Close
About
FAQ
Home
Collections
Login
USC Login
Register
0
Selected
Invert selection
Deselect all
Deselect all
Click here to refresh results
Click here to refresh results
USC
/
Digital Library
/
University of Southern California Dissertations and Theses
/
The impact of restructuring the language arts intervention program and its effect on the academic achievement of English language learners
(USC Thesis Other)
The impact of restructuring the language arts intervention program and its effect on the academic achievement of English language learners
PDF
Download
Share
Open document
Flip pages
Contact Us
Contact Us
Copy asset link
Request this asset
Transcript (if available)
Content
THE IMPACT OF RESTRUCTURING THE LANGUAGE ARTS INTERVENTION
PROGRAM AND ITS EFFECT ON THE
ACADEMIC ACHIEVEMENT OF ENGLISH
LANGUAGE LEARNERS
by
Denise L. Parnell
A Dissertation Presented to the
FACULTY OF THE ROSSIER SCHOOL OF EDUCATION
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulfillment of the
Requirements for the Degree
DOCTOR OF EDUCATION
May 2008
Copyright 2007 Denise L. Parnell
ii
DEDICATION
This work is dedicated to everyone who has ever been told that they will never
amount to much. Dream your dream and hang on for the exciting ride as you can do
all you dream!
Your vision will become clear only when you look into your heart.
Who looks outside, dreams. Who looks inside, awakens.
~ Carl Jung
iii
ACKNOWLEDGEMENTS
The author would like to acknowledge the generosity and support of the
following individuals for their understanding and support; Joanna, Laura, and Mary,
my daughter; Tiffani, my mother; Angie, my friends and colleagues; Marj and Scott,
but most of all for his unconditional love and support Jeff.
A special thank you to Dr. Hocevar, although he took the time and effort to
support others by sharing thoughts such as, “Your enthusiasm and compassion for the
kids really shows,” it was his enthusiasm and compassion for supporting others in
research that allows numerous people to touch and improve the lives of many children.
Thank you Dr. H!
iv
TABLE OF CONTENTS
Dedication ............................................................................................................. ii
Acknowledgements .............................................................................................. iii
List of Tables ........................................................................................................ vi
Abstract ................................................................................................................. vii
Page
CHAPTER ONE:
The Problem .................................................................................................. 1
Problem Identification ................................................................................... 1
Problem Analysis/Interpretation .................................................................... 2
Restructuring: Target Time ........................................................................... 4
Researched-based Strategies ......................................................................... 9
Accountability System ................................................................................... 11
Conclusion ..................................................................................................... 14
Summary ........................................................................................................ 20
Purpose, Design and Utility ........................................................................... 23
CHAPTER TWO:
Literature Review .......................................................................................... 25
Restructuring the Instructional Program ....................................................... 26
Research Develops Knowledge and Skills .................................................... 29
Developing Professional Learning Communities .......................................... 31
Perceived Organizational Barriers ................................................................. 32
Self-efficacy and Accountability ................................................................... 34
Summary of the Literature ............................................................................. 35
CHAPTER THREE:
Methodology .................................................................................................. 37
Summary of Research Design ....................................................................... 37
Participants/Sampling .............................................................................. 39
Intervention Description .......................................................................... 40
Quantitative Instrumentation ................................................................... 44
Qualitative Instrumentation ..................................................................... 45
Procedural Timeline ................................................................................ 46
Quantitative Analysis .............................................................................. 47
Qualitative Analysis ................................................................................ 46
Delimitations and Limitations of the Study ................................................... 47
v
CHAPTER FOUR:
Results ........................................................................................................... 49
Pre-Post Independent Group Results ....................................................... 50
Comparison School ................................................................................. 56
CHAPTER FIVE:
Conclusion ..................................................................................................... 61
Overview ....................................................................................................... 62
Purpose and Design ....................................................................................... 63
Summary of Findings: Mitchum Elementary School and Moonstruck
Elementary School ................................................................................... 63
Category 1: Restructuring Student Groups .............................................. 68
Category 2: Implementing Research-based Strategies ............................ 70
Category 3: Category 3: Accountability .................................................. 71
Summary of Findings: Mitchum Elementary School and Moonstruck
Elementary School ................................................................................... 73
Implications ................................................................................................... 74
Site-based Recommendations ........................................................................ 77
Limitations ..................................................................................................... 81
Conclusion ..................................................................................................... 82
REFERENCES .......................................................................................................... 84
APPENDIX ............................................................................................................... 87
vi
LIST OF TABLES
Table 1: Mitchum Elementary Percent of Student Proficient in Language
Arts: Three-year Comparison ............................................................ 3
Table 2: 2005-2006 Similar School Comparison: percent Proficient .................... 39
Table 3: 2005-2006 Similar School Comparison: School Demographic
Characteristics ................................................................................... 39
Table 4: 2005-2006 Similar School Comparison: API base by Subgroups ........... 39
Table 5: Pre/Post Intervention CST ELA Performance Band: Statistical ............. 51
Table 6: Pre/Post Intervention CST ELA Performance Band: Practical ............... 52
Table 7: Pre/Post Intervention CST ELA Percent: Basic and Above .................... 54
Table 8: Pre/Post Intervention CST ELA Percent: Proficient and Above ............. 55
Table 9: CELDT Assessment Results: Overall Performance ................................ 57
Table 10: API School-wide Comparison ................................................................. 58
Table 11: . API Subgroup Comparison ...................................................................... 58
Table 12: AYP ELA Experimental and Control Group Comparison ...................... 59
vii
ABSTRACT
Trends in the performance of students attending Mitchum Elementary School
in 2005-2006 school year compared to the performance of students attending in the
2006-2007 school year were examined. The study has a particular focus on the
achievement of English language learners, the mean annual school-wide Academic
Performance Indices and the percentage of proficient and above using the Annual
Yearly Progress indicator, as well as the EL learner’s performance on the California
English Language Development Test.
A formative and summative evaluation was used to evaluate EL learner’s
progress in language development after participating in a restructured ELD program.
Formative evaluation consisted of informal interviews during academic conferences,
site observations and analysis of student work. Summative evaluation results were
utilized to measure EL learners English language arts California Standards Test scores
and the California English Language Development Test scores from the 2005-2006
school year through the 2006-2007 school year. A comparison was made to a control
school, Moonstruck Elementary, which did not implement restructuring: Target Time
ELD sessions, research-based strategies or a focused accountability system.
The focus of this evaluation was to discover how much of an impact these
interventions have on school-wide achievement and especially, the EL learners’
achievement, both in English language development and the California Standards
Test. The formative data were used to determine how the implementation process
affected both teachers and students. Behaviors and outcomes will guide future changes
viii
that will increase student academic growth and engagement. The primary research
question for the formative evaluation is how Mitchum Elementary can improve EL
learner instruction.
The new program consisted of a restructuring, Target Time. In this design, EL
learners are grouped by language level and English only students are grouped by CST
results. In addition, a strong emphasis was given to research-based strategies and an
accountability system that allowed monitoring of student growth.
Implementing the restructuring of an intervention program was an enormous task, but
the impact had positive results, some academic and some professionally. Although the
results did not reflect enough change in the statistical or practical results of the EL
learners, this researcher would propose that the study had a tremendous impact on the
professional community that will extend to the academic success of EL learners in the
near future.
1
CHAPTER 1
THE PROBLEM
Problem Identification
Mitchum Elementary School, originally constructed in 1999, has grown from
524 students to 872 students. The ethnic make up of Mitchum Elementary is that of
56% Hispanic, 30% Caucasian, 11% Asian, and 3% of other ethnicities. English
learners make up 38% of the population and socioeconomically disadvantaged make
up 56% and both continue to increase year-after-year. All teachers at Mitchum
Elementary are highly qualified as deemed by the NCLB standards and hold either a
CLAD or BCLAD certification. Staff development is ongoing, but focused on English
language development and vocabulary building this year.
The Federal Government holds the California Department of Education
accountable for monitoring Adequate Yearly Progress (AYP) in California schools.
This is accomplished by administering annual state assessments. Schools are then held
accountable for school-wide growth on these assessments and for growth within
substantial (sub) groups within multiple subject areas. The subgroups that the State
monitors for Mitchum Elementary are Hispanic/Latino students, White students,
Socioeconomically Disadvantaged students and English learners.
In analyzing the AYP Report for Mitchum Elementary School in the subject of
language arts, an area of focus was that of the Hispanic subgroup, the Socioecon-
omically Disadvantaged (SD) students and English language (EL) learners. The 2006
AYP target for language arts stated that 24.4% of the student population needed to
2
score proficient or above on the California Standards Test (CST). Over the last 3 years
Mitchum Elementary had declined in the number of student’s school-wide who scored
proficient by 1.4%, even though Mitchum is scoring 36% of the population at
proficient or better and meeting the AYP target school-wide. There is a reason for
concern as the target will continue to increase for the next 7 years and Mitchum
Elementary is showing a decline. The White subgroup has shown growth from 48.7%
to 55.2%, a 6.5% increase in students scoring proficient or better over the past 3 years.
The concern comes when analyzing the three remaining subgroups over the past 3
years. The Hispanic subgroup has declined by 4.5%, Socioeconomically
Disadvantaged (SD) students grew in 2005 by 3.4%, but declined in 2006 by 7.6%,
English Language (EL) learners have grown every year, but fail to hit the target as
only 20.5% of them scored at proficient or better in 2006 (Table 1).
The results show that these three particular groups of students are not receiving
the support they need in order to reach at a minimum, a proficient level in grade level
standards. This is generating an inequality for the Hispanic, SD and the EL learners in
comparison to the English only students. The question was where and how do we
close the academic gap for these students?
Problem Analysis/Interpretation
Analyzing the CELDT scores for EL learners over the past 3 years, shows the
following percentage of students at each of the proficiency levels: 2% score at level
1 - Beginning (8% decrease), 16% score at level 2-- Early Intermediate (14%
decrease), 46% score at level 3--Intermediate (4% increase), 27% score at level
3
Table 1. Mitchum Elementary: Percent of Students Proficient or Higher in Language Arts Three-Year
Comparison
2003-2004
2004-2005
Loss/Gain
2003-2004
2004-2005
2005-2006
Loss/Gain
2004-2005
2005-2006
School-wide 37.4% 36.6% -8.0% 36.-% -6.0%
Hispanic/Latino 26.1% 25.9% -2.0% 21.6% -4.3%
White 48.7% 50.0% +1.3% 55.2% +5.2%
SD 22.5% 25.9% +3.4% 18.3% -7.6%
EL 18.7% 19.7% +1.0% 20.5% +8.0%
CDE Target 13.6% 13.6% 24.4%
4-Early Advanced (12 % increase) and 7% at level 5--Advanced (6% increase).
Growth has been slow, but increasing. Since the focus for students in levels 1-3 is
language development and not California standards, this means that in the past 3 years,
64% of the ELL students’ main instructional focus has been language development,
not California State standards. Although, an increase in language development is
needed to successfully improve in State standards, these students are not receiving the
same focus towards mastery of the standards as their English only classmates. Thus,
slow progress has been realized toward increasing mastery of standards.
As the administrator, it is my role to analysis the instructional program for all
students at Mitchum Elementary and develop a program that better meets the needs of
the entire population. In analyzing the data at Mitchum Elementary, many students are
represented in all three subgroups. For the purpose of this dissertation, the EL learner
subgroup was the main focus for two reasons: (a) many EL learners are counted within
all three subgroups; and (b) it is the EL learner subgroup that was designated as not
4
meeting the state targets for 2 consecutive years, resulting in placing Mitchum
Elementary in a Program Improvement (PI) status.
Based on the CELDT scores, I see far too many EL learners staying in the
beginning, early intermediate and intermediate levels for multiple years. My goal was
to develop an English language arts intervention that supported EL learners in
successfully moving through one language performance level each year, and allowed
English only students to receive instruction that best meets their needs as well. The
new program consisted of implementing a restructuring of the language arts
intervention program that included the use of research-based strategies and an
accountability system that monitored student growth.
Restructuring: Target Time
Alexander and Murphy list five general statements related to Learner-Centered
Principals (Alexander & Murphy, 1999, p. 26). The first two are Knowledge Base and
Strategic Processing or Executive Control. Stated by Clark and Estes (2002) as the
first of the “Big Three”: People’s Knowledge and Skills (p. 43). Considering the “Big
Three” approach, it is believed that teachers do possess the knowledge and skills that
are needed to successfully move students through the ELD levels. All teachers have
received their CLAD or BCLAD certification and have attended multiple district
provided trainings in the area of ELD. However, once the monitoring component is in
place, this belief will be proven or disproved.
The third element by Alexander and Murphy (1999) is stated as “Motivation
and Affect,” and as the second of the “Big Three,” Motivation to Achieve the Goal by
5
Clark and Estes (2002). There has not been an emphasis on EL learners’ progress
during the ELD sessions, as it has never been monitored like the goals of reaching
grade level standards. Teachers have been motivated to focus on students’ progress
toward meeting grade level standards and have not made the connection that EL
learners’ progress would greatly improve in the standards if their language acquisition
were improved. The ELD sessions have been seen as “another” thing to do that gets in
the way of teaching standards.
General statements four and five by Alexander and Murphy (1999);
Development and Individual Differences and Situational or context are embedded in
Clark and Estes (2002) final cause of the “Big Three,” Organizational barriers such as
lack of necessary equipment and missing inadequate work processes (Clark & Estes,
2002, p. 43), Considering this final cause, the restructuring of the ELD groups
removed the structural barrier of too many different language levels needing to be
addressed within the same ELD session as students were placed within like language
levels. When considering the knowledge gap of the EL learner, another area to be
addressed is the engagement of the learner during instruction.
During last year’s Academic conferences, EL learner progress was discussed.
It became quite clear that attributions, self-efficacy, fear and anxiety and teacher
expectations were areas of focus as restructuring of the program took place. These
areas can be considered in regard to teachers or students. However, in order to look at
these areas in students, we must first focus on the teacher and consider the students at
a later time. When asked why ELD sessions were difficult, unanimously teachers
6
stated the difficulty they have serving all proficiency levels at one time. These are
teachers who differentiate all day long, but have not transferred that skill to the ELD
session. They are locked into the statement of policy that states, “EL learners are to
receive a minimum of forty-five minutes of ELD instruction a day.” Once the idea of
only serving one ELD level during a single session (Target Time) was brought up,
teachers shared a sense of relief as comments were shared, “We can do that?” “That
would make things much easier,” etc. Attribution had played a role as they settled on
the fact that the task was too difficult (external), and that they did not have the ability
(internal) to be successful in this model of instruction. Teachers were even more
interested in the restructuring, when they realized that they could control how the
groups were configured. They were willing to provide more effort as the task became
more doable (Table 2, Pintrich, 2003, p. 672). This then led right into self-efficacy.
The fact that teachers had to serve all levels of ELD within the same session is
a case of self-efficacy. When asked about the organization barrier, inadequate work
process of serving all levels at once, the teachers in the experimental school, Mitchum
Elementary were quick to state it is impossible. The low self-efficacy has encouraged
the lack of monitoring during ELD sessions. By restructuring the ELD sessions, this
major obstacle was removed and by adding the requirement to monitor EL learners’
progress during the ELD sessions it allowed teachers’ self-regulation of student
progress. The restructuring approach aided in determining if the performance gap was
indeed an environmental issue or if there is lack of teacher knowledge regarding
language acquisition. The primary instructional objective was to move the EL learners
7
from their current ELD level to the next level by eliminating the obstacle of serving
too many different ELD levels at one time.
Teachers needed to retain a choice on how to approach students’ needs within
the group so staff and administration developed an action plan. Clark and Estes’
(2002), Three Facets of Motivated Performance (p. 81) was enacted. Teachers will
develop student lists (active choice) and then collaborated about who should serve
whom (mental effort) so that like proficiency levels were grouped together. Teachers
generated the plan (persistence) so success was owned by all. Having changed a once
unreachable goal, as seen by the teachers, into an obtainable “self-structured” goal,
enhanced their self-efficacy. Requiring teachers to meet monthly and discuss student
process, allowed administration to continue teachers’ mastery orientation as
improvement of the process was discussed, and altered along the way to prevent
teachers from moving into a performance orientation. Any adjustments were viewed
as informational and progress viewed as developed through the teachers’ efforts. As
Dembo states, “In a mastery goal orientation, the [teacher] is concerned with making
progress and mastering a skill and places value on the importance of effort” (Dembo,
2004).
Fear and anxiety may have set in at first for teachers, knowing they would be
discussing results with administration regularly at monthly academic conferences.
However, teachers developed trust in their colleagues and in administration as part of
the solution, motivation was enhanced. “The more people trust each other’s skills, the
more motivated they will become” (Bandura, 1997). Meetings were designed to share
8
achievement of goals. Using this organizational and management structure encouraged
personal and social responsibility and provided a comfortable environment (Table 2,
Pintrich, 2003, p. 672).
Often, teachers perceive students’ ability levels as fixed or stable; in other
words, they have a finite view of intelligence. Teachers’ expectations of students can
have a positive or negative effect on student progress. Reinventing the ELD session
structure, supported teachers in developing strong positive expectations of students as
they controlled the groupings; they established mastery goals verses performance
goals and they worked to meet social obligations in a safe, comfortable and
predictable environment (Pintrich, 2003, p. 672).
Offering a program structure for ELD that eliminated the overwhelming task of
meeting all ELD levels at once, assisted teachers in creating focused lessons that
consider the cognitive process of learning instead of “just” teaching the curriculum.
When considering the cognitive process, most staff seemed to be in need of moving
their knowledge from procedural to metacognitive. Teachers understand that the
lesson approach for levels 1-3 is different from levels 4 and 5. That is, they understand
the criteria, but have not applied this knowledge to the ELD sessions.
Explicit direct instruction lessons were designed around specific ELD levels so
that students could show growth within the ELD levels. Originally, students were
grouped together by grade level not by ELD level, making it difficult to address each
student’s needs in one session. The restructuring of the program allowed teachers to
group students by their ELD levels, and gave teachers the opportunity to provide
9
appropriate instruction using the EDI lesson format. Teachers developed lessons that
provided students with information that allowed EL learners to build language
acquisition and access standards at their ELD level.
Researched-based Strategies
A “sense of urgency” was needed at Mitchum Elementary to support clear
focused instruction every academic minute, and that students were engaged in their
own learning. Mitchum’s grade level teams discussed the percentage of students
scoring at proficient or above at grade level meetings. Further, they began the
implementation of students setting goals. Each month teachers assessed their students
using an agreed upon assessment, and then collaborated as a grade level to determine
ways to address students’ needs. Sharing student growth at Mitchum’s staff meetings
allowed grade level teams to celebrate student success and negotiate changes as
needed. In order to experience student success, grade level teams first developed
lessons that calibrated to the standards, aided students in monitoring their own
learning, and allowed teachers to monitor student growth through formative
assessments.
Teachers are trained in EDI instruction. Using this model, Merrill’s First
Principles of Instruction (Merrill, 2002) are considered. Merrill lists five principles--
the first concerns engaging learners in real-world problems (p. 44). EDI addressed this
as a classroom problem that is addressed within a lesson, as the teacher states the
learning objective in student language with the emphasis on what the student must
accomplish when the lesson is complete. Teachers then connect it to when students
10
may need the skill beyond the classroom. Merrill’s (2002) second principle states
learning is promoted when existing knowledge is activated (p. 44-45). The second step
of the EDI format is to Activate Prior Knowledge. Teachers review prior lessons or
experiences so that students can connect the new lesson to something already known.
Principle three, learning is promoted when knowledge is demonstrated to the learner
(Merrill, 2002, p. 45). The third step in an EDI lesson is Model, Explain and
Demonstrate. Teachers model each step and do “think alouds” to help students
understand how and why the teacher is doing each step. Bringing up Merrill’s (2002)
fourth principle, learning is promoted when applied by the learner (p. 45). This step is
called Guided Practice in the EDI format. Teachers can then monitor students
understanding of the concept as they apply the new knowledge under teacher’s
guidance. This is the area most lacking in the ELD session thus far. The final principle
stated by Merrill is that learning is promoted when new knowledge is integrated into
the learner’s world (Merrill, 2002, p. 45). In EDI, students are not released to
independent practice until they show mastery of the new concept during guided
practice. Therefore, students would not be released to the independent practice stage
unless they showed integration of the new concept into their own knowledge. Given
that teachers are implementing the EDI format for ELD instruction and have
demonstrated mastery of implementation of the EDI format, the primary teacher
knowledge gap then rests with principle four, as teachers needed to monitor for
mastery (long-term knowledge of skill) of the ELD lesson, not only during the lesson,
11
but also during core instruction throughout the day (short-term use of knowledge and
skill).
Beyond adherence to the EDI lesson plan during ELD and core instruction and
implementing assessments, a stronger focus was placed on using data from
performance-based assessments to guide instruction. Teachers have always completed
assessments and looked at the results, but designing lessons around the results has
been rare. Utilizing what Darling-Hammond (2002) calls the five “habits of mind”
regarding intellectual development, having the ability to weigh and use evidence, to
see and understand differing viewpoints, to see connections and relationships, to
imagine alternatives and to assess implementations and effects is an area that was
improved while evaluating student work. Teachers have worked with the standards,
but needed more experience, and planning time to develop lessons that allow standards
to be “bundled” and that would result in a product that students created, showing they
have a metacognitive understanding of the concepts within the standards. Calibration
between students’ needs, the assignments, and the expected outcome of the standards
needed to be aligned. Organized scheduled planning sessions allowed teachers to
accomplish this focus.
Accountability System
Another structure that needed to be implemented was student goals, the goals
were guided by teachers in that they shared students’ reading levels, show them what
gaps need to be worked on per the assessment, offered strategies that helped with the
gap and had them set goals. “Explicit goals for student learning and shared school-
12
wide values are the framework for restructuring” (Darling-Hammond, 2002, p. 150).
Administrative funding focused on push-in help that allowed gaps in learning to be
addressed, as well as decrease the teacher to student ratio. Darling and Hammond
(2002) shared that in their research, it is important to know the students in order to
gain growth, “Each school has created a way to know students well and to engage in
ongoing inquiry as a basis for continued improvement” (p. 150). Looking at collected
data on students by class during academic conferences with administration provided a
way of focusing on individual student’s needs.
One aspect of the program that needed to change was that of helping teachers
to monitor student progress, and to build their confidence in their own ability to offer
strong English language development (ELD) at the student’s ELD level. Monitoring
student progress throughout the year, verses only at the end of the year, allowed
teachers to adjust instruction as needed frequently. Students and teachers monitored
the improvement in language acquisition. The use of the ELD Matrix, a rubric of the
ELD standards, to design student centered lessons also created a structure that
considered students’ individual needs within a lesson and provided a monitoring
devise, resulting in students advancing through the ELD levels.
Teachers used daily observation, the EL Matrix and teacher created
assessments to monitor progress. Students monitored their own progress, thus
increasing student motivation using the EL matrix and assessments as well. Students
CELDT scores were compared at years end for overall growth. The monitoring of
13
student progress showed if the structural change was successful in creating teacher
self-regulation of EL learner growth and development.
Teachers are well trained in how to monitor student achievement, but were not
transferring that knowledge to the English language development instructional
sessions specifically, “They only need the information that they should apply the
approach they used in the past (or a slight variation) to the new challenge” (Clark &
Estes, 2002, p. 60). Teachers were aware of the 45 minutes a day requirement for
ELD, but were not monitoring the progress of the sessions. Holding teachers
accountable for showing growth within the ELD sessions offered the “information”
that they needed to apply the monitoring strategies to the ELD sessions.
Teachers addressed two important EL learner areas. The first was that of
monitoring EL learners’ progress throughout the year so that at the end of the year,
students show growth from one ELD level to the next level. The second was that of
lesson development, lessons designed around the results of daily monitoring and
regular assessments, instead of waiting for the annual assessment.
Teachers met at least once a month to determine students’ needs, and to
monitor the motivational goals and program changes so that the task of successfully
teaching ELD remained “doable” in the eyes of the teachers. Teachers recorded
decisions and alterations in minutes that were turned in to administration once a month
until June 2006. Administration monitored attribution and self-efficacy growth
through documentation turned in each month, observations and through individual
teacher academic conferences each month.
14
During the meetings teachers and administration reviewed student assessments
and analyzed student growth. Administration monitored teachers’ values and mood
through observations, walk-throughs and recorded meeting minutes. Administration
performed at least three walk-throughs a week to determine if the EDI format was
used and for student engagement. The overall organizational goal was monitored
every month during academic conferences. Through the analyzing of students’ on-
going assessments, student progress was monitored allowing for progress into the next
ELD level.
Conclusion
Teachers offered 45 minutes of direct ELD each day. Some teaching teams;
allowed one teacher to take all the EL learners and the other teacher took the English
only students. In analyzing the EL learner’s California Standards Test (CST) scores
over the last 3 years, the improvement in language arts towards the proficient level has
been less then 2% among EL learners. In order for students to succeed academically,
they must master the California State Standards for their grade level; not reaching this
goal sets EL learners up to fail. The EL learners must move into the Early Advanced
and Advanced levels within their first 3 years of learning English. Once students are
within the top two English language proficiency levels, teachers can then offer
Specially Designed Academic-Instruction in English (SDAIE) strategies that will
enable students to access the California state standards.
Although the ultimate goal is to develop a school that supports effort-based
education, that can take time to show up in assessment results. The above changes
15
support immediate results and aid in supporting the shift to effort-based learning.
Designated as a Program Improvement (PI) school in the 2005-2006 school year, a
much more focused approach was developed to show growth within all of the
students. “Although he [Schmoker] does not necessarily advocate a short-term view of
school reform, he asserts that obtaining results in the first year is critical to providing a
foundation of success on which to build” (Marzano, 2003). Targeted student grouping
for ELD instruction (restructuring), a strong focus on exposing staff to research-based
strategies and implementing a strong monitoring of student progress and teacher
efficacy was addressed during the 2006-2007 school year.
The focus intervention, “Target Time,” was implemented school-wide,
allowing teachers to address the needs of students by their level of proficiency as
deemed on the California English Language Development Test (CELDT) or on the
CST for English only students. EL learners scoring below proficient were grouped into
smaller groups and with students scoring in the same language level. This allowed
teachers to design lessons, using the EDI format, which best fit the students’ needs
instead of trying to serve the range of language levels in one class. English only
students were grouped by their CST proficiency level and EDI lessons were
constructed to meet their needs as well. The groupings were flexible and changed
regularly as assessments showed student growth. Teachers at Mitchum Elementary
believe all students can learn and work together to share ideas, and take it upon
themselves to design flexible groupings during core instruction that supports students’
needs.
16
I shared with staff articles about 90/90/90 schools, Accountability in Action
(Reeves, 2000) and two of Marzano’s books, What Works in Schools (2003) and
Classroom Instruction that Works (2001) in hopes to strengthen their understanding of
using research to inform decisions at the site. Marzano’s (2003) work supports
establishing governance structures that allow for teacher involvement in decisions and
policies for the school. Shared, informed, decision-making is just such the governance
structure needed at Mitchum Elementary. Sharing research that supports research-
based strategies allowed teachers to make informed decisions about their instruction
and began with administration allowing opportunities for staff to read and discuss the
research. The focus for the 90/90/90 articles was two-fold; the first was to build
teachers’ efficacy, to show that many of the strategies used at Mitchum Elementary
were proven to work, as they are research-based strategies. This led teachers to have a
stronger focus on implementing the strategies consistently and with fidelity. The
second goal was to strengthen the culture of “every child can learn,” by designing
ways to focus on higher expectations for all students. Monitoring of student growth
was a clear focus of many of our conversations, meetings and trainings.
Academic Conferences (AC) allow administration to meet with each teacher
individually each month. Students were grouped into two groups: 20/70 group
(students scoring 320-370 on the CST as this group often slips through the crack with
little attention), and the EL learners. During the AC meetings teachers, shared how
students progressed through the use of assessments, observation (data on grouping
students and why), and student participation (strategies used for checking for
17
understanding). A stronger focus was placed on using a higher level of checking for
understanding such as summarizing and note taking (Marzano, 2003), questions based
on Bloom’s Taxonomy so that students “own” the content, and in student goal setting
so students can self-regulate their own learning (Marzano, 2001; Resnick & Hall,
2003). Lessons supported metacognitive development in the teachers and in students.
To improve student achievement, leaders realized they would need to
fundamentally change instructional practice (Togneri & Anderson, 2003). This
statement stands true for Mitchum Elementary, as when Mitchum opened 8 years ago
the enrollment was 524 students. Today it houses 872 students with a shift in
socioeconomic disadvantage students from 31% free and reduced lunch to 58%.
Watching this shift take place, I realized that the structure used to deliver instruction
was going to need to change as students were entering with the need for more explicit
instruction and an increase in building background knowledge. I brought Data Works
in to train teachers in the EDI format and how to calibrate lessons to the standards.
Teachers became aware of how important modeling and checking for understanding
was for student growth, but it was clear that a need for effort-based education had
surfaced. “An effort-based system actually can create intelligence. Ability is created
through certain kinds of effort on the part of the learners and reciprocally on the part
of the educators who are working with those learners” (Resnick & Hall, 2003). Many
things stand to reason for Mitchum’s slow progress, but areas that needed to improve
immediately so that a foundation of effort-based learning could begin to develop
human capacity, people becoming more intelligent through sustained and target effort
18
(Resnick & Hall, 2003) were: (a) consistent implementation of the EDI format
(explicit instruction), (b) development of critical thinking in students using researched-
based strategies, (c) calibrating standards across the curriculum, and (d) an on-going
use of informative assessments.
A focus on instruction through a coherent systematic approach will allow all
students to achieve (Togneri & Anderson, 2003). Individuals’ possess measurable
qualities of intelligence, aptitude, and retain the capability to gain intelligence through
sustained and targeted effort of one’s self. Self-confidence and learning oriented goals,
lead to intelligence developing over time (Resnick & Hall, 2003). This human
capacity is developed in environments using a rigorous curriculum with a clear
focused approach that demands lots of mental work and exposure to Bloom’s
cognitive dimensions for all students. Teachers needed to supply the strategies that
allow students to reach the content.
Marzano (2003) shares nine crucial strategies that need to be actively present
in each classroom: (a) identifying similarities and differences, (b) summarizing and
note taking, (c) reinforcing effort and providing recognition, (d) homework and
practice, (e) non-linguistic representations, (f) cooperative learning, (g) setting
objectives and providing feedback, (h) generating and testing hypotheses, and
(i) questions cues and advance organizers. In order to know which strategies are
needed for a particular lesson, teachers needed to know the knowledge of each student
with regard to content. One strategy that allowed teachers to access the thoughts of
students was to allow them to brainstorm their knowledge on a subject, in the past this
19
took the form of a whole class KWL chart usually created by a few and recorded by
the teacher. This concept needed to be implemented individually by having each
student share their knowledge (K), in written form, in an organizer before asking the
class to share out. This allowed the teacher to understand each student’s present
schema and decisions could be made about the learning experiences that were needed.
Students also wrote questions about the subject on an organizer, sharing what they
wanted (W) to know about the topic. This too, aided teachers in knowing how to craft
their lessons so that the lesson supported students as researchers of their own learning
and ending with all students writing what they learned (L) about a subject.
Lessons needed to reflect cross-curriculum standards that were focused and
deliberate, and allow input experiences that help students identify what they already
know, provide direct links between the new content and the old and equip students
with a way to organize the information (Marzano, 2003). Desired outcomes were
displayed so that students could regulate their own learning in order to monitor
themselves. Students needed to have the opportunity to check new insights against
prior knowledge, building knowledge capacity. The EDI format allowed for such an
opportunity and for formative assessment along the way. Monitoring teachers’ fidelity
to the use of the EDI format with a focus on students’ obtainment of knowledge was
the challenge. Daily progress was monitored by each teacher and adjustments to
lessons were made as needed. Each trimester’s scores were analyzed for advancement
within the groups in regard to the ELD progress. This resulted in EL learners
advancing by at least one ELD level in a year.
20
Summary
“If at first you don’t succeed, try, try again,” effort-based education is the key
to developing young minds that can problem-solve and manage their own educational
learning. Building the human capital allows people to build intelligence. Lauren B.
Resnick, Director of the Learning Research and Development Center, shares the
relationship between a focus on aptitude versus effort. Aptitude has been seen as an
innate mental ability, a fixed commodity. Resnick and Hall (2003), express that
metacognitive strategies are both teachable and learnable, therefore, an effort-based
system actually can create intelligence.
Reflecting on Mitchum’s approach to educating children, it became clear that
most of the time we were focused on performance-based assessments that have
unknowingly supported a focus on aptitude not effort, and have aided in mainly
supporting the White middle class student.
This coupled with an inconsistent use of informative assessments to guide
instruction along the way resulted in slow progress at best. Although, we have been
working on the EDI format that guides teachers in developing lessons that assure
students know the objective, can connect the new lesson to prior knowledge, includes
teacher modeling, guided practice and independent practice, the focus was on the
format not on students’ development within the format. Lessons may have been well
planned, but without strong formative assessments along the way, teachers were
unaware if all students were actually learning. This perpetuating the “I taught it so
they must have learned it” mentality. Measuring and monitoring outcomes, program
21
effectiveness, and policies and practices at all levels needed to become part of
Mitchum’s “educational fabric.” Analyzing our outcomes and practices brought to the
surface biases and issues that needed to be addressed (Johnson, 2002). Teachers have
voiced, during academic conferences that they are willing to make changes, and low
morale showed that teachers were not comfortable with Mitchum’s program
improvement status. With data in hand, they were willing to restructure their approach.
It was time to start a “culture of inquiry,” strengthening the analyzing of relevant data,
probing perceptions about why things are as they are and examining the academic
culture, including issues of access, equality and opportunities to learn (Johnson, 2002).
Teachers were aware and used Bloom’s Taxonomy to structure some questions
for students when checking for understanding throughout the lesson. The question
became, are the same level of questions always targeted to the same students so the
gap of learning continued? Teachers needed to scaffold questions up the taxonomy so
that students moved into the next level of thought, “stretching” students’ knowledge.
This, “pigeon holing” extenuates the push to differentiate instruction. In an attempt to
aide students in having “access” to the standards, teachers were designing questions
that fell within the Remember and Understanding levels of Bloom’s taxonomy.
Scaffolding was used, but rarely moved students beyond identifying, retrieving and
interpreting information. Students were not consistently exposed to higher levels of
Bloom’s taxonomy such as applying, analyzing or evaluating new knowledge.
Leaving many students “stuck” in the lower levels. Walk-through, after walk-through
showed a fury of fact-finding and memorization without the understanding of concept.
22
Teachers needed to “bridge” students learning by providing questions and inquiries
that supported students in developing higher order thinking skills. Indicators need to
be connected to short- or long-term higher-learning outcomes for all students
(Johnson, 2003).
Although, it is said by teachers, that all students can learn, actions showed a
different philosophy; not necessarily because they secretly do not believe that all
students can learn, but they themselves have not built their own understanding of how
to build lessons that brought all students to the metacognitive stage. Supporting
teachers in moving from display-oriented goals to learning-oriented goals (Resnick &
Hall, 2003) allowed classrooms to shift from a focus on lesson format and “pigeon
hole questions” to a focus on students’ and teachers’ metacognition of knowledge.
“It was so exciting that he understood it, I had to have him say it again,” a
heart felt sharing from a teacher whose student, due to a lack of opportunities, is
restricted from having this experience daily. Just as students want to feel successful,
teachers do embrace the excitement when students’ “light bulbs” light up. Guiding
teachers to develop higher order thinking questioning established not only a
foundation of higher expectations, but enforced the same expectation for all students.
This limited the “missed opportunities” for students in their learning. “Without this
bigger picture, learners become dependent on someone else every step of the way to
tell them what is good or not good and what to do next” (Resnick & Hall, 2003).
Imagine the energy that is found in a room where all students are progressing and
sharing their findings with the teacher and their peers. Strengthened self-confidence in
23
students and in teachers was witnessed. Teachers carried the belief that they can make
a difference with all students and students developed characteristics such as initiative,
persistence and drive as their environment supported them as a learner (Northouse,
2004).
Purpose, Design and Utility
Formative and summative evaluation was used to evaluate EL learners’
progress in language development after participating in the restructured ELD program.
Formative evaluation consisted of informal interviews during academic conferences,
site observations and analysis of student work. Summative evaluation results were
measured utilizing the EL learners English Language Arts CST scores and the CELDT
scores from the 2005-2006 school year through the 2006-2007 school year. A
comparison was made between a control school, Moonstruck Elementary, who did not
implement restructuring: Target Time, research-based strategies or a focused
accountability system and Mitchum Elementary, the experimental school, who did
implement the restructuring.
The focus of this evaluation was to discover how much of an impact these
interventions had on the EL learners’ achievement, both in English language
development and the California Standards. The formative data was used to determine
how the implementation process affected both teachers and students. Behaviors and
outcomes guided future changes that increased student academic growth and
engagement. The research question used to determine the summative discovery was:
Did the restructuring of the language arts interventions implemented in the 2006-2007
24
school year, which included the use of research-based strategies and an accountability
system, have an effect on the academic achievement of English learners? The primary
research question for the formative evaluation was how could Mitchum Elementary
improve EL learner instruction?
25
CHAPTER 2
LITERATURE REVIEW
English Language (EL) learners are the fastest growing subgroup in the
nation’s schools. In California, 1 in every 7 children entering kindergarten is an EL
learner (National Education Association, 2006). By 2025, 25% of all students in
school may be EL learners (Committee on Education and Labor, March 2007). This
supports the need for the No Child Left Behind (NCLB) legislation to succeed.
California has a comprehensive accountability system based on state requirements,
established by the Public Schools Accountability Act (PSAA) of 1999, and Federal
requirements, established by the NCLB Act of 2001. Positive change to NCLB
legislation since its induction in 2001 now requires that test data be disaggregated to
assure all students in all subgroups are learning successfully (CDE, 2005). Although
under NCLB, EL learners are treated as a homogenous group as NCLB does not
account for family and cultural background characteristics. These very characteristics
can have a tremendous impact on what approach may be the most successful and on
the amount of time that may be needed for EL learners to reach the same proficiency
as their English only counterparts. This leads school districts to determine what
constitutes a quality program. Research holds the answers to which approach offers
success for all students, and more importantly EL learners.
The approach taken for this literature review is not to question the state or
federal accountability system, although there are many questions that can be asked,
but to question the quality of the instructional program at the school site. Is the
26
program designed to reach the needs of all students, especially EL learners? This
literature review will discuss how all learners learn and how conducting a performance
gap analysis can strengthen the instructional program for all students. It will delve into
researched best practices and offer insight to an accountability system that allows for
on-going assessment and program monitoring, all the while supporting the
development of teachers as professionals.
Restructuring the Instructional Program
The NCLB data reflects a need for change, as too many students are not
reaching proficient levels as rapidly as the federal government benchmarks require.
Add to that a layer of needing to acquire English proficiency for the ELL students, and
schools are left to design programs that address both language acquisition and state
standards at the same time. This leaves schools to determine what changes to make,
when and how. The answer lies with needing to know the right thing to do for the
community that is being served.
Knowing the right thing to do is the central problem of school improvement.
Holding schools accountable for their performance depends on having people
in the schools with the knowledge, skill, and judgment to make the
improvements that will increase student performance. (Elmore, 2003, p. 9)
In order to determine what needs to change in a learning environment to
improve student progress, one must first understand how others learn. Alexander and
Murphy (1999) have revised the 14 psychological principals of learning that were
developed in 1993 by the American Psychological Association (APA) Presidential
Task Force on Psychology in Education and Mid-Continent Regional Educational
27
Laboratory (McREL) to five general statements related to a learner-centered focus.
These five areas will provide an understanding for professionals as they seek to
facilitate the learning of others (Alexander & Murphy, 1999).
Alexander and Murphy (1999) contribute the devaluing of psychological
processes as a contributing factor for failing reform in education (p. 27). The five
essential dimensions are: (a) knowledge base, (b) strategic processing (executive
control), (c) motivation and effect, (d) development and individual differences, and (e)
situation or context. Alexander and Murphy (1999) concur that education has over
looked and undervalued the beneficial knowledge of psychological processes when
developing learning experiences for children as one’s existing knowledge serves as the
foundation of all future learning. It is this very foundation that serves to connect all
new knowledge. Prior knowledge is an extremely powerful force, not only in what
information is attended to or how the information is perceived, but also in what the
learner views as relevant or important. An individual’s knowledge base is the scaffold
that supports the construction of all new learning. It becomes extremely important that
educators be aware of misconceptions and work to challenge the misconceptions
through meaningful instruction (Alexander & Murphy, 1999, pp. 29-30).
Strategic processing is having the ability to reflect on one’s own mental
functioning and to assess one’s own performance. “Studies have demonstrated that
learning is enhanced when individuals have knowledge of and apply appropriate
monitoring or executive strategies during the learning process” (Alexander & Murphy,
1999, p. 31). Significant academic growth can be realized when students reflect on
28
their own thinking and learning performance. This brings even greater importance to
the use of Bloom’s taxonomy questioning throughout the learning process as using
higher order questioning will allow for such reflection. However, strategic processing
or executive control must be coordinated with the learner’s emotional state or
instructional climate if optimal learning is to be achieved. These strategies must also
vary as the demands of the task changes. Simply having students memorize and
routinely execute a set of strategies is a misconception of the strategic process.
Strategic processing must be purposeful and effortful for academic growth to be
realized.
It is difficult to bring about strategic process change without also addressing
motivation, interest and self-regulation (Alexander & Murphy, 1999, p. 33). Alexander
and Murphy (1999) share a quote from Edison, “Genius is 1% inspiration and 99%
perspiration” (p. 33). Learning is influenced by how invested the learner is in the
learning process. An individual’s needs and desires must be activated so that an
individual can direct their thoughts and behaviors towards the learning. Students with
positive self-concepts are more apt to set goals of understanding instead of performing
and achieve better in school. Teachers that acknowledge student goals and interest,
and set a climate that is supportive and encouraging, will more than likely experience
student success. It is imperative that the educational system considers a learner’s
needs, desires, self-perceptions and emotional state while developing an environment
that allows for personal goals.
29
In understanding motivation one must also consider an individual’s
development and differences. Growth is a process that not only considers
commonalties, but also variations within individuals. Alexander and Murphy (1999)
characterize learning as the continual interplay between the nomothetic and
idiographic components of human development, that is, between the generalizable
patterns that give human thoughts and actions more predictability and the
individualistic constructions that are inevitable. “Educators must know the typical
stages that mark the pathways through which individuals of shared characteristics
appear to travel (Alexander & Murphy, 1999, pp. 37-38). Giving this dimension
consideration, it brings great value in educators having a very clear understanding of
language acquisition and the stages associated with each level of acquisition as the
typical development and individual variations explain human development.
Human growth is influenced by inherited and experiential factors. Such
experiences are embedded in an individual’s situation or given context. Much research
recognizes that learning is continuous and shaped by social context. Learner-centered
environments acknowledge the social nature of learning and the value shared thinking
can have on learner performance. When these are aligned with Clark and Estes (2002)
focus on improved performance, it is clear that that the essential dimensions are
embedded in Clark and Estes reform process.
Research Develops Knowledge and Skills
Clark and Estes (2002) “Big Three,” the analysis phase, consists of gathering
data about the causes of performance gaps. The first area is the knowledge and skills
30
of the people accountable for the performance gap or dimensions one and two of
Alexander and Murphy’s psychological process. Teachers must have an internalized
understanding of the stages of second language acquisition. Hill and Flynn (2006)
affirm that one of the most important things is that teachers know which of the five
stages each of their English learners are in (p. 14). The five acquisition stages are:
(a) Stage 1: Beginning, (b) Stage 2: Early Intermediate, (c) Stage 3: Intermediate,
(d) Stage 4: Early Advanced, and (e) Stage 5: Advanced. The task then becomes one
of scaffolding the curriculum to meet the given levels. Krashen and Terrell (1983)
support that the experience (the input), must include structures that pull from the
approaching stage. In order for teachers to embed such structures, they must have
knowledge of the stages of language acquisition. Teacher performance requires
knowledge and skill enhancement. According to Clark and Estes (2002), this can come
in one of three ways: (a) information, (b) job aid or (c) training. Information can be
shared in the form of simply telling someone something about their job that they need
to know to succeed. A job aid is a slightly higher form of information that can be used
as self-help information to perform a task. Whereas, training is used when people must
acquire knowledge and skills and need to practice and receive corrective feedback
(p. 58).
Obtaining knowledge of the language stages comes in the form of a job aid, a
matrix, which describes the output of student’s language production depending on
their level of language acquisition. Closely related to knowledge of the language
stages is that of not only knowing “best practice” strategies, but also knowing when to
31
implement the right strategy that allows access to the curriculum dependent on the
student’s language level. Marzano (2003) shares a well-articulated list of research-
based strategies that are a collection of many well-known researchers in his book,
What Works in Schools. The most effective teachers use the most effective strategies
and an expert teacher knows “when” to use the right strategy.
Developing Professional Learning Communities
Research supports that professional learning communities are the best way to
build capacity and strong academic culture at a school site. The second area of Clark
and Estes (2002) causes of performance gaps and the third dimension of Alexander
and Murphy (1999), deal with motivation to achieve a goal. In order to sustain
improvement there must be a shared mission, collective inquiry, collaborative teams,
and an action orientation that is improvement based and guided by results (DuFour &
Eaker, 1998). Marzano (2003) states, “The impact of decisions made by individual
teachers is far greater than the impact of decisions made at the school level” (p. 71) so
by building the motivation of the learning community a strong influence on student
achievement will be realized. Site-based management, shared decision making, staff
teams with shared planning time and shared responsibility for student instruction
roosts at the core of restructuring as “It is imperative that [the] initiative also consider
what happens in the heart of the school’s enterprise-the classroom (DuFour & Eaker,
1998, p. 9). Today in order to positively effect student success, the industrial model of
educating students must end and preparing them for the global economy must reign.
This model of educating is best described as a professional learning community. The
32
term “community” places greater emphasis on relationships, shared ideas and strong
culture which are crucial to school improvement (DuFour & Eaker, 1998, p. 15).
Schools must embrace a new conceptual model; the issue is identifying what research
proves to be “best practices” for a given community to ensure school improvement
(DuFour & Eaker, 1998, p. 23).
Aside from the viable curriculum that Marzano (2003) refers to, exemplary
practices and common goals must be shared so that equality for all students is present
and meritocracy does not set in. Instructional feedback for teachers is just as important
as learning feedback for students. Teachers themselves recognize the marked
differences in effectiveness among their colleagues (Schmoker, 2006, p. 28),
embedded structured collegial and professional meetings that focus on instructional
delivery based on data driven need, must be brokered. It allows for self-reflection and
regulation of behaviors. A focus on successful implementation of best strategies can
be discussed and cultivated among grade level teams.
A successful face-to-face team is more than just collectively intelligent. It
makes everyone work harder, think smarter and reach better conclusions than
they would have on their own. (Schmoker as cited by Surowiecki, 2006,
p. 105)
Perceived Organizational Barriers
Either way you look at it, motivation to achieve the goal (Clark & Estes, 2002)
or motivation and effort (Alexander & Murphy, 1999) teachers must be afforded the
opportunity to develop a process that emphasizes a structured organized English
language development program that is monitored, adjusted and implemented by them,
33
the teachers. Intrinsic motivation comes from ownership and such personal
involvement contributes to greater learning (Alexander & Murphy, 1999). Developing
goals, persistence and investing mental effort will only come from a continued school
wide focus on student growth both in state standards and in language acquisition.
Alexander and Murphy’s (1999) final two dimensions are development of
individual differences and the situation. The final cause of performance gaps
according to Clark and Estes (2002) is that of organizational barriers or better stated
as, the over coming of organizational barriers. Performance is governed by people’s
beliefs about themselves and their environment, so it stands to be true that one must
learn what the beliefs are in an organization before a goal can be developed to institute
change. While gathering information about individual beliefs, it is crucial to listen and
determine if the performance gap is lack of knowledge and skills, insufficient
motivation or some organizational barrier (p. 45).
When reflecting on the industrial model of education that focused on
procedures verses results, it is clear that systems used to educate the child of the global
world must significantly change from those of the industrial world. The organization
must take into account the fact that mastery of rigorous content and developing
individuals who reason and think, is a must for all students as the new young adult is
expected to adapt and produce with a mind that embraces futuristic visions. This is a
far cry from producing individuals that only had to perform under well know
situations mainly within the confines of their hometown.
34
Embedded in culture are core values, goals, beliefs, emotions and processes
(Clark & Estes, 2002, p. 108). In order for change to occur individual differences and
structural situations must be addressed. A sense of value of group process and
collaboration must be instilled; the value of group performance must be seen as out
weighing individual performance. The structure of the approach must account for any
perception of barriers that prevent successfully accomplishing the goal. Teachers must
be able to offer a challenging standards-based approach to learning. When layered
with the need to also accommodate for language levels, one runs the risk of low
teacher expectations and doubts of teacher self-efficacy.
Self-efficacy and Accountability
“Perceived self-efficacy is defined as people’s beliefs about capabilities to
produce designated levels of performance that exercise influence over events. Self-
efficacy beliefs determine how people feel, think, motivate themselves and behave”
(Bandura, 1994, p. 1). Tackling an organization restructure brings with it anxiety and
ultimately questions one’s self-efficacy as the outcome is uncertain. Structuring the
new approach to include challenging goals that are supported within a social model
will allow teachers to see people similar to them succeed by sustained effort or to see
difficult tasks as challenges to be mastered rather than as threats to be avoided
(Bandura, 1994). Monitoring progress through evaluation will allow for intermediate
steps to be analyzed along the way. Evaluation supports whether or not something is
worthwhile. This rings true for both students and instructional staff. Evaluation
provides reliable and valid information about the state of the system that is being
35
changed, whether it is individual student growth or school wide program
improvement. Attempting to close the performance gap requires periodic systematic
evaluation so that mid-course corrections can occur (Clark & Estes, 2002).
Academic success or a need to adjust instruction can only be determined if an
accountability system is engaged. The first step of an accountability system is to
collectively set goals. As Marzano (2003) reports, Schmoker notes not only on the
impact of achievement, but that “Goals themselves lead not only to success but also to
effectiveness of cohesion of a team” (Marzano, 2003, pp. 35-36). Setting goals alone
will not accomplish academic success, but it is the act of monitoring goal accomplish-
ments that allow academic challenges to be met. Timely feedback on specific
knowledge and skills for specific students needs to occur at a minimum, every 9
weeks. An expert teacher has acquired a variety of learning strategies that support not
only grade level standards, but also acquisition of language and knows when to use the
appropriate strategy to ensure all students have an opportunity to develop essential
knowledge (Marzano, 2003).
Summary of the Literature
It comes as no surprise that EL learners are the fasted growing subgroup in
California. It also comes as no surprise that the federal government wants to hold
schools responsible for educating all students, and that the requirements are imposed
before a well thought out system was devised. The current system does not account for
family and cultural backgrounds and generalizes all EL learners as the same.
However, what the current system does prove is that not all students are being given
36
an equal opportunity, as many school site programs are not addressing the needs of all
students. It is the responsibility of local public educational system to develop
programs within their neighborhood schools that takes the wealth of specialized
research and applies it to the community they serve.
Determining the right thing to do for all students is the responsibility of all
educators starting with understanding the principles of learning and developing a
program that is learner-centered. The assessment results no matter how flawed, are
still founded in the fact that not all students are receiving an instructional program that
is preparing them for the global world they are about to enter. Schools need to reflect
on current industrial model programs and build professional learning communities that
together will address the multifaceted child that is entering the local classroom today
and the global world tomorrow.
A restructuring of instructional practice so that barriers can be addressed by
many individuals and not left to be resolved by one, only leads to dissolving restrictive
expectations and lack of self-efficacy. Add a researched-based accountability system
that monitors academic growth that allows for mid-course adjustment and all students,
especially EL learners will be on their way to developing the skills needed as they
enter the global economy.
37
CHAPTER 3
METHODOLOGY
Summary of Research Design
The purpose of this study was to determine if restructuring the English
Language Development (ELD) program at Mitchum Elementary would have an effect
on the academic achievement of English Learners, resulting in increased proficiency
levels on the California Standards Test (CST). The restructuring consisted of placing
English Language (EL) learners into homogeneous language proficiency groups:
(a) Target Time, (b) implementing research-based strategies, and (c) implementing an
accountability system.
A quantitative pre-post design was used to examine the impact of the
restructuring that was instituted in the 2006-2007 school year. The 2005-2006 data
was used as the comparative data. The 0 pre X 0 post design format was used as the
summative evaluation.
1. Pre-test Observation: 2006 California Standards Test
2. Treatment (X): Restructuring the ELD Program
3. Post-test Observation: 2007 California Standards Test
The 2006 CST results school-wide, by grade level and/or substantial (sub)
groups were compared to the 2007 CST results. This comparison determined if the
intervention resulted in improved proficiency school-wide, by grade level and/or
subgroup. Causality was not proven as other factors may account for the growth or
decline, but this study helped to determine if the restructuring had a positive impact on
38
the academic achievement of all learners, but especial EL learners. A nonequivalent
control group design consisted of comparing post-test data only between a control
group, Moonstruck Elementary and the experimental group, Mitchum Elementary.
Moonstruck Elementary was chosen as the control group based on data retrieved from
the 2006-2007 Similar Schools Report.
The intervention implemented at Mitchum Elementary was the restructuring of
the language arts intervention, Target Time, and includes the use of research-based
strategies and an accountability system. Post-test CST data was compared between
both Moonstruck (control school) and Mitchum Elementary (experimental school) and
the pre-post change will be examined in Mitchum Elementary.
Below, tables 2-4 above provide a comparison between Mitchum and
Moonstruck Elementary in three general areas: Percentage of students scoring
proficient on the CST in language arts for the sub groups reported on AYP reports,
school demographics and API scores. Moonstruck and Mitchum have comparative
scores of proficiency in each of the sub groups. The largest difference is in the
subgroup of social economically disadvantaged at 8.2%, but is not considered to be
statistically significant so the schools can still be considered comparative schools. The
second table shows each school’s demographics. Mitchum Elementary has slightly
more students in three of the subgroups, but relatively speaking the schools are still
closely aligned in their demographics and should not be considered unequally
matched. As for the comparison of each sub groups API scores, both schools are
39
Table 2. 2005-2006 Similar School Comparison Percent Proficient
School
H/L
White
SD
EL
Mitchum 21.6% 55.2 18.3 20.8
Moonstruck 24.5 52.7 26.5 22.1
Table 3. 2005-2006 Similar School Comparison School Demographic Characteristics
School
Hispanic
White
Participants in free
or reduced lunch
English
Learners
Mitchum 52% 35% 44% 32%
Moonstruck 40% 54% 38% 20%
Table 4. 2005-2006 Similar School Comparison API Base by Subgroup
School
Hispanic
White
Socioeconomically
disadvantaged
English
learners
School
base
Mitchum 692 802 692 668 741
Moonstruck 650 795 649 632 735
within 45 points of each other, providing a foundation for growth that is comparative
equal.
A qualitative evaluation component is included in chapter five, providing an
assessment of the restructuring intervention. Teachers were interviewed during
academic conferences using open-ended questioning and walk-through observations
provided an opportunity to monitor the implementation of strategies.
40
Participants/Sampling
Targeted participants in the study consisted of students in the second through
sixth grade, subgroups as defined by AYP and English Learners, as identified by the
California English Language Development Test (CELDT). The 2005-2006 CST
results were compared to the 2006-2007 CST results Grades 2-6 school-wide, by grade
and subgroups with regard to an increase or decrease of percentage of students scoring
in three proficiency levels: (a) Advanced, (b) Proficient and (c) Basic.
Teachers and administration were considered additional participants in the
restructuring of the accountability section of the ELD program, as information was
obtained through academic conferences, grade level meetings and walk-through
observations. The data collected from teachers by administration was not used as part
of the data comparison, but was used to determine the degree of implementation of the
strategies and how the accountability system influenced instructional decisions. This
data also forms the basis for the recommendation that is in chapter 5.
Intervention Description
In Darling-Hammond’s (2002) chapter 5: Structuring Learner-Centered
Schools, the case is made that there are organizational features and educational
commitments that support achieving success for all students. Mitchum emphasized
three of Darling-Hammond’s (2002) 12 characteristics for this study: (a) school
purpose and focus, (b) high and universal academic standards, and (c) performance-
based assessment and accountability aimed at clearly stated competencies. In
analyzing these areas Mitchum Elementary School made changes for the 2006-2007
41
school year. The staff understands the need to have a strong purpose and high
academic standards, as well as the need to use performance-based assessments.
Teachers focused on mastery for every student of the essential standards. Grade levels
utilized curriculum assessments to monitor students regularly. Language arts district
assessments were used each trimester to determine benchmark growth throughout the
year. Teachers implemented re-teaching as necessary.
In completing academic conferences with individual teachers in 2005-2006, a
barrier that presented itself was that of reaching multiple language levels and broad
proficiency levels of students within a single classroom. In developing the 2006-2007
academic schedule, a school-wide Target Time was implemented. During Target
Time, students are assigned to a group by their language level, as deemed by the
CELDT assessment (EL students) or proficiency levels, as deemed by the CST
assessment (English only students). Instruction is then provided for students’
individual needs within their language or proficiency level. Target Time is designed to
provide English language development for English learners, extra support for below
proficient English only students and enrichment activities for those students scoring
proficient or above. Students are reassigned as curriculum unit assessments prove
student mastery utilizing flexible groupings.
Target Time enables teachers to reach specific students’ needs, but it does not
take the place of core instruction. In order to support students during core instruction,
two things were developed in the classroom throughout the day: (a) the teacher’s use
of metacognitive skills for students, and (b) the use of formative informal assessments
42
daily during each academic lesson. Student set goals engages students in their own
learning.
Marzano (2003) and Resnick and Hall’s (2003) research support, students who
set their own learning goals are actively engage in their own learning process. Student
set goals were introduced to increase students metacognitive skills. Setting the goal of
instituting socialized intelligence; the belief of the right one has of knowing what is
going on around them, the skill of building metacognition to problem-solve and reason
and the disposition of creating habits of the mind and a positive, mindful disposition
toward learning (Resnick & Hall, 2003), the classroom routinely challenges learners
and teachers to use metacognitive strategies. Creating the metacognitive toolkit; the
direct teaching of ways to memorize, ways of using resources, a repertoire of smart
things to do in reasoning, problem-solving, and decision making (Resnick & Hall,
2003) will increase the quality of teaching and learning within each classroom.
The explicit direct instruction model that staff has been trained in for the last 3
years supports the endeavour to increase metacognaive skills and high universal
academic standards for all students. Embedded throughout the lesson format is
checking for understanding, using Bloom’s taxonomy, higher order thinking
questioning affords students the opportunity to develop their metacognative level of
engagement.
The restructuring of the language arts intervention has three areas of
implementation. Focusing the restructuring on the English language learner, the first
area is that of restructuring student grouping from heterogeneous language level
43
groups to homogeneous flexible language level group. A common uninterrupted ELD
time was established school-wide. Teachers grouped students in the beginning of the
year by their 2005-2006 CELDT language proficiency level, moving students up a
level as they become proficient. Groups were adjusted when the 2006-2007 CELDT
results became available.
The second area of implementation is the use of researched-based strategies by
the teachers. Using researched-based strategies created opportunities for students to
engage in daily metacognative development. Staff read, Classroom Instruction that
Works by Robert J. Marzano (2001) and Section II: Teacher-Level Factors in What
Works in Schools by Robert J. Marzano (2003). As a staff, we identify strategies
already being utilized. Strategies that are not researched based will be replaced with
researched-based strategies, creating a school-wide consistent use of research-based
strategies. Checking for understanding questioning was based in Bloom’s taxonomy,
enriching students understanding of concepts.
The final area of implementation is a multidimensional accountability system
that holds students, teachers and administration accountable. Through the use of
student engagement activities, such as “Think-Pair-Share,” white board response, and
summarizing activities, students were held responsible for their own learning during
direct instruct. Students’ responses allow teachers to monitor student achievement
daily for mastery. Teachers provided reteaching of content as needed. Regularly
scheduled grade level meetings allowed teachers to adjust the homogeneous language
level groups often as results are analyzed. Flexible grouping of students was a focus so
44
as students showed mastery of content they were moved to the next level. Teachers
meet individually monthly with administration and multiple times in grade level
groups to discuss student growth. Each trimester district assessments were reviewed.
Open-ended questioning was used by administration to generate teacher responses
revealing their planning ideology and implementation processes. Administration
conducted walk-through and formal observations to monitor implementation fidelity to
the research-based strategies and to the EDI lesson format.
Quantitative Instrumentation
Quantitative data consisted of data reported by the California Department of
Education Website, Standardized Testing and Reporting (STAR) Program. The state
reports students’ scores as the percentage of students scoring within each proficiency
level, Advanced, Proficient, Basic, Below Basic, and Far Below Basic by grade level.
A comparison was made between two groups: (a) percentage of students scoring
within Basic, Proficient and Advanced, and (b) then students scoring in Proficient and
Advanced. Percentages were compared school-wide, by grade level and by subgroups
from the 2006 to 2007 CST results to determine if there was an increase or decrease of
the percentage of students scoring within the two proficiency groupings.
Qualitative Instrumentation
Academic conferences were held throughout the year. Qualitative data was
collected using an informal interview format. Academic conferences consisted of one-
on-one meetings with each individual teacher and administration. The primary focus
was on the academic growth of EL learners. The interviews provided information with
45
regard to each teacher’s ideology of instruction, ELD focus if they were the ELD
teacher, implementation of research-based strategies, and the monitoring of EL
learners growth during core instruction. The interviews conducted by administration
consisted of open-ended questioning with a strong focus on EL learners:
1. Has Target Time made a difference in EL achievement? How do you
know?
2. What researched-based strategies have you implemented this year that
support your EL students?
3. How does your grade level use data when reformulating your EL student
groups?
4. How do you know if the strategy/strategies has/have improved student
learning?
5. How have you used the EL Matrix to improve instruction for the EL
learners during core instruction?
6. What data do you use to determine there has been growth or not?
7. What student engagement strategies have proven successful and how do
you know they have been successful?
8. How have you changed your instruction this year for EL students from last
year?
9. How has the ELL behavior changed in your classroom as a result of your
instructional changes?
46
10. How has the use of the EL Matrix helped EL students develop language?
11. Was Target Time Successful? How do you know?
Procedural Timeline
July 2006: Bell schedule reflected a common Target Time for all grade levels
Beginning of August 2006: Teachers used 2006 CELDT and CST results to
develop Target Time flexible groups. Adjustments to groups were on going
throughout the year. Staff read What Works in Schools; Section II by Robert J.
Marzano (2003) and Classroom Instruction that Works by Robert J. Marzano
(2001).
End of August 2006: First Academic conferences were held to determine
research-based strategies that would be implemented.
September 2006: Academic conferences--Questions 1 and 2.
October 2006: Academic conferences--Questions 3, 4 and 5.
End of October 2006: Language Arts district trimester data collected.
November 2006: Academic conferences--Questions 6 and 7.
February 2007: Academic conferences--Questions 8 and 9.
March 2007: Language Arts district trimester data collected.
March 2007: Academic conferences--Question 10 and 11.
May 2007: It was determined that Target Time will be implemented for the
2007-2008 school year.
47
Quantitative Analysis
A quantitative analysis was used to determine if the pre/post means were
statistically significant (p. 15). Statistical significance determined that the intervention
was responsible for academic gains/losses or if the gains/losses were purely due to
chance. The test used for a pre/post comparison was the dependent groups t-test. To
determine practical significance, effect size, the Cohen’s d test was used. The effect
size is the ratio of gain to the standard deviation of the pretest. Although scientists
argue that pre-post design gains can be influenced by other factors, this design was
useful in evaluating the intervention program. A second and third index of effect size
was the raw changes and the percentage gain on the CST.
Qualitative Analysis
The data collected from conducting informal interviews, Academic
conferences using an interview guide was structured as follows:
1. Review and analyze data from Academic conferences
2. Identify common themes from responses
3. Describe themes in detail
4. Personally interpret themes
Delimitations and Limitations of the Study
The study has limitations as it is not a randomly selected experiment and the
use of a pre-post design allows for an influence of many factors in this particular
setting, causing internal invalidity. Using disaggregated CST and CELDT data and a
control group increased the internal validity of the study. The analysis provided
48
important information related to student engagement, lesson design, implementation of
research-based strategies and the monitoring of student achievement.
49
CHAPTER 4
RESULTS
The summative evaluation portion of the study incorporated three dependent
variables and both a pre-post independent groups design and a nonequivalent
comparison group design for Mitchum Elementary. The nonequivalent comparison
group design was used for comparing the posttest results of English Learners in
Grades 2 through 6 at Mitchum Elementary (experimental group) and Moonstruck
Elementary (control group). The three dependent variables were: (a) CST ELA
performance band scores, (b) the percentage of students who scored “basic and above”
on the CST ELA, and (c) the percentage of students who scored “proficient and
above” on the CST ELA. The performance band scores were coded as follows: 0 =
Far Below Basic; 1 = Below Basic; 2 = Basic; 3 = Proficient; 4 = Advanced.
1. Pre-post independent groups design. This design used to analyze the
change at Mitchum Elementary from 2005-2006 (preintervention) to 2006-
2007 (post-intervention). The following statistics were used for the three
CST ELA performance band scores: (a) An independent groups t-test to
assess the statistical significance of the change (p < .15); (b) Cohen’s d to
assess effect size (criterion for practical significance (d > .20); (c) Raw
change from 2006 to 2007 to assess practical significance (criterion for
practical significance = 10% improvement); and (d) Percentage change to
assess practical significance (criterion for practical significance = 10%
improvement).
50
2. Nonequivalent comparison group design. This design included an
experimental group (Mitchum Elementary) and one comparison group
(Moonstruck Elementary), not randomly assigned. Mitchum and
Moonstruck Elementary were compared using the post-test CST ELA data.
The treatment was administered only at Mitchum Elementary. The
treatment of implementing the restructuring of the language arts
intervention, Target Time were students are grouped by language levels as
deemed by the CELDT and CST proficiency levels and includes research
based strategies that focus on aligning lessons to standards and an
accountability system.
Pre-Post Independent Groups Results
Table 5 shows the pre-post significant test findings (p < .15) for the
experimental school (Grades 2 through 6) and the school’s three significant subgroups:
(a) Hispanic or Latino (H/L), (b) Socioeconomically Disadvantaged (SD), and
(c) English Learners (EL).
Row one results shown in table 5 indicate that overall in the experimental
school there was not a significant increase in the CST ELA performance bands from
2006 to 2007, t (1232) = +.66, p < .150. However, statistically significant results
(p < .15) were found for three subgroups. Among the 4
th
grade students, the observed
gain from 2006 to 2007 was .27 (more than a quarter of a performance band), t (251) =
-1.84, p = .067. As for the 5
th
grade students, the observed gain from 2006 to 2007 was
.24 (just under a quarter of a performance band growth), t (243) = -1.68, p = .094. In
51
Table 5. Pre- versus Post-Intervention CST ELA Performance Band Differences: Statistical Findings
Grouping
Pre N
2006
Post N
2007
Pre M
2006
Post M
2007 Difference t-ratio
Observed
prob.
School 611 623 2.09 2.14 +.05 +.66 .511
Grade 2 120 126 1.97 2.10 +.13 +.82 .416
Grade 3 116 122 1.81 1.62 -.19 -1.33 .185
Grade 4 126 127 2.29 2.56 +.27* +1.84 .067
Grade 5 125 120 2.03 2.27 +.24* +1.68 .094
Grade 6 124 128 2.33 2.12 -.21* -1.46 .147
H/L 340 383 1.73 1.84 +.11 +1.34 .180
SD 290 341 1.67 1.76 +.09 +1.02 .308
EL 200 219 1.35 1.48 +.13 +1.36 .175
Note. * = p < .150
regard to 6
th
graders, the intervention had the reverse effect as 6
th
grade students
observed a lost from 2006 to 2007 as the effect size was -.21 (almost a quarter of a
performance band), t (250) = +1.46, p = .147. A decline, although not considered
statistically significant, was observed in 3
rd
grade. Students in 3
rd
grade declined from
2006 to 2007 and the effect size was -.19 (almost a quarter of a performance band),
t (236) = -1.33, p =.185. Disaggregation of the data on the basis of ethnicity and
language classification indicates positive findings for the Hispanic/Latino and English
Learners subgroups that are higher than the school-wide gains, but is not quite enough
to be considered statistically significant.
Because practical significance is highly dependent on sample size and very
large differences on a practical level can be “statistically” insignificant, the size of
each difference was assessed in three ways: (a) raw change from 2006 to 2007,
(b) effect size, and (c) percentage change. The raw change is the post-test score minus
52
the pretest score. Effect size was computed using the ratio of the change from 2006 to
2007 to the pretest standard deviations. Percentage change was assessed using the
ratio of the change from 2006 to 2007 to the pretest means. Results are shown in
Table 6.
Table 6. Pre- versus Post-Intervention CST ELA Performance Band Differences: Practical
Significance
Grouping Pre-M Pre-SD
Pre-Post
change
Effect
size
%
change
School 2.09 1.18 .05 +.04 .02
Grade 2 1.97 1.23 .13 +.11 .07
Grade 3 1.81 1.18 -.19 -.16 -.11*
Grade 4 2.29 1.21 .27 +.22* .12*
Grade 5 2.03 1.09 .24 +.22* .12*
Grade 6 2.33 1.14 -.21 -.18 -.09
H/L 1.74 1.09 .11 +.10 .06
SD 1.67 1.04 .09 +.09 .05
EL 1.35 1.02 .13 +.13 .10*
Note. * = effect size > .20 and % change > .10.
Except for the EL subgroup the pretest means in table 6 are all above 2.00. In
2006, a majority of the students assessed in the experimental elementary school
(Mitchum) were performing at the Basic level or higher in English-Language Arts.
Using Grades 2 through 6 of the experimental school as the sample: (a) 12% of the
students were performing at FBB, (b) 17.2% were at BB, 34.8% at the Basic level,
(c) 24.6% at the Proficient level, and (d) 11.6% of the students were at the Advanced
level on the CST ELA pretest.
53
Table 6 also displays the practical significance for each of the targeted groups.
Practical significance was examined in three ways:
1. Raw change. In two instances, the degree of improvement from 2006 to
2007 was about .25 of a performance band. The most substantial
observed gain from 2006 to 2007 was among the 4
th
grade students at .27
(more than a quarter of a performance band) and among 5
th
grade
students, the observed gain from 2006 to 2007 was .24 (not quite a
quarter of a performance band). Even though the English learner’s results
did not calculate as being statistically significant, it was not missed by
much and the observed gain from 2006 to 2007 was .13 of a performance
band.
2. Effect size. Two groups exceed the prestudy standard of .20, specifically,
4
th
and 5
th
grade students (both by .22). The effect size for both grades
was just less than a quarter of a standard deviation.
3. Percent change. Another way to assess practical significance is to
examine the percentage change in the performance band scores from pre-
to post-. The preset standard for practical significance was 10%. The last
column in table 2 shows that two grades exceeded 10% from 2006 to
2007, 4
th
and 5
th
grade (.12). The last column also shows that the EL
subgroup obtained the goal of exactly a 10% increase. The loss of over
10% in 3
rd
grade (-.11) and almost 10% in 6
th
grade (-.09) are both
noteworthy and concerning.
54
Tables 7 and 8 summarize the pre-post results for two additional indices:
Percentage of the students who scored basic and above and the percentage of the
students who scored proficient and above (per NCLB). The reason that the percentage
of students scoring Basic and above was added, in addition to the NCLB requirement,
was that in an urban school with a significant number of EL students, the NCLB goal
of 100% Basic and Above is a more pragmatic target for 2014.
Table 7. Pre- versus Post-Intervention CST ELA Percent Basic and Above Percent Basic and Above
Grouping Pre (2006) Post (2007) Pre-Post change % change
School .71 .70 -.01 -.01
Grade 2 .70 .68 -.02 -.03
Grade 3 .60 .53 -.07 -.12
Grade 4 .75 .83 .08 .11*
Grade 5 .71 .77 .06 .08
Grade 6 .79 .69 .10* .13*
H/L .62 .63 .01 .02
SD .57 .60 .03 .05
EL .48 .49 .01 .02
Note. * = % change > .10.
The findings in Tables 7 and 8 are quite different then what was reported in Tables 5
and 6. That is, a significant percentage change in CST ELA performance occurred in
students scoring proficient and advanced: 2
nd
(18%), 4
th
(21%), 5
th
(33%), 6
th
(13%)
grade and in the Hispanic/Latino (32%) and socioeconomically disadvantaged (26%)
subgroups as well as school-wide (14%). Tables 3 and 4 also illustrate the decline in
achievement in Grade 3, -.12% change for students achieving Basic and Above, and a
-28% change in students achieving Proficient and Above on the CST-ELA. Although
55
Table 8. Pre- versus Post-Intervention CST ELA Percent Proficient and Above
Percent proficient and above
Grouping
Pre (2006)
Post (2007)
Pre-Post change
% change
School .36 .41 .05 .14*
Grade 2 .34 .40 .06 .18*
Grade 3 .32 .23 -.09 -.28*
Grade 4 .44 .53 .09 .21*
Grade 5 .33 .44 .11* .33*
Grade 6 .38 .43 .05 .13*
H/L .22 .29 .07 .32*
SD .19 .24 .05 .26*
EL .12 .12 .00 .00
Note. * = % change > .10.
several of the changes were more than a 20% improvement, the results have to be
qualified to the extent that the pretest rates were generally in the low to mid 30%
range, and thus even a smaller change will exceed our prestudy criterion of 10%. Only
Grade 5 had more than a 10% change in the number of students scoring proficient and
above. A major concern is the sub group least positively affected was that of the
English learners.
A final measure used in this study is the California English Language
Development Test (CELDT), which is a test of English Language Proficiency required
in California public schools each year for English learners. The test which is aligned
with the English Language Development standards approved by the State Board of
Education measures a student’s proficiency of English language skills in four areas:
(a) listening, (b) speaking, (c) reading, and (d) writing. The CELDT scores define five
levels of performance on the test. The proficiency levels are: (a) Beginning, (b) Early
56
Intermediate, (c) Intermediate, (d) Early Advanced, and (e) Advanced. Students in
Kindergarten and the 1
st
grade receive an overall score for listening and speaking
while students in Grades 2-12 receive an overall score that incorporates listening,
speaking, reading and writing, as well as a comprehension score for listening and
reading. Table 9 depicts the overall performance levels for the CELDT assessment for
2006 and 2007.
CELDT Form F results, administered in 2006-2007, are reported using a new
common scale, which will allow year-to-year comparisons to be made in the future.
Prior to the 2006-2007 CELDT administration, the year-to-year comparisons were not
possible. A note of caution is that the summary results for Form F are not to be
compared with the CELDT results of any previous years (Forms A–E), and thus the
results herein are for descriptive purposes only (Table 10).
Comparison School
The design of this study incorporated the use of a comparison school,
Moonstruck Elementary School. As mentioned earlier in chapter 1, this school was
selected from the similar schools list of the 2005-2006 Accountability Progress
Reporting (APR) for the experimental school, Mitchum Elementary, from the CDE
website for schools with similar demographics. The experimental school and
Moonstruck Elementary, the control school, share similar demographics, namely,
Hispanic or Latino (H/L), Socioeconomically Disadvantaged (SD), and English
Learners (EL) subgroups.
Table 9. CELDT Assessment Results: Overall Performance Levels, 2006 and 2007
Year
Total
N
# of adv
students
%
of
adv
# of
early
adv
students
% of
early
adv
# of
intermed.
students
% of
intermed.
# of
early
intermed.
students
% of
early
intermed.
# of
students
%
of
beg
2007 275 15 15% 68 25% 130 47% 49 18% 13 5%
2006 258 14 5% 71 28% 110 43% 57 22%
6
2%
57
58
Table 10. API School-wide Comparison, 2006 and 2007
School API 2006 API 2007 Gain or loss
Experimental school 736 747 +11
Control school 735 727 -8
In the 2005-2006 school year, English learners as a cohort group were
comparable in numbers (control school = 192 English learners; experimental school =
329 English learners for students tested in Grades 2 through 6). Additionally, 22.1% of
the English learners in Grades 2 through 6 in the control school achieved grade level
proficiency on the 2006 CST-ELA, whereas only 20.8% of English learners in the
experimental school achieved grade level proficiency on the same assessment, falling
short of the State target of 24.4% by 3.6%. Table 6 shows the comparison between the
experimental and control schools for the State accountability of API in the 2006 and
2007 school years. Table 11 depicts the Subgroup API for the experimental and the
control schools for the Hispanic or Latino, socioeconomically Disadvantaged, and
English Learners subgroups. The API scores range from a low of 200 to a high of
1,000 and according to the Public Schools Accountability Act of 1998, every school is
expected to achieve an API score of at least 800.
Table 12 provides information for the experimental and control schools on the
federal accountability measure known as Adequate Yearly Progress (AYP), which
originates from the accountability system under No Child Left Behind (NCLB). AYP
focuses on the achievement of numerically significant subgroups of students within
schools. The primary measure of success is the achievement of a specific, and
59
Table 11. API Subgroups Comparison, 2006 and 2007
Subgroup Experimental school Control school
2006 2007 Difference 2006 2007 Difference
Hispanic or Latino 678 697 +19 666 667 +1
Socioeconomically
disadvantaged
674 683 +9 667 660 -7
English learners 668 689 +21 657 636 -21
gradually increasing, percentage of students in each subgroup scoring “proficient” or
“advanced” on the California Standards Tests in English-Language Arts and
Mathematics. In addition, schools are accountable for testing 95% of students within
each subgroup with sanctions applied for failure to do so. Escalating consequences
ensue for schools and districts that fail to make their AYP for 2 consecutive years
within a process known as Program Improvement (California Department of
Education, 2005).
Table 12. AYP English-Language Arts for the Experimental and Control Schools, 2007
Groups
Experimental
percent at or above
proficient
Control
percent at or above
proficient
School-wide 41.4% 37.3%
Hispanic or Latino 29.2% 22.6%
Socioeconomically Disadvantaged 24.8% 24.8%
English Learners 27.9% 18.2%
The AYP target for percent proficient for 2007 is 24.4% in English-Language
Arts. Table 8 shows that both the experimental and the control schools surpassed the
AYP target of 24.4% school-wide. When comparing the two schools, however, the
60
control school did exceptionally well considering that the AYP target for 2008
increases to 35.2% in English-Language Arts. The Hispanic or Latino subgroup at the
experimental school achieved the highest rate of 29.2% in comparison to the other
groups whereas in the control school the Socioeconomically Disadvantaged subgroup
achieved the highest rate of 24.8%.
61
CHAPTER 5
CONCLUSION
Overview
Implementing a restructuring of the English language arts intervention program
that included the use of research-based strategies and an accountability system has
influenced staff involvement and student success. In analyzing the data from this
study, it is important to consider the effect that the intervention had on teachers’
professional practice and students’ proficiency levels. The following chapter
concludes this study by elaborating on the implications of the quantitative findings and
conclusions drawn from the qualitative data gathered in this study. Finally,
recommendations for Mitchum Elementary are provided.
Mitchum’s English language (EL) learners’ growth, although positive, has
been slow in past years. Since the focus for EL learners in levels 1-3 is language
development and not California standards, this means that in the past 3 years 64% of
the EL learners’ main instructional focus has been language development, not
California State standards. Although, an increase in language development is needed
to successfully improve on the state standards, these students are not able to access the
standards the same as their English only classmates.
The goal of my interventions was to develop an English language development
program that supports EL students in successfully moving through one language
proficiency level each year, provide instructional strategies that allow EL learners to
access state standards’ and allow English only students to receive instruction that best
62
met their needs as well. The new program consisted of a restructuring, Target Time. In
this design, EL learners are grouped by language level and English only students are
grouped by CST results. In addition, a strong emphasis was given to research-based
strategies and an accountability system that allowed monitoring of student growth.
Purpose and Design
The purpose of this study was to examine the impact (positive, negative or
neutral) of restructuring Mitchum Elementary’s language arts intervention program
with inclusion of research-based strategies and an accountability system on the
academic achievement school-wide, by grade level and by subgroups: Hispanic (H/L),
socioeconomically disadvantaged (SD) and English language learners (ELL) at
Mitchum Elementary School. Specifically, the change in performance bands of the
California Standards Test (CST) from preintervention (2005) to post-intervention
(2006) were analyzed school-wide, by grade level for all students and by sub groups in
the experimental school, Mitchum Elementary School. Participants in this study
consisted of the student population (872 students), but results reported are only those
of students in second through 6
th
grade at Mitchum Elementary School, as CST
assessments are not administered to kindergarten or 1
st
grade. Of the participants,
approximately 383 (44%) of the students are identified as EL learners school-wide.
An additional group of participants was comprised of the staff at Mitchum
Elementary School. During the study, there were 38 certificated staff members
including teachers and administrators and four classified push-in paraprofessionals.
During academic conferences and grade level meetings the adult participants at
63
Mitchum Elementary analyzed data to determine the impact on student achievement
and shared their beliefs regarding the barriers involved in preventing EL learners from
accessing California state standards, how restructuring the program supported their
professional planning, which research-based strategies were implemented, and how
the accountability system impacted lesson design. The intent of including adult
participants was to gather information regarding on how decisions about lesson design
were determined and the effect of such decisions, as well as how the restructuring
affected teachers’ self-efficacy with regard to supporting EL learners in language
acquisition. The results of the qualitative data are reported as common themes and will
be discussed in this section concerning the experimental school, Mitchum Elementary
School. In total, every teacher at Mitchum Elementary was interviewed 6 times from
August 2006 to March 2007. Documents and material analysis consisted of meeting
agendas, curriculum assessments, lesson plans and trimester assessments. The
qualitative data collected supports this study in determining the self-efficacy of staff,
the staff’s use of research-based strategies and the school-wide use of data to guide
instruction.
A control school, Moonstruck Elementary, was used to compare average
growth both school-wide and by grade level. Trends can be compared either positively
or negatively. Moonstruck is considered comparative to Mitchum Elementary as they
are listed on the California Department of Education list of comparative schools and
are reasonable equal demographically, socioeconomically and within the same school
district.
64
Summary of Findings: Mitchum Elementary School
The following section provides evidence in answering the overarching research
question that guided this study: Does the restructuring of the language arts
interventions implemented in the 2006-2007 school year, which included the use of
research-based strategies and an accountability system, have an effect on the academic
achievement of EL learners as measured by CST scores? Overall, the depth of the
impact that the restructuring had on the academic achievement of all students at
Mitchum Elementary School was positive. Given that there was an 11 point increase
in the Academic Performance Index (API) scores of the entire school from 736 points,
preintervention (2006), to 747 points, post-intervention (2007), on the surface, the
intervention provided a positive outcome for Mitchum Elementary School.
Even more impressive growth was made when looking at the increase of API
scores by subgroups: (a) the Hispanic/Latino (H/L) moved from 678 to 697, an
increase of 19 points; (b) Socioeconomically Disadvantaged (SD) moved from 674 to
683, an increase of 9 points; and (c) the English Language (EL) learners increased the
most from 668 to 689, an increase of 21 points.
When reviewing the findings of performance bands by grade level for the
experimental school, Mitchum Elementary School, positive changes were made in all
grade levels except 3rd grade. Although, the 2nd and 6th grade difference from 2006
to 2007 is not considered significant (p < .15) there was a positive difference. The 4
th
and 5
th
grade change is considered statistically significant in size (p < .15). The
positive increase can be attributed in part to the reason Mitchum Elementary School
65
achieved its Adequate Yearly Progress (AYP) and API. Mitchum Elementary has been
frozen in Year 1 of Program Improvement. If a positive increase is experienced in the
2007-2008 school year, Mitchum Elementary will be out of Program Improvement
status.
Based on second grade overall findings, the 2nd grade had a mean change of
13%. Although not a statistically significant difference, it reflects a consistent positive
change in moving 2nd graders toward the performance categories of “proficient” and
“advanced.” Percentages of students decreased at the “far below basic” by 5%,
increased in the “below basic” by 6% and decreased in “basic” by 8%, an increase of
3% in the both performance bands “proficient” and “advanced.” Overall, 2nd grade
increased students performing in the proficient and advanced bands by 6%.
Third grade overall findings demonstrated an overall mean change of -19%.
Although the change is negative overall, it is important to note that the percentage of
students performing at the “far below basic,” “below basic,” and “basic” levels shifted
by a swing of -2% to -4%. A strong focus has to be given to the “proficient” and
“advanced” students who slid as that swing was much more significant at -9%.
The overall findings at the 4th grade level were significant. While there was an
overall positive mean change of 27%, the performance category bands reflect a more
significant development in proficiency. Fourth graders experienced a decrease in the
“far below basic” of 5%, a decrease in “below basic” of 2%, a decrease in “basic” of
1% and an increase in “proficient” of 2% with the largest increase in the “advanced”
performance band of 7%. Showing a strong shift toward proficient and advanced.
66
Overall, 4th grade increased students performing in the proficient and advanced bands
by 9%.
At the 5th grade level, the overall mean change reflected 24% growth. Fifth
graders decreased the percentage in “far below basic” by 1%, decreased in “below
basic” by 5% and increased “basic” by 11%. The performance bands of “proficient”
increased by 3%, but by far the most significant change for 5th grade was that of the
increase of 8% in the “advanced” performance band. Overall, 5th grade increased
students performing in the proficient and advanced bands by 11%.
At the 6th grade level, the overall mean change reflected -21% difference.
Sixth graders increased the percentage in “far below basic” and “below basic” by 5%
in each band. There was a huge decrease of 15% in the “basic” performance band, and
an increase in “proficient” of 5% so some students did show positive increase in
performance levels. The “advanced: performance band held steady at 12%. Overall,
sixth grade increased students performing in the proficient and advanced bands by 5%.
When viewing the school as a whole, Mitchum Elementary School, made
positive change across all performance bands except “below basic.” “Far below basic”
was decreased by 1%, “below basic” was increased by 2%, “basic” was decreased by
6%, “proficient” was increased by 2% and the performance band of “advanced” was
increased by 3%. The basic premise demonstrates that Mitchum Elementary School
made positive change towards moving their total student population towards higher
academic achievement. Mitchum Elementary School lowered the below proficient
levels of “far below basic” and “ basic” while increasing the “proficient” and
67
“advanced” groups of students, thereby moving more students closer to proficiency of
state standards as required by No Child Left Behind (NCLB).
Taking a closer look at the individual subgroups of H/L, SD and EL, the
primary reason for implementing the intervention, positive improvement was unveiled
with each of these subgroups as well. Although, none of the findings are considered
statistically significant (p < .15), all are found to be positive improvement. The H/L
subgroup showed an increase overall in the CST language arts performance bands by
11%, SD subgroup by 9% and the EL subgroup by 13%.
Findings for the H/L subgroup were positive overall as there was a mean
change of 11%, the performance category bands reflect a more significant develop-
ment in proficiency as great growth was experienced in moving students from “basic”
to “proficient” or higher. The H/L subgroup experienced a decrease in the “far below
basic” of 2%, an increase in “below basic” of 2%, a decrease in “basic” of 7% and an
increase in “proficient” of 5% and an increase in the “advanced” performance band of
2%. Even with the increase in “below basic,” it still supports a shift toward proficient
and advanced.
The SD subgroup findings were the lowest of the three subgroups, but
nonetheless they were still positive. While there was an overall positive mean change
of 9%, the performance category bands reflect a more significant development in
proficiency. The SD subgroup experienced a decrease in the “far below basic” of 2%,
an increase in “below basic” of 3%, a decrease in “basic” of 7% and an increase in
both “proficient” and “advanced” of 3%. Resulting in a 6% increase of students
68
scoring in the “proficient” and “advanced” performance bands. Once again, even
though there was an increase in “below basic,” there is still a shift towards students
scoring proficient or higher.
The EL subgroup findings were the most significant of the three subgroups.
While there was an overall positive mean change of 13%, the performance category
bands reflect some interesting movement. The EL subgroup experienced a large
decrease in the “far below basic” of 7%, an increase in “below basic” of 5%, an
increase in “basic” of 1% and a decrease in “proficient” of 2% with an increase in the
“advanced” performance band of 2%. Even with the increase in “below basic” and
“basic,” there is still a shift in proficiency including an increase in the “advanced”
performance band.
In conjunction with the above quantitative findings, it is extremely significant
to note the qualitative findings. These findings provided data in answering the
formative research questions. The questions have been grouped into three categories:
Category 1: Restructuring Student Groups
1. Has Target Time made a difference in EL achievement? How do you
know?
2. Was Target Time Successful? How do you know?
These two questions were purposely place at the beginning of the year
(question 1) and at the end of the year (question 2), so to determine teacher’s self-
efficacy and actual program performance. Question one allowed teachers to share their
69
excitement about how they now better serve EL learners because of their ability to
group students by language levels.
A year earlier, when asked why ELD sessions were difficult, unanimously
teachers stated the difficulty they have serving all proficiency levels at one time. Once
the idea of only serving one to two ELD levels during a single session (Target Time)
was implemented, the fact that the task was too difficult (External), and that they did
not have the ability (Internal) to be successful in this model, was replaced with
excitement as they controlled how the groups were configured and lesson design. They
were willing to provide more effort as the task became more doable (Table 2, Pintrich,
2003, p. 672). This then led right into increased self-efficacy.
The low self-efficacy had encouraged the lack of monitoring during ELD
sessions. By restructuring the ELD sessions this major obstacle was removed and by
adding the requirement to monitor EL progress during the ELD sessions teachers’ self-
regulation of student progress was heightened. The restructuring approach did
determine that in some cases there was a lack of teacher knowledge regarding
language acquisition.
Although the California Department of Education warns that the California
English Language Development Test (CELDT) of 2006-2007 cannot be compared to
the 2005-2006 test due to the new “common scale,” It is worth noting that the 2006-
2007 CELDT results did show a shift towards more students meeting the Annual
Measurable Achievement Objectives (AMAO) 1 and 2. AMAO 3 is based on the LEA
meeting the percentage proficient targets district wide.
70
NCLB requires Title III-funded districts and the state to meet three AMAOs,
the first two of which relate specifically to English language proficiency.
1. AMAO 1 measures improvement in the percent of LEP students who
make progress in learning English.
2. AMAO 2 measures improvement in the percent of LEP students who
attain English language proficiency.
3. AMAO 3 measures improvement in the percent of LEP students who are
successful on the state reading and mathematics tests used in AYP
determinations. The performance targets for AMAO 3 are the AYP
targets already in place. (CDE, 2006/2007)
Mitchum Elementary had an increase of 4% more EL students meeting the
AMAO 1 target. The AMAO 1 target is the percent of students making annual
progress in learning English, EL students in “beginning,” “early intermediate” or
“intermediate” meeting the annual growth target of gaining one proficiency level on
the CELDT. The AMAO 1 target for students in the “early advanced” and “advanced”
level are to achieve English proficiency on the CELDT. A substantial increase in
students meeting the AMAO 2, the percent of EL students attaining English
proficiency on the CELDT, was reported as a 40% increase. Even though a causal
relationship concerning the intervention cannot be made, it is clear that a considerable
amount of EL students at Mitchum Elementary did move towards English proficiency.
The second category of research questions regarding implementation of
research-based strategies follows:
71
Category 2: Implementing Research-based Strategies
3. What research-based strategies have you implemented this year that
support your EL students?
4. How do you know if the strategy/strategies has/have improved student
learning?
5. What student engagement strategies have proven successful and how do
you know they have been successful?
6. How have you changed your instruction this year for EL students, from last
year?
7. How has the ELL behavior changed in your classroom as a result of your
instructional changes?
As a site, Mitchum Elementary made the decision to focus on continuity across
the site with regard to implementing research-based strategies. Implementing the
restructure of language arts was going to be a large undertaking so the decision was
made to only choose two research-based strategies school-wide. Every teacher agreed
to implement the following two strategies. The first strategy focused on student
engagement. All teachers increased the amount of “think-pair-share” questioning. The
idea is to have all students engaged in answering checking for understanding questions
for each and every question. This then provided the opportunity for the teacher to
monitor students’ responses, enabling them to adjusted lessons or misunderstandings
as needed. Administration did observe a substantial increase in all students being held
accountable for answering all posed questions with a partner and a decrease in
72
volunteer questioning, calling on students who raise their hands. This was especially
beneficial for the EL learners as it provided an opportunity for these students to
participate in oral language practice with scaffolding and it provided an opportunity
for teachers to model complex sentence structure. “Scaffolding is essentially a way to
nudge a student toward a higher level of performance. With language development,
this can be done by modeling correct grammar or pronunciation, asking challenging
questions or providing direct instruction” (Hill & Flynn, 2006, p. 16).
The second research-based strategy that was agreed upon was the use of
graphic organizers. Marzano (2001) provides strong research results, a 9% to 29%
gain in knowledge, as graphic organizers allow students to connect new information to
prior knowledge (p. 117). His research shows that graphic organizers combine a
linguistic and nonlinquistic mode of representing information (Marzano, 2001, p. 75).
All grade level teams agreed on the use of certain graphic organizers for given lessons.
The third and final category of research questions regarding implementation of
an accountability system follows:
Category 3: Accountability
8. How does your grade level use data when reformulating your EL student
groups?
9. How have you used the English Learner Matrix to improve instruction
for the EL student during core instruction?
10. How has the use of the EL Matrix helped EL students develop language?
11. What data do you use to determine there has been growth or not?
73
The accountability system has two major components. The first is teacher
support and planning and the other is the use of data. One of the biggest obstacles was
to build confidence in the teaching staff. Academic conferences helped to support
teachers in developing their understanding of how to use data to guide their
instruction. During the academic conferences administration and the teachers would
discuss individual students and design plans about how to structure lessons so that
students’ needs were considered. The ELD Matrix would be referenced and imbedded
into the ELD and core curricular lessons. Through this process, teachers became better
aware of what their expectation should be for each of the language levels. Overall, the
majority of teachers underestimated what they should be expecting from their EL
students.
Purposeful planning of the checking for understanding revealed just how often
students did not have a clear understanding of the concept being taught. Teachers
shared that they “taught” the concept and thought that students were ready for
independent practice, after conducting the checking for understanding strategies,
teachers soon realized that more modeling or guided practice was needed. Teachers
reported that the more they experienced this situation the more they used the checking
for understanding strategies.
Implementing assessments regularly throughout the year has also had quite an
impact on lesson design. Teachers shared that the assessments allowed them to
construct small group instruction that was much focused, yet remained flexible
because they could monitor student progress. Even though at first teachers though it
74
was too much assessing, they soon realized how the assessments made their planning
and grouping much easier.
An area that created a weakness for implementing the restructure this year was
the fact that the district had adopted a new ELD curriculum so this was the first year of
implementation. We have already had some discussion about particular units and how
to pace the instruction better next year. Overall, staff members had positive response
to the program and want to continue the structure next year.
Staff members all agreed that the research best practice strategies and the
assessments were successful and plan to continue to implement them. The intent is to
perfect the chosen strategies before adding more. As for the assessments, staff has
come to an agreement on which curricular assessments will be used to support the
program. Administration will support staff by paying for an assessment team to come
in and conduct the district assessments as it was concluded this would be the least
amount of instructional disruption
Summary of Findings: Mitchum Elementary School
and Moonstruck Elementary School
In reviewing the findings of the study, a comparison of the control school,
Moonstruck with the experimental school, Mitchum Elementary was completed. A
nonequivalent control group design was used to compare post-test data between the
control group, Moonstruck Elementary and the experimental group, Mitchum
Elementary. Moonstruck Elementary was chosen as the control group based on data
retrieved from the 2006-2007 Similar Schools Report. The intervention of
75
implementing Target Time, research-based strategies and an accountability system
was implemented in the experimental group, Mitchum Elementary.
Mitchum Elementary showed a positive change of +11 points school-wide, 736
to 747, as deemed by the state accountability system, Academic Performance Index
(API). The control school, Moonstruck Elementary, showed a negative change of -8
points school-wide, 735 to 727. API scores range from 200 to 1000 and the
expectation for every school is a minimum of 800 or at least a 5% increase. Mitchum
Elementary also demonstrated substantial positive growth within each of the sub-
groups; Hispanic/Latino (H/L) increased by +19 points (678 to 697), Socioeco-
nomically Disadvantage (SD) increased by +9 points (674 to 683) and English
Learners (EL) increased by +21 points (668 to 689). Whereas, the control school,
Moonstruck Elementary, only showed a positive change in one subgroup, H/L
increased by +1 point (666 to 667) and negative change in the remaining two sub-
groups, SD decreased by -7 points and EL decreased by -21 points.
In comparing academic growth between the experimental school, Mitchum
Elementary, and the control school, Moonstruck Elementary, the strong positive
academic growth for Mitchum Elementary supports that the restructuring of the
language arts intervention program had positive academic effects on the student
population school-wide and within all subgroups.
Implications
Mitchum Elementary made a transformation during the 2006-2007 school year.
Statistically, not all the changes could be measured, but as a community of caring
76
individuals, the school grew tremendously professionally. Administration was able to
personally connect with every teacher every month. This forged a bond that built trust
and provided an avenue that allowed teachers to take chances without the fear of
retaliation. Each grade level revealed a personality that in itself provided admini-
stration with information that allowed changes to be crafted so that implementation
was smoother. Even though statistically the changes were small and in the 3rd grade
negative, much information was gathered that set a foundation for much growth in
years to come. After all, the growth that was experienced was enough to freeze
Mitchum Elementary in Program Improvement and place them on the road to exiting
Program Improvement.
Considering the results school-wide, the overall statistical findings were not
significant (p. < 15) as the difference of the means was only 5%. However, when
considering the NCLB goal is to move students into the level of proficient or above
then the results paint a different practical image as school-wide there was an increase
of 14% more students scoring in the proficient or above levels. The overall results
even get more interesting when you look at the data by grade level and by subgroups.
Second grade did not make enough of a difference to count statistically as their
mean difference was only 13%, but a practical difference was realized as they
increased the number of students scoring proficient or higher by 18%. This team was
very fragmented as half the team (3) was very comfortable teaching in isolation, while
the other half demonstrated a need to plan and problem solve together. By year end,
the team had pulled together and were observed planning and problem solving
77
together. Due to academic conferencing and walk-through observations, admini-
stration was well aware of each teacher’s areas of strength and weakness. It was
exciting to witness professionals as the trust was built within the grade level team as
they began to open up and addressed individual weaknesses. They have a ways to go
before they are completely open with each other, but they are one step closer due to
implementing the new intervention program and collaborating on which strategies best
fit their grade level.
Third grade results are of most concern as their data did make enough of a
difference to count statistically, but is was negative. Their mean difference was -19%
and their practical difference were also significant negatively as students scoring
proficient or higher dropped by -28%. This team has an interesting history. Two of the
members teach from the perspective that students need to emotionally connect to
learning. This in itself would not be bad, but it prevents them from moving through the
standards, as they believe they cannot move on until every child “gets it.” The rest of
the team knew it was an issue, but until this year, had the belief it did not affect them,
so why confront them with a possibility of causing stress on their relationships?
Administration was able to begin conversation about changes that need to take
place in instruction by posing questions about the data during the individual academic
conferences. Statements like, “It’s not fair to move on if some students don’t
understand” were countered by administration with, “Is it fair to never expose your
students to a standard yet assess them on it?” It is a slow process, but both teachers
have shown signs of realizing they have to make changes in their instruction.
78
Fourth and 5
th
made the most improvement. They both made statistical gains.
Fourth grade increased their mean by 27% and 5th grade by 24%. Both grade levels
made great practical growth and experienced a change of .21 of the students moving
into the levels of proficient or higher. Fifth grade moved .33 of their students. Fourth
grade growth was the biggest shock for administration as academic conferencing and
walk-through observations provided information that questioned the depth in which
standards were being addressed. This is a new team and the personality is one of
casual and free. Interim assessments did not support the growth presented in the CST
results. Nonetheless all were pleasantly surprised. Fifth grade also provided a pleasant
surprise as their methodical determination paid off. This team has spent much time
collaborating and analyzing their instruction. They are quite quick to share areas that
need improvement. It was nice to see the results that forced them to celebrate hard
work paying off.
Sixth grade results were very interesting and they made enough of a difference
to count statistically, but it was a negative difference of -21%. In contrast, as per
NCLB we moved students right were they needed to go as the students scoring in
proficient or higher increased by 13%. This team is made up of four classrooms, but
five teachers as one classroom houses a teaching team. Two of the members are quick
to understand research and apply changes immediately, usually beyond expectation as
they craft the implementation to meet their students’ needs. The third individual
member is very open, but has to work at understanding how the change can play out in
her instruction. The teaching team has interesting dynamics as one of the teachers has
79
extremely high expectations and the other has a very laissez-faire approach. All the
members work well together and support each other in many different ways.
When viewing the overall results, Mitchum Elementary School only
experienced statistically positive academic achievement changes in 4th and 5th grade.
However, practically significance, increasing the number of students who moved into
the proficient or higher levels on the California Standards language arts test (> .10),
was experienced in all but 3rd grade. An even bigger reward that should result in a
bigger pay off in the future, is the fact that each grade level team is well on their way
to establishing strong trust in each other, and have developed a sense of ownership of
all students in their grade level not just the 20 or 32 assigned to their classroom.
Collaboration is strong and positively accepted, making each teacher want to perfect
their instructional approach so that they offer the best instruction possible.
Site-based Recommendations
Although this study is limited, implementing the restructuring of the language
arts intervention that included the use of research-based strategies and an account-
ability system did result in two grades making significant statistical (p < .15) change
and all, but one grade level making significant practical change. The restructuring also
greatly supported professional growth as the qualitative results showed a great
increase in collaborative planning and problem-solving, as well as commitment to a
greater cause, all students being afforded improved instruction instead of isolated
incidence.
80
This study offers enough support, especially in the qualitative results, to
recommend that the intervention continue. Strengthening the capacity of the staff will
result in greater student academic improvement. Eaker, DuFour and DuFour (2002)
state it best, “The consensus of leading researchers from within and outside of
education who agreed that the characteristics of a professional learning community are
essential to sustained improvement of any organization” (p. 1). It was incredible to
watch teachers open up, analyze and take responsibility for their contributions to
student improve-ment. This year brought to the surface the answer that all professional
learning communities (PLC) must answer in order to function as a PLC. That is as a
site, it was discovered that each individual must reflect on how their very actions play
a roll in the success of all students and that being a strong team, far out weighs the
small successes of individual growth (Eaker & DuFore, 1998).
Reflecting back on the traits of a learner-centered principal (Alexander &
Murphy, 1999), it was essential to determine what the cause of the performance gap
was at Mitchum Elementary. Considering Clark and Estes (2002), “The Big Three”
causes of performance gaps, all were addressed within the restructuring of the
language arts intervention: (a) knowledge and skills of the staff, (b) motivation to
achieve the goal, and (c) organization barriers. Building collaboration, trust and
bringing in proven research-based strategies addressed the knowledge and skills of
staff. Motivation was addressed by developing a process that valued and allowed staff
to build upon their on strengths as they controlled the program design and supported
each other throughout the process. The biggest accomplishment was that of
81
diminishing the perceived barrier of the impossibility to serve all language levels in a
classroom. Looking at all students within a grade level and utilizing all teachers to
group the students by language needs, addressed the barrier.
A monitoring process of EL progress in ELD driven by the Avenues
curriculum needs to be developed as Mitchum Elementary goes into the second year of
implementing the Avenues curriculum. Qualitative data supports the need to embed
more explicit instruction in writing. The use of writing can also be used to inform
teachers of students’ clear understanding or misunderstanding of concepts. As teachers
enter into the second year of implementation, they are better familiar with the program
components and connecting the curricular components with the language development
levels can build a strong ELD program that is very student focused.
Mitchum Elementary is well on its way to becoming a strong professional
learning community. Next year will allow for the fine-tuning that needs to develop in
order to realize a stronger showing in the statistical results. A strong analysis needs to
occur in 3rd grade to determine how to best improve the instructional program and
teaching strategies so to support student growth and teacher professional growth. A
review of explicit direct instruction (EDI) is needed to revitalize the use of the lesson
format as with all the changes that occurred, the EDI format was not as consistently
used as in years past. A continued focus on higher order questioning is part of the EDI
format as embedded in the EDI format is the use of Bloom’s taxonomy questioning.
The final site-based recommendation that was considered, but only implemented in a
few classrooms, was that of student involvement in goal setting and monitoring of
82
their own leaning. Marzano (2001) established that successful people have mastered
the skill of establishing a direction for learning (p. 93). Students need the exposure of
how to establish direction and that begins in elementary school.
Strengthening the teachers recently found direction and providing an
opportunity for students to develop their own direction will move the school toward
the goal of all students presiding in the proficient or higher levels on the CST language
arts assessment. Further development in implementing the EL matrix and monitoring
EL learners language acquisition will provide a strong foundation to which EL
learners will reach English proficiency. A strong focus and continued commitment
will result in moving Mitchum Elementary out of Program Improvement status.
The present study focused on an evaluation of the impact that the restructuring
of the language arts intervention had on the academic achievement of EL learners.
Although, growth was experienced in the mean difference of EL learners, it was not
statistically significant. The practical significance in this group was weak. Further
work needs to be developed in the area of assessing language acquisition throughout
the year so that adjustments can occur within the school year. The current measure of
the California English Language Development Test (CELDT) that is administered
annually, only measures the level at which students are progressing in the ability to
listen, speak, read and write English. The CELDT does not measure academic content
area.
Given that the teacher is the school’s most important factor in students’
academic achievement, the impact of teacher efficacy is a dynamic that must be
83
studied and monitored (Marzano, 2003). The measurement of teacher efficacy and
competence is directly interrelated to the aspect of student performance. A strong
focus on professional training for teachers and the consistent implementation of
research-based strategies must be provided and monitored. Administration will
increase walk-through observation and provide feedback on implementation of
strategies to support accountability.
Limitations
Caution is exercised in the generalization of this study’s results. An internal
limitation was inherent in the use of a pre-post nonequivalent control group design.
Given the results, although positive overall, there are many extraneous factors that
were not addressed that may have influenced the outcome of the effects. One factor is
that of sampling variability, as it can hinder the true measure of the amount of change
exhibited by each grade level. Student mobility plays a significant confounding roll in
nonequivalent control group design.
Another limitation is that of disaggregated reporting from the state and federal
reporting agencies. California’s accountability system measures the performance and
progress of a school on results of statewide tests, CST. These tests are aligned to state-
adopted content standards. The content standards describe the knowledge and skills
that students should master at each grade level. A school API is a composite number
representing the results of different state tests. The test results used in calculating a
school’s API have different relative emphases. The amount of emphasis each content
area has in the API for a particular school (called the school content area weights), are
84
determined by statewide weighting (called test weights) and by the number of students
taking a particular test at a school. Obtaining disaggregated information allows
schools to group student who show the same areas of need within the most heavily
weighted areas reported.
As for the federal reporting, AYP, the accountability system requires that each
year schools must meet four sets of requirements to make their AYP. The require-
ments reflect statewide performance levels and are the same for all schools. The
requirements include: (a) student participation rate on statewide tests, (b) percentage
of students scoring at the proficient level or above in English language arts and
mathematics on statewide tests, (c) Growth API, and (d) graduation rate (if high
school students are enrolled). Numerically, significant subgroups at a school also must
meet participation rate and percent proficient requirements. A serious limitation in this
researcher’s point of view was that of the lack of CST EL learner data currently
available from the California Department of Education (CDE), as this provided an
external limitation to the study. The CDE does not provide disaggregated information
with regard to the percent of students who reside in the “far below,” “below basic,”
and “basic” performance bands for EL learners, as the reports only show students
scoring proficient or above. It does report all performance bands school-wide, but not
for sub-groups. There is a strong need to know if interventions are moving students
from the lower three proficient bands as well. The lack of such information prevented
the researcher from determining if the intervention supported growth within the lowest
performance bands.
85
New curriculum also played a role in limiting possible outcomes. This was the
first year the Avenues curriculum was used at Mitchum Elementary. Teacher’s
unfamiliarity to the program hindered fluid use and implementation of the curriculum.
Although the Avenues provided a better approach than the previous curriculum, the
results of growth in language acquisition cannot be directly determined in only one
year of implementation. Avenues was reported as being weak in the area of
developing writing.
The implementation of the study itself played a role in limiting possible results
as implementing a process for the first time brings uncertainty and creates a distraction
from lesson development. However, if the positive results experienced in this study
stand true, then in years to come the implementation of the intervention, will far out
weigh the single year of implementation.
Conclusion
Implementing the restructuring an intervention program was an enormous task,
but the impact had positive results, some academic and some professionally. Although
the results did not reflect enough change in the statistical or practical results of the EL
learners, this researcher would propose that the study had a tremendous impact on the
professional community that will extend to the academic success of EL learners in the
near future. One year is not enough time to truly see the benefits in the students’
results, but qualitative results showed a strong swing towards developing staff’s
understanding of student learning and their responsibility of providing an environment
86
that allows for all students to succeed. There has been a revitalization of confidence in
serving EL learners and a renewed sense of commitment to their success.
87
REFERENCES
Alexander, P. A., & Murphy, K. P. (1999). The research for APAs learner-centered
psychological principles. In N. M. Lambert and B. L. McCombs (Eds.) How
students learn: Reforming schools through learner centered education (p. 26)
Alexandria, VA: Association for Supervision and Curriculum Development
Bandura, A. (1994). Self-efficacy. In V. S. Ramachaudran (Ed.), Encyclopedia of
human behavior (Vol. 4, pp. 71-81). New York: Academic Press. (Reprinted in
H. Friedman [ed.], Encyclopedia of mental health. San Diego, CA: Academic
Press.
Bandura, A. (1997). Self-efficacy: The exercise of control. New York: W.H. Freeman
California Department of Education (CDE). (2005). 2005 Accountability Progress
Report. Retrieved December 4, 2007, from
http://www.cde.ca.gov/ta/ac/ay/documents
California Department of Education (CDE). (2005-2006). California English language
development test annual assessment–all students. Retrieved December 4, 2007,
from http://data1/cde/ca/gov/dataquest/CELDT/CELDT03
California Department of Education (CDE). (2006-2007). California English language
development test annual assessment–all students. Retrieved December 4, 2007,
from http://data1.cde.ca.gov/dataquest/CELDT/Celdt03
Clark, R. E., & Estes, F. (2002) Turning research into results: A guide to selecting the
right performance solutions. Atlanta, GA: CEP Press Committee on Education
and Labor (March 2007). Early Childhood, Elementary and Secondary
Education Subcommittee Hearing: Impact of NCLB on English Learners.
Prepared remarks of U.S. Rep. Dale Kildee (D-MI) chairman of the House
Subcommittee. Retrieved October 30, 2007 from www.edworkforce.house.
gov/hearingsstatements
Darling-Hammond, L. (2002). The right to learn (pp. 148-176). San Francisco, CA:
Jossey-Bass.
Dembo, M. H. (2004). Understanding learning and memory in motivation and
learning strategies for college success: A self management approach. (2
nd
ed.,
pp. 29-50). Mahwah, NJ: Lawrence Erlbaum Associates.
88
DuFour, R., & Eaker, R. (1998). Professional learning communities at work: Best
practices for enhancing student achievement. Alexandria, VA: Association for
Supervision and Curriculum Development.
Eaker, R., DuFour, R., & DuFour, R. (2002). Getting started: Reculturing a school to
become professional learning communities. Bloomington, ID: Solution Tree.
Elmore, R. (2002). Bridging the gap between standards and achievement.
Washington, DC: Albert Shanker Institute.
Elmore, R. (2003). Knowing the right thing to do: School Improvement and
performance-based accountability. Washington, DC: National Governors
Association Center for Best Practices.
Johnson, R. S. (2002). Using data to close the achievement gap: How to measure
equity in our schools. Thousand Oaks, CA: Corwin Press
Krashen, S.D., & Terrell, T. (1983). The natural approach: Language acquisition in
the classroom. Oxford, NY: Pergamon.
Marzano, R. J. (2001). Classroom instruction that works: Research-based strategies
for increasing student achievement. Alexandria, VA: Association for
Supervision and Curriculum Development.
Marzano, R. J. (2005). School leadership that works. From research to results.
Alexandria, VA: Association for Supervision and Curriculum Development.
Marzano, R. J. (2003). What works in schools: Translating research into action.
Alexandria, VA: Association for Supervision and Curriculum Development.
Merrill, M. D. (2002). First principles of instruction. Instructional Technology
Research and Development, 50(3), 43-59.
National Education Association. (2006, March). Meeting English language learners’
needs under no child left behind. Trends: Issues in Urban Education, 21, 1.
Northouse, P. G. (2004). Leadership theory and practice. Thousand Oaks, CA: Sage.
Pintrich, P. R. (2003) A motivation science perspective on the role of student
motivation in learning and teaching contexts. Journal of Educational
Psychology, 94(4), 667-686.
89
Reeves, D. (2004). Making standards work: How to implement standards-based
assessments in the classroom, school, and district. Englewood, CO:Advanced
Learning Press.
Resnick, L., & Hall, M. (2003). Principles of learning for effort-based education.
Institute for Learning (IFL). Pittsburgh, PA: University of Pittsburgh
Schmoker, Mike (2006). Results now: How we can achieve unprecedented
improvements in teaching and learning. Alexandria, VA: Association for
Supervision and Curriculum Development.
Stronge, J. H. (Ed.). (2006). Evaluating teaching: A guide to current thinking and best
practices. Thousand Oaks, CA: Corwin.
Togneri, W., & Anderson, S. E. (2003). Beyond islands of excellence: What districts
can do to improve instruction and achievement in all schools. Washington,
DC: The Learning First Alliance and the Association for Supervision and
Curriculum Development.
90
APPENDIX
ELD Matrix Grammatical Forms
91
92
Abstract (if available)
Abstract
Trends in the performance of students attending Mitchum Elementary School in 2005-2006 school year compared to the performance of students attending in the 2006-2007 school year were examined. The study has a particular focus on the achievement of English language learners, the mean annual school-wide Academic Performance Indices and the percentage of proficient and above using the Annual Yearly Progress indicator, as well as the EL learner s performance on the California English Language Development Test.
Linked assets
University of Southern California Dissertations and Theses
Conceptually similar
PDF
English learners' performance on the California Standards Test at Aviles Elementary
PDF
The effect of a language arts intervention on the academic achievement of English language learners
PDF
An evaluation of the impact of direct instruction intervention on the academic achievement of English language learners
PDF
The implementation of strategies to minimize the achievement gap for African-American, Latino, English learners, and socio-economically disadvantaged students
PDF
An evaluation of the impact of a standards-based intervention on the academic achievement of English language learners
PDF
Raising student achievement at Eberman Elementary School with effective teaching strategies
PDF
Examining the effectiveness of the intervention programs for English learners at MFC intermediate school
PDF
English language learners utilizing the accelerated reader program
PDF
Raising student achievement on the California Standards Test and California High School Exit Exam at the Phoenix Arts Charter School
PDF
An alternative capstone project: Evaluating the academic achievement gap for Latino English language learners in a high achieving school district
PDF
Implementation of dual language immersion to improve academic achievement of Latinx English learners
PDF
Examining the effectiveness of teacher training on the improvement of California standardized test scores at Eva B. Elementary School
PDF
The effects of open enrollment, curriculum alignment, and data-driven instruction on the test performance of English language learners (ELLS) and re-designated fluent English proficient students ...
PDF
The effectiveness of the cycle of inquiry on middle school English-learners in English-language arts
PDF
The impact of professional learning communities and block scheduling on English language learner students at Oak Point Middle School
PDF
A longitudinal investigation of reading in high-stakes tests for adolescent English language learners
PDF
An evaluation of the impact of a standards-based intervention on the academic achievement of algebra students
PDF
Teach them to read: implementing high leverage pedagogical practices to increase English language learner academic achievement
PDF
The block schedule and the English language learners: impact on academic performance and graduation rate at Oxbow High School
PDF
An analysis of the impact of the total educational support system direct-instruction model on the California standards test performance of English language learners at experimental elementary school
Asset Metadata
Creator
Parnell, Denise L.
(author)
Core Title
The impact of restructuring the language arts intervention program and its effect on the academic achievement of English language learners
School
Rossier School of Education
Degree
Doctor of Education
Degree Program
Education (Leadership)
Publication Date
05/07/2008
Defense Date
02/21/2008
Publisher
University of Southern California
(original),
University of Southern California. Libraries
(digital)
Tag
English learners,interventions,OAI-PMH Harvest,school reform
Place Name
California
(states),
educational facilities: Mitchum Elementary School
(geographic subject)
Language
English
Advisor
Hocevar, Dennis (
committee chair
), Hexom, Denise (
committee member
), Stowe, Kathy Huisong (
committee member
)
Creator Email
dparnell@usc.edu
Permanent Link (DOI)
https://doi.org/10.25549/usctheses-m1234
Unique identifier
UC1309937
Identifier
etd-Parnell-20080507 (filename),usctheses-m40 (legacy collection record id),usctheses-c127-69858 (legacy record id),usctheses-m1234 (legacy record id)
Legacy Identifier
etd-Parnell-20080507.pdf
Dmrecord
69858
Document Type
Dissertation
Rights
Parnell, Denise L.
Type
texts
Source
University of Southern California
(contributing entity),
University of Southern California Dissertations and Theses
(collection)
Repository Name
Libraries, University of Southern California
Repository Location
Los Angeles, California
Repository Email
cisadmin@lib.usc.edu
Tags
English learners
interventions
school reform