Close
About
FAQ
Home
Collections
Login
USC Login
Register
0
Selected
Invert selection
Deselect all
Deselect all
Click here to refresh results
Click here to refresh results
USC
/
Digital Library
/
University of Southern California Dissertations and Theses
/
An evaluation of the impact of a standards-based intervention on the academic achievement of algebra students
(USC Thesis Other)
An evaluation of the impact of a standards-based intervention on the academic achievement of algebra students
PDF
Download
Share
Open document
Flip pages
Contact Us
Contact Us
Copy asset link
Request this asset
Transcript (if available)
Content
AN EVALUATION OF THE IMPACT OF A STANDARDS-BASED
INTERVENTION ON THE ACADEMIC ACHIEVMENT OF ALGEBRA
STUDENTS
by
Mimi Rodibaugh-Woods
A Dissertation Presented to the
FACULTY OF THE ROSSIER SCHOOL OF EDUCATION
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulfillment of the
Requirements for the Degree
DOCTOR OF EDUCATION
August 2007
Copyright 2007 Mimi Rodibaugh-Woods
ii
TABLE OF CONTENTS
List of Tables iii
List of Figures v
Abstract vi
Chapter 1: The Problem 1
Problem Description 1
Problem Analysis 5
Problem Solution 12
Chapter 2: Literature Review 18
Chapter 3: Methodology 32
Summative Evaluation Design 32
Formative Evaluation Design 35
Intervention 37
Chapter 4: Findings 49
Chapter 5: Summary, Discussion, and Recommendations 79
Summary of Findings 79
Site-Based Recommendations 89
Recommendations for Further Study 92
Limitations 93
Bibliography 95
iii
LIST OF TABLES
Table 1: 2003 Algebra 1 Content Standards Test Scores for Huntington 4
Beach High School
Table 2: 2004 Algebra 1 Content Standards Test Scores for Huntington 4
Beach High School
Table 3: 2005 Algebra 1 Content Standards Test Scores for Huntington 4
Beach High School
Table 4: Selection Criteria for Experimental and Control Groups 33
Table 5: Curriculum Sequence for Algebra 1 at Huntington Beach High 40
School
Table 6: Mean Statistics by Class Level 50
Table 7: Effect Size Estimates 52
Table 8: Percentage of Change of Mean Score by Class Level 53
Table 9: Mean Statistics by Class Level: Proficient and Above 54
Table 10: Effect Size Estimates for Proficiency 56
Table 11: Percentage of Change by Class Level: Proficient and Above 56
Table 12: Mean Statistics by Class Level: Basic and Above 57
Table 13: Effect Size Estimates for Basic and Above 60
Table 14: Percentage of Change by Class Level: Basic and Above 60
Table 15: Tests of Between-Subjects Effects 61
Dependent Variable: Score
Table 16: Tests of Between-Subjects Effects 62
Dependent Variable: Proficiency
Table 17: Tests of Between-Subjects Effects 63
Dependent Variable: Basic
Table 18: Mean Statistics by Class Level: Control School Findings 64
iv
Table 19: Effect Size Estimates: Control School Findings 66
Table 20: Percentage of Change of Mean Score by Class Level: Control 66
School Findings
Table 21: Control School Mean Statistics by Class Level: Proficient and 67
Above
Table 22: Effect Size Estimates for Proficiency: Control School Findings 70
Table 23: Control School Percentage of Change by Class Level: 70
Proficient and Above
Table 24: Control School Mean Statistics by Class Level: Basic and 71
Above
Table 25: Effect Size Estimates for Basic and Above: Control School 73
Findings
Table 26: Percentage of Change by Class Level: Basic and Above 74
Table 27: Tests of Between-Subjects Effects: Control School Findings 75
Dependent Variable: Score
Table 28: Tests of Between-Subjects Effects: Control School Findings 76
Dependent Variable: Proficiency
Table 29: Tests of Between-Subjects Effects: Control School Findings 77
Dependent Variable: Basic
Table 30: Experimental versus Control: Change from 2005 to 2006 78
v
LIST OF FIGURES
Figure 1: Estimated Marginal Means by Class Level 51
Figure 2: Estimated Marginal Means by Class Level: Proficient and 55
Above
Figure 3: Estimated Marginal Means by Class Level: Basic and Above 59
Figure 4: Estimated Marginal Means by Class Level: Control School 65
Findings
Figure 5: Control School Estimated Marginal Means by Class Level: 69
Proficient and Above
Figure 6: Control School Estimated Marginal Means by Class Level: 72
Basic and Above
vi
ABSTRACT
The purpose of this study is to implement and evaluate a standards-based
intervention to improve math proficiency in Algebra 1 at Harrison Park High School
(name has been changed). To raise mathematics achievement at the experimental
school, the standards-based intervention was implemented for the school year 2005-
2006. The study was designed as pre-post nonequivalent control group design, with
the intervention taking place in several phases, and as post only quasi experiment
with a nearby school, Ericson High School (name has been changed), serving as a
control. The study was designed as a combination of both summative and formative
evaluation techniques. By analyzing the test data, the strengths and weaknesses of
the Algebra program were analyzed in an attempt to improve the current program in
subsequent years. Analyzing student achievement on the Algebra 1 Content
Standards Test (CST) directed further improvements in school curriculum and
classroom instructional delivery, leading to an alignment between state expectations
and academic achievement.
This study has revealed the impact of a standards-based intervention on the
academic achievement of Algebra students. Given the results of the findings, the
overall impact of the intervention was positive, and there was a clear correlation
between the class level of Algebra the student was enrolled in and academic
achievement as measured by the Algebra 1 Content Standards Tests (CST). This
study presents a standards-based intervention as a resource to assist in moving
Algebra students closer to proficiency.
1
CHAPTER ONE
THE PROBLEM
Problem Description
Recently there has been a tremendous push at both the state and federal levels
to reduce the achievement gap, and increase the academic performance of all
students. The passage of Public Law 107-110, commonly known as the No Child
Left Behind Act of 2001, clearly illustrates this. The purpose of this act was to
reform the public education system by more clearly identifying the standards to
which students should be held, and making schools and teachers more accountable
for the attainment of these standards by the students. As part of this initiative, one of
the primary goals was “…ensuring that high-quality academic assessments,
accountability systems, teacher preparation and training, curriculum, and
instructional materials are aligned with challenging state academic standards (Public
Law 107-110, p. 1439).” In the state of California, these state standards are
measured by the California Standards Tests (CSTs). Through this alignment of
teaching and learning with common state standards and assessments, the goal was to
facilitate “…improving and strengthening accountability, teaching, and learning
(Public Law 107-110, p. 1440).” Common state standards and expectations were
emphasized in mathematics, with the expectation that all students would have the
opportunity to be proficient in Algebra 1 by the time they graduated high school.
At Harrison Park High School (name has been changed), performance in
Algebra 1 as measured by the California Standards tests from 2003-2005 was far
2
below that which is desired. Harrison Park High School is located in Huntington
Beach, California, a large city with a population that is economically diverse,
although primarily considered upper-middle class. The school served 2,658 students
in the school year 2005-2006, and this population is predicted to continue to grow
over time as new housing continues to be built and the city’s population continues to
grow. Average class size is 31 students. School demographic data show that in the
school year 2005-2006, the population was primarily White (67% of total
enrollment), followed by a significant number of Hispanic and Asian (14% and 10%
of total enrollment, respectively). Parent education level is high, with most parents
graduating college. According to data from the school year 2005-2006, of the 10%
of students that responded, 20% reported their parents completed some college, 40%
had parents that graduated college, and 20% had parents that completed some
graduate school. For this reason, school and education are very important to the
parents and the community at large, and there is tremendous focus on academic
achievement as it is measured by state test scores, specifically the California
Standards Tests (CSTs).
The passage of the No Child Left Behind Act of 2001 has put further pressure
on the school to improve test scores, giving strict requirements for teachers to be
fully credentialed in the subject they teach to facilitate academic achievement for all
students. According to demographic data from the school year 2005-2006, the
average number of years of service for a teacher at Harrison Park High School is
12.9 years, and 52.9% of teachers have earned a masters degree or above. Since the
3
CSTs are written solely according the California State Standards (Edsource, 2005),
there has been a tremendous push within the school over the past few years to define
those standards for each subject, and then align curriculum accordingly.
In the year 2003, the first year for which CST data is available, it became
clear that there was much work to be done in the field of mathematics, and more
specifically Algebra 1, to improve math proficiency as measured by state test scores.
Of the 793 students tested that year, 37% scored Below Basic and 20% scored Far
Below Basic. This was of great concern, since over half of the students enrolled in
some form of Algebra 1 were scoring Below Basic. In 2004 there was little
improvement. Out of 911 students tested that year, 43% scored Below Basic and
15% scored Far Below Basic. Again little improvement was made the next year.
Out of 1026 students tested, 33% scored Below Basic and 16% scored Far Below
Basic. The data from these three years is summarized in the Tables 1-3 below.
Clearly something wasn’t working and an intervention was needed to improve math
proficiency in Algebra 1. As math department coordinator, I was in a position to
facilitate change, and began analyzing the problem in hopes of designing an
intervention that would result in improved math proficiency as measured by the
CSTs. In order to do so, I secured the support of the high school principal, who
could help with implementation if necessary.
4
Table 1:
2003 Algebra 1 Content Standards Test Scores for Harrison Park High School
9
th
Grade 10th Grade 11
th
Grade All Students
Students
Tested
401 251 141 793
%Advanced 3% 1% 1% 2%
%Proficient 23% 13% 7% 17%
%Basic 24% 26% 21% 24%
% Below
Basic
34% 36% 46% 37%
%F.B. Basic 16% 24% 26% 20%
Table 2:
2004 Algebra 1 Content Standards Test Scores for Harrison Park High School
9
th
Grade 10
th
Grade 11
th
Grade All Students
Students
Tested
469 283 159 911
%Advanced 2% 0% 0% 1%
%Proficient 22% 9% 10% 16%
%Basic 22% 29% 26% 25%
% Below
Basic
40% 46% 45% 43%
% Far Below
Basic
14% 16% 19% 15%
Table 3:
2005 Algebra 1 Content Standards Test Scores for Harrison Park High School
9th Grade 10
th
Grade 11
th
Grade All Students
Students
Tested
555 300 171 1026
%Advanced 3% 0% 0% 1%
%Proficient 29% 12% 8% 20%
%Basic 32% 32% 19% 30%
% Below
Basic
25% 40% 45% 33%
% Far Below
Basic
12% 16% 28% 16%
The purpose of this study is to design and implement a standards-based
intervention to improve math proficiency in Algebra 1. The delivery of the Algebra
1 content standards will be analyzed, insuring that the intervention leads to an
5
aligning of the curriculum with the content standards that are being tested at the state
level. This will involve close collaboration with the Algebra teachers, as well as
staff development in the implementation stages of the intervention. The study will
seek to answer the question; will a standards-based intervention have an impact on
math proficiency in Algebra 1?
Problem Analysis
The problem of low proficiency on the Algebra 1 California Standards Tests
at Harrison Park High School can be attributed to a variety of causes. A useful
framework for analyzing the problem is presented by Robert J. Marzano in his book,
What Works in Schools (2003). Marzano outlines three primary factors that affect
student achievement. First there are school-level factors. At Harrison Park High
School, these would include the types of Algebra courses being offered to students
before they are required to take the Algebra 1 CST and perhaps the school bell-
schedule itself, which is designed as a block schedule. These are school curricular
and structural issues, and in order for students to be successful they must be given
equal and consistent access to the Algebra 1 curriculum. The second factors are
teacher-level factors. In order for students to be successful, teachers must be
teaching the California State Standards that are being tested, and they must do so in
such a way that students are both motivated to learn and developing mastery of each
content standard. The final factors affecting math achievement are student-level
factors. In order for there to be success, students must “buy in” to the state tests in
terms of their importance to them individually, and they must be sufficiently
6
motivated to master what is being taught in the classroom. All three of these factors
will be analyzed in depth to determine the specific breakdown that is occurring
between math instruction at Harrison Park High School and the achievement of math
proficiency as measured by the state tests.
School-Level Factors
There are two school-level factors that are contributing to low academic
achievement in Algebra 1 at Harrison Park High School. The first factor is the types
of courses being offered to students in Algebra. At the time this intervention took
place, there were three algebra 1 sequences that students could follow. At the
highest level, students could enroll in a 1 year Algebra 1 course. This is a traditional
Algebra 1 class that covers the entire Algebra 1 curriculum in a year, and these
students naturally took the Algebra 1 CST at the end of the year. At the second
level, students could enroll in a two year Basic Algebra program, taking Basic
Algebra B the first year and Basic Algebra D the second year. These students would
only complete half of the Algebra 1 curriculum during the first year, but were still
required to take the Algebra 1 CST at the end of that first year. This would
contribute to lower test scores since only half of the standards would have been
introduced. These same students would again take the Algebra 1 CST after the
second year of Basic Algebra. At the lowest level, students can enroll in an
Essentials of Algebra course, taking Essentials of Algebra B the first year and
Essentials of Algebra D the second year. This is a two year algebra course that
meets the state algebra requirement, but moves at a much slower pace for students
7
who have weak math skills, with remediation given along the way. Again, these
students are required to take the Algebra 1 CST at the end of the first year of
instruction, even though only half of the curriculum was completed, and then again
after the second year of instruction. In summary, there were five different class
levels of Algebra that a student could be enrolled in at Harrison Park High School;
Algebra 1, Basic Algebra D, Basic Algebra B, Essentials of Algebra D, and
Essentials of Algebra B. Clearly, the students who are enrolled in the two year
courses are at a disadvantage at the end of the first year of instruction since they have
not been introduced to all the Algebra 1 standards being tested, and therefore score
poorly on the CST test. As Marzano (2003) states, “…if students do not have the
opportunity to learn the content expected of them, there is little chance that they will
(p. 24).” These are school curricular issues, and should be looked at as an ongoing
effort to improve performance on state tests.
The second school-level factor affecting mathematics proficiency at Harrison
Park High School is the bell schedule itself. This high school follows a block
schedule, with courses meeting three times a week for a longer block of time, instead
of the traditional schedule in which classes meet every day. In mathematics, it is
often necessary for students to receive information in small chunks, and then practice
the skills taught on a daily basis. With the block schedule, students receive more
information in one day, and then often go a day without practicing the skills taught.
This mode of instruction most likely affects academic achievement and consequently
performance on the state tests. Each year, a bell schedule committee forms to revisit
8
alternative bell schedules that might work best for students. However, the block
schedule works best for some subjects where it is not a requirement to have daily
practice. When put to a yearly vote, the block schedule has received the most votes.
Teacher-Level Factors
The most important factor affecting math proficiency is the classroom
teacher. The classroom teacher is the individual the students connect with each
week, and who has the greatest influence on student motivation. There are three
teacher-level factors that need to be addressed when analyzing math achievement.
The first thing that will be noticed when studying the Algebra teachers at Harrison
Park High School is a tremendous resistance to change. Half of the teachers who
were teaching Algebra in the year 2005 were veteran teachers who had been teaching
for more than twenty years. On the same note, the Algebra textbooks that were
being used were outdated, written before the emphasis on state standards. The
veteran teachers were extremely comfortable with these textbooks, and taught the
same curriculum that they had been teaching for years, without any thought to the
state standards or the California Standards Tests that would be given in the spring.
They relied primarily on the textbook for classroom instruction and individual
practice. These teachers had a tremendous sense of pride in their teaching, and felt
that what they had been doing for so many years was in fact working so there was no
need to change their instructional delivery. They failed to see a connection between
their methods of classroom instruction and the failure of students to attain
proficiency, instead shifting the blame for the problem on other school or student
9
factors, including the fact that students coming from middle school were poorly
prepared, and that the students at Harrison Park High School had a general relaxed
attitude and no motivation to learn. They also pointed to the fact that students who
had only completed one half of the Algebra 1 curriculum were required to take the
test, and that this was a main reason behind low achievement. Although all of these
are valid factors affecting low achievement, the classroom teacher is still the most
important factor, and any intervention designed to increase student math
achievement must involve the support and cooperation of the classroom teacher.
“…the impact of decisions made by individual teachers is far greater than the impact
of decisions made at the school level (Marzano, 2003, p. 71).”
The second teacher factor affecting student Algebra 1 proficiency is
instructional design and instructional strategies. According to Marzano (2003), the
more instructional strategies a teacher is familiar with, the more effective the teacher
will be (p. 78). Current teachers at Harrison Park High School are not familiar with
the California State Standards, and more importantly with the “essential standards”
that are being tested on the CSTs. These standards need to be taken into account
when designing lessons. This factor correlates strongly with teacher resistance to
changing what has been taught for years. As part of any intervention that is to take
place, teacher in-servicing needs to take place to help teachers design lessons that
revolve around the state standards and rely less heavily on the Algebra textbook.
Furthermore, the lack of incorporation of technology into the curriculum needs to be
addressed. There is an abundance of web-based, computer technology available to
10
classroom teachers to help meet the needs of technologically savvy students and
increase motivation by presenting real-world applications of algebra concepts. Any
intervention should also incorporate standards-based computer technology as part of
the process.
The final teacher factor that is contributing to low math achievement is the
lack of relevant professional development for Algebra teachers. Teachers need to be
continually striving to learn more about the changes that are taking place in
mathematics instruction, and seeking to find new ideas to help meet the needs of an
ever-changing student population. Many of the veteran teachers at Harrison Park
High School tend to close their doors, and focus solely on what is happening in their
own classrooms, without noticing the changes that are taking place in the school
culture and student population. Many new teachers come in with great new ideas
and strategies, and would be more than willing to share them with others. As part of
professional development, teachers should be required to observe other teachers with
different styles than their own to get new ideas and stay fresh in their knowledge.
Furthermore, professional development in math instructional strategies needs to be
an ongoing process, and teachers should attend conferences regularly as a means of
staying informed of the most current state policy changes, and obtaining new
instructional ideas for their students.
Student-Level Factors
There are several student-level factors that need to be analyzed with regard to
their effects on student achievement in mathematics. First, students need to “buy in”
11
to the importance of the state tests to them personally. High school students are
required to take at tremendous number of tests throughout their high school career,
including not only tests given by their teachers in the classroom, but also SAT and
ACT tests, Advanced Placement (AP) tests, the California High School Exit
Examination (CAHSEE), and the California Content Standards (CST) tests.
Students can clearly see the importance of the first three tests. SAT and ACT tests
are directly linked to college admissions, Advanced Placement tests will give them
college credit, and they must pass the CAHSEE test in order to receive a high school
diploma. However, students do not see the relevance of the CST tests. They do not
count as part of their grade in any way, and they are not directly linked to college
admissions or graduation requirements. For this reason, an effort needs to be made
to illustrate to the students the importance of these tests for them specifically. There
needs to be a link in their minds between a school’s API score as directly related to
the perception that colleges have of the academic integrity of the school. If students
can buy into the fact that their individual performance on the test will help the
perception that they attend a “better school,” the test will hold more relevance to
them.
The second student-level factor affecting mathematics achievement is student
motivation. Motivation tends to be a challenge, specifically in the lower level
Algebra courses. Students in these classes have traditionally been unsuccessful in
mathematics classes, and therefore have very low self-efficacy when it comes to
learning math. For this reason, they tend to give up and lose motivation as they
12
continue to be unsuccessful. Marzano (2003) states that continuous and immediate
feedback throughout the learning process can aid in allowing students to self-assess
before any final assessments are given (p. 37), thus increasing motivation to continue
with the learning process. An effort must be made to increase these students’
success, and therefore increase motivation to learn more.
The final student-level factor affecting academic achievement is the level of
parental involvement a student has on his or her learning. Students who do not have
a home life that encourages or supports learning will naturally have lower academic
performance than those that do. An effort should be made to identify and target
students who have a low level of parental involvement, and design ways to support
these students in their learning. According to Marzano (2003), school-based
interventions can go a long way in compensating for a negative home life (p. 123).
Problem Solution
To raise mathematics achievement at Harrison Park High School, a
standards-based intervention was implemented for the school year 2005-2006. As
mathematics department coordinator, I was in a position to lead the implementation
and ensure that all teachers of Algebra were included as participants of the study.
There were eight teacher participants in the intervention. My role as the researcher
was to make sure that all the participants were well informed of the phases of the
intervention, and that they were consistently implementing the intervention across
the Algebra classrooms. I secured the support of the administrator who oversees the
math department at Harrison Park High School to help in maintaining consistency in
13
implementation. Participants were advised of the importance of the intervention in
helping to determine the strength of the school’s Algebra program, and that the
outcome of the study could lead to improvements in this program.
The study was designed as pre-post nonequivalent control group
design, with the intervention taking place during the school year 2005-2006 in
several phases, and as post only quasi experiment with a nearby school, Ericson High
School (name has been changed), serving as a control. The study was designed as a
combination of both summative and formative evaluation techniques. The
summative evaluation involved quantitative methods, using the CST test score data
as the means of answering the research question, “Does a standards-based
intervention in Algebra have an impact in mathematics proficiency as measured by
the California Standards Tests?” The formative evaluation involved qualitative
methods to seek ways to improve the intervention.
For the first phase of the intervention, all Algebra teachers were sent to
several conferences focused on creating a model high school. During the
conference, the teachers were given a list of Algebra 1 California State Standards
and the likelihood that each standard would be tested on the California Standards
Test as determined by research of past tests. They were then given time to work
collaboratively on taking the long list of standards and reducing it to a list of
“essential” standards that must be covered in order for students to achieve on the
state test. This allowed teachers to become familiar with the standards, and start
assessing the level to which they had been teaching to the standards in the past.
14
They were now aware of exactly what their students were expected to know by the
end of the course.
For the next phase of the intervention, Algebra teachers were paid during the
summer to spend time aligning the Algebra 1 curriculum at Harrison Park High
School with the essential standards that they had decided upon. They created a
curriculum map for the entire year for each of the five class levels, specifying the
quarter in which each concept would be taught and the standard with which that
concept was aligned. This moved the teachers away from the textbook as the driving
force in their teaching. Although the textbook would still be used, Algebra would
now be taught with the essential standards determining the order in which concepts
would be taught, which concepts were most important, and which concepts could be
left out if time ran out. This curriculum map also ensured that every Algebra
classroom was consistent in terms of the standards being taught, which would
guarantee that all students are receiving equal access to the curriculum. In addition
to teaching a common curriculum, the Algebra teachers for each class level created
common assessments throughout the school year for each chapter and for the final
exam. This consistency again ensures that all students no matter whom their teacher,
are being taught a common standards-based curriculum.
As the school year began, another phase of the intervention began in which
teachers were given a limited amount of release time to go and observe other
Algebra teachers who have been teaching to the standards for several years. The
teachers would then meet with these mentors and discuss strategies that they could
15
use to help increase student awareness of the standards. In addition, all Algebra
teachers posted the California Standard that they were teaching that day on the
chalkboard. They emphasized to the students the standard of the day, and referred
back to that standard throughout the day’s lessons. The Algebra teachers also agreed
that they would do a daily “warm up,” where students were given several released
questions from previous CST tests to work on and discuss together. This gave the
students familiarity with the types of questions they would most likely see again on
that year’s Algebra 1 CST test. This way, the students knew exactly what they
needed to know, and how that knowledge would be assessed when it came time for
state testing.
One final important aspect of the Standards-Based intervention in Algebra
was the use of technology in instructional design. HPHS purchased a web-based
computer program called Riverdeep which was recently written to align directly with
California State Standards. The program includes tutorials for each Algebra 1
concept which connects the standard to a real-life situation. It then provides both
guided and independent practice, with multiple choice questions similar to those seen
on state tests. There is also an assessment component to the program so that teachers
can keep track of which standards their students had mastered, and those which still
needed work. This program can be used for whole-class instruction using an LCD
projector, or it can be used in a computer lab with each student using it
independently. Furthermore, since it is web-based, students can access if from home
once they are given a password. Incorporating this technology into the Algebra
16
curriculum was a way to reinforce the standards, while making learning more
interesting and relevant to the students. The goal would be that this would also help
increase motivation to learn since it is a more hands-on approach and is very
interactive. Teachers were given two in-service days to be trained on the program,
and the software was implemented for the 2006 school year.
In summary, the intervention designed in this study is the implementation of
a standards based curriculum at Harrison Park High School. There is a wide variety
of research available on standards-based teaching, and overall the research does
show that there is a positive correlation between a standards-based curriculum and
test scores. In their study, The Influence of Standards on K-12 Teaching and Student
Learning (2005), Lauer, Snow, Martin-Glenn, Van Buhler, Stoutemyer, and Snow-
Renner found that students who were given a standards-based curriculum scored
higher on state assessments than did students who received a more traditional
curriculum. This shows that being given a standards-based curriculum transfers to
performance on standards-driven assessments. Furthermore, they found positive
findings in teacher performance when implementing a standards-based curriculum.
Teachers were likely to change the way they presented material in such a way that it
was aligned more directly with the state standards, and in a way that better prepared
their students for state testing. In conclusion, the research very much supports the
use of a Standards-Based Intervention as a means of improving student achievement
on the Algebra 1 Content Standards Test. The standards based intervention in
Algebra at Harrison Park High School is important to all the stakeholders. By
17
analyzing the test data, the strengths and weaknesses of the Algebra program will be
analyzed in an attempt to improve the current program in subsequent years.
Analyzing student achievement on the Algebra 1 Content Standards Test (CST) will
help direct future improvements in school curriculum and classroom instructional
delivery, leading to an alignment between state expectations and academic
achievement.
18
CHAPTER 2
LITERATURE REVIEW
Introduction
In the state of California over the last ten years, it has become abundantly
clear that something needs to be done to increase the academic achievement of all
students. Student academic achievement, as measured by state test scores, show
tremendous inequity in the populations of students who are achieving on these tests
and those who are not. This is especially true in the area of high school
mathematics, specifically in algebra. For this reason, there has been a recent push
towards “standards-based reform.” This chapter will use current research and
literature on the subject to define what is meant by the term, standards-based reform,
and give a brief background into why this type of reform seems to have become
necessary. Furthermore, Robert J. Marzano’s (2003) framework for research-based
reform will be discussed as a model for achieving successful standards-based reform.
Finally, the literature will be used to discuss the implications of this type of reform
will be discussed in terms of its overall success in schools, its impact on equity, and
policy implications that develop as a result.
What is Standards-Based Reform?
In their research study “Incentives and Equity under Standards-Based
Reform,” Julian R. Betts and Robert M. Costrell (2001) present a very concise
definition of what is meant by the term standards-based reform. “Standards-based
reform is a strategy that includes specifying what is to be learned, devising tests to
19
measure learning, and establishing consequences of performance for students and
schools (Betts and Costrell, 2001, p. 9).” In other words, there are three primary
components to any successful standards-based intervention. Finn and Kanstoroom
(2001) refer to these three components as the “accountability tripod.” Teachers and
students must clearly know what is expected of them. In the state of California,
these expectations are delineated in the California State standards. Furthermore,
measurement devices must be created, such as the California Standards Tests
(CSTs), to assess the proficiency to each state standard, and the reliability and
validity of these tests must be researched to ensure that they are measuring mastery
of the standards that they are intended to measure. In other words, it is not good
enough to simply outline standards if there is no reliable way to measure whether the
standards have been met (Finn and Kanstoroom, 2001). In the state of California, the
CST test results are used to determine a standard of proficiency for each school,
known as adequate yearly progress (AYP). Furthermore, within each content area,
performance benchmarks are used to determine the level to which each student has
achieved proficiency. The five levels of proficiency which a student can achieve are
advanced, proficient, basic, below basic, and far below basic. Finally, consequences
must be put into place for individuals and organizations that do not improve, or meet
the minimum standards. Again, it should be researched to make sure that the
consequences that are put into place are designed to assist these schools, teachers,
and students to improve in the future. The goal of any standards-based reform is to
improve the academic achievement of all students, regardless of socio-economic
20
background, thus creating equity as well as raising achievement. Any successful
standards-based intervention must contain all three elements outlined above (clear
standards, assessments, and consequences) in order to be successful in both of these
areas.
Resnick and Zurawsky (2005) discuss a fourth essential component for
successful standards-based reform. This component is the development of standards-
based instructional programs and teacher professional development provided by
schools and districts. In other words, once the expected standards are clearly
defined, and the assessments for determining whether these standards have been met
are developed, it is essential that teachers are given the tools required to ensure that
their students can master the standards and can translate this mastery to the
assessments. For this reason, the standards-based intervention designed for this
study used all four key components in its development and implementation. The
research done to measure the success of standards-based reform, and the role of each
of these components, will be analyzed further in this review.
Background
It has been thought that the idea of standards-based reform first came into
being with the publishing of A Nation at Risk in 1983 by the National Commission
on Excellence in Education. This publication outlined the crisis in education at that
time in terms of equal access to a rigorous curriculum for all students, and the way in
which that curriculum was being delivered. Further emphasizing the shortcomings
of the American educational system was the Third International Mathematics and
21
Science Study (TIMSS), conducted in 1995-1996. This was a comparison study of
academic achievement across forty one nations, and was believed to highlight the
inadequacies of education in the United States. Both of these publications led to the
reintroduction to statewide testing and the creation of consistent content standards in
each subject area. In the state of California, the Public School Accountability Act of
1999 brought into being a statewide accountability system, which contained three
elements to reform and accountability. These elements are the Academic
Performance Index (API), the Immediate Intervention/Underperforming Schools
Program (II/USP), and the High Achieving/Improving Schools Program (HA/IS).
The API is an index used to evaluate and rank schools based on a variety of
measures. These measures include performance on state achievement tests, a high
school exit exam, attendance rates, and graduation rates. The API score is then used
to develop two performance targets. The first is an annual growth target, which is set
based on that year’s baseline score. This minimum growth target is 5% annually.
Second, state performance targets are set that ensure implementation of the state
standards. Schools are then ranked by their growth rates, and how these rates
compare to schools with similar characteristics. These similar characteristics include
ethnicity, socioeconomic status, percentage of students who are English language
learners, average class size, percentage of teachers who are fully credentialed,
percentage of teachers on emergency credentials, and whether the school is on a
multitrack, year-round calendar. “In recognition of the role that socioeconomic
status and language ability play in determining student achievement the state also
22
ranks schools into ten deciles after first grouping schools together with schools with
similar characteristics (Betts and Danenberg, 2002. p. 131)”.
The II/USP is a program implemented as a result of the API score in those
schools who are underperforming according to state accountability measures. This
program provides both state and federal grant money to these schools for both
planning and implementation of an action plan for improvement. “Improving
academic performance, involving parents or guardians, and providing effective and
efficient allocation of resources and school management are among the key elements
of the action plan (Betts and Danenberg, 2002, p. 132).” If a school does not
improve and meet their performance goals for two consecutive years, it will result in
state reassignment of school management, a reorganization of the school, or perhaps
even school closure in extreme cases.
The HA/IS is a program implemented as a result of the API score to reward
those schools who meet or exceed their performance growth targets. The emphasis
of this reward system is on rates of improvement as opposed to the actual
achievement test scores. This is to provide equity due to the fact that schools with
the highest levels of achievement are typically in high-income neighborhoods with
smaller class sizes and fewer English language learners. Lower income schools can
still receive these same rewards due to percent increased in achievement over time,
even if the actual achievement test scores are lower. These rewards include
classification as a distinguished school, placement on a pubic list of honor roll
23
schools, and public commendations from the state legislature (Betts and Danenberg,
2002).
Once this statewide accountability was put in place, schools began feeling the
pressure to ensure that their students achieved on statewide accountability measures,
most importantly the California Standards Tests (CSTs). Many schools began the
process of reforming their curriculum and teacher instructional strategies to ensure
that the California state standards were being taught. However, it was not until the
passage of Public Law 107-110, commonly known as the No Child Left Behind Act
of 2001, that the push for this type of reform was felt by all educators. This law
acknowledged the fact that educational reform was clearly necessary to increase the
academic achievement of all students, and thus close the “achievement gap,”
between students of different socio-economic backgrounds. As outlined in the law,
all students should be given “a fair, equal, and significant opportunity to obtain a
high-quality education and reach, at a minimum, proficiency on challenging State
academic achievement standards and state academic assessments (Public Law 107-
110, 2001, p. 1439).” With this in mind, schools began the task of reforming the
way they were teaching the material to ensure that these common standards were
being met. Schools and teachers began feeling the pressure of ensuring high
achievement for their students, and they were being held accountable for this
achievement. Clearly, something had to be done to align curriculum with the state
assessments and to ensure that all students have equal access to this curriculum.
Furthermore, schools and teachers knew that there were consequences for their
24
students not achieving proficiency, and this further drove schools to reform their
curriculum delivery.
Government Roles in Standards-Based Reform
With the passage of the No Child Left Behind Act of 2001, the U.S.
government made it extremely clear that schools would be held accountable, and
they were clear about the standards to which schools would be held. The direction
for reform had been determined, and incentives were in place to promote this reform.
However, it was felt by many schools that although state standards had been
outlined, and a push for change was obvious, there was little direction about how this
change should take place at the school and classroom level (Goertz, 2001). The list
of standards is broad, so it became clear that if proficiency on the state tests was the
goal, an effort needed to be made to narrow down this list to the most essential
standards, and a curriculum needed to be designed to deliver these essential
standards in the most rigorous way possible. “Teachers and districts frequently
complain that state standards are too general to effectively guide local curriculum
and instruction and that district and school staff members do not have the time or the
expertise to translate these broad goals into practice (Goertz, 2001, p. 63).”
Elmore and Fuhrman (2001) discuss the challenge of balancing the roles of
the government and the schools in implementing standards-based reform. It is again
emphasized that the government’s message about the need for school improvement is
strong, and that schools are paying attention. Schools and teachers need this impetus
to help improve and focus instruction. “New accountability systems help channel
25
teachers’ work to the most important goals, largely those included in the
performance measure (Elmore and Fuhrman, 2001, p. 68).” However, there is wide
and varied interpretation between schools about how this type of focus will be
implemented. Often the expectations underlying the accountability system is unclear
to schools, teachers, and students. Schools know they need to change, but are not
quite sure of the direction that change should take. Standards-based reform is
intended to help define and focus that change. The standards are clearly outlined and
the assessments are in place, so it is now up to the school-level stakeholders to
design how those standards will be mastered. The government is allowing flexibility
among school populations as to how this will take place.
A Framework for Standards Based Reform
A useful framework for designing any standards-based intervention in
schools comes from Robert J. Marzano in his book, What Works in Schools (2003).
Marzano outlines three primary categories of factors affecting student achievement,
each of which must be addressed in implementing any type of reform. The first
category he refers to as school-level factors. These factors are controllable at the
school level via school policies and school-wide decision making. These would
include guaranteed and viable access to the curriculum, parent and community
involvement, safe and orderly environment, and collegiality and professionalism.
The second category discussed is teacher-level factors. These factors are
controllable at the classroom level via teacher instructional strategies and curriculum
design. These factors include classroom management as well as content knowledge
26
and effective delivery of that content. The third and final category is student-level
factors. These factors are controllable at the home-level via parent support and home
environment. These factors include motivation, learned intelligence, and home
atmosphere. The role of all three of these factors must be addressed in any
standards-based reform in order for it to be successful, and this framework was the
driving force in the design and implementation of the intervention that is at the core
of this study.
School-Level Factors
According to Marzano (2003), there are five school-level factors that affect
student achievement. They are (1) a guaranteed and viable curriculum, (2)
challenging goals and effective feedback, (3) parental and community involvement,
(4) safe and orderly environment, and (5) collegiality and professionalism. These
five factors work together to create a school environment that promotes academic
achievement. Students must have equal access to a rigorous curriculum and a daily
schedule that promotes adequate access to this curriculum. Goals for teachers must
be made clear by administrators, and teachers must be given constant feedback as to
their progress in achieving these goals. Parents and students must be made aware of
school goals, and participate actively in the achievement of those goals. The school
campus must be secure, and there must be structure and routine to each school day
and a scholarly atmosphere that promotes learning at all times. Finally, teachers
must collaborate together and give each other constructive feedback as to the
delivery of standards-based instruction. There must be a common goal for all
27
stakeholders (administration, teachers, students, parents), and this goal needs to be
communicated as well as a solid rationale as to why reform is necessary to achieve
this goal.
Teacher-Level Factors
In his framework, Marzano (2003) identifies three teacher-level factors that
contribute to student academic achievement in schools. They are (1) instructional
strategies, (2) classroom management, and (3) classroom curricular design. These
factors work in unison in the classroom, and the strongest teachers are able to
balance the need for structure and discipline with the need to allow students to
explore, learn, and generate their own ideas and solutions. Classroom instruction
should be designed around the state standards, and should be designed in such a way
that the teacher is constantly checking for understanding and allowing guided
practice. A strong instructional design will often take care of classroom management
issues since students are engaged and actively participating in their learning.
Students need daily structure and a learning atmosphere that is academically driven.
Finally, the curriculum needs to be outlined and designed to teach the most essential
state standards for that content area. The curriculum needs to be clearly outlined and
accessible to all teachers in that subject area. Teachers should be given constant
professional development to keep up on current trends and methodologies
surrounding standards-based instruction.
28
Student-Level Factors
Marzano (2003) identifies three student-level factors that contribute to high
academic achievement. These factors are (1) home environment, (2) learned
intelligence and background knowledge, and (3) motivation. These factors are the
most difficult to control at the school and classroom level since they largely have to
do with the upbringing of the student, and the students’ past experiences both inside
and outside the classroom. However, steps can be taken at school to compensate for
negative student-factors. “…the research clearly shows that even some of the most
negative aspects of a student’s background can be mediated by school-based
interventions (Marzano, 2003, p. 123).” Schools need to focus on open
communication between the school and parents as far as the importance of education,
and the impact that a successful education will have on the future lives of the
students. Schools can provide workshops for parents about creating a home
atmosphere that supports learning, as well as send written materials on a regular
basis with advice on how to help their children succeed. The classroom teacher also
has a very powerful influence in changing students’ preconceived ideas about what
they are able to achieve. They can obtain information about a student’s background
knowledge, and use this to help increase the student’s motivation to learn. Teachers
can also use incentives to help increase motivation, and as students see themselves
succeed, this help to shape their ideas about their leaned intelligence. Finally,
teachers should make it a priority to communicate to their students the state
standards that they are being required to master, and refer to these standards often, as
29
well as provide a rationale as to why these standards are important to the students
themselves.
Policy Implications of Standards-Based Reform
Betts and Danenberg (2002) have revealed two important trends in the data
surrounding the movement toward school accountability in California. First, test
scores in California are, in fact, increasing. This is true overall for all students,
regardless of socioeconomic status or whether or not they are English language
learners. In fact, students from the lowest performing schools have had the largest
gains in achievement as measured by the California Standards Tests. Second,
teacher resources have declined, especially in the lowest-performing schools. So this
brings the question of why test scores are increasing while resources are decreasing.
Betts and Dannenberg (2002) present two possible reasons this seems to be
occurring. First, students may be becoming more familiar with the test, and teachers
may in fact be “teaching to the test.” However, an alternate explanation might be
that this growth despite declining resources truly represents improvement brought on
increased pressure of state accountability programs and publicly displayed data on
school performance. Regardless of the reasons for the improvement, the movement
in California for standards-based accountability has spurred some very important
positive changes within the schools (Betts and Danenberg, 2002). There is now a
clearly defined list of standards in each content area so that there is a consistent set
of expectations for all students about what they are expected to master.
Improvements in professional development for teachers have evolved to ensure that
30
they are informed about the changing educational trends in California. A statewide
testing system has been implemented to help schools evaluate their progress
longitudinally, and incentives are in place to motivate them to do so. Finally, high
expectations are in place for teacher training, and teacher preparation programs are
focused on improving instructional strategies for an ever-changing student
population.
Overall, the literature shows that schools who successfully implement
standards-based interventions have a high sense of internal accountability within the
school culture, and that this internal accountability far outweighs the impact of any
external accountability from the state. In other words, the schools that have
improved the most have collaborated and defined a common goal about what they
expect of their students academically and the means by which they will reach that
goal (Elmore and Fuhrman, 2001). The successful schools have taken state
accountability mandates, and incorporated them internally by creating a means of
reaching expectations in a way that best suits the needs of their student population.
Schools that are not successful tend to have teachers and/or students that view the
mandates as unrealistic for their student population, and therefore focus primarily on
test preparation as opposed to overall improvement. Therefore, any successful
standards-based intervention needs to have the active involvement and support of all
the stakeholders: teachers, students, and parents. They all need to be involved in the
decision making process that will determine the direction their school will take
toward school academic improvement.
31
Equity in Standards-Based Education
One final concern that should be addressed in standards-based education is
the issue of equity among students. In the field of mathematics specifically, it has
become clear that there was an achievement gap, particularly at the high school level.
The traditional mathematics curriculum in California high schools was designed with
college-prep students in mind, focusing on skills that would be necessary to be
successful in higher-level mathematics classes (Schoenfeld, 2002). The curriculum
did not focus on those skills that would help students in their everyday lives, or in
non-mathematics related careers. For this reason, all non-college bound students
were being left out, since they were traditionally not successful at the college prep
classes. Standards-based reform in mathematics was implemented with this thought
in mind. There must be equal access to a high quality mathematics curriculum, and
there must be standards to outline what that means in terms of curriculum content.
In addition to identifying the content students should know, the Standards
focused on process: there was to be a focus at all grade levels on problem
solving, reasoning, connections (between mathematical topics and to real
world applications), and the communication of mathematical ideas in written
and oral form (Schoenfeld, 2002, p. 8).
This concept of equity and equal access to the curriculum is extremely relevant to
this study since there are five different class levels of Algebra that a student can be
enrolled in, but all of these students are required to take the same California
Standards Test despite the fact that they were exposed to very different curricula.
This contradiction will be discussed further in chapter 5.
32
CHAPTER 3
METHODOLOGY
Summary of Research Design
The purpose of this research study was to determine whether a standards-
based intervention in Algebra at Harrison Park High School (name has been
changed) has an effect on academic achievement as measured by the Algebra 1
California Standards Test (CST), student grades, and teacher perception. The study
was designed as pre-post nonequivalent control group design, with the intervention
taking place during the school year 2005-2006 in several phases, and as post only
quasi experiment with a nearby school, Ericson High School (name has been
changed), serving as a control. The study was designed as a combination of both
summative and formative evaluation techniques, which will be discussed further
here.
Summative Evaluation Design
The summative evaluation involved quantitative methods, using the Algebra
1 CST test score data as the means of answering the research question, “Does a
standards-based intervention in Algebra 1 have an impact in mathematics proficiency
as measured by the California Standards Tests?” The study was pre-post
nonequivalent control group design, with pre-test data being analyzed from the
school year 2004-2005, with the intervention and post-test taking place during the
school year 2005-2006. Harrison Park High School was treated as the experimental
group, and there was one control group selected, Ericson High School, also located
33
in the same Union High School District. The standards-based intervention was
implemented in all five class levels of Algebra at HPHS (the experimental school),
and the CST scores of the two schools were then compared. The control school was
selected due to its similarities to Harrison Park High School in terms of the number
of students enrolled, average class size, the ethnic diversity of the students (or lack
thereof), percent of students who are English Learners, and percent of students on
Free or Reduced Price Lunch Program. Furthermore, the two schools are similar in
Academic Performance Index (API), according to data from the California
Department of Education. This information is summarized in the following table.
Table 4:
Selection Criteria for Experimental and Control Groups
School HPHS Ericson
Number Enrolled 2658 2501
Average Class Size 29.3 29
%EL 5.3% 2.8%
% Free/Reduced Lunch 7.9% 5.2%
API Score 2003 727 748
API Score 2004 746 791
API Score 2005 765 799
Math proficiency in Algebra 1 was defined based on the number of students scoring
Far Below Basic, Below Basic, Basic, Proficient, and Advanced on the California
Standards test. This design is modeled by the following conventional scientific
notation.
E O1 X O2
C O1 O2
34
This notation represents the following experimental design.
E Experimental Group (HPHS) Pre2005 X Intervention Post2006
C Control Group (EHS) Pre2005 Post2006
The quantitative data used in this study were the results of the California
Standards Test (CST) in Algebra 1. Three years of test data (2003, 2004, and 2005)
were collected prior to the standards-based intervention, in addition to the number of
students performing in each of the five proficiency categories for each of these years.
The percentage of students scoring proficient and above on the Algebra 1 CST was
determined, as this was defined as a “baseline” level for proficiency in Algebra 1 as
per state and national legislation. Furthermore, the percentage of students scoring
basic and above on the Algebra 1 CST was determined, as this is another useful
measure of academic achievement. The intervention took place during the school
year 2005-2006, and the 2006 CST test was given after one year of this intervention.
Three dependent variables were analyzed and compared from pre-intervention
(2005) to post-intervention (2006) to determine whether the standards-based
intervention in Algebra 1 had a “practical” impact on math proficiency at Harrison
Park High School. First, mean performance scores were compared across years.
Second, the percentage of students scoring proficient and advanced in each year was
analyzed. And finally, the percentage of students scoring basic, proficient, and
advanced in each year was compared. Furthermore, the post-test data for 2005-2006
was compared with that of the control school which did not receive the intervention
to further analyze the intervention’s impact.
35
Formative Evaluation Design
The formative evaluation involved qualitative methods to seek ways to
improve the intervention. The evaluation focused on three primary aspects of the
intervention. The first was the design of the intervention itself in terms of
maximizing strengths and minimizing weaknesses. This aspect also focused on the
question of how the standards-based program could be improved in the future. The
second was the actual implementation of the intervention in terms of teacher buy-in
and teacher expectations. Finally, student and teacher reaction to the intervention
was analyzed as to whether it helped to increase student motivation to be successful
in Algebra, and whether the increased focus on the California state standards helped
to increase student proficiency in this subject.
The method of data collection that was used to analyze the design of the
standards-based intervention was primarily document-analysis, including curriculum
maps that were developed during the first phase of the intervention and the
California state Algebra 1 standards. Interviews with Algebra teachers were also
used to determine the perceived areas of improvement that could be made in the
future. Methods of data collection that were used to analyze the implementation of
the intervention were unobtrusive observations, interviews with teachers and
students, and further document analysis. Finally, student and teacher reaction were
analyzed with classroom observations and interviews. The foundation of the
formative research was driven by the following questions, which were incorporated
into the guideline questions used in the general interview guide.
36
1) How is standards-based instruction implemented in the
mathematics classrooms at HPHS?
2) How do the California state standards for Algebra 1 drive
instruction?
3) Are students knowledgeable about the standards they are being
taught?
4) What are the strengths of standards-based instruction?
5) What are the weaknesses of standards-based instruction?
6) To what extent does standards-based instruction affect student
motivation?
7) To what extent does standards-based instruction affect teacher
expectations of student achievement?
8) Does standards-based instruction improve student mastery of
algebra concepts?
Again, all of these data were utilized to determine the extent to which the standards-
based intervention in Algebra helped to increase student proficiency in that subject.
The data was further utilized to facilitate Algebra program improvement for the
future.
Intervention
The driving force behind the Algebra 1 standards-based intervention is the
California Standards Test (CST) that is given at the end of each school year. In
analyzing test scores from 2003-2005, it became clear that improvement must be
37
made in student mastery of the standards being tested. The curriculum being
delivered in the classroom must be aligned to the California state standards being
tested. In order to begin this process of alignment, it must be a priority that each
Algebra teacher is closely familiar with the standards. Therefore, Phase 1 of the
intervention at Harrison Park High School involved a staff development component
to ensure that teachers were aware of current standards. Algebra 1 teachers were
sent to a Model Schools conference which focused on schools that had achieved
success in teaching to the standards. At this conference, teachers were given a list of
all the state Algebra 1 standards, as well as the statistical likelihood that each
standard would actually appear on the California standards test as determined from
their appearance on past tests. Teachers were then given time to work
collaboratively to reduce this long list of standards to a more condensed list of
“essential” standards that must be covered in the Algebra curriculum for students to
be successful. For example, two standards that were determined to be tested the
most often were Algebra 1 standards 10.0 and 14.0. Standard 10.0 requires that
students can add, subtract, multiply, and divide monomials and polynomials and
solve multi-step problems, including word problems, by using these techniques.
Standard 14.0 requires that students can solve a quadratic equation by factoring or
completing the square. Since these standards were considered to be part of the list of
“essential” standards, Algebra teachers would make it a priority that these standards
were taught over others that were not tested as often. Throughout this process of
prioritizing the standards, teachers became very familiar with all the standards, and it
38
allowed them to focus their curriculum on just the essential standards to make the
task of teaching the standards more manageable. They could now move into Phase 2
of the intervention, which involved re-designing the curriculum of each of the five
class levels of the Algebra curriculum according to these essential standards.
For Phase 2 of the intervention, Algebra teachers were paid during the
summer to design a common Algebra curriculum for each of the five class levels of
Algebra that they would all follow throughout the year. Marzano (2003) discusses
the importance of organizing the curriculum with a common sequence so that all
students have equal opportunity to learn it. The teachers at Harrison Park High
School designed a curriculum “map” for each of the five class levels of Algebra,
which outlined this common sequence for each class level quarter-by-quarter, as well
as listing each of the Algebra 1 standards that would be addressed in each unit. The
teachers used the results of their work at the Model Schools conference to help guide
this process. In this way, they narrowed the focus of each course to primarily the
essential standards, and if time permits, the additional standards as well. They also
aligned the standards to the corresponding chapters in the textbook for easy
reference.
Once the curriculum “map” is created, any new teacher could come in to
teach any of the five class levels of Algebra, take a look at the curriculum map, and
know exactly what they were expected to teach each quarter. Because of this,
students were being guaranteed equal access to the curriculum at each level,
regardless of who was teaching it. Furthermore, teachers also worked during this
39
time to create common assessments for the end of each chapter, and for the semester
final exams. Again, this further encourages teachers to teach to the same standards,
and guarantees that all students are receiving a rigorous standards-based education in
Algebra at each class level. The resulting curriculum map for the one year Algebra 1
course created during this time is included in the following table. The curriculum
map for the two year Basic Algebra and Essentials of Algebra course consists of the
same standards and concepts, but instead of being divided into four quarters, it is
divided into four semesters, therefore taking two years to complete. The Essentials
of Algebra course covers the standards at a much more basic level, providing a great
deal of remediation along the way, while the Basic Algebra course covers the
standards much more in depth, providing enrichment to reinforce the concepts.
40
Table 5:
Curriculum Sequence for Algebra 1 at Harrison Park High School
First Quarter Second Quarter Third Quarter Fourth Quarter
Ch. 6-Polynomials
Standards 2.0,
10.0, 11.0
Sections 6.2, 6.3,
6.4, 6.5, 6.6, 6.7,
6.8, 6.9
Ch. 8-Rational
Numbers
(continued)
Ch. 11-Systems of
Equations
Standards 9.0,
15.0
Sections 11.2,
11.3, 11.4, 11.5,
11.6
Ch. 13-
Quadratics
Standards 14.0,
19.0, 20.0, 21.0,
22.0
Sections 13.1,
13.2, 13.4, 13.5,
13.6, 13.7
Ch. 7-Factoring
Standards 11.0,
14.0
Sections 7.1, 7.2,
7.3, 7.5, 7.6, 7.7,
7.8, 7.9, 7.10
Ch. 9-Graphing
Standards 16.0,
17.0, 18.0
Sections 9.1, 9.2,
9.3, 9.4, 9.5
Ch. 12-Radical
Expressions
Standards 1.0, 2.0
Sections 12.2,
12.3, 12.4, 12.5,
12.6, 12.7, 12.8
Ch. 4-
Applications of
Rational Numbers
Standards 15.0,
23.0
Sections 4.1, 4.2,
4.3, 4.4, 4.6, 4.7
Ch. 8-Rational
Numbers
Standards 12.0,
13.0
Sections 8.1, 8.2,
8.3, 8.4, 8.5, 8.7,
8.8, 8.9
Ch. 10-Graphing
Linear Equations
Standards 6.0,
7.0, 8,0
Sections 10.1,
10.2, 10.4, 10.5,
10.6
The final phase of the intervention was the implementation stage during the
school year 2005-2006. Each teacher followed the curriculum map as they began the
first quarter, and they all agreed to write the standard they were teaching each day
clearly on the chalkboard so that the students were also aware of which standard they
were focusing on that day. The standard was referred to throughout the lesson to
further reinforce student awareness. Furthermore, as part of the intervention,
teachers agreed to have students do a daily “warm up,” where students were given
questions from previous California Standards tests to help give them more familiarity
of the types of questions they would encounter during the state testing. During the
41
first semester, teachers were also given a limited amount of release time to observe
other Algebra teachers who had been teaching to the standards for years both on site
and at other schools in the district to get ideas about how to improve delivery of
these standards. The teachers would then meet with these mentor teachers, and
discuss strategies to further strengthen their implementation of the standards-based
curriculum.
Another important part of the implementation phase of the intervention was
the incorporation of technology in instruction. Harrison Park High School purchased
a web-based computer program called Riverdeep during the spring of 2005, and all
Algebra teachers were given two days of training on how to use the program as part
of their classroom instruction. This program was recently written, and is directly
aligned with the California state standards. The program has a tutorial component
that can be used for whole-class instruction with the use of a LCD projector, or it can
be used for individual student use in a computer lab. If the students are working in a
lab, after the tutorial they are given a quiz consisting of multiple choice questions
similar to those given on the CST Algebra 1 test. The computer program will score
the quiz and let the student know which standards he or she has mastered, and which
need further practice. The student is then automatically given a remedial tutorial on
the standards that need work, and then quizzed again until mastery is reached.
Teachers can monitor these assessments as part of the program to make sure their
students are mastering the standards. Additionally, the program is web-based, so
students can access it from home with the use of a computer, allowing the teacher to
42
assign additional practice as needed. Because computer technology allows a very
hands-on approach to learning, the goal is to make the learning more relevant and
interactive for the student, thus increasing motivation. Familiarity with the state
standards is also being reinforced throughout the process. As part of the standards-
based intervention in Algebra, all eight Algebra teachers incorporated this program
into the curriculum, utilizing the computer lab on a regular basis.
Participants and Setting
The participants in this study consisted of the students and teachers of
Algebra. 997 students were enrolled in one of the five class levels of Algebra during
the year of the intervention, and eight teachers taught the five class levels. The
Algebra students can be broken down into five distinct groups. The first group was
those students who are enrolled in a one year Algebra 1 course. This group
completed all the Algebra 1 standards in one year, and took the corresponding
Algebra 1 Content Standards test at the end of that year. The second and third
groups of students were those that were enrolled in Basic Algebra B, the first year of
a two year Basic Algebra course, or Basic Algebra D, the second year of that same
two year course. This course moves at a slower pace, and completes the same
standards as the Algebra 1 course over a two year period. These students were
required to take the Algebra 1 Content Standards Test at the end of the year, even if
they were only enrolled in the first year and had only completed half the Algebra 1
curriculum. The fourth and fifth groups of students were those that were enrolled in
Essentials of Algebra B and Essentials of Algebra D. Again, this is the first and
43
second year of a two year algebra course, but is a remediation course and moves at a
very slow pace and only covers the surface of each essential Algebra 1 standard.
Again, these students were required to take the Algebra 1 Content Standards Test at
the end of the year, even if they were only enrolled in the first year. Three years of
pre-intervention data was analyzed. For the years 2003, 2004, and 2005, the
percentage of students scoring far below basic, below basic, basic, proficient, and
advanced was determined. The intervention was given to Algebra students during
the school year 2005-2006. The 2006 California Standards Test, given after one year
of the standards-based intervention, was compared to that of the previous year to
determine the impact the intervention had on math proficiency.
The second group of participants was the Algebra teachers. Eight certificated
teachers taught either the one-year or two-year Algebra 1 course during the
intervention period. All eight teachers hold a California single-subject credential in
Mathematics. These teachers agreed to participate in the implementation of the
standards-based intervention, and their input will be solicited via interviews in
gathering information about the impact the intervention had on student achievement
in mathematics.
The setting of this study was Harrison Park High School, located in
Huntington Beach, California. This is a large city with an economically diverse
population, although most if its residents are considered upper-middle class. School
demographic data shows that the school served 2,658 students during the 2005-2006,
the year of the intervention. Average class size was 31 students. The student
44
population was primarily White (67% of total enrollment), followed by a significant
number of Hispanic and Asian students (14% and 10% of total enrollment,
respectively). There was a very active parent population, with a strong PTSA group
that supports a rigorous curriculum and has high expectations for school test scores.
Instrumentation: Achievement
The instrumentation used to measure student Algebra 1 achievement was
publicly available data compiled by the California Department of Education
outlining the results of the 2003-2006 California Standards Tests in Algebra 1. This
data divides student results into five performance bands. These are the percent
scoring Advanced, the percent scoring Proficient, the percent scoring Basic, the
percent scoring Below Basic, and the percent scoring Far Below Basic. According
to state and national legislation, the goal is for every student assessed to achieve
proficient or above, representing a high level of mastery (EdSource, 2005). The state
and federal government for California then summarizes this information using what
is known as the Academic Performance Index (API). This score ranges from the
lowest score of 200 to the highest scores of 1000, and every school is expected to
have a score of at least 800 (Edsource, 2005). This score is used to rank schools into
ten deciles, each representing 10% of the schools, providing information on how a
school compares to other schools in the state. Next, schools are compared to other
schools that are similar to them in terms of student characteristics and demographics,
providing information on how a school compares to other schools with similar
45
obstacles. Finally, schools are given a target for improving their API score for the
following academic year.
Procedures: Achievement
Prior to the intervention, achievement data were collected from the California
Department of Education regarding the test results for the Algebra 1 Content
Standards Test for the years 2003, 2004, and 2005. These data were collected for
both the experimental school, Harrison Park High School, and the control school,
Ericson High School. The percentage of students scoring basic, proficient, and
advanced was calculated. After the completion of the intervention, test score data for
2006 were collected in the same manner for both schools and compared to that of the
previous data to determine the extent to which the standards-based intervention
impacted mathematics achievement.
Instrumentation: Interviews
Before and during this study, interviews were conducted with teachers and
administrators to determine the extent to which the standards-based intervention
impacted student achievement in mathematics. Two types of interviews were
conducted. The first type of interview used was the informal conversational
interview. For these interviews, the researcher would initiate an informal
conversation about the intervention with the Algebra 1 teacher, but no formal set of
questions was used. The second type of interview was the general interview guide
approach. For this interview, a set of guideline questions was developed with the
main topics to be covered during the interview, but the actual questioning remained
46
open-ended and flexible. In this way, the researcher could change the direction of
the interview based on the answers given by the interviewee, but would still cover
the same main topics with each person being interviewed, maintaining some
consistency in questioning. The guideline questions were as follows:
1) How is standards-based instruction implemented in mathematics
classrooms at HPHS?
2) How do the California state standards for Algebra 1 drive instruction?
3) Are students knowledgeable about the standards they are being
taught?
4) What are the strengths of standards-based instruction?
5) What were the weaknesses of standards-based instruction?
6) To what extent does standards-based instruction affect student
motivation?
7) To what extent does standards-based instruction affect teacher
expectations of student achievement?
8) Does standards-based instruction improve student mastery of algebra
concepts?
Procedure: Interviews
Interviews were conducted at Harrison Park High School during the school
year 2005-2006, the year of implementation of the standards-based intervention. All
eight Algebra teachers were interviewed with the general interview guide approach.
Approximately seven additional mathematics teachers and administrators were
47
interviewed with both the informal conversational interview and the general
interview guide approach. Interviews lasted from 15-30 minutes in length. The
researcher recorded information received in the interviews with note-taking, both
during and directly following the interviews for the general interview guide
approach, and only following the interview for the informal conversational approach.
Instrumentation: Observations
The researcher conducted observations of the Algebra classrooms at Harrison
Park High School. The main focus of the observations was the extent to which the
standards-based Algebra 1 curriculum was being implemented in the classroom. The
researcher looked for clear posting of the standards in the classroom, and for how
often the classroom teacher referred to those standards throughout the lesson.
Furthermore, observations also took place in the computer lab to ascertain the extent
to which Riverdeep was being used to support the standards-based instruction. Both
teacher and student actions were analyzed during these observations. Consistency
across classrooms was analyzed as well to determine the rigor with which teachers
were implementing the structured curriculum.
Procedure: Observations
Classroom observations took place during the 2005-2006 school year, the
year of the intervention. All eight Algebra teachers were observed twice during this
time, with visits lasting from 45-60 minutes. The researcher recorded observations
with note-taking both during and after the observations. Both teacher and student
behavior was recorded and analyzed. The researcher sat quietly in the back of the
48
classroom during all observations, and was therefore not an active participant in the
classroom activities.
Instrumentation: Document Analysis
Documents were analyzed during this study as a final source of information
in data collection. The primary documents utilized were the California State Content
Standards for Mathematics, curriculum guides and maps, and common assessment
tools used by Algebra teachers. These documents were analyzed to verify the extent
to which all Algebra teachers were teaching a common, standards-based Algebra
curriculum to their students.
Procedure: Document Analysis
These documents were read and analyzed as a final source of qualitative data
as to the impact that the standards-based intervention had on Algebra 1 achievement.
The standards, maps, and assessment tools were used together to evaluate the
consistency of the intervention in the different Algebra classrooms. These
documents were used to help tie together the information collected from achievement
data, as well as the information collected through the interviews and observations.
49
CHAPTER 4
FINDINGS
Introduction
For the purposes of analyzing test data from the California Standards Test in
Algebra 1, the five proficiency categories reported by the state of “far below basic,
below basic, basic, proficient, and advanced” were converted to a numerical scale
ranging from 0 through 4, respectively, to represent the five categories. “Far below
basic” is represented by a value of 0, “below basic” is represented by a value of 1,
“basic” is represented as a value of 2, “proficient” is represented as a value of 3, and
“advanced” is represented as a value of 4. Furthermore, the five different levels of
algebra courses at Harrison Park High School in which students are enrolled were
converted to a numerical scale ranging from 0 through 4 to represent the course.
Essentials of Algebra B is represented as 0, Essentials of Algebra D is represented as
1, Basic Algebra B is represented as 2, Basic Algebra D is represented as 3, and
Algebra 1 is represented as 4. Tables 6 through 17 and Figures 1 through 3 in the
following analysis reflect the data analysis of the experimental school, Harrison Park
High School (HPHS).
Mean differences were tested for statistical significance (p < .05) using a 2
(year) by 5 (class level) ANOVA. Mean differences also were tested for practical
significance using Cohen’s d (d > .30) and percentage change (change > 10%).
Tables 15-17 show the results for performance scores (scaled as discussed above),
percent proficient and above and percent basic and above, respectively. Because 2 of
50
3 two way (class level by year) interactions were significant, Tables 6 though 14 and
Figures 1-3 break down the means by sequence and year, and the discussion focuses
on the practical significance of the differences.
Tables 18 through 29 and Figures 4 through 6 reflect the data analysis of the
control school, Ericson High School (EHS). This analysis parallels the analysis
discussed above. Finally, Table 30 summarizes the differences between HPHS and
EHS.
Effects on Algebra 1 CST Performance: 2005-2006
Table 6 shows the mean performance changes for all students tested on the
Algebra 1 CST. All five class levels are listed with a pre-intervention mean in 2005
and a post-intervention mean in 2006 at Harrison Park High School. The last column
reflects the amount of change from pre-intervention mean (2005) to post-intervention
mean (2006).
Table 6:
Mean Statistics by Class Level
Class Level 2005 2006 Difference
Essentials of
Algebra B
.75
(n = 160)
.86
(n = 109)
+.11
Essentials of
Algebra D
.77
(n = 119)
.95
(n = 132)
+.18
Basic Algebra B 1.23
(n = 207)
1.35
(n = 238)
+.12
Basic Algebra D 1.61
(n = 175)
1.43
(n = 225)
-.18
Algebra 1 2.38
(n = 365)
2.49
(n = 293)
+.11
Total 1.57
(n = 1026)
1.60
(n = 997)
+.03
51
Table 6 reflects positive mean changes made in all class levels at Harrison
Park High School except Basic Algebra D, which had a change of -.18. Overall,
Essentials of Algebra D made the largest positive change with +.18, Basic Algebra B
with the next largest positive change at +.12, and Essentials of Algebra A and
Algebra 1 making the smallest amount of positive mean change of +.11 over the one
year period. The total mean difference of all class levels combined results in +0.03,
which means that the standards-based intervention in algebra contributed to modest
gains in overall algebra achievement at Harrison Park High School as measured by
the Algebra 1 CST. Figure 1 shows a graph of the mean statistics by class level
summarized in Table 6. The class levels have been labeled with the numerical scale
assigned to them as described in the introduction of this chapter.
Figure 1:
Estimated Marginal Means by Class Level
0
0.6
1.2
1.8
2.4
3
012 34
2005 2005
2006 2006
Class Level
Estimated Marginal Means
52
Effect size estimates for the mean changes by class level on the Algebra 1
CST are shown in Table 7. Using effect size estimates, data are analyzed to
demonstrate practical significance of the academic progress made pre-intervention
(2005) to post-intervention (2006). Effect size was calculated by dividing the
difference from pre-intervention (2005) to post-intervention (2006) by the observed
standard deviation of the pre-intervention (2005) Algebra 1 CST classification.
Table 7:
Effect Size Estimates
Essentials
of Algebra
B
Essentials
of Algebra
D
Basic
Algebra B
Basic
Algebra D
Algebra 1
Gain +.11 +.18 +.12 -.18 +.11
2005 SD .614 .630 .809 .837 .848
Effect Size +.179 +.286 +.148 -.215 +.130
***The effect size estimate was +.029 for the total sample.
Table 7 shows that the effect size for Essentials of Algebra D was nearly
practically significant at +.286, but the effect size at the other class levels at Harrison
Park High School was small (less than ± .30).
Percentage of Change of Algebra 1 CST Means: 2005 vs. 2006
The percentage of change of Algebra 1 CST means are reflected in Table 8.
Percentage of change estimates are a second measure used to demonstrate the
practical significance of the academic progress made pre-intervention (2005) to post-
intervention (2006). Both pre-intervention (2005) and post-intervention (2006)
means are listed, as well as the percentage of change from pre-intervention to post-
intervention.
53
Table 8:
Percentage of Change of Mean Score by Class Level
Essentials
of Algebra
B
Essentials
of Algebra
D
Basic
Algebra B
Basic
Algebra D
Algebra 1
Mean 2005 .75 .77 1.23 1.61 2.38
Mean 2006 .86 .95 1.35 1.43 2.49
Difference +.11 +.18 +.12 -.18 +.11
Percentage
of Change
+14.7% +23.4% +9.8% -11.2% +4.6%
Table 8 shows that the percent gains made for Essentials of Algebra B,
Essentials of Algebra D, and Basic Algebra B were practically significant at +14.7%,
+23.4%, and +9.8%, respectively. The percent gain for Algebra 1 was small, and
there was a percent loss in Basic Algebra D.
Effects on Algebra 1 CST Performance: Proficient and Above
Mean changes for the percentage of students who scored at the proficient and
advanced levels for each class level at Harrison Park High School on the Algebra 1
CST are shown in Table 9. According to No Child Left Behind (NCLB), students
who are considered meeting the guidelines of proficiency achieve at the proficient
and advanced levels on the CST. Again, all five class levels are listed with a pre-
intervention mean in 2005 and a post-intervention mean in 2006 at Harrison Park
High School. The last column reflects the amount of change from pre-intervention
mean (2005) to post-intervention mean (2006).
54
Table 9:
Mean Statistics by Class Level: Proficient and Above
Class Level 2005 2006 Difference
Essentials of
Algebra B
.000
(n = 160)
.046
(n = 109)
+.046
Essentials of
Algebra D
.000
(n = 119)
.053
(n = 132)
+.053
Basic Algebra B .058
(n = 207)
.084
(n = 238)
+.026
Basic Algebra D .120
(n = 175)
.111
(n = 225)
-.009
Algebra 1 .499
(n = 365)
.553
(n = 293)
+.054
Table 9 reflects positive mean changes in all class levels at Harrison Park
High School, with the exception of Basic Algebra D, which had a change of -.009.
Overall, Algebra 1 made the largest positive change with +.054, Essentials of
Algebra D with the second largest positive change at +0.053, Essentials of Algebra B
with the third largest positive change of +.046, and Basic Algebra B making the
smallest amount of positive mean change of +.026 over the one year period. The
total mean difference of all class levels combined was +.010, showing that the
standards based intervention in Algebra 1 at Harrison Park High School contributed
to modest gains in the percentage of students scoring proficient and above on the
Algebra 1 CST. Figure 2 shows a graph of the mean statistics summarized in Table
9. The class levels have again been labeled with the numerical values assigned to
them as described previously.
55
Figure 2:
Estimated Marginal Means by Class Level: Proficient and Above
0
0.12
0.24
0.36
0.48
0.6
0 12 34
2005 2005
2006 2006
Sequence
Estimated Marginal Mean of Proficiency
Effect size estimates for the mean changes in proficiency by class level on the
Algebra 1 CST are shown in Table 10. Using effect size estimates, data are analyzed
to demonstrate practical significance of the academic progress made pre-intervention
(2005) to post-intervention (2006). Again, effect size was calculated by dividing the
difference from pre-intervention (2005) to post-intervention (2006) by the observed
standard deviation of the pre-intervention (2005) Algebra 1 CST classification.
56
Table 10:
Effect Size Estimates for Proficiency
Essentials
of Algebra
B
Essentials
of Algebra
D
Basic
Algebra B
Basic
Algebra D
Algebra 1
Gain +.046 +.053 +.026 -.009 +.054
2005 SD .000 .000 .234 .326 .501
Effect Size NA NA +.111 -.028 +.108
***The effect size estimate was +.025 for the total sample.
Table 10 shows that the effect size for all class levels at Harrison Park High
School was small (less than ± .3). Effect sizes for Essentials of Algebra B and
Essentials of Algebra D were not calculated due to the .000 standard deviation in
2005 for both of those classes.
Percentage of Change Estimates of Algebra 1 CST Proficiency: 2005 vs. 2006
The percentage of change estimates of Algebra 1 CST proficiency are
reflected in Table 11. Percentage of change estimates are a second measure used to
demonstrate the practical significance of the academic progress made pre-
intervention (2005) to post-intervention (2006). Both pre-intervention (2005) and
post-intervention (2006) means are listed, as well as the percentage of change from
pre-intervention to post-intervention.
Table 11:
Percentage of Change by Class Level: Proficient and Above
Essentials
of Algebra
B
Essentials
of Algebra
D
Basic
Algebra B
Basic
Algebra D
Algebra 1
Mean 2005 .000 .000 .058 .120 .499
Mean 2006 .046 .053 .084 .111 .553
Difference +.046 +.053 +.026 -.009 +.054
Percentage
of Change
NA NA +44.8% -7.5% +10.8%
57
Table 11 shows that the percent gains made for Basic Algebra B and Algebra
1 were practically significant at +44.8% and +10.8% respectively. Again, there was
a percent loss in Basic Algebra D. Percentage of change was not calculated for
Essentials of Algebra B and Essentials of Algebra D due to the .000 mean in 2005
for these classes.
Effects on Algebra 1 CST Performance: Basic and Above
According to NCLB, a student is considered “proficient” in the subject area if
they score a level of proficient or advanced on the CST. However, it is also useful to
look at the number of students who score at the basic, proficient, and advanced levels
in analyzing student achievement. Mean changes for the percentage of students who
scored basic and above on the Algebra 1 CST are shown in Table 12. Again, all five
class levels are listed with a pre-intervention mean in 2005 and a post-intervention
mean in 2006 at Harrison Park High School. The last column reflects the amount of
change from pre-intervention mean (2005) to post-intervention mean (2006).
Table 12:
Mean Statistics by Class Level: Basic and Above
Class Level 2005 2006 Difference
Essentials of
Algebra B
.094
(n = 160)
.193
(n = 109)
+.099
Essentials of
Algebra D
.109
(n = 119)
.212
(n = 132)
+.103
Basic Algebra B .352
(n = 207)
.416
(n = 238)
+.064
Basic Algebra D .594
(n = 175)
.462
(n = 225)
-.132
Algebra 1 .871
(n = 365)
.874
(n = 293)
+.003
58
Table 12 reflects positive mean changes in all class levels at Harrison Park
High School, with the exception of Basic Algebra D, which had a change of -.132.
Overall, Essentials of Algebra D made the largest positive change with +.103,
Essentials of Algebra B with the second largest positive change at +0.099, Basic
Algebra B with the third largest positive change of +.064, and Algebra 1 making the
smallest amount of positive mean change of +.003 over the one year period. The
total mean difference of all class levels combined was -.0002, showing that the
standards based intervention in Algebra 1 at Harrison Park High School did not have
an impact on the percentage of students scoring basic and above on the Algebra 1
CST. Figure 3 shows a graph of the mean statistics summarized in Table 12. The
class levels have again been labeled with the numerical values assigned to them as
described previously.
59
Figure 3:
Estimated Marginal Means by Class Level: Basic and Above
0
0.2
0.4
0.6
0.8
1
012 34
2005 2005
2006 2006
Sequence
Estimated Marginal Means
Effect size estimates for the mean changes in percent of students scoring
basic and above by class level on the Algebra 1 CST are shown in Table 13. Using
effect size estimates, data are analyzed to demonstrate practical significance of the
academic progress made pre-intervention (2005) to post-intervention (2006). Again,
effect size was calculated by dividing the difference from pre-intervention (2005) to
post-intervention (2006) by the observed standard deviation of the pre-intervention
(2005) Algebra 1 CST classification.
60
Table 13:
Effect Size Estimates for Basic and Above
Essentials
of Algebra
B
Essentials
of Algebra
D
Basic
Algebra B
Basic
Algebra D
Algebra 1
Gain +.099 +.103 +.063 -.132 +.003
2005 SD .292 .313 .479 .492 .335
Effect Size +.339 +.329 +.132 -.268 +.009
***The effect size estimate was -.0004 for the total sample.
Table 13 shows that the effect size for Essentials of Algebra B and Essentials
of Algebra D were practically significant at +.339 and +.329, respectively. The
effect size for the other three classes was small (less than ± .3).
Percentage of Change Estimates of Students Scoring Basic and Above on the
Algebra 1 CST: 2005 vs. 2006
The percentage of change estimates of students scoring basic and
above on the Algebra 1 CST are reflected in Table 14. Percentage of change
estimates are a second measure used to demonstrate the practical significance of the
academic progress made pre-intervention (2005) to post-intervention (2006). Both
pre-intervention (2005) and post-intervention (2006) means are listed, as well as the
percentage of change from pre-intervention to post-intervention.
Table 14:
Percentage of Change by Class Level: Basic and Above
Essentials
of Algebra
B
Essentials
of Algebra
D
Basic
Algebra B
Basic
Algebra D
Algebra 1
Mean 2005 .094 .109 .353 .594 .871
Mean 2006 .193 .212 .416 .462 .874
Difference +.099 +.103 +.063 -.132 +.003
Percentage
of Change
+105.3% +94.5% +17.8% -22.2% +0.3%
61
Table 14 shows that the percent gains made for Essentials of Algebra B,
Essentials of Algebra D, and Basic Algebra B were practically significant at
+105.3%, +94.5%, and +17.8%, respectively. Again, there was a percent loss in
Basic Algebra D.
Effects on Algebra 1 CST Performance: Interaction between Class Level and
Performance
The following factorial Analysis of Variance (ANOVA) analyzes data
reflective of the statistical main effects and interaction between the five different
class levels at Harrison Park High School and the two years. Class Level has been
labeled as “sequence” in the factorial analysis. The result for the impact of the
interaction on mean score was significant, suggesting that a follow-up analysis for
each class level is necessary.
Table 15:
Tests of Between-Subjects Effects
Dependent Variable: Score
Source Type II
Sum of
Squares
df Mean
Square
F Sig.
Corrected
Model
Intercept
sequence
year
sequence*year
Error
Total
Corrected Total
812.155
a
5090.302
804.431
2.279
7.415
1374.543
7277.000
2186.698
9
1
4
1
4
2013
2023
2022
90.239
5090.302
201.108
2.279
1.854
.683
132.155
7454.681
294.520
3.338
2.715
.000
.000
.000
.068
.028
a. R Squared = .371 (Adjusted R Squared = .369)
62
Interaction between Class Level and Performance: Students Scoring Proficient
and Above
The following factorial Analysis of Variance (ANOVA) analyzes data
reflective of the statistical main effects and interaction between the five different
class levels at Harrison Park High School and the two years, this time focusing on
the percentage of students who scored at a level of proficient and advanced. Class
Level has been labeled as “sequence” in the factorial analysis. The result for the
impact of the interaction on proficiency level was significant in this analysis,
suggesting that a follow-up analysis for each class level is necessary.
Table 16:
Tests of Between-Subjects Effects
Dependent Variable: proficiency
Source Type II
Sum of
Squares
df Mean
Square
F Sig.
Corrected
Model
Intercept
sequence
year
sequence*year
Error
Total
Corrected Total
95.488
a
93.107
95.150
.588
.287
245.405
434.000
340.893
9
1
4
1
4
2013
2023
2022
10.610
93.107
23.787
.588
.072
.122
87.030
763.738
195.123
4.819
.588
.000
.000
.000
.028
.672
a. R Squared = .280 (Adjusted R Squared = .277)
Interaction between Class Level and Performance: Students Scoring Basic and
Above
The following factorial Analysis of Variance (ANOVA) analyzes data
reflective of the statistical main effects and interaction between the five different
63
class levels at Harrison Park High School and the two years, this time focusing on
the percentage of students who scored at a level of basic, proficient and advanced.
Class Level has been labeled as “sequence” in the factorial analysis. The result for
the impact of the interaction on the percent of students scoring at a level of basic and
above was not significant.
Table 17:
Tests of Between-Subjects Effects
Dependent Variable: basic
Source Type II
Sum of
Squares
df Mean
Square
F Sig.
Corrected
Model
Intercept
sequence
year
sequence*year
Error
Total
Corrected Total
164.900
a
525.438
161.547
.106
3.352
340.662
1031.000
505.562
9
1
4
1
4
2013
2023
2022
18.322
525.438
40.387
.106
.838
.169
108.267
3104.852
238.648
.625
4.952
.000
.000
.000
.429
.001
a. R Squared = .326 (Adjusted R Squared = .323)
Effects on Algebra 1 CST Performance 2005-2006: Control Group Findings
Table 18 shows the mean performance changes for all students tested on the
Algebra 1 CST for the control school, Ericson High School. All five class levels are
listed with a pre-intervention mean in 2005 and a post-intervention mean in 2006.
The last column reflects the amount of change from pre-intervention mean (2005) to
post-intervention mean (2006).
64
Table 18:
Mean Statistics by Grade Level: Control School Findings
Class Level 2005 2006 Difference
Essentials of
Algebra B
.81
(n = 47)
.73
(n = 73)
-.08
Essentials of
Algebra D
1.04
(n = 46)
.87
(n = 46)
-.17
Basic Algebra B 1.23
(n = 194)
1.36
(n = 256)
+.13
Basic Algebra D 1.39
(n = 109)
1.18
(n = 118)
-.21
Algebra 1 2.27
(n = 281)
2.28
(n = 257)
+.01
Total 1.65
(n = 677)
1.56
(n = 750)
-.09
Table 18 reflects positive mean changes for Ericson High School were made
in Basic Algebra B and Algebra 1. Overall, Basic Algebra B made the largest
positive change with +.13, with Algebra 1 making the smallest amount of positive
mean change of +.01 over the one year period. There were negative mean changes
made in Essentials of Algebra B, Essentials of Algebra D, and Basic Algebra D, with
changes of -.08, -.17, and -.21, respectively. The total mean difference of all class
levels combined results in -.09, which means that Ericson High School, which did
not receive the standards-based intervention in algebra, had losses in overall algebra
achievement as measured by the Algebra 1 CST. Figure 4 shows a graph of the
mean statistics by class level summarized in Table 18. The class levels have been
labeled with the numerical scale assigned to them as described in the introduction of
this chapter.
65
Figure 4:
Estimated Marginal Means by Class Level: Control School Findings
0
0.6
1.2
1.8
2.4
3
012 34
2005 2005
2006 2006
Sequence
Estimated Marginal Means
Effect size estimates for the mean changes by class level on the Algebra 1
CST for Ericson High School are shown in Table 19. Using effect size estimates,
data are analyzed to demonstrate practical significance of the academic progress
made pre-intervention (2005) to post-intervention (2006). Effect size was calculated
by dividing the difference from pre-intervention (2005) to post-intervention (2006)
by the observed standard deviation of the pre-intervention (2005) Algebra 1 CST
classification.
66
Table 19:
Effect Size Estimates: Control School Findings
Essentials
of Algebra
B
Essentials
of Algebra
D
Basic
Algebra B
Basic
Algebra D
Algebra 1
Gain -.08 -.17 +.13 -.21 +.01
2005 SD .613 .759 .670 .758 .801
Effect Size -.131 -.224 +.194 -.277 +.012
***The effect size estimate was -.098 for the total sample.
Table 19 shows that effect size for all class levels at Ericson High School was
small (less than ± .30).
Percentage of Change of Algebra 1 CST Means 2005 vs. 2006: Control School
Findings
The percentages of change of Algebra 1 CST means for Ericson High School
are reflected in Table 20. Percentage of change estimates are a second measure used
to demonstrate the practical significance of the academic progress made pre-
intervention (2005) to post-intervention (2006). Both pre-intervention (2005) and
post-intervention (2006) means are listed, as well as the percentage of change from
pre-intervention to post-intervention.
Table 20:
Percentage of Change of Mean Score by Class Level: Control School Findings
Essentials
of Algebra
B
Essentials
of Algebra
D
Basic
Algebra B
Basic
Algebra D
Algebra 1
Mean 2005 .81 1.04 1.23 1.39 2.27
Mean 2006 .73 .87 1.36 1.18 2.28
Difference -.08 -.17 +.13 -.21 +.01
Percentage
of Change
-9.9% -16.3% +2.4% -15.1% +0.4%
67
Table 20 shows that the percent gains made for Basic Algebra B and Algebra
1 were not practically significant, with gains of +2.4% and +0.4%, respectively.
There were percent losses in Essentials of Algebra B, Essentials of Algebra D, and
Basic Algebra D.
Control School Findings for Effects on Algebra 1 CST Performance: Proficient
and Above
Mean changes for the percentage of students who scored at the proficient and
advanced levels for each class level at Ericson High School on the Algebra 1 CST
are shown in Table 21. According to No Child Left Behind (NCLB), students who
are considered meeting the guidelines of proficiency achieve at the proficient and
advanced levels on the CST. Again, all five class levels are listed with a pre-
intervention mean in 2005 and a post-intervention mean in 2006 at Ericson High
School. The last column reflects the amount of change from pre-intervention mean
(2005) to post-intervention mean (2006).
Table 21:
Control School Mean Statistics by Class Level: Proficient and Above
Class Level 2005 2006 Difference
Essentials of
Algebra B
.000
(n = 47)
.000
(n = 73)
.000
Essentials of
Algebra D
.022
(n = 46)
.000
(n = 46)
-.022
Basic Algebra B .021
(n = 194)
.051
(n = 256)
+.030
Basic Algebra D .046
(n = 109)
.059
(n = 118)
+.013
Algebra 1 .420
(n = 281)
.463
(n = 257)
+.043
68
Table 21 reflects positive mean changes at Ericson High School for Basic
Algebra B, Basic Algebra D, and Algebra 1. Overall, Algebra 1 made the largest
positive change with +.043, Basic Algebra B with the second largest positive change
at +.030, and Basic Algebra D making the smallest amount of positive mean change
of +.013 over the one year period. Essentials of Algebra B made no change, and
Essentials of Algebra D had a negative mean change of -.022. The total mean
difference of all class levels combined was -.004, showing that at Ericson High
School, which did not receive the standards based intervention in Algebra 1, had
overall losses in the percentage of students scoring proficient and above on the
Algebra 1 CST. Figure 5 shows a graph of the mean statistics summarized in Table
21. The class levels have again been labeled with the numerical values assigned to
them as described previously.
69
Figure 5:
Control School Estimated Marginal Means by Class Level: Proficient and
Above
0
0.2
0.4
0.6
0.8
1
012 34
2005 2005
2006 2006
Class Level
Estimated Marginal Means
Effect size estimates for the mean changes in proficiency by class
level on the Algebra 1 CST for Ericson High School are shown in Table 22.
Using effect size estimates, data are analyzed to demonstrate practical
significance of the academic progress made pre-intervention (2005) to post-
intervention (2006). Again, effect size was calculated by dividing the
difference from pre-intervention (2005) to post-intervention (2006) by the
observed standard deviation of the pre-intervention (2005) Algebra 1 CST
classification.
70
Table 22:
Effect Size Estimates for Proficiency: Control School Findings
Essentials
of Algebra
B
Essentials
of Algebra
D
Basic
Algebra B
Basic
Algebra D
Algebra 1
Gain +.000 -.022 +.030 +.013 +.043
2005 SD .000 .147 .142 .210 .494
Effect Size NA -.150 +.211 +.062 +.087
***The effect size estimate was -.010 for the total sample.
Table 22 shows that the effect size for all class levels at Ericson High School
was small (less than ± .30). Effect size for Essentials of Algebra B was not
calculated due to the .000 standard deviation in 2005 for this class.
Percentage of Change Estimates of Algebra 1 CST Proficiency 2005 vs. 2006:
Control School Findings
The percentages of change estimates of Algebra 1 CST proficiency for
Ericson High School are reflected in Table 23. Percentage of change estimates are a
second measure used to demonstrate the practical significance of the academic
progress made pre-intervention (2005) to post-intervention (2006). Both pre-
intervention (2005) and post-intervention (2006) means are listed, as well as the
percentage of change from pre-intervention to post-intervention.
Table 23:
Control School Percentage of Change by Class Level: Proficient and Above
Essentials
of Algebra
B
Essentials
of Algebra
D
Basic
Algebra B
Basic
Algebra D
Algebra 1
Mean 2005 .000 .022 .021 .046 .420
Mean 2006 .000 .000 .051 .059 .463
Difference .000 -.022 +.030 +.013 +.043
Percentage
of Change
NA -100.0% +142.9% +28.3% +10.2%
71
Table 23 shows that the percent gains made for Basic Algebra B, Basic
Algebra D, and Algebra 1 were practically significant at +142.9%, +28.3%, and
+10.2%, respectively. There was a percent loss for Essentials of Algebra D.
Percentage of change was not calculated for Essentials of Algebra B due to the .000
means in 2005 for this class.
Control School Findings for Effects on Algebra 1 CST Performance: Basic and
Above
According to NCLB, a student is considered “proficient” in the subject area if
they score a level of proficient or advanced on the CST. However, it is also useful to
look at the number of students who score at the basic, proficient, and advanced levels
in analyzing student achievement. Mean changes for the percentage of students who
scored basic and above on the Algebra 1 CST for Ericson High School are shown in
Table 24. Again, all five class levels are listed with a pre-intervention mean in 2005
and a post-intervention mean in 2006. The last column reflects the amount of change
from pre-intervention mean (2005) to post-intervention mean (2006).
Table 24:
Control School Mean Statistics by Class Level: Basic and Above
Class Level 2005 2006 Difference
Essentials of
Algebra B
.106
(n = 47)
.069
(n = 73)
-.037
Essentials of
Algebra D
.261
(n = 46)
.196
(n = 46)
-.065
Basic Algebra B .325
(n = 194)
.422
(n = 256)
+.097
Basic Algebra D .468
(n = 109)
.254
(n = 118)
-.214
Algebra 1 .858
(n = 281)
.817
(n = 257)
-.041
72
Table 24 reflects a positive mean change for Basic Algebra D, with a mean
change of +.097. There was a negative mean change for the four remaining classes.
The total mean difference of all class levels combined was -.067, showing that
Ericson High School, which did not receive the standards based intervention in
Algebra 1, had a decrease in the percentage of students scoring basic and above on
the Algebra 1 CST. Figure 6 shows a graph of the mean statistics summarized in
Table 24. The class levels have again been labeled with the numerical values
assigned to them as described previously.
Figure 6:
Control School Estimated Marginal Means by Class Level: Basic and Above
0
0.2
0.4
0.6
0.8
1
012 34
2005 2005
2006 2006
Class Level
Estimated Marginal Means
Effect size estimates for the mean changes in percent of students scoring
basic and above by class level on the Algebra 1 CST at Ericson High School are
shown in Table 25. Using effect size estimates, data are analyzed to demonstrate
73
practical significance of the academic progress made pre-intervention (2005) to post-
intervention (2006). Again, effect size was calculated by dividing the difference
from pre-intervention (2005) to post-intervention (2006) by the observed standard
deviation of the pre-intervention (2005) Algebra 1 CST classification.
Table 25:
Effect Size Estimates for Basic and Above: Control School Findings
Essentials
of Algebra
B
Essentials
of Algebra
D
Basic
Algebra B
Basic
Algebra D
Algebra 1
Gain -.037 -.065 +.097 -.214 -.041
2005 SD .312 .444 .469 .501 .350
Effect Size -.119 -.146 +.207 -.427 -.117
***The effect size estimate was -.134 for the total sample.
Table 25 shows that the effect size for Basic Algebra D was practically
significant at -.427. The effect size for the other four classes was small (less than
± .30).
Control School Percentage of Change Estimates of Students Scoring Basic and
Above on the Algebra 1 CST: 2005 vs. 2006
The percentage of change estimates of students scoring basic and
above on the Algebra 1 CST for Ericson High School are reflected in Table 26.
Percentage of change estimates are a second measure used to demonstrate the
practical significance of the academic progress made pre-intervention (2005) to post-
intervention (2006). Both pre-intervention (2005) and post-intervention (2006)
means are listed, as well as the percentage of change from pre-intervention to post-
intervention.
74
Table 26:
Percentage of Change by Class Level: Basic and Above
Essentials
of Algebra
B
Essentials
of Algebra
D
Basic
Algebra B
Basic
Algebra D
Algebra 1
Mean 2005 .106 .261 .325 .468 .858
Mean 2006 .069 .196 .422 .254 .817
Difference -.037 -.065 +.097 -.214 -.041
Percentage
of Change
-34.9% -24.9% +29.8% -45.7% -4.8%
Table 26 shows that the percent gain made for Basic Algebra B was
practically significant at +29.8%. There were percent losses in the remaining four
courses.
Control School Effects on Algebra 1 CST Performance: Interaction between
Class Level and Performance
The following factorial Analysis of Variance (ANOVA) analyzes data
reflective of the statistical main effects and interaction between the five different
class levels at Ericson High School and the two years. Class Level has been labeled
as “sequence” in the factorial analysis. The result for the impact of the interaction on
mean score was significant, suggesting that a follow-up analysis for each class level
is necessary.
75
Table 27:
Tests of Between-Subjects Effects: Control School Findings
Dependent Variable: Score
Source Type II
Sum of
Squares
df Mean
Square
F Sig.
Corrected
Model
Intercept
sequence
year
sequence*year
Error
Total
Corrected Total
435.442
a
3649.281
427.057
.023
5.440
805.277
4890.000
1240.719
9
1
4
1
4
1417
1427
1426
48.382
3649.281
106.764
.023
1.360
.568
85.136
6421.429
187.867
.040
2.393
.000
.000
.000
.841
.049
a. R Squared = .351 (Adjusted R Squared = .347)
Control School Interaction between Class Level and Performance: Students
Scoring Proficient and Above
The following factorial Analysis of Variance (ANOVA) analyzes data
reflective of the statistical main effects and interaction between the five different
class levels at Ericson High School and the two years, this time focusing on the
percentage of students who scored at a level of proficient and advanced. Class Level
has been labeled as “sequence” in the factorial analysis. The result for the impact of
the interaction on proficiency level was not significant in this analysis.
76
Table 28:
Tests of Between-Subjects Effects: Control School Findings
Dependent Variable: proficiency
Source Type II
Sum of
Squares
df Mean
Square
F Sig.
Corrected
Model
Intercept
sequence
year
sequence*year
Error
Total
Corrected Total
56.104
a
49.957
55.978
.249
.122
160.938
267.000
217.043
9
1
4
1
4
1417
1427
1426
6.234
49.957
13.994
.249
.030
.114
54.887
439.855
123.216
2.194
.268
.000
.000
.000
.139
.899
a. R Squared = .258 (Adjusted R Squared = .254)
Control School Interaction between Class Level and Performance: Students
Scoring Basic and Above
The following factorial Analysis of Variance (ANOVA) analyzes data
reflective of the statistical main effects and interaction between the five different
class levels at Ericson High School and the two years, this time focusing on the
percentage of students who scored at a level of basic, proficient and advanced. Class
Level has been labeled as “sequence” in the factorial analysis. The result for the
impact of the interaction on the percent of students scoring at a level of basic and
above was significant, suggesting that a follow-up analysis for each class level is
necessary.
77
Table 29:
Tests of Between-Subjects Effects: Control School Findings
Dependent Variable: basic
Source Type II
Sum of
Squares
df Mean
Square
F Sig.
Corrected
Model
Intercept
sequence
year
sequence*year
Error
Total
Corrected Total
104.021
a
377.544
98.697
.251
3.736
252.434
734.000
356.456
9
1
4
1
4
1417
1427
1426
11.558
377.544
24.674
.251
.934
.178
64.879
2119.287
138.504
1.409
5.243
.000
.000
.000
.235
.000
a. R Squared = .292 (Adjusted R Squared = .287)
Control versus Experimental School Comparisons
A series of year by school 2x2 ANOVAs were conducted for each of the
three class levels and each of the three outcomes. The interaction was used to test
whether the change from 2005 to 2006 was different between schools. The results in
Table 30 show that for the most part, the changes from 2005 to 2006 were not
different. The one exception is at the lower class levels on both the performance
band scores and the percentage basic and above. For Class Level 0 at HPHS,
performance band scores increased by .11. In contrast, there was a decrease of .08 at
EHS. Consistent with this finding, the increase in the percentage of Class Level 0
students at HPHS scoring basic and above was .10 in contrast to a .04 decrease at
EHS. The results for Class Level 1 paralleled these findings. For Class Level 1 at
HPHS, performance band scores increased by .18 in contrast to a decrease of .17 at
EHS. Consistent with this finding, the increase in the percentage of Class Level 1
78
students at HPHS scoring basic and above was .10 in contrast to a .06 decrease at
EHS.
Table 30:
Experimental versus Control: Change from 2005 to 2006
Performance Score Proficient and Above Basic and Above
F probability F probability F probability
0 1.64 .201 3.60 .058
1 3.88 .050 3.24 .073
2 .01 .921 .02 .893 .27 .605
3 .09 .759 .21 .643 1.01 .315
4 .04 .848 .04 .848 1.11 .292
79
CHAPTER 5
SUMMARY, DISCUSSION, AND RECOMMENDATIONS
Overview
A standards-based intervention in Algebra 1 at Harrison Park High School
(name has been changed) is evaluated in this study in terms of its impact on student
achievement on the California Standards Test (CST) in Algebra 1. The implications
of the quantitative findings will be discussed, as well as corresponding conclusions
from the qualitative data gathered in the study. Based on these conclusions,
recommendations for the algebra program at the experimental school will be
discussed, as well as recommendations for further study.
Summary of Findings
The purpose of this study was to evaluate the impact of a standards-based
intervention in Algebra 1 at Harrison Park High School on student academic
achievement as measured by the California Standards Test (CST). The percentage of
change of student performance from pre-intervention (2005) to post-intervention
(2006) was analyzed for the experimental school for each class level in three ways;
the percentage of change of mean score, the percentage of change of the percent of
students scoring proficient and advanced, and the percentage of change of the
percent of students scoring basic, proficient, and advanced. Furthermore, the
interaction between student achievement and the five class levels of Algebra was
analyzed. Finally, the percentage of change of student performance from pre-
intervention (2005) to post-intervention (2006) was analyzed in the same way for the
80
control school, Ericson High School (name has been changed), as well as the
interaction between student achievement at the five class levels of Algebra at this
school.
The primary participants of this study were the student population of
Harrison Park High School enrolled in one of the five levels of Algebra (997
students). These class levels are Essentials of Algebra B, Essentials of Algebra D,
Basic Algebra B, Basic Algebra D, and Algebra 1. School demographic data show
that in the school year 2005-2006, the student population was primarily White (67%
of total enrollment), followed by a significant number of Hispanic and Asian
students (14% and 10% of total enrollment, respectively).
A second group of participants in the study was the Algebra teachers of
Harrison Park High School, as well as the administrator in charge of overseeing the
math department. In the year of the study, there were eight certificated teachers
teaching the five class levels of Algebra. Each of these participants agreed to be
interviewed to share their beliefs about the implementation of the standards-based
intervention and its impact on student achievement at Harrison Park High School.
All eight teachers were interviewed, as well as the administrator. The results of the
qualitative data will be discussed in the findings section of this chapter for the
experimental school, including the findings regarding the interaction of class level
and student achievement. Fifteen interviews and sixteen observations were
conducted at Harrison Park High School during the intervention. Document analysis
was conducted to verify the extent to which all Algebra teachers were teaching a
81
common, standards-based Algebra curriculum to their students for each class level
offered. The primary documents utilized were the California State Content
Standards for Mathematics, curriculum guides and maps, and common assessment
tools used by Algebra teachers. The qualitative findings will serve to support the
conclusions reached utilizing the quantitative data regarding the impact of the
standards-based intervention in Algebra, and will contribute to the implications for
further study and recommendations for improvement at HPHS.
Summary of Quantitative Findings: Harrison Park High School
This section will discuss evidence that will serve to answer the research
question this study was designed to answer: Does a standards-based intervention in
Algebra have an impact on student achievement in Algebra as measured by the
California Standards Test (CST) in Algebra 1? Overall, the intervention had a
positive impact on the achievement test scores in Algebra 1 at Harrison Park High
School. There was an increase in the schools overall Academic Performance Index
(API) from 765 points pre-intervention (2005) to 783 points post-intervention
(2006), and the school met its growth target for each subgroup of the student
population.
When reviewing the findings by class level for the experimental school,
Harrison Park High School, positive changes were made for all class levels with the
exception of Basic Algebra D, which had a loss of .18 in mean score. Although the
increase in mean score for the remaining class levels was modest, increases were
also made in these classes in the percent of students performing at the proficient and
82
advanced level, as well as the percent of students performing at the basic, proficient,
and advanced levels. These consistent increases can be considered in part the reason
for Harrison Park High School’s increased API score and achievement of its growth
targets.
Findings for Essentials of Algebra B students show an overall positive mean
change in score of +.11. Although this change seems small, it is important to note
that the percent of change of mean score for this class was +14.7%, which is
practically significant. Furthermore, the percent of students performing at the
proficient and advanced levels was increased by +4.6%. The percent of students
performing at the basic, proficient, and advanced levels was increased by +9.9%.
The latter is a percent of change of 105.3%, which is practically significant. This
indicates that students in the lowest class level, Essentials of Algebra B, are moving
toward higher levels of proficiency.
Essentials of Algebra D findings also show an overall positive mean change
in score of +.18, which was the largest mean change of the five class levels. The
percent of change of mean score for this class level was +23.4%, which is again
practically significant. The percent of students performing at the proficient and
advanced levels was increased by +5.3%. The percent of students performing at the
basic, proficient, and advanced levels was increased by +10.3%. The latter is a
percent of change of +94.5%, which is practically significant. Again, this indicates
that students in the class level Essentials of Algebra D are moving toward higher
levels of proficiency.
83
Based on Basic Algebra B findings, Basic Algebra B had an overall positive
mean change of +.12. Again, although this change seems small, it is important to
note that the percent of change of mean score for this class level was +9.8%, which
is practically significant. Furthermore, the percent of students performing at the
proficient and advanced levels was increased by +2.6%, which is a percent of change
of +44.8%, making it practically significant. The percent of students performing at
the basic, proficient, and advanced levels was increased by +6.4%, which is a percent
of change of +17.8%, again making it practically significant. This indicates that
students in the class level Basic Algebra B are moving towards higher levels of
proficiency.
The findings for the Basic Algebra D class level show that this was the only
class level that had a negative change in mean score. The overall change was -0.18,
which is a percent of change of -11.2%. The percent of students performing at the
proficient and advanced levels was decreased by -0.9%, which is a percent of change
of -7.5%. The percent of students performing at the basic, proficient, and advanced
levels was decreased by -13.2%, which is a percent of change of -22.2%. This
indicates that students in the class level, Basic Algebra D, are not moving towards
higher levels of proficiency.
Algebra 1 findings show that this class level had an overall positive mean
change of +.11. The percent of change of mean score for this class level was +4.6%,
which is not practically significant. However, it is important to note that the percent
of students performing at the proficient and advanced levels was increased by
84
+5.4%, which is a percent of change of +10.8%, making it practically significant.
The percent of students performing at the basic, proficient, and advanced levels was
increased by +0.3%, which is a percent of change of +0.3%. This indicates that
students in the class level Algebra 1 are moving towards higher levels of proficiency.
Summary of Qualitative Findings: Harrison Park High School
In addition to the quantitative findings discussed above, it is important to
discuss the qualitative findings. The essential formative research questions which
this data seek to answer are:
1) How is standards-based instruction implemented in mathematics
classrooms at HPHS, and
2) Does standards-based instruction improve student mastery of
algebra concepts?
Out of the fifteen interviews and sixteen observations, teachers and
administrators agreed that there was uniformity in the way that the Algebra 1
standards-based intervention was implemented, with the one exception of the
frequency with which each teacher utilized the standards-based computer program,
Riverdeep. The computer program was utilized most often (on average twice a
week) by teachers of the lower class levels, Essentials of Algebra B and Essentials of
Algebra D, and the least often (on average once a month) by teachers of the basic
classes, Basic Algebra B and Basic Algebra D. Other than this, all Algebra teachers
implemented the standards-based program uniformly following the same sequence in
each class level, gave the daily warm-up exercises, emphasized the standard taught,
85
and gave the commonly developed assessments. This uniformity was confirmed in
every interview and observation. One teacher interviewed commented, “the
consistency of the curriculum across the classrooms for each class level made it
extremely easy for students who had to switch classes or teachers in the middle of
the semester due to scheduling conflicts involving athletics or class changes.” This
uniformity therefore guaranteed equal access to the curriculum for students in each
class level, regardless of their teacher.
Additional qualitative findings at the experimental school found that by
emphasizing the standard being taught in each lesson, and giving a daily warm-up on
that standard by using previously-released questions from past Algebra 1 CST tests,
the students became more aware of state expectations, and expressed more
confidence in their performance on the CST test. An Algebra 1 teacher commented,
“I had a number of students who had traditionally had test anxiety actively
volunteering to explain test questions to the class during class warm-ups. I could see
their confidence building with familiarity with the standards and the constant
reinforcement of those standards.” Teachers and administrators unanimously
expressed that they saw an increased awareness of California state standards in
Algebra 1, and an increased awareness of the connection between what was being
taught in the classroom and what was being tested on the Algebra 1 CST test.
Weaknesses expressed by the staff generally related to the disadvantage that
the students enrolled in Essentials of Algebra B and Basic Algebra B faced when
taking the Algebra 1 CST due to the fact that they had only learned half of the
86
standards on which they were being tested. This interaction between class level and
mean performance on the Algebra 1 CST will be discussed further in the next
section. This sparked a discussion about the curriculum at the experimental school,
and the practicality of having two-year Algebra programs given the fact that students
in the first year of the program would still be required by the state to take the
Algebra 1 CST. Furthermore, teachers expressed dissatisfaction with having five
different class levels of Algebra, and discussion about collapsing this into three
levels was discussed. However, there was overall satisfaction in the standards-based
curriculum designed and implemented during the study for teachers of each class
level, and teachers expressed a desire to expand this intervention to the mathematics
subject areas. Teachers have already begun working on creating uniform curriculum
guides for Geometry and Algebra II.
Summary of Findings: Interaction between Class Level and Performance
In reviewing the findings of the study, the changes in mean performance of
the experimental school must be discussed in terms of the interaction between the
class level a student was enrolled in, and their performance on the Algebra 1 CST.
Based on the Analysis of Variance (ANOVA), the results shown in Table 15 indicate
that the interaction sequence by year was significant when the mean score was the
dependent variable, as evidenced by the observed probability of p. = .028. This
implies a moderating of class level on achievement gains as a factor in the
experimental school’s findings for mean score. The results shown in Table 16
indicate that the same interaction was not significant when proficiency was the
87
dependent variable, as evidenced by the observed probability of p = .672. However,
it is important to note that the results in Table 17 indicate that the interaction
between sequence and year was significant when basic was the dependent variable,
as evidenced by the observed probability of p = .001. This implies significance of
class level as a factor in the experimental school’s findings for the percent of
students performing at the basic, proficient, and advanced levels.
Implications
During the study, interesting results began to emerge across all class levels at
the experimental school, Harrison Park High School in terms of achievement in
Algebra 1. The two lowest class levels, Essentials of Algebra B and Essentials of
Algebra D, made the largest increases in both percent of change of mean score and
percent of change of the students performing at the basic, proficient, and advanced
levels, while the highest class level, Algebra 1, made the smallest increase in these
areas. Based on teacher interviews and observations, this result could be the result of
the fact that the Essentials of Algebra students scored so low in 2005, that any
amount of change would be considered a significant percent increase. By contrast,
Algebra 1 students had higher mean scores in 2005, so it would take a much larger
increase for the percent of change to be significant.
Another surprising result was that Basic Algebra D had an overall decrease in
mean score, percent of students performing at the proficient and advanced levels, and
percent of students performing at the basic, proficient, and advanced levels.
Teachers at the experimental school were unsure as to the reason behind this result.
88
One common interpretation was that these students have learned the Algebra 1
standards over a two year period, so perhaps they were weak on the standards that
they learned in the first year since it had been so long since they had been exposed to
the standards. One solution may be to include a strong review component in the
Basic Algebra D curriculum in the future to ensure retention. This will be discussed
further in the recommendations section of this chapter.
It is important to emphasize the statistical significance of the interaction
between class level and performance on the Algebra 1 CST test. This interaction
between class level and mean score had an observed probability of .028, and the
interaction between class level and percent of students performing at the basic,
proficient, and advanced levels had an observed probability of .001. This indicates
that the class level a student was enrolled on had an impact on the student’s
achievement on the CST. As Marzano (2003) states, “Opportunity to learn (OTL)
has the strongest relationship with student achievement of all school-level factors
identified… (p. 22).” The students at Harrison Park High School that are enrolled in
the first year of the two-year Algebra course do not have the opportunity to learn all
of the Algebra content standards that they will be tested on. In essence, this is a form
of “tracking” in which lower level students are grouped together, and the curriculum
is presented to them in a different sequence and using different instructional
strategies. Schmidt (2004) discusses this common use of tracking in mathematics
classes in the United States.
89
Tracking is another practice in the United States that changes the curriculum
for different students. Practiced widely in the United States during the middle
grades but especially in the 7th and 8th grades, content tracking is clearly not
consistent with the vision of NCLB. Tracking sorts students by various
criteria into different courses so extensively that a given school can have
more than five different variations of 8th grade mathematics (Schmidt, 2004,
p. 7)
This is truly the case at the experimental school, where five class levels of Algebra
are offered, each offering the curriculum in a different sequence and at a different
depth.
When viewing the overall results, Harrison Park High School experienced
positive academic achievement changes at all class levels except Basic Algebra D.
The practical significance lies in the evidence that most students are moving closer to
the level of proficiency on the Algebra 1 CST test. The positive results may be
attributed to using the uniformity with which the standards-based intervention in
Algebra was implemented in this study. In discussing mathematics standards-based
reform, Schoenfeld (2002) states, “The bottom line is that standards-based reform
appears to work when it is implemented as part of a coherent systemic effort in
which curriculum, assessment, and professional development are aligned (p. 17).”
Site-Based Recommendations
Based on this study in Algebra 1, the first site-based recommendation for
Harrison Park High School is to expand the standards-based intervention to all other
levels of mathematics taught. At the conclusion of this study, teachers had already
begun working on developing common curricular maps for the Geometry and
90
Algebra 2 courses. The implications of this study show that the standards-based
intervention increased levels of proficiency for most students, so expanding
standards-based instruction to all class levels should lead to an increased in the
school’s overall performance in mathematics, thus leading to further increased in
API scores. The change should be done incrementally so as not to seem
overwhelming to teachers. As Marzano (2003) explains, “…administrators and
classroom teachers are often overwhelmed by the sheer amount of change attempted
and the work involved (p. 159).” By implementing standards-based instruction one
class level at a time, teachers will be given a chance to review data as to the success
of the implementation, thus providing further evidence that standards-based
instruction in mathematics should be expanded at the site-level.
A second recommendation for the site level involves taking a look at the
current Algebra curriculum, which offers Algebra instruction at five different levels.
This study showed that the class level a student was enrolled in had an impact on
their achievement on the CST test. For this reason, it is important to provide every
student with an opportunity to learn the same standards in the same amount of time.
Schmidt (2004) states, “The content standards must be the same for everyone—a
standard curriculum for all students within a state (p. 6).” Perhaps, eliminating one
of the two-year course sequences, and placing a larger number of students in the one
year Algebra 1 course would be a good start. Furthermore, the rationale behind a
two-year sequence given the current state expectations for Algebra 1 should be
studied and discussed, and recommendations should be made for a viable way to
91
present the Algebra 1 standards to the lower level students in a single year, whether
it be to provide additional support electives or condense the essential standards even
further to focus on those that are most often tested. As Schmidt (2004) discusses,
Individual districts or even schools can direct everything from curriculum to
tracking policies—such as creating special programs for gifted students in
middle school mathematics—and can even "adjust" the content standards for
students who come to school disadvantaged by such factors as poverty or
limited English language skills (p. 6).
One final site-based recommendation is that in the case that the two year
Basic Algebra program remains a part of the school’s curriculum, it must be
analyzed as to why the level of achievement Basic Algebra D earned decreased in
the year of the standards-based intervention. Algebra teachers should continue
standards-based instruction and look at data from next year’s Algebra 1 CST test to
determine whether this loss was just unique to this one year’s group of students or
whether there is an underlying weakness in the delivery of the Basic Algebra D two-
year curriculum. If this trend continues, interventions should be made to increase
professional development and strengthen teacher instructional strategies at this class
level to ensure that these students are receiving a strong standards-based curriculum.
Elmore and Fuhrman (2001) support this conclusion that teachers must be willing to
make changes in their current classroom practices based on past student
performance.
92
Responding to external performance-based accountability systems is not
simply a matter of reorienting existing teaching methods and organizational
routines toward new pursuits. Virtually all schools, no matter what their
demographic characteristics or prior performance, must do different things,
not just the same things differently (Elmore and Fuhrman, 2001, p. 69).
Recommendations for Further Study
The present study focused on an evaluation of the impact of a standards-
based intervention on the academic achievement of Algebra students. One of the
primary findings was that there was a strong interaction between a student’s class
level and their achievement on the Algebra 1 CST. The effectiveness of two year
Algebra programs for high school students should be studied further. Furthermore,
the question of whether tracking students into different levels of Algebra is actually
helping to increase academic achievement.
A second recommendation for further study is that grade level characteristics
should be analyzed as another factor that influences achievement. A student who has
traditionally moved more slowly through the levels of math, and is therefore older
than his or her peers, may have low self-efficacy in mathematics in general, and this
may lead to lower academic achievement, whereas a student who has been in an
accelerated track has high self-efficacy which may lead to higher academic
achievement. In other words, a ninth grader enrolled in Algebra 1 may have higher
achievement than an eleventh grader since they have been able to move through
previous levels of mathematics more quickly, causing higher levels of efficacy in
their ability to be successful. Marzano (2003) states, “…how students perceive the
causes of their prior successes and failures is a better determinant of motivation and
93
persistence than is learned success or failure avoidance orientation (p. 146).” Future
research should study the impact that grade level has on Algebra 1 academic
achievement.
One final recommendation for further study that presents itself as a result of
this study is the impact of the individual teacher on the academic achievement of
Algebra students. Teacher instructional strategies may vary from classroom to
classroom, and this may have an impact on student academic achievement in
Algebra. Marzano (2003) emphasizes the tremendous impact that classroom
teachers have on the academic achievement of their students. “…it is clear that
effective teachers have a profound influence on student achievement and ineffective
teachers do not. In fact, ineffective teachers might actually impede the learning of
their students (Marzano, 2003, p. 75).” Future study should analyze this impact on
Algebra 1 academic achievement.
Limitations
Due to the limitations of this study, caution is exercised in the generalization
of results. An internal limitation was the use of a pre-post nonequivalent control
group design. This limits the sampling variability which hinders the true measure of
the amount of change exhibited by each class level. For instance, the observation of
an Algebra 1 class in the pre-intervention (2005) school year is not the same Algebra
1 class that is observed in the post-intervention (2006) school year, due to the fact
that the 2005 students have moved up to the next level of mathematics. Therefore,
94
academic achievement could have been altered by the disproportionate numbers of
low performing and high performing students enrolled in each class level for each
year due to the effect of sampling variability.
Conclusion
This study has revealed the impact of a standards based intervention on the
academic achievement of Algebra students. Given the results of the findings, the
overall impact of the intervention was positive, and there was a clear correlation
between the class level of Algebra the student was enrolled in and academic
achievement as measured by the Algebra 1 California Standards Test (CST). This
study presents a resource to assist in moving Algebra students closer to proficiency.
As a result, additional research must be pursued to determine additional factors that
impact academic achievement in Algebra, as well as methods of increasing equity in
achieving proficiency across all class levels of Algebra.
95
BIBLIOGRAPHY
Betts, J.R. & Costrell, R.M. (2001). Incentives and equity under standards-based
reform. Brookings Papers on Education Policy, 9-74.
Betts, J.R. & Danenberg, A. (2002). School accountability in California: An early
evaluation. Brookings Papers on Education Policy, 123-197.
California Department of Education (2005). 2005 California Standards Test Scores.
Retrieved September 12, 2006, from
http://star.cde.ca.gov/star2005/viewreport.asp.
California Department of Education (2005). 2005 Accountability Progress Report.
Retrieved September 12, 2006, from
http://api.cde.ca.gov/AcntRpt2006/2005BaseSch.aspx?allcds=306654830329
43.
California Department of Education (2006). 2006 California Standards Test Scores.
Retrieved October 19, 2006, from
http://star.cde.ca.gov/star2006/viewreport.asp.
California Department of Education (2006). 2006 Accountability Progress Report.
Retrieved October 19, 2006 from
http://api.cde.ca.gov/AnctRpt2007/2006BaseSch.aspx?allcds=306654830329
43.
EdSource (2005, June). The state’s official measures of school performance.
Mountain View, CA: EdSource, Inc.
Elmore, R.F. & Fuhrman, S.H. (2001). Holding schools accountable: Is it working?
Phi Delta Kappan, 83(1), 67-72.
Finn, C.E., Jr. & Kanstoroom, M. (2001). State academic standards. Brookings
Papers on Educational Policy, 131-179.
Goertz, M.E. (2001). Redefining government roles in an era of standards-based
reform. Phi Delta Kappan, 83(1), 62-66.
Lauer, P.A., Snow, D., Martin-Glenn, M., Van Buhler, R.J., Stoutemyer, K., &
Snow-Renner, R. (2005). The influence of standards on K-12 teaching and
student learning: A research synthesis. Regional Educational Laboratory
(Contract #ED-01-CO-0006). McREL.
96
Marzano, R.J. (2003). What works in schools: Translating research into action.
Alexandria, VA: Association for Supervision and Curriculum Development.
National Commission on Excellence in Education (1983). A Nation at Risk.
Retrieved September 12, 2006, from
http://www.ed.gov/pubs/NatAtRisk/risk.html.
Schmidt, W.H. (2004). A vision for mathematics. Educational Leadership, 61(5), 6-
11.
Schoenfeld, A.H. (2002, January/February). Making mathematics work for all
children: Issues of standards, testing, and equity. Educational Researcher
31(1), 13-25.
United States Code (2002). Public Law 107-110, also know as the No Child Left
Behind Act of 2001.
United States Department of Education (1998). Third international mathematics and
science study (TIMSS). Retrieved June 18, 2007, from
http://nces.ed.gov/timss/results.asp.
Abstract (if available)
Abstract
The purpose of this study is to implement and evaluate a standards-based intervention to improve math proficiency in Algebra 1 at Harrison Park High School (name has been changed). To raise mathematics achievement at the experimental school, the standards-based intervention was implemented for the school year 2005-2006. The study was designed as pre-post nonequivalent control group design, with the intervention taking place in several phases, and as post only quasi experiment with a nearby school, Ericson High School (name has been changed), serving as a control. The study was designed as a combination of both summative and formative evaluation techniques. By analyzing the test data, the strengths and weaknesses of the Algebra program were analyzed in an attempt to improve the current program in subsequent years. Analyzing student achievement on the Algebra 1 Content Standards Test (CST) directed further improvements in school curriculum and classroom instructional delivery, leading to an alignment between state expectations and academic achievement.
Linked assets
University of Southern California Dissertations and Theses
Conceptually similar
PDF
An evaluation of the impact of a standards-based intervention on the academic achievement of English language learners
PDF
An evaluation of the impact of direct instruction intervention on the academic achievement of English language learners
PDF
The effect of a language arts intervention on the academic achievement of English language learners
PDF
The effects of culturally responsive standards based instruction on African American student achievement
PDF
The effect of site support teams on student achievement in seven northern California schools
PDF
Alternatives for achievement: a mathematics intervention for English learners
PDF
The impact of restructuring the language arts intervention program and its effect on the academic achievement of English language learners
PDF
Examining the effectiveness of the intervention programs for English learners at MFC intermediate school
PDF
Closing the science achievement gap for ninth grade English learners through standards- and inquiry-based science instruction
PDF
The teacher evaluation: key components that positively impact teacher effectiveness and student achievement
PDF
The block schedule and the English language learners: impact on academic performance and graduation rate at Oxbow High School
PDF
The impact of professional learning communities and block scheduling on English language learner students at Oak Point Middle School
PDF
In the implementation of standards-based reform: what is the leadership role of the principal in building school capacity and accountability to sustain student academic growth?
PDF
Raising student achievement at Eberman Elementary School with effective teaching strategies
PDF
Factors including student engagement impacting student achievement in a high performing urban high school district: a case study
PDF
English learners' performance on the California Standards Test at Aviles Elementary
PDF
Quantifying student growth: analysis of the validity of applying growth modeling to the California Standards Test
PDF
Examining the effectiveness of teacher training on the improvement of California standardized test scores at Eva B. Elementary School
PDF
The impact and sustainability of programs, practices and norms on student academic performance: a case study
PDF
An analysis of the impact of the total educational support system direct-instruction model on the California standards test performance of English language learners at experimental elementary school
Asset Metadata
Creator
Rodibaugh-Woods, Mimi
(author)
Core Title
An evaluation of the impact of a standards-based intervention on the academic achievement of algebra students
School
Rossier School of Education
Degree
Doctor of Education
Degree Program
Education (Curriculum
Publication Date
07/24/2007
Defense Date
06/26/2007
Publisher
University of Southern California
(original),
University of Southern California. Libraries
(digital)
Tag
2188,OAI-PMH Harvest
Language
English
Advisor
Hocevar, Dennis (
committee chair
), Baker, Robert L. (
committee member
), Mayberg[?], Connie (
committee member
)
Creator Email
mwoods@hboilers.com
Permanent Link (DOI)
https://doi.org/10.25549/usctheses-m649
Unique identifier
UC1197192
Identifier
etd-RodibaughWoods-20070724 (filename),usctheses-m40 (legacy collection record id),usctheses-c127-526858 (legacy record id),usctheses-m649 (legacy record id)
Legacy Identifier
etd-RodibaughWoods-20070724.pdf
Dmrecord
526858
Document Type
Dissertation
Rights
Rodibaugh-Woods, Mimi
Type
texts
Source
University of Southern California
(contributing entity),
University of Southern California Dissertations and Theses
(collection)
Repository Name
Libraries, University of Southern California
Repository Location
Los Angeles, California
Repository Email
cisadmin@lib.usc.edu
Tags
2188