Close
About
FAQ
Home
Collections
Login
USC Login
Register
0
Selected
Invert selection
Deselect all
Deselect all
Click here to refresh results
Click here to refresh results
USC
/
Digital Library
/
University of Southern California Dissertations and Theses
/
Examining the effectiveness of teacher training on the improvement of California standardized test scores at Eva B. Elementary School
(USC Thesis Other)
Examining the effectiveness of teacher training on the improvement of California standardized test scores at Eva B. Elementary School
PDF
Download
Share
Open document
Flip pages
Contact Us
Contact Us
Copy asset link
Request this asset
Transcript (if available)
Content
EXAMINING THE EFFECTIVENESS OF TEACHER TRAINING ON THE
IMPROVEMENT OF CALIFORNIA STANDARDIZED TEST SCORES
AT EVA B. ELEMENTARY SCHOOL
by
Richard Anthony Tapia
______________________________________________________
A Dissertation Presented to the
FACULTY OF THE ROSSIER SCHOOL OF EDUCATION
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulfillment of the
Requirements for the Degree
DOCTOR OF EDUCATION
May 2008
Copyright 2008 Richard Anthony Tapia
ACKNOWLEDGMENTS
First and foremost I would like to acknowledge and thank my loving
wife, Elizabeth Hoover Tapia. Her long hours of editing as well as her
continued support, encouragement, and unwavering love made this journey
possible and less daunting. Thank you Elizabeth. WATPOMH!
I would also like to extend sincere appreciation to Dr. Dennis Hocevar,
whose guidance, support, and knowledge made the completion of this endeavor
possible. I would also like to extend my thanks to Dr. Kathy Stowe and Dr.
Denise Hexom for their insight and expertise in the evaluation of this study.
My gratitude is extended to the students and staff at Cordova Lane
Elementary School and Williamson Elementary School for their patience,
cooperation, and tireless hours of supporting the students of our community.
Keep up the great work and I appreciate the support both staffs and
administrations have provided me during this process. Go Cougars!
Finally I would like to extend my appreciation and thanks to my family.
The lifelong support and encouragement from my brothers and sisters assisted in
the accomplishment of my dreams and goals. It has not been easy being their
little brother. Thanks for the love! I would also like to thank my mother and
father, God rest their souls, Eva Baquera Tapia and Jose Ramirez Tapia, without
whose constant love, support, and encouragement during their life time this
ii
dream and the wonderful educational opportunities presented to me would never
have been realized. I love and miss you both very much!
iii
TABLE OF CONTENTS
ACKNOWLEDGMENTS .......................................................................... ii
LIST OF TABLES......................................................................................vi
ABSTRACT ..........................................................................................vii
CHAPTER 1. PROBLEM IDENTIFICATION .........................................1
Problem Analysis...........................................................................10
Problem Solution ...........................................................................16
Purpose, Design, and Utility..........................................................19
CHAPTER 2. LITERATURE REVIEW..................................................26
Essential Program Components.....................................................26
High Risk Children Need Quality Teachers ..................................27
Schools That Succeed....................................................................31
Professional Development.............................................................33
Research to Increase Teacher Quality ...........................................40
CHAPTER 3. METHODOLOGY............................................................47
Design Summary ...........................................................................47
Participants, Sampling, and Setting...............................................49
Intervention Description ................................................................50
Quantitative....................................................................................52
Comparing CST Results ................................................................54
Equating and Scaling .....................................................................55
Qualitative......................................................................................56
Instrumentation and Procedure......................................................57
Observational Checklists...................................................57
Informal Interviews...........................................................58
Scaled Questionnaires........................................................59
Procedural Timeline.......................................................................60
Data Analysis.................................................................................61
Qualitative Analysis...........................................................62
Quantitative Analysis.........................................................62
Delimitations and Limitations of the Study...................................63
iv
CHAPTER 4. RESULTS..........................................................................65
Design Summary and Results Overview .......................................65
Eva B. Elementary, ELA Results ..................................................67
Eva B. Elementary, Mathematics Results......................................67
Eva B. Elementary ELA Performance Band Summary.................70
Eva B. Elementary Mathematics Performance Band Summary....72
Summary of NCLB Results at Eva B. Elementary in
ELA and Mathematics ................................................................75
Similar School Comparison Results ..............................................78
Overview of Results ......................................................................80
CHAPTER 5. CONCLUSIONS ...............................................................82
Overview........................................................................................82
Purpose and Design .......................................................................84
Summary of Findings ....................................................................86
Eva B. Elementary School.................................................86
AB 466 Training Summary ..............................................90
Eva B. Elementary School and
Ramirez Elementary School ...........................................95
Implications...................................................................................98
Site-Based Recommendations.....................................................102
Limitations...................................................................................105
Conclusion...................................................................................106
REFERENCES ........................................................................................108
APPENDIX: DETERMINING THE EFFECTIVENESS OF
AB 466 TRAINING ON STUDENT ACHIEVEMENT.............112
v
LIST OF TABLES
Table 1. Grade Level and Number of Students .......................................6
Table 2. Ethnic Background of Students .................................................7
Table 3. STAR Results on the Academic Performance Index.................9
Table 4. Similar Schools’ Profile...........................................................11
Table 5. Pre- Versus Post–Intervention CST ELA Performance
Band Differences: Statistical Findings ................................67
Table 6. Pre- Versus Post-Intervention CST Math Performance
Band Differences: Statistical Findings ....................................68
Table 7. Pre- Versus Post-Intervention CST ELA Performance
Band Differences: Practical Significance ...............................70
Table 8. Pre- Versus Post-Intervention CST Mathematics
Performance Band Differences: Practical Significance..........73
Table 9. Pre- Versus Post-Intervention CST ELA Percent Basic
and Above................................................................................76
Table 10. Pre- Versus Post-Intervention CST ELA Percent Proficient
and Above................................................................................76
Table 11. Pre- Versus Post-Intervention CST Math Percent Basic
and Above................................................................................77
Table 12. Pre- Versus Post-Intervention CST Math Percent Proficient
and Above................................................................................77
Table 13. API School-wide Comparison, 2005 and 2007 .......................78
Table 14. AYP School-wide Comparison, 2005 and 2007......................79
vi
ABSTRACT
The purpose of the study was to determine the effectiveness of a teacher
intervention and its application to the school-wide program at Eva B.
Elementary. The goal was to determine the effectiveness of AB 466 teacher
training, identified as the fourth Essential Program Component in the SAIT
(School Assistance and Intervention Team) process, on student achievement.
The situation that was presented placed Eva B. Elementary in the state program
improvement process. The application of the nine EPC through the SAIT
process were in process for 2 years at Eva B. Elementary and the researcher
examined the effect of teacher training on student achievement as measured by
the California Standards Tests (CST).
The teachers at Eva B. Elementary school attended training in the State
Board of Education adopted core curriculum of English/Language Arts and
Math, AB 466 training. Each teacher attended the five day intensive training
over a two-year period. The goal of the training was to prepare teachers to
present strategies for all students to increase the school’s overall performance
after attending the intensive training.
The results determined that although the achievement, as measured by
the school’s API (Academic Performance Index) increased over the time frame
assessed, students’ achievement increased only slightly in general. Students in
English Language Arts showed minimal levels of improvement across the grade
vii
levels and students in Mathematics exhibited a decrease in three of the four
grade levels assessed. Interestingly, the third grade students at Eva B.
Elementary showed a significant increase in English/Language Arts and
Mathematics, a grade level that has not fared so well historically statewide in
standardized assessments administered.
The staff and administration at Eva B. Elementary understood the need
for continued training, application and collaboration of the skills and practices
shared at the AB 466 training. The students benefited from the application of
the practices implemented in the SAIT process and the teachers understood that
a continued application of those practices will assist holistically in the
improvement of student achievement for the children of Eva B. Elementary
School.
viii
CHAPTER 1
PROBLEM IDENTIFICATION
Eva B. Elementary School was located in an emerging city in the east
end of Sacramento County and in the northern portion of the state of California.
The school was in Program Improvement (PI) under the guidelines of the
Federal No Child Left Behind Act of 2001 (NCLB) (United States Department
of Education, 2002). Although the school had shown improvement in the state’s
standardized testing program, it had not fully complied with all of the
regulations related to the achievement status of the students who attended this
elementary school. As a school site administrator in a neighboring school to
Eva B. Elementary, my goal was to identify specifically the validity and effect
of the following essential component: “Fully credentialed teachers and teacher
participation in AB 466 training on SBE-adopted instructional materials”
(California Comprehensive Center and American Institute of Research, 2006, p.
6). This was the fourth essential component, mandated by the state and
implemented by the Tapville Unified School District, for the staff at Eva B.
Elementary School in light of their Program Improvement status. The nine
Essential Program Components (EPC), as measured on the Academic Program
Survey (APS), were designed to support the improvement of student academic
performance in English Language Arts and Mathematics. By focusing on the
instructional core-curriculum and instruction, and supports for curriculum and
1
instruction, the EPC aim was to ensure that the School Assistance and
Intervention Team (SAIT) process remains focused intensively on the
improvement of student achievement in key academic subject areas (California
Comprehensive Center and American Institute of Research, 2006, p. 2). A list
of the nine Essential Program Components are as follows (California
Comprehensive Center and American Institute of Research, 2006):
1. Use of State Board of Education (SBE) adopted (kindergarten through
grade eight) or standards aligned-(grades nine through twelve) English-
language arts and Mathematics instructional materials including
intervention materials. (p. 3)
2. Instructional time: adherence to instructional minutes for English
Language Arts and Mathematics (K-8) and high school access to
standards-aligned core courses. (p. 4)
3. Principals’ instructional leadership training Assembly Bill (AB) 75
training on SBE-adopted instructional materials. (p. 5)
4. Fully credentialed teachers and teacher participation in AB 466
training on SBE-adopted instructional materials. (p. 6)
5. Student achievement monitoring system (use of data to monitor
student progress on curriculum-embedded assessments and modify
instruction). (p. 7)
6. Ongoing instructional assistance and support for teachers (use of
content experts and instructional coaches). (p. 8)
7. Monthly teacher collaboration by grade level (K-8) and department
(9-12). (p. 10)
8. Lesson and course pacing schedule (K-8) and master schedule
flexibility for sufficient numbers of intervention courses (9-12). (p. 11)
9. Fiscal support. (p. 12)
2
The EPC were designed to serve as the foundation for an effective
improvement process. The strength of each component was bolstered by
addressing all components in combination. In this researcher’s opinion, these
elements were good practices and processes for all schools, not just those
determined to be in need of improvement based on their achievement scores.
Schools that performed below the 50th percentile on the achievement
tests pursuant to Education Code Section 60640 (California Department of
Educatin, 2007a) were invited to participate in the Immediate
Intervention/Underperforming Schools Program (II/USP). The participating
II/USP schools that failed to meet growth requirements of the program were
identified and deemed state-monitored by the State Board of Education and the
State Superintendent of Public Instruction (California Department of Education,
2007a). This was one of a varying level of SAIT support that a Title 1 school
can volunteer to be a part of. When a school enrolled in the program, funding
was also attached to the process.
The state had earmarked a total of approximately $20,000,000 to support
the SAIT process statewide. Schools enrolled in the process were eligible to
receive $150 per student to accomplish the goals of the SAIT process and
implement the nine EPC. A school had three years once they were identified as
state-monitored to make significant growth for two consecutive years to be
eligible to exit state-monitored status under the II/USP program (California
3
Department of Education, 2007a). If the school did not achieve this level of
success, the sanctions from the state could increase. Those sanctions could lead
up to, and include, state control of the school or district in question (California
Department of Education, 2007b). This was an interesting situation to monitor
as the levels of proficiency should increase significantly between now and
2014—the year the guidelines related to the NCLB were expected to be
achieved. Although the NCLB guidelines were federal mandates, the goals of
proficiency and student achievement at the state, SAIT process, and national
level were very similar.
In the literature, the following comment was of interest, “While each
individual component was considered an important piece of the school
improvement process, no one component should be seen as a silver bullet to
improvement” (California Comprehensive Center and American Institute for
Research, 2006 p. 2). While there was absolute truth to the above statement, this
researcher was interested in addressing the professional development component
as the focus of this study. It is difficult to specifically identify the effects of the
fourth essential component on student achievement, however Linda Darling-
Hammond and Youngs (2002) proposed the following, “current research
indicate student achievement gains were more influenced by teacher quality than
by many other factors, such as class size or student demographics” (p. 17).
4
Teacher quality was of extreme interest to this researcher and an effort to relate
this concept to student achievement was the goal of this endeavor.
The question remained however, what is teacher quality and how can it
be measured? Some of the prominent researchers in the area have defined
teacher quality in research terms. Marzano (2001) shares his opinion on how a
teacher of high quality functions, “teachers should rely on their knowledge of
their students, their subject matter, and their situation to identify the most
appropriate instructional strategies” (Marzano, 2001 p. 9). Another expert in the
field, Darling-Hammond (Darling-Hammond and Youngs, 2002), identifies the
following scientific based guidelines that are to be considered when determining
teacher quality. “Looking across studies, several aspects of teachers’
qualifications have been found to bear some relationship to student achievement.
These include teachers’ (a) general academic and verbal ability, (b) subject
matter knowledge, (c) knowledge about teaching and learning as reflected in
teacher education courses or preparation experiences, (d) teaching experience,
and (e) the combined set of teacher qualifications measured by teacher
certification, which includes most of the preceding factors” (p.16). These
definitions will be taken into consideration specifically when considering the
effects our intervention will have on student achievement.
Eva B. Elementary School served approximately 400 kindergarten
through fifth-grade students on its campus. The school had 18 classes which
5
included 2 special education Special Day Classes (SDC) while the remaining 16
classes were regular education classes. Table 1 illustrates the students who
attended the school by grade level:
Table 1
Grade Level and Number of Students
Grade Level Number of Students
Kindergarten 67
First 69
Second 69
Third 62
Fourth 65
Fifth 66
The school serves students from very diverse cultural backgrounds and a
number of languages were spoken on the campus including, but not limited to:
Armenian, English, Farsi, Russian, Spanish, and Ukrainian. Table 2 identifies
the ethnic background of students enrolled at the school:
The background of the school population was as follows: the school was
bordered by a number of apartment complexes and housing options that were
traditionally utilized by families that were often very mobile.
6
Table 2
Ethnic Background of Students
Racial/Ethnic
Background
Number of
Students
Percentage of School
Population
African
American/Black
55 13.6
American Indian or
Alaskan Native
1 0.2
Asian 17 4.2
Filipino 9 2.2
Hispanic 77 19
Pacific Islander 0 0.0
White (Non-Hispanic) 238 58.8
Other 8 2.0
The students at the school and their parents had a mobility factor, as
measured by the California Department of Education (CDE) in its Demographic
Characteristics—School Report, of 15%. The average parent-education level
per STAR was 2.70 with 96% of the parents responding. The high school
graduate and some college categories were the two largest groups with 39% and
26%, respectively. The participants in the federal free or reduced-price lunch
program, according to STAR data, was 83% at Eva B. Elementary (California
Department of Education, 2007c). This information supplies a back drop of the
population served. Implementation of the nine EPC and the SAIT process had
begun at the school and the feedback and results of those processes are shared in
this study.
7
Although an emerging and potentially thriving community, Rancho
Cordova and the population that forms the community of Eva B. Elementary
School qualify for Title I services as defined by district parameters. Being
enrolled in the Title I program, the school was held accountable to the federal
mandates and guidelines of the No Child Left Behind Act of 2001 (United States
Department of Education, 2002). The assessment data from the State of
California, California Standardized Test (CST) or Standardized Test And
Reporting (STAR) was the measure that was utilized to identify the growth and
progress of all schools in the state of California on an annual basis. The
guideline for schools in the district which determines their eligibility for the
Title I program was related to the school’s percentage of students who were on
or eligible for the federal free and reduced-lunch program. The percentage of
students on this program was 83%, or approximately 330 of the reportedly 400
that were enrolled in the school. This measure was also defined as
socioeconomically disadvantaged families.
Listed on Table 3 are the STAR results in Academic Performance Index
(API) for Eva B. Elementary School for the timeframe 2002–2005,
8
Table 3
STAR Results on the Academic Performance Index
627
697
699
693
580
600
620
640
660
680
700
2002 2003 2004 2005
Eva B. Elementary API 2002 - 2005
API
As shown above, the Academic Performance Index (API) for Eva B.
Elementary for 2002 was 627, in 2003 it increased to 697, in 2004 it increased to
699, and in 2005 it decreased to 693 (California Department of Education,
2007d). The school entered program improvement status in the 2005-2006
school year for failing to meet its target growth in two consecutive years. In
2005, the school’s Academic Performance Index (API) decreased from 699 to
693, although between the years of 2003 and 2004, the school increased its API
two points from 697 to 699. The growth target was to increase by five points in
each of the two assessment cycles thus creating program improvement status for
the school in the 2005-2006 school year. Since that time, the staff and students
had been immersed in the implementation of the nine Essential Program
Components (EPC) in an effort to improve academic achievement.
9
Administration, both site and district, as well as students, staff, and
parents had a vested interest in the outcomes of the efforts made by the Program
Improvement guidelines and the SAIT process. Information, research, and
feedback provided in this study attempted to verify the validity of those efforts.
The level of educational productivity, like other types of productivity, depends
on how well inputs were turned into outputs by the behaviors or processes of
individuals and organizations (Milanowski, Kimball & Odden, 2005, p. 2).
Problem Analysis
What were the factors that were contributing to the underachievement of
the students at Eva B. Elementary School? Glancing at the most recent API
scores, one would conclude that the children had met their capacity for learning.
The students would seem to be similar and the scores in the last three years were
similar as well, falling within a 10 point range of one another. The goal for the
school and the entire state of California was for all schools to achieve and
maintain a score of 800 on the CST’s. With an average score in the last three
years of 696, Eva B. Elementary School was more than 100 points from that
mark. Table 4 shows the school’s similar schools profile as determined by CDE
in 2005 (California Department of Education, 2007e):
10
Table 4
Similar Schools’ Profile
832 807 791 788 693
0
200
400
600
800
1000
2005 API Scores
EE VSE GE RE EBE
Similiar Elementary Schools
Eva B Elementary Similiar Schools 2005
Elizabeth Elementary
Vernon Street Elementary
Garfield Elementary
Ramirez Elementary
Eva B Elementary
With the state expecting schools to reach the goal of an 800 point API
score and students from schools defined as similar by CDE achieving or nearing
this goal, what were the factors that were creating borders for Eva B.
Elementary School to reach the statewide expectation of 800 points?
The factors that can have an effect on academic achievement, student
improvement, and school growth can be multiple and varied. The focus was on
the significance of these factors and their relation to the school and the people
who constituted this educational community. The idea was to focus the energy
and attention on factors that the school and the people in it had control over and
the ability to change. Those factors can be related to knowledge, motivation,
students, and the organization.
The goal was to determine that the EPC related to staff training and
teacher development had the most influence on the improvement of academic
11
achievement and student learning. This would be considered a knowledge
factor. It was the goal of CDE to make certain that the teachers in these
underperforming schools had the credentials, expertise, and training to make the
students in their educational community successful. The fourth component
specifically states
Teachers in the school will be fully credentialed and the teachers will
participate in AB 466 training on State Board of Education (SBE)
adopted instructional materials. The training features the district’s
adopted core program and/or intervention programs for reading/language
arts and district’s adopted basic programs for Mathematics for each
teacher’s grade level and program. (California Comprehensive Center
and American Institute for Research, 2006, p. 6)
Research by Marzano (2003) in the area of school improvement and its
related factors shares that of all things considered, it was the teacher level
factors that have the most profound impact on achievement. Consider this,
“That is, about 13% of the variance in student achievement in a given subject
area was due to what the teacher does” (Marzano, 2003, p. 74). The three
teacher-level factors: instructional strategies, classroom management, and
classroom curriculum design cannot be isolated in terms of their classroom
application or their impact on student achievement. Additionally, the impact of
the individual classroom teacher could have a greater impact on student
achievement than any of the school level-factors (Marzano, 2003).
Research continually reflects the importance of teacher effectiveness and
its affect on student achievement. The training that schools enrolled in program
12
improvement and implementing the SAIT process and its support should, after
all of the teacher training takes place, experience effective student outcomes,
progress, and growth.
This was where the motivation of teachers in these SAIT schools came
into play. A discussion with a group of relatively new teachers, who had been
trained to provide the support and training for schools implementing the AB 466
training, felt that motivation of veteran teachers was a key factor. In some cases
the newer teacher trainers were not always well received by the more veteran
teachers, who sometimes had the attitude or mentality of “What possibly could
this new teacher show me that I don’t already know?” The issue, in some cases,
was that the veteran teachers felt they were forced to attend this training and no
matter how effective the training was, they were not going to receive any value
from the sessions. There was not always a sense of trust or a validation of the
newer teachers’ expertise in the area in which they were trained. The years of
experience of a teacher presenter seemed to be the most important and validating
factor for the veteran teachers attending the training.
Clark and Estes in their book Turning Research into Results (2002) state
the following related to human motivational factors, “motivation gets us going,
keeps us moving, and tells us how much effort to spend on work tasks” (Clark &
Estes, 2002, p. 44). According to Clark and Estes’ research, the three facets of
motivated performance were related to active choice, persistence, and mental
13
effort (pp. 80-81). Confidence in making an effort was paramount to motivation
as well. Both over and under confidence can have a negative effect on mental
effort. Like most things, the amount of mental effort applied to make a task
successful is just the right amount. Perhaps Goldilocks was motivated by the
appropriate mental effort in her encounter with the three bears’ belongings,
knowing after a couple of trial and errors, she would ultimately find the one
thing that was just right for her.
Belief has the most influential affect on motivation. Consider the
placebo effect: “When people believe that a pill is going to make them better,
they very often get better even though they were taking an inert sugar pill that
does not contain the active ingredient in the antidepressant medication” (Clark
& Estes, 2002, p. 82). When people think they have the appropriate capacity
and ability to be effective and benefit from getting the job done that is when
they will expend the appropriate amount of mental effort or motivation to get the
job done most efficiently.
It was imperative that the teachers who were attending the AB 466
trainings were motivated by the need to improve student growth and
achievement. If the confidence of the teachers was too high, the issues some of
the newer teacher trainers had would continue to be a hindrance. It was
essential that these teachers had a sense of the importance the improvement of
their students’ achievement had on themselves and society as a whole. Student
14
growth in an academic environment must be a belief in which teachers have a
vested interest regardless of factors that were out of their control. “The
difference between us was that different people have very different ideas about
what makes them effective” (Clark & Estes, 2002 p.83).
Student factors are always going to be at issue when it comes to school
and learning. The students’ motivation, prior knowledge, and home life have
approximately an 80% effect on student achievement. With those numbers
significantly supporting the fact that educators only affect a fifth of how students
learn, motivation can be easily cast asunder. It is because of the 80% that we are
not able to control that the 20% we do control must be at an effort of 120%. By
creating a belief in ourselves and our students that working diligently together
we can be successful, we can rise above the control of one of the three student
factors (Marzano, Pickering, & Pollock, 2001, p. 2). Also, creating situations
for students to call upon their prior knowledge in an effort to expand their new
knowledge will increase the belief and motivation of students that they can
achieve academic success and grow their knowledge. It is the home-life factor
that we cannot control. Using our energy and beliefs to positively affect the two
student factors related to motivation and prior knowledge, and accepting that we
cannot change what takes place in the students’ home, will make educators
much more successful in the long run. “The objective of adopting a more
positive motivational climate is to increase individual and team confidence,
15
interpersonal and organizational trust, collaborative spirit, optimism, positive
emotions, and values about work” (Clark & Estes, 2002, p. 100). In an attempt
to reinforce these important motivational factors we will also reap the reward of
their efficient and effective implementation.
Problem Solution
In solving the need to grow the school’s CST scores, the School
Assistance and Intervention Team (SAIT) was developed to implement the nine
Essential Program Components (EPC). The SAIT process was placed into law
in 2001 when the state revised laws related to the state takeover of schools, or
schools deemed state monitored. Schools that were enrolled in the Intervention/
Underperforming Schools Program (II/USP) were able to enter into a contract
with a SAIT and develop an action plan to improve school achievement.
Guidelines, timelines, and approvals are a large part of the development and
implementation of the SAIT process initialization as part of the state mandate.
Many of the commonly known components of the mandate correspond to the
development of an action plan developed by a team of highly knowledgeable
skilled professionals with experience in the areas of school leadership,
curriculum, and instruction aligned to state content standards. The members of
this team were to also have expertise in improvement of low-performing
schools, as well as evaluation and research-based reform strategies. These
16
teams would provide intensive support to implement reform initiatives that
would be driven by data and feedback from the lessons and instruction in the
classroom. The school and the team are responsible for sharing the information
to the governing board on a regular basis. Some of the lesser known initiatives
in this Education Code, related to the SAIT process, were the recommendations
to extend the school day, the opportunity to extend the school year, making the
employees of the school 12-month employees, and providing instruction to show
significant progress growth in assessments in two consecutive years (California
Department of Education, 2007a). This process would address the need to
improve student achievement but would surely raise concerns of the employee
groups that were working in these schools if they were to suddenly become 12-
month employees.
The goal was to create a sense of ownership to the students and the
improvement plan of the school. “Only when schools develop a shared
understanding of current reality can a commitment to change be initiated and
sustained” (Bamburg, 1994, p. 23). This was the goal of the SAIT and the
administration of the school and the district—a shared and common
understanding of the need to improve instruction to enhance student scores and
academic achievement for the students.
Teachers were working on the nine EPC to accomplish the necessary
growth with all student groups in an effort to have the school removed from the
17
Program Improvement list and be free of the sanctions. Teachers had made
efforts in reform to improve the opportunities for students and for their
colleagues to develop and support a sense of grade level and school-wide
camaraderie and collaboration for the benefit of the students. “Learning is as
much a socially shared undertaking as it is an individually constructed
enterprise” (Alexander & Murphy, 1999, p. 39). The administration had
supported and encouraged the growth and interaction of colleagues related to
student improvement and academic achievement. The staff, at times, was reared
in frustration, but constant reminders by the SAIT, and the administration
refocused their energies and direction.
When defining the significance of the nine components presented time
and again, the quality and effectiveness of teachers was presented as the most
important factor. Training and professional development were the two most
common factors presented in preparing teachers to increase their effectiveness in
the classroom and with students. Having identified those two common
reoccurring factors leads this researcher to believe that the most important
component was that which refers to teacher training and support in the SBE
curriculum. Teachers had received the training in the core curriculum areas and
the outcome for advanced student achievement was the expected outcome of
that effort. As most of the staff had completed the AB 466 training, renamed
18
AB 430 training, the expectation of the stakeholders was that student scores in
the CST’s would increase.
The training was quite extensive and comprehensive. Trainers
transported staff and the curriculum they were implementing on an in-depth
journey related to the multitude of options, exercises, and interventions that
were available within, and closely surrounding, the curriculum being applied.
Staff received curriculum and teacher strategy training to support all levels of
students in the school. The goal was that each classroom teacher would deliver
the core curriculum at any need necessary for students of all levels to be
supported and academically successful. Application of the curriculum at, above,
and below level were researched and presented in an effort to offer the teachers
the insight and ability to support students at any and all levels of need. The staff
received 5 days of 8 hours a day training to better understand, prepare, and
present the curriculum to students of varying levels in their classrooms and to
assist in the achievement and improvement of student achievement and scores at
their school. The trainings for staff was offered and presented in two main
curricular areas: Mathematics and English-Language Arts.
Purpose, Design, and Utility
The state and the federal government have invested a significant amount
of funding, over $20,000,000, in an effort that all students will meet a
19
government-defined level of proficiency in Language Arts and Mathematics.
These guidelines have been established through the NCLB. Each state has had
the opportunity to develop and define its set of guidelines to determine
proficiency in these two areas locally, and statewide in this case. One of the
programs that has a significant level of financial support, $150 per student, and
has been significantly developed and implemented was the School Assistance
and Intervention Team (SAIT) process (California Department of Education,
2007a).
The process includes and addresses Nine Essential Components (EPC).
One of those components, specifically number four, states as follows: “Fully
credentialed teachers and teacher participation in AB 466 training on State
Board of Education(SBE)-adopted instructional materials” (California
Comprehensive Center and American Institute for Research, 2006,p. 6). The
goal of this study was to determine how effective the training, application, and
implementation of the SBE instructional materials, specifically through the
attendance and completion of the AB 466 trainings by teachers, would effect
and improve student achievement as well as the achievement scores on the
CST’s of a Title 1 school that was enrolled in the SAIT process (California
Department of Education, 2007a).
An attempt to collect quantitative data in the form of surveys; class,
teacher, and student observations; and teacher interviews occurred on multiple
20
occasions. The level of achievement on the CST’s by Eva B. Elementary
School students was compared to their previous year’s test scores as well as
schools similar, as defined by the state, to Eva B. Elementary School’s student
population and demographics. These multiple forms of evaluations provided an
opportunity to measure the success rate of a portion of the SAIT process. By
utilizing these multiple measures, pros and cons of the program could be
identified and solutions or applicable interventions could be addressed and
applied in an attempt to provide a more effective direction of this statewide
intervention.
The purpose of the study was varied. The initial determination was to
measure the effectiveness of a program—a summative evaluation. However,
naturally engrained in that process was an attempt at addressing an effective
outcome that would be a portion of a formative evaluation, as well as the area of
action research, which will in some form, perhaps not very significantly, address
an outcome that would correct or identify a specific problem. The varied
methods of the data collected, and the manner in which it was collected,
logically addressed other forms of purpose in the study.
The focus of the study was depth and a qualitative nature of outcome; the
study utilized data that was quantitative in nature. The collection of interviews,
observations, and surveys leaned toward a qualitative nature of inquiry—one
that offered a significant insight into the effectiveness of the measured program.
21
However, one of the measures utilized in an attempt to evaluate the
effectiveness of the fourth Essential Program Component in the SAIT process
were longitudinal and comparative CST scores. This data was one type of
information that assists in the determination of the effectiveness of the program
and process itself. Therefore, the mixed-methods of research attempted to
holistically analyze and determine the effectiveness of the SAIT process
specifically related to the fourth Essential Program Component and teacher
training.
The units of analysis of this study were introspective as we measured the
growth of student achievement over a period of time. The units of analysis were
comparative across groups with similar characteristics and demographics. CST
scores from similar schools offered analysis to define differing manners or
efforts in improving student achievement. In all cases in this study, the units of
analysis was people-focused, and the outcomes and determinations attempted to
assist in the improvement of an already existing program. “The key issue in
selecting and making decisions about the appropriate unit of analysis is to decide
what it is you want to be able to say something about at the end of the study”
(Patton, 2002, p. 229).
A variety of sampling types were implemented in this study. By nature,
the type that may have most reflected the variety of measurements and data
22
collected were the theory-based sampling, in cohort with operational construct
sampling, and theoretical sampling.
The construct sampling was related to the longitudinal study of the
school’s CST scores over a time period of 3 years. The comparison of the
scores that were prior to the intervention were compared to the scores that were
achieved after the intervention had been put in place. The outcomes determined
if the implementation of the SAIT process, more specifically the teacher training
component, had a positive affect on the achievement of students. The most
recent scores, although from different populations, had some sense of
comparability to the scores from previous years.
The operational construct sampling was applied in the area of case study.
The real-world applications in the form of student and teacher observations and
surveys gave a view of the program from a variety of stakeholders involved in
implementing, receiving, and validating the outcomes related to the intervention.
The participants’ insight, experience, and attitude toward the intervention and its
application had a bearing on the outcome. That information had to be
disseminated and disclosed in a comprehensive and user-friendly format.
The theoretical sampling was applied in the comparison of similar
schools. The goal was to address the differing conditions of the multiple
situations and attempt to determine if other mitigating factors outside of the
intervention had an affect on the outcome. The outcome or comparison was the
23
comparison of CST scores at similar schools to see if the differing conditions of
the intervention had an affect on the outcome of the measurement.
In grounded theory, theoretical sampling supports the constant
comparative method of analysis. That is, one does theoretical sampling
in grounded theory in order to use the constant comparative method of
analysis. The two go hand in glove, connecting design and analysis.
(Patton, 2002, p. 239)
The evaluation design was both a qualitative and quantitative format.
The results and comparisons were shared in a formative and summative basis.
The comparisons of CST scores from similar schools to Eva B. Elementary
School were the formative evaluations of the intervention and the interview.
Observation and surveys were the summative portion of the evaluation. The
mixed-method proposed here allowed for multiple outcomes to be addressed and
measured in an effort to completely measure the true effectiveness of the
intervention and its usefulness in an effort to improve student outcomes and
student achievement.
The final outcome of the study had an impact on the stakeholders of Eva
B. Elementary School. The goal was to determine through the application of
varying theories, samples, and evaluations that information related to the
students, staff, and administration of the school and the district had a resulting
impact, regardless of whether it was positive or negative. The goal was that the
information would be applicable to maintain a continuous improvement model
of educational application at the school, as well as all of the other schools
24
located in the school district. “Creating a culture of inquiry involves analyzing
relevant data, probing perceptions about why things were as they are, and
examining the academic culture including opportunities to learn.” (Johnson,
2002, p. 12).
The application of this evaluation to teacher training strategies not only
presented by the nine EPC, but also by teacher training in general, will support
what efforts need to be made to support the core curriculum. Was it the focused
straightforward design of AB 466 training related to the SBE-approved core
curriculum, or could it be strategies in effective teaching methods and areas of
differentiation that were at the core of improved student achievement?
Measuring the effectiveness of this specific training on a school involved in the
SAIT process, with 100% of the teaching staff trained specifically in the core
curriculum, offers insight into the value of specific core curriculum training.
The ability to utilize multiple measures in this evaluation, CST scores,
observations, interviews, and questionnaires increases the information relevant
to the evaluation of the training. Effect of the EPC, in general, will be critical,
however, the influence of training mandated in AB 466 and prescribed in
Essential Program Component four will be the focus of the outcome specifically.
25
CHAPTER 2
LITERATURE REVIEW
Many researchers concur that one of the most important keys to student
success is teacher quality. Marzano (2003), Darling-Hammond (2007), Darling-
Hammond and Youngs (2002), and Reeves (2003) among others share varied
levels of research that support this claim. The California Comprehensive Center
in its report Research Summary Supporting the Nine Essential Program
Components and Academic Program Survey, a study designed to support the
improvement of student academic performance in English Language Arts and
Mathematics (California Comprehensive Center and American Institute for
Research, 2006), identifies one of the nine essential components as teacher
quality through preparedness.
Essential Program Components
The California Comprehensive Center and American Institute for
Research (CACC) study, based on the compilation of numerous studies related
to school improvement, devotes an entire Essential Program Component (EPC)
to the quality of teachers who were presenting the curriculum to students in
under-performing schools. The CACC identifies the nine essential components
as nine of the most influential factors identified in the functioning of schools
26
that were beating the odds by demonstrating success with challenging student
populations.
Number four on the list of the nine essential components was the
identification of fully credentialed teachers. Further, the fourth component
advises that these teachers be appropriately trained in the State Board of
Education-adopted materials and curriculum related to English Language Arts
and Mathematics. The research continually supports the need for highly
qualified and fully credentialed teachers. That concern was also identified in the
national NCLB which mandates that schools must employ teachers who possess
appropriate credentials and the qualities outlined in the NCLB. It was the effort
of this research project to emphasize the importance of the claim that teacher
quality was a significant factor in the preparation, development, and growth in
the achievement of students in all schools, most importantly lower-performing
schools. The goal of this research effort was to recognize the immense
importance of teacher quality and preparedness to effectively instruct students in
the classroom, especially those classrooms of our most challenging students.
High Risk Children Need Quality Teachers
Research continues to confirm that the most challenging and needy
populations were mostly minority students who may also be from a socio-
economic disadvantaged (SED) status and were identified as English-Language
27
Learners(ELL). These were the population that, for the most part, can benefit
from the influence of a highly qualified and fully credentialed teacher.
Unfortunately, this population was least likely to reap the benefits of quality and
the NCLB-mandated skills from the teachers who were implementing
instruction in their classrooms. In many cases, teachers who serve these
populations were recently entering training to develop these qualifications and
credentials and were less experienced and less skilled in implementing the
curriculum and practices that were deemed essential and of high quality in
supporting student learning and achievement.
The need for teachers to serve this population necessitates that districts
with significant groups of students from SED status and ELL populations
employ the best quality individuals to fill the role of teacher in these classrooms.
It was a concern that continues to grow as the number of significantly
disadvantaged students continues to fill our classrooms and schools. The goal
was that districts accomplish this endeavor by hiring teachers who have the
appropriate qualifications. In absence of that accomplishment, it was imperative
that individuals who were employed in these roles be enrolled in effective and
extensive credentialing programs to adequately support those students in the
direst need.
Darling-Hammond (1997) presents a number of interesting factors. Her
report found that although the nation would be attempting to field more than two
28
million teachers over the next decade, these future teachers had no guarantee
that a system was in place to insure their access to the type of knowledge
necessary to assist in the support of their students. She also shared that one-
quarter of newly hired teachers lacked the qualifications for their positions and
23% of all secondary teachers hired did not even have a minor in their main
teaching field.
The staggering statistic that stood out continues to be the lack of support
for our most needy students. Darling-Hammond (1997) states that schools with
the highest minority enrollments had less than a 50% chance of getting a science
or math teacher who held a license or degree in the field they taught. These
statistics were unfathomable now and this information dates back to 1991. More
recent statistics reveal the following: 21% of all public secondary teachers had
less than a minor in their main assignment field, 59% had less than those
qualifications in their secondary teaching field, and 27% of new entrants to the
field of teaching had not met the minimum requirements to enter teaching and
were practicing with a substandard or no teaching license at all (Darling-
Hammond, 1997).
Quality and qualifications of teachers do matter and have an enormous
impact on the learning and achievement to our students. This factor was
continuously confirmed. One study in Texas analyzed 900 school districts and
found, teacher’s expertise—as measured by scores on a licensing examination,
29
master’s degrees, and experience—accounted for about 40% of the measured
variance in students’ reading and Mathematics achievement at grades 1 through
11. In New York City, a study of high- and low-achieving schools with similar
populations found that differences in teacher qualifications accounted for more
than 90% of the variation in student achievement in reading and Mathematics.
The effects of these findings were so strong and the variations in teacher
expertise so great that after controlling for socioeconomic status, the largest
disparities in achievement between black and white students were almost
entirely accounted for by differences in the qualifications of their teachers. No
other intervention can make the difference that a knowledgeable, skillful teacher
can make in the learning process (Darling-Hammond, 1997).
The identification of teachers who were fully credentialed was not the
end all be all of teacher quality. Research also supports the need to continually
develop all teachers effectively to maintain an exemplar level of professional
competence. A coherent, comprehensive plan for teachers to maintain a high
level of quality also was important. In many instances, teachers do not take the
opportunity to increase their knowledge and implementation base for curriculum
and instruction. The implementation of a new curriculum plan or new
instructional strategies was often met with criticism and cynicism. Often a
“been there done that” attitude prevails when plans or practices that constitute
change were introduced. Many teachers feel that the change that takes place
30
was more harmful than effective. Ironically many times the same argument was
made each time veteran teachers were presented with changes in curriculum or
instructional strategies.
The most effective way to combat and defend the ideas of a new
curriculum and instructional strategy change was to identify and share effective
results. Nothing was more convincing in changing the attitude of skeptical
educators than positive results from student populations similar to their own. It
was imperative that districts and administrations share energy and enthusiasm
when implementing efforts that attempt to increase student achievement. Many
times these efforts were sabotaged by the people in charge of implementing
them by the mere fact that subliminally they too, have a bias or concern that
these efforts will not be successful. The instructional and educational leaders
have to have a plan about which all share the same fidelity and commitment.
Any wavier or skepticism toward any change, no matter how minor, can have a
significant impact on the support of that effort.
Schools That Succeed
Reeves (2003) shares the following strategies in his effort to improve
student achievement through his research efforts with schools that were low-
poverty and had high multi-ethical populations. These schools were labeled
90/90/90 Schools. These schools had been identified with the following
31
characteristics: 90% or more of the students were eligible for free and reduced
lunch, 90% or more of the students were members of ethnic minority groups,
and 90% or more of the students met the district or state academic standards in
reading or another area. In brief, these findings include the following from
research completed in the Milwaukee Public School systems and independently
verified by Schmoker (2001):
1. Techniques used by the 90/90/90 Schools were persistent. The
students were still poor and their economic opportunities have not
improved. Nevertheless, more than 90% of the students in these schools
continue to meet or exceed state standards.
2. Techniques used by the 90/90/90 Schools were replicable. The first
time the district tracked these schools; only seven 90/90/90 Schools were
identified. In the most recent report, 13 schools meet the criteria for this
distinguished label.
3. Techniques used by the 90/90/90 Schools were consistent. These
schools were not lurching from one fad to another. While they differ in
some respects with regard to implementation, they were consistent with
regard to the following areas of emphasis:
Writing—students write frequently in a variety of subjects.
Performance Assessment—the predominant method of assessment was
performance assessment. This does not mean that these schools never
use multiple-choice items. However, it was performance assessment in
several different disciplines that local observers have associated with
student progress.
Collaboration—teachers routinely collaborate, using real student-work
as the focus of their discussion.
Focus—teachers in these schools do not try to “do it all” but were highly
focused on learning. (Reeves, 2000, pp. 7-8)
32
Sharing the depth of research and providing educators an opportunity to
review, relate, and interact with this information and the strategies shared within
can be an effective exercise in developing changes that significantly support the
effort in student achievement and teacher quality.
Professional Development
The level of teacher quality relates to multiple factors and the two that
were of paramount importance were preparedness and continued professional
development. In many cases, a fully credentialed and prepared educator can
lack some of the most important skills necessary to being effective. It was then
the charge of districts to develop an effective professional development plan that
supports the efforts of teachers in developing strategies to increase student
achievement at all levels. The goal was to develop the level of skill in teachers
to achieve success with all populations in our educational settings. Many times
the expectation of teachers was to just do it when implementing new curriculum,
texts, ideas, or strategies to improve student learning, without providing
appropriate training to achieve the appropriate level of aptitude that will have an
effective impact on student achievement.
The curriculum for teacher education should be shaped both by what
teachers need to learn and by how they learn. This means, first of all,
that the content should be organized so that teachers gain a mental map
of what was involved in effective teaching and what factors influence
student learning. With this schema for their work and learning, teachers
33
can seek out and add knowledge about specific techniques throughout
their careers. (Darling-Hammond & Baratz-Snowden, 2007, p. 115)
The foresight of this approach was a proactive measure to assist in the
support of teachers to develop and maintain expertise in their field.
It was imperative that the responsible agency also include continual and
consistent training to support teacher efforts to implement, master, and
effectively utilize the skills of their trade. I liken most professional development
in education to a short discussion or change that was to be implemented.
However, the way the individual (educator) implements the change in a manner
in which they define it, may or may not be the intention of the program or the
district when the plan to make the instructional changes were developed.
Unfortunately this may be more the norm than the exception in education.
Training and preparation must be fully supported and implemented to reap the
most significant benefits from the effort possible. A comprehensive plan related
to the time frame of the changes in SBE-adopted curriculum and materials was
essential. As we prepare teachers in any curricular or subject matter adjustment,
a timeline for levels of comprehensive training must also be included. Time for
teachers to internalize the information, process, practice, and collaborate on their
efforts was invaluable. Teachers should be allowed an opportunity to apply the
tools of their trade, receive refreshers on the SBE curriculum and the important
concepts of that information, and re-implement their strategies. They should
34
also have time for more collaboration and practice which would provide
monumental improvement in student achievement.
Togneri and Anderson (2003) outlined commitments made by school
districts to improve instruction and achievement in all schools. The following
were the seven principal findings Togneri and Anderson presented:
1. Districts had the courage to acknowledge poor performance and the
will to seek solutions.
2. Districts put in place a system-wide approach to improving
instruction.
3. Districts instilled visions that focused on student learning and guided
instructional improvement.
4. Districts made decisions based on data, not instinct.
5. Districts adopted new approaches to professional development.
6. Districts redefined leadership roles.
7. Districts committed to sustaining reform over the long haul. (p. 7)
All of the findings presented were significant to the area of teacher
quality and student achievement, however, the findings that stand out as most
effective in support of these efforts were the following. Districts put in place a
system-wide approach to improving instruction and districts instilled visions that
focused on student learning and guided instructional improvement. Most
importantly, in my opinion, was that districts committed to sustaining reform
over the long haul. Togneri and Anderson’s (2003) findings support a
35
comprehensive consistent approach to student achievement, increasing teacher
quality, and a district’s commitment to those important efforts.
We would hope that medical personnel, when trained in a new or
developing medical procedure, would take a significant amount of time to fully
understand the intricacies of a heart or brain surgery strategy prior to their
implementation of that strategy on oneself or a loved one. It is imperative that
the same attention to training and detail be present in the field of education and
teaching. Although we may not be physically operating on vital organs in the
body of the students we teach, we have an insurmountable affect on the growth
and development of their minds and their learning. Our influence is far and
wide, and many in education may not be fully aware of the significantly
monumental impact our practices have on the students under our charge.
In the compilation of his book, Classroom Instruction That Works
(Marzano, Pickering, & Pollock, 2001), Marzano and his colleagues presented
the review of a multitude of research projects and efforts that have identified the
nine most effective instructional strategies. Repeatedly these strategies support
the efforts of teachers and researchers in increasing student learning and
achievement. The nine strategies are as follows:
1. Identifying similarities and differences.
2. Summarizing and note taking.
3. Reinforcing effort and providing recognition.
36
4. Homework and practice.
5. Nonlinguistic representations.
6. Cooperative learning.
7. Setting objectives and providing feedback.
8. Generating and testing hypothesis.
9. Cues, questions, and advanced organizers.
A meta-analysis was developed to determine the average effect of a
given technique and the results were then presented in numbers related to effect
size and percentile gain to illustrate the scientific effectiveness of each strategy.
Throughout the discussion of the strategies in Marzano’s book the results were
defined and reported in effect size and percentile gain (Marzano, Pickering, &
Pollock, 2001).
It was important to the teaching profession that research related to
effective teaching strategies and practices as complied by Marzano and his
colleagues be implemented when developing teacher preparation programs.
Also important was the implementation of effective professional development
for educators presently in the classroom. In many instances, solid fool-proof
research and legitimate educational practices were not consistently immersed in
the endeavor to support teacher quality, growth, and efforts that will support
increased sustainable student learning—growth and achievement. It can be the
37
simplest of measures available that have the most profound effects on student
outcomes and achievement.
Marzano (2003) continues to share more applicable information. He
states teacher effectiveness is the most important factor affecting student
achievement. Marzano includes the following teacher-level factors as having
the most impact related to student achievement: instructional strategies,
classroom management, and curriculum design (Marzano, 2003). The reference
to the need for professional development and a plan for consistent or continuing
training in these very important teacher-level factors related to student
achievement were presented as school-level factors. The factors Marzano states
were incredibly important, and for those factors he cites to maintain a lasting
effect or impact, there was the need to develop and maintain a comprehensive
level of training and support for teachers in those three areas (Marzano, 2003).
Although Marzano (2003) mentions professionalism in his text related to
school-level factors, the importance of teacher training and professional
development was somewhat short sighted. He references a study by Darling-
Hammond (1997) that identifies teacher quality as a significant variance to the
increase of student achievement and applies one action step to the concept. The
importance of teacher training in an effort to increase teacher quality was, again,
overshadowed by other factors. This was a common occurrence among many of
the recommendations or research areas that were presented in support of
38
improving student achievement. Many researchers share the finished product
for student success and teacher effectiveness, but many miss the mark of
including important, relevant, and effective training as part of the plan. Without
sincere consideration of the importance of teacher training to support an
effective increase in significant student achievement, the other factors do not
have the opportunity to meet the high level of expectations on teacher
effectiveness and student achievement.
Therein may lie the major issue related to the increase in student
achievement—the lack of a continued effort or plan to support teachers in the
development and understanding of any and all instructional and curricular
changes. The efforts made in curricular changes need to be followed by a
comprehensive integration of a training plan that supports teacher development
and implementation of those changes to effectively influence the instruction and
strategies in their classrooms. A strategy or strategies that effectively implement
curricular changes to increase student achievement must be presented with
positive results for populations that historically have had difficulty consistently
increasing their achievement and academic growth.
Teachers also develop dispositions about what it means to be a teacher.
Chief among them was the disposition to continue to seek strategies for
reaching students who were not initially successful. Being a professional
involves not simply “knowing the answers” but also having the skills and
the will to evaluate one’s practice and search for new answers when
needed, at both the classroom level and the school level. Teacher
education that helps teachers diagnose what was going on in problematic
situations—when students were not learning, for example, and seek out
39
other resources or knowledge to address the problem, helps develop this
crucial disposition. (Darling-Hammond & Baratz-Snowden, 2007, p.
116)
Simultaneously these efforts must be supported by a comprehensive
training plan for teachers. Districts must maintain a commitment to sustaining
reform over the long haul. The target of these efforts must attempt to increase
the fidelity and commitment to the curricular or instructional changes and focus
on successful practices and programs that show promise in their implementation.
Once this effort is presented in a thorough and judicious effort supported by
educational and curricular leaders in academic settings, we can then expect the
teachers whose efforts were supported to have the expected impact on
curriculum, instruction, and student achievement in a comprehensive and
compelling effort. Supporting teachers in training and application of new and
effective ventures or changes in curriculum can go a long way in the
implementation of any effort. This, along with effective results from other
successful curricular practices, can be two of the most compelling aspects to
take place in an attempt to increase student achievement through the continued
enhancement of teacher quality.
Research to Increase Teacher Quality
Recruitment needs to focus not only on ensuring that we have enough
teachers, but also on recruiting a diverse teaching force that represents the
American population if majority and minority students are to experience diverse
40
role models. The proportion of minority teachers, approximately 13%,
continues to be far less than the proportion of minority students, just over 33%
in public schools (Darling-Hammond, 1997 p.15).
Teacher quality has many attributes and identities and there were a
number of considerations to make when assessing the quality of a teacher,
preparation, experience, content knowledge base, and skill set. Many of these
topics were often overlooked when teachers were recruited and placed in
classrooms in our nation and state. This most significantly occurs in classrooms
that require a considerable amount of support—students from socioeconomic
disadvantaged and non-native English speaking families. All students suffer
from inferior teaching; however, research continues to reinforce that students
with the most severe needs were the ones who were constantly short changed by
being served by teachers who were less qualified. The attributes shared above
were increasingly important and need to be successful in the implementation of
effective curriculum and increased student achievement. Universally, increasing
the quality of teachers in the classroom will significantly facilitate the need for
filling the role of effective, quality teachers for all students, particularly those
students identified as most needy.
Teacher preparation and professional development were among the most
significant areas to target when attempting to increase teacher quality. Many
teachers come to the practice of teaching with little or no knowledge or
41
understanding of child development philosophies, theories, or practices. An
extensive knowledge base in child development, as well as effective and
applicable teaching strategies, continues to be increasingly important as the
diversity of our student population grows dramatically. The need to support
students who come from diverse knowledge backgrounds, who were non-native
English speakers, and have varied levels of special needs in every classroom,
continues to be the major emphasis. The growth of diverse populations in our
communities continues to necessitate these practices and efforts in teacher
quality to expand as well. Teachers who have trained years ago must be
provided the opportunity to update their skill set, practices, and pedagogies to
collectively support the diverse needs of our students. Cultural sensitivity also
needs to be considered when applying and placing into action the appropriate
and effective teaching practices these students necessitate.
Research continually supports the assertion that teacher preparation and
professional development were increasingly effective in the efforts and
developments in increasing student achievement. The increased level of teacher
preparation and professional development consistently shows increases and
improvements in student achievement. The knowledge of curriculum and
diversified strategies to implement those skills make a difference in the quality
and equality for students in learning. Teachers new and old cannot survive in
the high-stakes endeavor of increased student achievement without the
42
reliability of a solid curriculum plan that maintains a consistency and support for
teachers to develop, practice, collaborate, and interact with new curriculum and
effective research-based teaching strategies, when provided, to support their
efforts to increase student achievement.
Teachers should not rely on the practices that they were educated with in
their experiences as a student, nor should they solely rely on the strategies they
were educated in during their teacher preparation programs, even if they were
fortunate enough to be appropriately prepared prior to their induction to a
classroom of students. A plan for consistent training that supports the goals and
expectations of the state, district, and school system should be implemented
consistently and provided with resources and practiced interactively. These
efforts alone will not be the culminating factors in developing and maintaining
teacher quality; however, it was a strong positive step in the right direction. As
Togneri and Anderson (2003) share in their findings, districts that committed to
sustaining reform over the long haul showed that making a difference takes
time. The plans and strategies for improvement were developed and remained
consistent over a number of years. There was no false pretense of quick fixes or
phenomenal overnight developments. District leaders encouraged practitioners
to try new ideas and did not expect immediate results.
Strategies to increase teacher quality were of the utmost importance and
groups, teams, and schools of teachers cooperating in this effort were paramount
43
to the success. Teachers alone cannot have as significant a level of
effectiveness, as a group of teachers from varying levels of expertise
collaborating together. Marzano (2003) and Reeves (2003) both concur that
spending time together as a team, lesson planning, developing, collaborating,
and processing data was of the utmost importance to increasing teacher quality
and student achievement. Marzano specifically states “Engage teachers in
meaningful staff development activities” (Marzano, 2003 p.65). Reeves adds
the following related to effective and necessary collaboration,
The schools devoted time for teacher collaboration. This was not merely
an exercise in idle discussion nor at attempt to get along in a friendly and
collegial fashion. Rather, collaboration meetings were focused on an
examination of student work and a collective determination of what the
word “proficiency” really means. (Reeves, 2003, p. 9)
Teachers practicing and training as a team can solidify the positive
efforts of all students in a more efficient and effective method. Working as a
team developing ideas, strategies, lessons, and applications in concert, and then
supporting one another in those efforts through observation and overview of
those practices, cooperatively will have a dynamic and lasting effect on the
instruction of all students and lessons developed collectively. The team support,
reflection, and evaluation of the endeavor is beneficial to all who participate.
The effect can be endless in increasing teacher quality and supporting student
achievement.
44
Darling-Hammond and Baratz-Snowden (2007) share the following
collaboration strategies in increasing the effectiveness and quality of teachers.
Programs and professional development efforts that teach teachers to do more
than simply implement particular techniques; they help teachers learn to think
pedagogically, reason through dilemmas, investigate problems, and analyze
student learning to develop appropriate curriculum for a diverse group of
learners (Darling-Hammond & Baratz-Snowden, 2007, p. 119). Groups of
educators who share norms and practices can be especially powerful influences
on learning, especially when there were collective knowledge and common
goals across frameworks and courses. Teachers learn best in a community
which enables them to develop a vision for their practice; knowledge about
teaching, learning, and children; dispositions about how to use this knowledge;
practices that allow them to act on their intentions and beliefs; and tools that
support their efforts (Darling-Hammond & Baratz-Snowden, 2007).
It is the application of these comprehensive, consistent, and practical
strategies for teaching and learning that are significant for educators at all levels
to become aware of, practice, and apply in their educational and pedagogical
settings. It is the responsibility of the professionals who make the decisions
related to the effect of these efforts to also maintain the vision for educational
and instructional programs that provide the cutting-edge effective practices and
systems that make a difference for all students. The management of these
45
directives is for the benefit of our youth and our students from all walks of life.
The onus is ours—that of a collective society—an investment in success for all
is of paramount importance. Our future and the future of an educated society
depend on it!
46
CHAPTER 3
METHODOLOGY
Design Summary
The method of data collection for this study necessitated the comparison
of California Standardized Test (CST) scores for Eva B. Elementary School for
the 2005 and 2007 assessment cycles. A pre/post design was implemented when
comparing scores from the two assessment time frames, as well as a comparison
to a similar school to Eva B. Elementary, as identified by the California
Department of Education (CDE).
The scores that were accomplished in the 2005 assessment year were the
base year or control group, and as the intervention, AB 466 teacher training
(California Comprehensive Center and American Institute of Research, 2006)
took place over a three year period: 2004-2005, 2005-2006, and 2006-2007
school years. The point of comparison was the CST scores from the 2007
school year—the year that the completion of the intervention of teacher training
had been accomplished. The similar-school scores were utilized as a
comparison to measure the level of effectiveness of the intervention. The
following summative evaluation format was implemented: 0 pre X 0 post. The
2005 scores were utilized as a base for Eva B. Elementary as well as the scores
from Ramirez Elementary School, a school on Eva B. Elementary’s 2005
47
similar-schools list. The 2007 scores were considered the post-intervention and,
thus, reflect the effects of the AB 466 state mandated teacher training
intervention in both the English-Language Arts (ELA) and Mathematics core
curriculum.
Pre-test Observation: 2005 California Standards Test
Treatment (X): AB 466 Teacher Training in the Core Curricular Areas
Post-test Observation: 2007 California Standards Test Scores
Results of the 2007 CST scores were compared to the base year of 2005
CST scores for Eva B. Elementary School to determine the level of effectiveness
of the fourth Essential Program Components (EPC) (teacher training) in the core
curricular areas through AB 466 (California Comprehensive Center and
American Institute of Research, 2006) training and its effect on student
achievement and CST scores. A comparison of results related to Eva B.
Elementary’s similar schools in 2005 were analyzed as a basis of the effect,
positive or negative, of the AB 466 training.
With the mixed-method approach of analysis being applied in this study,
the evaluation also included various formative components. Interviews with the
staff and administration, as well as observations in the classroom attempted to
determine the level of implementation and the skills and strategies present in the
classroom that were presented in the AB 466 training in which teachers had
participated. The interviews and observations of the Eva B. Elementary staff
48
took place in the 2007-2008 school year. Questions during the interview
process related to the specific effectiveness of the AB 466 training and
classroom observations evaluated the application of strategies presented in the
classroom during instruction time.
Participants, Sampling, and Setting
Participants in this study consisted of teachers who had completed the
AB 466 training at Eva B. Elementary School and their students who were
participating in the California Standards Tests (CST) in the spring of 2007. The
base year data from assessments implemented in the spring of 2005 prior to the
intervention of AB 466 teacher training were compared to the results from the
spring of 2007 after most teachers had received and completed the AB 466
training. Assessment results were attained by receiving school reports from the
STAR site located within the California Department of Education website
(California Department of Education, 2007d).
There were multiple additional participants such as: the similar school,
Ramirez Elementary, that was compared to Eva B. Elementary School to
provide a comparison of the training in different educational environments; and
the principal of Eva B. Elementary, who provided insight and support of the
efforts, application, and fidelity of the AB 466 training that teachers had
49
implemented to the students at Eva B. Elementary School. As the researcher, I
provided an outsider’s point of view of the application and implementation of
the training through observations and interviews, professional dialogue, and an
analysis of the findings while comparing pre- and post-CST results from the
students at Eva B. Elementary School.
The backdrop for this study was the population of students and staff at
Eva B. Elementary School. This school maintained a population of
approximately 400 kindergarten through fifth-grade students. The school was
located in the emerging community of Rancho Cordova. The community had
just become a city and was bordered by the American River to the north, the
City of Sacramento to the west, the City of Folsom to the east and the City of
Elk Grove to the south. Civic pride had always been present but had taken a
very positive up turn since city-hood had been granted to the Rancho Cordova
community in 2003. Eva B. Elementary’s population was quite diverse and
multi-lingual. There were a number of languages spoken on campus including
but not limited to: Armenian, English, Farsi, Russian, Spanish, and Ukrainian.
The student population included African-American, Asian, Filipino, Hispanic,
Native American, and Anglo students. The school was surrounded by a number
of apartment complexes that made the population very transient. Eighty-three
percent of the student population participated in the free and reduced-lunch
program which made the school eligible for its participation in the district’s Title
50
1 program. This classification was what constituted the school’s involvement in
the SAIT and program improvement process. The student participant sampling
was very mobile, of a SED status, and an extremely culturally diverse
population which was reflective of most elementary schools in the state of
California.
Intervention Description
AB 466 (California Comprehensive Center and American Institute for
Research, 2006) teacher training, mandated by the California Department of
Education in July 2002, was the intervention treatment for teachers at Eva B.
Elementary School. The intervention is Essential Program Component number
four that addresses the following concerns: Fully credentialed teachers and
teacher participation in AB 466 training on State Board of Education adopted
instructional materials. While the Essential Program Component number four
has three parts, only two pertain to elementary programs and thus this study:
1. The district staffs the majority of its classrooms in all schools with
fully credentialed teachers.
2. A majority of K-8 teachers attend the AB 466 Professional
Development Program (40 hours of institute and 80 hours of
practicum). The training features the district’s adopted core program
and/or intervention programs for reading/language arts and district’s
adopted basic programs for Mathematics for each teacher’s grade
level and program. (California Comprehensive Center and American
Institute of Research, 2006, p.6)
51
“While teacher quality is the foundation for improved classroom
instruction, relevant professional development for teachers on the effective
implementation of the core program is also necessary to improve student
achievement and instructional programs” (Corallo & McDonald, 2001, p.9). It
is imperative that teachers are trained appropriately and consistently to
implement the core curriculum. It is also of extreme importance for teachers to
have the opportunity to plan, collaborate, and communicate with their peers at
grade level to coordinate the implementation of appropriate and relevant
instructional practices.
The intervention of AB 466 teacher training for all teachers at Eva B.
Elementary School required a 2-year timeline. The training was very time
consuming and intensive and required a significant time commitment from the
staff trained in the core curricular areas of Mathematics and English-language
arts. The availability, scheduling, implementation of the curricular training, and
methods and strategies were an investment that required a significant amount of
time and commitment from all parties involved in the process.
Quantitative
Quantitative data related to the California Standard Test results STAR
(Standardized Testing and Reporting) were the basis for the measurement of the
effectiveness of the AB 466 training intervention. Results provided by the
52
California Department of Education from Eva B. Elementary School were
compared from 2005, the year prior to the AB 466 training taking place, to the
post-intervention results of 2007, the year all teachers had completed the AB
466 training. The results for all students in the core subject areas of
Mathematics and English-language arts were analyzed and compared to attempt
to measure the effectiveness of the AB 466 training on student growth and
achievement at the school. The information posted on the California
Department of Education website, dated 11/18/05 (California Department of
Education, 2007f), indicated the following related to the California Standards
Test information and STAR results:
The ELA CSTs for grades two and three consist of 65 multiple-choice
questions with an additional 6 field-test questions. For grades four
through eleven, the tests consist of 75 multiple-choice questions with an
additional 6 field-test questions. At grades four and seven, the ELA
CSTs also include a writing component, the California Writing Standards
Test, which addresses a writing applications standard selected for testing
each year. (California Department of Education, 2007f, n.p.)
In math, the following information was posted: “Students in grades two
through seven take grade-level Mathematics tests. The CSTs in Mathematics
consist of 65 multiple-choice questions with an additional six field-test
questions” (California Department of Education, 2007g, n.p.). The following
information was also posted from the STAR test website related to comparing
and compiling of scaled scores. “It is important to have an insight of the
information when making comparisons of multiple years of CSTs. The
53
understanding allows for more consideration of the information and results
shared” (California Department of Education, 2007g, n.p.).
The purpose of the CSTs is to determine students’ achievement of the
California Content Standards for each grade or course. Students’ scores
are compared to preset criteria to determine if the students’ performance
on the test is advanced, proficient, basic, below basic, or far below basic.
The state target is for all students to score at the proficient and advanced
levels.
The 2006 STAR Program includes four components:
• California Standards Tests (CSTs)
• California Alternate Performance Assessment (CAPA)
• California Achievement Tests, Sixth Edition Survey (CAT/6
Survey)—grades 3 and 7
• Aprenda: La prueba de logros en español, Tercera edición
(Aprenda 3)
The CSTs are a major component of California’s accountability system
for schools and districts. CST and CAPA results are the major
component used for calculating each school’s API. These results are
also used for determining if elementary and middle schools are making
adequate yearly progress in helping all students become proficient on the
state’s content standards as required by the federal No Child Left Behind
Act of 2001. (California Department of Education, 2007h, p. 6)
Comparing CST Results
When comparing results for the CSTs, results only within the same
subject and grade should be compared; that is, grade two English-
language arts (ELA) compared to grade two English-language arts, or
grade six Mathematics compared to grade six Mathematics. No direct
comparisons should be made between grades or between content areas.
Two types of comparisons are possible:
54
1. Comparing the average scale score, or
2. Comparing the percentage of students scoring at each performance
level.
The reviewer may compare results for the same grade and subject across
years within a school, between schools, or between a school and its
district, county, or the state. When making comparisons, the reviewer
should consider comparing the percentage of students scoring proficient
and advanced, since the state target is for all students to score at or above
proficient. Comparisons may also be made by calculating the overall
percentage of students within a school who scored proficient and
advanced and comparing this to the overall percentage of students in
another school, the district, the county, or the state who scored proficient
or advanced. To do this, first calculate the number of students who
scored proficient and advanced for the subject area at each grade level
[(%PRO + %ADV) * number tested for the grade and subject area =
number scored PRO/ADV]. Then add the number scored PRO/ADV for
all grades and divide by the total enrollment. (California Department of
Education, 2007h, p. 12)
Equating and Scaling
Educators and parents need to be able to compare the test results of
students who take different forms of the same test, such as forms
administered in different years but in the same content area and grade
level. It is important, therefore, that results are reported in a way that
assures that a given score indicates the same level of knowledge or skill,
regardless of the form taken. To accomplish this, number-correct scores
from different forms are aligned through a process called scaling and
equating and are reported as “scale scores.” (California Department of
Education, 2007h, p. 20)
Equating is a statistical procedure that is used to adjust scale scores on
different test forms so that scores can be compared in an accurate manner. Raw
scores, or number of correct scores, from different test forms cannot be fairly
compared because the test forms can be of different difficulty. Scale scores are
55
comparable across forms of the same test because differences between the
forms—such as in average test question difficulty—have been taken into
account” (California Department of Education, 2007h). Equating and scaling
are described in more detail in the California Standards Tests Technical Report.
The technical reports are posted at California Department of Education (2007b).
Qualitative
Qualitative data were collected in the form of informal interviews,
observations, and scaled questionnaires. The administrator and the researcher
had the opportunity to interview teachers related to their perception of the
effectiveness of the training and their implementation and fidelity to the
instructional strategies they received in the AB 466 training. The administrator
and the evaluator had an opportunity to observe and compare first-hand
observations of the actual implementation of the instructional strategies and
practices implemented by the teachers in their classrooms. Finally, the evaluator
conducted a short survey that measured effectiveness and teacher beliefs of the
training and implementation of the training. The two documents are included in
Appendix A.
56
Instrumentation and Procedure
Observational Checklists
Observations of the classroom took place during English-language arts
and Mathematics lessons to determine the fidelity to the State Board of
Education (SBE) core curriculum in the core curricular areas. The following
checklist helped to calibrate and evaluate the level of implementation that was
observed:
1. Was the lesson observed presented taken directly from the SBE
adopted core curriculum text?
2. What was the student engagement?
3. What was the teacher engagement?
4. Was direct instruction or feedback observed?
5. Did students have the opportunity for extra support, if applicable?
The researcher and the administrator visited multiple classrooms on a
varying schedule during core curricular times. After the information observed
was calibrated and delineated, a professional dialogue took place between the
researcher and the administrator. A plan to share the information with the staff
was developed.
57
Informal Interviews
Informal interviews of teachers took place during and after the
observation process. Teachers were given an opportunity to dialogue
professionally with the researcher and the administrator of the school. The topic
of discussion was the information gathered from the observation visits and
checklists. The informal nature of the dialogue gave the researcher and the
administration the opportunity to document feedback in a less structured manner
to promote a reduced level of anxiety. Prompts similar to the following were
used:
1. We observed your class today during a lesson and this is what we
observed (reference to notes and checklists from observation). How do you feel
this is benefiting student achievement?
2. Was the lesson an activity acquired from your recent AB 466
training?
3. Did the AB 466 training increase your comfort with the SBE
approved curriculum?
4. How would you rate on a scale of one to five the
relevance/applicability of the AB 466 training to your daily lessons?
5. How would you rate on a scale of one to five the effectiveness of the
AB 466 training on the increase of student achievement at Eva B. Elementary
School?
58
When the results from the interviews were collected and organized, a
professional dialogue took place between the researcher and the administrator to
determine the manner in which to share the information most effectively with
the Eva B. Elementary staff.
Scaled Questionnaires
Scaled questionnaires were implemented to the entire teaching staff at
Eva B. Elementary. The questionnaires were in reference to the AB 466 training
and its effectiveness in the classrooms of the teachers on a daily basis. When
the information was collected and calibrated, the results were shared with the
staff and administration of the school.
The scaled questionnaires were on a rated scale from one to five, one
being strongly agree and five being strongly disagree, with varying degrees of
agreement and disagreement along the scale. The following are a couple of the
examples from the questionnaire:
1. Was the AB 466 training relevant to your day-to-day classroom
activities?
2. Will the AB 466 training increase student achievement?
3. Do you have an increased understanding and ability to apply the SBE
approved curriculum to students of Eva B. Elementary?
59
4. Did the AB 466 training increase your understanding and ability to
implement and utilize differentiated instruction in your lessons?
5. Did AB 466 training have the most impact on instruction in your
classroom compared to the nine EPC?
The questions created an opportunity for professional dialogue at the
school between the staff, administration, and the researcher. With teacher
quality having a major affect on student achievement, the goal of the process
was to assist the Eva B. Elementary staff specifically, while supporting staffs
throughout the district, in determining the effects of the level of teacher training
on teacher quality. Providing an opportunity to support teachers by providing
them with the appropriate and necessary tools to teach, and fostering their
opportunities to sharpen skills to improve student achievement and their skill
level in the classroom was the goal of this effort. “Not only should our students
have high self-efficacy about their ability to succeed in the classroom, but so,
too, should we, as teachers, have high self-efficacy about our ability to help
them succeed” (Ormond, 2006, p.346)
Procedural Timeline
The following timeline was adhered to in order to complete the
evaluation in the timelines proposed. The timeline was maintained for data
collection and evaluation results:
60
September, 2007: Quantitative data in the form of the CST results from
Eva B. Elementary School were recorded. Interviews and observations of
teachers were scheduled.
October, 2007: Data analysis utilizing SPSS on the CST scores from the
school were implemented. Interviews and observations of teachers in the
classrooms were conducted. Surveys for teachers were conducted during a staff
meeting.
November 2007: Data collected from interviews, surveys, and
observations were interpreted and analyzed and results were defined.
Data Analysis
The following sections define the manner in which the data derived from
the CST scores were analyzed by comparing and evaluating the results. CST
scores from 2005 to 2007, both internally and externally, were the basis for a
qualitative analysis. Internally, the CST scores were evaluated and compared
for differences from 2005 to 2007. Externally, the scores were compared to
Ramirez Elementary School—a similar school as defined by the CDE.
The data from interviews, surveys, and observations of the students and
staff at Eva B. Elementary were complied and evaluated. This quantitative
analysis gave the researcher an opportunity for professional dialogue, and a first-
hand examination of the processes that had been implemented in the classrooms
61
at the school. This effort offered a personal view of the efforts of the teachers
and feedback from students, as they attempted to increase student growth and
achievement.
Qualitative Analysis
The interpretation of the qualitative data compiled for this study was
around Kirkpatrick’s model of compiling data which Champion (2002)
transformed to become user friendly in education. Champion outlined the major
points of impact as follows:
1. Reaction
2. Learning
3. Use
4. Results
Each of these areas draw on the various approaches to collecting data to make
the research user friendly and applicable.
Quantitative Analysis
The measurement of the intervention and its effectiveness and statistical
significance was applied in a dependent groups t-test. The measurement of
practical significance was accomplished through the application of the Cohen’s
d. Percentage gain and difference was complied and displayed in a 2 by 4 effect
62
size display. An attempt to maximize effect size and effectiveness was
extrapolated.
Delimitations and Limitations of the Study
The limitations in this study can be related, but not limited, to
instrumentation, selection, intervention Type 1 errors, generalizability, and
external validity. Each of these areas of concern have specific issues related to
the limitations and validity of the study. The instrumentation, CST, is the one
that has the most validity as it is a measure developed by Educational Testing
Systems and supported by the California Department of Education. The
measures of security and resources related to validity are of the utmost
importance in the state’s efforts to support this assessment. The selection limits
are related to the attendance area of Eva B. Elementary School, which in and of
itself, is a limitation. The intervention of AB 466 training may be limited
because of presentation and application fidelity. Type 1 errors have a significant
opportunity to create limitations when dealing with an assessment system of
such significance with the CST program. An attempt to maximize external
validity through the use of disaggregated data from the California Department of
Education related to the STAR results from Eva B. Elementary School was
applied in this study.
63
The concerted efforts of the state administration at CDE and the district
to implement, support, and provide current training and professional
development through the School Assistance and Intervention Team process and
the AB 466 training to increase student achievement at Eva B. Elementary
School could be seen as a limitation. The intervention to provide the growth and
academic achievement of students has been supported by AB 466 training. The
results and evaluation of the CST assessments provide validity to the success of
this endeavor.
64
CHAPTER 4
RESULTS
Design Summary and Results Overview
The summative evaluation of the study incorporated three dependent
variables and both a pre/post independent group design and a non-equivalent
comparison group design for Eva B. Elementary School. The nonequivalent
group design was used for comparing the California Standardized Test (CST)
results of students at Eva B. Elementary (experimental group) to a state defined
similar school for Eva B. Elementary in 2005, Ramirez Elementary (benchmark
group). In 2005 Ramirez Elementary was identified as a similar school on the
Eva B. Elementary STAR report (California Department of Education, 2007i).
Ramirez had a similar number of students completing the test and achieved an
API score of 788 in 2005. Ramirez surpassed the state goal in API of 800 in
2007 with a reported API score of 811.
The dependent variables were: CST English-Language Arts (ELA) and
CST Math performance band scores, the percentage of students who scored
“basic and above” on the CST ELA and the CST Math, and the percentage of
students who scored “proficient and above” on the CST ELA and CST Math.
The performance bands were coded as follows:
0 = Far Below Basic, 1 = Below Basic, 2 = Basic, 3 = Proficient, and 4 =
Advanced.
65
Pre/post independent groups design: This design was implemented to
analyze the change at the experimental school (Eva B. Elementary) from 2004-
05 (pre-intervention) to 2006-07 (post-intervention). The following statistics
were executed for the CST ELA and CST Math performance band scores: (a) an
independent groups t-test to assess the statistical significance of the change (p <
.15), (b) Cohen’s d to assess practical significance (criterion for practical
significance (d >.10), (c) raw change from 2005 to 2007 to assess practical
significance, and (d) percentage change to assess practical significance (criterion
for practical significance = 10% improvement).
Nonequivalent comparison group design: This design includes an
experimental group and one comparison group that was not randomly assigned.
The experimental and control groups were compared on the post-test CST data.
The treatment was administered only to the experimental group. The treatment,
AB 466 training in the State Board of Education-adopted curriculum in English-
Language Arts and Math, was administered in a 2-year cycle. In the case of Eva
B. Elementary School, the SBE in English-Language Arts was the Houghton-
Mifflin curriculum and the SBE in Mathematics was the Scott Foresman
curriculum. The English-Language Arts training took place in 2004-2005 and
the Math training took place in the 2005-2006 school years.
66
Eva B. Elementary, ELA Results
Table 5 shows the pre/post test statistical findings (p< .15) for the
experimental school (grades 2-5) in English-Language Arts.
Table 5
Pre- Versus Post–Intervention CST ELA Performance Band Differences:
Statistical Findings
Grouping
Pre N
2005
Post N
2007
Pre M
2005
Post M
2007
Difference
t-ratio
Observed p
School 232 213 1.84 1.95 .11 1.1 .29
Grade 2 53 59 1.53 1.66 .13 .63 .53
Grade 3 59 58 1.24 1.69 .45* 2.3 .02*
Grade 4 59 45 2.42 2.60 .18 .96 .34
Grade 5 61 51 2.13 2.02 -.11 -.55 .58
*p < .150
The results in row 1 indicate that overall in the experimental school,
there was significant increase in the CST ELA performance bands between the
2005 and 2007 assessment period, t(444) = 1.1, p < .150. Statistically
significant results (p < .150) were found in only the third-grade subgroup,
however, there were positive increases in ELA student achievement at all grade
levels except grade 5. Among the third-grade students, the observed gain from
2005 to 2007 was .45 (nearly one-half of a performance band growth), t(116) =
2.3, p = .22. Although a significant amount of growth occurred at the third-
grade level, it is significant to note that the pre-mean scores of those students
67
were almost a quarter percent lower than the next closest level assessed. The
results overall identify a level of growth realized significantly at the third-grade
level and that growth, along with consistent positive scores at the other grade
levels, supported the overall growth for the school in English-Language Arts
collectively.
Eva B. Elementary, Mathematics Results
The information shared in Table 6 represents a comparison of student
assessment results from the 2005-2007 CST, in the area of Mathematics. The
pre/post significance test findings (p < .15) for the experimental school (grades 2
through 5) in Mathematics are compared in Table 6.
Table 6
Pre- Versus Post-Intervention CST Math Performance Band Differences:
Statistical Findings
Grouping
Pre N
2005
Post N
2007
Pre M
2005
Post M
2007
Difference
t-ratio
Observed
p
School 233 213 2.24 2.23 -.01 -.02 .98
Grade 2 54 59 2.17 2.12 -.05 -.22 .53
Grade 3 59 58 1.85 2.31 .23 2.0 .05*
Grade 4 59 45 2.58 2.31 -.30 -1.2 .23
Grade 5 61 51 2.33 1.82 -.50 -2.4 -.02
* =p < .150
68
Table 6 row 1 identifies the lack of change in the area of Mathematics
school wide. The one subgroup that realized a significant amount of growth in
the area of Mathematics as identified in Table 6 was third grade. The levels of
significant growth found (p < .150) was .23 performance band points in third
grade t (113) = -2.0, p = .05. Of concern was the significant loss in fifth-grade
Mathematics. Students realized a loss of a half of a performance band .50, t
(102) = 2.4, p = .02, in fifth-grade Mathematics. The growth in third grade,
although significant, exhibits a similar caveat as in the area of English-Language
Arts. The third-grade students’ pre-scores were more than half a point lower in
Mathematics, in comparison to the other grade-level Mathematics’ achievement.
The increase brought the third-grade achievement to a level consistent with the
other grades in the school. The loss in fifth-grade Mathematics is of significant
concern as the students lost almost half of a performance band of achievement in
this subject area. The loss in fifth grade was not significantly offset by the gain
in third grade as the other grades stayed relatively consistent, thus the school, as
a whole, experienced a small loss in Mathematics achievement as measured by
the CST assessments.
69
Eva B. Elementary ELA Performance
Band Summary
Because statistical significance is highly dependent on sample size, and
very small differences on a practical level can be statistically significant, the
size of each difference was assessed in three ways: raw change from 2005 to
2007, effect size, and percentage change. The raw change is the post-test score
minus the pre-test score. Effect size was computed implementing the ratio of
the change from 2005 to 2007 to the pre-test standard deviation. Percentage
change was assessed using the ratio of the change from 2005 to 2007 to the pre-
test mean. Results of those assessments for English-Language Arts are shown
in Table 7.
Table 7
Pre- Versus Post-Intervention CST ELA Performance Band Differences:
Practical Significance
Grouping
Pre M Pre SD
Pre-Post
Change
Effect
Size
Percent
Change
School 1.84 1.13 .11 .10 .06
Grade 2 1.53 1.15 .13 .12 .09
Grade 3 1.24 1.04 .45 .36* .38*
Grade 4 2.42 .855 .18 .21* .07
Grade 5 2.13 1.06 -.11 -.11 -.05
* = Effect size > .20 and % change > .10.
70
Table 7 shows that the pre-test means in English-Language Arts at Eva
B. Elementary School are near 1.00 at the second- and third-grade levels. This
delineation shows that a majority of the students at those levels were performing
between the “below basic” and the “basic” level in 2005. The students in fourth
and fifth grades showed mean results above 2.00 which identifies that the
preponderance of students at those grade levels were performing at the “basic”
level in 2005. Identifying statistically the entire experimental school, the
students were performing as follows in English-Language Arts in 2005: 5% at
“Far below basic,” 21% at “Below basic,” 32% at “Basic” level, 27% were at
the “Proficient” level, and 5% of the students were at the “Advanced” level on
the CST ELA pretest.
Table 7 also displays the practical significance for each of the targeted
groups. Practical significance was examined in three ways:
Raw Change: In third grade the degree of ELA improvement from 2005
to 2007 was .46, nearly a half of a performance band increase. No other grade
level displayed this level of increase in the ELA results.
Effect Size: Two groups exceed the pre-study standard of .20—
specifically third grade (.36) and fourth grade (.21). The effect size of the third-
grade students was more than one-third of a standard deviation. An increase in
fourth grade was identified although not as significant. The increases at these
71
two grade levels assisted in the overall growth of the school’s achievement in
English-Language Arts
Percent Change. Another manner in which to assess practical
significance is to examine the percentage change in the performance band scores
from pre to post. The preset standard for practical significance was 10%. The
last column in Table 7 shows that there was a 10% increase from 2005 to 2007
in only one subgroup, third grade. The growth in third grade, 38%, was of
significance, however the reader should take into consideration the low level of
achievement at the starting point that had been identified in the 2005 pretest
results.
Eva B. Elementary Mathematics Performance
Band Summary
The practical significance results in Mathematics for Eva B. Elementary
School are identified in Table 8. The manner in which each was calculated is
consistent with the scores in English-Language Arts and is shared in the text
preceding those tables of identified results:
A significant characteristic evident in Table 8 is that the pretest means at
all grade levels fell at or near 2.00, which is identified as level “basic.”
72
Table 8
Pre- Versus Post-Intervention CST Mathematics Performance Band Differences:
Practical Significance
Grouping
Pre M Pre SD
Pre-Post
Change
Effect
Size
Percent
Change
School 2.24 1.15 -.01 -.00 -.00
Grade 2 2.17 1.26 -.05 -.04 -.02
Grade 3 1.85 1.19 .23 .20 .13*
Grade 4 2.58 1.02 -.27 -.26* -.10
Grade 5 2.33 1.06 -.50 -.48* -.22
* = Effect size > .20 and % change > .10
Students on the average were scoring at the “basic” levels during the pre-
test era in 2005. The following is the percentage breakdown for all students in
the area of Mathematics: 6% were scoring “Far Below Basic,” while 21% were
“Below Basic,” 29% were identified “Basic,” 27% were identified as
“Proficient," and 16% of the student population at Eva B. Elementary School
scored “Advanced” in the area of Mathematics.
Practical significance for Mathematics was also evaluated in three ways:
Raw Change, Effect Size, and Percentage Change as shared above in Table 7.
The result for each of the subgroups is as follows:
Raw Change: In the subject area of Mathematics, the degree of
improvement was significant in third grade (.23), which equals an increase of
almost one-fourth of a performance band at that grade level . Fifth (-.50) and
fourth (-.27) grades showed significant losses in the area of Mathematics. All
73
grade levels exhibited a loss in the area of Mathematics, except for third grade.
As previously shared, the increase in third-grade levels, although significant,
brought the grade level to the school-wide average of achievement in
Mathematics that had been realized in the pre-test of 2005 by the other grade
levels at the school.
Effect Size: Third grade (.20) met the pre-study standard of .20. Fourth
grade (-.26), and most significantly fifth grade (-.48) exhibited a practically
significant loss as compared to their 2005 pre-test standard deviation. The effect
size at fifth grade was nearly one-half of a standard deviation loss. The decrease
in achievement in Mathematics was consistent across the grade levels, save for
third grade.
Percent Change: The preset standard for practical significance as
identified in the ELA section above was 10%. The last column in Table 8
illustrates that three of the four subgroups showed practical significance in the
area of Mathematics, two were losses however. The third grade at .13 showed
the only growth. Fourth grade (.10) and fifth grade (.22) significantly exhibited
a loss in this measure. These losses in grades four and five were a concern. The
level of achievement remained consistent at the school-wide level considering
the losses at three of the four grade levels.
74
Summary of NCLB Results at Eva B. Elementary
in ELA and Mathematics
Tables 9 and 10 summarize the display of pre/post results for two
additional indices: percentage of the students who scored basic and above and
percentage of the students who scored proficient and above in ELA (as per
NCLB).
The results from the study of ELA CST scores at Eva B. Elementary
School indicated overall that the level of achievement remained consistent from
the pre-test (2005) to the post-test (2007) in the percentage of student achieving
basic and proficient levels of achievement on the CST assessments.
The results indicate limited growth and a loss in the areas of “Basic” and
“Proficient” as determined by the CSTs from 2005 to 2007. The gain of
students achieving a level of “Proficient” or above in ELA doubled in the third
grade from 2005 to 2007. The gain of (.11%) in fourth grade is also of note.
Tables 11 and 12 share information that compare levels of students who
reach “Basic” or “Proficient” levels on the CST results in the subject area of
Mathematics. The results indicated a very limited level of change in the area of
“Basic and above” in Mathematics CST proficiency. In contrast, the third and
fourth grades’ growth in “Proficient and above” students was extremely positive
with a 38% and 32% increase, respectively.
75
Table 9
Pre- Versus Post-Intervention CST ELA Percent Basic and Above
Grouping Pre (2005) Post (2007)
Pre-Post
Change
Percent
Change
School .63 .62 -.02 -.03
Grade 2 .51 .53 .02 .04
Grade 3 .39 .55 .16 .41
Grade 4 .90 .91 -.01 -.01
Grade 5 .72 .68 -.04 -.05
Table 10
Pre- Versus Post-Intervention CST ELA Percent Proficient and Above
Grouping Pre (2005) Post (2007)
Pre-Post
Change
Percent
Change
School .32 .36 .04 .13
Grade 2 .25 .22 -.03 -.12
Grade 3 .12 .24 .12 1.00
Grade 4 .44 .49 .05 .11
Grade 5 .44 .33 -.11 -.25
76
Table 11
Pre- Versus Post-Intervention CST Math Percent Basic and Above
Grouping Pre (2005) Post (2007)
Pre-Post
Change
Percent
Change
School .31 .31 .00 .00
Grade 2 .66 .53 -.13 -.20
Grade 3 .52 .55 .03 .06
Grade 4 .90 .91 -.01 -.01
Grade 5 .77 .68 -.09 -.12
Table 12
Pre- Versus Post-Intervention CST Math Percent Proficient and Above
Grouping
Pre (2005) Post (2007)
Pre-Post
Change
Percent
Change
School .43 .42 -.01 -.02
Grade 2 .45 .37 -.08 -.18
Grade 3 .32 .44 .12 .38
Grade 4 .44 .58 .14 .32
Grade 5 .49 .32 -.17 -.35
77
Similar School Comparison Results
The final results compare the level of growth in the experimental school,
Eva B. Elementary School, to the control school, Ramirez Elementary School, in
API scores from 2005 to 2007 (Table 13).
Table 13
API School-wide Comparison, 2005 and 2007
School API 2005 API 2007 Increase
Eva B. Elementary School 693 725 +32
Ramirez Elementary School 788 811 +23
Ramirez Elementary School was chosen as a comparison school from the
list of 100 similar schools provided by the California Department of Education
that provides the Similar Schools Report for 2005 Academic Performance Index
(API) Base Report (California Department of Education, 2007i). Ramirez
Elementary was chosen for its geographical proximity to Eva B. Elementary. As
with all similar school reports, the predicted similarity of the two institutions is a
limitation of the actual similarities. Thus, the ability to compare the two schools
is limited.
The increases at both schools were positive. The increase at Eva B.
Elementary, although larger, did little to close the gap between the two schools.
78
The increase at Ramirez was not specifically studied, however, the achievement
of a school at those levels, with all things consistent, would be an expected
increase of achievement consistent with or above those levels previously
accomplished.
Table 14 compares the two schools in the measure of Adequate Yearly
Progress (AYP) for their federal accountability in student achievement
(California Department of Education, 2007j). Table 14 charts the level of
growth in the experimental (Eva B. Elementary) and control (Ramirez
Elementary) schools related to AYP. Both schools exhibited a level of growth
in this measure from 2005 to 2007. When comparing these results with the
levels of growth or decline in the measurements of practical or statistical
significance, the outcomes are varied.
Table 14
AYP School-wide Comparison, 2005 and 2007
Eva B. Elementary
2005 Levels of
Percent Proficient
2007 Levels of
Percent Proficient
Difference
English-Language
Arts
32 36 +4
Mathematics 43 42 -1
Ramirez
English-Language
Arts
46 56 +10
Mathematics 57 65 +9
79
The experimental school increased both in the area of API and AYP
although not showing significant growth in the areas measured for statistical or
practical significance.
Overview of Results
The increase prevalent in multiple areas of achievement in English-
Language Arts and Mathematics for Eva B. Elementary School, covering the
two years of the study, provided a level of growth that fostered improvement in
the levels of the students’ achievement at “Proficient” and above. The student
achievement in the two core-curricular areas was significant enough to provide
an increase in AYP of 4% in ELA and 2% in Mathematics. The increase was
positive and provided support to the growth that was recognized through these
measures at Eva B. Elementary School.
What is the conclusion of the results derived from this study on the
student achievement and growth in English-Language Arts and Mathematics at
Eva B. Elementary School? The information indicates a CDE reported level of
growth both in the experimental school’s API and AYP scores. The practical
and statistical significance, as measured by a pre/post independent groups design
and a nonequivalent comparison group design, showed little or no change in
student achievement. There was some significant change at various grade
levels; however, the results shared limited, if any, overall significant growth in
80
student achievement. The determination of reportable significance was very
different in those measures when comparing to the evaluation of results as
determined by the schools’ API and AYP scores.
81
CHAPTER 5
CONCLUSION
Overview
The level of teacher quality and its effect on the growth of student
achievement at Eva B. Elementary School are examined in this study. A
comparison of California Standards Test scores before and after teachers had
received extensive training in the California State School Board of Education
adoption of curriculum in the core areas of English-Language Arts (ELA) and
Mathematics, Essential Program Component four, was identified as the
intervention for the improvement of student achievement at Eva B. Elementary
School. The following is a synopsis of that quantitative data and a discussion of
qualitative findings. This chapter also contains recommendations for
improvement in student achievement for the students enrolled at Eva B.
Elementary School.
Eva B. Elementary had been labeled a Program Improvement (PI) school
and participated in the process of being supported by a School Assistance and
Intervention Team (SAIT) by virtue of their students’ scores on the annual
California Standards Test (CST). Although the school had shown progress in
the area of student achievement through an increase in their API scores, the
growth was not at the significant annual rate expected by the Federal NCLB act
82
(United States Department of Education, 2002). This assessment landed Eva B.
Elementary on the list of schools that were provided extra support and training
in the State Board of Education (SBE) adopted curricular materials in ELA and
Mathematics, while being labeled a PI school and taking part in the SAIT
process.
The process of SAIT provided, first and foremost, very specific and
comprehensive training in the SBE-adopted curricular materials that teachers at
Eva B. Elementary School were implementing in their classrooms in the core
curricular areas of ELA and Mathematics. The training included all teachers
working in the classrooms at Eva B. Elementary and provided them with an in-
depth examination of the curricular materials, as well as strategies for the most
effective implementation of lessons in the core curriculum that support student
growth in an effort to increase student achievement. The nine components have
been identified in numerous research studies as key factors for school
improvement and for the functioning of schools that are “beating the odds” by
demonstrating success with challenging student populations (California
Comprehensive Center and American Institute of Research, 2006
p. 2).
There are nine essential program components that were developed by the
California Department of Education and they are to be implemented at schools
83
not attaining the necessary levels of achievement as determined by NCLB
standards.
By focusing on the instructional core-curriculum and instruction, and
supports for curriculum and instruction, the essential program
components aim to ensure the School Assistance and Intervention Team
(SAIT) process remains focused intensively on the improvement of
student achievement in key academic subject areas. (California
Comprehensive Center and American Institute of Research, 2006 p. 2)
The purpose of this study was to determine the effectiveness of the
fourth essential program component, which specifically states the following :
“Fully credentialed teachers and teacher participation in AB 466 training on
SBE-adopted instructional materials” (California Comprehensive Center and
American Institute of Research, 2006 p. 6).
Purpose and Design
The purpose of the study was also to determine the effectiveness of the
mandated teacher training, essential program component four, on student
achievement in ELA and Mathematics at Eva B. Elementary School. The initial
determination was to evaluate the effectiveness of program improvement
through the measurement of CST scores after teacher training had occurred for
the staff of Eva B. Elemenary. The comparison of pre to post intervention CST
scores would assist in the determination of the program effectiveness after the
training was implemented through the SAIT process, as a portion of the
implementation of the nine essential program components.
84
A collection of interviews, observations, and survey were used to address
the qualitative nature of inquiry. This offers a significant insight into the
effectiveness of the measured program from the point of view of the staff and
administration. In addition, the effectiveness of the fourth essential program
component and teacher training provided by the SAIT process was evaluated
using school achievement on the CST before and after the intervention.
CST scores from a similar school were also used in the evaluation. The
goal was to address the differing conditions of varied situations in an attempt to
determine if other mitigating factors outside of the intervention have an effect,
positive or negative, on the outcome of the intervention implemented,
specifically teacher training in the SBE-adopted core curricular areas. “While
we know a great deal more than we once did about how people learn and how to
teach effectively, much of this knowledge is only haphazardly available to those
who most need it to do their work” (Darling-Hammond & Baratz-Snowden,
2007, p. 113). Ramirez Elementary school, a similar school, as defined by the
California Department of Education (CDE), was chosen because it is
demographically similar, as determined by CDE similar-school standards, and
located within a geographical proximity to Eva B. Elementary School.
85
Summary of Findings
Eva B. Elementary School
The following is an overview of the data that were analyzed to determine
the effectiveness of the intervention applied and the research question of this
study: What is the effectiveness of the fourth essential program component,
specifically AB 466 teacher training in the core curriculum, to student
achievement at Eva B. Elementary School? In measuring achievement using
CST scores, the increase at Eva B. Elementary was large. In the 2 years
covering the study, Eva B. Elementary’s API increased by 32 points. The base
API score of 693 (2005) increased to 725 (2007), a substantial two-year growth.
The level of students achieving at the “Basic” and “Proficient” benchmarks also
increased a slight percentage over the 2 years the study covered.
The increase of API scores and the increase of students identified as
achieving “Basic” and “Proficient” supported the positive outcomes for the
students and staff at Eva B. Elementary School. The increase in third grade
scores in ELA and Mathematics was significant statistically and accounted for
the overall growth of achievement at Eva B. Elementary.
Specifically, analyzing the data by grade-level scores, and comparing
their statistical findings in ELA for the students at Eva B. Elementary School, all
grade levels showed an increase in achievement except for fifth grade.
Although improvement took place in three of the four grade levels accessed, it
86
was only statistically significant (p<.15) at third grade (p=.02). It is also
important to note the increase at third grade brought that grade to a similar
achievement level at which the other grade levels had consistently achieved,
suggesting in part that the third grade increase was due to their low performance
in the base year.
The statistical data in Mathematics indicated a loss of achievement at
three of the four grade levels assessed. The level of growth that was statistically
significant (p<.15) was again at third grade (p=.05). Although a loss was
evident at the other three grade levels, only the fifth grade (p=.02) achievement
decline was statistically significant (p<.15). The decrease in Mathematics
achievement scores school-wide amounted to a small difference of -.01 overall
from 2005 to 2007 at Eva B. Elementary School.
Using effect size and percentage to assess practical significance, the gain
in ELA achievement at third grade was significant (ES=.36, PC=.13). The
fourth grade (ES=.21) also showed a significant growth in effect size, but did
not yield a significant percentage growth in ELA. As previously shared, growth
in three of the four grade levels was evident, however the practical significance
of growth in ELA was only evident in both effect size and percentage change at
third grade and in the significant effect size increase at fourth grade.
In analysis of the area of achievement in ELA for the students of Eva B.
Elementary School, three of the four grade levels showed improvement in their
87
CST scores when comparing 2005 results with those accomplished in 2007. The
students in third grade in 2007 exhibited the largest amount of growth, and the
school, as a whole, improved its achievement. The students in fifth grade
showed a decrease in achievement.
In defining the practical significance data for Mathematics, the growth in
third grade (ES = .20) (PC = .13) showed a significant effect size and percentage
growth for that level. However the other three grade levels showed a decrease
in growth and negative practical significance. Fourth grade achieved a
significant effect size change (ES=.26) and a positive percentage change
(PC=.10). Fifth grade decreased almost half a performance band change and
realized a 22% negative growth change. Although there were three levels of
negative change, the school lost only .01 in overall achievement in Mathematics.
This would lead one to determine the training and application of the
Mathematics curriculum would need to be reevaluated and upgraded to reverse
the drop off in negative Mathematics achievement scores.
The ELA increase in third grade is surprising, as historically students in
the third grade had shown a consistent or negative level of achievement on the
CST. The manner in which the test is administered at second grade, mostly
teacher read, compared to third grade where students are responsible for reading
and understanding the text individually, is the consideration for this
phenomenon. The fifth grade fall-off can also be attributed to a qualitative
88
consideration in the level of understanding students have at fourth grade as
compared to fifth grade when taking the CST. Consideration for the level of
comprehension growth necessary in one year is critical to the success of
achievement being assessed in fifth grade.
The results in Mathematics were surprising, as well as the qualitative
findings that acknowledged the fact that staff felt that the training in
Mathematics was more relevant and applicable to their daily lessons and
implementation of the curriculum when compared to the training experienced in
ELA. The third grade again showed the most significant improvement in
achievement, increasing almost a quarter of a performance band, while the fifth
grade students exhibited a decrease of almost a half of a performance band.
The increase at third grade over the 2-year span was an increase that
supported the consistency of achievement at Eva B. Elementary School, while
the other grade levels experienced a decrease across the board in Mathematics.
The results again defy convention as the third grade students increased their
achievement significantly. As stated above, the transition of the test
implementation from second to third grade is significant and consistently has
had a negative effect on CST results in third grade statewide.
In further analysis, as students historically scored markedly better in the
universal language of numbers, it is important to take into consideration the
effects of their second language backgrounds and lower SES environments. It
89
may be more realistic, fruitful, and effective for the goal of NCLB to take into
consideration a model that recognizes schools and students that move from one
level to the next in a progressive growth model, as opposed to the current all or
nothing stakes of meeting proficient levels in all areas regardless of limitations
related to student populations. This type of model, I am almost certain, would
also improve teacher morale and effectiveness as the results recognize both the
growth of their students and the achievement of the necessary benchmarks.
AB 466 Training Summary
The findings shared by staff were enlightening as well. In the survey the
staff completed which related to the perceived effectiveness of the AB 466
training in the SBE adopted core-curriculum, the results were of note.
Consistently teachers at Eva B. Elementary School, when asked the level of
effectiveness of the AB 466 training, shared the fact that the training in
Mathematics was much more personable and relevant to their day-to-day
application than those in ELA.
This information would not be consistent with the level of achievement
that was reported by the CST in Mathematics. The students of Eva B.
Elementary consistently increased their level of achievement in ELA. Three of
the four grade levels assessed showed improvement, and the school, as a whole,
90
improved in ELA achievement, as compared to the results in Mathematics where
three of the four grade levels experienced a decrease in assessed achievement.
The perception of the training shared by the teachers, although relevant,
did not have an effect on the results realized by the students in the area of
achievement in Mathematics. Specifically the fifth grade students exhibited a
decrease in achievement of nearly one half of a performance band in
Mathematics. Third grade realized an increase while second and fourth grades
joined the fifth grade class in a decreased level of achievement. The school,
collectively, displayed a decrease of achievement in the area of Mathematics.
Ironically, when taking into consideration the responses of the teachers
in the survey and the interviews, the results heavily favored the effectiveness
and quality of the Mathematics training to the training that was presented in
support of the ELA curriculum. Comments shared by staff consistently
confirmed that the ELA curriculum training presentation was a review of what
was already being implemented in their classrooms, and the minutia of the
training hampered its effectiveness. Trainers were covering information they
were mandated to cover, as opposed to allowing teachers’ input on what they
felt would be the most relevant manner of implementation of the training to
support an increase of achievement in their classrooms.
Teachers also felt the group being trained was not consistent and did not
provide them the opportunities to work with others at their specific grade level
91
or school to collaborate the most effective approach to implementation in their
classrooms. Specifically one teacher shared, “There was, in my opinion, an
extreme disconnect between what the teachers in the training thought would be a
relevant and effective use of our efforts as opposed to what had to be covered
for the trainers to be compliant. It was extremely frustrating.” The
appropriateness of the training model may be what needs to be evaluated in this
case.
Other comments consistently shared by teachers addressed the timing of
the ELA training in respect to their classrooms and the cycle of the ELA
adoption. One teacher commented, “We had received the curriculum and
training in Houghton-Mifflin ELA, just the summer previous. We were anxious
to expand on what we learned in our first year of implementation, but what took
place was a review of the training we had received the summer before.”
Another teacher commented, “It was too hard on the school and on the students
for such a large number of teachers to be gone for such a long amount of time
(five consecutive days out of the classroom), for what was actually
accomplished in this training.”
The feedback shared in reference to the Mathematics training was
markedly different. Teachers consistently stated that they felt it was the most
effective and relevant of the two trainings they attended. The comments were
again related to the cycle of the adoption and its correlation to the time frame in
92
which the training was presented. Most teachers had had little or no previous
training in the Scott-Foresman Mathematics curriculum that was being
implemented at Eva B. Elementary School. Teachers also shared that the
familiarity with the presenters of the Mathematics curriculum helped support the
effectiveness of the training.
Teachers shared the following comments, “The ability to work with our
grade level and school partners assisted in the ability to apply and make relevant
the time and efforts that were expended in the Mathematics training.” “The
presenters heard our needs and addressed them. This was particularly important
for two reasons. First, because we had a bad taste in our mouth from the ELA
training. Second, because we really needed time to collaborate with the
information and curriculum that was being shared. The time that was given,
along with the format of working directly with our school colleagues allowed for
an increased effectiveness of the Mathematics training.” “Having had the
experience of the ELA training, we went into the Mathematics training better
prepared. Better prepared to have our needs met and better prepared to make the
training applicable to our daily instruction and student needs.”
After reviewing the surveys and notes from the interviews, it was evident
that the teachers had a much more beneficial experience in the Mathematics
training than the ELA training. This was consistently attributed to the timing of
the two trainings and to the effectiveness of the trainers’ ability to address the
93
needs of the teachers. The timing I refer to is the timeframe in which the cycle
of adopting the particular curriculum has been implemented. When considering
the training in ELA with the Houghton-Mifflin curriculum, teachers felt the
training was a repeat and not an expansion of what they had very recently
experienced when the curriculum was introduced to them the year prior to their
AB 466 training taking place. On the other hand, the teachers interviewed had
little experience and training in the Mathematics adoption that was being shared.
The teachers also felt the trainers of the Mathematics curriculum were more
supportive of their needs and input. The training provided relevance through the
ability to share ideas and information with colleagues from the school and at
grade level. This time these opportunities assisted in the planning and
implementation of the core curriculum in Mathematics to their daily lessons and
the manner in which they were going to present the information to their students.
I recognized the need for relevance and application of the curricular
materials for the teachers. It was important for teachers to have time to make
arrangements during the trainings to decide in which manner it would be most
effective to apply the curriculum presented. This information is directly
supported by Reeves’ research in the success of underachieving schools. He
states, “In the course of many sessions, the most effective schools made time for
collaboration very frequently and in some cases did this every day” (Reeves,
2003, p. 9). Teachers of first and second grade were allowed the time to
94
collaborate and developed a Mathematics universal access in the training to
better assist their students on a daily basis with support in specific areas that
would assist in increased student achievement. Given this information, it may
be more relevant; in the teachers’ perception, to evaluate the effectiveness of
trainings and the direct effect on teacher efficacy and student achievement.
Ironically, the results on the CST do not assess the results the teachers
experience related to the effectiveness of the two AB 466 trainings in the core
curricular areas and their affect on student growth and achievement. This
consideration, in the long run, could be more reflective of the efforts and effect
of the AB466 training on student growth and achievement.
Eva B. Elementary School and
Ramirez Elementary School.
The nonequivalent control group design was applied to compare post-test
data between a control group, Ramirez Elementary School, and the experimental
group, Eva B. Elementary School. Ramirez Elementary School was chosen as
the control school because it was on Eva B. Elementary School’s similar school
list in 2005 as defined by the California Department of Education. The
intervention of AB 466 training, teacher preparation in the SBE-adopted core
curriculum of English/Language Arts and Mathematics, was the intervention
95
that was implemented and evaluated at the experimental school, Eva B.
Elementary.
Eva B. Elementary showed a positive amount of growth school-wide in
its Academic Performance Index (API), a measure calculated by the state
Standardized Testing and Reporting (STAR) accountability system. Between
2005 and 2007, the increase at Eva B. Elementary School was a total of +32
points growing from a 2005 API of 693 to a realization of 725 in 2007. The
control school, Ramirez, also experienced an increase, although not as
significant, of +23 points from 2005 to 2007. Their API was 788 in 2005 and
increased to 811 in 2007. API scores range from 200 to 1000 and the goal for
schools in the state of California is a score of 800 or an increase of a minimum
of 5% annually.
Eva B. Elementary had an increase in three of the four grade levels of
assessment in English/Language Arts. The collective effort resulted in an
overall increase in the percentage of students achieving at a level of “Proficient”
or above in ELA from 32.2% in 2005 to 35.9% in 2007. This was an increase of
3.7% in the 2007 assessment.
The students at Eva B. Elementary School also experienced an increase
in the number of students achieving a level of “Proficient” or above in
Mathematics. The increase in the percentage of students accomplishing these
96
results grew from 43.4% in 2005 to 45% in 2007. Although the increase was
small, 1.6%, it was a positive increase nonetheless.
The results were also positive for the control school as Ramirez
Elementary school also had positive growth in the percentage of students
achieving at a level of “Proficient” or above in both English/Language Arts and
Mathematics. The increase in ELA in 2007 was 9.5% and an improvement of
8.7% in Mathematics.
Although Eva B. Elementary experienced a larger increase in API than
Ramirez Elementary, 32 points as compared to 23 points, the reverse was true
when comparing the percentage of students accomplishing a level of
“Proficient” or above. Ramirez exhibited a larger increase in their percentage of
student growth than students at Eva B. Elementary, an increase of 9.5% for
Ramirez as compared to 3.7% for Eva B. Elementary in ELA and an 8.7%
increase in Mathematics for Ramirez as compared to Eva B. Elementary’s
increase of 1.6%.
The results demonstrate the difficult manner in which the STAR results
evaluate the increase in student achievement on the state assessment. One
school increased more significantly in API score while the other achieved a
higher level of growth in students exhibiting competency at a level of
“Proficient” and above. The need to delve more deeply to understand and create
significance of the results from these assessments is of extreme importance. The
97
ability to identify which interventions and practices are more noteworthy and
effective in the increase of student achievement is of tremendous value to the
educational system. In the case of these two schools, the fact that they were
both able to show significant growth when implementing strategies and practices
specific to their populations to increase student achievement and exhibiting that
growth is of significance.
Implications
The increase of achievement for the students at Eva B. Elementary
school was of note. The process of this growth continues to transform the
school site. Was the implementation of the AB 466 training significant? Yes, in
a varying degree of effectiveness and in the different core-curricular areas. The
staff divulged in surveys and interviews their opinions of the AB 466 training
and its effectiveness in the application of the core-curricular areas. The
trainings received both positive and negative feedback; however, the results as
measured by Eva B. Elementary’s CST scores did not directly reflect the
perception shared by teachers related to which trainings they thought most
effective.
Teachers also shared other areas of support that they claim have been
more conducive to the support and increase in student achievement.
Consistently the staff shared that grade level and school-wide collaboration has
98
made their job and their direction in supporting students very consistent and
unwavering. Darling-Hammond and Baratz-Snowden (2007) concur, “It also
means finding ways for teachers to learn about practice in practice, so that
concrete applications can be made and problems of practice can be raised,
analyzed, and addressed” (p. 115). One teacher shared “I can focus on a skill set
and strategy with the students I work with in universal access, because we have
identified their specific needs during our grade-level collaboration time.”
Another teacher shared why effective collaboration does happen, “Yes, we were
under the watchful eye of the SAIT process; however, the trust between my
colleagues and me was the factor that made for efficient data driven
collaboration and effective universal access time that supported growth for our
students.”
These comments directly support the findings stated earlier in the
research by the CACC, “no one component should be seen as a silver bullet to
improvement” (California Comprehensive Center and American Institute for
Research, 2006, p. 2). Teachers understand and recognize the need for
curriculum fidelity and competence; however, they also recognize that they
alone cannot get the job done.
The administration has noticed that most of the teachers and staff at Eva
B. Elementary understand the need for school improvement and support those
efforts. The visits to the classrooms confirmed this as teachers and students
99
were engaged in meaningful instruction that supported the students at their level
and in the core-curricular areas. Time and effort in the core curriculum were
consistent and presented in a manner which provided students the ability to
accomplish the goals and expectations presented.
There are still a few naysayers that remain; however, they are being won
over by the positive results the school is accomplishing. Nothing can be more
effective and conducive in changing direction than positive results of efforts in
place, and those positive results are taking place at Eva B. Elementary School.
The consistency and effectiveness of the results and the ability of those results
moving Eva B. Elementary out of Program Improvement status is proof for all
that effective practices and strategies can have a consistently positive effect on
student learning and achievement. Research also supports the effective
application of data in changing these attitudes and efforts, “Data can be a
foundation for school-wide examination of patterns. Properly used, data can be
a compelling means of launching, sustaining, and institutionalizing a reform
effort” (Johnson, 2002, p. 35).
The result to consider most noteworthy was the increase in growth in
third grade. In both ELA and Mathematics, third grade achievement at Eva B.
Elementary school increased notably. ELA experienced a statistical significant
(p =.022) increase of .45 in mean over their 2005 achievement, nearly one-half
of a performance band. In Mathematics the growth was also statistically
100
significant at third grade (p = .048) with an increase in mean of .23 over their
2005 achievement. This constitutes almost a quarter of a performance band of
improvement in Mathematics. The proficiency levels in third grade ELA
increased .12 which constituted a 1.00% increase of students achieving
proficiency. The .12 level of growth in Mathematics at third grade represents a
.38% change in the number of students producing at a level of Proficient and
above. This level of growth in both core-curricular areas is significant and
represents a positive increase at a grade level that historically has not
transitioned well in the state assessment.
The district, as a whole, has done well in assessments in third grade as
reported by the Assistant Superintendent of Curriculum and Elementary
Instruction. The growth at third grade, although significant, allowed the
students at that grade level to meet the school-wide norm of achievement that
had been consistently realized at the other grade levels at Eva B. Elementary
School.
When sharing these results with the administrator at Eva B. Elementary,
she confirmed that when comparing the group of third grade students in 2005 to
the group of third grade students in 2007, their levels of achievement can be
attributed to various factors. The 2005 group was more transient and had more
behavioral issues than the 2007 group. The administrator also pointed out the
same group from 2005 was the group that dropped almost a half of performance
101
band in Mathematics as fifth graders. This group of students seemed to have a
noticeable level of aptitude challenges, and those results were present and
notable in assessment results throughout their progress through the grade levels.
Site-Based Recommendations
The limitations of this study being considered, the implementation of the
AB 466 training in the core curriculum of English/Language Arts and
Mathematics can be credited for a portion of the results of improvement in the
achievement and growth in student learning. This information was confirmed
quantitatively with an increase in three of the four grades levels’ growth in the
ELA assessment and qualitatively by the results shared by the teachers who had
experienced the training. Although positive and negative feedback was shared
regarding the training by teachers, positive results were evidenced by the
observation of effective standards-based instruction taking place in the
classrooms. The outcome of the surveys and interviews with teachers showed a
variation of the effectiveness of the AB 466 trainings in the core curriculum;
however, the resulting behavior of consistent, standards-based instruction
observed in the classroom reinforced the importance of fidelity to the instruction
of the core curriculum in the classroom.
Effective training in the core curriculum is essential in the consideration
of teacher quality and the effectiveness of instruction in the classroom. It is
102
important that when the State Board of Education adopts a curriculum in the
core areas, that prudent and substantial training be offered in those curricular
areas. Teacher preparation and teacher quality go hand in hand, Ingersoll (2007)
states, “this is a crucial issue because highly qualified teachers may become
highly unqualified if they are assigned to teach subjects for which they have
little training or education” (p. 6). Appropriate training and support in the core
curriculum is essential in the development of student achievement and growth at
all levels. Teachers found the importance of the AB 466 training and applied it
to their daily practices; it was very evident in the observations of the classrooms
and the instruction that was implemented. When considering the research
related to effective instruction and effective instructional practices, Marzano
(2007) shares the following,
In short, research will never be able to identify instructional strategies
that work with every student in every class. The best research can do is
tell us which strategies have a good chance (i.e., high probability) of
working well with students. Individual classroom teachers must
determine which strategies to employ with the right students at the right
time. In effect, a good part of effective teaching is an art. (p. 5)
The passion and energy teachers shared in the interviews that took place
after the administrator and I had made classroom observations demonstrated the
importance the teachers placed on their efforts in the classrooms. As previously
stated, the feedback related to the AB 466 training was across the board in
relation to effectiveness; however, the application of the SBE-adopted core
curriculum was evident. “The quality of teachers and teaching is undoubtedly
103
an important factor in shaping students’ growth and learning” (Ingersoll, 2007 p.
20).
It is also important to continue the efforts of support and collaboration
between teachers in their grade levels as well as school-wide. Ingersoll(2007)
continues to support teacher mentoring and collaboration, “teachers who have a
mentor or support from a teacher in the same field; had common planning time
or regularly scheduled collaboration with other teachers and had regular and
supportive communication with administration, collectively, had a very large
and statistically significant impact” (Ingersoll, 2007 p.6). This confirms the
aforementioned caveat that individually the EPC can be effective, however the
EPC were developed to be implemented collectively to have the most significant
impact on student growth and achievement. This point was clearly stated at the
outset and the research continues to conclude that the claim is valid and
accurate.
A continuance of the strategies for teacher training, collaboration, and
the application of the core curriculum in predetermined and consistent time
frames can have a statistical and practical significance on the growth of student
learning and achievement at Eva B. Elementary School. This recommendation
has been supported by the information complied in this research effort and
supported by the response and observation of the teachers placing these sound
educational strategies and practices into action on a daily basis in their
104
classrooms, and in their school collectively and consistently as Eva B.
Elementary School strives to increase student achievement and reach its goal of
being removed from the state Program Improvement and SAIT process.
Limitations
The validity of the study must take into consideration multiple internal
and external factors. Internally the study must reference the following issues:
selection bias, student mobility, and sample size. Externally the instrumentation
and the application of multiple interventions and their affect on the outcomes of
the study should be considered when generalizing the study’s findings.
In this case, the most significant factors related to internal validity are
selection bias and sample size; however, the sample size limitations were
addressed in applying an independent t-test to establish statistical significance
and indices of effect size and percentage change to assess practical significance.
Selection bias was the most significant internal validity factor. When
dealing with independent samples, growth cannot be concluded due to the
intervention unless the people are randomly assigned. The application of
practical significance partially addressed this issue. Sampling error may explain
the results. The mobility factor of students also plays a major factor in the
validity of the assessment in this study.
105
External validity begins with the use of implementing the California
Standards Test (CST) as the instrumentation of the pre/post test evaluation. The
consistency at grade levels of the assessment, as well as the variability of the
participants, does not accurately allow the research and information to be
directly reflective of the achievement of one group at grade level as compared to
another in differing years of assessment. These factors alone could be the
considerations for the level of growth realized from the second to third grade in
this study, both in ELA and Mathematics from 2005 to 2007.
Multiple interventions and the implementation of the other eight EPC
play a major part in the external validity of the effect of AB 466 training on
student growth and achievement. The information the teachers shared
continually supported other factors in concert with the AB 466 training they feel
had significance on the accomplishment of student growth and achievement on
the students at Eva B. Elementary School. Validity, in general, is a significant
consideration when attempting to duplicate this study to any outside
implementation and or application.
Conclusion
Was teacher quality affected by the implementation of AB 466 training?
Did student outcomes improve because of increased understanding of the core
curriculum by teachers after the training was implemented? When conducting
106
this study, these and many other questions came into consideration to determine
the effectiveness of the teacher training intervention implemented. One thing
was repeatedly confirmed by the research and was shared in conclusions of
Marzano’s (2007) work, “Among elements such as a well-articulated curriculum
and a safe and orderly environment, the one factor that surfaced as the single
most influential component of an effective school is the individual teachers
within that school” (Marzano, 2007, p. 1).
The implementation of the AB 466 training alone cannot be the only
intervention considered when deeming whether the students at Eva B.
Elementary School benefited from this intervention specifically. The
consideration of all of the Essential Program Components must be taken into
account when addressing the level of achievement experienced through the
timeline of the study.
Collectively, the growth and progress of student achievement at Eva B.
Elementary has been significant in English/Language Arts. Students have also
maintained a consistency of achievement in the area of Mathematics. These two
findings suggest the implementation of practices, strategies, and the nine
essential components in place at Eva B. Elementary School will continue to
support further the efforts of teachers to assist students in the growth of learning
and achievement.
107
REFERENCES
Alexander, P.A. & Murphy, K.P. (1999). The research for APA’s learner-
centered psychological principles. In N. M. Lambert & B.L. McCombs
(eds.), How students learn: Reforming schools through learner centered
education (pp 25-50). Alexandria, VA: Association for Supervision and
Curriculum Development.
Bamburg, J.D. (1994). Raising expectations to improve learning. Oak Brook,
IL: North Central Regional Educational Library.
California Comprehensive Center and American Institute for Research. (2006).
Research summary supporting the nine essential program components
and Academic Program Survey. San Francisco: WestEd.
California Department of Education (2006). Principal training. Retrieved April
22, 2007, from http://www.cde.ca.gov/fg/fo/profile.asp?id=194.
California Department of Education. (2007a). SAIT corrective actions Title I.
Retrieved April 22, 2007, from
www.cde.ca.gov/fg/fo/profile.asp?id=716.
California Department of Education. (2007b). Program resources and
supplemental information for the STAR program. Retrieved April 22,
2007, from www.cde.ca.gov/ta/tg/sr/resources.asp.
California Department of Education. (2007c). Standardized testing and
reporting information. Retrieved April 22, 2007, from
http://api.cde.ca.gov/reports/API/2005Grth_SchDem.asp?SchCode=6033
286&DistCode=67330&AllCds=34673306033286.
California Department of Education. (2007d). Standardized testing and
reporting information. Retrieved April 22, 2007, from
http://www.cde.ca.gov/ta/tg/sr/.
California Department of Education. (2007e). Standardized testing and
reporting information Retrieved December 23, 2007, from.
http://api.cde.ca.gov/reports/API/APISearchName.asp?TheYear=&cTopi
c=API&cLevel=School&cName=williamson&cCounty=&cTimeFrame=
S.
108
California Department of Education. (2007f). California Standards Tests
English-Language Arts Blueprint Preface. Retrieved January 19, 2008,
from (http://www.cde.ca.gov/ta/tg/sr/documents/elapreface1105.doc).
California Department of Education. (2007g). California Standards Tests
Mathematics Blueprint Preface. Retrieved January 19, 2008, from
(http://www.cde.ca.gov/ta/tg/sr/documents/elapreface1105.doc).
California Department of Education. (2007h). Post-test guide technical
information for STAR district and test site coordinators and research
specialists. Retrieved January 19,2008, from
(http://www.startest.org/pdfs/STAR.post-test_guide.2006.pdf).
California Department of Education. (2007i). Similar Schools Report 2005
Academic Performance Index (API) Base Report. Retrieved December
23, 2007 from,
(http://api.cde.ca.gov/APIBase2006/2005BaseSchSS.aspx?allcds=34673
306033286).
California Department of Education. (2007j). 2007 Adequate Yearly Progress
Report. Retrieved January 19, 2008 from,
(http://data1.cde.ca.gov/dataquest/APR/APRSearchName.asp?TheYear=
&cTopic=AYP&cLevel=School&cName=EvaB.&cCounty=&cTimeFra
me=S).
Champion, R. (2002). Taking measure: Choose the right data for the job.
Journal of Staff Development, 23 (3), 78-83.
Clark, R. E. &Estes, F. (2002). Turning research into results: A guide to
selecting the right performance solutions. Atlanta, GA: Center for
Effective Performance Press.
Corallo, C., & McDonald, D. (2001). What works with low-performing schools:
A review of research literature on low-performing schools. Charleston,
WV: Appalachia Educational Laboratory, Inc.
Darling-Hammond, L. (1997, November). Doing what matters most: Investing
in quality teachers. New York: National Commission on Teaching and
America’s Future.
109
Darling-Hammond, L. & Baratz-Snowden, J. (2007). A good teacher in every
classroom: Preparing the highly qualified teachers our children
deserve. Washington, DC: The National Academy of Education
Committee on Teacher Education, Educational Horizons.
Darling-Hammond, L. & Youngs, P. (2002, December). Defining “highly
qualified teachers:” What does “scientifically based research” actually
tell us? Educational Researcher, 31 (9), 13-25.
Ingersoll, R. M. (September, 2007). Short on power, long on responsibility.
Educational Leadership, Teachers as Leaders 65, (1), 20-25.
Johnson, R.S. (2002). Using data to close the achievement gap: How to
measure equity in our schools. Thousand Oaks, CA: Corwin Press
Kirkpatrick, D. (2001). The four level evaluation process. In L.L. Ekens (Ed.),
What smart trainers know: The secrets of success from the world's
foremost experts (pp. 122-132). San Francisco: Jossey-Bass/Pfeiffer.
Marzano, R. J. (2003). What works in schools: Translating research into
action. Alexandria, VA: Association for Supervision and Curriculum
Development.
Marzano, R. J. (2007). The art and science of teaching: a comprehensive
framework for effective instruction. Alexandria, VA: Association for
Supervision and Curriculum Development.
Marzano, R. J., Pickering, D., & Pollock, J. E. (2001). Classroom instruction
that works: Research-based strategies for increasing student
achievement. Alexandria, VA: Association for Supervision and
Curriculum Development.
Milanowski, A. T., Kimball, S. M., & Odden, A. (2005). Teacher accountability
measures and links to learning. In L. Stiefel (Ed.), Measuring school
performance and efficiency: Implications for practice and research.
Yearbook of the American Education Finance Association (pp. 1-37).
Larchmont, NY: Eye on Education.
Ormrod, J.E. (2006) Educational psychology: Developing learners (6
th
ed.).
Upper Saddle River, NJ: Merrill.
Patton, M.Q. (2002). Qualitative research and evaluation methods (3rd ed.).
Thousand Oaks, CA: Sage Publications.
110
Reeves, D.B. (2000). Accountability in action in blueprint for learning
organizations. Denver: Advanced Learning Press.
Reeves, D.B. (2003). High performance in high poverty schools: 90/90/90 and
beyond. Denver: Advanced Learning Press.
Schmoker, M. (2001). The results fieldbook. Alexandria, VA: Association for
Supervision and Curriculum Development.
Togneri, W., & Anderson, S. E. (2003). Beyond islands of excellence: What
districts can do to improve instruction and achievement in all schools.
Washington, DC: The Learning First Alliance and the Association for
Supervision and Curriculum Development.
United States Department of Education. (2002). No Child Left Behind Act of
2001. Retrieved April 22, 2007, from
http://www.nclb.gov/next/overview/index.html.
111
APPENDIX
DETERMINING THE EFFECTIVENESS OF AB 466
TRAINING ON STUDENT ACHIEVEMENT
1. Did the AB 466 training in English/Language Arts assist in your
effectiveness of presenting the instructional program with the Houghton
Mifflin curriculum?
1
Strongly
Disagree
2
Somewhat
Disagree
3
Neither Agree
or Disagree
4
Somewhat
Agree
5
Strongly
Agree
2. Did the AB 466 training in Math assist in your effectiveness of
presenting the instructional program with the Scott Foresman
curriculum?
1
Strongly
Disagree
2
Somewhat
Disagree
3
Neither Agree
or Disagree
4
Somewhat
Agree
5
Strongly
Agree
3. In your opinion did the AB 466 training increase student achievement?
1
Strongly
Disagree
2
Somewhat
Disagree
3
Neither Agree
or Disagree
4
Somewhat
Agree
5
Strongly
Agree
4. What is your belief in the following statement?
“Ultimately, students benefit most when a community of qualified teachers,
armed with professional development and a collective goal to raise
achievement, effectively deliver instructional programming to all students.”
1
Strongly
Disagree
2
Somewhat
Disagree
3
Neither Agree
or Disagree
4
Somewhat
Agree
5
Strongly
Agree
5. In your opinion has the SAIT process supported student achievement?
1
Strongly
Disagree
2
Somewhat
Disagree
3
Neither Agree
or Disagree
4
Somewhat
Agree
5
Strongly
Agree
112
Appendix: Continued
6. Was the AB 466 training relevant to your day-to-day classroom
activities?
1
Strongly
Disagree
2
Somewhat
Disagree
3
Neither Agree
or Disagree
4
Somewhat
Agree
5
Strongly
Agree
7. Do you have an increased understanding and ability to apply the SBE-
approved curriculum in English/Language Arts to the students of Eva B.
Elementary School?
1
Strongly
Disagree
2
Somewhat
Disagree
3
Neither Agree
or Disagree
4
Somewhat
Agree
5
Strongly
Agree
8. Do you have an increased understanding and ability to apply the SBE-
approved curriculum in Mathematics to the students of Eva B.
Elementary School?
1
Strongly
Disagree
2
Somewhat
Disagree
3
Neither Agree
or Disagree
4
Somewhat
Agree
5
Strongly
Agree
9. Did the AB 466 training increase your understanding and ability to
implement and utilize differentiated instruction in your lessons?
1
Strongly
Disagree
2
Somewhat
Disagree
3
Neither Agree
or Disagree
4
Somewhat
Agree
5
Strongly
Agree
10. Did the AB 466 training have the most impact on instruction in your
classroom compared to the other nine Essential Program Components?
1
Strongly
Disagree
2
Somewhat
Disagree
3
Neither Agree
or Disagree
4
Somewhat
Agree
5
Strongly
Agree
113
Appendix: Continued
Interview Questions
What was the most effective part of the SAIT process related to the nine
Essential Program Components in your opinion?
In your opinion do you feel the AB 466 training or the coaching/collaboration
was most effective?
Could the professional development (AB 466 training in ELA or Math ) have
been better tailored to meet your curricular needs?
We observed your class during a lesson. How do you feel that is benefiting
student achievement?
Was the lesson or activity we observed acquired in your AB 466 training?
How would you rate on a scale of one to five the relevance/applicability of the
English/Language Arts AB 466 training on your daily lessons?
How would you rate on a scale of one to five the relevance/applicability of the
effect on student achievement?
How would you rate on a scale of one to five the relevance/applicability of the
Mathematics AB 466 training on your daily lessons?
How would you rate on a scale of one to five the relevance/applicability of the
effect on student achievement?
Would you like to share anything related to your experience in the AB 466
training?
114
Abstract (if available)
Abstract
The purpose of the study was to determine the effectiveness of a teacher intervention and its application to the school-wide program at Eva B. Elementary. The goal was to determine the effectiveness of AB 466 teacher training, identified as the fourth Essential Program Component in the SAIT (School Assistance and Intervention Team) process, on student achievement. The situation that was presented placed Eva B. Elementary in the state program improvement process. The application of the nine EPC through the SAIT process were in process for 2 years at Eva B. Elementary and the researcher examined the effect of teacher training on student achievement as measured by the California Standards Tests (CST).
Linked assets
University of Southern California Dissertations and Theses
Conceptually similar
PDF
English learners' performance on the California Standards Test at Aviles Elementary
PDF
Raising student achievement at Eberman Elementary School with effective teaching strategies
PDF
Raising student achievement on the California Standards Test and California High School Exit Exam at the Phoenix Arts Charter School
PDF
The impact of restructuring the language arts intervention program and its effect on the academic achievement of English language learners
PDF
The effects of culturally responsive standards based instruction on African American student achievement
PDF
Distributed leadership practices in schools: effect on the development of teacher leadership - a case study
PDF
The effect of site support teams on student achievement in seven northern California schools
PDF
What is the relationship between self-efficacy of community college mathematics faculty and effective instructional practice?
PDF
The implementation of strategies to minimize the achievement gap for African-American, Latino, English learners, and socio-economically disadvantaged students
PDF
The effectiveness of the literacy for success intervention at Wilson Middle School
PDF
School-wide implementation of the elements of effective classroom instruction: lesson from a high performing high poverty urban elementary school
PDF
Examining the effectiveness of the intervention programs for English learners at MFC intermediate school
PDF
Achievement gap and sustainability: a case study of an elementary school bridging the achievement gap
PDF
Effective factors of high performing urban high schools: a case study
PDF
Evidence-based study of resource usage in California’s program improvement schools
PDF
Standards based report card: teachers' perception on the development, transition, and implementation
PDF
Examining the applications of data-driven decision making on classroom instruction
PDF
Sustainability of a narrowed achievement gap: A case study
PDF
Teachers as bystanders: the effect of teachers’ perceptions on reporting bullying behavior
PDF
Evaluation of the progress of elementary English learners at Daisyville Unified School District
Asset Metadata
Creator
Tapia, Richard Anthony
(author)
Core Title
Examining the effectiveness of teacher training on the improvement of California standardized test scores at Eva B. Elementary School
School
Rossier School of Education
Degree
Doctor of Education
Degree Program
Education (Leadership)
Publication Date
04/22/2008
Defense Date
02/21/2008
Publisher
University of Southern California
(original),
University of Southern California. Libraries
(digital)
Tag
Education,OAI-PMH Harvest,professional development,Teacher Training
Place Name
California
(states),
educational facilities: Eva B. Elementary School
(geographic subject)
Language
English
Advisor
Hocevar, Dennis (
committee chair
), Hexom, Denise (
committee member
), Stowe, Kathy Huisong (
committee member
)
Creator Email
ratapman@comcast.net
Permanent Link (DOI)
https://doi.org/10.25549/usctheses-m1182
Unique identifier
UC1273855
Identifier
etd-Tapia-20080422 (filename),usctheses-m40 (legacy collection record id),usctheses-c127-62707 (legacy record id),usctheses-m1182 (legacy record id)
Legacy Identifier
etd-Tapia-20080422.pdf
Dmrecord
62707
Document Type
Dissertation
Rights
Tapia, Richard Anthony
Type
texts
Source
University of Southern California
(contributing entity),
University of Southern California Dissertations and Theses
(collection)
Repository Name
Libraries, University of Southern California
Repository Location
Los Angeles, California
Repository Email
cisadmin@lib.usc.edu
Tags
professional development