University of Southern California Dissertations and Theses

A study of the relationship between student achievement and mathematics program congruence in select secondary schools of the Archdiocese of Los Angeles

(USC Thesis Other)

A study of the relationship between student achievement and mathematics program congruence in select secondary schools of the Archdiocese of Los Angeles

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content INFORMATION TO USERS This manuscript has been reproduced from the microfilm master. U M I films the text directly from the original or copy submitted. Thus, some thesis and dissertation copies are in typewriter face, while others may be from any type of computer printer. The quality of this reproduction is dependent upon the quality of the copy submitted. Broken or indistinct print, colored or poor quality illustrations and photographs, print bleedthrough, substandard margins, and improper alignment can adversely affect reproduction. In the unlikely event that the author did not send UMI a complete manuscript and there are missing pages, these will be noted. Also, if unauthorized copyright material had to be removed, a note will indicate the deletion. Oversize materials (e.g., maps, drawings, charts) are reproduced by sectioning the original, beginning at the upper left-hand comer and continuing from left to right in equal sections with small overlaps. ProQuest Information and Learning 300 North Zeeb Road, Ann Arbor, M l 48106-1346 USA 800-521-0600 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. A STUDY OF THE RELATIONSHIP BETWEEN STUDENT ACHIEVEMENT AND MATHEMATICS PROGRAM CONGRUENCE IN SELECT SECONDARY SCHOOLS OF THE ARCHDIOCESE OF LOS ANGELES by Mark Patrick Ryan A Dissertation Presented to the FACULTY OF THE ROSSIER SCHOOL OF EDUCATION UNIVERSITY OF SOUTHERN CALIFORNIA In Partial Fulfillment of the Requirements for the degree DOCTOR OF EDUCATION May 2002 Copyright 2002 Mark Patrick Ryan Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. UMI Number: 3073844 — _______ (B ) UMI UMI Microform 3073844 Copyright 2003 by ProQuest Information and Learning Company. All rights reserved. This microform edition is protected against unauthorized copying under Title 17, United States Code. ProQuest Information and Learning Company 300 North Zeeb Road P.O. Box 1346 Ann Arbor, Ml 48106-1346 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. UNIVERSITY OF SOUTHERN CALIFORNIA The Graduate School University Park LOS ANGELES, CALIFORNIA 90089-1695 This dissertation , w ritten b y Mark Patrick Ryan. Under th e d irection o f D issertation Com m ittee, an d approved b y a ll its m em bers, has been p resen ted to and a ccep ted b y The Graduate School, in p a rtia l fu lfillm en t o f requirem ents fo r th e degree o f DOCTOR OF PHILOSOPHY Dean o f Graduate Stu dies D ate / '/I ' ) n — DI SSER TA TION COMMITTEE w Chairperson - /\A/I Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Mark Patrick Ryan Dr. David Marsh ABSTRACT A STUDY OF THE RELATIONSHIP BETWEEN STUDENT ACHIEVEMENT AND MATHEMATICS PROGRAM CONGRUENCE IN SELECT SECONDARY SCHOOLS OF THE ARCHDIOCESE OF LOS ANGELES The purpose of this dissertation was to determine the relationship, if any, between student achievement gains and congruence in a sample of 11 high schools in the Archdiocese of Los Angeles. This study focused on research literature that analyzed student performance in mathematics in both the national and international arenas, as well as on reform efforts aimed at increasing achievement. Specifically, the study measured student performance on pre- and post-tests of first- and second-year algebra skills. A research-developed analysis instrument then analyzed the extent of vertical and horizontal articulation (termed congruence in the study) by those teachers regarding content standards, assessment strategies, and instructional methodologies. The study revealed that student performance in first- and second-year algebra classes is poor, with the highest performing students answering only less than half of the questions correctly on the Mathematics Diagnostic Testing Project (MDTP) tests of first- and second-year algebra skills. The study also revealed that varying degrees of congruence existed in participating schools. A strong positive correlation (r=.82) was I Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. found to exist between the level of congruence and the extent of mean student achievement gain on the MDTP tests - that is, schools in which significant efforts have been made at vertical and horizontal articulation are more likely to have higher student achievement gains. Implications for teachers, school site administrators, school district personnel, and Schools of Education are presented. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. TABLE OF CONTENTS DEDICATION AND ACKNOWLEDGEMENTS LIST OF TABLES LIST OF FIGURES CHAPTER 1 INTRODUCTION Background to the Problem Statement of the Problem Purpose of the Study Research Questions Theoretical Framework Definition of Terms Methodological Assumptions Delimitations Limitations Organization of the Study 2 REVIEW OF RELATED LITERATURE Introduction The Performance of American Secondary Students: Historically and From an International Perspective The Role and Importance of Math Education for Our Schools UC Expectations Reform in Mathematics Education High School/University Alignment and Continuity Conclusion 3 METHODOLOGY. Study Sample Instrumentation Data Collection Data Analysis Page No. ii vi 1 1 8 9 10 10 16 18 18 18 19 21 21 22 33 40 44 35 59 61 62 64 68 70 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. TABLE OF CONTENTS (continued). Page No. CHAPTER 4 FINDINGS Pre- and Post-Test Comparisons School Congruence Levels Summary Relationship Betwwen Student Achievement And Congruence Level Discussion 5 SUMMARY, CONCLUSIONS, AND RECOMMENDATIONS 109 Summary 109 Purpose 111 Research Questions 112 Methods and Procedures 112 Variables 113 Data Analysis 113 Selected Findings 114 Conclusions 117 Recommendations 118 72 74 80 95 96 105 REFERENCES 122 APPENDIX A B MATH STUDY COMMUNICATION MATH DEPARTMENT CONGRUENCE STUDY QUESTIONNAIRE MDTP TESTING INFORMATION 129 132 139 iv Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. LIST OF TABLES TABLE Page Is 1. National Assessment of Educational Progress Content Domain Distribution 22 2. NAEP Summary for Grade 12 (1990) 23 3. Achievement Levels by Subgroup: Grade 12 (1996) 24 4. Achievement Levels by Subgroup: Grade 12 (1992) 25 5. Achievement Levels by Subgroup: Grade 12 (1990) 26 6. Comparison of Average Scales Score by Year 26 7. NAEP Proficiency Levels 27 8. Participating Students, Teachers, and Classes in the Study 63 9. Algebra 1 Gain Scores by School 75 10. Algebra 2 Gain Scores by School 77 11. Total Gain Scores by School 79 12. Congruence Levels by School 82 13. Algebra I Achievement Gains in Rank Order with Congruence Levels. 97 14. Algebra 2 Achievement Gains in Rank Order with Congruence Levels. 98 15. Ranked Overall Gain Scores and School Congruence Levels 98 16. Congruence Levels and Overall Student Gains by Course 99 v Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. LIST OF FIGURES FIGURE 1. 2. Page No. Conceptual Model of Congurence Including Within-School and School-to-University Articulation 11 Summary Description of the Components Present at Each Level of Congruence 15 Congruence Level Characteristics 81 vi Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. CHAPTER 1 INTRODUCTION Background to the Problem Poor mathematics performance is nothing new in America. Since data on student performance were first collected in the late 1960’s and early 1970’s, American students have performed below par in mathematics and in practically every other area assessed by the National Assessment of Educational Progress (NAEP, Reese et al. 1997). To make matters worse, United States students’ math performance has improved only slightly since the data were first collected. The 1999 Digest o f Education Statistics reported that mean scores for all students on the NAEP have increased only 7.6 points (on a 500-point scale) since testing first began in 1973 (p. 139). On international comparisons outlined in the Digest, U.S. students scored at the international average in fourth-grade mathematics, 21st out of 26 countries in 8th grade mathematics, and 15th out of 21 countries in 12th grade mathematics (p. 448). The American College Board concluded in its study of algebra that high school graduates who had taken one or two years of algebra were more than twice as likely to go to college than those who had not (Hawkins, 1993). Because students who never make it into algebra are never afforded the opportunity to take more advanced math and science courses they, in effect, are excluded from the possibility of post secondary study immediately after high school. 1 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. In the early 1800’s Harvard, Yale and Princeton listed algebra as one of the required courses of admission. As a result, this was the first time algebra became part of the secondary curriculum. Educational psychologists, such as Thorndike, argued that students whose IQ’s were below 110 would “be unable to understand the symbolism, generalizations, and proofs of algebra.” In effect Thorndike’s view excluded more than two-thirds of the population from enrolling in algebra. The view Thorndike espoused, though not politically correct in the twenty-first century, is never the less operationalized in the vast majority of American secondary schools. Though most teachers and administrators blame failure in algebra on poor student ability, the Third International Mathematics and Science Study identified the lack of primary and intermediate-grade algebraic learning opportunities as a potentially significant factor in the poor performance of American eighth-graders. Rather than placing the onus on students and their lack of effort or skills, TIMSS maintained that systemic change was necessary in how and to what extent algebra is to be taught. It is no shock then, that research reveals algebra teachers emphasizing classroom management and covering the textbook rather than ensuring student achievement (Slovan, 1990). Since the vast majority of American students are not exposed to algebra until they enroll in a traditional Algebra I course in eighth or ninth grade, the abstractness of algebra causes many students to become frustrated and feel incapable of meeting set standards. 2 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. It is obvious that deficiencies in mathematics achievement are the result of systemic problems. The factors that help or hinder student achievement need to be at the forefront of discussions and analyses of those systemic problems. At the secondary school level, since algebra is normally the first college-prep course taken by students, the analysis should begin there. Various reform efforts have been undertaken during the past two decades to address the deficiency in mathematics achievement, particularly in algebra. Those reform efforts include attempts to identify exactly what mathematics should be taught, how it should be assessed, and how best to teach the variety of learners in American classrooms. The National Advisory Committee on Mathematics and Education argued as early as 1975 that increased math standards were the key to reform. Organizations such as the National Council of Teachers of Mathematics responded to such concerns by publishing content and performance standards which included the inculcation of algebraic thinking throughout a child’s elementary and middle school experience. Meserve and Suydam in their 1992 research identified algorithmic instruction as prevalent in mathematics classrooms because teachers and students feel pressure to achieve on standardized tests. Thus, problem solving is de emphasized and in some cases obliterated from the curriculum. Of course research such as that conducted by Wood and Sellers (1996) contest the view that algorithmic instruction must supplant problem solving. Exposure to a rich 3 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. problem-solving climate has been shown to raise the levels of student achievement on standardized assessments. Perhaps the most significant reform effort is characterized by Tucker and Codding when they argue that “standards, common assessment materials, instructional methods, and instructional technology” must be aligned with each other. Mathematics achievement reform efforts can be classified into three categories - those that have focused on “a return to basics,” those that have emphasized problem solving and mathematical thinking, and those that have argued for systemic changes in how mathematical learning is assessed. The “back to basics’ movement stresses algorithmic ability such as computation with whole numbers, decimals, fractions, and percents. Those who advocate the basics argue that standardized testing measures aptitude in these areas, so teaching should focus on these areas. Parent groups and legislators tend to be among the most vocal in this camp of reformers. The advocates for deeper mathematical thinking are led by professional organizations such as the National Council for Teachers of Mathematics (NCTM). Spurred on by research such as the Stigler Video Study of eighth grade mathematics classrooms in the U.S., Germany, and Japan, the NCTM and others argue that students need to be exposed to rich problem solving situations in substantially greater amounts than they do algorithmic instruction (Mumane & Levey, 1996). 4 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Furthermore, there are those who call for substantive changes in how mathematical knowledge and skills are assessed (Henningsen & Stein, 1997 and Stigler & Hiebert, 1998). They argue that the performance of mathematical tasks is more authentic and “real life.” Such tasks should involve collaboration and communication of mathematical thought rather than merely solving problems. Reformers in this camp argue that a paradigm shift in how mathematical thinking is assessed will result in a parallel paradigm shift in what is taught. Perhaps, the most compelling evidence in favor of one approach over the other has been reported by the Stigler Video Study (1997). Japanese teachers were found to approach teaching from a very different perspective than their American counterparts, specifically focusing on a small number of “bigger” problems, encouraging students to communicate their rationale for solving problems, and engaging students in meaningful discussion about solution strategies. The California and NCTM Frameworks for math instruction told us, long before TIMSS, that non-routine problems with high student interest, the focus on big mathematical ideas, and student reflection on teaching and learning were essential elements of efforts to improve student performance in math. (Clarke, 1997). The systemic reform required to support such a paradigm shift is extensive, however. NCTM articulates five components of such a shift: • toward classrooms as mathematical communities — away from classrooms as simply a collection of individuals; 5 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. • toward logic and mathematical evidence as verification — away from the teacher as the sole authority for right answers; • toward mathematical reasoning—away from merely memorizing procedures; • toward conjecturing, inventing, and problem solving - away from an emphasis on mechanistic answer-finding; • toward connecting mathematics, its ideas, and its applications - away from treating mathematics as a body of isolated concepts and procedures. (National Council of Teachers of Mathematics, 1991.) Essential to such reform efforts appears to be a dramatic change in classroom techniques. Firestone, Mayrowetz, and Fairman (1998) propose that performance- based assessment in mathematics become central, focusing only in part on student answers, focusing instead primarily on how students arrive at those answers. This paradigm shift faces many obstacles, including teacher reticence (Wood, Cobb, and Jacket, 1990), lack of professional development (Clarke, 1997), and the “anti assessment culture” that has pervaded so many of our schools. Despite the myriad factors affecting student achievement already noted, the premise of this study is that the most serious obstacle may be the lack of congruence and articulation between the K-12 educational system and the college/university community; K-12 education and colleges/universities have long been isolated from each other. A lack of articulation between what math is taught, how it is taught, and how it is assessed at both of these levels represents major 6 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. stumbling blocks to reform efforts. One significant attempt was begun in California over 20 years ago (MDTP, 1975). Firestone, Mayrowitz, and Fairman point out that colleges and universities have tremendous potential to impact the way mathematics is taught in the secondary classrooms. However, that potential impact is often ignored or devalued (1998). No other research can be found which explicitly examines the necessity for or value of articulation between secondary schools and colleges and universities. The University of California, California State University, and Community Colleges of California have established an impressive model of articulation based on the Mathematics Diagnostic Testing Project (MDTP) tests. Representatives from all three post-secondary levels in California gathered over 20 years ago to discuss the need for tools to diagnose student readiness for mathematics course work. The MDTP was the result of these discussions, and since the inception of the MDTP, nearly every UC, CSU, and Community College in California has utilized the MDTP Diagnostic tests as the metric against which students are measured for readiness to enter various levels of math instruction. Today in California, a student graduating from a secondary school who enters UC, CSU, or Community College in California has a greater than 80% chance of being administered an MDTP Diagnostic test to help determine placement in Freshmen mathematics courses (interview with Barbara Wells, MDTP). This model of cooperation, high expectations, and common assessment has made a significant impact on the design and function of high school and college math courses in California. 7 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Statement of the Problem Since American students at all levels clearly are not meeting national and international standards for mathematics performance, schools have been barraged with criticism for their “failure” to produce world class math students (Bracey, 1998). Colleges and universities blame secondary teachers for the lack of foundational mathematics skills possessed by incoming college freshmen. In turn, secondary teachers point the finger at elementary teachers, and elementary teachers point the finger at parents who did not provide adequate foundation for student learning. Since the stakeholders at no one level of schooling have yet to accept full responsibility, the mathematics community continues to be divided over how best to promote articulation and congruence between what is expected at various levels and how student achievement ought to be assessed. Clearly then there is a need for models of strong articulation between and among secondary mathematics teachers and their college/university counterparts. Such articulation needs to focus on three aspects of curriculum - what is taught, how it is assessed, and how it should be taught. Little has been written in the research literature about the importance of challenges of matriculation within a school and between schools. Researchers know that one systemic problem leading to poor math performance is the inclusion of too many topics within a single course (Tucker and Codding, 1998; NELS 1998, TIMSS Data). Therefore, there is a need to identify the most important topics on which to focus math instruction. But other questions regarding matriculation 8 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. remain. Is there a value in teachers discussing common sets of expectations (objectives) for a particular course? Are common assessment tasks of any use? Firestone, Mayrowitz, and Fairman (1998) argue that such common assessments can alter teaching behavior for the better (1998). Many schools face problems of students being ill prepared for the next course in a sequential math program. Is the lack of preparation something that can be countermanded by greater internal congruence between teachers and courses (Tucker & Codding, 1998)? This study will attempt to explore the two issues: a) matriculation between secondary schools and the university community; and, b) internal school congruence within mathematics programs. Purpose of the Study Specifically, this research analyzes the extent to which some secondary schools have achieved within-school and school-to-university congruence in their mathematics programs and the impact that congruence may have on student performance gains. The major goals of the study were twofold: • to assess the extent of within-school and school-to-university congruence in eleven secondary school mathematics programs as measured by survey and interview/observation data on each participating school; • to analyze possible relationships between levels of within-school and school-to university congruence and student achievement. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Research Questions This study investigates the following research questions: 1. What student performance gain exists in first and second year algebra courses for catholic high schools involved in the study? 2. To what extent is there mathematics instructional congruence within a sampling of Los Angeles Archdiocesan Catholic Secondary Schools? What are the levels of congruence within those schools? 3. What is the relationship between student performance gain scores and congruence levels within schools studied? Theoretical Framework This study operated under the conceptual framework that congruence is based on the two broad areas of within-school congruence and school-to-university congruence. Within-school congruence includes three components: common syllabi, common final exams and placements tests, and lesson caucuses (meetings of teachers to discuss teaching strategies and lesson plans for particular concepts.) School-to-university congruence consists of assessments which are linked to the expectations colleges have of their freshmen. The conceptual model is represented by Figure I. From this model, the study analyzed the extent to which secondary schools have achieved congruence in their mathematics programs. That assessment was 10 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. made using a five-level congruence scale developed by the researcher based on a thorough review of the relevant literature. FIGURE 1. Conceptual Model of Congruence Including Within-School and School-to-University Articulation. CONGRUENCE WITHIN-SCHOOL SCHOOL-TO-UNIVERSITY Common Syllabi Lesson Caucuses Assessment linked to college Expectations Common Finals/Placement Tests The highest level (five) exists in schools with curriculum, instruction, and assessment, which is common across classrooms within that school. Too, level five schools have an established program of articulation with the colleges and universities into which students matriculate. Level one schools experience tremendous differences in how mathematics is taught, what math concepts are taught and how student achievement is assessed. Schools in levels two, three, and four experience varying degrees of articulation between faculty members and with university faculty. Level One (Isolation). Individual classroom teachers at this level typically teach without any reference to standards and without input from peers or specific 11 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. knowledge of college expectations. Teaching is largely textbook driven. Students generally have little to no knowledge of outside assessments or the expectations colleges have for student achievement in mathematics. The same courses in a school may use different textbooks. Each teacher’s syllabus is developed independently without input from others; similarities of class organization are for the most part coincidental. The difficulty level of assessments may vary widely because different teachers will have different standards for student achievement. Teachers are generally unaware of the problems inherent in teacher isolation. Level Two (Partial Isolation). The elements of level one exist here, but teachers at this level of congruence recognize the value of interacting with external standards and with peers in the design of what is taught and how it is assessed. Some discussion exists between teachers; little common assessment or approaches to teaching has been achieved. This level might also be called the “readiness” level because, as opposed to level one where the teachers are unaware of the problems of isolation, these teachers recognize the value of discourse between teachers about what is taught, how it is taught, and how student achievement is assessed. Teachers at this level, however, still lack a vision for connecting what is done in the secondary school to what is expected at the collegiate level. National and state standards are largely ignored. Level Three (Beginnings of congruence). The teachers at level two who recognize the importance of in-school and/or external congruence become engaged in taking the first steps toward such congruence and placement on level three. 12 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Discussions begun at level two result in common syllabi with many common course expectations (Expected Course Learning Results- ECLRs), the use of identical teaching resource materials in some cases (such as worksheets, computer software, and activities), attempts at common assessment tasks and rubrics, and a common approach to the teaching of course content. Lesson caucuses to plan lessons on a particular topic or section of the text are held occasionally, with teachers increasing their level of comfort with peer evaluation and discourse. Though there are not yet common final exams, teachers share their own final exams with other teachers and input received from colleagues. There is growing consensus about what students ought to know and be able to do after taking the course, and a growing agreement about how to assess that knowledge and ability. National and state standards are seriously considered in discussion about course content. The expectations of college and universities are considered in the design of assessment; some placement testing occurs to insure proper assignment to course levels. Level Four (Solid Congruence). At this level, teachers utilize common final exams and are increasingly using lesson caucuses to plan teaching stratagems. There are common syllabi across the same course on a school campus with the same ECLRs and identical assessment tasks throughout the course, including a common final exam that is analyzed closely by the teachers and math department leadership for results and trends. The common final also impacts discourse about how to teach more effectively to maximize student results. Teachers agree on what 13 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. students need to know and do at the completion of the course, and they are in full agreement about how to assess student understanding and skill. National and state standards are relied upon as a firm foundation for course content. Placement tests are given for each course to insure proper assignment to a course that corresponds to student ability. These placement tests are loosely aligned with college placement tests for mathematics courses. Much discourse takes place about how to best prepare students for successful performance on college placement tests. Students are minimally aware of how their course work and performance in secondary schools links with and prepares them for success in college mathematics. Level Five (The Ideal). There is clear articulation here between the expectations of the secondary school college preparatory mathematics courses and those courses the students will experience in the colleges to which they matriculate. The components of level four exist, except that in addition, the placement tests given at the secondary school are aligned with those given at the collegiate level (in fact, they may administer the same tests - such as the MDTP in California). Placement tests exist for each level of college preparatory mathematics. Teachers and students are keenly aware of how the secondary course work prepares students for success in college. The expectations of the colleges and universities are matched to those of the secondary school (see University of California Academic Senate, undated). The operative definitions of what students ought to know and be able to do in Algebra 1, Algebra 2, Geometry, Pre-Calculus, and Calculus classes 14 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. are common between the secondary school and the colleges into which the majority of that school’s students matriculate. Teachers agree on how to best teach those concepts and skills and the assessments they utilize are themselves assessed for validity and reliability by the members of the department. In cases where validity and reliability of tests come into question, outside sources for checking these measures may be employed. Regular lesson caucuses are conducted in which teachers critically assess peer instructional strategies using agreed upon standards for teaching (Stevenson & Stigler, 1992; Stigler & Hiebert, 1997; Stigler, et al. 1997). Much discourse takes place about how to improve the teaching and learning of all students. The NCTM “Community of Learners” model exists both between teachers and students and between teachers and their colleagues (1992). Figure 2 shows the various components of congruence and their presence at each level. FIGURE 2. Summary Description of the Components Present at Each Level of Congruence. COMPONENT LEVEL 1 LEVEL 2 LEVEL J LEVEL 4 LEVEL 5 Instruction is textbook driven Yes Maybe Somewhat No No Assessment is textbook driven Yes Maybe Somewhat No No Awareness o f College Expectations by students and teachers No No Limited Somewhat Clear Common ECLRs No Some Yes Yes Yes Common Syllabi No No Yes Yes Yes Common Assessment Tasks No No Some Many All Common Final Exam No No No Possibly Yes Placement Exams No No No Some Yes Reference to National/State/University standards No No Limited Somewhat Fully Assessment aligned to college expectations No No Limited Somewhat Fully Lesson Caucuses No No Rare Limited Extensive 15 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Definition of Terms The following terms will be used throughout the study and require some operational definitions: Congruence: The Latin etymology for the term comes from that which means “a meeting.” For the purposes of this study, congruence refers to the alignment and continuity experienced within a schools’ mathematics courses - the meeting of minds between secondary teachers and their university counterparts. Congruence also refers to the extent to which a particular course is taught in the same way by more than one instructor. The extent to which common assessment is present in a course so that one teacher’s expectations for student achievement in a course are identical to the extent to which secondary school curricula is aligned to the expectations of the colleges and universities into which that school’s students matriculate. For the purposes of this study, the term congruence is meant to encompass the term ‘curricular alignment.’ Common Syllabi: A syllabus used by all teachers within a school teaching the same course. It includes clear course objectives, a description of course materials, and an outline of the order in which course material is presented. Lesson Caucuses: Meetings in which teachers teaching the same course gather to engage in common lesson planning, assessment of teaching effectiveness and strategies, assessment of student learning, and analysis of school-wide efforts at congruence. 16 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Common Finals/Placement Tests: A system of assessments that is common across classrooms of the same course. Teachers administer these exams to determine placement in the next level course and to assess student achievement of the stated course learning objectives. Horizontal Articulation: The communication that takes place between teachers teaching at different grade levels or course levels within a department. For instance, the discussions that take place between first and second year algebra teachers about what to teach, how to teach it, and how to assess it. Vertical Articulation: The communication which takes place between teachers teaching the same course. This communication is about such topics as what course expectations should be, how to bets teach concepts being studied, and how and when to assess student understanding and use assessment to inform instruction. Assessment: The process of determining the extent to which a student knows and is able to do the things expected in a particular course. Assessment includes such things as teacher-made tests, textbook-made tests, standardized tests, portfolios, performance tasks, teacher observation, peer observation, and writing which is scored using a rubric. Standards: Local, state or national guidelines about what students should know or be able to do after a particular course of instruction. 17 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Methodological Assumptions The following assumptions were made about the data upon which this study was based: 1. The math teachers who participated in the study responded with truthful and realistic information. 2. Math teachers abided by the conditions for test administration as provided by the UCLA Math Diagnostic Testing Project. 3. The interview and observation data were recorded and analyzed accurately. Delimitations The study was subject to the following delimitations: 1. Student performance data from the first and second year algebra classes is limited to students enrolled in those courses within the eleven schools studied. 2. The study was based on data obtained from the MDTP tests as well as from surveys, interviews, and observations made at participating schools. 3. The UCLA Mathematics Diagnostic Testing Project limited the researcher to summary classroom data rather than individual student results. Limitations 1. Surveys may only reflect the opinions and perspectives of the person filling out the survey. At least three math teachers at each school were surveyed, and 18 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. discrepancies in their responses were addressed in follow up interviews, some observations, and telephone calls. 2. Survey, interview, and observation data only reflect the participating eleven schools. Though those schools represent a cross section of socioeconomic, enrollment, and staffing variables with the Archdiocese, care should be taken when making generalizations to the entire catholic secondary school population. 3. Since catholic and public schools have very different organization structures, care should also be given to the generalizability of these data to public schools. Organization of the Study Chapter 1 includes background about math reform efforts, the statement of the problem, the purpose of the study, the questions being researched, and the delimitations and limitations of the study. Chapter 2 reviews literature on the role and importance of math education in our schools, considers the math performance of American secondary school students (both historically and from an international perspective), analyzes the key elements of recent and proposed math education reform, and considers views of the alignment and high/school-to-university continuity that should be attempted in secondary math education. Chapter 3 discusses the methodology for the data gathering and analysis used in this study. 19 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. CHAPTER 2 REVIEW OF RELATED LITERATURE Introduction Math skills such as number concept, computation, and problem solving are essential for success in the work force and in college. At best however, the mathematics performance of American secondary school students can be described as weak. Studies such as the Third International Mathematics and Science Study (TIMSS) paint a disheartening picture of our students’ achievement in mathematics. The myriad of reform efforts in mathematics education, including the alignment of high school and university math curricula, show promise toward improvement of that performance, but their implementation has been slow and incomplete. This chapter will examine the literature regarding: a) the mathematics performance of American secondary school students with respect to a wide range of indicators, historical trends and international perspectives; b) the role and importance of mathematics education in our schools, more specifically, why students should be proficient in mathematics and the type of mathematics to be studied; c) the key elements of recent and proposed reform in mathematics education in secondary school, including an integrated curriculum, proposed curriculum and 21 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. instructional improvements, use of AP and other structured courses, assessment strategies; and, d) views of alignment and high school/university continuity that might be attempted in secondary mathematics education. The Performance of American Secondary Students: Historically and From an International Perspective In 1996, the National Assessment of Educational Progress (NAEP) examined student performance in science, mathematics, reading, and writing, and in February of 1997 published the results showing student performance in these areas during the 1996 assessment. The published results also include analysis of trends in academic progress in each of these areas based on three similar studies in mathematics achievement conducted nationally since the inception of NAEP in 1969. The NAEP data in mathematics is collected at grades 4, 8, and 12. The NAEP assessment in 12th grade mathematics contained five broad content areas, with items distributed as outlined in Table 1. TABLE 1. National Assessment of Educational Progress Content Domain Distribution Number sense, properties, and operations 20% of the test Maintenance 15% of the test Geometry and spatial sense 20% of the test Data analysis, statistics, and probability 20% of the test Algebra and functions 25% of the test 22 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. The sample for NAEP’s grade 12 math assessment was a representative group of public and non-public school students, including geographically diverse areas, and probability samples of students within the schools tested. Nearly 7,000 (6,904) 12th grade students were tested nationally in 264 schools. Results for NAEP are based largely on three distinguishing levels of student achievement: basic (partial mastery of prerequisite knowledge and skills fundamental for work at that grade), proficient (solid academic performance - competency over challenging subject matter, including subject-matter knowledge, application of that knowledge, and analytical skills), and advanced (superior performance). A 0-to-500 scale was used in the assessments, with 288 being the minimum score for attaining the basic achievement level in 12th grade tests, 336 being the minimum for proficiency, and 367 being the minimum required for advanced status. Nationally, only 2% of American students scored at or above the advanced level, 16% scored at or above the proficient level, and only 69% scored at or above the basic level of achievement. That means 31% of American 12th graders scored below even the most basic level of competency expected of them in the NAEP assessments. Table 2 summarizes the 1990 results for grade 12. TABLE 2. NAEP Summary for Grade 12 (1990) Percent at advanced level 1% Percent at or above proficient level 12% Percent at or above basic level 58% Percent below basic level 42% 23 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Males are doing slightly better than females on these assessments, though both are doing abysmally. Black, Native American, and Hispanic students did significantly worse than their White counterparts, and Asian/Pacific Islander students did slightly better than their White counterparts. Non-public and Catholic schools did slightly better than their public school counterparts, and those who participate in Title I or the Free/Reduced-Price lunch program under-perform those who do not participate in these programs. The following table summarizes NAEP 12th grade performance. TABLE 3. Achievement Levels by Subgroup: Grade 12 (1996) SUBGROUP Advanced ACHIEVEM ENT LEVELS At/Above Proficient At/Above Basic Below Basic Gender: | Mate 3 18 7 3 Female 1 14 69 31 Ethnicity: W hite 2 20 79 21 Black 0 4 38 62 Hispanic 0 6 50 50 Asian/Pacific 7 33 81 19 American Indian 0 3 34 66 School Category: Public Schools 2 15 68 32 Non-Public 2 24 82 18 Catholic 2 20 79 21 SES: Title I Participant 0 1 25 75 N on-Title I 2 17 70 30 Lunch Eligible 0 4 40 60 N ot Lunch Eligible 3 18 74 26 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Student performance in 1992 and 1990 NAEP assessments was generally similar but slightly lower as indicated on Table 4. TABLE 4. Achievement Levels by Subgroup: Grade 12 (1992) SUBGROUP | 1 PROFICIENCY LEVELS Advanced At/Above Proficient At/Above Basic Below Basic Gender: I Male ; 2 15 64 36 Female ■ 1 13 63 37 Ethnicity: | White j 2 18 72 28 Black ! 0 2 34 66 Hispanic 0 6 45 55 Asian/Pacific j American Indian 4 not available 30 81 19 School Category: Public Schools ! 1 13 61 39 Non-Public 3 25 81 19 Catholic | 2 21 79 21 SES: i Title I Participant 0 6 29 71 Non-Title I 4 26 67 33 Lunch Eligible j 1 8 39 61 Not Lunch Eligible j i 5 30 71 29 The 1990 results were also generally lower than the 1992 and 1996 results, with the same basic patterns of males slightly outperforming females, Asian Pacific Islanders outperforming Whites and Whites significantly outperforming their Black and Hispanic counterparts, non-public school students outperforming public schools, and participants in Title I and Government-sponsored lunch programs under performing those who do not participate in these programs. 25 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. TABLE 5. Achievement Levels by Subgroup: Grade 12 (1990) SUBGROUP Advanced ACHIEVEM ENT LEVELS At/Above Proficient At/Above Basic Below Basic Gender: Male 2 15 60 40 Female 1 9 56 44 Ethnicity: White 2 14 66 34 Black 0 2 27 73 Hispanic 0 4 36 64 Asian/Pacific 5 23 75 25 American Indian not available School Category: Public Schools t 12 57 43 Non-Public I 12 65 35 Catholic 1 10 61 39 SES: Title I Participant 0 3 31 69 Non-Title I 3 26 74 26 Lunch Eligible 0 9 42 58 Not Lunch Eligible 3 30 75 25 The average scale score of 17-year-olds taking the NAEP math assessments has increased only slightly from 304 in 1973 to 307 in 1996, with slight drops in 1978, 1982, and 1994 (See Table 6). TABLE 6. Comparison of Average Scales Score by Year YEAR GRADE 12 SCORES 1973 304 1978 300 1982 299 1986 302 1990 305 1992 307 1994 306 1996 307 26 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. In summary, there is no significant gain in student achievement in grade 12 mathematics between 1973 and 1996. The NAEP Report Card also includes reference to five mathematical performance levels (150,200, 250, 300, and 350), with each increasing level representing a more complex level of algorithmic, understanding, problem-solving, and reasoning ability. Performance level 350 includes the ability to perform multi- step problems and algebra, level 300 is described as the ability to perform moderately complex procedures and reasoning tasks, and level 250 is the ability to perform numerical operations with proficiency. Level 200 is classified as beginning skills and understandings while level 150 is the ability to recall basic arithmetic facts. As can be seen from Table 7, the percentages of students reaching levels 350, 250,200, and 150 has remained constant since 1978. The number reaching performance level 300 has increased slightly between 1978 and 1996. TABLE 7. NAEP Proficiency Levels LEVEL PERCENT AT THE LEVEL IN THE FOLLOWING YEARS 1978 1982 1986 1990 1992 1994 1996 350 7 6 7 7 7 7 7 300 52 49 52 56 59 59 60 250 92 93 96 96 97 97 97 200 100 100 100 100 100 100 100 150 100 100 100 100 100 100 100 These data reveal that only seven percent of our high school seniors are graduating with multi-step problem solving and basic algebra skills. Considering the 27 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. complexity of skills identified in the following section of this chapter, and the fact that many of those skills are significantly more challenging than even level 350 skills, it is clear that almost none of our graduating seniors in American high schools are graduating with the necessary requisite skills to experience success in college and the changing work force. The results specific to California college bound students are no more encouraging. The California State University system reports that the mean score of students on the ELM test is 490 out of 700; with a score of 550 being passing (ELM 1991-1992 Report on the Performance of First-time Freshmen). The University of California reports that nearly 25% of its entering freshmen require some remedial math course prior to acceptance into a math course for which undergraduate credit is given (University Academic Senate Report, 1996, p. 14). The performance of American secondary students in the international arena is even more disheartening. The Third International Math and Science Study (TIMSS) included quite a few alarming findings. Among the most damning were: 1) The mean U.S. score in 12th grade mathematics literacy achievement was 461, compared to the international mean of 500; 2) The United States sample achieved a score statistically and significantly lower than the international mean; and 3) A particularly startling finding was that compared to the 8th grade TIMSS data, the United States scored worse on 12th grade scaled scores by 28 points (-11 compared to the 8th grade mean and -39 compared to the 12th grade mean). 28 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. What is more, only Cyprus and South Africa scored significantly lower than the United States mean, while Italy, the Russian Federation, Lithuania, and the Czech Republic had no statistically significant different score from the United States. The advanced mathematics results are similarly appalling, with the United States at the bottom of the sample of 13 countries (negatively surpassed only slightly by the Czech Republic). Other significant findings are: • Sweden, the top scoring country, had a mean of 552 out of a possible 800. The Netherlands, Sweden, Denmark, and Switzerland were the top four countries in the 21 country sample. • 34% of the students in the TIMSS sample from the United States reported not taking a senior year mathematics course, compared to 22% and 30% in Denmark and Sweden, the top performing countries. • The mean U.S. score on the science literacy test was 480, a difference statistically significant at the .05 level. • There was no significant difference in math achievement between males and females in the United States, though there was a slight difference (466 for males and 456 for females). • There seems to be little difference in the extent to which seniors in America and seniors in top-performing countries such as Denmark or Sweden liked mathematics, or spent time on homework and/or studying in the area of mathematics. 29 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Interestingly, United States students perceive themselves as doing just fine in mathematics — (76%) the highest percentage of students agreeing or strongly agreeing with the statement that they are doing well in mathematics. Of course, the TIMSS data (indeed the entire TIMSS process) has come under vocal opposition from educators and policy makers such as Bracey, who in his May 1998 article entitled “TIMSS, Rhymes with ‘Dims,’ as in “Witted, pp. 686-687”, argued that all of the following are reasons to discount some, if not all, of the TIMSS results: 1) differences in age and grade level comparisons between countries; 2) enrollment rates differ between countries; 3) the United States advanced mathematics population lacks definition (Pre-Calculus students taking an exam which included calculus); 4) some high scores are very questionable; 5) many countries do not meet the TIMSS criteria for participation; and 6) cultural differences exist between countries. I say again as I have said in the past: there are serious problems in American schools, and our poor rural and urban schools need the equivalent of the Marshall Plan. But we should attend to the problems that actually exist. Those that were purportedly revealed in the official story of TIMSS do not. Forgione, the U.S. Commissioner of the National Center for Education Statistics, has called TIMSS ‘rough around the edges.’ I say rotten to the core. The official TIMSS story is an exercise in political rhetoric and comes very close to being a hoax perpetrated on the whole world. In fact, Bracey argues, “If the math and science literacy scores were accurately calculated, factoring in appropriate variables, the United States would be about average. Not a cause for celebration, to be sure, but not the disaster so far painted, (p. 686).” 30 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Bracey’s specific objections to the TIMSS methodology include: • The TIMSS tests supposedly test “what is needed to function effectively in society.” Who determines this and how? • The graduation rates of 12th graders in various countries varies significantly. • The ages o f‘final year’ students varies significantly - in Iceland, for instance “the TIMSS kids are about the same age as American college seniors.” • Some countries who did better have multi-year programs where students study the same subjects for as many as three years of secondary school, whereas American students study some of these subjects for only one. • Students in other countries were selected out of the testing population because they were in vocational schools, whereas American schools are more comprehensive. • American teens work substantially more than their counterparts in other countries and in cases where they worked up to 15 hours per week, they scored above the international average, while those who worked more than 25 hours per week scored 60 points lower. In his June, 1998 response to Bracey, NCES Commissioner Pascal Forgione characterized many of Bracey’s concerns as misguided. To the argument that there are differences in age and grade levels, Forgione rebutted that the study compared students at “a similar point in the education system: the end of secondary school (p. 769).” To the argument that in other countries “only the best students are still enrolled in secondary school in the late teenage years,” NCES tells us that most of Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. the countries who participated in TIMSS had over 85% of the school-age cohort enrolled in secondary schools. He also said that “in none of the countries were the students chosen on the basis of whether they had taken calculus (p. 770).” Then Forgione pointed out that, “of the 21 countries participating in the general knowledge assessments, 14 excluded less than 10% of the internationally defined eligible population from the TIMSS sample (p. 771).” When confronted with the notion that the test results lack true relevance to college, work, or life success, Forgione adeptly argues for high standards and improved performance and for the validity of the TIMSS results: As crucial as academic achievement may be to success in the labor market, the importance of science and mathematics knowledge reaches beyond economic productivity. The TIMSS general knowledge assessments were designed to test students’ ability to apply their knowledge to situations they might encounter in everyday life, which is becoming more complicated. As adults, today’s students will need to make sense of a rapidly changing world. They will need to make difficult decisions for themselves and their families regarding finances and health care. Collectively, as members of a community, they will need to understand and make decisions regarding how to treat the world in which they live (p. 772). Achieving Forgione’s vision and building student skills up to the level of national and international standards will be challenging. Concerted reform efforts are essential to “fixing” many of the problems discussed. The poor performance of American math students begs the question, so what? Why are math skills so important? This chapter now turns to a discussion of 32 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. the importance of math education and the skills necessary for success in the world of the 21st century and beyond. The Role and Importance of Math Education for Our Schools Math skills are life skills; they help prepare students for entry into the work force and into college. The California Mathematics standards tell us, Numbers play a crucial role in our daily lives, whether we are buying a car or understanding the world and financial news reports we encounter every day...Everyone needs to have broad, deep and useful knowledge of mathematics, a fact other countries have long understood. Our global economic competitors expect and demand the acquisition of mathematical knowledge from all of their citizens. We need to keep pace. If they can do, we can do it (p.l). Supporting this contention is a recently published work, High School Mathematics at Work: Essays and Examples for the Education o f All Students (The National Science Education Board and the National Research Council Center for Science, Mathematics and Engineering Education, 1998), which highlights the mathematics needed for careers (p. ix). This text argues that “society’s technological, economic, and cultural changes of the last 50 years have made many important mathematical ideas more relevant and accessible in work and in everyday life” (p.l). A year earlier, a 1997 report entitled Preparing fo r the 21st Century: The Education Imperative contended that: ...today, an understanding of science, mathematics, and technology is very important in the workplace. As routine mechanical and clerical tasks become computerized, more and 33 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. more jobs require high-level skills that involve critical thinking, problem solving, communicating ideas to others and collaborating effectively. Many of these jobs build on skills developed through high-quality science, mathematics, and technology education. Our nation is unlikely to remain a world leader without a better educated work force. (The National Academy of Sciences, p. I). What’s more, the California and NCTM Frameworks for math instruction tell us that to be competitive in the global economy, “We must develop the mathematics and science skills of all of our students, not simply the very best” (National Goals Report, 1994 p.38). U.S. Secretary of Education Richard Riley, in his January 9, 1998 “State of Mathematics Education: Building a Strong Foundation for the 21st Century” speech, supports the need for all students to develop more substantial math skills: “almost 90% of new jobs require more than a high school level of literacy and math skills (pp. 1-2).” There really is substantial agreement about what students ought to learn in mathematics courses. The James Stigler TIMSS video study revealed that Japanese classrooms focus on a smaller number of “bigger” problems, encouraging students to communicate their rationale for solving problems, and engaging students in meaningful discourse about solution strategies. Careers are but one area where math skills are important. College bound high school students need significant skills in math to be successful in college. The Educational Equality Project of the College Board reported that “the single most important predictor for further education success is a student’s performance in math through at least geometry.” (Tucker and Codding, p.25) 34 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Secretary Riley’s 1998 State of Mathematics Education speech included the alarming fact that .. .young people who have taken gateway courses like Algebra 1 and Geometry go on to college at much higher rates than those who do not — 83% to 36%. The difference is particularly stark for low-income students. These students are almost 3 times as likely — 71% versus 27% -- to attend college. In fact, taking the tough courses, including challenging mathematics, is a more important factor in determining college attendance than is either a student’s family background or income. The 1992 follow-up to the 1988 National Educational Longitudinal Study reinforces the Secretary’s assertions: • Only 26.8% of high school seniors who complete less than the minimum requirement of algebra and geometry plan to attend a four-year college. • 70.2% of high school seniors completed Algebra 2 and Geometry, but only 52.9% completed Algebra 2. Of that 70.2% who completed these ‘gateway’ courses, 81.6% had plans to attend a four-year college. • 16% of high school seniors took Pre-Calculus while 10% took Calculus. Nearly all of those students planned to attend a four-year college (98.5%). Such research suggests some compelling reasons why students should be proficient in mathematics - it is necessary for them to attain good jobs and gain success in college. But what exactly are experts saying students ought to study in school? Mumane and Levy (1996) provide one answer: “The ability to manipulate fractions and decimals and to interpret the graphs and bar graphs, along with a bare 35 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. minimum of algebra” is the lowest level of acceptable math skill required of employees at companies such as Diamond-Star Motors, Honda of America Manufacturing, and Northwestern Mutual Life” (p.33). Diamond-Star Motors articulated a list of skills that all DSM production and maintenance associates would need: • the ability to read at a high school level • the ability to do math at a high school level • the ability to solve semi-structured problems and to originate improvements • the ability to work in teams • skills in oral communication • skills in inspection (the ability to detect errors) (p. 21) Arguing for an even wider range of skills than DSM’s high school mathematics requirement, Taylor maintains that three broad areas of mathematics are required; estimation, trigonometry, and algebra. Estimation is the ability “to get a rough check on the accuracy of a calculation.” Trigonometry includes the use of “sine and cosine to model periodic phenomena such as going around and around in a circle, going in and out with tides, monitoring temperature or smog components changing on a 24-hour cycle, or the cycling of predator-prey populations ... A lack of algebraic skills puts an upward bound on the types of careers to which a student can aspire (The Importance o f Workplace and Everyday Mathematics p.31).” Stevenson and Stigler (1992) include nine areas that mathematics experts, psychologists, educators, and politicians agree ought to be included on assessments 36 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. of student progress: word problems; number concepts and equations; estimation; operations (focusing on the ability of students to explain uses of arithmetic operations); geometry; graphing; visualization and mental folding; and, mental calculation (p. 40). The State of California Mathematics Framework (1992) for public schools and the National Council o f Teachers o f Mathematics Standards (1996) for high school consist of 14 distinct yet interwoven ideas “as to what mathematics might be addressed in coherent high school units of instruction” (p. iii). Those standards include problem solving, communication, reasoning, mathematical connections, algebra, functions, geometry from a synthetic perspective, geometry from an algebraic perspective, trigonometry, statistics, probability, discrete mathematics, conceptual underpinnings o f calculus, and mathematical structure (California Mathematics Framework, 1992, pp. 136-140). The New Performance Standards advocated by Tucker (1998) include seven broad areas: arithmetic and number concepts, geometry and measurement concepts, function and algebra concept, statistics and probability concepts, problem solving and mathematical reasoning, mathematical skills and tools, and mathematical communication (p. 295). These broad areas complement the California Math Standards which call for “facts, skills, procedural knowledge, conceptual understanding, problem solving, application, reasoning, and the eventual communication of the entire process” of learning in five domains - Number Sense, Algebra and Functions; Measurement 37 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. and Geometry; Statistics, Data Analysis and Probability; and Problem Solving and Mathematical Reasoning (p. 295). Gail Burrill, the President of the National Council of Teachers o f Mathematics reported in last year’s President’s Report: Choices and Challenges, that addition, subtraction, multiplication, and division are not the only basics students need to know: To conceptualize, visualize, and predict, students must comprehend basics that include algebraic reasoning, understanding space and shapes, and working with data and measurement. Students must be allowed to use skills in tackling meaningful problems as a way to learn the thinking and reasoning processes with numbers that they will need for their future, whatever that may be...One basic skill is understanding formulas that are used in real life.. .Part of what is basic is to understand that writing a formula using mathematical symbols is much more efficient and easier to use than writing the formula in words. (JRME 28:5 p. 610) BurriU’s predecessor, Jack Price, stated in his 1996 address that the strength of the Standards rests in their call for the continuation of the “basics” as most people know them - addition, subtraction, multiplication, and division - as well as the introduction of the business-and-industry “basics”, which include reasoning, problem solving, communicating, making mathematical connections, and collaborating. We do teach the basics - the basics of today and tomorrow. (JRME 75:5 p. 606) The Commission for the Establishment of Academic Content and Performance Standards recently published, with the approval of the State Board of Education and input from the California Education Round Table (a group of college and business leaders), Mathematics Content Standards for Grades K-I2. Those standards said, in part, 38 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Our goal is for California students to: • develop fluency with basic computational skills; • develop understanding of mathematical concepts; • become mathematical problem-solvers so that they can find ways to reach a goal where no routine path is apparent; • communicate mathematically about quantities, logical relationships and unknowns via the use of signs, symbols, models, graphs and terms; • reason mathematically by gathering data, analyzing evidence and bundling arguments to support or refute hypotheses; and, • make connections among mathematical ideas and between mathematics and other disciplines. The specific grade level standards for twelfth grade state that students understand and can justify advanced and abstract ideas in algebra, geometry, and trigonometry. They can perform complex algebraic simplifications and manipulations as required to solve problems. Students use algebraic and geometric arguments to prove important mathematical ideas. They have a deep understanding of families of functions, their use in the world and the mathematical techniques required to write, solve, simplify and interpret features of standard functions. Students understand and play the connections between a function and its inverse; between right triangle trigonometry and circular functions; and between coordinates in polar, vector and rectangular form (p. 32). These standards are purposely at high levels but they assume that the educational community in California will work to build student mathematical literacy to the point where almost all high school graduates are completing two years of algebra, 39 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. one year of geometry, and one year of trigonometry, and all students complete at least one year of algebra and one year of geometry. UC Expectations The University of California outlined similar competencies in the “Statement on Competencies in Mathematics Expected of Entering College Students” (available at http://www.ucop.edu/senate). The University of California system has long recommended four years of high school mathematics, including Pre-Calculus. The UC competencies include “certain approaches, attitudes, and perspectives” on mathematics that students “should have gained” in their high school course work. These “dispositions” include: • a view that math should make sense; • an ease in using math to solve unfamiliar problems in both concrete and abstract situations; • a willingness to work on problems requiring time and thought, problems that aren’t solved merely by mimicking examples that have already been seen; • a readiness to discuss the mathematical ideas involved in a problem with other students and to write clearly and coherently about mathematical topics; • an acceptance of responsibility for their own learning; • the understanding that assertions require justification based on persuasive arguments, and an ability to supply appropriate justifications; 40 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. • an openness to the use of appropriate technology, such as graphing calculators and computers, in solving math problems and the attendant awareness of the limitations of this technology; and • a perception of mathematics as a unified field of study, (pp. 1 -6) Integral to achieving these dispositions is quality math instruction that includes • modeling mathematical thinking; • solving problems; • developing analytic ability and logic; • appreciating the beauty and fascination of mathematics; • building student confidence; • communicating processes that lead to solutions and reasoning behind solutions; and • becoming fluent in the ‘standard’ skills of mathematics, (pp. 1-6) The University of California has identified six mathematical subjects that are an essential part of the knowledge base and skills base for all students who enter higher education. Students are best served by deep mathematical experiences in these areas: • variables, equations, and algebraic expressions; • families of functions and their graphs; • geometric concepts; • probability 41 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. • data analysis and statistics; and • argumentation and proof, (pp. 1-6) The UC “Statement” includes specific explication of these six areas, including a list of twelve specific types of problems students need to be able to solve in order to be successful in college: • arithmetic with signed numbers, including fractions and percents; • combining like terms; • using the distributive law; • factoring polynomials; • solving linear equations for one variable; • using the Quadratic equation; • applying the laws of exponents; • plotting points in the coordinate plane and graphing a function; • finding the measure of a third angle in a triangle, given the measure of the other two angles; • finding the areas of right triangles; • using ratios to solve problems involving similar triangles; and • finding the length of the third side of a triangle given the length of the other two sides, (pp. 1-6) In a similar vein, the California State University system has articulated its expectations of mathematics skills required by entering freshmen, and administered 42 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. an entry level mathematics examination (ELM) consisting of 60 questions in two basic areas — elementary algebra and geometry. The algebra section contains questions on elementary operations of real numbers, scientific notation, absolute value, and applications such as averages, estimations, percents, word problems, charts, and graphs. The second algebraic topic is polynomials, including a) evaluation of polynomials, b) adding, c) subtracting, d) multiplying and dividing them, and e) factoring polynomials (including the difference of squares, common factors, perfect squares, and sums and differences of cubes). The third and fourth topics are computation of rational expressions with one or more variables and powers and roots (including integer exponents, radicals, and fractional exponents). The fifth topic in algebraic skills is solutions of equations, (including a) equations in one unknown, b) systems of linear equations in two and three unknowns, c) quadratic equations, d) equations involving square roots, absolute values, and rational expressions, and e) ratio and proportion involving equations). Inequalities (linear and quadratic), graphing (points on the number line, equations, and inequalities—including absolute values), and functions of one variable round out the algebraic skills required of entering freshmen who demonstrate competency on the ELM. Topics in geometry include a) perimeter and area of triangles, squares, rectangles, and parallelograms; b) properties of circles, including their radius, diameter, circumference, and area; c) volumes of rectangular solids, cylinders and spheres; d) properties of triangles (including sum of interior angles, isosceles, 43 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. equilateral, similar, and congruent triangles); e) the Pythagorean Theorem and its applications; f) parallelism; and g) perpendicularity. Even the California Community Colleges have adopted a policy in support of the UC and CSU requirements that effectively requires all students to pass proficiency tests in intermediate algebra and geometry prior to transferring from a California Community College to one of the State Universities. The mathematics community has come to unprecedented agreement on what math ought to be learned and for what purposes, much more so than the social studies or language arts communities. As seen in the first part of this chapter, the data on our students’ math performance are disappointing on many fronts, yet have come under fire from various groups for inaccuracies in reporting and measurement techniques. This chapter now turns to a discussion of various approaches to reforming mathematics education which might help improve student performance and prepare students with the skills necessary to compete in the global community. Reform in Mathematics Education Several themes have emerged in mathematics reform that have as their ultimate goal the improvement of student performance. These reform themes include: higher standards; increased problem-solving; inventing, and conjecturing tasks in math classrooms; constructivism; the use of non-routine problems; the use of technology (including graphing calculators and computers); the view of 44 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. classrooms as learning communities; and a shift toward authentic and performance assessment. Math standards The need for clear and challenging math standards has been a clarion call since the failure of “new math” reform efforts, a failure sounded by the National Advisory Committee on Mathematics Education (NACOME, 1975). In 1992, the U.S. Department of Education gave its blessing to the establishment of 22 national pilot projects “charged with the development of skills standards (Standards: Making Them Useful - Lessons from the Research and Development Period, found at http://www.ed.gov/pubs/ Standards/lessons.htmD.” The Education Department found that one of the first difficulties encountered by these projects was finding common ground on operative definitions. Following are some of the definitions these 22 projects agreed upon after much debate: Content standards refer to what we expect learners to know and be able to perform. Performance Standards indicate levels of achievement, or competency within a content area (e.g. advanced, proficient, or basic). Performance standards can be set for an individual content standard or across groups of standards. The National Council of Teachers of Mathematics took the leadership in developing content and performance standards in mathematics, beginning by identifying five major necessary shifts in K-12 (indeed K-graduate school) mathematics education: 45 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. • toward classrooms as mathematical communities - away from classrooms as simply a collection of individuals; • toward logic and mathematical evidence as verification - away from theteacher as the sole authority for right answers; • toward mathematical reasoning -away from merely memorizing procedures; • toward conjecturing, inventing, and problem solving - away from an emphasis on mechanistic answer-finding; • toward connecting mathematics, its ideas, and its applications - away from treating mathematics as a body of isolated concepts and procedures. (National Council of Teachers of Mathematics, Professional Standards For Teaching Mathematics, 1991, p.3) The epistemic secondary school mathematics standards proposed by the NCTM included rigorous work in algebra, geometry, trigonometry, and functions, specifically setting the expectation that high schools develop problem solving, mathematical communication, math reasoning, and an understanding of the “interplay among various mathematical topics and their applications (California Mathematics Framework, p.137). The California Education Roundtable recently published its own version of mathematics standards for California students, emphasizing very high expectations for students at all grade levels, and specifying precalculus-level skills as the desirable exit skills to be possessed by all California graduates. 46 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. The push for higher standards has sparked much debate among school teachers, parents, and curriculum leaders, especially in a state such as California with such a high non/limited-English proficient student population. Some argue that standards set too high discourage students from trying at all. Others argue that higher expectations result in higher achievement. The research and school administrative communities have been embattled in a debate over the role and importance of high math standards, but many classroom teachers continue to ignore any published standards, instead choosing to teach from the textbook beginning at page one and working through as many chapters as can be reasonably accomplished in a 180-day school year (Wells, 1999). Problem solving The importance of problem-solving, inventing, and conjecturing in math classrooms is perceived by some teachers as being at odds with the push for higher standards. The need to teach algorithms overshadows problem solving in many math classrooms (Meserve & Suydam, 1992), in part, because standardized testing most often requires students to perform algorithms rather than demonstrate problem-solving capacity. Teachers then are reticent to spend too much time on problem-solving activities for fear the students will do poorly on these algorithm laden standardized assessments. This, of course, is the opposite of what research tells us about problem-centered mathematics programs. Wood and Sellers (1996) researched children enrolled in problem-centered classrooms for two, one and zero 47 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. years with the results that those in problem-centered classrooms did significantly better on standardized algorithmic assessments than those who were taught using only the textbook. The Wood and Sellers study included 17 schools in a single district in Indiana, focusing on 305 students in grades 1-4 who had been enrolled in the Problem-Centered Math Curriculum for two, one and zero years: At the end of the third grade a comparison was made of students in problem-centered class for 2 years (Project 2), problem-centered classes for one year (Project 1), and textbook classes. An ANOVA revealed significant difference for Project-2 pupils on the Computation subtest (F=12.95, p<.05) and Total test (F=7.84, p<.05). A similar comparison was made at the end of the fourth grade using ANOVA. Students in Project-2 classes for second and third grade continued to score significantly higher on the Computation subtest (F=10.27, p<.05), on the Concepts and Applications subtest (F=l 1.22, p<.05), and on the Total test (F=13.27, p<.05) (JRME, 28:2 p.173). Constructivism Problem solving is not the only paradigm shift proposed by NCTM. Problem posing involves students formulating their own problems based on rich mathematics contexts in which the problems are placed. Silver and Cai (1996) found that “good problem solvers generated more mathematical problems and more complex problems than poor problem solvers did” (p.521). The NCTM Professional Standards for Teaching Mathematics (1991) recommends that teachers provide many opportunities for students to “formulate problems from given situations and create new problems by modifying the conditions of a given problem 48 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. (p.95)Brown and Walter (1993) published a text specifically providing teachers with suggestions for problem solving in the mathematics classroom. Wood and Sellers (1997) define constructivism as “a process in which students reorganize their activity to resolve situations found to be personally problematic,” — a philosophy which has impacted every academic discipline since its inception. In constructivist settings, “children attempt to resolve personal conflicts or differences between their existing ways of thinking and the aspects of their experiences that are new (p. 167).” Constructivism, then, argues that children construct meaning based on prior experience and make connections between prior knowledge and the current problems with which they are faced. Henningsen and Stein (1997) researched the importance of rich mathematical tasks that require learners to engage in higher order thinking. Specifically, Henningsen and Stein looked at the components of mathematical tasks that engaged learners for the longest periods of time. They found that those tasks that built on student’s prior knowledge were most often those tasks in which learners stayed engaged (82% of the time). This emphasis on prior knowledge, coupled with sustained pressure for explanation and meaning, reinforced the research of Bennett and Desforges (1988) which found that “tasks that build on students’ prior knowledge are likely to maintain high-level cognitive demands.” Interestingly, Clarke (1997) found a similar connection to teachers’ engagement with the teaching process. He studied two teachers from the Midwestern United States, analyzing their engagement with posing non-routine problems to students. 49 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. The teacher who had strong professional development experiences (prior and continuing knowledge) and who reflected daily on the teaching and learning process was significantly more comfortable with the posing of rich mathematical problems and the changing role of the teacher in the classroom. The teacher with limited professional development experiences was far less comfortable with the teacher’s role in mathematical discourse and problem-solving/problem-posing. Teacher as co-problem solver Non-routine problems often involve the teacher and students struggling together toward a solution. The paradigm shift from teacher as omniscient being toward teacher as co-problem-solver-with-students has met with some serious resistance, both from teachers and students (Wood, Cobb, & Jackel, 1990). But in situations where the teacher has effectively relinquished the omniscient role, Lampert (1990) found that communities of discourse resulted in ideas brought to a public forum, arguments refereed by the teacher, and the teacher sanctioning student’s intuitive use of mathematical principles (Clarke p.280). Experiments such as the Middle Grades Mathematics Project (MGMP), the Cognitively Guided Instruction (CGI) program, and Reality in Mathematics Education (RIME) project in Victoria, Australia have all used seven guiding principles about the role of the mathematics teacher in the classroom (each to varying degrees): • Non-routine problems are the starting point for class instruction and there is no provision of procedures for their solution. 50 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. • Student interest and needs are the basis for adapting materials. • Classrooms are organized in a variety of ways (individual, small group, whole class). • Teachers are fellow players with students in the solution process. • Big ideas of mathematics are the focus. • Informal assessment methods are used to inform instructional decisions. • Teachers facilitate student reflection on teaching and their own learning processes. (Clarke, JRME 28:3. P. 280) Technology The use of technology in mathematics classrooms has also been a topic for reform efforts. Tucker and Codding argue that “standards, assessment, materials, instruction, and instructional technology” must be aligned with each other (p. 99). The NCTM Standards assume that calculators, computers, and other mathematical tools are used to help promote mathematical discourse. These forms of technology become the vehicles for students to more efficiently solve rich problems posed by the teacher. Rather than being an end unto themselves, technological resources are means to assist students in solving problems and communicating their solutions methodologies to others, including their peers and teachers. Duke (1998) argues that technology is important but cannot be allowed to diminish relationships between students. “Technology has the capacity to isolate as well as to connect people...Designs for new learning environments must provide opportunities for 51 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. people to build and sustain relationships” (p. 692). Mumane and Levy (1996) encourage schools to make use of modem technology to provide context-rich, real- life examples for students of how technology is used in the work force. Using the example of underwriters at Northwest Mutual Life, they describe a work force in which employers use technology to communicate, solve complex problems, collaborate on customer support and corporate improvement, and expedite client processing. Still, in all, the typical American math classroom uses limited technology (Blum & Niss, 1991; Everybody Counts, p. 62). But in places where math teachers make effective use of technological tools, students’ performance and engagement are enhanced (Zbiek, 1998). Classrooms as learning communities: The Japanese Model The view that classrooms are learning communities is nothing new. Horace Mann, Freidrich Froebel, Jane Addams, Maria Montessori, and others have long articulated the view that classrooms must be places of discourse, ground in the principles that the teacher is a facilitator of student learning and that student interaction with peers and teachers is the single most important way students learn. TheTIMSS Video Study concluded that mathematical discourse in Japanese classrooms was far more aligned to some aspects of the NCTM standards than were U.S. classrooms (1997, p. 18). Japanese students are expected to explain their thinking processes and their rationale for attacking challenging problems in a 52 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. particular way. Interestingly, the videos “never observed calculators being used in a Japanese classroom” (p. 19). In Japanese classes, math lessons focus on one or two key problems, rich in mathematical dimension and requiring deep mathematical thought. Students work both independently and collaboratively to solve problems and discuss their answers with peers and with the teacher. Much discourse about problem-solving strategies occurs. In American classrooms videotaped, it was clear that there was an emphasis on algorithmic procedures and “the application and acquisition of skills” as opposed to the development of problem solving capacity (p. 19). Stigler and Hiebert (1999) argue that there must be a reexamination of the teaching process, not merely the existence of standards for learning. They propose that the Japanese model of discourse among teachers be examined for its possible application to American schools. In Japan, teachers regularly conduct peer observation and assessment and systemically review teacher performance and effectiveness (a process they call lesson study), critically examining ways the teaching process can be perfected. Japanese lesson study involves finding and implementing ways to make problem-centered math classrooms more authentic to the problems encountered in the work force. In lesson study sessions, teachers look for authentic problems and ways to assess students’ problem-solving ability. The shift toward authentic and performance assessment has been a lengthy part of school reform efforts in American schools. Requiring students to demonstrate their knowledge in more 53 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. “real-life” settings seems like an appropriate paradigm shift: but it too, has met with some resistance, not the least of which is the lack of teacher training at designing and assessing authentic performance scenarios (Corbett & Wilson, 1991; Smith, 1991). Firestone, Mayrowetz, and Fairman (1998) examined mathematics performance assessment in Maryland and Maine, two states that have “recently adopted performance-based assessments. Maryland was selected because the state linked formal sanctions to test performance, while Maine did not have such formal stakes” (p. 98). Their study reinforced the views of Sigler and Hiebert (1997) “that rather strong forces in the educational system maintain an approach to teaching that emphasizes practicing on many, small problems and shallow coverage of many topics” (p. 111). Firestone, Mayrowetz, and Fairman also hypothesized that “instructional methods for teaching mathematics did not change greatly in either state.” However, “performance-based assessment can change specific behaviors and procedures in the classroom more easily than the general paradigms for teaching a subject... state assessment policies do more to organize existing learning opportunities for teachers than to increase them. State policies are not the only forces promoting change in instructional practice, especially in mathematics. Colleges and universities...are among the many sources of information about how to change practice” (p.l 11). 54 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. High School/University Alignment and Continuity As Firestone, Mayrowitz, and Fairman point out, colleges and universities have tremendous potential impact on the way mathematics is taught in secondary classrooms. But how well do both the university and high school faculties understand what each is doing? Do high school teachers know what their students need to know and be able to do when they enter college? Do university mathematics professors know what is being taught in the high school math courses from which their freshmen math students hail? Are secondary math instructors designing coursework specifically with the goal of preparing students for entry into collegiate math programs? Is there any effort on the part of high school and university mathematics leadership to engage in dialogue about how high schools can better prepare students for entry into college math and how universities can support the efforts of high school math programs? Perhaps the best example of high school university congruence can be found in the California State University/University of California Mathematics Diagnostic Testing Project (CSU/UC MDTP). The MDTP, which begun in 1970, is concentrated on a set of diagnostic tests in algebra readiness, elementary algebra, geometry readiness, second-year algebra readiness, math analysis readiness, and calculus readiness. University researchers developed these tests with much consultation from secondary mathematics teachers and curriculum leaders. The tests cover material mathematics teachers at both the secondary and university levels believe a student ought to know in order to be successful in the level of 55 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. mathematics for which the diagnostic test is being given. Over the last 28 years, the tests have been refined in regular cycles, insuring the construct validity and test reliability. Used in the proper way at the secondary level, teachers administer these tests at various times during the school year to determine progress toward clear objectives for each course. In many California secondary schools, the MDTP tests have become placement tests “designed not to limit access to high levels of mathematics, but to determine appropriateness of course selection” (Wells, p .4). Specifically, the MDTP tests help teachers and students “identify areas of deficiency in terms of the preparedness of students for the next level of college preparatory mathematics” (p. 11). In 1995, more than 346,000 MDTP tests were scored (free of charge) at UC and CSU scoring centers throughout California (MDTP, 1996, p.l). “It is the hope of MDTP that “the tests will help teachers and students to strengthen areas of weakness in order to enhance the students’ chance for success in their mathematics course work” (Wells, p. 9). MDTP has taken an important step toward congruence in California’s secondary schools and universities. Beginning with what is expected of students at each level of college preparatory mathematics course work, the tests have become tools for curricular improvement. Every public university in California (and 85% of the Community Colleges in California) uses the MDPT tests to determine appropriate course placement in freshman math courses. Students must meet 56 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. minimum cutoff scores (established by the individual university or college campus) in order to gain entrance into the desired mathematics course. High school mathematics teachers are well advised then, to make use of the MDTP. Since a large number of California high school graduates go on to public universities and community colleges in their own state, and since these institutions require the MDTP tests as placement instruments, high schools should be using these to help students prepare for the inevitable requirement that they pass such a test in order to gain entrance into math courses they wish to take. What makes MDTP an example of desirable congruence? Because there is widespread agreement between college and high school math teachers that the MDTP tests are both statistically valid and reliable, the MDTP has successfully answered the question “What ought students know at the end of each college preparatory mathematics course?” Furthermore, MDTP is a standardized assessment instrument that both the secondary and university personnel can use to plan instruction and gauge student achievement. The question still remains, however: to what extent is the use by secondary schools of the MDTP impacting student success and preparation for college mathematics courses? Common assessments such as the MDTP are but one means of measuring congruence between a high school and the universities into which its students matriculate. On another, perhaps more difficult level, congruence can and should exist within a school building through such novelties as common syllabi, lesson caucuses, and common final exams. 57 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. The existence of a common syllabus for a course is one such factor for in school congruence. In schools with a common syllabus for a course, there is one set of learning objectives, regardless of who is teaching it. What might be termed Expected Course Learning Results (ECLRs) are developed and refined with broad based input from teachers, parents, students, and curriculum frameworks such as California’s Mathematics Framework. Too, the order in which material is presented is common across classrooms. The manner in which material is presented can and should vary by teacher, but in places with high levels of in-school congruence, teachers meet to plan lessons together, adopting the best strategies for student success. What might be termed lesson caucuses begin with a discussion of the ECLR being taught. The caucus then moves to a dialog about how to best teach the ECLR using best practices. Possible stumbling blocks to student success are analyzed and anticipated, and means of assessing student understanding are agreed upon. In the ideal situation, peers observe their fellow teachers and give critical feedback on how to improve teaching and learning. Common final exams are another form of in-school congruence. Currently, most high schools have each teacher develop his or her own final exam, an exam which is most often not linked to the exams of other teachers with the same course and which is linked only to objectives developed by the individual teacher, if it is linked to any at all. In schools with high levels of internal congruence, ECLRs are the basis for developing assessments (preferably performance/authentic forms of 58 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. assessment) which determine the extent to which ECLRs have been achieved by all students. The common final exam for Algebra I is not unique to an individual teacher; rather, every Algebra I teacher uses the same final exam and the results of that exam are analyzed for trends in student performance and effectiveness of instructional strategies. Conclusion Mathematics success has long been a concern of American policymakers. Defining the purposes and types of mathematics in which our students should gain proficiency has proved challenging, but organizations such as the National Council of Teachers of Mathematics have made noble attempts to do so. The performance of American students, especially when compared to international norms, is abysmal. Yet, promising trends in school reform have proven helpful in increasing student achievement. One such trend is the push toward aligning secondary school and collegiate expectations so that entering college freshmen possess the algorithmic and problem solving skills and attitudes that lead toward success in college mathematics courses. This chapter has examined the role and importance of mathematics education in our schools, the mathematics performance of American secondary school studies, the key elements of recent and proposed reforms in mathematics education, and views of alignment/high school-> university continuity that might be attempted in secondary school mathematics education. If improved student 59 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. performance is a key element of reform efforts, the congruence between what we expect in high school and what our colleges and universities expect must be improved. Greater links between secondary and university math departments need to be forged. When such links have been made, the researcher believes student performance will drastically improve. The next chapter will discuss the methodology employed in the study of the performance gain of students enrolled in 11 schools at varying levels of math instruction congruence. 60 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. CHAPTER 3 METHODOLOGY This chapter presents a description of the sample that participated in a study of congruence in secondary mathematics departments of catholic high schools in the Archdiocese of Los Angeles. The chapter then sets out the instrumentation used to investigate the research questions, the data collection procedures, and the data analyses. The unit of analyses for this study was the school. Quantitative data were collected and analyzed on student performance gains in first- and second-year algebra courses. In addition, written surveys were administered to participating school faculty to provide a basis for assigning congruence levels. This qualitative method assessed the extent to which there was articulation vertically and horizontally between teachers regarding what to teach, how to teach it, and how to assess it in first- and second-year algebra courses. The assigned levels were statistically analyzed to determine their relationship to student performance gains among first- and second-year algebra students in each of these schools. Survey methodology was selected as the most appropriate means of data collection for congruence levels because anonymous surveys allowed respondents to provide detailed descriptions of the extent to which congruence existed within the school. Such anonymity was a necessity in that, prior to agreement regarding participation in the study, serious concerns were raised about the survey’s potential to reflect negatively on the institution where the respondents were employed. 61 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Three research questions guided the study: 1. What student performance gain exists in first and second year algebra courses for catholic high schools involved in the study? 2. To what extent is there mathematics instructional congruence within a sampling of Los Angeles Archdiocesan Catholic Secondary Schools? What are the levels of congruence within those schools? 3. What is the relationship between student performance gain scores and congruence levels within schools studied? Study Sample The study sample consisted of 1386 first-year algebra students and 1023 second-year algebra students from eleven schools in the Archdiocese of Los Angeles. A total of 40 out of a possible 48 teachers of those first- and second-year algebra students also were study participants. Schools were selected to participate in the study if they were under the auspices of the Archdiocese of Los Angeles. The superintendent of schools and assistant superintendent of secondary schools gave permission for the study to be announced at an annual principal’s meeting. Each of the 54 catholic high schools within the Archdiocese were invited to participate, provided they had at least two sections each of first- and second-year algebra and at least two different teachers for each of the first- and second-year algebra sections. This insured anonymity among participating teachers and classes. Of the 54 possible schools, these criteria eliminated 8, leaving a potential 62 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. participant pool of 46 schools. The final requirement for participating schools was that no more than 50% of the school’s math department could be new to the school that school year. That eliminated another 5 schools from the pool. O f the remaining 41 schools, 11 returned an interest form by the September 5,1998 deadline. Those eleven schools were then asked to participate in the study. Participating schools were assigned fictitious names to assure their anonymity. Within each school, all of the students in each first- and second-year algebra course were pre-and post- tested using the MDTP tests. In all, this involved a sample size of 84 classrooms (48 Algebra 1 and 36 Algebra 2) and 2409 students (1386 Algebra 1 and 1023 Algebra 2). Table 8 below summarizes the sample for the study. TABLE 8. Participating Students, Teachers, and Classes in the Study SCHOOL ALG 1 CLASSES ALG 1 STUDENTS ALG 2 CLASSES ALG 2 STUDENTS NO. OF TEACHERS NO. RESPOND A 4 114 2 71 6 6 B 5 170 3 98 4 3 C 3 91 2 63 3 3 D 4 120 3 89 3 3 E 4 n o 2 55 3 3 F 4 128 4 120 35 35 G 4 75 3 60 4 3 H 5 133 5 152 5 4 J 7 204 4 119 6 4 K . 4 130 3 72 5 3 L 4 111 5 124 4 3 TOTALS 48 1386 36 1023 48 40 63 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Instrumentation Three measures were utilized to investigate the research questions: 1) The University of California/California State University Mathematics Diagnostic Testing Project (MDTP) Algebra Diagnostic and Intermediate Algebra Diagnostic exams; 2) a completion survey to assess congruence within the school; and, 3) a rating scale to assign congruence levels. The first research question dealt with student performance gains. The MDTP Algebra Diagnostic and Intermediate Algebra Diagnostics tests were selected as the measures to assess gain scores. The MDTP test is designed to diagnose student skills in preparation for various levels of mathematics coursework. The content of the tests has remained much the same since their inception 2 years ago. The following are the domains assessed in the Elementary Algebra Diagnostic Test (given to the first year Algebra students in the study): • Graphical Representation • Informal Geometry and Logic • Linear Equations and Inequalities • Polynomials, Including Quadratic Equations • Rational Expressions/Graphical Representation • Integers, their Operations & Applications There are 7-8 questions in each domain, a total of 50 questions on the test, and 70% correct indicates mastery of the content contained in the test. 64 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. The Intermediate Algebra Diagnostic Test contains the following domains: • Functions, including Logarithmic and Exponential • Geometric Applications • Geometric Concepts • Conceptual Geometry • Linear Equations and Inequalities, Absolute Values, and their Graphs • Polynomials, including Quadratic Equations • Rational Expressions There are approximately 7 questions from each domain on the Intermediate Algebra Diagnostic Test. A passing score is generally 70%. It is worth noting that the MDTP exams have played a critical role in mathematics instruction in California since their inception in 1977; however, the leadership of the MDTP has articulated a strong desire to keep the tests from becoming a political bargaining chip. Therefore, mean scores by school, district, and across the state have not and will not be released for public consumption. The MDTP leadership feels strongly that the tests must remain diagnostic tools rather than high stakes exams. Of course, this is in opposition to the reality that students who fail to meet minimum cut-off scores in California Community Colleges, Cal State Universities and UC schools are not admitted to certain math courses. The fact that these tests are used by so many California colleges and universities, is the reason they were selected as the instrument for this study. As part of the agreement reached between this researcher and the MDTP staff, the information provided in 65 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. this study is limited to a general description of test domains, quantity of items on the tests, and anonymously-reported school mean scores. The second research question addressed the issue of congruence levels within schools. To establish congruence levels, two assessments were developed by the researcher. A teacher survey elicited responses regarding various congruence behaviors. This survey included both short answer and free-response sections with the goal of ascertaining the level of congruence that existed within each school. Questions on the survey included opportunities for respondents to indicate the extent to which they discussed course objectives with other teachers, the nature of assessment in first- and second-year algebra courses, and the content of course syllabi in these classes. The survey was organized as follows: 1. Question one asked respondents to identify (YES or NO) whether the following existed with in the school: common final exams, common assessments, common syllabi, placement exams, team instructional planning, UC/CSU MDPT Testing, department meetings on expectations of entering college freshmen, and standards discussions. 2. Question two asked respondents to use a five point Likert scale to assess such factors as textbook vs. standards-driven curriculum, student preparation of high school and college coursework in mathematics, and analysis of teaching methods. 3. Question three provided a free-response written section for respondents to clarify responses to the Likert Scale. 66 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 4. Question four asked respondents to self-assess their overall congruence level using the researcher-developed congruence factors. For purposes of this study, congruence refers to the extent that teachers discuss and align expectations about what is taught, how it is taught, and how it is assessed in first and second year algebra courses. The following behaviors were identified as indicators of congruence within a school math department: a. Discussions by teachers of the same course regarding what is taught, how it is taught, and how it is assessed. b. Discussions between teachers of various level courses regarding what is taught, how it is taught, and how it is assessed. c. Agreement by teachers teaching the same course regarding what is taught, how it is taught, and how it is assessed. d. Agreement by teachers of various level courses regarding what is taught, how it is taught, and how it is assessed within each course. The third study measure was a congruence rating scale devised by the researcher. The scale served to establish a congruence level for each school and consisted of five levels as follows: Level 1 - Minimal or no communication between teachers about what is taught, how it is taught, and how it is assessed in first- and second-year algebra courses. Level 2 - Some communication between teachers regarding curriculum, instruction, and assessment with only minimal impact on teaching and learning. 67 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. given to all participating teachers and those teachers signed statements acknowledging the proper use of those procedures During September 1998, surveys were mailed and/or emailed to each participating teacher at the 11 participating schools. In their initial notice of intent to participate, respondents were asked which method of communication was most preferred. Surveys were to be returned by mail or email. Forty out of a possible 48 teachers returned the survey, about half of which were returned by email. All returned surveys were useable and each survey respondent answered all the questions. Those who responded via email tended to give more lengthy answers to open-ended questions, with the average respondent writing 7-8 sentences for each open-ended question in Part 3 of the survey. Respondents who wrote responses tended to give 2-3 sentence answers. By the November I, 1998 deadline, 31 teachers had returned to the survey. The remaining teachers were contacted via email and phone as a reminder about the need to return the surveys. By November 20, 1998,40 respondents had returned the surveys, and data analysis began. The third research question looked at the relationship between achievement gain and congruence. The statistical test of Pearson Product Moment Correlation Coefficients (PPMCC) was selected to assess the numerical relationship between the student achievement gains and congruence levels within the school. The PPMCC was appropriate to measure the relationship because the coefficient of linear correlation r is the numerical measure of the strength of the linear relationships between two variables. The coefficient reflects the consistency of the 69 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. effect that change in one variable has on the other. The value of the linear correlation coefficient helps us to answer the question, “Is there a linear correlation between the two variables under consideration?”(Johnson & Kuby, 2000, pp. 146- 147) Data Analysis Gain scores were computed for each of the tests by calculating the mean difference between pre- and post-test scores. The UCLA Mathematics Diagnostic Testing Project staff scored the tests and provided classroom mean scores that were used by the researcher to tabulate overall school gain scores. Overall levels of congruence were designated for each school based on the researcher-developed congruence scale. This was accomplished by analyzing teacher responses to the congruence survey items. In cases where there appeared to be discrepancies between various teachers’ responses regarding congruence level within a school, follow-up interviews were undertaken to reconcile any differences so that one level of congruence could be assigned to each school. Because of the potential subjectivity in assigning a congruence level, three persons (the researcher, and two math department chairpersons from schools not participating in the study) independently proposed a congruence level for each participating school based on the survey and interview data. The mean of these three congruence level assignments was rounded to the nearest whole number. In no cases was there a difference of more than one proposed congruence level 70 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. between the three persons considering congruence. Should a difference have been noted, a conference would have held between the researcher and the two independent math department chairpersons to attempt to reach some consensus. There was the possibility that one teacher within a school would have a stronger sense of congruence than others in the same school. Since congruence necessarily involves alignment with other teachers and the universities into which students matriculate, it was not considered desirable or possible for an individual to be assigned a “separate” congruence level. The school was assessed as a single unit of measurement. Individuals within the school who had a vision for articulation could help move a school beyond level one, but could not move it to levels three or beyond without the involvement of other staff members at the school. Relationships between student performance gain (pre-to-post) and the school’s level of congruence were evaluated. Chapter 4 of this dissertation will examine the results and findings of the study described in this chapter. 71 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. CHAPTER 4 FINDINGS This study focused on the relationship between curriculum alignment and student achievement in high school mathematics classes. The following research questions were posed: • What student performance gain exists in first-and second-year algebra courses for high schools involved in the study? • To what extent was the mathematics curriculum aligned within a sample of schools? What was the level of congruence within the schools? • What is the relationship between student performance gain scores and congruence levels within school studied? The term congruence included two aspects, the first being the extent to which classroom teachers within a school communicated with each other about what to teach, how to teach it, and how to assess it. The second was the extent to which a school understood the expectations of colleges and universities into which their students matriculated, designed the curriculum content for courses around those expectations, and linked assessments to those expectations. Eleven schools participated in the study, with a total of slightly more than 2400 students enrolled in 80 sections of first- or second-year algebra. The eleven schools participated voluntarily, but only after they were guaranteed that the results of the study could not be identified in terms of any given school. Because of the 72 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. size of the Los Angeles Archdiocese and the ease with which certain demographic data could be associated with a particular school, demographic information about the schools has been omitted from the report of findings. For the purposes of clarity, however, a fictitious name has been assigned to each participating school. To address the first research question, student performance gains were assessed by pre and post testing using the University of California/California State University Diagnostic Mathematics Project exams. These exams are diagnostic in nature, and consequently allow for independent assessment of student achievement. Pre-tests were administered early in the year, prior to any substantial teacher instruction. The same test was administered after at least 100 days of instruction had taken place in first and second year courses. With respect to the second research question dealing with curriculum alignment, congruence levels were set by the researcher based on questionnaire data gathered from participating teachers. Forty teachers out of a possible 48 completed questionnaires regarding the level of curricular congruence present in the participating schools. The third research question sought the relationship between the findings for achievement gains and the findings for congruence levels. Statistical analyses were applied to the data to determining significant relationships, if any. 73 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Pre- and Post-Test Comparisons The first research question investigated achievement gains in algebra. To assess achievement for both first-year and second-year algebra, Algebra Diagnostic Test scores were analyzed for differences. First Year Algebra The test measures student proficiency in seven domains identified as essential to first year algebra competency. Pre- and post-test average percentage scores were calculated for each participating school and differences were compared. Table 9 represents the algebra readiness (students currently enrolled in Algebra 1) gain scores of schools participating in the study. The scores were calculated using the difference between scores on pre-tests administered in early fall of the 1998-1999 school year, and post-tests administered to the same students in the late spring of the same school year. The table includes the number of students who took both the pre- and post-tests in Algebra 1, as well as the average pretest score in percentages correct, average post-test score in percentages correct, and gain score as a percentage difference between post-test and pre-test scores. 74 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. TABLE 9. Algebra 1 Gain Scores by School SCHOOL NO. OF STUDENTS PRE TEST AVG % POST TEST AVG % GAIN SCORE AVG % St. Elizabeth 114 19 47 28 St. Martin 170 27 43 16 St. Gerard 91 30 36 6 St. Luke 120 25 40 15 St. Cecilia 110 19 24 5 St. Thomas 128 39 57 18 St. Vincent 75 20 32 12 St. Agnes 133 19 24 5 Cardinal McIntyre 204 23 42 19 Bishop Fisher 130 11 17 7 O ur Lady o f the Rosary 111 23 50 27 The mean pretest score was 23.2%, indicating that students knew little of the Algebra 1 curriculum upon entry into the first-year algebra course. This is not surprising because the content of the test was material students had not previously been exposed to in a systematic way. Inspection of post-test scores, however, shows that all schools did evidence performance gains. The mean post-test score was 37.5% correct, meaning students correctly answered only slightly more than one-third correct after more than three-quarters of a year of instruction. The highest post-test score was 57% correct and the lowest was 17%. The mean performance gain was 14.3 percentage points, with the highest gain at 28% and the lowest at five percent. Noted also is a clear disparity between participating schools, with St. Elizabeth High School, St. Thomas High School, Cardinal McIntyre High School, and Our Lady of the Rosary High School performing above the mean, while St. Gerard High School, St. Cecilia High School, St. Agnes High School, and 75 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Bishop Fisher High School performed well below the mean. Only one school’s average percentage correct was equal to more than half of the points possible. Three of the participating schools has post-test scores in first-year algebra of lower than 25% and the remaining seven schools had post-test scores between 25 and 50% correct. It is also important to note here that the minimum performance level accepted by most mathematics departments at the community colleges and state universities is 60%. The post-test scores of participants in this study indicate that few, if any, of the more than 2400 students who took the tests would meet minimum performance levels. Second-Year Algebra The intermediate algebra readiness test measures student proficiency in algebra and geometry skills required for success in second-year algebra. Both the validity and reliability of this test have also been determined over the past 20 years of regular usage by hundreds of thousands of students in over 10,000 classrooms. Table 10 represents the intermediate algebra readiness (students currently enrolled in Algebra 2) gain scores of schools participating in the study. The scores were calculated using the difference between scores on pre-tests administered in early fall of the 1998-1999 school year, and post-tests administered to the same students in the late spring of the same school year. The table includes the number of students who took both the pre- and post-tests in Algebra 2, as well as the average 76 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. pre-test score in percentages correct, average post-test score in percentages correct, and gain score as a percentage difference between post-test and pre-test scores. The table also includes significance levels using a t-test comparing each score to the mean achievement gain score in second-year algebra. TABLE 10. Algebra 2 Gain Scores by School SCHOOL NO. OF STUDENTS PRE TEST AVG % POST TEST AVG % GAIN SCORE AVG % St. Elizabeth 71 13 64 51 St. Martin 98 25 43 18 St. Gerard 63 34 44 10 St. Luke 89 29 37 8 St. Cecilia 55 29 35 6 St. Thomas 120 33 54 21 St. Vincent 60 30 45 15 St. Agnes 152 23 24 1 Cardinal McIntyre 119 27 34 7 Bishop Fisher 72 12 11 -1 O ur Lady o f the Rosary 124 24 41 17 On the pre-test, the range of scores was from a minimum of 12 to a maximum of 34, indicating that even St. Gerard High School with a pre-test score of 34 averaged only slightly more than one-third of the questions correct. The post test averages were not substantially better, with a range from 11 to 64. All but two of the 11 participating schools had post-test scores below 50%, indicating that students in Algebra 2 courses at those schools performed well below average. The gain scores for students in Algebra 2 ranged from — 1 to 51, with the greatest gain seen for St. Elizabeth High School (a gain of 51 percentage points). More than half 77 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. of the 11 participating schools (six in all) showed gain scores of ten or less percentage points. St. Elizabeth High School is an outlier with an exceptionally high performance gain among its second-year algebra students. The next best performing schools are St. Martin High School, St. Thomas High School, St. Vincent High School, and Our Lady of the Rosary High School — all with gain scores near the mean Algebra 2 gain score. These four schools are all at least thirty percentage points lower than St. Elizabeth High School. In contrast, St. Agnes High School and Bishop Fisher High School showed negligible growth among second-year algebra students, with Bishop Fisher High School actually showing a performance loss after more than 100 days of instruction. The remaining sample schools showed only slight performance gains for the period between pre and post testing. Overall gain scores The overall achievement gain data (including first and second year algebra students) is summarized in Table 11. These data show substantial improvement in some schools, with negligible improvement in other schools. Participating schools might be grouped into three categories — those with total average gain scores less than nine percentage points (St. Gerard, St. Cecilia, St. Agnes and Bishop Fisher High Schools), those with gain scores between 10 and 17 points (St. Martin, St. Luke, St. Vincent and Cardinal McIntyre High Schools), and those with gain scores 78 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. of 18 or greater (St. Elizabeth, St. Thomas, and Our Lady of the Rosary High Schools). TABLE 11. Total Gain Scores by School SCHOOL TOTAL GAIN SCORE (IN %) St. Elizabeth 39.5 St. Martin 17 St. Gerard 8 St. Luke 11.5 St. Cecilia 5.5 St. Thomas 19.5 St. Vincent 13.5 St. Agnes 3 Cardinal McIntyre 13 Bishop Fisher 2.5 Our Lady of the Rosary 22 MEAN 14.1 Though these gain scores are not stellar by any measure, they give some indication as to the nature of student learning in the mathematics classrooms tested. St. Elizabeth High School in particular is noteworthy for a total gain score of nearly 40 percentage points. Such a gain score is indicative of substantial differences in student learning between the beginning and end of a course. Gain scores substantially lower than 40 percentage points indicate much less mastery of the course content by students after a year of instruction. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. School Congruence Levels The second research question addressed curriculum alignment that was assessed by establishing the congruence level of each participating school. This researcher developed a five-point congruence scale to measure the extent of both vertical and horizontal “fit” between teacher and student understanding of what is taught, how it is taught, and how it is assessed in classes within a school system (See Figure 3). Utilizing this scale, the researcher analyzed survey, interview, and observation data from participating schools and designated a congruence level for each school. These data are summarized in Table 12. In circumstances where the researcher had difficulty placing a school at a particular level of congruence, the researcher asked three independent reviewers to consider the data and assign a level of congruence to the school. The consensus judgments of the reviewers resulted in the final congruence levels. It should be noted that in no case was there any substantive disagreement about the levels to which a school should be assigned. 80 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. FIGURE 3. Congruence Level Characteristics. LEVEL CONGRUENCE LEVEL CHARACTERISTICS I (ISOLATION) • No reference to standards • Little/no communication with other teachers about what to teach • Textbook driven instruction • Methodologies vary widely across classrooms • Assessments vary widely across classrooms 2 (READINESS OR PARTIAL ISOLATION) • Standards largely ignored • Communication the same as level one conditions, except teachers at this level want to begin talking with other teachers about their isolation • Textbooks drive instruction • Few if any varied methodologies; inconsistent across classrooms • Assessment unique to each classroom ; primarily textbook driven 3 (BEGINNINGS OF CONGRUENCE) • Common syllabi with common course expectations (standards) • Occasional lesson caucuses with consideration given to standards • Textbook still drives the curriculum but there is sharing o f resources outside the textbook • Varied methodologies used, mostly as a result of collaborative discussions about how to best teach particular topics • Growing consensus about how students ought to be assessed; college expectations are considered in developing assessments 4 (SOLID CONGRUENCE) • Common syllabi with clear standards for student performance • Lesson caucuses regularly in which discussion surround both how to best teach topics and how to best assess student understanding; much discussion about how to maximize results • Textbook is a teaching and learning tool, but the course standards guide instruction and learning • Varied methodologies focused on improving student understanding • Common final exams, placement tests, and assessments aligned with college/university expectations 5 (TOTAL CONGRUENCE) • Common syllabi with clear standards for student performance linked with prior and future learning experiences and expectations; college/university expectations are integrated • Lesson caucuses occur regularly in which discussion surround both how to best teach topics and how to best assess student understanding; much discussion about how to maximize student results, particularly as they relate to college/university success • Textbook is a teaching and learning tool, but the course standards guide instruction and learning • Proven, varied methodologies improve student understanding • Common final exams, placement tests, and assessments aligned with college/university expectations; regular review o f the results from these assessments- with the goal o f informing instruction and improving achievement 81 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. TABLE 12. Congruence Levels by School SCHOOL LEVEL OF CONGRUENCE St. Elizabeth 4 Our Lady of the Rosary 4 St. Martin 4 St. Thomas 3 St. Vincent 3 Cardinal McIntyre 3 St. Luke 2 St. Gerard 2 St. Cecilia 2 St. Agnes 2 Bishop Fisher 2 Supporting data for the assignment of congruence levels are summarized by each participating school. St. Elizabeth High School — Level 4 St. Elizabeth’s student achievement gain was impressive, higher than any other school. Algebra 2 students went from pre-test means of 13% to post test means of 64% (a gain of 51 percentage points) while first-year algebra students went from pre-test means of 19% to post-test means of 47% (a gain of 28 percentage points). The net gain for all students tested was slightly less than 40%. Not surprisingly, St. Elizabeth’s congruence level was set at 4. St. Elizabeth had all six of its Algebra 1 and 2 teachers respond to the questionnaire. It is solidly at level four on the congruence scale developed by this researcher. There are common course outcomes at all levels of instruction. 82 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Common final exams are administered for all courses and common unit and lesson assessments are frequently administered. Common syllabi exist for all courses such that each teacher publishes common Expected Course Learning Results, a common assessment scale and system, and adheres to a common schedule for what is taught and when. The department has developed a portfolio system with agreement having been reached on what “non-negotiable” work will be stored in the portfolios and what type and quantity of student reflections will be included. The math department has reviewed California State and UC expectations of entering freshmen. Two members of the department teach at community colleges and have been involved in articulation committees between secondary schools and colleges. Each course has a placement exam modeled closely after the UC/CSU MDTP exams, and the department regularly discusses state and national standards at department gatherings. Teachers teaching the same course during a school year meet at least monthly to discuss what is being taught, how it is being taught, and how it is being assessed. Agreement is sought on answers to these essential questions. St. Elizabeth High School also has a unique summer school program which allows students to remediate deficiencies when they do not pass placement tests in math, or to take an entire year of first or second-year algebra, geometry, or precalculus. These courses also follow the curricular program set forth during the regular academic year and the math department chairperson gives strong leadership to the summer school faculty with regards to what to teach and how to assess it. 83 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. The same common final exams given during the regular school year are administered during the summer session. It appears that St. Elizabeth is quickly approaching level five on the congruence scale. Because lesson caucuses are not a regular staple of the department activities, the school cannot be placed at that level. It should be noted that St. Elizabeth High School perceives itself as having “made major improvements in the area of math instruction in the last five years. The numbers of students taking advanced math courses has skyrocketed and the number of students succeeding at all levels of math has increased dramatically” (Math Department Chairperson, St. Elizabeth High School). Our Ladv of the Rosarv High School - Level 4 Our Lady of the Rosary High School is solidly at level four on the congruence scale developed for the study. As with the other Level 4 schools, Our Lady of the Rosary’s student achievement gain was noteworthy. Algebra 2 students went from pre-test means of 23% to post-test means of 50% (a gain of 27 percentage points) while first-year algebra students went from pre-test means of 24% to post-test means of 41% (a gain of 17 %). The net gain for all four Algebra 1 and Algebra 2 teachers indicated common course outcomes at all levels of instruction and these are based largely on the expectations outlined in the UC/CSU MDTP exams. Common final exams are administered for all algebra one courses. They are planned for the next school year in algebra 2 courses and are being 84 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. developed for other courses for implementation as soon as possible. Common unit and lesson assessments are sometimes administered, especially in the first-year algebra classes where there is a common final examination. The math department has reviewed Cal State and UC expectations of entering freshmen and made changes to course expectations based on those published standards. The math chairperson sits on a community college math articulation committee. Each course has a placement exam modeled after the UC/CSU MDTP exams, and the department regularly discusses state and national standards at department gatherings. Teachers teaching the same course during a school year meet about once every six weeks to discuss what is being taught, how it is being taught, and how it is being assessed. Agreement is sought on answers to these essential questions. One teacher aptly stated, “We feel like we have a long way to go before we get our students to perform at the levels we would like, but we have a clear vision for how to do it and we are all on the same wavelength about it.” St. Martin High School — Level 4 St. Martin’s student achievement gains were substantial but not as dramatic at St. Elizabeth’s. Algebra 2 students went from pre-test means of 25% to post-test means of 43% (a gain of 18 percentage points) while first-year algebra students went from pre-test means of 27% to post-test means of 43% (a gain of 16 percentage points). The net gain for all students tested was 17%. 85 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. St. Martin had three of its four Algebra 1 and 2 teachers respond to the questionnaire. It is also solidly at level four on the congruence scale. There are common course outcomes at all levels of instruction. Common final exams are administered for all courses and common unit and lesson assessments are sometimes administered. The math department has reviewed Cal State and UC expectations of entering freshmen. Nearly two -thirds of the school’s graduates go onto Cal State and UC schools. The performance of students on the Cal State Entry Level Math exam is among the highest in the state. Each course has a placement exam modeled closely after the UC/CSU MDTP exams, and the department regularly discusses state and national standards at department gatherings. Teachers teaching the same course during a school year often meet weekly to discuss what is being taught, how it is being taught, and how it is being assessed. Agreement is sought on answers to these essential questions. Cardinal Mclntvre — Level 3 Cardinal McIntyre High School had four of six algebra 1 and 2 teachers respond to the questionnaire. It is at level three on the congruence scale developed by the researcher. The math department chair reports that, “instruction is largely based on the textbook. We do not deviate that much from what is covered there. We like the textbooks and feel as though they do a good job of preparing students for the math they will face in college.” The school has common syllabi which are published in a department handbook. Teachers must use a common grading system 86 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. and scale. There are no common final exams or assessment tasks, though one teacher reports, “the other teacher who also teaches Algebra 2 and I get together every week to talk about where we are at in the book and how our kids are doing.” At one time, the school utilized the UC/CSU MDPT exams but found them too difficult for the majority of students and abandoned their use. The common syllabi include course objectives which were developed over 10 years ago, but which are reviewed every two years or so by the department staff. Incoming freshmen are given placement tests, which the math chair reports “show us all too well how abysmal the math instruction is at the feeder elementary schools,” but no other placement system exists for math coursework. The prerequisite for placement into the next level course is a passing grade (which can be as low as a D-). McIntyre High’s student achievement gain was moderate. Algebra 2 students went from pre-test means of 27% to post-test means of 34% (a gain o f 7 percentage points) while first-year algebra students went from pre-test means of 23% to post-test means of 42% (a gain of 19 percentage points). The net gain for all students was 13%. St. Vincent High School — Level 3 St. Vincent’s student achievement gain was judged to be moderate because the net gain for all students tested was slightly less than 14%. Algebra 2 students went from pre- test means of 30% to post-test means of 45% (a gain of 15 87 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. percentage points) while first- year algebra students went from pre-test means of 20% to post-test means of 32% (a gain of 12 percentage points). St. Vincent High had three of its four Algebra 1 and 2 teachers respond to the questionnaire. It is solidly at level three. There are common course outcomes at all levels of instruction. Common final exams are administered for the pre algebra and algebra courses and common unit and lesson assessments are sometimes given. The math department has reviewed Cal State and UC expectations o f entering freshmen. Each course has used a placement exam modeled loosely after the UC/CSU MDTP exams, and the department occasionally discusses state and national standards at department gatherings. Teachers teaching the same course during a school year meet once per year to discuss what is being taught, but little other dialogue takes place about how assessments might be standardized. Student achievement on the CSU ELM test is above the national average. St. Thomas High School — Level 3 St. Thomas High School is at the border between levels two and three on the congruence scale, but closer to level three. The student achievement gain for all students tested was slightly less than 20%. Algebra 2 students faired better than Algebra 1. Algebra 2 students went from pre-test means of 33% to post-test means of 54% (a gain of 21 percentage points) while first-year algebra students went from pre-test means of 39% to post-test means of 57% (a gain of 18 percentage points). 88 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. All five teachers responded that there are common course outcomes published in the department handbook for all levels of instruction, though one teacher indicates that “the objectives are something I looked at once about twelve years ago and one for a WASC visit last year, but have not really paid much attention to.” Students and teachers are very much aware of college expectations; all of the recent graduates went on to four-year colleges or universities. The teachers have begun monthly department meetings to discuss what is being taught and how it is being assessed, but those discussions have only recently begun and the fruits are not yet evident. Common final exams are planned for all courses beginning with the next school year and common unit and lesson assessments should follow shortly thereafter. Each course uses the UC/CSU MDTP exams for placement purposes though one teacher argued, “The parents and students have the real power to place students because if they yell loud enough, the student gets into whatever course s/he wants.” Teachers teaching the same course during a school year meet quarterly to discuss what is being taught and how it is being assessed. Agreement is sought on answers to these essential questions, but no discussion takes place about how to best teach particular topics. There is some desire on the part of a few teachers to engage in such discussion. St. Gerard High School - Level 2 St. Gerard High School is solidly at level two with all three of its Algebra 1 and 2 teachers responding to the questionnaire. The student achievement gains for 89 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. St. Gerard High School were fair, averaging 8%. Algebra 1 students rose from an average of 30% to 36% and Algebra 2 students rose from 34% to 44%. Some common course outcomes at various levels of instruction were noted, but no common final exams existed for any courses. To a limited extent, the math department has reviewed Cal State and UC expectations of entering freshmen. Student performance on the CSU ELM exam is above average. There is a real problem of teacher turnover in the math department; a different person has chaired the department each of the past four years. The UC/CSU MDTP exams are administered as placement exams for upper level courses, but students and their parents have the option of overruling the placement test results. The department has had infrequent meetings, but the questionnaires reveal a sense on the part of teachers that the students “would do well no matter what because they are naturally talented.” St. Luke High School — Level 2 St. Luke High School was designated as a Level 2 school. There has been little turnover in staff the last ten years, and teachers report that they “have not really changed instruction much, except to focus more on basic skills because students come in from 8th grade lacking so many basics.” The math department has not reviewed Cal State and UC expectations of entering freshmen. Each course uses the UC/CSU MDTP exams as a placement tool, but grades in previous coursework are considered the “single most important factor in determining 90 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. placement.” The department rarely discusses state and national standards at department gatherings. One teacher described the situation in a phone interview as being “very isolated., no one else in the department really knows what I am doing until they get them (the students) the next year and see how well prepared they 55 are. St. Luke’s student achievement was slightly less than 12% overall. First- year algebra students showed greater gains than their Algebra 2 counterparts. Comparison of Algebra 1 means indicated an increase of 15 percentage points whereas for Algebra 2 means, gain of only 8 percentage points was noted. St. Cecilia High School - Level 2 St. Cecilia High is marginally at level two on the congruence scale. Responses from all three Algebra 1 and Algebra 2 teachers reported common course outcomes at all levels of instruction, though the math department chairperson states, “People seem to follow them at will.” Common final exams are a goal for the department, but one of the teachers commented, “There is little communication because the teachers are so far apart physically in the school plant. We never talk except at formal meetings. There is never a chance for us to dialogue about what we are teaching and how we could do better.” The math department has only informally reviewed Cal State and UC expectations of entering freshmen. There is a placement exam to determine proficiency of entering freshmen, but course grades are the only thing that determines placement after that. 91 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. One teacher pointed out, “It is hard to give grades lower than a ‘C’ when you know that will prevent them from taking the next course; I am not sure if my course is too rigorous or not rigorous enough so I tend not to give too many low grades.” The school has a voluntary program of administering the UC/CSU MDTP exams. As a result of those exam results, students are encouraged to enroll in summer remediation courses if their scores indicate a deficiency. Teachers teaching the same course during a school year meet twice a year (once in August and once in January) to discuss what is being taught, how it is being taught, and how it is being assessed. St. Cecilia’s overall student achievement gain was slightly less than 6%.. Algebra 2 students went from pre test means of 29% to post test means of 34% (a gain of 6 percentage points), while first-year algebra students went from pre test means of 19% to post test means of 24% (a gain of 5 percentage points). St. Agnes High School - Level 2 St. Agnes High had four of five Algebra I and 2 teachers respond to the questionnaire. It is at level two on the curriculum congruence scale. Common course outcomes are in place for some levels of instruction, including Algebra 1 and 2, but there are no common finals, no common syllabi, and no common assessment tasks. The math department has reviewed Cal State and UC expectations of entering freshmen and has discussed the possibility of creating some commonalities among what is taught and how it should be assessed. Each 92 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. course uses the UC/CSU MDTP exams for placement purposes, and cutoff scores of 35% have been set for placement into the next level course. The teachers complained that “there is just so much to cover in the textbook that I have to pick and choose what to teach based on what I think is important. I wish we could sit down together and agree upon what the most important things are in each course so I could know what to leave out. The text is simply too mammoth a beast.” Another teacher argued, “The isolation between teachers is the biggest problem. We just don’t talk about what kids need to know to be ready for the next class; then the next teacher complains that the kids are ill-prepared. I feel guilty about it, but I am doing the best I can.” The fact that St. Agnes is so aware of the problem is what places them at level two on the scale. A level one school would have little or no awareness of the isolation issue. St. Agnes’s student achievement gain was minimal. Algebra 2 students went from pre test means of 23% to post test means of 24% (a gain of 1 percentage point), while first year algebra students went from pre test means of 19% to post test means of 24% (a gain of 5 percentage points). The net gain for all students tested was 3%. Bishop Fisher High School — Level 2 Bishop Fisher’s math instruction is almost entirely textbook driven. Its math department meets regularly to discuss the challenges of their situation, but teachers perceive that the principal is an obstacle to substantive change. Teachers 93 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. are aware of college expectations in the field of math; three are recent college graduates with advanced degrees in math or hard sciences. Student achievement in math is self-reported to be poor. An outdated faculty handbook includes course descriptions for each course, but one teachers puts it best when she says, “When kids come into your class unable to do even half of what the last course was supposed to teach, you have no choice but to start back where the kids are at and get them as far as you can in the time you have them.” Another teacher reports, “We have no idea what other people are teaching and we constantly sit around in meetings complaining about how terrible everything is; unfortunately, we never come up with any real solutions. We have asked for placement testing and summer school and higher standards but the principal keeps saying we can’t exclude kids from taking higher level math course because the UC’s and Cal States require them.” The math teachers at Bishop Fisher seem genuinely interested in improving student achievement, but appear stymied by their perception that the principal is a roadblock. Interestingly enough, the researcher met with this principal after the questionnaires were received and got the sense that she was equally concerned about poor student performance in math and was very willing to try new approaches to turn around student failure. The communication between the principal and math chairperson is very poor and this seems to be the single most important factor that contributes to the lack of student achievement gain. 94 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Bishop Fisher High School’s student achievement gain was abysmal. Algebra 2 students went from pre-test means of 13% to post-test means of 12% (a loss of 1 percentage point) while first-year algebra students went from pre-test means of 11% to post-test means of 17% (a gain of 6 percentage points). The net gain for all students tested was slightly less than 3%. In a follow-up session with the department chair, the researcher asked why the chairperson thought there would be a performance loss among second-year algebra students. The chairperson reported that there was a long-term substitute in one classroom for several months of the year, and that those students had been complaining of not learning the material. Summary None of the schools in the study were assigned a congruence level of one. This may be due to the nature of Catholic Schools. There is more administrative oversight and faculty collaboration in smaller schools than in larger ones, and the schools studied all appeared to have strong administrations with a focus on student achievement. The schools assigned to Level Two on the scale had in common dedicated staffs who work hard to teach students, but who lack the opportunities to collaborate with peers for the purpose of defining curriculum, standardizing assessment, and improving instruction. Level Three schools had time to discuss issues of curriculum, instruction, and assessment, but were only beginning the 95 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. process of using those discussions as springboards for the improvement of teaching and learning. Level Four schools had engaged in adequate horizontal and vertical articulation about curriculum, instruction, and assessment to see some results in improved teaching and learning. They self-reported successes at focusing instruction around standards and focusing assessments around high expectations. None of the schools surveyed were Level Five schools. Though St. Elizabeth High School has worked hard at such things as lesson caucuses, common exams, common syllabi, and vertical/horizontal articulation, they have not yet created full congruence of curriculum, instruction, and assessment. Relationship Between Student Achievement and Congruence Level The last research question looked at the relationship, if any, between level of school congruence and student achievement gain. To answer this question, schools were rank ordered according to gain scores for Algebra 1 and Algebra 2. This allowed the analysis of gain scores in light of the school’s corresponding congruence level. Table 13 below includes the rank of each school in order from least achievement gain to greatest achievement gain in Algebra 1. To the right of school names and ranks is their Algebra 1 Achievement Gain scores and the school’s congruence level. 96 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. TABLE 13. Algebra 1 Achievement Gains in Rank Order with Congruence Levels RANK SCHOOL NAME ALGEBRA 1 ACHIEVEMENT GAIN CONGRUENCE LEVEL I Bishop Fisher 17 2 2 St. Agnes 24 2 2 St. Cecilia 24 2 3 St. Vincent 32 3 4 St. Gerard 36 2 5 St. Luke 40 2 6 Cardinal McIntyre 42 3 7 St. Martin 43 4 8 St. Elizabeth 47 4 9 Our Lady of the Rosary 50 4 10 St. Thomas 57 3 The data in Table 13 show that there is a strong positive relationship between Algebra 1 gain scores and congruence levels of schools participating in the study. A Pearson r test revealed a correlation of r = 0.69. In general, a school’s level of congruence increases as its student achievement increases. Table 14 includes ranked achievement gains for Algebra 2 and the corresponding congruence levels of participating schools. The Pearson Product Moment Correlation Coefficient for the relationship between the Algebra 2 gain scores and school congruence levels was r = .65. As was the case with Algebra 1 gain scores, there is a positive relationship between gain scores of Algebra 2 students and the level of congruence present in the school’s math departments. 97 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. TABLE 14. Algebra 2 Achievement Gains in Rank Order with Congruence Levels RANK SCHOOL NAME ALGEBRA 2 ACHIEVEMENT GAIN CONGRUENCE LEVEL I Bishop Fisher 11 2 2 St. Agnes 24 2 3 Cardinal McIntyre 34 3 4 St. Cecilia 35 2 5 St. Luke 37 2 6 Our Lady of the Rosary 41 4 7 St. Martin 43 2 8 St. Gerard 44 2 9 St. Vincent 45 3 10 St. Thomas 54 4 11 St. Elizabeth 64 3 The next area of interest in this analysis was that of overall gain scores and congruence level. To obtain the overall gain score, first- and second-year gain scores were averaged together. Table 15 shows the ranked order of schools by overall gain score and congruence level. TABLE 15. Ranked Overall Gain Scores and School Congruence Levels SCHOOL OVERALL GAIN CONGRUENCE SCORE LEVEL St. Elizabeth 39.5 4 Our Lady of the Rosary 22 4 St. Thomas 19.5 3 St. Martin 17 4 St. Vincent 13.5 3 Cardinal McIntyre 13 3 St. Luke 11.5 2 St. Gerard 8 2 St. Cecilia 5.5 2 St. Agnes 3 2 Bishop Fisher 2.5 2 98 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. These data reiterate the trend seen in the individual course analyses. Schools with higher levels of congruence tend to have higher levels of student achievement. All five schools with a congruence level of 2 had achievement gains less than 15. The three schools with congruence levels of 3 had achievement gains between 10 and 20 percentage points. The two highest performing schools also had the highest congruence levels. A Pearson Correlation coefficient of r = 0.82 was found to exist between these two variables (overall achievement and congruence level). To explore this trend, another level of analysis was undertaken. Schools were grouped by congruence level; average mean gain scores per congruence group were calculated and compared (see Table 16). The trend noted above appeared to hold. TABLE 16. Congruence Levels and Overall Student Gains by Course CONGRUENCE LEVEL ALGEBRA 1 MEAN GAIN ALGEBRA 2 MEAN GAIN 4 23.7 28.6 3 16.3 14.3 2 12.3 4.8 Table 16 reveals that schools with a congruence level of 4 had the highest level of mean achievement gain while schools at levels 3 and 2 had proportionately smaller achievement gains. Further, the data from the student achievement 99 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. measures and the school questionnaires support a presence of certain characteristics of congruence and student achievement in participating schools. Level Four Characteristics Common to the schools at Level Four is a focus on high-powered teaching and learning. Schools at Level 4 are standards-driven and link standards to assessment. “We meet each week to be sure we know what we are teaching, how we teaching it, and how we will know if the students get it” (teacher, St. Elizabeth High School). “I feel like all of us are a unit. We work together to help all students achieve. When one of us is having a problem teaching a particular topic, we all sit down to talk about ways we can be better at helping students understand. When the end of the semester comes, we don’t just throw away the exam results. We do an item analysis to plan ways we can make students more successful” (St. Martin’s Department Chair). Level Four schools have agreed on the essential skills for success in the next level of math, and have based these upon college and university expectations. A teacher from Our Lady of the Rosary High School stated, “We all went to a conference on the ELM (Entry Level Mathematics Exam administered to incoming freshmen by all California State Schools) and came back and decided what things we would eliminate from the courses. It was amazing how freeing that felt. 100 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Instead of having to cover all the textbooks, we were able to focus our attention on the really important things.” Further, Level Four schools exhibit strong departmental leadership. “Our Department Chair really focuses our attention on helping all students learn. Every department meeting has a time set aside to plan ways we can get higher performance out of the kids” (math teacher from St. Martin High School). The principal at St. Elizabeth praised the math chairperson for being, “One of the strongest department chairs. He runs a good meeting. The members of his department know that the department meetings will include some time to evaluate test results and discuss how those results can be improved.” Level Four schools also exhibit an awareness that they have not yet reached the panacea of teaching and learning. The chairperson at St. Martin said, “We know how far we have to go and that sometimes gets us depressed. But we can point to so many notable improvements in our curriculum, our teaching, the way we assess.” A final characteristic of Level Four schools deals with the role of the textbook. “Rather than serve as the curriculum itself, our textbooks provide the various problems from which we can draw examples and student work. Our teachers use the final exam as the basis of the curriculum and the textbooks to get the problems we need for students to solve. Every other school I’ve been at does it the other way around” (Math Chairperson, St. Elizabeth High School). 101 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Level Three Characteristics Level Three schools have begun congruence. They have common syllabi based on standards, but the source and rigor of those standards is sometimes unclear. “We use the California Math Framework, but I hear that is now out of date,” said one math teacher at St. Thomas High. The textbook is still the basis of the curriculum in Level Three schools. “We pretty much follow the textbooks. We have talked about changing the order of chapters, but all we’ve done is talk.” (Math teacher, Cardinal McIntyre High School) Assessments are still unique to each classroom and there is little agreement about the nature and rigor of assessments. “I have tried to get people to consider a common final exam for each course, and last semester we did pilot one in the Pre- Algebra class, but people are still reticent to move ahead with that” (Math Chairperson, St. Vincent High School). “I want to be sure I am grading the same way others are grading. As a first year teacher, the kids keep telling me I am too hard, but I have no idea if that’s true. It would be very helpful if we could have the same final and I could know I was grading students with the same expectations as the teacher across the hall who has been here for 15 years” (Math teacher, St. Thomas High School). Level Three schools use varying instructional methodologies and occasionally conduct lesson caucuses to improve teaching and learning. “One of the highlights of the past year was going to the CMC (California Mathematics 102 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Council) Conference in Palm Springs. All of us went and had a great time. At lunch one day, we talked about writing across the curriculum assignment that would satisfy the requirements of the vice-principal. We agreed to try something we learned in one of the workshops and it went great” (Math teacher, Cardinal McIntyre High School). A final characteristic of Level Three schools is their desire to improve teaching and learning and an awareness of specific ways to do so. St. Thomas High’s math chairperson said, “I know we need to complain less and do more. The key to improving is to try something we think will work and then talk about it and make it better. The Pre-Algebra final we gave last semester is a good example. If we had followed through on that, we could have made that the final for all Pre- Algebra courses in the future, and then moved to a new level the next year. Instead we let it drop.” One of Cardinal McIntyre’s teachers said, “We have plans to meet at least once a month next year to develop and implement a plan to improve our test scores and student success in higher level of math.” Level Two Schools Schools at Level Two on the congruence scale are ready to move away from isolation, but don’t yet have all the pieces necessary to implement a congruence plan. Teachers are aware of the problems with student achievement, but there is no consensus about how to remedy those problems. “I know our test scores suck, and kids keep coming back from college saying we need to do a better 103 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. job of preparing them. I just can’t seem to get any of the people in the department to see the problem and do anything about it” (Math chairperson, Bishop Fisher High School). Instruction is focused around completing the textbook rather than on meeting specific standards. “Once a long time ago, we had a workshop on setting course goals and objectives, and we wrote up some stuff for each course, but that all ended up with the department chair and hasn’t been seen since.. ..I try to cover what I can in the book and do what it takes not to get much grief from the administration” (Math teacher at St. Agnes High). Level Two schools lack consistency. “We have a teacher whom I’d nominate for Disney Teacher of the Year. She is awesome. Her kids are learning and they enjoy it. But across the hall, we have a slug who is only collecting a paycheck. We need to get everyone in the same room and come to agreement about standards and assessment. The curriculum course I recently took taught me a whole bunch of ways to do that, but I’m not sure I really can fight the system here.” (Math chairperson, St. Gerard High School) One of the teachers at St. Luke’s agrees, “We have a couple of teachers who are great and others who just don’t seem to care. Some of us sit down at lunch to talk about how to make things better, but when we invite the others, we get the cold shoulder.” 104 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Discussion Each of the three research questions of this study has resulted in valuable findings with potential to impact K-12 education. The first question sought to identify the nature of student achievement in first- and second-year algebra students enrolled in schools involved in the study. The research revealed that student performance was poor, even in the best-performing schools. The vast majority of students were performing well below level after a year had elapsed between pre- and post-testing. There were some schools where the performance gains were substantial; however, even in those schools, students’ performance was well below average. These findings are in consonance with the findings of both national and international studies of student performance. The National Assessment of Educational Progress (NAEP) and Third International Mathematics and Science Study (TIMSS) both paint a dismal portrait of student achievement. Mumane and Levy (1999) confirm that student performance in United States schools is well below that outlined in national standards. The second research question investigated the level of congruence in mathematics department, focusing on the extent to which there was both vertical and horizontal articulation between teachers about what mathematics to teach, how it should best be taught, and what evidence of learning (assessment) would be acceptable. Varying degrees of congruence were found to exist, ranging from schools where very little articulation took place to schools where measurable 105 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. efforts have been made to change the paradigm of classroom teacher from one of isolation to one of teaming with others to find best practices in curriculum, teaching, and assessment. Henderson and Hawthorne (2000) argue that such teaming is the only way to authentically and meaningfully improve schools. They maintain that “transformative educators seek insight about what students can do with what they know as well as how they are becoming active agents of their own learning” (p. 87). Transformative educators: 1) seek to identify clear and appropriate standards for student learning; 2) design performance-based assessments that challenge students to apply their knowledge and skills; and 3) evaluate both teaching and learning with an eye toward continual improvement. This research affirmed the existence of faculties with transformative educators. The final research question pondered the relationship, if any, between congruence and student achievement. A strong correlation of r=0.82 was found to exist between these two variables. The premise of the 1999 text, The New American High School, by David Marsh and Judy Codding is that student achievement can be significantly increased if there is collaboration by teachers and other decision makers about rigorous standards, meaningful assessments, and quality teaching to provide evidence of student learning. In places where such a paradigm is being implemented, student achievement seems to be on the rise as evidenced by such a strong correlation between congruence and achievement. Lord (2000) argues for “critical colleagueship,” an approach in which classroom practice is critically examined by the practitioners and redesigned in 106 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. light of national standards (pp. 190-197). Lord asserts that if teachers are retrained to consider teaching and learning in light of new standards, they will also need ample opportunity to develop new assessment skills and instructional strategies which align themselves with the rigor of the new curriculum. Overall these data gathered as a result of the three research questions do point to a relationship between levels of curricular congruence in a school and the student performance achievement gains in the school’s first- and second-year algebra courses. In general, schools involved in the study which had greater levels of curricular alignment (that is agreement about what should be taught, how it should be taught, and how it should be assessed) had greater student achievement gains during the course of a typical year of academic instruction. Those with less agreement about what should be taught, how it should be taught, and how it should be assessed had poorer performance gains among the first- and second-year algebra classes studied. Then too, alignment was associated with perhaps the factor most associated with gains in student achievement. It is this researcher’s opinion that the combination of common syllabi, common assessments, and lesson caucuses created conditions most conducive to increased student achievement. In schools where there was a plan for alignment of curriculum, instruction, and assessment, student achievement was greater. Schools in which there was substantive variety in curriculum, instruction, and assessment had poor student performance. The awareness on the part of teachers as to the importance of common standards, common instructional strategies, and common measurement methods created 107 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. climates where teacher and students could focus on high performance and achievement of rigorous standards. This chapter has presented the findings of the research study. The next chapter offers a summary of the study, major findings, conclusions, and implications. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. CHAPTERS SUMMARY, CONCLUSIONS, AND RECOMMENDATIONS Summary The extent to which there is alignment between what secondary mathematics is taught, how it is assessed, and how it is taught has been the focus of this study. That alignment and the articulation between and among teachers teaching the same course as well as those teaching courses prior to and after a particular course is termed “congruence” for the purposes of this study. This research specifically sought the relationship, if any, between congruence levels and student performance. Poor performance in mathematics by American students is not a new topic. The National Assessment of Educational Progress (NAEP) and other research consistently point to well below-average performance on mathematical algorithms and problem solving across grade levels. No one group of American students does substantially better than any other group. When compared to international averages, United States students perform well below the norm. Math education remains one of the foremost purposes of American schools. Mumane and Levy (1999) report that an understanding of underlying mathematical concepts is a skill required of high school graduates who want to be competitive in the workforce of the 2000s and beyond. The National Goals Education Panel (2001) reported that a lack of math skills, particularly algebra had the potential to “doom a child to a twentieth century lifestyle in the twenty-first century (p. ix).” 109 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. When mastered, algebraic concepts serve as a significant underpinning to so many of the skills required of those in the current workforce. Mumane and Levy’s “new skills for high school graduates” include the ability to analyze problems, decipher formulas, tackle dilemmas from a variety of thought processes, and think critically. The skills called for by the National Education Goals Panel and researchers such as Mumane and Levy are really nothing new. There is substantial agreement about what mathematics needs to be taught and learned at the various stages of education. If this premise is true, then the question becomes how to teach and assess student understanding. Congruence as used in this study, is an answer to that question. Articulation about the important topics, how to assess student understanding and how to best produce student understanding all are keys to improving student performance. National and International studies, such the Third International Mathematics and Science Study (TIMSS), have found that American mathematics teachers work very hard to teach students a great many topics. Those topics tend to be of minimal difficulty and voluminous in quantity. The emphasis is on performing algorithms rather than on critical thinking about mathematical processes. However, a model of effective congruence that produces impressive student achievement can be found in the Japanese system. Japanese teachers spend substantially more time than American teachers discussing with colleagues how to best approach the teaching of difficult subject matter. Japanese lesson caucuses include extensive peer coaching 110 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. and feedback and critical analysis of both teaching and learning processes during lessons. The Japanese national system of common exams in algebra and other mathematics courses results in a very clear understanding by Japanese teachers and students about what is to be learned and how learning will be assessed. The culture in American schools however does not lend itself to such collaboration. The individuality of Americans and the push for differentiated instruction has led many American teachers to oppose certain aspects of the standards, assessment, and accountability movements. (Fraryetal., 1993) Specifically, “the testing backlash” identified by the 2001 Business Roundtable found that teachers disliked standardized assessments when they perceived those assessments to be detached from standards and when accountability was linked with student performance judged primarily by those standardized tests. The lack of alignment across courses and classrooms breeds lower student achievement. That lack of alignment is due in large part to the lack of time teachers have for the necessary discourse about standards and assessment. Though this issue was not a focus of the study, several teachers and department chairpersons mentioned their concerns about the time it takes to agree upon course expectations, write common assessments, and plan lessons with colleagues. Purpose This study set out to determine the relationship, if any, between curriculum alignment and student achievement in high school algebra classes. I l l Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Research Questions The following research questions were posed: • What student performance gain exists in first and second year algebra courses for schools involved in the study? • To what extent was the mathematics curriculum aligned within a sample of schools? What was the level of congruence within the schools? • What is the relationship between student performance gain scores and the congruence levels within schools studied? Methods and Procedures The sample included 11 schools and slightly more than 2400 students enrolled in 80 sections of first- or second-year algebra classes at those schools. Students were given pre- and post-tests created by the University of California/California State University Mathematics Diagnostic Testing Project (UC/CSU MDTP). Students in the first-year algebra classes took an Algebra 1 Diagnostic test which measures student understanding of first-year algebra topics while students enrolled in the second-year algebra courses were administered the MDTP Intermediate Algebra Diagnostic test which measures student understanding of second-year algebra topics. Only students who took both the pre- and post-tests were included in the data analyzed for this study; students absent from one or both days of testing were excluded from the data. 112 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Data were collected to determine congruence levels from the department chairpersons and mathematics teachers at schools participating in the study. A survey measuring congruence of curriculum, instruction, and assessment and horizontal and vertical articulation about these topics was administered to all mathematics faculty at participating institutions. Variables The two major variables at work in the study were student achievement and congruence level. Student achievement gains were calculated by taking the difference in mean post-test scores and pre-test scores for each course within a school. Congruence levels were calculated using a five-point scale developed by the researcher for this study, and were assigned to each school. Data Analysis. Student achievement gains for first- and second-year algebra students were analyzed separately and then collectively for participating schools. Average gain scores by course and for both courses were compared. Survey data and interview/observation data were analyzed by the researcher using the congruence scale developed for this study. These data were reviewed for the extent to which mathematics teachers at participating schools accomplished the following: 113 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. • Agreed upon and published course syllabi with common course learning outcomes linked to the expectations of college and university mathematics programs; • Discussed teaching strategies and analyzed the teaching and learning processes as a tool for improving student achievement; and • Administered common assessments that measured student learning of the common outcomes. Selected Findings The major findings of the study are as follows: 1. Student performance in first- and second-year algebra courses was very poor. Even the highest performing schools had the majority of students scoring less than 70 percent correct after a year of algebra instruction. Schools in the study could be divided into three subgroups - those who students made very little or no growth during the year, those whose students made measurable, but small growth during the year, and those whose student performance gains were substantial. In general, the performance level of students in first-year algebra mirrored that of their school’s counterparts in second-year algebra courses. These findings parallel those of the National Assessment of Educational Progress which found that less than ten percent of graduating seniors were at the “proficient” level in second-year algebra skills. 114 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 2. The second research question revealed that schools in the study ranged in congruence levels from two to four. Those at Level Two had an awareness of the need to discuss standards for teaching and learning but had made little progress toward engaging in such discussions. These schools evidence a willingness to engage in discussions about effective assessment and standardized syllabi but no such discussions were yet in place. Schools at Level Three had begun such discussions and made some progress toward unifying the curriculum, standards, and assessment, while schools at Level Four had achieved common syllabi, common assessments, and some articulation about how to best teach for student learning. The highest levels of congruence were synonymous with schools in which substantial efforts had been made to agree upon course content, pedagogy, and assessment methodologies. 3. A strong relationship (r = 0.82) between congruence levels and student achievement gains was reported for schools participating in the study. Though such a relationship cannot be said to be causal, it appears from the study that when teachers engage in discussions about what to teach, how to assess it, and how to best teach for understanding, student performance is better than in situation where such congruence is lacking. Like the highest performing countries in the TIMSS study, those students whose teachers caucused about what to teach, howto teach it, and how to assess it were the highest performing. 4. Teachers in the study reported that lack of available time for articulation was a major factor in preventing such articulation from taking place. Odden (2001) 115 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. agrees that financial resources must be used to support time for teachers “to collaborate with other teachers” (p. 89). The Brazosport Independent School District in Texas is an example of where such focused collaboration has positively impacted student performance (Schmoker, 2001, pp. 105-106). 5. Schools in which common course expectations did not exist were also schools where the lowest performance gains could be seen. Conversely, agreement about course content was associated with higher student performance. Wiggins (1998) argued that, “If everything is important, then nothing is important.” Schmoker (2001) calls the agreement about what is important “common instructional currency.” He maintains that, “this commonality ... mightily promotes clarity, focused interaction, and thus collective invention” (p. 88). 6. Schools in which common assessments were administered were more likely to outperform schools in which no common assessments were administered. Schmoker (2001) concluded in his analysis of high achieving school districts that “end-of-course-assessments for every course at every grad level” was one of several key strategies for improving student achievement. 7. Some dissatisfaction with standardized tests, particularly the SAT9, was noted for low-performing schools. One of the teachers in this study complained that “these tests don’t measure the same things we teach, and yet somehow we are going to be evaluated on the student performance on these tests. We are told not to teach to the test and then the test is so different from the textbook (teacher from St. Martin High School).” 116 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Conclusions The findings led to the following conclusions: 1. Students in mathematics classes in schools in which teachers of the same course agree upon a curriculum outperform their counterparts in schools with teachers who are isolated and have not reached consensus with peers about what content should be mastered within a course. 2. Students in mathematics classes in schools in which teachers of the same course agree upon the best instructional strategies for optimal student learning outperform their counterparts in schools with teachers who are isolated and do not dialog with peers about quality teaching and learning. 3. Students in mathematics classes in schools in which teachers of the same course agree upon and administer common assessments linked to college/university expectations outperform their counterparts in schools with teachers who administer only teacher-made and textbook-provided assessments. A teacher in this study commented, “There is so much disconnect between what tests like the SAT expect kids to know and what we are teaching in schools. It is no wonder our kids do so poorly. If our high school curriculum was aligned with these tests, our kids would do better (math chairperson from Bishop Fisher High School).” 4. Vertical and horizontal articulation by teachers and school district staff can help increase student achievement by insuring that secondary teachers know the expectations of college academic departments. When such expectations are 117 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. clear, secondary curricula, instruction, and assessment can be aligned to those expectations. 5. Several inhibitors exist to congruence, most notably the lack of time for collaboration, an attitude by some that external assessment is unnecessary and intrusive, and a disagreement by a few about what the most important topics are in secondary math courses. Recommendations The following recommendations evolved from this study: 1. At the school level, principals, department chairs, and teachers need to give serious consideration to the value of common course expectations for multiple teachers teaching the same course. These expectations should be aligned with those of both previous courses and college and universities into which the students will matriculate. No longer should the textbook drive the curriculum; rather, serious consideration needs to be given to the most significant topics for student understanding. 2. School administrators should realize that teachers need to be given time to collaborate with each other in evaluating the teaching and learning process. Teachers must be allowed to closely analyze teaching strategies with peers horizontally and vertically (that is, with those who teach the same course and with those who teach courses prior to and after their particular course) and improve teaching strategies such that student achievement improves. This 118 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. caucusing must allow teachers to identify the best way to teach mathematical concepts and to implement common assessments linked to specific expectations of colleges and universities into which students matriculate. Research indicates that education remains one of the most isolated professions (Miles & Hombeck, 2000). Teachers, unlike most other professionals, are not often encouraged and prodded to engage in regular discourse with peers about the expected outcomes, the best means to achieve those outcomes, and the best ways to evaluate the effectiveness of the means used to achieve the expected ends. Such discourse is at the heart of congruence as understood in this study. Teachers are put into classrooms with textbooks and told to teach. Faculty meetings focus on school operation. Occasionally, an administrator comes into the classroom for a 40-50 minute period for a formal evaluation of teaching. Student outcomes are largely ignored (Mohrman et al., 1995). Assessments are viewed as “evil intrusions” and valued little. Examining student work and the development and use of assessment outside those developed by individual teachers are only beginning to emerge as acceptable practices in schools (Mumane, 1996). The culture of isolationism in schools needs to be replaced by a culture of collaboration, a culture in which teachers do not fear critical feedback from colleagues, but welcome it is a vehicle for improving both instruction and student achievement. 3. District, state, and national policymakers should insist that common assessments be administered regularly in courses, especially courses in which 119 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. different teachers are teaching the same course. These assessments should be carefully analyzed for alignment to the course curriculum and should be used to inform teaching and learning. Teachers should teach to the skills and knowledge assessed on these tests. We must move out of a culture in which teaching to the test is frowned upon and move toward a culture in which students are well prepared for what they will face on high-stakes assessments. This is not to say that teachers should teach specific problems and solutions, but students should know the types and nature of assessments they will undertake, as well as the best strategies to master material and demonstrate understanding. 4. Teacher education programs must encourage and teach student-teachers the kind of faculty collaboration this study found positively associated with student achievement. If schools of education promote congruence and teach skills that support vertical and horizontal articulation, new teachers can reshape school cultures from the bottom up. 5. Finally, the American public needs to support teachers and schools in their efforts to move out of the culture of isolationism. Financial resources should be brought to bear on these efforts. Time for teacher collaboration, professional development on teaching to high-stakes assessments, and retraining teachers to participate in critical conferences to improve teaching and learning all will cost money. School leaders may need to prioritize expenditures, but additional money will most certainly be required. 120 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. This research demonstrates the need for teachers and school leaders to engage in regular discourse about what should be taught, how it should be taught, and how it should be assessed. That discourse should take place between those teaching the same levels and courses, as well as with those at lower and higher levels of instruction. This vertical and horizontal congruence cannot help but improve the caliber of teaching and learning going on in America’s schools. 121 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. REFERENCES Bennett, N. and Desforges. C. (1988). Matching classroom tasks to students' attainments. Elementary School Journal 88.221-234. Blum, W. andNiss, M. (1991). Applied mathematical problem solving, modeling, applications, and links to other subjects -- State trends and issues in mathematics instruction. Educational Studies in Mathematics 22. 37-68. Bracey, G. (1998). TIMSS, rhymes with'Dims,'as in'Witted'. Kappan 19(9}. 686- 688. Brown, S. and Walter, M. (1993). Problem posing: Reflection and applications. Hillsdale, NJ: Erlbaum. Burrill, G. (1997). President's report: Choices and challenges [Adapted from the Presidential Address at the 75th Annual meeting of the National Council of Teachers of Mathematics - Saint Paul, MN, 18 April 1997]. Journal for Research in Math Education 28(51.602-612. California Academic Standards Commission. (1997). Mathematics content standards for grades K-12. Sacramento, CA: California Department of Education. California State Board of Education. (1992). Mathematics framework for California public schools: Kindergarten through grade twelve. Sacramento, CA: California Department of Education. California State University. (1990). Entry Level Mathematics Examination: A primer. Office of the Academic Vice President: California State University. Clarke, D. (1997). The changing role of the mathematics teacher. Research in Mathematics Education 28(3i. 278-308. Commission on Teaching Standards for School Mathematics. (1991). Professional standards for teaching mathematics. Virginia: National Council of Teachers of Mathematics. Corett, H. and Wilson, B. (1991). Testing, reform, and rebellion. Norwood, NJ: Ablex. 122 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Duke, D. (1998). Challenges of designing the next generation of America's schools. Kappan 79(91.688-694. Firestone, W., Mayrowetz, D., and Fairman, J. (1998). Performance-based assessment and instructional change: The effects of testing in Maine and Maryland. Educational Evaluation and Policy Analysis 20(2). 95-114. Forgione, P. (1998). Responses to frequently asked questions about 12th-grade TIMSS. Kappan 79(10). 769-773. Frary, G. et al. (April 15, 1993) “The opposition to testing: The real reasons” [Presented at the National Conference on Standards and Assessment in Las Vegas, NV]. Green, P., Dugoni, G., Ingel, S., and Cambum, E. (1992). Statistical analysis report of the national education longitudinal study of 1988: A profile of the American high school seniors in 1992. Washington, DC: U.S. Department of Education Office of Educational Research and Improvement. Greenleaf, D. (1997). Common course syllabi, placement examinations, and end- of-course examinations for first- and second-year algebra courses. Retrieved January 15, 1997, from St. Matthias High School Database. Greenleaf, D. (1998). Electronic References: St. Matthias High School Mathematics Department Handbook. Retrieved October 4, 1998, from http://www.stmatthiashs.org. Henningsen, M. and Stein, M. (1997). Mathematical tasks and student cognition: Classroom-based factors that support and inhibit high-level mathematical thinking and reasoning. Journal for Research in Mathematics Education 28(5). 524-549. Johnson, D. and Kuby, R. (2000). Elementary Statistics. New Jersey: Woodcock Press. Lampert, M. (1990). When the problem is not the question and the solution is not the answer: Mathematical knowing and teaching. American Educational Research Journal 27. 29-63. Lord, B. (March 12,2000). Telephone interview based on the article pending publication “Critical Colleagueship.” 123 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Manaster, A. (1998). Some characteristics of eighth grade mathematics classes in the TIMSS videotape study. American Mathematical Monthly. November, 793-805. Mathematics Diagnostic Testing Project. (1990). Informational Brochure (Available from the UC/CSU MDTP Scoring Centers across California). Mathematics Diagnostic Testing Project. (1996, Fall). Newsletter (Available from the UC/CSU MDTP Scoring Centers). Marsh, D. and Codding, J. (1999). The New American High School. Thousand Oaks, CA: Corwin Press. Mazzo, J. et al. (1998). NAEP Findings in mathematics and science: A comparison of data from the past twenty years. Retrieved July 1, 1998, from United States Department of Education (on CD-ROM). Meserve, B. and Suydam, M. (Eds.). (1992). Mathematics education in the United States. Paris: UNESCO. Miles, D. and Hombeck, A. (March 15, 2000). “Teacher Isolationism” [part o f a presentation at the National Conference on Standards and Assessment in Las Vegas, NV]. Mullis, L, Martin, M., Beaton, A., Gonzalez, E., Kelly, D., and Smith, T. (1998). Mathematics and science achievement in the final year of secondary school: lEA's Third International Mathematics and Science Study (TIMSS). TIMSS International Study Center. Mumane, R. and Levy, F. (1996). Teaching the new basic skills: Principles for educating children to thrive in a changing economy. New York: The Free Press. Mumane, R. and Levy, F. (1999). Presentation to the Association for Curriculum Supervision and Development [based on Teaching the new basic skills: Principles for educating children to thrive in a changing economy. New York: The Free Press]. National Advisory Committee on Math Education. (1975). NACOME Report to the Nation. Washington, DC: United States Government Printing Office. National Advisory Committee on Math Education. (2000). NACOME Report to the Nation. Washington, DC: United States Government Printing Office. 124 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. National Education Goals Panel. (1994). The national education goals report: Building a nation of learners. Washington, DC: United States Government Printing Office. National Education Goals Panel. (2001). The national education goals report: Building a nation of learners. Washington, DC: United States Government Printing Office. National Education Goals Panel. (1998). National education goals panel report on the progress toward the President's stated national goals [CD-ROM]. National Research Council. (1989). Everybody counts: A report to the nation on the future of mathematics education. Washington, DC: United States Government Printing Office. National Research Council. (1997). Preparing for the 21st century: The education imperative. Washington, DC: National Academy Press. National Science Education Board and National Research Council. (1998). High school mathematics at work: Essays and examples for the education of all students. Washington, DC: National Academy Press. Odden, A. (2001). The new school finance: A Kappan Special Section on school reform. Kappan 83:1. 85-91. Office of the Academic Senate, and Office of the President. (1996). Statement on competencies in mathematics expected of entering college students. Retrieved August 15, 1996, from http://www.ucop.edu/senate. Price, J. (1996). President's report: Building bridges of mathematical understanding for all children [Adapted from the Presidential Address at the 74th Annual Meeting of the National Council of Teachers of Mathematics in San Diego, California on 26 April 1996]. Journal for Research in Mathematics Education 27(51 603-609. Reese, C. et al. (1997). National Assessment of Educational Progress: 1996 report card for the nation and the states. Washington, DC: United States Government Printing Office. Riley, R. and Stigler, J. (1995). An overview of the TIMSS Videotape study: Findings and implications (Videocassette). Washington, DC: United States Government Printing Office. 125 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Riley, R. (January 9,1998). State of mathematics education: Building a strong foundation for the twenty-first century. In A speech presented at the annual meeting of the American Mathematical Society. Washington, DC: Publisher. Schmoker, M. (2001) The results fieldbook: Practical strategies from dramatically improved schools. Alexandria, VA: The Association for Supervision and Curriculum Development [ASCD Product No. 101001]. Silver, E. and Cai, J. (1996). An analysis of arithmetic problem-posing by middle school students. Journal for Research in Mathematics Education 27(5 J. 521-540. Smith, M. (1991). Put to the test: The effects of external testing on students. Educational Researcher 20(5J. 8-12. Stevenson, H. and Stigler, J. (1992). The learning gap: Why our schools are failing and what we can learn from Japanese and Chinese education. New York: Touchstone. Stigler, J. and Hiebert, J. (1997). Understanding and improving classroom mathematics instruction. New York: Touchstone. Stigler, J. (1997). TIMSS video study of eighth grade mathematics classes in Germany, Japan, and the United States (Videocassette). Washington, DC: United States Government Printing Office. Stigler, J. et al. (1997). The TIMSS videotape classroom study: Methods and findings from an exploratory research project on eighth-grade mathematics instruction in Germany, Japan, and the United States. Washington, DC: United States Department of Education. Taylor, J. (1998). The importance of workplace and everyday mathematics [Published in High School mathematics at work]. Washington, DC: National Academy Press. Tucker, M. and Codding, J. (1998). Standards for our schools: How to set them, measure them, and reach them. San Francisco: Jossey-Bass. United States Department of Education. (1997). Standards: making them useful — Lesson from the research and development period. Retrieved August 15, 1997, from http://www.ed.gov/pubs/Standards/lessons.html. 126 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. United States Department of Education. (1998). NAEP assessment data through 1996: CD-ROM including all tables, standard error tables, and supplemental information [CD-ROM]. Washington, DC: United States Government Printing Office (available from EDPUBS at www.ed.gov). University of California Office of the Academic Senate. (1996). Remedial course taking patterns: A report to the academic senate. Berkeley, CA: University of California (available from the Office of the President of the University of California). Wells, B. (January 18, 1998). Purpose and activities of the Mathematics Diagnostic Testing Project for the University of California and California State University systems (Telephone interview). Los Angeles, CA: unpublished. Wells, B. (February 8, 1999). Purpose and activities of the Mathematics Diagnostic Testing Project for the University of California and California State University systems (Telephone follow-up interview). Los Angeles, CA: unpublished. Wiggins, G. (1998). Understanding by Design. Alexandria, VA: Association for Supervision and Curriculum Development. Wood, T. and Sellers, P. (1996). Assessment of a problem-centered mathematics program: Third grade. Journal for Research in Mathematics Education 27(3). 337-353. Wood, T., Cobb, P., and Yackel, E. (1990). The contextual nature of teaching: Mathematics and reading instruction in one second-grade classroom. Elementary School Journal 90.497-513. Wood, T. and Sellers, P. (1997). Deepening the analysis: Longitudinal assessment of a problem-centered mathematics program. Journal for Research in Mathematics Education 28(2). 163-186. Zbiek, R. (1998). Prospective teachers' use of computing tools to develop and validate functions as mathematical models. Journal for Research in Mathematics Education 29(2). 184-201. 127 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. APPENDICES Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. APPENDIX A MATH STUDY COMMUNICATION Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Mark P. Ryan 11635 Downey Avenue #8 Downey, CA 90241 562-861-6205 November 9, 1998 Dear Math Study Participant: Enclosed is the math congruence questionnaire for you to complete. If you could possibly have one or two other teachers at the school fill out separate questionnaires, that would be wonderful. Feel free to duplicate the questionnaire as necessary. You can return them to me via FAX at 562-869-8652 or mail at 7851 Gardendale Street, Downey, CA 90242. A stamped, self-addressed envelope is included if you indicated mail was your preferred method of communication. If you have any questions as you are completing the study, please feel free to call me at 562-861-2271. Thank you in advance for your assistance. Sincerely, Mark P. Ryan 130 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. MATH CONGRUENCE STUDY Survey Format Preference Sheet FAX FOR: Mark Ryan 562-869-8652 FROM: School______________________________ Contact person ______________________ My preference for survey format is: send it to me by mail to this address: _____________ City ____________ send it to me by e-mail at this e-mail address: _____ send it to me by FAX at this number: (____)_______ let’s do the survey over the phone. Call me at (____) best time to call __________ Reproduced with permission of the copyright owner. Further reproduction prohibited without permission APPENDIX B MATH DEPARTMENT CONGRUENCE STUDY QUESTIONNAIRE Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. MATH DEPARTMENT CONGRUENCE STUDY QUESTIONNAIRE (This survey should take approximately 30 minutes to complete. Responses will not be linked by name to a particular individual or school.) I. Does you school math department currently have or engage in any of the following (Answer YES or NO): • Common final exams within a course (the same final given by more than one teacher) ______ • Common quizzes or tests at times other than final exam time (the same quizzes or tests given by more than one teacher) ______ • Common syllabi within a course (the same syllabus used by more than one teacher) ________ • Placement exams for students to take before they enter a particular math course at your school _______ • Common instructional planning time where teachers discuss instructional strategies for a particular topic within a course _______ • A published set of course objectives for each math course ________ • Regular administration of the UC/CSU Mathematics Diagnostic Testing Project Tests to one or more groups of your students ________ • Department meetings (at least once a year) in which college expectations of entering freshmen are discussed ________ • Copies of one or more of the following documents which are DISCUSSED at department meetings: NCTM Standards, California Math Standards, other standards established by ‘outside’ agencies such as the federal or state government_________ 133 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 2. On the following scale, rate these statements about your math students, teachers and department as a whole: 5 = STRONGLY AGREE 4 = AGREE 3 = AGREE SOMEWHAT 2 = DISAGREE SOMEWHAT I = STRONGLY DISAGREE Math instruction at this school is largely textbook driven: 5 4 3 2 1 We very often use textbook assessments (such as those that come in teacher resource kits): 5 4 3 2 1 Our students are aware of college math expectations: 5 4 3 2 1 We have a common idea of what our objectives are for classes: 5 4 3 2 1 We are aware of national and/or state standards in math: 5 4 3 2 1 Our students are well prepared for college math course work: 5 4 3 2 1 Our teachers are aware of college math expectations: 5 4 3 2 1 All of our teachers assess student math achievement similarly: 5 4 3 2 1 We get together as a math department regularly to discuss ways to improve teaching and learning: 5 4 3 2 1 We give placement tests for entrance into math course work that adequately predict student success in those courses: 5 4 3 2 1 Our math teachers have high expectations for students: 5 4 3 2 1 Students at our school perceive our math classes as preparing them well for college math course work: 5 4 3 2 1 Our teaching methods are aligned with current standards as established by groups such as the National Council of Teachers of Mathematics: 5 4 3 2 1 Students come into math courses ill-prepared for the level of work expected of them: 5 4 3 2 1 134 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 3. Would you please comment on the reasons for your responses to previous statements? Math instruction at this school is largely textbook driven: We very often use textbook assessments: Our students are aware of college math expectations: We have a common idea of what our objectives are for classes: We are aware of national and/or state standards in math: Our students are well prepared for college math course work: Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Our teachers are aware of college math expectations: All of our teachers assess student math achievement similarly: We get together as a math department regularly to discuss ways to improve teaching and learning: We give placement tests for entrance into math course work that adequately predict student success in those courses: Our math teachers have high expectations for students: Students at our school perceive our math classes as preparing them well for college math course work: 136 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Our teaching methods are aligned with current standards as established by groups such as the National Council of Teachers of Mathematics: Students come into math courses ill-prepared for the level of work expected of them: 137 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 4. For the purposes of this study, the research is examining the extent to which student achievement is related to math department congruence. Congruence refers to such elements as: • The extent to which teachers have common expectations for students • The extent to which there are common assessments beyond a single classroom • The extent to which high school course work is aligned to the expectations of colleges and universities • The extent to which NCTM standards for teaching and learning are being met • The extent to which the standards of other groups such as the California Commission on Standards are being met • The extent to which teachers engage in common planning time • The extent to which instruction and assessment are driven by standards and objects rather than textbooks • The extent to which students and teachers are aware of college math expectations How would you describe your school’s level of math department congruence on the following scale (CIRCLE ONE): VERY HIGH HIGH SOMEWHAT HIGH SOMEWHAT LOW LOW Why did you place your school at that level? School name Your position at the school_____________________ 138 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. APPENDIX C MDTP TESTING INFORMATION 139 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Dear Math Congruence Study Participants: Please include this signed letter with your Answer sheets when you return them to the UCLA MDTP Scoring Center. Thank you, Mark Ryan TO: UCLA MDTP Scoring Center FROM: ______________________ High School You have our school’s permission to send classroom summary data to Mark Ryan as part of his study on Math Department Congruence. We understand he is NOT receiving information on particular student scores; he is only receiving teacher summary sheets showing entire- class performance. Signed_______________________ Date 140 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Id t s S k P A R T A : A ll In fa rn ia ta n m u t bm am M m tL Contact School: Addrats: City: Hava you mad MDTP tacts batore? □ y a * □ no [School coda: PART A: indicate thm number needed In dia boaaa undemeetfi tfia aoomerittm tcet numb*. TEST TYPE wuaearor eueaaens CALCULATOR OPTIONAL CALCU RffOl LATOR KRMD Enoueii 1 SBMWfl # I b ie in n j 1 SOMMh i A LGEBRA READINESS SO 50 m in u « AW 090* 1 AW090S* / V ARS0X92 / 1 AW0X92S / ELEMENTARY ALGEBRA A (gofer*. / C t c s s e ^ 45 SO 4 5 nunutts SO mtmms ^ E A 5 Q A 9 ^ \ e m 9x» i i \ p 4 5 X 9 f i GEOM ETRY READINESS 45 4 5 nunutas GR45A83 (M4SASS 9)49X 91 G U A M S IN TE R M ED IA TE A L G EB R A f c r f \ /s«-t>•" .i s5 C /a S i v j 45 45 minutes ^ IA 4 5 A 9 0 ^ s m t m z s srV t 4 sfJ& xsA S f \ MATH ANALYSIS READINESS 45 minutes MR4S&92 Mn«Jks3S A l \ •Vf4ik*j4^ PRECA LCU LU S 60 minutes PC40A33 p e J o jio s I I / 1 P C L a : / \ Sf'iC X E S PRECA LCU LU S minutes PCfiSAai \ j PtjcOASls PkSOXSI 1 1 ICS3X5CE CALCULUS READINESS a ; S i minutes / \ tn4C X I / \ CALCULUS READINESS i 30 minutes ; ! i / \ I \ I \ 1 1 1 BEGINNING CALCULUS 1 6C minutes * / \ j 6C33X3* 1 j i 1 1 k \ You are ancouragea to m e the lasts mora man once. so o m s a otaer witn this in mine. Please ao not allow stuoents to write m tne test ooohiets. Pius X • St. Matthias Digit School A C tcnoiic E d U C ltto n It A n A a v a n u f * For u f* M ark P . Ryan Vica Principal £ C iaraen d alc S: U n w n o . C A « 0 2 4 : - 4 t<X * <56:» AM -22?! FA X ! S t Z \ *►»•.<*,«- Tctai n u m se r s? c a s s e s ! i a /A _ _ ■ Tsrai n u rrs e r c f tea c n e rs -v., .._ _ _ _ 1 >oi<ii ftum oer c t a n sw e r s n e e ts reo u e^tcc ! i A / . - i _ _ _ _ _ A/w - ___ t ‘ ! Cate o rd e r received anir t 1 iwiate s u c s t .e s s h is o e c C a te *cCfc:vec ts r s s sr> rc 141 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. CALIFORNIA STATE UNIVERSITY/UNIVERSITY OF CALIFORNIA MATHEMATICS DIAGNOSTIC TESTING PROJECT TEST RESULTS INTERPRETATION GUIDE Fall 1991 These materials have been prepared with the support o f the California Slate University, the University o f California, and the California Academic Partnership Program. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. T h e T est R esu lts Repo r t DESRIPTION AND SUGGESTIONS The Test Results Report begins with several pages of summary information intended to be used primarily by the instructor. These pages are followed by a set of individual student letters. Each student should be given his or her own individual letter, encouraged to discuss the results with his or her instructor and parents, and should keep the letter for possible later use in tutorials or specially designed summer courses that may allow students to concentrate on those topics needing review. CLASS RESULTS The class averages for each topic and for the entire test are reported on the Class Results page. These averages are expressed both as raw scores and as percentages. The mastery level is also reported for each topic. Finally, both the number and the percentage of students scoring at or above the master level is reported for each topic. The information reported on the Class Results page provides some indication of the class’s overall level of preparedness for the next mathematics course and in each of the topics of the test. The topic results may be helpful in identifying areas in need of further review. They also provide an indication of which topics to examine first on the following page of the report. The total score may be a useful indication of the level at which the next course should begin. GRAPHIC DISPLAY OF CLASS RESULTS The Graphic Display o f Class Results page contains bar graphs indicating the percentage of students in the class answering each item correctly. These graphs are grouped by the topics of the test. Within each topic, the graphs are ordered by the percentage of students correctly answering each item. The graphs show the relative difficulty of each item for the entire class. The bar graphs provide a quick way of identifying those items that were most difficult for a class. These more difficult items might be examined first on the following page in the report. Similarly, the bar graphs provide a quick way of identifying the items and topics on which the students did best and, therefore, on which the curriculum and instruction are working most successfully. 143 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. ITEM ANALYSIS The Item Analysis page reports the performance of the class on each item in the test. The results are presented in terms of the percentage of students choosing each response. For each item, the following is listed: the item number; the key (i.e., the correct response for the item); the topic of the item (or a code for the topic — the codes are listed on the cover letter of the report); the percentage of students who did not select any of the possible answers for the item (the “omits”); and, for each possible response (A) through (E), the percentage of students who selected that response. An asterisk (*) appears with the percentage of students who selected the key. Suppose that the following analysis is given for item 43: OMIT A B C D E 4 *17 4 56 4 13 Suppose further that item 43 is the following: (A)-a b (B) ab (C) (a - b)2 (D) a2 + b1 (E) 2 - (a/b) - (Jb/a) The popularity of response (C) suggests that most of the students converted the denominator to 1/a - b) before inverting and multiplying. It appears that they are still adding (or subtracting) fractions by adding denominators instead of first finding a common denominator. Of course, other mistakes could also lead to the selection of (C). For example, each term in the denominator may have first been inverted, giving a - b, and then the result inverted and multiplied to yield (C). In any case, an analysis of students’ responses to this item would show that a large percentage of students are having difficulty combining literal fractions. INDIVIDUAL STUDENT RESULTS The Individual Student Results page gives summary information about each student’s performance on the test. This information includes the total number of items that the student answered correctly, the number of items in each topic that the student answered correctly, the number of items to which the student responded, and the last item to which the student responded. An asterisk (*) appears with each 144 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. score that is below master level. The number of items attempted and the last item attempted may provide some indication of the extent to which a student had difficulty in finishing the test rather than in solving certain problems on it. INDIVIDUAL STUDENT LETTERS Following the summary information reporting the class results as a whole are individual one-page letters addressed to each student in the class. These letters give the student his or her total score on the test along with scores in each topic. Further, the topics are separated into those in which the student scored at or above master level, those in which the student needs some review, and those in which the student needs substantial review. 145 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

An analysis of program planning in schools with emerging excellence in science instructional design

PDF

Examining the factors that predict the academic success of minority students in the remedial mathematics pipeline in an urban community college

PDF

Factors influencing teachers' and administrators' identification of diverse students for gifted programs in Title I schools

PDF

How effective schools use data to improve student achievement

PDF

Early career teachers' views of their elementary science teaching methods courses: The relationship between preservice preparation and the realities of the first years of teaching

PDF

A cross-site analysis of the extent of implementation of the California mathematics and science frameworks.

PDF

A history of the development of the California Science Content Standards: 1990--2005

PDF

An analysis of the use of data to increase student achievement in public schools

PDF

Design, implementation and adequacy of using student performance data and the design's link to state context for assessing student performance: A case study

PDF

Environmental education in community -based coastal resource management: A case study of Olango Island, Philippines

PDF

An investigation of the impact of selected prereading activities on student content learning through laboratory activities

PDF

Districtwide instructional improvement: A case study of a high school in the Los Coyotes High School District

PDF

Connecting districts and schools to improve teaching and learning: A case study of district efforts in the Los Coyotes High School District

PDF

Districtwide instructional improvement: A case study of an elementary school in the Beach Promenade Unified School District

PDF

Districtwide instructional improvement: A case study of a high school in the media unified school district

PDF

Education for change in a changing Nigerian Igbo society: Impacts of traditional African and western education on the upbringing of Igbo children

PDF

Factors contributing to the academic achievements of Hispanic gifted children

PDF

How districts and schools utilize data to improve the delivery of instruction and student performance: A case study

PDF

An evaluation of the use of microcomputer-based laboratory instruction on middle school students' concept attainment and attitudes towards computer -based instruction

PDF

Connecting districts and schools to improve teaching and learning: A case study of district efforts in the Orchard Field Unified School District

Asset Metadata

Creator Ryan, Mark Patrick (author)

Core Title A study of the relationship between student achievement and mathematics program congruence in select secondary schools of the Archdiocese of Los Angeles

School Rossier School of Education

Degree Doctor of Philosophy

Degree Program Education

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag education, curriculum and instruction,Education, Mathematics,education, philosophy of,Education, Religious,OAI-PMH Harvest

Language English

Contributor Digitized by ProQuest (provenance)

Advisor Marsh, David D. (committee chair), [illegible] (committee member), McComas, William (committee member)

Permanent Link (DOI) https://doi.org/10.25549/usctheses-c16-231192

Unique identifier UC11339304

Identifier 3073844.pdf (filename),usctheses-c16-231192 (legacy record id)

Legacy Identifier 3073844.pdf

Dmrecord 231192

Document Type Dissertation

Rights Ryan, Mark Patrick

Type texts

Source University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Access Conditions The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the au...

Repository Name University of Southern California Digital Library

Repository Location USC Digital Library, University of Southern California, University Park Campus, Los Angeles, California 90089, USA