University of Southern California Dissertations and Theses

Universal algebra: an evaluation

(USC Thesis Other)

Universal algebra: an evaluation

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content

UNIVERSAL ALGEBRA: AN EVALUATION

By

Alicia R. Roman

A Dissertation Presented to the
FACULTY OF THE USC ROSSIER SCHOOL OF EDUCATION
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulfillment of the Requirements for the Degree
DOCTOR OF EDUCATION

August 2021

Copyright 2021 Alicia R. Roman

ii

Dedication
I dedicate this dissertation to my wife. Your support, love, and endurance through the last three
years has been nothing short of amazing. The true measure of love is pushing me through the
hard times and encouraging me when I thought I could not finish. Although I wrote this
dissertation, I truly believe we earned it together.
iii

Acknowledgements
Thank you to my family who always believed in me. Thank you to daughter who also knows
what it is to a smart black woman in a world that was not designed for you. I can’t wait for you
to call me Dr. Mommy! Thank you to my mom who was trail-blazing in the 1980s and I didn’t
even notice or appreciate it. Thank you to my fellow members of Cohort 9. You are all clearly
the best Cohort ever and I am grateful for all your support through my illness and I hope we
never lose touch. Thank you to Dr. Linda P. Chen who told me the best dissertation is a complete
dissertation and always knew with certainty that I would complete it and who also provided me
with a committee member. Finally, thank you to the best chair ever, Dr. Yates, for all the
counseling, help, support and encouragement. I am truly going to miss our weekly meetings
where we talked about not only the progress on my dissertation but also work and life! There is
simply no better chair out there

Finally, I thank God because through him all things are possible.

iv

Contents
Dedication ....................................................................................................................................... ii
Acknowledgements ........................................................................................................................ iii
List of Tables ................................................................................................................................. iv
List of Figures ............................................................................................................................... vii
Abstract ........................................................................................................................................ viii

Chapter 1: Overview of the Study ..................................................................................................1
Background of the Problem ..........................................................................................................1
Importance of Addressing the Problem ....................................................................................2
Organizational Context and Mission ........................................................................................2
Organizational Performance Status ..........................................................................................5
Organizational Performance Goal ............................................................................................5
Description of Stakeholder Groups ..........................................................................................5
Stakeholders’ Performance Goals ............................................................................................6
Stakeholder Group for the Study ..............................................................................................7
Purpose of the Project and Questions .......................................................................................7
Conceptual and Methodological Framework ...........................................................................8
Definitions ................................................................................................................................8
Organization of the Project .......................................................................................................9

Chapter 2: Review of the Literature..............................................................................................10
Influences On The Problem of Practice..................................................................................10
Background of Higher-Level Mathematics in Public Education ...........................................11
Role of Stakeholder Group of Focus ......................................................................................14
Stakeholder Knowledge, Motivation and Organizational Influences .....................................15
Conceptual Framework: The Interaction of Stakeholders’ Knowledge and Motivation and
the Organizational Context .....................................................................................................27

Chapter 3: Methods .......................................................................................................................31
Purpose of the Project and Methodology ...............................................................................31
Conceptual and Methodological Framework .........................................................................32
Adaptation of the Gap Analysis Framework as an Evaluation Model ...................................33
Direct Data Sources ................................................................................................................33
Alignment of the KMO Influences and Methods ...................................................................36
Data Analysis..........................................................................................................................37

Chapter 4: Results And Findings ..................................................................................................40
Purpose of the Project and Questions .....................................................................................40
Organization of the Chapter ...................................................................................................41
RQ 1: To What Extent Is The Department Meeting Their Goal Of Offering Algebra I To
100% Of 8th And 9th Graders? Did Student Achievement Increase? ..................................42
RQ 2: To what extent were there opportunities for professional learning and certification in
mathematics for teachers in the Universal Algebra program? ..............................................48
v

RQ 3. To what extent was there broad city-wide support for the program as well as the
availability of curricular and coaching resources? ................................................................51
RQ4: What are the systems’ knowledge and motivation influences related to achieving this
organizational goal? ...............................................................................................................53
Discussion of Findings ...........................................................................................................53
Knowledge .........................................................................................................................53
Motivation ..........................................................................................................................56
Organization .......................................................................................................................57
Chapter 5: Recommendations and Evaluation ...............................................................................61
Recommendations to Address Knowledge, Motivation, and Organization Influences .....62
Integrated Implementation and Evaluation Plan ................................................................72
Organizational Purpose, Need and Expectations ...............................................................72
Level 4: Results and Leading Indicators ............................................................................73
Level 3: Behavior ...............................................................................................................74
Level 2: Learning ...............................................................................................................77
Level 1: Reaction ...............................................................................................................81
Data Analysis and Reporting .............................................................................................84
Summary of the Implementation and Evaluation Plan ......................................................86
Limitations and Delimitations ............................................................................................87
Future Research .................................................................................................................88
Conclusion .........................................................................................................................89
References ......................................................................................................................................95

vi

List of Tables
Table 1 Organizational Mission, Global Goal and Stakeholder Performance Goals ......................6
Table 2 Knowledge Influences, Types, and Assessments for Knowledge Gap Analysis ..............20
Table 3 Assumed Motivation Influences .......................................................................................24
Table 4 Assumed Organizational Influences .................................................................................26
Table 5 KMO Influences and methods ..........................................................................................36
Table 6 Number of Schools by School Level offering Algebra I Courses ....................................43
Table 7 Number of Algebra I Test Takers and Scoring Percentile citywide 2015-2019 ...............44
Table 8 Summary of Influences – Cause, Asset, or Need .............................................................58
Table 9 Summary of Knowledge Influences and Recommendations ............................................63
Table 10 Summary of Motivation Influences and Recommendations ...........................................67
Table 11 Summary of Organization Influences and Recommendations .......................................70
Table 12 Outcomes, Metrics, and Methods for External and Internal Outcomes ..........................74
Table 13 Critical Behaviors, Metrics, Methods, and Timing for Evaluation ................................75
Table 14 Required Drivers to Support Critical Behaviors .............................................................76
Table 15 Evaluation of the Components of Learning for the Program ..........................................80
Table 16 Components to Measure Reactions to the Program ........................................................81
Table 17 Immediate Evaluation Summary ....................................................................................83
Table 18 Delayed Evaluation Summary ........................................................................................84

vii

List of Figures
Figure 1 Student Demographics by Percentage ...............................................................................4
Figure 2 Conceptual Framework for Universal Algebra ...............................................................27
Figure 3 Conceptual and Methodological Framework ..................................................................32
Figure 4 Student Performance on NAEP Urban Math Assessment by Year .................................46
Figure 5 Teacher responses to professional development questions .............................................50
Figure 6 Sample Universal Algebra Data Dashboard ....................................................................85

viii

Abstract
This study utilizes the Clark and Estes (2008) gap analysis framework as an evaluation and
analytic tool to examine and identify performance gaps, program goals, and intended outcomes.
The purpose of this evaluation study was to analyze how the gaps in knowledge, motivation, and
organization influences impacted the efficacy of the Universal Algebra initiative. Secondary
data, both direct data sources and contextual data sources, were used to verify the knowledge and
skills, motivation, and organization influences and identify the causes of those gaps. The
validated assumed influences for declarative factual knowledge, conceptual knowledge,
procedural knowledge, metacognitive knowledge, self-efficacy, value, cultural models, and
cultural settings were identified, and confirmed influences were analyzed. The results and
findings illustrated that although the program had made significant progress, it failed to meet its
performance goal of 100% of student taking Algebra I by the end of ninth grade. The study
illustrates how that gap analysis framework can be used as an evaluation tool to identify and
remediate gaps in programs to increase efficacy of those programs.
Keywords: equity, algebra, achievement, under-resourced

Chapter 1: Overview of the Study
During the last quarter-century, mathematics pedagogues have advocated for the
incorporation of an innovative concept into mathematics education. Traditionally, mathematics
education has emphasized the calculation and computational aspects of teaching and learning.
However, societal shifts demand a distinctive definition of mathematical literacy, necessitating
unique methods of classroom teaching that inculcate students with reasoning and problem-
solving skills promoting higher-level mathematics aptitude. This approach positions mathematics
as a subject wherein students can give meaning to processes and actively participate in their own
learning rather than passively receive information (National Council of Teachers of
Mathematics, 1989). Classroom teachers are critical to implementing change in the classroom
(National Council of Teachers of Mathematics, 1991), and they need sustained support and
improved resources to actualize this vision. Although various resources have been designed to
aid teachers as they engage in these pedagogical changes (National Council of Teachers of
Mathematics, 2014), research suggests that mere access to these resources is inadequate and that
teachers require job-centered and supported professional learning to process these pedagogical
shifts and implement them effectively.
Background of the Problem
Education researchers have struggled to describe how students can engage in sense-
making while participating in classroom learning. Sense-making refers to the process by which
students give meaning to and understand what they are learning. Myriad definitions of sense-
making have been offered by mathematics research and education, with one of the earliest
accepted definitions concerning constructing relationships and fashioning links between
2

mathematical concepts (Brownell, 1935). A later definition differentiated instrumental and
relational understanding (Skemp, 2006), explaining that more traditional pedagogical methods
emphasize instrumental understanding, which focuses on rules and procedures rather than
higher-level thinking. In contrast, relational understanding describes the knowledge of why and
how rules and procedures work. Both forms of sense-making are critical to understanding larger
mathematical concepts. However, higher-level mathematics requires an even deeper
understanding of mathematical skills that incorporates conceptual understanding, factual
knowledge, and procedural fluency (NCTM, 2000). Among the many different definitions of
mathematical understanding, various commonalities arise. Students who possess higher-level
mathematical reasoning and understanding can grasp and conjoin mathematical concepts,
understand the “why” of mathematics, apply rules and procedures to unfamiliar problems and
use reason to justify their mathematical ideas. Consequently, they are more engaged in the
classroom and can see the value in what they are learning.
Importance of Addressing the Problem
A drastic realignment is required for teaching to support the type of learning that higher-
level mathematics requires. Although this realignment encompasses many factors, this study
focuses on three areas where the Universal Algebra initiative seeks to positively impact the
instructional core: (1) professional learning for teaching higher-level mathematics; (2) in- and
out-of-school resources for higher-level mathematics; (3) in-school coaching for teachers.
Conventional mathematics pedagogy and early mathematical literacy emphasize procedures and
facts rather than meaning and understanding. However, higher-level mathematics requires a
deeper understanding of why a concept works, a procedure is used, or a particular skill is
applied. Using techniques such as reasoning, mathematical communication, problem-solving,
3

and justification, classrooms that emphasize sense-making cultivate a culture that grasps
concepts more profoundly than is possible for a conventional classroom. That is, classrooms can
be empowered to promote mathematical understanding. However, this requires teachers who are
skilled in and trained to promote higher-level mathematics in the classroom. Although the
literature illustrates that students can make sense of mathematical concepts if given the
opportunity for communication and reflection (Hiebert et al., 1997), without teachers trained to
facilitate sense-making activities in the classroom, students are likely to experience a knowledge
deficit.
Organizational Context and Mission
The Office of Curriculum, Instruction, and Professional Development (OCIPD) is part of
the Metropolitan Department of Education (MDOE) in Metropolitan City. The OCIPD is tasked
with the citywide implementation and alignment of curricula, instructional methodologies, and
professional learning for teachers. According to the organization’s website, the MDOE is one of
the largest school systems in the United States, providing services to over one million students
and featuring over 10,000 teachers and administrators guiding the academic trajectory of those
students. The city is ethnically diverse, and the school system is representative of that, as Figure
1 demonstrates.
4

Figure 1
Student Demographics by Percentage

The OCIPD comprises eight departments, seven of which offer professional learning.
There are four subject area departments: Social Studies; Literacy; Arts; Science, Technology,
Engineering and Mathematics (STEM). Additionally, the OCIPD is responsible for instructional
and curricular cohesion, with the departments responsible for those areas including Periodic
Assessments and Core Curriculum, Instructional Strategy and Evaluation. Meanwhile, the
OCIPD’s operational department is responsible for various curricular and instructional
implementation objectives, including large-scale project management. The OCIPD has
approximately 700 staff members, which includes teachers, administrators, instructional
specialists, and various operational support staff.
According to the organization, the OCIPD provides over 1,500 teacher training events a
year, as well as several camps and intensive programs for students that aim to positively impact
the instructional core and improve student achievement. Professional learning offerings from the
5

OCIPD are part of the organization’s core values, demonstrating its mission to support the field
(i.e., superintendents, Field Support Center staff, principals, and teachers) by providing
opportunities that build capacity, facilitate collaboration, shift mindsets, and deepen cycles of
professional learning that are responsive to schools’ needs while centering student agency.
Organizational Performance Status
The organizational performance problem at the root of this study is the lack of teachers
qualified to teach algebra, which hinders the MDOE’s ability to offer algebra in all its high
schools. To fulfill its mission and provide access to Algebra I, it is imperative for the OCIPD to
promote instructional competence in teaching higher-level mathematics. The failure to achieve
this can produce students who are inadequately prepared for college and teachers who are
incapable of preparing them, as well as contributing to an ongoing achievement gap.
Organizational Performance Goal
The goal of the Universal Algebra initiative is for all students to complete an algebra
course by the end of 9th grade, which would empower them to take advanced math courses in
high school and prepare them for higher education and careers. By 2022, all Metropolitan City
students will have been given the opportunity to take an algebra course by the 9th grade and will
have access to academic support in elementary and middle school to promote algebra readiness.
The OCIPD’s achievement of this goal will be measured by the number of high schools offering
Algebra I or a vehicle for taking Algebra I.
Description of Stakeholder Groups
The Executive Director for STEM will frame and provide thought leadership as well as
advocate for Universal Algebra projects and tasks.
6

Meanwhile, the Senior Executive Director of the OCIPD will provide thought leadership
and act as an advocate to the Department of Education and city leadership.
Teachers will attend professional learning opportunities to improve their acumen for
teaching advanced math in schools that did not previously provide advanced math courses.
Administrators will provide time, space, and opportunity for teacher learning
communities as well as the leadership to secure curricula and provide algebra courses across the
city. Additionally, they will departmentalize math in schools that did not previously have a math
department.
Stakeholders’ Performance Goals
Table 1
Organizational Mission, Global Goal, and Stakeholder Performance Goals
Organizational Mission
The OCIPD’s mission is to support the field (i.e., superintendents, Field Support Center staff,
principals, and teachers) with opportunities that build capacity, facilitate collaboration, shift
mindsets, and deepen cycles of professional learning that are responsive to schools' needs
while holding student agency at the center.
Organizational Performance Goal
By 2022, all students will have access to an algebra course in 8th grade, and to academic
supports in elementary and middle school to ensure greater algebra readiness.
The Executive
Director of STEM

By June 2020,
over 400
teachers from 5th to
10th grade will return
to their classrooms
across the city with
expanded expertise in
math instruction and
strategies.
Senior Executive
Director for OCIPD

By June 2022, Math
scores will have
increased 2 points in
the state Math
assessment in
statistically under-
represented districts.
Math Teachers

By June 2020, 100%
of students will have
exposure and access
to advanced math.

Administrators

By June 2022, 67
elementary
schools will have
departmentalizing
math to ensure a
specialized math
teacher is helping
students toward the
goal of universal
algebra.

7

Stakeholder Group for the Study
All of Metropolitan City’s stakeholders have key roles in the success of the Universal
Algebra initiatives, and each will uniquely contribute to achieving the organizational goal of
empowering 100% of schools to offer advanced math courses and ensuring all students complete
an Algebra I course by 9th grade. Given it is critical to understand where the city is positioned
regarding this performance goal, this study’s stakeholders of focus are all teachers involved in
the Universal Algebra initiative. The stakeholders’ goal, supported by the Senior Executive
Director, is for 100% of teachers to implement advanced math in their classrooms. Advanced
math includes algebra and all coursework leading up to the completion of Algebra I, including
pre-algebra and Math II. The failure to accomplish this goal will produce a student population
that continues to be incapable of entering college due to a lack of preparation and pre-requisite
classwork. In turn, these ill-prepared students will continue to suffer diminished career options,
which adversely affects the health of the city and its students.
Purpose of the Project and Questions
This project’s purpose is to evaluate the implementation of the Universal Algebra program
which includes offering Algebra I courses to all students by 9th grade and the impact of this
shift. The analysis focuses on the knowledge, motivation, and organizational influences of
stakeholders in relation to achieving this organizational goal. Although a complete evaluation
process would address all Universal Algebra stakeholders, for practical purposes, this evaluation
focuses on mathematics teachers. Accordingly, the following questions guide this study:
1a. To what extent is the Department meeting their goal of offering Algebra I to
100% of 8th and 9th graders?
b. Did student achievement increase?
8

2. To what extent were there opportunities for professional learning and certification
in mathematics for teachers in the Universal Algebra program?
3. To what extent was there broad city-wide support for the program as well as the
availability of curricular and coaching resources?
4. What are the systems’ knowledge and motivation influences related to achieving
this organizational goal?
Conceptual and Methodological Framework
This study uses Clark and Estes’ (2008) gap analysis as the framework for a
methodological schema that can elucidate organizational goals while ascertaining the gap
between an organization’s performance and its defined goals and objectives. The assumed
knowledge, motivations, and organizational influences of stakeholders are analyzed using a
literature review as well as personal knowledge and the organization’s experience. The
organization's influences are evaluated using secondary direct data sources, such as surveys and
assessments, as well contextual data such as news articles. The evaluation’s conclusion makes
recommendations based on the literature, the data collected, and the subsequent analysis.

Definitions
Algebra: The study of mathematical symbols and the rules for manipulating these symbols,
which constitute a thread unifying almost all of mathematics.
Professional development or Professional Learning Professional development comprises
professional workshops and conferences designed to enhance teachers’ knowledge, skills,
attitudes, and beliefs in service of improved student achievement.
9

Organization of the Project
This study is organized into five chapters. This current chapter has introduced key
concepts and definitions associated with the MDOE. Importantly, the MDOE’s organizational
mission, status, goals, and stakeholders are analyzed using the gap analysis framework. Chapter
Two reviews the literature that is germane to the study’s scope. This study addresses teacher
preparedness, algebra readiness, and issues of access, including access to resources. Chapter
Three outlines the assumed influences on outcomes and the data selection and analysis
methodologies. Chapter Four assesses and analyzes the collected data and results. Chapter Five
offers recommendations for closing the perceived gaps––based on the data collected and the
literature review––and an evaluation plan for the proffered solutions.
10

Chapter 2: Review of the Literature
To better understand the current state of higher-level mathematics on a national level, this
chapter frames mathematical education historically and within the context of currently available
relevant literature. Mathematical literacy for post-secondary readiness is critical to bridging the
equity gap experienced by many traditionally under-resourced populations. The Universal
Algebra program has been created to address gaps in such communities. Students in
Metropolitan City need access to Algebra I courses and teachers that can proficiently teach the
subject in the interest of not only equity but also post-secondary readiness. This chapter
examines the background to the problem of practice, including current and historical influences,
as well as considering stakeholder knowledge, motivation, and organizational influences.
Following an investigation of the role of teachers in the problem of practice, Clark and Estes’
(2008) framework is utilized to examine knowledge, motivation, and organizational influences of
stakeholders. The chapter concludes by presenting the conceptual framework used to guide this
study.
Influences On The Problem of Practice
This section critically determines what current research proposes concerning themes
related to this project. It is arranged in four parts: 1) background of higher-level mathematics in
public education; 2) the current state of higher-level math programs in secondary schools; 3)
expanding programs for teaching higher-level mathematics; 4) evaluation of mathematics
programs; 5) challenges to higher-level mathematics pedagogy. The background subsection
addresses the policy, legal, and historical context of the problem. Then, the current state
subsection explores algebra offerings nationally and their impact on students’ post-secondary
success. The next subsection considers the relationship between expanded programs and post-
11

secondary success. Then, the value of evaluating mathematics programs is addressed to
determine the efficacy of mathematics intervention programs. The final subsection addresses
challenges to mathematics pedagogy.
Background of Higher-Level Mathematics in Public Education
There have been ongoing concerns about the efficacy and distribution of higher-level
mathematics, especially algebra. The literature indicates that certain elements, such as tradition,
can hinder meaningful change. However, since the release of a Nation at Risk, the public,
politicians, and educators have felt an impetus to improve education in order to compete on a
global scale. Ongoing policy decisions (e.g., No Child Left Behind, Common Core Standards),
societal demands (high school graduates with math literacy), and technological improvements
have nurtured and propelled the call for change. Access to higher-level math courses has
historically been limited; therefore, expanding that access would afford greater opportunities to
under-resourced students. Forty-eight states and several territories have some form of “college
and career-ready” high school graduation aspirations and common mathematics standards (Reys
et al., 2010), with data indicating that higher-level math preparation is generally considered to be
a barometer of college and career readiness.
The lack of access to algebra courses has been shown to negatively impact college
readiness (Karp & Schubring, 2014). Meanwhile, McCall (2015) investigated the impact of the
level of math courses a student completes in high school on their chances of attending and
completing college, concluding that completing classes in high school not only increases a
student’s chances of success in college but also of completing college. Thus, the literature
indicates that higher-level math courses can improve post-secondary performance.
12

Current State of Higher-Level Math Programs in Secondary Education
States and cities nationwide are considering programs to address deficiencies in math
literacy. Metro-Craig et al. (n.d.) crafted a qualitative descriptive case study to describe features
at-risk students brought to online Algebra I credit recovery courses aimed at improving their
proficiency. The study considered eight high school students who participated in an online
Algebra I course., with data collected through interviews with the students before and after the
course, with observations made to access emerging themes. The researchers posit that higher-
level math courses not being broadly available can have a life-long impact, particularly on under-
served populations. In addition to access, the quality and content of mathematics teaching have
also been correlated with ongoing participation in STEM programs.
Elsewhere, Bottia et al. (2015) emphasize the significance of superior STEM academic
preparation in high school and the function of informal and formal contact with STEM as crucial
influences on students’ chances of embarking upon a STEM career. The researchers suggest
several policy revisions, including providing a variety of STEM opportunities, and call for
increased access and exposure to STEM given increased access to STEM courses predicts
interest in the field and pursuit of careers in it.
Expanding Programs for Teaching Higher-Level Mathematics
Many districts are implementing programs to expand the reach of higher-level
mathematics education, some including private partnerships. For example, Miami-Dade has
adopted a math literacy program in all its high schools to improve student performance
(NewsRx). Elsewhere, in a program conceived by cognitive and computer scientists from
Carnegie Mellon University in concert with master mathematics teachers, Carnegie Learning is
trying to help reinvent mathematics teaching to empower students and improve math scores in
13

diverse populations. Meanwhile, Slavin et al. (n.d.) examined two cooperative learning programs
that had positively impacted learning, positing that programs impacting daily teaching practices
and student interactions demonstrate more promise than those emphasizing technology or
textbooks alone. Thus, the available literature suggests that Metropolitan City is not the only
urban district concerned about and taking action to address higher-level math literacy.
Evaluation of Mathematics Programs
Program evaluation is a valuable tool that can be used to gauge the efficacy and reach of
an initiative. Vanderheyden et al. (n.d.) conducted a randomized evaluation of an intervention,
illustrating the ability of evaluation to determine the integrity with which an intervention can be
implemented in classrooms and predict summative measures of mathematics achievement and
growth on curriculum-based measures. The researchers also discuss the implications
for mathematics instruction in general and mathematics intervention in response to intervention
models. Program evaluation, regardless of subject area focus, is generally considered a
barometer for determining a program’s efficacy.
Challenges to Higher-Level Mathematics Pedagogy
The need to expand course offerings in high schools to include algebra is complicated by
a lack of teachers with the skills and background to teach the subject. According to Daughtry et
al. (n.d.), increased computer literacy promotes higher levels of classroom computer use, with
teachers more likely to cultivate technology self-efficacy and technological content knowledge
after participating in training. That study’s findings indicated that 100% of exposure increased
daily usage and interest. This suggests that the quality of the mathematics instruction provided
by teachers could have implications for student success.
14

Role of Stakeholder Group of Focus
This study’s stakeholder group of focus for this study is the teachers who are
participating in the Universal Algebra program. Mathematics teachers are integral to promoting
understanding of higher-level mathematics because they can directly impact the instructional
core. Their connection to the classroom cultivates their ability to achieve knowledge transfer and
effectively teach the material essential to increasing higher-level mathematical understanding.
Clark and Estes’ (2008) Knowledge, Motivation and Organizational Influences Framework
Clark and Estes (2008) postulate a thorough and rich framework to structure and
contextualize stakeholder and organizational performance goals and determine the gap between
how an organization is performing and its actual goals. Following gap analysis, this model
researches the knowledge, motivation, and organizational influences of stakeholders that could
impede the achievement of performance goals (Clark and Estes, 2008). Krathwohl (2002) split
knowledge and skills into four distinct categories––factual, conceptual, procedural, and
metacognitive––to understand stakeholders’ ability to realize performance goals. Conversely,
influences on motivation focus on how an employee perseveres towards achieving an identified
goal, as well as the mental effort involved in achieving that goal (Clark & Estes, 2008). It is
important that motivational beliefs, such as self-efficacy, are a component of research aimed at
addressing a performance gap (Rueda, 2011).
Meanwhile, organizational influences on performance include organizational barriers like
a school’s culture and its access to resources (Clark & Estes, 2008). These influences, derived
from the gap analysis, are discussed in-depth and used to frame the knowledge and skills that
would enable Universal Algebra teachers to incorporate higher-level mathematics pedagogy into
their classrooms. Accordingly, the following section addresses how knowledge and skills impact
15

stakeholder performance, considers the assumed motivational impediments, and examines
organizational issues in the context of stakeholder goals. All identified stakeholder influences on
performance are further analyzed using the methodology outlined in Chapter Three.
Stakeholder Knowledge, Motivation, and Organizational Influences
Knowledge and Skills
The research regarding math teachers reveals that performance improves when staff
members are provided with models for increased learning and self-efficacy (Aguinis & Kraiger,
2009; Grossman & Salas, 2011; Martin, 2010). Investigating how knowledge and skills impact
how staff members perform and achieve organizational goals is critical to the study of
performance problems (Clark & Estes, 2008), with assessment of employee knowledge and skills
providing insight into their job performance guiding improvement in service of better outcomes
(Clark & Estes, 2008). That is, assessing the nuances and components of employees’ knowledge
and skills can increase understanding about how well an employee achieves their goals, solves
problems, and performs overall (Clark & Estes, 2008). The research illustrates that
organizational success is determined by an employee’s adeptness at not only acquiring skills but
also positively transferring those skills to their colleagues (Aguinis & Kraiger, 2009; Grossman
& Salas, 2011; Alexander, Schallert, & Reynolds, 2009). Thus, it is beneficial for the employee,
both personally and professionally, and enhances their level of engagement to relate their
knowledge and skills successfully to their undertakings and projects (Grossman & Salas, 2011).
The framework used to understand knowledge classifies knowledge into four categories:
factual knowledge, conceptual knowledge, procedural knowledge, and metacognitive knowledge
(Krathwohl, 2002; Rueda, 2011). Factual knowledge is considered to be basic information that
can be quickly recollected (Krathwohl, 2002), with one illustration of an individual’s grasp of
16

factual knowledge being knowing their organization’s vision and objective and the departments
that made up the greater division. The next category of knowledge dimension would be
conceptual knowledge. Conceptual knowledge describes understanding how separate elements
function within a larger structure (Krathwohl, 2002; Rueda, 2011). Another way of framing this
relationship is by locating conceptual knowledge at the nexus of interpretation of new knowledge
in the context of prior knowledge through its incorporation into one’s existing mental schema.
One example is the need for employees to know what they do well and what they need to
improve upon as it relates to their job. In practice, this could involve an individual revising
purchasing policy information and incorporating it into their present knowledge of purchasing
policy to cultivate a new understanding of that policy.
Procedural knowledge refers to understanding how something is done, such as a
mathematical order of operations (Krathwohl, 2002; Rueda, 2011), with an employee knowing
when to apply a certain process in the context of their professional duties being an example.
Finally, metacognitive knowledge is the ability to understand not only one’s own
cognition but also the cognition of others (Krathwohl, 2002; Rueda, 2011). Employees reflect on
what and how they learn by utilizing metacognition, the process by which individuals reflect on
how they learn and the process in which they learn (Mayer, 2011). This knowledge dimension
transcends procedural or factual knowledge to guide individual reflection on not only their
knowledge but also how they possess that knowledge, where their knowledge gaps are, and how
they process information (Mayer, 2011; Krathwohl 2002; Rueda, 2011).
Each of the four knowledge dimensions influences how employees realize the
organizational goal of improving student achievement in mathematics. This study addresses the
factual, conceptual, procedural, and metacognitive knowledge dimensions in relation to this
17

organizational goal. knowledge influences include a) Teachers need to know specific examples
of Higher level mathematics teaching strategies, b) Teachers will have knowledge of higher
level math pedagogy., c) Teachers will be able to implement improved math instruction
strategies in the classrooms, d) Teachers need to be able to self-reflect and consider how
effectively they are implementing the higher level mathematics strategies in their classrooms.
This metacognitive knowledge influences, outlined––detailed in the following subsections––are
used to assess the knowledge and skills that teachers involved in the Universal Algebra program
need to achieve the organizational goal of algebra readiness.
Teachers Know Specific Examples of Higher-Level Mathematics Teaching
Strategies. Procedural knowledge is crucial to the success of teachers in the classroom. Without
knowing how to complete mathematical processes, teachers lack the knowledge necessary to
instruct their students. Quality formative feedback can meaningfully improve learning processes
and outcomes (Shute, 2008). Meanwhile, integrating procedural knowledge with improved
instruction in the classroom requires an understanding of the areas in which students need to
improve. Teachers can attain this information by reviewing test items for items with a high
failure rate (Wasserman, 2016). This will provide insight into areas that need to be targeted.
Rather than simply administering a test and applying grades, teachers should revisit high failure
items to ensure their class’s understanding.
Teachers Will Have Knowledge of Higher-Level Math Pedagogy. Although
conceptual knowledge of mathematics provides a strong foundation in the classroom, to
successfully teach higher-level mathematics, teachers need to understand pedagogies that can
promote knowledge transfer. Ross and Willson (2012) conducted a study that found that teachers
with strong conceptual knowledge of mathematics pedagogy could better engage their students in
18

the classroom. By receiving instruction and modeling on pedagogy to increase their conceptual
understanding, Universal Algebra teachers can improve their instruction and increase knowledge
transfer.
Teachers Will Be Able to Implement Improved Math Instruction Strategies In The
Classroom. Procedural knowledge is often critical to school reform programs like Universal
Algebra. Thus, resources, including materials and professional development opportunities, are
being provided to Universal Algebra teachers. In addition to focused professional learning for six
weeks during the summer, these teachers also receive school-based coaching, including
modeling of instructional practices and sample lesson plans. The Universal Algebra program also
provides ongoing support via monthly professional development and features an email inbox
capable of providing support to teachers on an ad hoc basis. By providing myriad forms of
support for teachers’ instruction in the classroom, the program can improve student achievement
and enhance teachers’ professional acumen.
Showalter (2017) conducted a study on the impact of higher-level math courses
on higher-level math pathways and the probability of students taking such courses avoiding
remedial math courses in post-secondary education. That study parallels the objective of the
Universal Algebra program, defining higher-level mathematics as trigonometry, algebra, and
calculus, with the researchers concluding that students need a strong foundation in higher-level
mathematics to be successful. Universal Algebra teachers, who undergo professional
development during the summer, need to be able to apply this knowledge in the classroom.
Notably, two other studies have indicated that teachers’ fundamental understanding and
knowledge about effectively completing mathematical processes and composing coherent and
aligned lesson plans greatly improves student achievement (Wasserman, 2016; Ross & Willson,
19

2012). Given Universal Algebra teachers are frequently in under-resourced schools––where
students receive poor preparation and have limited access to resources––this procedural
knowledge is even more integral to the Universal Algebra model, which provides essential
pedagogical resources to teachers. Thus, while teachers do need to know basic math information,
they also need to be able to apply and model this information in the classroom.
Teachers Need To Be Able To Self-Reflect And Consider How Effectively They Are
Implementing The Higher-level Mathematics Strategies In Their Classrooms. It is essential
for teachers to hone their metacognitive knowledge to expand their grasp of the connection
between learning and development and personal growth and improved organizational
performance (Aguinis & Kraiger, 2009). Teachers benefit from establishing robust relationships
with their work and their own pedagogy (Wasserman, 2016). This is particularly important for
teachers being able to probe their own metacognition and better understand their own learning.
When teachers use metacognition skills to process information before, during, and after
professional learning, they improve retention of skills and knowledge into long-term memory,
which, in turn, improves knowledge transfer (Grossman & Salas, 2011; Deans of Learning,
2018).
Furthermore, the research suggests that the intersection between a teacher’s knowledge
about themselves and their learning processes and their training and development cultivates a
framework for improving classroom instruction (Wasserman, 2016; Ross & Willson, 2012). That
is, a teacher’s attainment of personal growth and development can be realized via professional
learning opportunities (Aguinis & Kraiger, 2009; Clark & Estes, 2008), which expands their
metacognitive abilities and their ability to understand the learning and developmental needs of
students in the classroom.
20

Table 2 illustrates the organizational mission and organizational goal and provides
information concerning knowledge influences, knowledge types, and assessments of knowledge
influence in terms of factual, conceptual, procedural, and metacognitive knowledge.
Table 2
Knowledge Influences, Types, and Assessments for Knowledge Gap Analysis.
Organizational Mission
The OCIPD’s mission is to support the field (i.e., superintendents, Field Support Center staff,
principals, and teachers) with opportunities that build capacity, facilitate collaboration, shift
mindsets, and deepen cycles of professional learning that are responsive to schools' needs while
holding student agency at the center.
Organizational Global Goal
By 2022, all students will have access to an Algebra I by the end of 9
th
grade, and to academic
supports in elementary and middle school to ensure greater algebra readiness.
Stakeholder Goal
June 2020, over 400 teachers from 5th to 10th grade will return to their classrooms across the
city with expanded expertise in math instruction and strategies.
Knowledge Influence Knowledge Type Knowledge Influence
Assessment
Teachers need to know specific
examples of Higher-level mathematics
teaching strategies
Declarative Annual School Survey.
Teachers will have knowledge of
higher-level math pedagogy.
Conceptual Annual School Survey
Teachers will be able to implement
improved math instruction strategies in
the classrooms.
Procedural Annual School Survey,
Summative Assessments
Teachers will be able to implement
improved math instruction strategies in
the classrooms
Metacognitive Annual School Survey,
Newspaper Articles

Motivation
Although there a multitude of motivational theories exists, this study uses self-efficacy
theory and value theory to study teachers’ confidence in their own abilities and to what teachers
attribute poor performance. According to Mayer (2011), motivation determines the engagement
of an individual from the beginning of a task to its successful completion, with research positing
21

that individuals perform at higher levels by consciously choosing to become involved, applying
mental effort and persistence to reach organizational and personal goals (Clark & Estes, 2008;
Grossman & Salas, 2011). It is essential to understand motivational influences and how they
impact performance problems when addressing an organization’s needs regarding improving
performance (Rueda, 2011). As discussed, Clark and Estes (2008) present motivation as the
second element in a three-tier model that can assess and resolve performance problems. Within
this context, the motivational factors that influence achievement, issues, assets, and needs should
be evaluated by applying the concepts of choice, persistence, and mental effort (Clark & Estes,
2008; Jensen, 2012; Mayer, 2011; Rueda, 2011).
Self-Efficacy Theory. Self-efficacy is an essential component of social cognitive theory
and emphasizes how individuals create their own circumstances (Bandura, 2000). Self-efficacy
theory addresses an individual’s belief that they can accomplish a task or realize a goal (Bandura,
2000). Self-efficacy beliefs provide the foundation for the abilities that individuals believe that
they have and the outcomes that they can achieve (Pajares, 2006). Given individuals cultivate
their own self-efficacy, they are motivated by their assessment of their experiences and are
motivated by social interactions, emotional and psychological reactions, and the observations of
others performing the same task (Pajares, 2006). Such interactions affect the development of an
individual’s self-efficacy, either positively or negatively impacting their motivation to perform
(Bandura, 2000). The research maintains that positive self-efficacy motivates individuals to
believe in their own abilities so that they can successfully achieve organizational goals (Clark &
Estes, 2008; Rueda, 2011).
Self-Efficacy and Teaching Higher-Level Mathematics. Individual and collective
efficacy contributes to whether or not a person or group is motivated to persist with an
22

organizational goal (Pajares, 2006), with their level of involvement and engagement influenced
by the degree to which they believe that they are able to contribute to a task (Hayward, 2010).
The research indicates that student performance in mathematics is directly linked to their
teachers’ beliefs about their own self-efficacy (Chatzistamatioum et al., 2014) and that teachers’
perceived self-efficacy increases through participation in growth opportunities and professional
learning and through positive modeling from their colleagues (Tang et al., 2014). Individuals are
more confident about engaging and completing their work when they believe they have the skills
and knowledge to complete their tasks and achieve organizational goals (Hayward, 2010;
Chatzistamatioum et al., 2014).
Expectancy-Value Theory. Performance improves when people have high expectations
and strong beliefs in their own ability and the value of the project with which they have been
tasked (Eccles, 2006). The two critical components of expectancy-value theory are the belief that
one has the ability to complete the task (i.e., expectancy) and whether they want to complete the
task (i.e., value). That is, expectancy can be gauged by the level of confidence one displays
concerning their skills and their capacity to achieve a goal (Clark & Estes, 2008), and value
refers to the degree of significance an individual assigns to a task (Rueda, 2011).
Attainments, intrinsic, utility, and cost constitute the four dimensions of value (Eccles,
2006; Rueda, 2011). Attainment value concerns the degree to which a task matches one’s own
values and preferences. Intrinsic value denotes the enjoyment one derives from the task. Utility
value concerns the extent to which a goal or task fulfills an individual’s needs, and cost value is
associated with the time and effort taken from other activities (Eccles, 2006; Rueda, 2011).
Ultimately, based on these four dimensions, if an individual believes that their effort will
23

contribute to the realization of a critical task, they are likely to be more motivated to devote time
and effort to realize a goal.
Values and Teacher Confidence. An individual’s degree of engagement depends on the
value placed on the task being performed (Eccles, 2006). Universal Algebra teachers need to feel
motivated and confident that their contributions in the classroom can positively impact student
achievement. Teachers who are engaged and believe in their abilities and that their work has a
meaningful impact on their students are better able to motivate their students (Ross & Willson,
2012). The research illustrates that teachers who are cognizant of the value of their pedagogy
have a greater impact in the classroom (Wasserman, 2016).
Table 3 identifies two motivational influences that emphasize self-efficacy and
expectancy-value theory. These influences are used to understand how self-efficacy theory and
expectancy-value theory can explain Universal Algebra outcomes.

24

Table 3
Assumed Motivation Influences
Organizational Mission
The OCIPD’s mission is to support the field (i.e., superintendents, Field Support Center staff,
principals, and teachers) with opportunities that build capacity, facilitate collaboration, shift
mindsets, and deepen cycles of professional learning that are responsive to schools' needs while
holding student agency at the center.
Organizational Global Goal
By 2022, all students will have access to an Algebra I course by the end of 9
th
grade and to
academic supports in elementary and middle school to ensure greater algebra readiness.
Stakeholder Goal
By June 2020, over 400 teachers from 5th to 10th grade will return to their classrooms across the
city with expanded expertise in math instruction and strategies. .
Assumed Motivation Influences Motivational Influence Assessment
Self-Efficacy: Teachers need to believe that they
are well-versed and can effectively teach higher-
level mathematics.

Annual School Survey Data
Expectancy-Value: Teachers feel that the
instruction they provide is important to their
students’ achievement.
Annual School Survey Data

Organization
In concert with knowledge and motivation influences, organizational influences factor in
performance gaps in teachers’ adeptness at implementing higher-level mathematics pedagogy in
their classrooms. According to Clark and Estes (2008), although a stakeholder group may
possess the requisite knowledge and motivation, performance gaps can persist due to problems
with an organization’s cultural models and settings. Cultural models are the structures that exist
within an organization, and cultural settings are cultivated within the organization (Schein,
2004). An organization cannot achieve its performance goals in the context of cultural models
featuring teachers unwilling to alter teaching techniques and tensions arising from the
introduction of new pedagogies. Meanwhile, cultural settings, such as the availability of
25

resources and the targeting of support at under-resourced communities, can also adversely
impact the achievement of performance goals. Thus, enduring organizational obstacles contribute
significantly to a teacher’s ability to successfully deliver higher-level mathematics teaching.
The Organization Needs To Have A Culture That Supports Achievement
Irrespective of Geography or Racial or Ethnic Background. Both Clark and Estes (2008) and
Schein (2010) have examined the component of organizational culture founded on the lived
experience of that organization’s members and which, therefore, reflects their attitudes, beliefs,
and values. The conviction that the African American communities bear responsibility for the
achievement gap has been documented in the literature as a facet of the cultural models in
schools (Ford, 2014; Ford & Grantham, 2003; Frye & Vogt, 2010; Hertzog, 2005). Implicit
biases in classrooms and in the school system challenge the school system’s ability to enact
equitable change. However, the Universal Algebra program’s goals require the provision of
scaffolded support to minorities to ensure equitable access to Algebra I courses citywide.
The Organization Needs To Have A Culture That Provides Targeted Support to
Historically Under-Resourced Communities. Clark and Estes (2008) examine the function of
value streams as the way diverse facets of an organization relate to each other. Therefore, the
schools that receive. Thus, receiving additional resources tacitly communicates the importance of
a goal. The mayor’s commitment to equity could be observed in his announcement and funding
of the Universal Algebra program (as well as in several other equity-driven initiatives). Central
administrators must also be willing to address the challenges that are unique to the local-level
school culture, especially given there is frequently a disconnect between timeframes, training,
and the resources provided for a change initiative and the ability of the school to actually
26

implement the change. This is particularly true of schools that have historically been under-
resourced and feature marginalized populations.
Metropolitan City Is Ethnically And Racially Diverse. Tensions Over Equity In
Education, Including The Allocation Of Resources, Is High. Equitable Distribution Of
Resources Is Necessary To Achieve Program Goals. As addressed by Clark and Estes (2008),
a program’s success requires that the internal capacity of an organization is aligned with the
program’s capacity. When a school system’s internal funding mechanism inhibits teachers’
abilities to actualize higher-level math instruction in the classroom because there is insufficient
funding to support implementation, the organization’s efforts be futile. That is, inadequate
resources can mean that teachers can know how to effectively implement higher-level
mathematics and still fail to do so (Clark & Estes, 2008).
Table 4 presents the organizational influences on teachers meeting their goals.
Table 4
Assumed Organizational Influences
Organizational Mission
The OCIPD’s mission is to support the field (i.e., superintendents, Field Support Center
staff, principals, and teachers) with opportunities that build capacity, facilitate collaboration,
shift mindsets, and deepen cycles of professional learning that are responsive to schools'
needs while holding student agency at the center.
Organizational Global Goal
By 2022, all students will have access to an Algebra I by the end of 9
th
grade and to
academic supports in elementary and middle school to ensure greater algebra readiness.
Stakeholder Goal (If Applicable)
By June 2020, over 400 teachers from 5th to 10th grade will return to their classrooms across
the city with expanded expertise in math instruction and strategies.
Assumed Organizational Influences

Organization Influence Assessment
Cultural Model Influence 1: The
organization needs to have a culture that
supports achievement irrespective of
geography, or racial or ethnic background
Annual School Survey, Newspaper Articles,
Data Collected by Non-Profit Organizations
27

Cultural Model Influence 2: The
organization needs to have a culture that
provides targeted support to historically
under-resourced communities.
Newspaper Articles, Resource Distribution
Data
Cultural Setting Influence 1: Metropolitan
City is ethnically and racially diverse.
Tensions over equity in education, including
the allocation of resources, is high. Equitable
distribution of resources is necessary to
achieve program goals.
Funding Allocation Data

Conceptual Framework: The Interaction of Stakeholders’ Knowledge and Motivation and
the Organizational Context
A conceptual framework organizes schema of ideas, beliefs, and assumptions
surrounding a research study (Maxwell, 2013), as well as identifying the variables in a study and
the relationships between the variables. Additionally, a conceptual framework can highlight
potential biases in the design of both previous studies and the study in question. It also
illuminates gaps in existing research and where the research in question might fit (Merriam &
Tisdell, 2016). Although knowledge, motivation, and organizational influences have been
presented here separately, they interact to provide a context for the research problem, as
demonstrated by Figure 2.
28

Figure 2
Conceptual Framework for Universal Algebra
The three circles in Figure 2 each represent a different aspect of the conceptual
framework for Universal Algebra’s implementation. The green circle represents the ability and
motivation of Universal Algebra teachers to improve their classroom instruction. The blue circle
represents the cultural model and settings, which combine with knowledge and motivation
factors to impact the achievement of a program’s overall goals. The blue circle is slightly larger
to represent the scale of the organization, an organization tasked with educating 1.1 million
school children. The program’s goals are presented in the purple oval. The green and blue circles
converge to ultimately impact the goal. Although there are other possible manners of illustrating
this framework, Figure 2 represents the structure best because, at the local level (i.e., the school
level), the issue is focused on teachers and their impacts on the instructional core. However,

Teachers
Conceptual
(pedagogy),
Procedural
(implementation,
Expectancy-value,
Self-efficacy

Metropolitan City
School System
Equity & Access

Over 400 teachers
from 5th to 10th
grades will return to
their classrooms with
expanded expertise in
math instructions and
strategies
29

when training, outreach, communication, and selection processes occur, the program is
contextualized against citywide issues such as highly segregated classrooms and the need to
provide resources for traditionally under-resourced populations.
Notably, the knowledge and motivation circle groups elements of conceptual knowledge
and procedural knowledge. Here, conceptual knowledge, refers to teachers being able to identify
and apply advanced mathematical concepts. To achieve the program’s goals, teachers need to
acquire the higher-level math skills that can enable them to facilitate the expansion of Algebra I
courses citywide. To effectively teach higher-level mathematics, teachers must grasp the
pedagogy such that knowledge transfer occurs. According to Ross and Willson (2012), teachers
with a robust understanding of conceptual knowledge about mathematics pedagogy can more
successfully engage students in their classrooms. In addition to providing instruction essential to
mathematics, teachers must also possess a procedural understanding of the work, with procedural
knowledge critical for school reform programs such as Universal Algebra. Teachers need to be
able to craft lesson plans that exhibit the methodology necessary to communicate advanced math
skills to their students, with several studies having illustrated that a teacher’s fundamental
understanding of effective execution of mathematical processes and their composition of
coherent aligned lesson plans greatly improve their students’ achievement levels (Wasserman,
2016; Ross & Willson, 2012).
Having recontextualized the knowledge required of teachers, it is necessary to reframe
the motivational factors that impact the program. Research has connected student achievement in
mathematics to teacher attitudes regarding their own self-efficacy (Chatzistamatioum et al.,
2014), with teacher engagement observed to be dependent on the value they assign the task they
are undertaking (Eccles, 2006). Universal Algebra teachers must feel driven and confident that
30

their contributions can positively impact student performance. Additionally, teachers should feel
confident in their ability to teach and develop resources that can improve student outcomes.
The scale of the organization directly impacts the convergence of knowledge and
motivation factors. The MDOE employs over 10,000 teachers and educates 1.1 million students
across hundreds of individual schools and over 30 districts. Each school and district has its own
culture and its own agenda, some aspects of which conflict with Universal Algebra. The culture
of equity and access that has birthed the Universal Algebra program has also produced
impediments to or facilitators of improvements at the school, district, and city levels.
Conclusion
This research project aims to ascertain how successfully Universal Algebra teachers have
implemented higher-level math pedagogy in their classrooms. Accordingly, Chapter Two has
introduced literature discussing higher-level math pedagogy in connection to the Universal
Algebra program, including assessing why it is critical to teaching mathematics and how it can
impact student achievement, both at school and in post-secondary education. Notably, the
literature affirms the importance of higher-level math instruction. This chapter has also proposed
a gap analysis to frame Universal Algebra’s implementation and program design in conversation
with the knowledge, motivation, and organizational influences of stakeholders. While the
literature reviewed discusses myriad facets of these components, no research has yet directly
applied the gap analysis framework to teacher implementation of the Universal Algebra program.
Therefore, Chapter Three explains the methodology used to apply the gap analysis framework in
the context of the Universal Algebra program’s implementation, enabling a better understanding
of factors potentially impeding its success.
31

Chapter 3: Methods
Purpose of the Project and Methodology
This project aims to evaluate the assumed knowledge, motivation, and organizational
factors that impacted the MDOE’s implementation of the Universal Algebra program and,
specifically, its ability to offer Algebra I courses system wide. The evaluation concentrates on
stakeholder resources in terms of knowledge and skill, motivation, and organizational influences
and opportunities. Although a complete evaluation would analyze all stakeholders, this analysis
focuses on the stakeholder group that is teachers participating in the Universal Algebra program.
This chapter also discusses the framework provided by Clark and Estes (2008) and how it
is implemented in this analysis.
The questions that guide this study are the following:
1a. To what extent is the Department meeting their goal of offering Algebra I to
100% of 8th and 9th graders?
b. Did student achievement increase?
2. To what extent were there opportunities for professional learning and
certification in mathematics for teachers in the Universal Algebra program?
3. To what extent was there broad city-wide support for the program as well as
the availability of curricular and coaching resources?
4. What are the systems’ knowledge and motivation influences related to
achieving this organizational goal?

32

Conceptual and Methodological Framework
This evaluation adapts the Clark and Estes (2008) human performance model as an
evaluation model to analyze secondary data that is publicly available for the Universal Algebra
program and outcome data concerning students, teachers, schools, and course offerings across
the city, enabling determination of whether the Universal Algebra program has attained its stated
goals. The steps adopted from the Clark and Estes model are represented in Figure 3.
Figure 3
Conceptual and Methodological Framework

33

1. Step 1: Identify organizational goals.
2. Step 2: Identify performance goals related to the organizational goals.
3. Step 3: Determine performance gaps.
4. Step 4: Analyze gaps to identify root causes.
The final three steps in the Clark and Estes (2008) gap analysis framework are addressed in
Chapters Four and Five of this paper:
5. Step 5: Identify solutions to close the gap.
6. Step 6: Plan for implementing improvements.
7. Step 7: Proposal to evaluate and monitor improvements
Adaptation of the Gap Analysis Framework as an Evaluation Model
The gap analysis framework (Clark & Estes, 2008; Rueda, 2011) has been modified to
examine the primary knowledge, motivation, and organizational resources and obstacles impacting
the effective implementation of the Universal Algebra program in terms of teachers learning
higher-level math pedagogy and expanding course offerings. Resources and obstacles, along with
their root causes, are evaluated using the gap analysis framework, leading to recommendations for
improvements that are derived from the identification of potential solutions.
Direct Data Sources
Data available to the public from the MDOE are used to analyze the 2018–2019 school
year, which constitutes the Universal Algebra program’s third year. Test data from 2016 were used
as a baseline to assess progress towards the program’s publicly stated goals. These direct data are
publicly available data concerning Regents Exam offerings for 2018–2019. The Regents Exam
included an Algebra I component for schools offering Algebra I classes. The Regents Exam is
administered annually to high school students. Additionally, citywide math scores and MDOE
34

school survey data are utilized for the 2016–2017, 2017–2018, and 2018–2019 school years. These
constitute citywide representative data for public schools from all 32 districts, as well as data
regarding the teachers and principals that work in those schools and, in the case of public schools,
data for school districts. For any given public school, it was possible to access test data and school
district data. Finally, National Assessment of Educational Progress (NAEP) data, including
assessments and questionnaire data, are utilized to draw empirical conclusions.
This analysis employs a causal-comparative research design (Gall et al., 2007) that has “the
purpose of explaining educational experiences through the investigation of cause and effect
connections. In this design, the assumed cause is referred to as the independent variable and the
effect is the dependent variable (Gall, et al., 2007). This analysis is ex post facto because the
elements being analyzed have not been interfered with by the researcher, and the data were
collected in the past as part of another research endeavor.
Contextual Data Sources
According to Yanchar (2011), contextual sources “offer explicitly interpretive, contextual
accounts of practical human action within dynamic systems.” He defines a contextual source in
qualitative research as a supplementary source that aids definition of the context based on the
research objectives. In this case, a contextual source can provide additional meaning to the
research by framing the environment in which the program is operating. Contextual sources for
this study include newspaper articles, journal articles, and news reports, as well as other forms of
publicly available material published between September 2016 and August 2018. These
contextual sources are important because they can help to define the community, society, or
organization being studied.
35

According to the publicly stated goals of Universal Algebra program, one district-wide
goal is to accelerate student achievement in Algebra I, especially in historically under-resourced
schools. By being more responsive to this subgroup of students, the MDOE can increase student
achievement and decrease academic inequity citywide. Realizing higher-level mathematics
instruction should provide students more opportunities and better prepare them to enter higher
education. Therefore, the citywide target is for 100% of teachers to consistently implement
higher-level mathematics instruction in their classroom teaching. While a systematic analysis of
all stakeholders would better recognize all causes of the issue, it is critical to distinguish
stakeholder groups to illuminate the challenges facing it and help the organization achieve its
performance goals (Maxwell, 2013). Thus, this study’s stakeholder population of interest is
Universal Algebra teachers.
Explanation for Choices
Although there are various approaches to acquiring data for a study, the setting may
prohibit certain approaches being viable. For this study, although an interview-based data
collection process was chosen, survey and observation approaches could also be considered
viable methods for establishing the knowledge, motivation, and organizational influences of
teachers in the Universal Algebra program. The researcher was unable to complete survey or
interview-based research due to the advent of COVID and the prohibition on outside research in
schools. Therefore, available secondary data was utilized to examine program efficacy.
Alignment of the KMO Influences and Methods
Table 5 presents the methods for assessing knowledge, motivation, and organizational
influences of teachers implementing the Universal Algebra program in terms of primary and
contextual data sources.
36

Table 5
KMO Influences and methods
Knowledge Influence Direct Data Sources Contextual Data Sources

Teachers need to know
specific examples of
Higher level mathematics
teaching strategies.

Declarative
Annual School Survey,
Public professional
learning series events and
opportunities
Reception to opportunities as
reported in the media
Procedural

Teachers will be able to
implement improved math
instruction strategies in
the classrooms.
Year to year comparison
of schools offering
Algebra I courses.

NAEP results

Teachers will have
knowledge of higher-level
math pedagogy.

Conceptual
Annual School Survey
Teachers need to be able
to self-reflect and consider
how effectively they are
implementing the higher-
level mathematics
strategies in their
classrooms.

Metacognitive
Annual School Survey

Assumed Motivation
Influences
Direct Data Sources Contextual Data Sources

Self - Efficacy - Teachers
believe their ability to
teach higher level
mathematics will improve
their students’ learning.

Annual School Survey News Articles
37

Value: Teachers feel that
the instruction they provide
is important to their
students’ achievement.
Annual School Survey News Articles
Assumed Organizational
Influences

Direct Data Sources Contextual Data Sources
Cultural Model Influence
1: The organization needs
to have a culture that
supports achievement
irrespective of geography,
or racial or ethnic
background

Annual School Survey News Articles, Press Releases
Cultural Model Influence
2: The organization needs
to have a culture that
provides targeted support
to historically under-
resourced communities.
Data Concerning
Resource Distribution
Non-Profit Reports

Cultural Setting Influence
1: Metropolitan City is
ethnically and racially
diverse. Tensions over
equity in education,
including the allocation of
resources, is high.
Equitable distribution of
resources is necessary to
achieve program goals.

Data Concerning
Funding Allocation,
Annual School Survey

News Articles

Data Analysis
The data analysis involved methodically analyzing quantitative data, including test
scores, and qualitative data, such as news articles, to develop findings and make
recommendations (Bogdan & Biklen, 2007). Descriptive statistics were used for all data
analyzed.
38

Analytic memos were completed within 24 hours of review of each secondary data
source. The researcher recorded potential biases, concerns, and initial conclusions about the data
in conversation with the conceptual framework and research questions. During the first analysis
stage, the researcher applied a priori codes that aligned with the conceptual framework and then
used open coding to identify themes outside the framework. The next analysis phase focused on
developing axial codes from the open codes. Then, during the final data analysis phase, the
researcher focused on patterns and themes that had surfaced from the data.
Credibility and Trustworthiness
When engaging in qualitative research, it is critical to establish credibility and
trustworthiness to safeguard the study’s success. This can usually be assured by maintaining
anonymity and confidentiality. Given the research involves reviewing existing public data, news
articles, and other artifacts, there is no confidentiality risk. Additionally, data triangulation is
utilized to maintain credibility and produce insights that respond to the research questions
(Merriam & Tisdell, 2016). When data sets are complementary, the data are considered credible,
and it is possible to address the research questions using the outlined data collection methods.
Data triangulation also promotes reflexivity, which guarantees trustworthiness and
credibility by helping to reduce personal bias in the data collection process, one of the most
significant dangers to the validity of qualitative research (Maxwell, 2013). Bias could also be
reduced by ensuring that the research focuses on collecting only data that specifically aligns with
the research questions and chosen framework.
Ethics
This study involves analysis of data, articles, and artifacts produced in relation to the
Universal Algebra program between 2015 and 2019 to develop a better understanding of the
39

program’s impact on teacher pedagogy, course availability, and student achievement. It is
important to consider the ethics of data analysis, especially when utilizing secondary data
sources such as news articles, especially given the noteworthy threats to the validity of
qualitative research. Qualitative data is vulnerable to bias, leading the subsequent analysis to
negatively impact the recommendations produced (Maxwell, 2013). A researcher’s biases can
slant different parts of the research process. Consequently, this study has distinguished the ways
in which the researchers’ values or backgrounds could influence data collection or analysis.
According to Patton (2015), a researcher’s credibility is essential to the credibility of the data
collected (2015).
Limitations and Delimitations
Limitations
Limitations are the factors that are beyond the scope of the researcher to control. Despite
taking various precautions, several limitations are likely to remain. First, the research uses
existing data, the collection of which the researcher was not involved in. To more effectively
study the impact of the Universal Algebra program on high school teachers and their
implementation of higher-level math pedagogy, teachers would need to be interviewed and
surveyed over a longer period.
Delimitations
In contrast to limitations, delimitations are aspects of a study that the researcher can control.
First, the researchers have endeavored to extract unbiased viewpoints from the secondary data
analyzed. However, implicit researcher bias could persist. Furthermore, although the study is
restricted by the researcher’s access being limited to publicly available data, the study’s results
should be generalizable for the population.
40

Chapter 4: Results and Findings
The study’s primary goal was to examine the increased access to resources for math
teaching and expanded availability of Algebra I courses as part of a citywide comprehensive
Universal Algebra program. The study also sought to determine whether the Universal Algebra
program had impacted student achievement and examine organizational influences on the
program, including public perception, institutional support, and teacher perceptions of the
program.
As discussed, data pertaining to the Universal Algebra program are examined according
to the following research questions:
Purpose of the Project and Questions
1a. To what extent is the Department meeting their goal of offering Algebra I to 100% of
8th and 9th graders?
b. Did student achievement increase?
2. To what extent were there opportunities for professional learning and certification in
mathematics for teachers in the Universal Algebra program?
3. To what extent was there broad city-wide support for the program as well as the
availability of curricular and coaching resources?
4. What are the systems’ knowledge and motivation influences related to achieving this
organizational goal?

41

Organization of the Chapter
This chapter’s discussion is organized according to the research questions. The data
guiding these responses––the availability of Algebra I courses and their impact on student
achievement, professional learning including teacher certification, curricular resources, and
citywide support for the Universal Algebra program––include assessment data and news articles,
as well as publicly available information found on district and institutional partner websites. The
Universal Algebra program’s success is predicated on improving teacher competency through in-
classroom interventions such as coaching and curricular resources, math certification programs,
and professional learning. These initiatives are ultimately intended to improve student
achievement and equity across the system by providing educators with the skills necessary to
deliver Algebra I courses citywide.
The data are reported for each research question in terms of both direct data sources and
contextual data sources. Direct data are data that provide primary information; for the purposes
of this study, they refer to raw assessment data. These data include school-level data by year,
such as the number of students who took the Regents exam, their grade level, and their test
scores. Contextual data provides background data to enable the data to be broadly framed in the
context of events, climate, and organizational influences. For the purposes of this study,
contextual data include media coverage, reports prepared by education agencies, and web pages
that provide information on the availability of different resources.

42

RQ 1: To What Extent Is The Department Meeting Their Goal Of Offering Algebra I To
100% Of 8th And 9th Graders? Did Student Achievement Increase?
Direct Data Sources
During the period 2015–2019, the MDOE was home to 620 middle schools and 548 high
schools, with an 8th-grade population ranging from 73,855 to 76,137 and a 9th-grade population
ranging from 88,483 to 86,353. Based on the direct data sources examined, it appears that
MDOE had achieved approximately 75% of its goal of offering Algebra I to all 8th and 9th
graders in the school system as of 2019. Moreover, the data revealed that student achievement
significantly improved between 2015 and 2019.
To answer the first research question, data on the availability of Algebra I courses
citywide was examined to determine whether the MDOE reached their goal. Additionally, data
for student achievement in courses pertaining to the Universal Algebra program were analyzed to
determine the program’s impact on student achievement.
Availability Of Algebra I Courses. Metropolitan City assessment data provided rich
information on the availability of Algebra I courses citywide, the number of sites where it was
offered, class composition of those students, as well as test data for students who took the Math
exam. In this section, assessment data from a state test as well as one national test was examined
to ascertain if the program was able to reach their goal of offering courses citywide.
Assessment data from a state and a national test were examined to ascertain whether the
program could reach its goal of offering courses citywide. The high school state exam assesses
student achievement in advanced courses such as Algebra I and Algebra II. For an MDOE
student to graduate, they must pass five exams. These exams can derive from the content areas of
English Language Arts, math, science, or social studies; additionally, other state-approved exams
43

or comparable assessment methods can be used instead. In context, this means that if a school
offers Algebra I, the corresponding Algebra I assessment is offered in that school. Therefore,
although the majority of test-takers are high school students, students in lower grades still take
the exam if they have taken the higher-level course. Table 6 provides data for the number of
students who took the Algebra I test during the period 2015––2019. These data reveal the extent
to which more exams were administered citywide, indicating an increased number of students
taking Algebra I.
Table 6
Number of Schools by School Level Offering Algebra I Courses
Source: https://infohub.nyced.org/reports/academics/test-results
Given schools must offer Algebra I coursework to students before testing can occur, the
number of test sites directly correlates to the number of schools offering Algebra I. According to
Table 6, there was a small increase in high schools offering Algebra I from 2015 to 2016 (428 to
442). However, Algebra I course offerings returned almost to their 2015 levels in 2017, and a
slight decrease in the number of sites was observed in 2019. However, there was an increase in
the number of schools offering Algebra I coursework in 8th grade, with 103 more schools
offering Algebra I in 2019 compared to 2015, a shift from 386 testing sites to 489. These data
demonstrate that roughly 78% of schools offering 8th grade have Algebra I coursework available
for their students at the beginning of the program; meanwhile, approximately 77% of high
44

schools offer Algebra I. Thus, the availability of Algebra I coursework does not meet the
MDOE’s 100% goal.
Table 7
Number of Algebra I Test Takers and Scoring Percentile Citywide 2015–2019
Source: https://infohub.nyced.org/reports/academics/test-results
Table 7 provides metrics for the number of students tested and their achievement level.
Given students can only take the test if they have taken an Algebra I course, the number of tests
taken effectively indicates the expansion of Algebra I course offerings citywide. The table
demonstrates that the total number of students tested increased by 42% from 2015 to 2016.
However, although there was a modest increase in the number of test-takers from 2016 to 2017,
the number of participating students remained fairly constant from 2017 to 2019. These data
align with the data from Table 6, which demonstrated a substantial increase at the site level for
students taking Algebra I in 2016, followed by the numbers remaining constant for the
subsequent study years.
Student Achievement. The second part of this research question pertains to student
achievement. For the period 2015–2019, the data revealed a significant increase in student
achievement, with students scoring above 80% on the exam increasing from 5,321 in 2015 to
29,674 in 2019. Additionally, the number of students achieving a passing grade (65%–79%)
increased from 36,091 to 79,080 (see Table 7). Upon accounting for the increased number of
exam-takers, the improved scores remain statistically significant, with 7% of students in 2015
45

scoring above 80% compared to 24% of students in 2019. The improvement was even more
pronounced for students achieving a passing grade (65%–79%), with 45% of students scoring in
this tier in 2015 compared to 65% of students in 2019. It is worth noting that there was a
substantial increase in students achieving a passing grade from 2017 to 2018. However, more
research is required to determine whether this resulted from a program intervention, a change in
program design or leadership, or a change in the design or scoring of the exam. Ignoring this
particular spike, modest increases in the number of passing students were observed, along with a
corresponding reduction of failing students.
Contextual Data Sources
The congressionally mandated NAEP assessment, supported by the U.S. Department of
Education and overseen by the National Center for Educational Statistics, provides the most
comprehensive available assessment for gauging national mathematics achievement. The scores
are released via the Nation’s Report Card and provide a dependable national representation of
student performance in core subjects such as mathematics. Score summaries, subscale scores,
and achievement levels correspond to proficiency levels (Glaser et al., 1997). In addition to
national assessments in mathematics, NAEP creates an annual snapshot of urban data; this
includes Metropolitan City schools.
Districts asked to participate in NAEP assessments are required to meet specific criteria.
They must have a population above 250,000, and half of the population must comprise minority
students or students eligible for free or reduced-price lunch. The school sample used for the
NAEP assessment represents all students in the district. Then, a random student sample is picked
from each school. These sample students constitute an extension of the NAEP’s state and
national samples and facilitate reliable reporting of student groups in selected districts. The
46

number of students assessed varies between districts. Students with disabilities and English-
language learners are incorporated into the assessments, with relevant adjustments made as
necessary. For the MDOE, the sample comprised 3,100 students. Figure 3 presents the scaled
average NAEP urban math assessment student performance scores by year.
Figure 4
Student Performance on NAEP Urban Math Assessment by Year

Source: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of
Educational Progress (NAEP), 2015, 2017, 2019 Urban School District Snapshot

The average scale score is a representation of the test items that a student has answered
correctly which have been converted into a standard scale for consistent measurement. The
average scale score for students who are below basic has increased 3% from 2015 to 2019. By
contrast, students who are considered advanced, which means that they have exceeded
expectations for mathematical knowledge according to their grade level, have only seen a 2%
increase year over year and both basic and proficient students have declined 3% and 2%
respectively year over year. This result is contrary to what was anticipated on the MDOE
promotional materials, namely, that the greater the school participation in the Universal Algebra
Program, the greater the gains in student achievement.
47

However, the NAEP urban snapshot of 8th-grade mathematics achievement indicates that
this unexpected inverse relationship is not statistically significant. Therefore, although the scores
demonstrate a slight decrease, they are not significant enough to indicate an actual overall
decline in student achievement. Additionally, the figures are not significantly different from
other urban districts.
Meanwhile, although the expansion of the Algebra I program was intended to solve
equity and access problems although it did not specifically address student-readiness, contextual
sources indicate that improvement was not realized for students of color. In April 2018,
Patch.com (Patch, 2018) published an article criticizing student achievement in mathematics
based on urban NAEP test scores. Based on the representative NAEP data, the article described
public dissatisfaction in Metropolitan City with the city’s declining NAEP scores, particularly
for students of color (Patch, 2018). Indeed, the gap between white and black student achievement
in mathematics, according to NAEP results, increased by 7% between 2015 and 2019. Although
the NAEP reported the difference as statistically insignificant, according to the Patch article
(2018), parents and policymakers considered the decline significant. That is, the NAEP data had
cultivated a negative perception of the impact of the Universal Algebra program on student
achievement. The Universal Algebra program represented a significant amount of effort to
improve scores leveraging millions of dollars as well as human capital, any decline in scores,
even though statistically insignificant were perceived as referendum on the success of the
program to parents and policymakers.
Summary
Despite an increased number of schools offering Algebra I in 8th and 9th grade, the
program does not meet the MDOE’s stated goal of achieving 100% availability. Based on the
48

data presented in Table 6, the program failed to establish itself in high schools: the number of
high schools offering Algebra I peaked at 442 (of 548 high schools in the system) or 76.6%.
However, schools offering Algebra I courses in 8th grade increased substantially to reach 78%
participation. Thus, although the MDOE has not attained 100% participation in 8th grade, they
have made significant steps (the addition of 103 8th grade sites) towards improving access for
students citywide. This should theoretically positively impact student achievement.
Meanwhile, Figure 4 reveals an inverse relationship between the Universal Algebra
program and high school student achievement. As the program became more prevalent city wide,
achievement declined, suggesting that teachers might not have attained the procedural
knowledge necessary to positively impact student achievement. Notably, Table 7 also
demonstrates that student achievement at high schools did not dramatically increase, particularly
after the first year, with the year 2015–2016 witnessing a large increase in the number of
students scoring above 80% and a corresponding decline in the number of students scoring below
65%. However, those increases were not built upon over subsequent years, with those
achievement level increases not replicated during years two through four of the program.
RQ2: To what extent were there opportunities for professional learning and certification in
mathematics for teachers in the Universal Algebra program?
Professional learning offers teachers opportunities to improve their pedagogy. Traditional
professional learning usually removes teachers from the classroom to attend workshops or
conferences focused on teaching theory and its classroom application. However, these kinds of
detached experiences are often unsuccessful (Archibald et al., 2011; Hargreaves, 2000) because
the professional learning does not align with the teachers’ actual daily classroom work. This
means that such approaches to professional learning might not impact the classroom learning
49

environment. Thus, many researchers have advocated for more in-depth approaches to
professional learning, approaches that connect content to practice (Knapp, 2003; Desimone,
2009). The Universal Algebra program was designed to associate professional learning with in-
classroom coaching, an approach discussed later in this chapter, with the provision of
professional learning to teachers considered a fundamental component of the Universal Algebra
program design.
Direct Data Sources
Information that is publicly available on the MDOE signup pages indicates that the
program used an annual cohort model, with each cohort receiving two years of professional
learning (MDOE, 2016). The professional learning comprised several components: a
professional learning series for teachers, a professional learning series for administrators,
membership in the National Council for Teaching Mathematics, and an online community. The
professional learning series for teachers was quite extensive, with all mathematics teachers for
grades 6–8 and all teachers delivering Algebra I offered 20 days of professional learning, which
included an intensive ten-day summer institute and ten days of professional learning throughout
the school year. This incorporated grade-appropriate training for all teachers. Additionally, all
participating teachers received payment for non-school days, or their school received payment
for a substitute to teach their class if sessions were held on school days (MDOE, 2016). The
professional learning was a combination of mandatory sessions in the school year and voluntary
sessions in the summer.
In addition to these professional learning opportunities, there were also “micro-
credentialing” opportunities offered through the local state college, as well as a partnership with
Bank Street College of Education to provide a Master of Education in Mathematics (CUNY,
50

2020; BSCE, 2020). The micro-credentialing option did not represent a genuine path to
certification, instead offering expanded coursework that could possibly be utilized by a
mathematics teacher in pursuit of a mathematics degree. Meanwhile, the Master of Education in
Mathematics program does indicate certification, requiring intensive participation over the
course of several years. Although the data provided confirms the availability of credentialing
resources, it does not confirm the level of participation by mathematics teachers.
Contextual Data Sources
Although no contextual data was available––for example, survey data concerning how
teachers perceived the effectiveness and comprehensiveness of the professionally learning
series––the MDOE does conduct an Annual School Survey considering city teachers’ attitudes,
beliefs, and knowledge about a variety of topics, including school culture, parental interaction,
professional learning, curriculum, and classroom management. The survey is subject area
specific with targeted questions for teachers in core content areas like mathematics, science,
English, and social studies. However, the data does not provide program-specific insights, it does
provide a broad picture of these areas. However, given the size of the MDOE––including over 1
million school children and over 30 districts––representative districts in the highest- and lowest-
performing quartiles were used for this section’s analysis, with District 2 representing the
highest-performing quartile and District 12 representing the lowest-performing quartile
(NYSED, 2017).
As Figure 5 shows, the Annual MDOE School Survey asked four targeted questions
specifically concerning professional learning.

51

Figure 5
Teacher Responses to Professional Development Questions

Overwhelmingly––and regardless of school performance––math teachers gave positive feedback
about their professional learning experiences.
However, there is no available data indicating how many teachers attended the various
professional learning opportunities. Nonetheless, the efficacy of the professional learning
sessions can be inferred from responses to the Annual School Survey, with 85% of teachers
surveyed indicating that they had the necessary acumen and skills to teach in the classroom.
Summary
The MDOE offered a robust series of opportunities for teachers to develop
professionally, as well as opportunities for certification in their field to improve their pedagogy.
However, there it is not possible to definitively determine the impact, scale, or scope of these
opportunities without additional usage data.
52

RQ3. To what extent was there broad city-wide support for the program as well as the
availability of curricular and coaching resources?
The primary responsibility of public schools is to effectively educate their students.
Substantial evidence positions teacher efficacy in causal connection with student learning
(Connor, 2017; Kraft & Blazar, 2018; Walkowiak, 2016). Therefore, schools have increasingly
examined ways of improving their professional learning frameworks to develop the capacity of
teachers to provide high-quality instruction (Quintero, 2019). Existing studies promote
personalized, embedded instructional coaching as a way of supporting professional learning and
improving classroom impact (Kane & Rosenquist, 2018; Kraft & Blazar, 2018). City-wide
support for this program was analyzed using the amount of funding and human capital dedicated
to the success of the program.
Direct Data Sources
School allocation information from the MDOE website indicated that, for the year 2016–
2017, $1.3 million was provided to schools (via a grant application process) for Universal
Algebra professional learning partner organizations, direct services to students, and family and
community mathematics education. Specifically, the school allocation memorandum indicated
that funding could be used to cover expenses including teacher overtime, supplies, supplemental
curricula, materials, or math manipulatives. In 2018 and 2019, funding increased to $2.7 million
and $2.5 million. According to the MDOE website, there were no substantive changes to the
grant process or funding purpose in subsequent years. However, the number of funded schools
tripled from approximately 50 to approximately 150. There is no substantive data available
concerning the grant application format or the requirements for the schools applying. Notably,
53

the allocation of funding in 2016 and 2017 occurred late in the fiscal year, potentially
complicating its expenditure as schools had planned.
In the mayor’s preliminary budget plan in 2016, a year after the program’s introduction,
increased funding for education programs was introduced to support the mayor’s previously
announced agenda of Equity and Excellence in Education (IBO, 2016). The executive budget
plan also included additional funding for the Universal Algebra program. Meanwhile, the budget
included additional funding for both the MDOE and the City University in support of their
collaborative micro-credentialing program. This demonstrates the mayoral office’s interest in the
program’s success and indicates the reasons for the increased funding provided to schools.
However, a review of the allocation of funding to schools indicated that the program
failed to allocate funding to an historically under-performing section of the City in 2017, a
predominantly black and Hispanic area comprising several historically poorly performing
districts (MDOE, 2018). Furthermore, a review of the allocation table pertaining to the school
allocation memorandum indicates that the program did not provide additional funding to four of
the eight most poorly performing districts (Students First, 2020; MDOE, 2017).
Contextual Data Sources
The MDOE website indicates that multiple professional learning contracts and
curriculum contracts were available for schools to utilize a simplified bid process to select from
pre-approved vendors. These vendors typically support in-school coaching as well as curricula
and professional learning related to that curricula. However, there is no data indicating whether
schools used their grant funding to derive additional support from such services. Nonetheless,
there is evidence that overall efforts to improve schools were prevalent, with 87% of teachers
responding to the Annual School Survey valuing school improvement efforts (MDOE, 2017) and
54

86% indicating that their principal valued professional development (MDOE, 2017).
Accordingly, although the data does not indicate specific support for the Universal Algebra
program, it does suggest broad support of professional development or in-school coaching
activities.
RQ4: What are the systems’ knowledge and motivation influences related to achieving this
organizational goal?
This research question is addressed in Chapter Five of this study.
Discussion of Findings
Knowledge
The Universal Algebra program’s efficacy is predicated on teachers attaining factual,
conceptual, procedural, and metacognitive knowledge. These knowledge areas address the
preparedness of teachers to realize the Universal Algebra program’s goals.
Factual Knowledge
Teachers Will Have Knowledge Of Formative And Summative Assessment. The
factual influence identified from the literature in Chapter Two indicated that the teachers in the
Universal Algebra program have access to the summative assessment data for statewide tests for
their schools. Although this is not student specific information, it does provide broad knowledge
of achievement at the school level. Teachers also have access to summative assessment
information using Measures of Student Learning assessments (MOSL). Teacher evaluations are
predicated on two baseline measurements: Measures of Teacher Performance and Measures of
Student Learning are local formative assessments that measure what students are learning in the
classroom. As a required component of teacher evaluations, all teachers employed by the MDOE
have access to these tools. Additionally, these formative assessment tools are free for schools,
55

meaning funding availability does not impact a school’s ability to use them. Notably, the data
also clearly provide information on summative assessments, another key indicator of student
performance.
This influence was determined to be an asset because according to the available data,
Universal Algebra teachers did have access to resources that could support the development of
the factual knowledge needed to enhance student achievement levels. However, there was a
discrepancy in teachers’ level of satisfaction with their access to professional learning and
resources, with teachers in the bottom quartile 7%–8% less satisfied than teachers in the top
quartile.
Conceptual Knowledge
Teachers Will Have Knowledge Of Higher Level Math Pedagogy. One identified
impediment to the citywide expansion of Algebra I was many teachers not possessing the
expertise necessary to teach higher-level mathematics to students. Therefore, this influence has
been determined to be a cause because the lack of knowledge spurred the inception of the
program. As part of the Universal Algebra program’s offerings, professional learning was
offered to teachers to enhance their ability to teach higher-level mathematics. The data available
indicates that these professional learning series occurred and that teachers generally responded
positively to these opportunities. Additionally, there were both in-school options and
opportunities for higher education leading to specific certifications. Thus, the data suggest that
teachers were provided access to the conceptual knowledge necessary to achieve the program’s
goals. However, without further research, it is not possible to ascertain the extent to which
teachers utilized these resources or to understand the impact on classroom pedagogy that these
resources might have had.
56

Procedural Knowledge
Teachers Will Be Able To Implement Improved Math Instruction Strategies In The
Classrooms. In alignment with the prior influence, this has been determined to be a cause. The
data indicate that Universal Algebra teachers had the resources necessary to implement improved
math instruction strategies in the classroom. In addition to in-school resources, teachers had
access to professional development, certifications, and degrees. Additionally, there was an online
learning community providing opportunities to speak with both instructional specialists and
peers. However, it is worth noting that certain additional resources were only available to schools
that applied for additional funding via the program’s centralized grant process, established by a
school allocation memoranda. Further research would be required to determine the extent to
which additional resources were used and what impact they had on in-class instruction.
Metacognitive Knowledge
Teachers Need To Be Able To Self-reflect And Consider How Effectively They Are
Implementing The Higher Level Mathematics Strategies In Their Classrooms. Although
the extant data do not explicitly address teacher self-reflection, it does enable correlations to be
identified between the data collected and the need for self-reflection prompting response to the
survey. As discussed, the Annual School Survey asked teachers various questions concerning
their professional learning experiences and how they felt those experiences had impacted their
classroom performance. Additionally, by inquiring about teacher’s abilities to teach
mathematical concepts, these questions required reflection to enable teachers to answer the
questions. This indicates that teachers reflect on their own abilities at least once annually as part
of the Annual School Survey. Data on self-reflection may have existed as part of the feedback
57

cycle for the professional learning, however, that data, if it exists, is not publicly available.
Therefore, it is not possible to draw conclusions about opportunities for self-reflection beyond
the Annual School Survey. Given the lack of conclusive data in this area, it is not possible to
determine if this was a cause, need, or asset.
Motivation
Teachers Need To Believe That They Are Well-versed And Can Effectively Teach Higher
Level Mathematics
As indicated, teachers citywide overwhelmingly expressed confidence in their ability to
teach higher-level mathematics concepts. However, an estimated 10% of teachers citywide still
did not indicate such a level of confidence, meaning roughly 1,000 teachers in MDOE schools
perceived their ability to teach higher-level mathematics as deficient. Although the data are too
general to provide specific insight into the impact on students of teachers lacking acumen in
mathematics instruction, this statistic indicates that a significant number of teachers were not
providing quality instruction in the classroom. However, the Universal Algebra program does not
feature any components specifically designed to target teachers who are deficient in higher level
math knowledge and there is no mechanism to measure or assess whether a teacher’s confidence
in their own ability is based on skills they actually possess.
Additionally, there is no evidence indicating that fewer teachers are teaching outside of
their subject area. If the program had truly increased knowledge through certification, this would
be demonstrated by a reduction in teachers operating outside of their subject area. The MDOE
state websites indicate that there has not been a significant increase in teachers certified to teach
in their subject area (NYSED, 2020). It is possible that teachers have acquired the knowledge
58

but decided not to pursue certification in the subject area. However, there is no data available
that indicates that this has occurred at the state or city level. Given the results of the survey
overwhelming support that teachers have the skills to teach, it can be concluded that this
influence is an asset.
Teachers Believe That The Instruction They Provide Is Important To Their Students’
Achievement
The above goal, referenced in chapter two is an example of expectancy-value theory
proposes that motivation for a given activity is predicated on two components: the probability of
the desired outcome being achieved by engaging in specific behavior and the value of that
outcome. In the case of the Universal Algebra program, these two components are represented by
engagement with professional learning and improvements in student achievement. Based on the
Annual School Survey, it is apparent that teachers and principals have dedicated themselves to
school improvement efforts oriented towards improving students’ performance. Data from the
survey indicate persistent engagement with professional learning and support and encouragement
from school leadership to engage in such initiatives. Survey items for this survey includes
questions to administrators about their support of teacher professional development as well as
questions related to how much support teachers felt that they received from their administrators
and both overwhelming indicated support in these areas. In addition, 88% of teachers city-wide
indicated that they were having consistent and long-term support through professional learning.

59

Organization
Cultural Model Influence 1: The Organization Needs To Have A Culture That
Supports Achievement Irrespective Of Geography, Or Racial Or Ethnic Background. The
MDOE features a long history of lower achievement levels for African-American, Hispanic, and
other minority populations, likely deriving from Metropolitan City’s history of heavily
segregated neighborhoods. For many years, lines were drawn to intentionally prevent people
from poorer and more minority-dominated neighborhoods from having autonomy over the
choice of school for kindergarten through to 8th grade. That is, for much of the last century, the
neighborhoods above 96th Street and below 14th Street were predominantly non-white. All
schools between 14th and 96th Streets could choose any school within those geographic
boundaries (New York School Talk, 2016). However, schools outside of those geographic
boundaries were not afforded the same choice.
Meanwhile, for entrance to high schools, the city utilizes a standardized high school
entrance exam, zoning, and application processes. Additionally, the city’s most elite high schools
rely solely on the entrance exam and application, with no students zoned to automatically enter.
This has produced a contentious battle over admission, especially given these schools frequently
provide pathways to enrolment at elite colleges (Apex for Youth, 2018). Efforts to reform the
system have been met with fierce resistance by upper-class parents and alumni, contributing to
the departure of the previous Chancellor. This situation prompted the development of the
Equality and Achievement umbrella under which the Universal Algebra program was introduced.
The institutional support for equity which can be seen in the creation of the nine equity programs
under the current Mayor and the infusion of over $500 million dollars in equity initiatives over 5
years (IBO, 2016) indicates that this influence is an asset.
60

Cultural Model Influence 2: The Organization Needs to Have A Culture That
Provides Targeted Support to Historically Under-Resourced Communities. The MDOE has
been criticized for decades for the lack of opportunity afforded to and the disparate treatment of
minority students. A 1996 ACORN report cited multiple instances of institutional bias in
Metropolitan City, ranging from parental access to educators to access to quality programming,
including gifted programs. However, plans targeting minority and historically under-resourced
communities broadly did not emerge until 2015, when a series of targeted programs attempted to
address systemic inequity. These programs, developed under the Equality and Excellence
umbrella, focused on access to computer science, advanced placement courses, college readiness,
literacy, and algebra, among other areas (MDOE, 2015). Additionally, one program incentivized
teachers to teach in poorer neighborhoods by paying them a supplement to commit to teaching in
those locations and aggressively recruiting teachers to teach in this area of the city. Educators
also recruited more minority teachers to teach in these schools via the Minority Men’s Initiative
(New Amsterdam News, 2015). As addressed earlier in this chapter, the MDOE also challenged
admissions to specialized high schools and gifted programs (Apex for Youth, 2018), as well as
soliciting private funding for school improvement efforts for multilingual learners by engaging
the Gates Foundation to provide additional support in the area (MDOE, 2019).
Given the many programs that the MDOE has implemented since 2015, it could be
concluded that they have increased and sustained efforts to address historic inequities and
institutional racism through targeted approaches to improving the academic outcomes of poorly
performing schools. However, despite the targeted teacher recruitment for one area of the city,
many areas continue to be impoverished and in need of sustained recruitment and retention
efforts to keep quality teachers in those districts. Additionally, supplemental funding programs,
61

such as the grants offered through the Universal Algebra program, do not necessarily target these
communities. Therefore, this influence is a need.
Cultural Setting Influence 1: Metropolitan City Is Ethnically And Racially Diverse.
Tensions Over Equity In Education, Including The Allocation Of Resources, Is High.
Equitable Distribution Of Resources Is Necessary To Achieve Program Goals. The
disproportionate allocation of resources to public schools in wealthier and poorer neighborhoods
has been a point of contention across public education for decades. The MDOE has a history of
under-resourcing Title I schools (Rivera Rodas, 2019), with the data collected from the Annual
School Survey indicating that teachers at schools in the lower quartile––all of which are Title I
schools––reported less satisfaction with the resources provided to them than their peers working
in better-performing school districts. Additionally, the data collected regarding the Universal
Algebra program’s distribution of grant funding does not indicate increased funding for poorer-
performing school districts. Instead, it indicates that school districts that were already better
performing were more likely to receive funding than their poorer performing peers (MDOE,
2017).
Thus, the MDOE has failed to achieve this goal by not monitoring how additional
resources offered through internally and externally restricted funding are allocated to ensure
different districts are equitably resourced. Accordingly, Chapter Five explores remediation
efforts that could improve the structural weaknesses in the Universal Algebra program.

62

Table 8
Summary of Influences – Cause, Asset, or Need
Assumed Influence: Type Cause, Need, or Asset
Teachers need to know
specific examples of Higher-
level mathematics teaching
strategies. (D)
Knowledge Asset

Teachers will have
knowledge of higher-level
math pedagogy. (C)
Knowledge Asset
Teachers will be able to
implement improved math
instruction strategies in the
classrooms. (P)
Knowledge Need
Teachers need to be able to
self-reflect and consider how
effectively they are
implementing the higher-level
mathematics strategies in
their classrooms. (M)
Knowledge Need
.
Self-Efficacy: Teachers
believe their ability to teach
higher -level mathematics.
Motivation Need
Utility Value: Teachers feel
that the instruction they
provide is important for their
students’ achievement.
Motivation Asset
Cultural Model Influence 1:
The organization needs to
have a culture that supports
achievement irrespective of
geography, or racial or ethnic
background
Organization Asset
Cultural Model Influence 2:
The organization has a culture
that provides targeted support
to historically under-
resourced communities.

Organization Need
63

Cultural Setting Influence 1:
Metropolitan City is
ethnically and racially
diverse. Tensions over equity
in education, including the
allocation of resources, is
high. Equitable distribution of
resources is necessary to
achieve program goals.
Organization Asset

64

Chapter 5: Recommendations and Evaluation

Chapter Four used findings from the available data to address the first three research
questions, identifying factors concerning knowledge, motivation, and organizational influences
to evaluate the efficacy of the Universal Algebra program in terms of student achievement,
access to coursework, teacher attitudes, and the program’s institutional support. Chapter Four’s
discussion also analyzed these factors to evaluate whether the Universal Algebra program had
reached its stated goals during the study period. Chapter Five responds to the final research
question by proposing solutions to the needs identified in Chapter Four and framing those
solutions within the context of the MDOE. The chapter concludes by suggesting an evaluation
plan that responds to these recommendations.
This project’s purpose is to evaluate the complete system-wide implementation of the
Universal Algebra program, especially access to Algebra I courses and its impact on student
achievement. The evaluation concentrates on stakeholder resources in terms of knowledge and
skill, motivation, and organizational influences and opportunities. Although a complete
evaluation would analyze all stakeholders, this analysis focuses on the stakeholder group that is
teachers participating in the Universal Algebra program.
The following questions guide this evaluation process:
1a. To what extent is the Department meeting their goal of offering Algebra I to 100% of
8th and 9th graders?
b. Did student achievement increase?
2. To what extent were there opportunities for professional learning and certification in
mathematics for teachers in the Universal Algebra program?
65

3. To what extent was there broad city-wide support for the program as well as the
availability of curricular and coaching resources?
4. What are the systems’ knowledge and motivation influences related to achieving this
organizational goal?
Recommendations to Address Knowledge, Motivation, and Organization Influences
The assumed knowledge, motivation, and organizational influences that impacted the
implementation of the Universal Algebra program in the MDOE have been analyzed to assess
the program’s weaknesses and provide recommendations for improvement. This chapter
provides detailed recommendations for each assumed knowledge, motivation, and organization
influence identified in Chapter Four, with the tables included to clearly present each assumed
influence alongside its conceptual principle, reference in the literature, and a context-specific
recommendation. These assumed influences were not collected from teacher data. Each table is
followed by a discussion building upon the supporting literature.
Knowledge Recommendations
Organizations must address knowledge gaps to realize their goals (Clark & Estes, 2008).
Here, knowledge is classified into four categories: declarative, procedural, conceptual, and
metacognitive (Krathwol, 2002). Declarative knowledge constitutes the basic information about
a particular subject or discipline that teachers must be acquainted with; procedural knowledge
describes how to do something such as the needed methods, techniques, and knowing when to
use certain procedures; conceptual knowledge refers to the knowledge of, or understanding of
concepts, principles, theories, models, classifications metacognitive knowledge is “knowledge
of [one's own] cognition and about oneself in relation to various subject matters” (Krathwol,
2002). Organizations can provide assistance to support growth in these areas in the form of, for
66

example, models of teacher success, professional learning, information, or education (Clark &
Estes, 2008).
This study results in the recommendation for a revised implementation and evaluation
plan that incorporates all four knowledge areas with complimentary support. This approach will
improve teacher performance and ultimately positively impact student achievement. The
literature maintains that teachers must be clearly aware of the expectations associated with
student-centered learning in connection with professional learning experiences to effectively
implement student-centered learning in the classroom (Estes, 2004; Lee & Hannifan, 2016). To
efficiently engage knowledge transfer in the classroom, teachers must have a developed
cognitive schema and clearly outlined conceptual understanding of their own and their students’
learning.

67

Table 9
Summary of Knowledge Influences and Recommendations
Assumed Knowledge
Influence: Cause, Need, or
Asset
Principle and Citation Context-Specific
Recommendation
Teachers need to know
specific examples of Higher-
level mathematics teaching
strategies. (D)

Creating schemata helps
learners to organize
declarative knowledge in a
domain (Schraw, Veldt, &
Olafson, 2009).
Teachers will be provided
with training on specific
structures, resources, and/or
scaffolding in the classroom
to support higher mathematics
pedagogy.
Teachers will have
knowledge of higher-level
math pedagogy. (C)
To develop mastery,
individuals must acquire
component skills, practice
integrating them, and know
when to apply what they
have learned (Schraw &
McCrudden, 2006).

Teachers will be provided
with training on higher-level
math concepts and will utilize
on-site support and coaching
to encourage knowledge
transfer.
Teachers will be able to
implement improved math
instruction strategies in the
classrooms. (P)
To develop mastery,
individuals must acquire
component skills, practice
integrating them, and know
when to apply what they have
learned (Schraw &
McCrudden, 2006).
Teachers will be given in
classroom coaching including
guidance and sample rubrics
for lesson plans that support
higher level math pedagogy.
Teachers need to be able to
self-reflect and consider how
effectively they are
implementing the higher-level
mathematics strategies in
their classrooms.
(M)
The use of metacognitive
strategies facilitates learning
(Baker, 2006).
Teachers will be provided
with a model to analyze and
self-reflect on their processes
and knowledge.
.

Note. An asterisk indicates the knowledge type for each influence listed using these
abbreviations: (D)eclarative; (P)rocedural; (M)etacognitive; (C)onceptual

68

Declarative and Conceptual Knowledge. This evaluation has demonstrated that 15% of
mathematics teachers require more comprehensive declarative knowledge about the types of
scaffolding necessary to teach different learners. A proposal grounded in information processing
theory has been chosen to bridge this knowledge gap. Shraw, Veldt, and Olfsan (2006) have
posited that creating schemata helps learners to organize declarative knowledge in a domain,
implying that providing learners with tools and aids to help them organize their work
thematically helps them to process information more effectively. Therefore, the recommendation
is to provide Universal Algebra teachers with an Ishikawa diagram (Ishikawa, 1990), with the
fish-shaped lines of this map demonstrating the connections between pedagogy and student
learning.
Additionally, high-quality formative feedback can meaningfully improve learning
processes and outcomes (Shute, 2008). Integrating factual knowledge with improved instruction
in the classroom requires an understanding of the areas in which students need to improve.
Teachers can attain this information by reviewing test items for items with a high failure rate
(Wasserman, 2016). Rather than simply administering a test and assigning grades, teachers
should revisit high failure items to ensure the class’s understanding.
Procedural Knowledge. The Universal Algebra program strives to provide myriad forms
of support for teachers’ classroom instruction, thereby improving student achievement and those
teachers’ professional acumen. Only 15% of the mathematics teachers surveyed in the Annual
School Survey indicated possessing the procedural knowledge necessary to improve classroom
instruction. Teachers will be provided with on-site coaching and models of teacher success and
opportunities for feedback that facilitate lesson plan development. Showalter (2017) conducted a
study on the impact of higher-level math courses on higher-level math pathways, identifying the
69

probability of students in those courses avoiding remedial math courses in post-secondary
education. This study parallels the objective of the Universal Algebra program. Defining higher-
level mathematics is defined as knowledge of trigonometry, algebra, and calculus, the
researchers concluded that students needed a strong foundation in higher-level mathematics to be
successful. Universal Algebra teachers, who receive professional development during the
summer, must be able to apply this knowledge in the classroom.
Two other studies have also maintained that teachers’ fundamental understanding and
knowledge about how to effectively complete mathematical processes and compose coherent,
aligned lesson plans greatly improve student achievement (Wasserman, 2016; Ross & Willson,
2012). Notably, Universal Algebra teachers frequently work in underserved schools, where
students receive poor preparation and have limited access to resources. Therefore, this procedural
knowledge is even more integral to the Universal Algebra model, which accordingly provides
essential pedagogical resources to teachers. Ultimately, while teachers must know basic math
information, it is also apparent that they must be able to apply and model this information in the
classroom.
Metacognitive Knowledge. The available data did not indicate that teachers were given
opportunities to substantively reflect on their own learning. Therefore, the recommendation for
metacognitive knowledge focuses on opportunities and resources that would allow teachers to
reflect on their own learning through journaling and professional learning communities. Teachers
profit from establishing robust relationships with both their work and their own pedagogy
(Wasserman, 2016). This is particularly important for teachers applying their own metacognitive
skills to better understand their own learning.
When teachers use metacognitive skills before, during, and after professional learning to
70

process information, it enhances the retention of skills and knowledge in long-term memory,
which, in turn, improves knowledge transfer (Grossman & Salas, 2011; Deans of Learning,
2018). The research suggests that the intersection between the knowledge that a teacher has
about themselves and their learning processes and their training and development produces a
framework for improving classroom instruction (Wasserman, 2016; Ross & Willson, 2012). A
teacher’s attainment of personal growth and development can be realized via professional
learning opportunities (Aguinis & Kraiger, 2009; Clark & Estes, 2008) that expand their
metacognitive abilities alongside their ability to understand the learning and developmental
needs of their students. Therefore, teachers should participate in teacher teams and reflective
journaling to better understand their own pedagogy.
Motivation Recommendations
Of the multitude of theories of motivation, this study has focused on self-efficacy theory
and expectancy-value theory to study teachers’ assurance in their own abilities and to understand
the factors to which teachers attribute poor performance. Motivation determines an individual’s
engagement from the beginning of a task to its successful completion (Mayer, 2011). The
research posits that individuals perform at higher levels by consciously choosing to become
involved, applying mental effort, and persisting to achieve organizational and personal goals
(Clark & Estes, 2008; Grossman & Salas, 2011). It is essential to understand motivational
influences and how they impact performance problems when addressing an organization’s needs
in the context of improving performance (Rueda, 2011). Motivation is the second element in the
Clark and Estes (2008) three-tiered model for assessing and resolving performance problems.
The motivational factors that influence achievement, issues, assets, and needs have been
evaluated using the concepts of choice, persistence, and mental effort (Clark & Estes, 2008;
71

Jensen, 2012; Mayer, 2011; Rueda, 2011). Table 9 presents the motivation influences identified
alongside principles derived from the literature and recommendations for improved teacher
motivation.
Table 10
Summary of Motivation Influences and Recommendations
Assumed Motivation
Influence
Principle and Citation Context-Specific
Recommendation
Self-Efficacy: Teachers
believe their ability to teach
higher -level mathematics.

Self-efficacy beliefs provide
the foundation for the
abilities that individuals
believe that they have and the
outcomes that they are able
to achieve (Pajares, 2006).
Provide models of teacher
success and opportunities for
feedback and practice.
Expectancy-Value: Teachers
feel that the instruction they
provide is important for their
students’ achievement.
Performance improves when
people have high
expectations and a strong
belief in their own ability and
the value of the project with
which they have been tasked
(Eccles, 2006).
Detail the merits of
completing the task and the
benefits to the students.

Self-Efficacy. Approximately 15% of math teachers in the lower quartile of schools
reported they were not certain that they have the skills to effectively teach higher-level
mathematics to their students. A proposal founded in self-efficacy theory has been chosen to
bridge this gap in declarative knowledge. Teachers’ self-efficacy increases as they participate in
growth opportunities and professional learning and receive positive modeling and feedback from
their colleagues (Tang et al., 2014), implying that providing teachers with professional learning
opportunities would increase their confidence in their ability to perform their work. Thus, the
MDOE should provide professional learning opportunities to assist teachers to identify and refine
their pedagogy. Combining professional learning with models of teacher success will positively
impact teacher self-efficacy.
72

According to Daughtry, B., Akagi, and Strickland (n.d.), increased computer literacy
promotes higher levels of classroom computer use, with teachers more likely to cultivate
technological self-efficacy and technological content knowledge following participation in
training. Furthermore, that paper’s findings indicate that 100% of that exposure increased daily
usage and interest. Given the implications for student success of the quality of mathematics
instruction provided by teachers, this suggests that it is essential to improve teacher self-efficacy
by enhancing their skills by providing learning opportunities.
Expectancy-Value. According to Eccles (2006), performance improves when people
have high expectations, a strong belief in their own ability, and value the project with which they
have been tasked. The two critical components of the expectancy-value theory are the belief that
one can complete the task (i.e., expectancy) and whether they want to complete the task (i.e.,
value). That is, expectancy can be gauged by the level of confidence one displays concerning
their skills and their capacity to achieve a goal (Clark & Estes, 2008), and value refers to the
degree of significance an individual assigns to a task (Rueda, 2011).
Eccles and Wigfield (2002) have postulated that achievement choices are determined by a
mixture of peoples’ expectations of accomplishment and the subjective value of that undertaking,
and Clark and Estes (2008) have indicated that people are more willing to complete a task if they
see the value in that task and are interested in the task. Therefore, administrators should include
opportunities to communicate the value of a task and the benefit of that task and to undergo
professional learning to maintain teacher interest in their field.
Organization Recommendations
School districts must commit to addressing systemic and historic inequality within their
districts to affect widespread change and improve student achievement. Under-achieving schools
73

have typically received fewer resources, less access to funding, and impediments to attracting top
teaching talent. A cursory value stream analysis––based on the work of Clark and Estes (2008) –
–of the MDOE identifies a connection between student achievement and a lack of resources,
including human capital. In conjunction with value streams, Clark and Estes (2008) have also
appraised the significance of value chains for organizational efficacy. Value chains utilize the
knowledge from a value stream to isolate the impediments to organizational efficacy. The
difference in resources and access provided to wealthier communities and those provided to
schools in historically under-resourced communities promulgates student achievement inequity
across race and socio-economic lines. The research indicates that teachers in more poorly
performing districts consistently report lower satisfaction with their access to resources and are
less confident in their ability to teach higher-level mathematics. The deficits within these districts
hinder teachers’ ability to implement higher-level mathematics in their classrooms. Change is
needed at the city level to ensure all schools have equitable––rather than equal––access to
resources enabling improved student achievement.

74

Table 11
Summary of Organization Influences and Recommendations
Assumed Organization
Influence

Principle and Citation

Context-Specific
Recommendation
Cultural Model Influence 1:
The organization needs to
have a culture that supports
achievement irrespective of
geography, or racial or
ethnic background
Equity, diversity and access
are important goals in private
and public sectors (Darling-
Hammond, 2007; Lim,
Haddad & Daugherty,
2013)
More buy-in needs must be
achieved at the school leader
level. Teachers in lower
performing schools need
additional support and
engagement. This can
involve including principals
in the change process.

Cultural Model Influence 2:
The organization has a
culture that provides targeted
support to historically under-
resourced communities.

The organization must
provide adequate knowledge,
skills, and motivational
support to everyone (Clark &
Estes, 2008).
Teachers need the training
and information necessary to
ensure that targeted support
can be offered to their
students.

Cultural Setting Influence 1:
Metropolitan City is
ethnically and racially
diverse. Tensions over equity
in education, including the
allocation of resources, is
high. Equitable distribution
of resources is necessary to
achieve program goals.
Ethical decision-making can
be applied to various tasks
and different sectors (Strike,
Haller & Soltis, 2005).
School funding allocation
methodology must be
examined to ensure that
resources are being equitably
distributed.

Organizational influences operate in concert with knowledge and motivation influences
to factor in the gaps in teachers’ adeptness at implementing higher-level mathematics pedagogy
in their classrooms. According to Clark and Estes (2008), although a stakeholder group may
possess the requisite knowledge and motivation, performance gaps persist because of problems
surrounding the organization’s cultural models and settings. Cultural models are the structures
that exist within the organization, and cultural settings are cultivated within the organization
75

(Schein, 2004). An organization cannot achieve its performance goals in the context of cultural
models featuring teachers unwilling to alter teaching techniques and tensions arising from the
introduction of new pedagogies.
The recommendation to address inequity is to examine the system holistically. This
includes looking beyond the traditional funding model for schools which is weighted to account
for many different aspects of a schools’ population including additional funding for English
Language Learners, Special Education students, and immigrant populations. The systemic
inequity also includes myriad other factors such as curriculum and instruction, the allocation of
supplemental funding like the Universal Algebra grant program, as well as teacher and principal
recruitment and retention.
Although the Universal Algebra program is intended to offer targeted support to under-
resourced communities, its implementation has perpetuated inequality by failing to provide
additional support to struggling districts that would enable them to improve student achievement,
support such as in-classroom coaching. Changing this would entail engaging in a system-wide
analysis by area of focus. Additionally, teachers and principals must be involved in the change
process in the form of focus groups or individual contributions to strategies, such as utilizing best
practices from better-performing schools; this would increase buy-in at the local level.
Ideally, the MDOE would be able to engage trained consultants who could lead teachers
and administrators, as well as central employees, in this process. For the MDOE to successfully
address this problem, all parts of the system need to be actively engaged in changing that system.
Teachers and administrators must work upwards from examining practice at the classroom level,
while policy and central administrators must work from the top down to provide solutions
leading to the equitable reallocation of resources.
76

Integrated Implementation and Evaluation Plan
This Universal Algebra evaluation plan is grounded in the New World Kirkpatrick Model
(Kirkpatrick & Kirkpatrick, 2016), which advocates for a methodology that originates with the
goals of the organization and then works backward towards achieving those goals. By adopting
this approach, leading indicators on the path to achieving these organizational goals can be more
readily identified and aligned with the organization’s goals. The reversal posited by this model
encompasses three actions; the creation of solution outcomes designed to assess work behaviors,
the identification of leading indicators during implementation and any indicators of staff
satisfaction. The model is designed to promote organizational support and connections with
immediate solutions and broader organizational goals (Kirkpatrick and Kirkpatrick, 2016).
Organizational Purpose, Need and Expectations
One of the OCIPD’s goals in its implementation of the Universal Algebra Initiative is for
all students to have access to Algebra I courses by the 9th grade. The research suggests that this
will embolden students to pursue advanced mathematics in high school, better preparing them for
post-secondary and career success. By 2022, all Metropolitan City students should have access to
an Algebra I course in 8th grade, with scaffolding in elementary and middle schools designed to
promote algebra readiness. The OCIPD’s goal attainment will be gauged by the number of
schools offering Algebra I or a means to take Algebra I.
This evaluation’s stakeholder group of interest was math teachers, whose goal was to
provide students with access to Algebra I. The biggest impediment to broadening access to
Algebra I was determined to be a lack of teachers with the content knowledge required to teach
mathematics. School administrators have had to departmentalize mathematics with staff
members incapable of teaching higher-level mathematics. Training teachers to teach higher-level
77

mathematics will enable them to teach more higher-level math classes; this, in turn, will increase
access for students.
As mentioned, the program’s stated goal is for all students to have access to Algebra I
classes by 2022. Additionally, the program has been mandated to address performance gaps in
traditionally under-resourced communities. Accordingly, this evaluation has examined the
knowledge, skills, motivational, and organizational barriers impeding teachers from teaching
higher-level mathematics. Contrasting with the preceding theoretical discussion, the following
recommendations constitute an action plan promoting improved teacher buy-in and confidence
and improved student performance. This action plan utilizes the results and leading indicators to
evaluate the program’s progress towards its goals.
Level 4: Results and Leading Indicators
Kirkpatrick and Kirkpatrick (2016) have identified Level 4 as the extent to which
targeted results occur due to training in conjunction with support and accountability measures
associated with that training. The organization creates a framework for achievement linking
training to results and the organizational mission and goals. Leading indicators provide short-
term benchmarks assessing whether critical behaviors can achieve the desired results. Level 4
results and leading indicators are enumerated in Table 11 as outcomes, metrics, and methods for
the program’s internal and external outcomes. If internal outcomes are met, it is understood that
external outcomes should also be met.

78

Table 12
Outcomes, Metrics, and Methods for External and Internal Outcomes
Outcome Metric(s) Method(s)
External Outcomes
Improved performance
on standardized math
assessments

State test scores Improved instruction in the
classroom through professional
development and coaching.
Increased access to
Algebra I classes
Auditing of offerings
citywide
Data gleaned from student
information systems
Internal Outcomes
Improved teacher
satisfaction with student
performance
Report card comments,
performance reviews
Review report card comments and
performance feedback, monthly
surveys

Level 3: Behavior
Critical Behaviors. Kirkpatrick and Kirkpatrick (2016) define Level 3 as the extent to
which participants utilize what was taught during training upon returning to work. Critical
behaviors are the essential behaviors that teachers must reliably execute to realize the targeted
outcomes. With teachers are the stakeholders of interest, the first critical behavior is their
documentation of their progress towards achieving the organizational goals for the program. The
second critical behavior is teacher use of Universal Algebra resources to remove barriers to
content knowledge and low self-efficacy. The third critical behavior is teacher advocacy for
targeted support for struggling students. Metrics, methods, and timing for outcome behaviors are
listed in Table 12.

79

Table 13
Critical Behaviors, Metrics, Methods, and Timing for Evaluation
Critical Behavior Metric(s)
Method(s)
Timing
Teachers will
document their
progress to
organizational
goals for the
program
Number of goals
achieved
Journal of
progress towards
organizational
goals
Monthly journal entries.
Teachers will
identify and
utilize the
appropriate
resources to
improve their
pedagogy.
Number of
resources used
Survey Monthly surveys regarding
resources utilized and purpose
of utilization
Teachers will
identify students
in their
classrooms who
require additional
supports and will
procure those
supports for them
.
Self-reported
data from
teachers on their
use of services
Survey Quarterly survey

Required Drivers. In the context of this evaluation and based on the model provided by
Kirkpatrick and Kirkpatrick (2016), required drivers are those “processes and systems that
reinforce, monitor, encourage, and reward performance of critical behaviors” in the classroom.
Teachers need the support of school and district staff to reinforce professional learning and
encourage implementation in the classroom. To optimize organizational support of teachers,
rewards should be established for the achievement of performance goals wherever union
contracts allow it. Table 13 lists drivers required to support critical behaviors.

80

Table 14
Required Drivers to Support Critical Behaviors
Method(s) Timing
Critical Behaviors Supported
1, 2, 3.
Reinforcing
Provide information and
updates on organizational
goals

Weekly at staff meetings 1
Provide information on
resources available to
support learning e.g. in-
school coaching and
professional development
Weekly and monthly at one
on ones. Also, via newsletter
2
Reminders for teachers to
reinforce goals and
resources available.

Create monthly reminders
(text, emails, Teacher
learning platform) about
goals and resources

1,2,3
Encouraging
Coaching for teachers and
administrators on
organizational goals
Monthly 1,2,3
Coaching for teachers on
strategies to improve
pedagogy and improve
connections with students.

Monthly 1,2,3
Rewarding
Provide rewards to teachers
that have students with
good or improved
attendance.

Monthly 1
Recognition of teachers that
are demonstrating success
with their students

Monthly 1,3
Incentivize administrators
that are recognizing
teachers making
improvements and
improving student
academic progress.
Monthly 1,3
81

Monitoring
Report outs about progress
for students and self-
reflection on pedagogy
Monthly 2
Results of report cards and
state exams
Yearly, quarterly 1,2,3

Organizational Support. Organizational support is critical to the success of training
programs. Schools that reinforce training on-site can expect as much as 85% of training material
to be applied in the classroom (Kirkpatrick & Kirkpatrick, 2016). Required drivers can be
strengthened in schools through the cultivation of small professional learning communities.
These can provide opportunities for both reinforcement and encouragement by creating regular
opportunities for teachers and administrators to share best practices and self-reported data about
classroom adoption and student achievement. Teacher evaluations provide formal approaches to
monitoring, encouraging, and rewarding teachers who have performed well and demonstrated
successful program adoption in the classroom. Additionally, progress can be monitored by
quarterly reviews of student learning, summative assessments, and measures of teacher
performance.
Level 2: Learning
According to Kirkpatrick and Kirkpatrick (2016), Level 2 outcomes describe the extent to
which training participants gain “knowledge, skills, attitude, confidence, and commitment” from
their participation in that training.
Learning Goals. The following learning goals summarize the learning expectations for
teachers participating in the Universal Algebra program:
1. Teachers will be able to apply knowledge of higher-level mathematics in their
classrooms. (Procedural knowledge)
82

2. Teachers will be able to explain the relationship between pedagogy, resources, and
student achievement. (Conceptual knowledge)
3. Teachers will be able to reflect on their pedagogy using journaling as well as peer-to-
peer sharing. (Metacognitive knowledge)
4. Teachers will be able to communicate that their involvement in the Universal Algebra
program has a positive impact on student achievement. (Expectancy- value)
5. Teachers will be able to analyze student performance to provide appropriate
scaffolding for improved student performance. (Procedural knowledge)
6. Teachers will be able to observe modeling and provide feedback on the teaching of
higher-level mathematics. (Self-efficacy)
Program. These learning goals will be accomplished by a program that combines
training for teachers with in-school coaching. The Universal Algebra program will should
provide training for teachers at school sites and provide a workshop series for teachers that
focuses on higher-level mathematics pedagogy. The program will begin with implementation for
Universal Algebra teachers and includes the implementation of new accountability structures for
teachers’ outcomes city wide.
The Universal Algebra program comprises a cohort model, with participants in the
program for five years. After five years, the teachers should be knowledgeable and confident
enough to teach higher-level mathematics to their students. Each cohort participates in an eight-
week summer intensive. The first cohorts should be teachers from schools and districts with
fewer teachers teaching mathematics and schools and districts which have demonstrated lower
achievement scores than the rest of the city. The summer intensive features six weeks of
professional learning and coaching by instructional specialists for teachers and includes a
83

curriculum designed to engage teachers deeply in the acquisition of the math fluency necessary
to engage students. Accordingly, a formative assessment of teacher skill level in mathematics
should be employed prior to the intensive to determine existing skill levels and tailor their
learning program to fill knowledge gaps. Following the intensive, a formative assessment is
conducted to determine the degree to which teacher pedagogy has improved and the areas in
which continued professional development is necessary.
Teacher learning should be reinforced via monthly professional learning sessions that
focus on their knowledge gaps. Teachers should also participate in monthly professional learning
community meetings of Universal Algebra teachers to reinforce their pedagogy and pursuit of
organizational goals by evaluating progress towards those goals, along with sharing best
practices. The program will also provide in-school coaching at the request of teachers or
principals based on their knowledge gaps and the needs of the school. Each school should also be
provided with a minimum of one instructional coach to provide regular on-site support and in-
classroom coaching to math teachers.
Evaluation of the Components of Learning. Although a significant component of
learning is the exhibition of declarative knowledge, it is critical to assess both declarative and
procedural knowledge acquisition, along with the value learners place on professional
development. Additionally, it is necessary to assess how confident learners are in employing
their knowledge and skills and how dedicated they are to applying them. Table 14 presents
evaluation methods and timing for different components of learning.

84

Table 15
Evaluation of the Components of Learning for the Program
Method(s) or Activity(ies) Timing
Declarative Knowledge
Knowledge checks using multiple choice

During professional learning sessions
Knowledge checks during discussions in
individual and group activities (role-plays,
pair share)

During teacher learning community
meetings
Procedural Skills

Use scenarios with multiple-choice items. During professional learning sessions
Demonstration in groups and individually of
job aid use

During teacher learning community
meetings and professional learning
sessions

Quality feedback from peers During teacher learning community
meetings and professional learning
sessions

Individual application of skills During teacher evaluations

Pre- and post-training assessment survey
regarding proficiency level

Before, during, and quarterly after
professional learning sessions and in-
school coaching
Attitude

Observations of teacher comments or actions
regarding the benefits of strategies presented

During professional learning sessions
Discussions of the value of the strategies that
have been presented

During and after professional
learning sessions
Pre- and post-evaluation items Before and after professional learning
sessions

Confidence
Survey questions After professional learning sessions

Discussions during or after feedback After professional learning sessions

Post-evaluation items After professional learning sessions

Commitment
85

Discussions and feedback During professional learning sessions

Action plans During professional learning sessions

Pre- and post-evaluation items Before and after professional learning
sessions

Level 1: Reaction
According to Kirkpatrick and Kirkpatrick (2016), reaction outcomes measure the degree
to which participants react favorably to the program, are satisfied with its content, and find it
relevant to their teaching. Table 15 illustrates the evaluation techniques that will be used to
measure how participants in the program’s professional learning sessions grow through that
professional learning. The table also specifies the timing of these methods in relation to the
sessions.
Table 16
Components to Measure Reactions to the Program
Method(s) Timing
Engagement
Attendance of professional
learning sessions

During the professional learning sessions
Asking meaningful questions

During the professional learning sessions
Completing scenarios

During the professional learning sessions
Participating in discussions During the professional learning sessions

Relevance
Survey items After every professional learning session
Course evaluation At the conclusion of the professional
learning series

86

Customer Satisfaction
Survey items After every professional learning session
Course evaluation At the conclusion of the professional
learning series

Evaluation Tools
Immediately Following Program Implementation. Kirkpatrick and Kirkpatrick (2016)
propose the “Blended Evaluation approach,” a multiple evaluation methodology that collects
data about the efficacy of a training program. This approach uses immediate and delayed
evaluation tools to measure the efficacy of professional learning (Kirkpatrick & Kirkpatrick,
2016). As recommended by Kirkpatrick and Kirkpatrick (2016) data will be gathered during the
series of professional learning sessions for teacher’s data and immediately after each of the
professional learning sessions. For Level 1, during each of the session’s facilitators will perform
recurring checks with teachers about relevance and will also observe their level of participation.
Data will be about the level of engagement that teachers evinced through their attendance and
participation during the professional learning sessions. In addition, surveys will be conducted
immediately after each session that will gauge the significance and satisfaction with each of the
sessions for Level 2. The learning of declarative and procedural knowledge will be examined
during each session directly with teachers participating in interactive portions of the sessions and
by utilizing provided models of teacher success and reference materials. Additionally, surveys
will be conducted following each session to measure acquisition of the declarative and
procedural knowledge content presented. Table 16 exemplifies content evaluation using the
observation and survey instruments for Level 1 and 2 outcomes.

87

Table 17
Immediate Evaluation Summary
Evaluation Level Evaluation tools content (Item description)
Level 1: Reaction
Engaged Teacher attended XX# of professional learning
sessions

Teacher participated in the professional
learning sessions

Relevant The coaching was relevant to my instruction.
The professional learning sessions were useful
to me in the classroom.

Satisfied The information was presented well.
I understood the strategies and pedagogy being
presented.
Level 2: Learning
Declarative
Knowledge of higher-level math terms and
standards
Knowledge of performance goals
Knowledge of services available for the
Universal Algebra program

Teachers know math terms and standards.
Teachers know school district performance
goals for students.
Teachers know what services are available via
the Universal Algebra program.
Procedural
Understanding how pedagogy relates to
students’ academic progress and post-
secondary readiness

I know how my pedagogy impacts my
students’ academic progress and post-
secondary readiness.
Teachers demonstrate that they can review
their assessments to indicate how their
pedagogy has impacted their students’ grades
and academic performance.
88

Understanding how academic progress is
related to student performance and career and
post-secondary readiness.
I know how my pedagogy is related to student
performance.
Teachers demonstrate that they can review
their own lesson plans to assess the quality of
their teaching.
I know my pedagogy is related to career and
post-secondary readiness.
Teachers demonstrate the ability to connect
pedagogy with career and post-secondary
choices.

Delayed For A Period After The Program Implementation. Approximately four
weeks after the conclusion of the professional learning sessions for teachers, program staff will
conduct a survey of scaled items to assess teacher perspectives on their satisfaction and the
relevance of the professional learning (Level 1), their confidence and the value in applying the
content (Level 2), their application of the content in the classroom and the support they have
received from facilitators and peers (Level 3), and the extent to which their performance has
become more effective in terms of content knowledge (Level 4). Table 17 exemplifies the types
of items that will be incorporated into the survey and evaluated by the survey instruments for
Level 1 through 4 outcomes.
Table 18
Delayed Evaluation Summary
Evaluation Level Evaluation Item
Level 1: Reaction What I learned in the professional learning sessions
has been valuable to my school performance and
success.
Level 2: Learning 1. I was able to learn higher-level math terms and
strategies.
89

2. I was able to learn performance metrics.
3. I was able to learn the value of higher-level
mathematics.
4. I was able to learn the connection between my
pedagogy and my students’ academic performance and
success.
5. I was able to learn the value of advanced
mathematics instruction for students’ future career and
post-secondary success.
Level 3: Behavior I use the worksheets and information sheets provided
during the professional learning sessions and in-school
coaching to organize and plan for my classroom
instruction.
Level 4: Results 1. I am more confident in my pedagogy and in my
content knowledge than before these professional
learning sessions.
2. I have set short-term goals for my performance.
3. I have set long-term performance and/or career
goals.

Data Analysis and Reporting
The Level 4 outcomes for teachers measure their regular attendance of Universal Algebra
professional learning opportunities and the number of times they access Universal Algebra
program services. Each month, program staff will track teachers participating in professional
learning and their in-school use of program services to produce a comparison with the previous
school year. The information will be collected by the school and by the program office to
examine progress, foster accountability, and showcase the effective implementation of
professional learning and service use across the city. Furthermore, monthly reports for the city
90

will be prepared that demonstrate the attendance at professional learning sessions and use of
program services for the teachers compared to the prior school year.
Figure 5
Sample Universal Algebra Data Dashboard

Summary of the Implementation and Evaluation Plan
The New World Kirkpatrick model (Kirkpatrick & Kirkpatrick, 2016) emphasized
embarking on planning while examining expected outcomes. This can effectively direct planning
to derive positive returns on expectations. The key idea in this implementation structure is its
uncomplicated concentration on outcomes from beginning to conclusion. For example,
incorporating evaluation into implementation can guide effective implementation in terms of
recognizable results. This technique is especially relevant for the field of education, where there
are various issues to discuss and many stakeholders. A well-defined goal and model for focusing
outcomes by connecting different implementation aspects, including evaluation, guarantees that
the material can be evaluated in terms of efficacy at the beginning of the process, throughout the
process, and after the process. Education institutes often enact multiple initiatives across entire
91

organizations without identifying tangible outcomes or enacting a strategic plan. In addition to
outcomes, this methodology enables the organization to evaluate implementation, obstacles, and
areas to be strengthened, which is critical because Universal Algebra teachers risk failure if they
are not afforded appropriate support or mechanisms for success.
Limitations and Delimitations
One limitation of this study is that data specific to study participants has not been
available or obtainable due to the COVID-19 pandemic. Given research was halted in
Metropolitan City, the researcher was limited to data concerning all mathematics teachers rather
than data specific to the Universal Algebra program. However, the outcomes discussed are not
designed to be generalizable to the entire population of mathematics teachers, instead applying
only to those pertaining to the program.
Another limitation is the study’s failure to address the impact of the COVID-19 pandemic
on the program’s outcomes. Although the program’s goals were expected to be met in 2022, the
program was defunded of all funding that was not personnel related. The impact of this decision
on students and teachers has not been considered and nor has student achievement in connection
to the shift to remote learning in the COVID-19 context. Additionally, neither the removal of
resources because of the program’s defunding nor the impact of that change has been addressed.
These changes have undeniably significantly impacted the program’s goals.
This evaluation’s final limitation concerns its use of secondary data. Secondary data
describe data not collected by the researcher but collected by another individual or group for a
different purpose. Notably, the secondary data used here are neither published nor peer-
reviewed, and the potential biases of the researcher who collected these data are unknown.
92

Additionally, the underlying assumptions made during the collection of the survey data are also
unknown.
This study has applied the delimitation of focusing on one group of stakeholders:
mathematics teachers. Neither students who had teachers in the program nor district- or school-
level leadership were included in this study. Teachers have been used as the stakeholder group of
interest because the program is focused on improving their pedagogy as the change lever for
student achievement in the classroom. However, effective change encompasses the action of
leadership in addition to teachers. Accordingly, understanding the level of support that the
program received from district and school leadership during the study period would have
provided further insight into the program’s success.
Future Research
Recommendations for future research on the Universal Algebra program should address
the allocation of funding at the school level in conjunction with student achievement. This would
determine whether annual funding has shifted to address the needs of more poorly performing
schools and ascertain whether achievement has improved as a result of these additional
resources. Future research should also utilize original data collected from teachers involved in
the Universal Algebra program. Ideally, data collection would include surveying multiple
stakeholders, including students, families, and administrators.
Future research should also examine the impact of funding and teacher recruitment on the
Universal Algebra program. The program aims to increase the number of teachers teaching
within their content area and provide funding and support towards achieving that goal. Future
research should examine whether the number of teachers teaching within their content area has
increased. It should also examine whether there have been recruitment efforts targeting math
93

teachers, particularly in the context of historically under-resourced communities. Future research
should also address the efficacy of vendors providing math support in classrooms.
Finally, future research should examine the program’s overall success in terms of
achieving equity, its stated core goal. This means inquiring as to whether the expansion of
Algebra I courses has been achieved in historically under-resourced districts and the impact of
this on student achievement. It also means considering how students performed if Algebra I
courses were offered. This information would allow districts to target interventions towards
schools, teachers, and students and enable performance improvements directed at meeting the
program’s goals.
Conclusion
This study, in concert with the whole-system reform theory of addressing systemic
inequities, emphasizes that the key to success system-wide is ensuring system goals are aligned
with participant motivation (Fullan, 2015). The recommendations made by this evaluation should
serve as an improvement roadmap for large urban school districts. Notably, this study’s findings
also indicate that performance improves when teachers feel prepared and empowered to teach in
their subject area, receive adequate resources, and are provided with meaningful professional
development. The researcher expects that by implementing these recommendations, student
performance should increase in tandem with teacher ability, ultimately improving instruction and
achievement overall.
Given that the Universal Algebra program has been developed to achieve its proposed
goals by meeting the pedagogical needs of teachers, it remains necessary to ask, simply, whether
the program is effective. Although myriad factors have impacted this program’s outcomes, the
program is likely to be effective if the underlying cultural influences can be addressed. Despite
94

the existence of uncontrollable variables that may affect outcomes, this optimism persists, with
the main concern being the consistency and constancy of citywide implementation. Differences
in the approaches of facilitators, principals, and in-school coaches can produce varying levels of
buy-in, comfort of teachers, participant experience and knowledge, and adherence to program
materials. However, it is likely that the program will be effective if school leaders are invested in
the program’s success and if the resources available are targeted at the communities that most
need them. Therefore, outreach to superintendents and principals is required before continued
implementation of the program to increase buy-in, support as many schools as possible, and
ensure that the schools in need of support are the subject of targeted interventions tailored to
their population.

95

References
Aguinis, H., & Kraiger, K. (2009). Benefits of training and development for individuals and
teams, organizations, and society. Annual Review of Psychology, 60, 451–474.
10.1146/annurev.psych.60.110707.163505.
Alexander, P. A., Schallert, D. L., & Reynolds, R. E. (2009). What is learning anyway? A
topographical perspective considered. Educational Psychologist, 44(3), 176–192.
Anderson, L., & Krathwohl, D. (2001). A taxonomy for learning, teaching, and assessing (1st
ed.). Longman.
Apex for Youth “UP FOR DEBATE: EXAMINING THE CONTROVERSY BEHIND THE
SHSAT.” 9 July 2018, www.apexforyouth.org/debate-examining-uproar-behind-
shsat/#:~:text=Eliminating%20the%20SHSAT%20means%20that%20families%20will%
20no,increase%20black%20and%20Latino%20representation%20to%2045%20percent.
Anonymous. Computers, Networks & Communications [Atlanta] (n.d.). Algebra; Miami-Dade
adopts Carnegie Learning  math programs for all 42 high schools. NewsRx.
http://search.proquest.com/docview/748918736/
Bandura, A. (1991). Social cognitive theory of self-regulation. Organizational Behavior and
Human Decision Processes, 50(24), 8–287.
Bandura, A. (2000). Exercise of human agency through collective efficacy. Current Directions in
Psychological Science, 9(3), 75–78.
Bogdan, R. C., & Bilken, S. K. (2007). Chapter 3: Fieldwork. In Qualitative research for
education: An introduction to theories and methods (5th ed.). Allyn and Bacon.
96

Bottia, M., Stearns, E., Mickelson, R., Moller, S., & Parker, A. (2015). The relationships among
high school STEM learning experiences and students’ intent to declare and declaration of
a STEM major in college. Teachers College Record, 117(3).
Brownell, W. A. (1935). Psychological considerations in the learning and teaching of arithmetic.
In W. D. Reeve (Ed.), The teaching of arithmetic. Tenth Yearbook of the National
Council of Teachers of Mathematics. Teachers College Press.
Clark, R. E., & Estes, F. (2008). Turning research into results: A guide to selecting the right
performance solutions. CEP Press.
Chatzistamatioum M., Dermitzaki, I. & Bagiatis, V. (2014). Self-regulatory teaching in
mathematics: relations to teachers’ motivation, affect and professional commitment.
European Journal of Psychology of Education, 29(2), 295–310
Daughtry, B., Akagi, C., & Strickland, J. (n.d.). Investigating Middle and High School Algebra 1
Teachers’ Technology Self-Efficacy, Training, Usage, and Commonalities. ProQuest
Dissertations Publishing. http://search.proquest.com/docview/2046766661/
Fullan, M., & Quinn, J. (2015). Coherence: The right drivers in action for schools, districts, and
systems. Corwin; Ontario Principals’ Council.
Fink, A. (2013). Chapter 4: Sampling. In How to conduct surveys: A step-by-step guide (5th ed.)
(p. 79–98). SAGE Publications.
Grossman, R., & Salas, E. (2011). The transfer of training: What really matters. International
Journal of Training and Development, 15(2), 103–120.
Hayward, S. (2010). Engaging employees through whole leadership. Strategic HR Review, 9(3),
11–17. http://doi.org/10.1108/14754391011040028.
97

Hiebert, J., & Grouws, D. A. (2007). The effects of mathematics teaching on students’ learning.
In F. Lester (Ed.), The second handbook of research on mathematics teaching and
learning. Information Age.
Karp, A., & Schubring, G. (2014). Handbook on the history of mathematics education. Springer
New York. http://doi.org/10.1007/978-1-4614-9155-2
Krathwohl, D. R. (2002). A revision of Bloom’s Taxonomy: An overview. Theory into Practice,
41(4), 212–218.
Maxwell, J.A. (2013). Qualitative research design: An interactive approach (3rd ed.). SAGE
Publications.
Mayer, R. E. (2011). Applying the science of learning. Pearson Education.
Mccall, B. (2015). Completing Algebra II in high school: Does It increase college access and
success? The Journal of Higher Education, 86(4), 628–662.
http://doi.org/10.1080/00221546.2015.11777377.
Merriam, S. B. & Tisdell, E. J. (2016). Qualitative research: A guide to design and
implementation (4th ed.). Jossey-Bass.
Metro-Craig, K., Curda, L., Jacobs, H., & Shriner, M. (n.d.). A Case study exploring at-risk high
school students’ perceptions of interactive feedback and supports in online Algebra I
credit recovery. ProQuest Dissertations Publishing.
http://search.proquest.com/docview/2054014359/
National Council on Mathematics (1989). Curriculum and evaluation standards. National
Council on Mathematics.
National Council on Mathematics (1991). Professional standards for teaching mathematics.
National Council on Mathematics.
98

National Council on Mathematics (2000). Principles and standards for school mathematics.
National Council on Mathematics.
National Council on Mathematics (2014). Principles to action: Ensuring mathematical success
for all. National Council on Mathematics
Nyschooltalk. “Do NYC School Zones Violate Federal Law?” New York School Talk, 26 May
2020, newyorkschooltalk.org/2020/05/do-nyc-school-zones-violate-federal-law/.

Patton, M.Q. (2002). Chapter 7: Qualitative interviewing. In Qualitative research & evaluation
methods (3rd ed.). SAGE Publications.
Reys, B., Reys, R., & Rubenstein, R. (2010). Mathematics curriculum: Issues, trends, and future
directions. National Council of Teachers of Mathematics.
Ross, A & Willson, V. (2012). The effects of representations, constructivist approaches, and
engagement on middle school students’ algebraic procedure and conceptual
understanding. School Science and Mathematic, 112 (2), 117-128
Rueda, R. (2011). The 3 dimensions of improving student performance. Teachers College Press.
Skemp, R. (2006). Relational and instrumental understanding. Mathematics Teaching in
The Middle School, 12(2), 88–95.
Showalter, D.A. (2017) To math or not to math: The algebra-calculus pipeline and postsecondary
mathematics remediation. The Journal of Experimental Education, 85(4), 674–688,
http://doi.org/10.1080/00220973.2017.1299080
Slavin, R., Lake, C., & Groff, C. (n.d.). Effective programs in middle and high school
mathematics: A best-evidence synthesis. Review of Educational Research, 79(2), 839–
911. http://doi.org/10.3102/0034654308330968
99

Vanderheyden, A., Mclaughlin, T., Algina, J., & Snyder, P. (n.d.). Randomized evaluation of a
supplemental grade-wide mathematics intervention. American Educational Research
Journal, 49(6), 1251–1284. http://doi.org/10.3102/0002831212462736
Wasserman, N. H. (2016) Abstract algebra for algebra teaching: Influencing school mathematics
instruction. Canadian Journal of Science, Mathematics and Technology Education,
16(1), 28–47, http://doi.org/10.1080/14926156.2015.1093200

Abstract (if available)

Abstract This study utilizes the Clark and Estes (2008) gap analysis framework as an evaluation and analytic tool to examine and identify performance gaps, program goals, and intended outcomes. The purpose of this evaluation study was to analyze how the gaps in knowledge, motivation, and organization influences impacted the efficacy of the Universal Algebra initiative. Secondary data, both direct data sources and contextual data sources, were used to verify the knowledge and skills, motivation, and organization influences and identify the causes of those gaps. The validated assumed influences for declarative factual knowledge, conceptual knowledge, procedural knowledge, metacognitive knowledge, self-efficacy, value, cultural models, and cultural settings were identified, and confirmed influences were analyzed. The results and findings illustrated that although the program had made significant progress, it failed to meet its performance goal of 100% of student taking Algebra I by the end of ninth grade. The study illustrates how that gap analysis framework can be used as an evaluation tool to identify and remediate gaps in programs to increase efficacy of those programs.

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

The academic implications of providing social emotional learning in K-12: an evaluation study

PDF

Factors impacting the effectiveness of mentor teachers in a national teacher residency

PDF

Practices supporting newcomer students

PDF

Gender diversity in optical communications and the role of professional societies: an evaluation study

PDF

A framework for customer reputation evaluation: an innovation study

PDF

Emerging adult peer provider specialists and successful college participation: An innovation study

PDF

Lean construction through craftspeople engagement: an evaluation study

PDF

Future educator programs in high schools: an evaluative study

PDF

Underutilization of employer-funded higher education: a gap analysis

PDF

One to one tablet integration in the mathematics classroom: an evaluation study of an international school in China

PDF

Employee standardization for interchangeability across states: an improvement study

PDF

Increasing organizational trust within financial services during times of change: an improvement study

PDF

Educator professional development for technology in the classroom: an evaluation study

PDF

Critical behaviors required for successful enterprise resource planning system implementation: an innovation study

PDF

An evaluation study: quality contextual professional development

PDF

Faculty’s influence on underrepresented minority (URM) undergraduate retention

PDF

Remote learning and parent engagement during a crisis

PDF

Narrowing the English learner achievement gap through teacher professional learning and cultural proficiency: an evaluation study

PDF

Millennial workforce retention program: an explanatory study

PDF

Child welfare turnover: an evaluation study of work-life balance practices

Asset Metadata

Creator Roman, Alicia R. (author)

Core Title Universal algebra: an evaluation

School Rossier School of Education

Degree Doctor of Education

Degree Program Organizational Change and Leadership (On Line)

Degree Conferral Date 2021-08

Publication Date 07/24/2021

Defense Date 07/21/2021

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag achievement,algebra,equity,OAI-PMH Harvest,under-resourced

Format application/pdf (imt)

Language English

Contributor Electronically uploaded by the author (provenance)

Advisor Yates, Kenneth (committee chair), Foulk, Suzanne (committee member), Mora-Flores, Eugenia (committee member)

Creator Email ardorsey@usc.edu

Permanent Link (DOI) https://doi.org/10.25549/usctheses-oUC15619736

Unique identifier UC15619736

Legacy Identifier etd-RomanAlici-9856

Document Type Dissertation

Format application/pdf (imt)

Rights Roman, Alicia R.

Type texts

Source University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Access Conditions The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the author, as the original true and official version of the work, but does not grant the reader permission to use the work if the desired use is covered by copyright. It is the author, as rights holder, who must provide use permission if such use is covered by copyright. The original signature page accompanying the original submission of the work to the USC Libraries is retained by the USC Libraries and a copy of it may be obtained by authorized requesters contacting the repository e-mail address given.

Repository Name University of Southern California Digital Library

Repository Location USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA

Repository Email cisadmin@lib.usc.edu