Close
About
FAQ
Home
Collections
Login
USC Login
Register
0
Selected
Invert selection
Deselect all
Deselect all
Click here to refresh results
Click here to refresh results
USC
/
Digital Library
/
University of Southern California Dissertations and Theses
/
How extending time in developmental math impacts persistence and success: evidence from a regression discontinuity in community colleges
(USC Thesis Other)
How extending time in developmental math impacts persistence and success: evidence from a regression discontinuity in community colleges
PDF
Download
Share
Open document
Flip pages
Contact Us
Contact Us
Copy asset link
Request this asset
Transcript (if available)
Content
HOW EXTENDING TIME IN DEVELOPMENTAL MATH IMPACTS STUDENT
PERSISTENCE AND SUCCESS:
EVIDENCE FROM A REGRESSION DISCONTINUITY IN COMMUNITY COLLEGES
by
Federick J. Ngo
Holly Kosiewicz
A Thesis Presented to the
FACULTY OF THE USC DORNSIFE COLLEGE OF LETTERS, ARTS & SCIENCES
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulfillment of the
Requirements for the Degree
MASTER OF ARTS IN ECONOMICS
(M.A. ECONOMICS)
DECEMBER 2015
EXTENDING TIME IN DEVELOPMENTAL MATH 2
Table of Contents
Acknowledgements............................................................................................................................................... 3
Abstract ..................................................................................................................................................................... 4
Introduction ............................................................................................................................................................ 5
Does Increasing Time in Math Help? .............................................................................................................. 7
Investment and Persistence in College Math Remediation .................................................................... 9
Setting ......................................................................................................................................................................12
Placement Into and Delivery of Developmental Math .............................................................................12
Extended Algebra Courses..............................................................................................................................13
Data ...........................................................................................................................................................................15
Outcomes .............................................................................................................................................................16
Empirical Strategy ...............................................................................................................................................20
Regression Discontinuity Design ...................................................................................................................20
Issues of Non-Compliance ...............................................................................................................................22
Fuzzy Regression Discontinuity Design .......................................................................................................24
Findings ...................................................................................................................................................................27
Enrolling in Math ..............................................................................................................................................30
Attempting and Passing Gatekeeper Courses ............................................................................................31
College Persistence ............................................................................................................................................33
Generalizability, Robustness, & Falsification Tests ................................................................................33
Sensitivity & Robustness .................................................................................................................................34
Discussion & Conclusion ...................................................................................................................................36
References ..............................................................................................................................................................39
Appendix .................................................................................................................................................................45
EXTENDING TIME IN DEVELOPMENTAL MATH 3
Acknowledgements
The authors wish to thank Professor Tatiana Melguizo in the Rossier School of Education for
access to the data, and for serving on the thesis committee with Professors Jeff Nugent and
Anant Nyshadham in the Department of Economics. We thank Kristen E. Fong and Ayesha
Hashim for comments on prior versions of the manuscript.
EXTENDING TIME IN DEVELOPMENTAL MATH 4
Abstract
Extending a one-semester math course over two semesters is thought to be beneficial for student
achievement since students have more time to master math concepts. We investigate the efficacy
of this practice in community college math remediation, where the cost to additional time may
outweigh the academic benefits and influence students’ persistence decisions. Drawing on data
from four large California community colleges, we use regression discontinuity to identify the
effect of taking two-semester extended algebra courses relative to typical single-semester
courses. We find enrolling in extended algebra significantly decreased student persistence and
success. Implications for developmental math reforms are discussed.
EXTENDING TIME IN DEVELOPMENTAL MATH 5
Introduction
Improving the outcomes of students in developmental or remedial math remains a puzzle
in higher education. Nationally, about 60 percent of incoming community college students are
deemed academically under-prepared for college-level math and referred to take remedial
coursework (Bailey, Jeong, & Cho, 2010). Of these students, only about 40 percent persist and
complete the gatekeeper college-level math courses typically required to earn postsecondary
credentials. The lower that students’ initial level of math remediation is the less likely they will
earn their desired credentials or transfer to four-year institutions (Bahr, 2012; Fong, Melguizo, &
Prather, 2015). While some of these outcomes can be explained by students’ academic
preparation, they also may be related to the amount of time students spend in remediation
(Edgecombe, 2011), to the additional costs of taking remedial courses (Melguizo, Hagedorn, &
Cypers, 2008), or to the quality of instruction in these courses (Grubb, 1999). The puzzle is that
the very intervention that is aimed at preparing students to be successful in gatekeeper college-
level courses may at the same time be an obstacle and deterrent to their persistence in college.
Concerns with low persistence and completion rates have motivated proponents of reform
to reconsider the delivery of developmental math (Burdman, 2012; Rutschow & Schneider,
2011). Moving away from the traditional sequence of multiple semester-long courses taught in
lecture-based formats, practitioners and policy-makers have explored alternative models of
delivery that accelerate student progress, contextualize curriculum and instruction, or provide
additional supports to students in developmental courses (Rutschow & Schneider, 2011). But in
an examination of math course offerings in a set of urban community colleges in California,
Kosiewicz, Ngo, and Fong (2014) found that a far more prevalent model of delivery was to
extend time in remediation, essentially slowing down the delivery of math content. Specifically,
EXTENDING TIME IN DEVELOPMENTAL MATH 6
some of the colleges implemented placement policies that assigned students to either an extended
two-semester
1
or a traditional one-semester elementary algebra course that serve as prerequisites
for college-level math. Both types of courses start with roughly the same content from day one
but the extended algebra courses move at a slower pace and force students to make two
enrollment decisions: one to enroll in the first half of the course, and another to enroll in the
second.
Lengthening the amount of time in math in this way is thought to be an intervention that
improves academic achievement. The underlying logic is that lower-skilled students may need
more time and instruction to master necessary algebraic concepts (Aronson, Zimmerman, &
Carlos, 1999). However, while there is some evidence that increasing the amount of instructional
time in algebra benefits middle and high school students (Cortes, Goodman, & Nomi, 2015;
Taylor, 2014), it is unclear whether this practice is beneficial to community college students, for
whom the additional costs in terms of time and money may outweigh the benefits of remediation.
Does extending time in algebra by a semester help community college students persist and
succeed in developmental math and college?
We investigate this research question using data from four large urban community
colleges in California. The nature of math course placement policies within these colleges
provides the opportunity to use a regression discontinuity (RD) design to estimate the impact of
assigning students to extended math courses relative to single-semester courses on student
achievement outcomes. While other researchers have used RD designs in the setting of
developmental math to identify the effects of placement in remediation, they have predominantly
examined placement into disparate math courses (e.g., the effects of placement in elementary
versus intermediate algebra on student outcomes). Thus it is difficult to disentangle whether
1
Here we use extended elementary algebra synonymously with two-semester elementary algebra
EXTENDING TIME IN DEVELOPMENTAL MATH 7
differences in outcomes are attributable to differences in time spent in remediation or to the
academic preparation within remedial courses. This study is unique because we are able to focus
on the effects of requiring students to spend additional time in math remediation. Students
around the placement cutoffs are considered statistically identical, but one group must take two
semesters of algebra instead of one.
This paper proceeds as follows. We first review the literature on increasing time in
algebra, drawing largely from studies of middle and high school settings. We then introduce the
literature on student persistence decisions in the community college setting and highlight the
ways it interacts with developmental math reforms. We next describe the data and the
methodological approach – regression discontinuity design – that we used to estimate the effects
of enrollment in the extended algebra sequence, focusing on four California community colleges
that assign students who earn lower placement scores to an extended two-semester elementary
algebra course instead of a typical one-semester course. Results from this study suggest that
extending the amount of time in algebra and adding the need for an extra persistence decision is
more harmful than beneficial for students at the margin of the cutoff. These students are much
less likely to enroll in and complete gatekeeper courses and persist towards credential attainment.
We conclude with a discussion of how developmental math reforms can increase persistence and
success for community college students.
Does Increasing Time in Math Help?
Increasing instructional time is not a new idea, as several states have implemented
reforms to either increase instructional minutes in classes or extend the school year (Aronson et
al., 1999; Patall, Cooper, & Harris, 2010). At the core of these initiatives is the notion that
allocating more time to instruction will increase the amount of time students can actively
EXTENDING TIME IN DEVELOPMENTAL MATH 8
participate in classroom exercises and learn course material. With additional time, teachers may
feel less hurried and have more time to cover material in more depth (Aronson et al., 1999; Patall
et al., 2010).
Most of the evidence on the advantages of increasing time in math stems from middle
and high school settings and focuses on algebra, a gatekeeper course that can be a significant
barrier to overall success in school (Cortes et al., 2015; Stein, Kauffman, Sherman, & Hillen,
2011). This evidence suggests that increasing time in math is beneficial for middle and high
school students and improves student achievement. Taylor (2014) examined a policy in Miami-
Dade County schools and found that increasing math instruction by requiring that students
concurrently take two courses instead of one had positive effects on the math achievement of
middle school students. Similarly, Cortes et al. (2015) studied the Chicago Public School system,
which implemented an algebra policy that required students scoring below the national median
on an 8
th
grade math test to take two periods of freshman algebra instead of the typical one-
period course. Students affected by this “double-dose” policy were taught approximately 45
minutes of lectured-based instruction in algebra alongside non-affected students, and received
an additional 45 minutes of non-course based instruction where they worked in small groups,
engaged with course-based materials absent from textbooks, and focused more on concepts they
had difficulty mastering (Nomi & Allensworth, 2009). Evaluations of this double dose of algebra
demonstrate that it largely generated positive effects on student achievement. Employing a
regression-discontinuity framework, these researchers found that the double dose algebra policy
increased math test scores, closed the black-white achievement gap on the math portion of the
ACT by fifteen percent, and improved high school graduation and college enrollment rates
(Cortes et al., 2015). However, because the double dose algebra policy not only doubled
EXTENDING TIME IN DEVELOPMENTAL MATH 9
instructional time but also provided professional development training and greater instructional
flexibility to teachers, Cortes and colleagues acknowledge that it is difficult to disentangle which
mechanisms improved academic performance. The likelihood that extending instructional time
alone boosted achievement falls flat when considering the body of literature that consistently
reveals little or no correlation between allocated time and achievement (Aronson et al., 1999).
Investment and Persistence in College Math Remediation
Whether the positive benefits produced by doubling the dose of algebra in middle and
high school actually translate into similar benefits at the college-level is difficult to determine,
mainly because college students have more autonomy than high school students in making
educational investment decisions. Postsecondary students can choose to enroll or not to enroll in
the courses that are offered to help them succeed. From an economic perspective, these decisions
can be considered as investments in human capital (Becker, 1964), and involve a host of factors
such as intended course of study and career goals, the likelihood of completion, total costs and
debt value, and expected lifetime earnings (Oreopoulos & Petrojinevic, 2013). More specifically,
Paulsen (2001) characterizes the choice to enroll in a course as a persistence decision that is
related to the expected return on human capital investment. Therefore, when examining college
student choices, it is not only important to consider achievement outcomes in higher education
such as credentials obtained or subsequent earnings, but also measures of persistence from
semester to semester, which are also indicative of students’ human capital investments.
A potential problem with the practice of extending math from one semester to two –
which we note is different from the double dose algebra policies described above – is that the
additional time requirement may influence students’ persistence decisions. Rather than doubling
the amount of time within one school day or semester, this practice of extending courses
EXTENDING TIME IN DEVELOPMENTAL MATH 10
inherently increases the number of points at which a student can exit the developmental math
sequence. The first human capital investment decision is to enroll in the two-semester course
sequence knowing that the road will be longer and the costs higher. Students face a second
persistence decision after completing the first course in the two-course sequence where they
must again consider the costs of an additional semester of math remediation.
Evidence shows that community college students may be sensitive to policies that
lengthen the amount of time they need to spend to reach their academic goals. In fact, students
who are assigned to longer developmental course sequences have the toughest time obtaining an
associate’s degree or meeting the requirements for upward transfer (Bailey et al., 2010; Crisp &
Delgado, 2014; Fong et al., 2015). While this may reflect rational decisions based on economic
choice models, it is important to note that student persistence can also be understood through
sociological and psychological lenses (Melguizo, 2011). Sociological theories would highlight
institutional practices and uncover the roles of academic, social, and cultural capital in
explaining patterns of student achievement (Bourdieu, 1986). For example, it may be that
students in developmental math courses systematically fail to receive the high quality instruction
and support services necessary to keep them committed to pursuing their goals, and that
academic environments are constructed in ways to reproduce existing inequalities.
Psychological theories might suggest that persistence is heavily influenced by non-cognitive
abilities such as a student’s level of motivation, sense of self-efficacy, and preference for long-
term goals. The concept of self-efficacy, for example, would suggest that people choose actions
that they determine to be consistent with their perceived abilities (Bandura, 1995). It is possible
that placement exam results could shape students’ perceptions of their abilities and thus
influence college persistence decisions (Deil-Amen & Tevis, 2010). Drawing from these
EXTENDING TIME IN DEVELOPMENTAL MATH 11
frameworks, theories of college student persistence have underscored the important roles of such
factors as social and academic integration (Tinto, 1993), support systems (e.g., parents and
financial aid) (Crisp & Nora, 2009), high quality instruction (Pascarella, Salisbury, & Blaich,
2011), and validating interactions with faculty (Barnett, 2011; Rendon, 1994) in influencing
college student persistence.
While these sociological and psychological theories provide important frameworks for
understanding college student persistence, the practice of extending math remediation over two
semesters inherently increases costs and thus can be characterized as a human capital investment
decision (Paulsen, 2001). We therefore frame our study using human capital theory, which
suggests that students fail to persist because the direct costs (i.e., tuition, books, fees) and the
indirect costs (i.e., opportunity cost) of taking an additional math course far outweigh the private
returns that results from investing in that course, or that the desired goal feels too far off and
unattainable and therefore not worth the investment (Manski & Wise, 1983; Paulsen, 2001). A
policy that lengthens time in developmental math may thus deter students from persisting and
succeeding in college, even if it promises to build student skill and knowledge.
As we will describe next, we can specifically test how increasing the cost of remediation
in terms of time and money affects student persistence in community colleges. Our study
contributes empirical evidence to the literature by examining the effects of extending
developmental math from one to two semesters. While other empirical studies have evaluated
alternative models of delivery such as acceleration, student success courses, learning
communities, and supplemental instruction, our study examines the impact of a far more
prevalent model of delivery in some California community colleges, which may also exist in
other community colleges across the country. Although extending time in developmental math
EXTENDING TIME IN DEVELOPMENTAL MATH 12
may provide academic benefits to students, a human capital framework and theories of college
student persistence posit that the additional time students must spend in math remediation may
also deter or discourage students from persisting towards their academic goals. Therefore,
determining the achievement and persistence impacts of extending time in math remediation can
provide important direction for community colleges and their delivery of developmental math.
Setting
The setting for the study is four large urban community colleges in a large metropolitan
area of California. About one-quarter of all community college students in the U.S. are enrolled
in California community colleges, many of which are in located in urban centers (Foundation for
Community Colleges, n.d.). Being open-access institutions, these four community colleges serve
a widely diverse body of students, with more than a quarter of students over 35 years of age, and
over 40 percent indicating that their native language is not English. Close to 90 percent of
students report having completed a high school level education.
2
This student population is
different from the national community college student population since about two-thirds of all
students in these California colleges identify as African-American or Latina/o. In contrast, the
majority of students who enter a community college in the U.S. are White, and just over one-
third identify as African-American or Latino (NCES, 2014).
Placement Into and Delivery of Developmental Math
Students with the goal of earning an associate’s degree or transferring to a four-year
university in these four colleges must take placement tests in math and English to determine the
courses in which they should enroll. These courses are often of a remedial or developmental
nature and designed to prepare students to be successful in college-level courses. The
2
Source: California Community College Chancellor’s Office DataMart (http://datamart.cccco.edu/datamart.aspx)
EXTENDING TIME IN DEVELOPMENTAL MATH 13
developmental math sequence can include as many as four or five courses, and typically follows
the progression of arithmetic, pre-algebra, elementary algebra, and intermediate algebra.
Students must pass these latter two courses to be considered eligible for an associate’s degree
3
or
to transfer
4
to the California State University or University of California four-year systems,
respectively. Due to the decentralized nature of California community college governance,
colleges have considerable discretion in determining how students are placed into the
developmental math sequence and how the sequence is taught. This includes autonomy in
selecting placement instruments
5
(e.g., ACCUPLACER; COMPASS), setting cut scores,
choosing additional measures to incorporate into placement decisions (i.e., multiple measures
such as high school GPA or prior achievement), as well as flexibility in designing the courses
themselves (Melguizo, Kosiewicz, Prather, & Bos, 2014; Perry, Bahr, Rosin, & Woodward,
2010).
Extended Algebra Courses
Despite some variation in choice of placement test and associated cutoffs, the four
colleges that we focus on in the study deliver developmental math in the same way. They have
veered from the typical developmental math sequence described above by lengthening the
amount of time some students spend in elementary algebra from one to two semesters and
assigning students to these courses based on placement test results. Figure 1 portrays this
extended course sequence and Table 1 shows the cut scores used to determine whether students
are assigned to one or two semesters of elementary algebra based on placement test results.
According to our examination of descriptions in course schedules from each college, as well as
those available from math department websites, students in either type of course are expected to
3
In 2009, California increased the minimum math requirement for an Associate’s degree from elementary algebra to intermediate algebra.
4
While the State of California determines the academic requirements for the associate’s degree, two- and four-year institutions establish local
articulation agreements that determine which math courses transfer and which do not.
5
California is moving towards a common placement system for community colleges (See Burdman, 2015).
EXTENDING TIME IN DEVELOPMENTAL MATH 14
cover roughly the same content from day one. Both types of algebra courses typically begin with
a review of pre-algebra topics (e.g., algebraic expressions, exponents), move on to linear
equations, and conclude with quadratic functions. The only difference is that students in
extended algebra courses cover these topics over the duration of two semesters instead of one.
Figure 1. Delivery of Developmental Math in LUCCD.
Table 1. Cut scores differentiating placement into two versus one semester math courses
Two Semesters One Semester
Two Semesters One Semester
ACCUPLACER AR Subtest
ACCUPLACER EA Subtest
College A
50<=PS<65 65<=PS<76
College B PS>73.5 -
35.5<=PS<45.5 45.5<=PS<60.5
College C 100<=PS<113 113<=PS<=120
20<=PS<34 34<=PS<77
COMPASS PA Subtest
COMPASS Algebra Subtest
Two Semesters One Semester
Two Semesters One Semester
College D 40<=PS<49 PS>=49 1<=PS<=19 21<=PS<=40
Note: PS denotes the student's adjusted placement score.
Although Table 1 shows that these colleges use different cut scores and placement tests to
sort students into the regular versus extended elementary algebra course, a common
EXTENDING TIME IN DEVELOPMENTAL MATH 15
characteristic of these placement policies is that assignment to either course can be determined
by a single point. For example, students in College B who obtained a placement score of 45 on
ACCUPLACER’s Elementary Algebra subtest would have been placed into a two-semester
elementary algebra course. Had those students obtained an additional point on their placement
score, they would have been subsequently placed into the typical one-semester elementary
algebra course. In the next section, we describe the data used for the study and argue that the way
in which these placement policies are implemented creates a natural experiment that can be
exploited to estimate the causal effects of placement into a two- versus one-semester elementary
algebra course on academic achievement.
Data
We utilized student-level administrative records that link students’ demographic data,
assessment records, course enrollment, and academic outcomes. The sample from the four
colleges included student cohorts assessed for math for the first time between the spring 2005
and the spring 2012 semesters. We tracked enrollment and performance outcomes for the 2005-
2008 cohorts through spring 2012, and the 2008-2012 cohorts through fall 2013.
6
This study
window was chosen to include only those cohorts that were subject to placement in an extended
algebra course and those where no other placement policy changes occurred. This is the 2009-
2012 cohorts in College A, the 2005-2011 cohorts in Colleges B and C, and all cohorts in
College D. To create a sample that most closely resembled typical degree-seeking community
college students, we excluded students who were concurrently enrolled in high school, over the
age of 65, or had already received an Associate’s or Bachelor’s degree at the time of testing.
6
The last cohort in the sample consists of those assessed in spring 2012. We can observe their enrollment outcomes for spring and fall of 2012,
and spring and fall of 2013. We also included any summer or winter quarter enrollments.
EXTENDING TIME IN DEVELOPMENTAL MATH 16
Table 2 presents summary statistics for the students placed into extended versus one-
semester elementary algebra course in the four colleges. The pooled sample from Colleges A-D
includes 12,805 students, of which 6,228 (48.6 percent) were placed in an extended course
sequence based on their test score. Because we used a regression discontinuity framework for
this analysis (described further below), we show statistics for the full sample alongside those for
a restricted sample of students who scored within a one, half, third, and quarter standard
deviation (SD) test score bandwidth from the placement cutoff in each college. While there are
some differences in group means for demographics and outcomes between those placed in two-
and one-semester algebra, the samples are more similar within narrower bandwidths around the
placement cutoffs.
Outcomes
We also present sample mean outcomes at the bottom of Table 2. The short-term
outcomes of interest are whether the student: 1) enrolled in any math course within a year after
taking the math assessment, 2) passed elementary algebra (i.e., completing the elementary
algebra sequence), 3) attempted
7
the next course in the sequence, intermediate algebra, and 4)
passed intermediate algebra. We note that we inputted zeroes for these short-term outcomes. If,
for example, a student attempted and passed elementary algebra but decided not to enroll in
intermediate algebra, then they were assigned a value of zero for both attempting and passing
intermediate algebra. We reasoned that the goal of offering developmental math courses and
assigning students to them via placement policies is to prepare students to pass these upper-level
math courses, and if students do not progress, that is largely a consequence of the initial
placement decision. Also, while students who earn a C in a developmental math course are
7
An attempt is defined as an enrollment past the no-penalty drop date, after which the student would receive a mark on his/her transcript. The no
penalty drop date is usually about 1-2 weeks after classes begin.
EXTENDING TIME IN DEVELOPMENTAL MATH 17
allowed to enroll in the next course, we also examined the “B or better” outcome because it is a
less contentious evaluative outcome. It is more likely that a student just below the cutoff and
assigned to extended algebra could have earned a C in one-semester algebra, but it presumably
would be more difficult for that same student to earn a B. In addition, research has shown that
placement tests are more accurate at identifying students who would earn a B or better versus
some lower criterion (Scott-Clayton et al., 2014). Thus our estimate using the B or better
criterion may be more related to course placement than to placement test accuracy.
The data also enabled us to examine longer-term outcomes of interest in the community
college setting. We calculated the total units completed within one year of the assessment, as
well as total degree-applicable units completed through spring 2012 for the 2005-2008
assessment cohorts, and through fall 2013 for the 2009-2012 cohorts. This means that we could
track student degree-progress for up to seven years for the 2005 cohort, and one and a half years
for the 2012 cohort. While we could not observe whether or not students actually earn a degree
or transfer to a four-year college, the number of degree-applicable units completed served as an
indicator of progress towards these goals.
EXTENDING TIME IN DEVELOPMENTAL MATH 18
Overall, the descriptive results show low rates of persistence and completion for students
placed into two-semester or one-semester algebra. Only 26 percent go on to attempt intermediate
algebra, and only about 18 percent complete the course with a C or better (which is the minimum
for an associate’s degree and as a prerequisite for some STEM courses). Only 9 percent earn a B
or better. When disaggregating these results by initial placement, we observe that 19 percent of
students assigned to extended algebra ever attempted IA and 7 percent passed it with a B or
Table 2. Summary statistics, students placed in extended versus one-semester elementary algebra
Full Sample of Students Bandwidth Around Cutoff
Total Extended
One-
Sem.
1SD .5SD .33SD .25SD
Student Characteristics
Age 22.9 23.2 22.6
22.4 22.6 22.7 22.6
Female .503 .513 .493
.505 .511 .505 .500
Asian/PI .087 .084 .089
.090 .090 .088 .089
Black .193 .211 .176
.173 .180 .200 .187
Latina/o .480 .479 .481
.502 .504 .496 .501
White .151 .140 .161
.150 .144 .132 .137
Other .088 .085 .092
.086 .083 .084 .086
English not primary language .260 .257 .262
.274 .274 .270 .276
Permanent Resident .099 .098 .099
.101 .099 .095 .098
Other Visas .080 .074 .084
.084 .083 .085 .084
Outcomes
Enrolled in Any Course .854 .839 .869
.855 .850 .846 .845
Enrolled in Any Math Course .620 .575 .662
.620 .607 .592 .599
Attempted EA .481 .337 .618
.481 .475 .466 .477
Passed EA with a C or better .320 .224 .410
.320 .310 .295 .303
Passed EA with a B or better .182 .125 .237
.182 .176 .164 .164
Attempted IA .260 .191 .326
.261 .250 .237 .239
Passed IA with a C or better .178 .133 .222
.180 .172 .163 .168
Passed IA with a B or better .094 .072 .114
.095 .090 .087 .090
Total units w/in one year 11.1 10.1 12.0
11.1 11.0 10.9 11.1
Total degree-applicable units 21.6 19.4 23.7
21.8 21.5 21.3 21.7
N 12805 6228 6577 11425 7545 5387 4114
EXTENDING TIME IN DEVELOPMENTAL MATH 19
better, whereas about 33 and 11 percent of students placed directly in one-semester algebra
achieved these outcomes.
Conditioning student persistence outcomes on enrollment (i.e., without inputting zeroes
for unobserved outcomes) presents a different but equally important picture of student
progression (Fong et al., 2015). In Figure 2, we present progression rates conditional on
attempting each course. Doing so enables us to examine persistence decisions (attempting
courses) and completion rates (passing courses) for students who chose to progress through the
math sequence. We see that at the first persistence decision point, students placed in extended
algebra courses are less likely than students placed in one-semester algebra to attempt the first
course (71 versus 89 percent). Yet surprisingly, after one semester of coursework these students
have a lower rate of attrition (19 percent) than one-semester algebra students (27 percent).
However, the third persistence decision results in another decrease in cohort size. Of those who
pass the second course in the extended algebra sequence, 23 percent choose not to enroll in
intermediate algebra, resulting in just 13 percent of the original sample completing the
developmental math sequence, compared to 30 percent students initially placed in one-semester
algebra. While this is suggestive of a problem with extended algebra, these progression and
completion rates are conditional on attempting the course and may reflect the role of unobserved
factors such as student motivation and effort. We next describe the empirical strategy we used to
account for these unobservable factors and to isolate the effects of extended algebra courses on
the persistence and completion outcomes of the students in our sample.
EXTENDING TIME IN DEVELOPMENTAL MATH 20
Figure 2. Progression through Algebra Sequence
Empirical Strategy
Regression Discontinuity Design
Each college in the study utilizes a placement policy where a cutoff score differentiates
assignment into two-semester versus one-semester algebra so we were able to use a regression
discontinuity (RD) approach that exploits the exogenous variation in student outcomes resulting
from initial course assignment. This enabled us to estimate the causal effect of placement into
two-semester algebra relative to one-semester algebra on student outcomes. Because the RD
approach assumes baseline equivalency for students at the margin of a cutoff score, differences
in outcomes between groups can be attributed directly to assignment to extended algebra courses.
A basic model for the RD design is:
(1)
EXTENDING TIME IN DEVELOPMENTAL MATH 21
where Y
i
are the outcomes of interest, β
1
is our coefficient of interest and captures the estimated
treatment effect of assignment to extended elementary algebra. In order to pool the four colleges
into one analytical sample, we normalized each student’s test score around the mean test score
for each college and assessment cohort. We then centered this around the standardized cutoff
value for each college. Thus while the raw test scores may be different across colleges, the
transformed test score enables comparison across colleges. β
2
describes the relationship between
the test score running variable and student outcomes, and β
3
describes the two-way interaction
between assignment to extended algebra and test scores. Since RD estimation hinges on correct
specification of functional form, we also tested higher-order polynomials of the running variable
to account for the possibility of non-linearity (Lee & Lemieux, 2010). The coefficients within γ’
capture the influence of the following student-level demographic variables on the outcomes of
interest: age at assessment, sex, race/ethnicity; primary language, and residence/visa status.
These were gathered from student records. Although covariates are not necessary in RD design
since discontinuities in outcomes should only be correlated with treatment status, including them
can increase precision since they may explain some of the variation in the outcome measures.
Finally, since there may be unobservable differences across campuses and cohorts that
may be correlated with student outcomes, we included fixed effects by campus μ
j
and assessment
cohort φ
t
that enable us to account for systematic differences between colleges or cohorts. For
example, if College A had more experienced faculty than College B or had other unobserved
characteristics that influenced student outcomes, then this would potentially bias the RD
treatment effect estimate. Including a campus fixed effect in this case would account for time-
invariant factors associated with student outcomes on each campus and enable us to identify the
EXTENDING TIME IN DEVELOPMENTAL MATH 22
RD estimate within each college.
8
The average of these RD estimates gives the overall estimate
of the treatment effect.
Issues of Non-Compliance
The estimate for β
1
obtained from equation (1) above would be unbiased if there was
perfect compliance with placement results, that is, all students who were assigned to extended
algebra actually enrolled in it. However, we did observe a group of non-compliers – students
who simply did not enroll, and of students who enrolled, those who did not comply with their
placement decision. Table 3 presents overall enrollment and compliance rates for the full sample.
About 85 percent of students enrolled in any course in any department and 62 percent enrolled in
a math course. It is important to note that students assigned to extended algebra in these colleges
were about three percentage points less likely to enroll in any course (p<.001), and about nine
percentage points (p<.001) less likely to enroll in a math course after the assessment compared
with students assigned to a typical one semester course. This is consistent with findings
indicating that students assigned to the lower levels of the developmental math sequence are less
likely to “show up” after the placement test (Bailey et al., 2010; Fong et al., 2015).
8
The same logic applies for a cohort fixed effect.
EXTENDING TIME IN DEVELOPMENTAL MATH 23
Table 3. Enrollment and Compliance
Full range around cutoff 0.5 SD Bandwidth around cutoff
All Extended
One-
semester Difference
All Extended
One-
Semester Difference
Enrolled 0.85 0.84 0.87 -.031***
0.85 0.84 0.87 -.030***
Enrolled in Math 0.62 0.58 0.66 -.087***
0.61 0.56 0.65 -.090***
N 12805 6228 6577 7545 3882 3663
Complied with
Placement 0.81 0.71 0.89 -.180***
0.80 0.69 0.90 -.213***
First Math Course
Arithmetic 0.02 0.02 0.01
0.02 0.02 0.01
Pre-algebra 0.05 0.07 0.03
0.04 0.06 0.02
Extended algebra 0.34 0.71 0.04
0.35 0.69 0.03
One-sem. algebra 0.57 0.17 0.89
0.56 0.19 0.90
Int. algebra 0.02 0.02 0.03
0.02 0.02 0.02
Transfer-level 0.01 0.01 0.01
0.01 0.01 0.01
N 7917 3570 4347 4571 2184 2387
Note: *** p<.001, **p<.01, *p<.05
Figure 3. Enrollment in Extended Algebra Courses, 4 Colleges
Figure 3 also reveals imperfect compliance around the cutoff, and we attribute this
mainly to a California community college policy that allows students to challenge their
EXTENDING TIME IN DEVELOPMENTAL MATH 24
placement results. Doing so generally involves a process of providing evidence of prior math
achievement and obtaining permission to enroll in an alternative course from an instructor.
Overall, we observe that about 81 percent of students placed in one- or two-semester algebra
subsequently enrolled complied with their placement. However, the disaggregated rates are
different between groups. Among students assigned to extended algebra, 71 percent complied
with their placement, and about 20 percent enrolled in a higher-level course (e.g., one-semester
algebra, intermediate algebra, or transfer-level math). Among students assigned to one-semester
algebra, nearly 90 percent complied with their placement and four percent enrolled in a higher-
level course.
These differences in compliance pose a threat to the internal validity of the RD estimate
outlined in equation (1). Compliers may be different from non-compliers along observable
characteristics such as race or language status, or unobservable characteristics such as effort or
motivation. If these variables predict non-compliance, then they may also be correlated with the
outcomes of interest and result in biased estimates of the effect of extended algebra courses. We
discuss a strategy to remedy this problem in the next section.
Fuzzy Regression Discontinuity Design
Instrumental variables (IV) estimation can be used in scenarios where there is imperfect
compliance with the assignment to treatment, and in the RD setting this is commonly referred to
as “fuzzy” RD design (Murnane & Willett, 2010). Here, the endogenous variable is enrollment in
the extended algebra course (i.e., compliance with the placement rule), and the exogenous
instrument is assignment to or placement into extended algebra based on the placement cutoff.
The key assumptions underlying the IV strategy are that the instrument, in this case assignment
to extended algebra, strongly predicts treatment compliance, and that assignment is only
EXTENDING TIME IN DEVELOPMENTAL MATH 25
correlated with the outcomes of interest through compliance with the treatment assignment
(Murnane & Willett, 2010). Below, we outline the two-stage least squares estimation and discuss
tests of the instrument below.
In the first stage of the fuzzy RD, we predicted compliance with the treatment (i.e.
enrollment in extended algebra) using the same predictors as the model above:
(2)
We used the estimated coefficients to obtain the predicted probabilities of compliance with the
treatment. The predicted probability of compliance serves as the regressor of interest in the
second stage regression. Per the recommendation of Murnane and Willett (2010), we included
the same baseline covariates from the first stage in the second stage model shown here:
(3)
Here, β
1
is the local average treatment effect (LATE) of complying with placement in an
extended algebra course on student success outcomes. In other words, β
1
approximates the
expected impact of being placed into extended algebra on academic achievement. Before
presenting the results of analyses using this IV estimation method in a RD setting, we discuss
two checks we performed to test key assumptions underlying the validity of the RD approach.
Manipulation of the running variable. A requirement for obtaining internally valid
estimates in the RD setting is that some running variable, here the test score, discontinuously
assigns students to the treatment and control conditions based on an exogenously determined
cutoff and students cannot manipulate their scores or treatment status (Murnane & Willett,
2010). One way this exogeneity assumption might be invalidated is through retesting, since
EXTENDING TIME IN DEVELOPMENTAL MATH 26
students may be able to repeat tests in order to achieve a certain score. While retesting may be a
cause for concern in other testing contexts, students in the LUCCD typically are not allowed to
retest until after one year from the original assessment date and cutoffs are not made public so
students would not know what score they would need to attain. Furthermore, both tests used for
this analysis, the ACCUPLACER and COMPASS, are computer-adaptive placement tests that
would be difficult to manipulate to attain scores just above or just below the cutoffs.
9
We
therefore include students who retested in our analyses.
10
We also ran all models with retesters
excluded and the results we obtained were of very similar magnitude and statistical significance
compared to those presented below.
Covariate balance and continuity. A second key assumption underlying the internal
validity of the RD design concerns the continuity of other covariates along the score distribution.
If other covariates vary discontinuously at the test score cutoff, then the estimate of the treatment
effect may be biased (Murnane & Willett, 2010). We visually examined this by plotting trends in
covariates around the placement cutoffs using a local polynomial smoothing function (see
Appendix). Following the recommendations of Lee and Lemieux (2010), we also conducted a set
of parallel RD analyses to estimate discontinuities in observable covariates (e.g., age at
assessment, sex, race/ethnicity; primary language, and residence/visa status). The results, which
are available in the Appendix, indicate that there are no discontinuities in covariates around the
placement cutoff.
11
With these internal validity checks completed, we proceed with presentation
of the main RD findings.
9
Nevertheless, if students were aware of the test score cutoff and manipulated their scores in order to surpass it, or alternatively, if colleges
utilized assessment practices that resulted in non-smooth distributions of test scores, then this would threaten the validity of the RD estimates.
We therefore conducted McCrary (2008) density tests to examine manipulation of the test score running variable around the treatment cutoff in
each college and provide these results and corresponding figures in the Appendix. There is no evidence of manipulation in any college at the
cutoff for assignment to extended courses.
10
Retesters constitute about 8 percent of the sample
11
There is some indication of a discontinuity in the age variable. While this may provide a threat to the internal validity of the RD estimate, it is
possible that a discontinuity may have been estimated by chance. To test this, Lee and Lemieux (2010) suggest computing a set of Seemingly
EXTENDING TIME IN DEVELOPMENTAL MATH 27
Findings
We first present graphical displays of discontinuities in select outcomes. In Figure 4, we
plot the relationship between test score and these outcomes around the cutoff between extended
and one-semester algebra using locally weighted scatterplot smoothing (Lowess) curves.
12
On
the left of each cutoff are the mean outcomes of students assigned to extended algebra, and on
the left are the outcomes of students assigned to one-semester algebra. We observe evidence of a
discontinuity for each outcome at the cutoff, suggesting that assignment to extended algebra
resulted in lowered probabilities of achieving each outcome.
Figure 4. Discontinuities in Persistence and Completion Outcomes
(a) Pass EA with a B or better
(b) Attempt IA
(c) Pass IA with a B or better
(d) Total Degree-Applicable Units
Completed
Unrelated Regressions, in which the errors across the regressions described above are allowed to be correlated across regression equations. We
then conduct a joint test that the coefficient for treatment assignment (indicating any discontinuities at the cutoff) is zero. This joint test indicates
that there is no discontinuity in the set of covariates at the cutoff. We therefore include all covariates in the final RD models to control for these
observable differences and increase precision of the treatment effect estimates.
12
The curves are drawn using a Lowess smoother with running means, which is a local linear regression model.
EXTENDING TIME IN DEVELOPMENTAL MATH 28
The two-stage estimation of fuzzy RD design complements these graphical analyses. The
results of these first stage regressions for each college are shown in Table 4. It is clear that
assignment to extended algebra is a strong predictor of actual enrollment in an extended algebra
course. The coefficient for the assignment indicator variable is .682 for the full sample (p<.001),
and the estimates are similar for narrower bandwidths. Furthermore, the F-statistics reported in
Table 4 indicate that we find evidence of a strong and relevant instrument for each bandwidth. It
is interesting to note that higher-scoring students, younger students, African-American students,
and white students were less likely to comply with their placement assignments. Female students
were more likely to comply with their placement assignments.
EXTENDING TIME IN DEVELOPMENTAL MATH 29
Table 4. First Stage IV Regressions, Enrollment in Extended Algebra
All 1 SD 0.5 SD 0.33 SD 0.25 SD
Assignment to Extended Algebra .682*** .665*** .655*** .654*** .637***
.061 .070 .074 .081 .084
Centered Test Score -.009 -.058 -.143 -.246* -.235*
.033 .055 .118 .117 .103
Treatment*Test Score .003 .006 .012 .021* .015***
.004 .004 .008 .005 .002
Age At Assessment -.003** -.002** -.001 -.001 -.002
.0009 .001 .001 .001 .001
Female -.004 -.002 .003 .017*** .030***
.004 .004 .003 .005 .009
Asian -.010 -.008 -.007 .016 -.002
.013 .013 .019 .023 .022
Black -.031** -.025*** -.026 -.002 -.004
.010 .007 .014 .018 .027
Hispanic .002 .005 .013 .030 .029
.008 .005 .012 .016 .017
White -.026*** -.026*** -.010** .008 .007
.007 .006 .004 .012 .021
English not primary language .001 .002 .010 .011 .027
.011 .011 .014 .024 .036
Permanent Resident -.001 -.003 -.017 -.002 -.005
.013 .013 .013 .018 .020
Other Visas .011 .009 -.001 .030 -.027
.015 .016 .022 .016 .028
Constant .170*** .162*** .202** .199** .185**
.042 .048 .045 .069 .074
F-test 95.1*** 67.4** 58.6** 59.2** 43.6**
7384 7080 4579 3188 2465
Note: *** p<.001, **p<.01, *p<.05
Table 5 outlines the main results from the second stage regressions. The dependent
variables include the following main outcomes of interest: enrolling in any math course within
one year of assessment, attempting and passing elementary algebra (EA), attempting and passing
intermediate algebra (IA), total units completed within a year of the assessment date, and total
EXTENDING TIME IN DEVELOPMENTAL MATH 30
units completed through the study period. We present estimates for the entire range of students
placed in two- and one-semester algebra, along with the estimates obtained when restricting the
sample to various bandwidth sizes. This is recommended for testing sensitivity of the results to
different bandwidths in the RD design (Lee & Lemieux, 2010). We present bandwidths of 1.0,
0.5, and 0.33, and 0.25 standard deviations (SD) above and below each cutoff, but most of our
discussion of the results will focus on the fuzzy RD estimates within a bandwidth of 0.5 SD. We
reason that the bandwidth of 0.5 SD above and below the cutoff increases statistical power
relative to a bandwidth of .33 SD, and is more likely to maintain equality in expectation around
the cutoff relative to a bandwidth of 1 SD. In addition, we calculated the optimal bandwidth for
each outcome using the method outlined in Imbens and Kalyanaraman (2012), and found that
they generally ranged from 0.3-0.6SD, so we use 0.5SD for consistency of interpretation. The
full regression models shown include the covariates and fixed effects described earlier but these
estimated coefficients are not shown here for simplicity of presentation.
13
Enrolling in Math
We previously documented in Table 3 that students assigned to extended algebra were
less likely to enroll in any math regardless of the level after receiving their placement results, and
this is in accordance with other findings showing that lower-ability students are less likely to
enroll after initial assessment (Bailey et al., 2010). However, it is possible that students faced
with the decision to enroll in the two-semester sequence are deterred by the cost of an additional
semester of remediation. To investigate whether this may be attributable to the placement policy,
we use the regression discontinuity setup in equation (1) to examine differences in enrollment
decisions between students scoring just above and below the cutoff. We use the “sharp” RD
approach in equation (1) since the outcome of interest is initial enrollment and there is no second
13
The full results are available from the authors upon request.
EXTENDING TIME IN DEVELOPMENTAL MATH 31
stage. The results of this are presented in Table 5 and indicate that assignment to extended
algebra did not affect the decision to enroll within any of the bandwidths shown. Thus while
enrollment rates are lower on average for the whole group, it appears that two-semester algebra
students of similar ability near the one-semester cutoff did not have a different propensity to
enroll compared to their one-semester counterparts. This corroborates findings from other
researchers showing that assignment to remediation does not discourage students from enrolling
in math (Martorell, McFarlin, & Xue, 2015; Scott-Clayton & Rodriguez, 2015).
Attempting and Passing Gatekeeper Courses
We next examined completion of the EA course (either the second part of two-semester
sequence, or the one-semester course) and IA, the next course in the sequence, using the fuzzy
regression discontinuity approach indicated by equations (2) and (3). In Table 5, the FRD
estimates indicate that there is a significant negative effect of placement in extended courses on
eventually completing the EA level. Students at the margin of the cutoff and placed in extended
algebra were about 8-10 percentage points less likely to eventually pass EA with a B or better.
These discontinuities are demonstrated visually in Figure 5. As expected, point estimates were
larger for the C or better criterion, with the likelihood of passing EA being about 20 percentage
points lower for those placed in the extended courses.
14
14
These results are available from the authors upon request.
EXTENDING TIME IN DEVELOPMENTAL MATH 32
Table 5. The Effect of Extended Algebra Courses on College Outcomes, Fuzzy RDs
Bandwidth
All 1 SD 0.5 SD 0.33 SD 0.25 SD
Enrolled in Any Math (Sharp RD)
Extended Alg. -.049 -.031 -.019 -.011 -.024
.030 .033 .031 .029 .027
Constant .730*** .709** .605** .570*** .559***
.067 .086 .065 .044 .041
Passing EA (C or better)
Extended Alg. -.279*** -.207*** -.205*** -.190*** -.218***
.030 .033 .043 .049 .058
Constant .516*** .471*** .456*** .441*** .515***
.063 .071 .096 .114 .127
Passing EA (B or better)
Extended Alg. -.158*** -.081*** -.085* -.082^ -.107*
.028 .030 .039 .044 .051
Constant .154*** .071 .112 .081 .105
.058 .065 .087 .102 .113
Attempting IA
Extended Alg. -.193*** -.127*** -.128** -.130** -.173**
.030 .033 .042 .049 .057
Constant .480*** .445*** .490*** .403*** .447***
.063 .071 .096 .113 .126
Passing IA (C or better)
Extended Alg. -.108*** -.058^ -.080* -.101* -.124*
.028 .031 .039 .045 .053
Constant .295*** .266*** .348*** .288** .313**
.058 .066 .088 .104 .116
Passing IA (B or better)
Extended Alg. -.056* -.028 -.062* -.050 -.060
.022 .025 .031 .036 .042
Constant .071 .039 .129 .075 .075
.047 .053 .070 .083 .093
Units Completed in One Year
Extended Alg. -1.64* -.452 -.141 -.262 -1.43
.791 .878 1.12 1.30 1.53
Constant 15.1*** 14.2*** 15.7*** 14.6*** 16.8***
1.65 1.87 2.53 3.01 3.37
Degree-Applicable Units through Fall 2013
Extended Alg. -5.25** -3.41 -2.94 -4.29 -8.92*
1.77 1.97 2.50 2.93 3.48
Constant 37.0*** 36.4*** 42.4*** 43.9*** 49.8***
3.71 4.21 5.66 6.81 7.68
N 7384 7080 4579 3188 2465
Note: Instrumental variables regression with campus fixed effects. Covariates include: age, race, sex, language, residence
status, and cohort. ***p<.001 **p<.01 *p<.05 ^p<.10
EXTENDING TIME IN DEVELOPMENTAL MATH 33
The RD results also indicate that students placed into a two-semester versus a one-
semester algebra course had a lower probability of attempting and passing IA courses, the next
course in the sequence, which is a gatekeeper course to transfer-level math and some
introductory STEM courses. The estimates indicate that enrollment in extended algebra reduced
the likelihood of attempting IA by about 13 percentage points, passing IA with a C or better by
about 8 percentage points, and passing with a B or better by about 5-6 percentage points,
depending on bandwidth size.
College Persistence
Finally, the data enable us to examine the total number of units completed after taking the
math assessment for each cohort. The estimates have a higher degree of variation across
bandwidths, since a host of other unobservable student-level variables such as motivation or
financial need may affect persistence decisions. Overall, the RD results indicate that there was no
difference in total units completed within one year of the assessment between students assigned
to extended algebra and those assigned to one-semester algebra at the margin of the cutoff.
Examination of the completion of degree-applicable credit produced a slightly different result. It
appears that students in extended algebra completed fewer degree-applicable units than their
counterparts in one-semester algebra, ranging from about 3 to 8 units depending on the
bandwidth. The estimates are not all statistically significant but their similar magnitude and
negative direction suggest that enrollment in the extended algebra course points towards
deleterious long-term effects on persistence and progress towards obtaining a college degree.
Generalizability, Robustness, & Falsification Tests
The RD design produces causal estimates of the impact of extending time in math
remediation, but it is important to emphasize that the estimated treatment effect only applies to
EXTENDING TIME IN DEVELOPMENTAL MATH 34
students at the margin of the cutoff and does not extrapolate down to students with lower test
scores. While our estimates show a significant negative effect for students with scores that are
within narrow bandwidths around the cutoff, it is possible that these effects may be different for
students even further below the cutoff. The extended algebra sequence, by providing additional
time to master algebra skills and progressing through content at a slower pace, may be more
beneficial to these students who obtained lower placement scores. However, Scott-Clayton,
Crosta, and Belfield (2014) find that there are actually relatively weak correlations between
placement test scores and student outcomes, lending credence to the idea that RD estimates can
be extrapolated further down the continuum of scores. Indeed, some of the estimates we obtained
(e.g., for passing EA) are fairly robust for different bandwidth sizes, suggesting that the results
are internally valid and generalizable to a broader range of students around each cutoff.
Sensitivity & Robustness
We conducted several sensitivity analyses to check the robustness of the estimates
discussed above and present these in Table 6. In column 1, we show the estimated effect from
our preferred specification (0.5 SD, with a linear interaction of the test score and treatment
variable, with covariates, and with campus and cohort fixed effects). In column 2 we show the
model without any covariates and excluding cohort and campus fixed effects. The main concern
with larger bandwidth sizes in the RD design is that the data are less likely to be modeled best by
just a linear specification. In columns 3-4 we show the inclusion of quadratic and cubic forms of
the running variable and the interaction of these with the treatment indicator. Overall, these
sensitivity analyses indicate that the results we obtained are robust to model specifications.
EXTENDING TIME IN DEVELOPMENTAL MATH 35
Table 6. Sensitivity Analyses for Outcomes
(1)
Full
(2)
No Cov.
or Fixed
Effects
(3)
Quadratic
(4)
Cubic
(5)
Cutoff at
-0.25SD
(6)
Cutoff at
+0.25SD
Passed EA (C or better) -.205*** -.211*** -.258*** -.214*** .023 .010
.043 .045 .055 .070 .024 .024
Passed EA (B or better) -.085* -.082^ -.143** -.075 .023 .010
.039 .044 .050 .063 .024 .024
Attempted IA -.128** -.136*** -.185*** -.180** -.002 .005
.042 .030 .054 .069 .046 .013
Passed IA (C or better) -.080* -.092*** -.106* -.078 -.017 .012
.039 .024 .050 .064 .022 .009
Passed IA (B or better) -.062* -.063*** -.078^ -.074 -.017 .012
.031 .019 .040 .051 .022 .009
Total units completed
w/in 1 yr of assessment -.141 .001 -.619 1.03 -.243 -.173
1.12 .886 1.44 1.83 .998 .866
Total degree-applicable
units completed -2.94 -3.47 -5.35 -3.53 -.419 .017
2.50 2.14 3.22 4.10 1.73 2.90
Note: Full model is 0.5SD, with covariates, campus and cohort fixed effects, and linear interaction term of treatment and assignment
variables.
*** p<.001, **p<.01, *p<.0.5, ^p<.10
Finally, to ensure that the RD estimates are local to the placement cutoffs and not just an
artifact of the RD design, we also performed falsification tests by running sharp RD analyses at
placebo cutoffs 0.25 SD above and 0.25 SD below the actual cutoff. These are shown in columns
5 and 6 of Table 6. We cannot use the fuzzy RD approach here since we do not observe
compliance under these hypothetical scenarios. The results of these tests fail to show any
significant treatment effects at any of the placebo cutoffs above and below the actual cutoff for
any of the outcomes. We are therefore confident that the estimates we obtained are specific to the
extended algebra course placement policy.
EXTENDING TIME IN DEVELOPMENTAL MATH 36
Discussion & Conclusion
This study provides insight into the effectiveness of an alternative model of delivery of
developmental math in community colleges, one for which there is little empirical evidence.
Intuition suggests that extended math sequences can provide students with the time and learning
context they may need to develop math skills that might be lacking. This extra preparation may
enable students to be more successful in subsequent STEM courses or their postsecondary
careers. At the same time, extending math courses increases the amount of time students are
expected to be in math remediation, and increases the number of opportunities that a student can
exit the developmental math sequence. This is a concern because having more exit points is
associated with higher rates of attrition (Bailey et al., 2010; Fong et al., 2015), and additional
time in remediation can incur substantial costs in terms of time and money that students may not
be able or willing to pay (Melguizo et al., 2008). According to the colleges’ fee schedules for
spring 2013, the actual cost of the additional course would be about $184 ($46/unit x 4 units).
However, this estimate does not include any additional costs associated with attending college
(e.g., books, materials, transportation), or the opportunity cost associated with time spent in the
additional remedial course.
Overall, the RD results show that placement in extended math courses in these four
colleges resulted in unfavorable student outcomes. Although students who were at the margin of
assignment to extended algebra courses were just as likely to enroll in math as students assigned
to the one semester course, they were less likely over time to complete subsequent courses
compared to their counterpart right above the cutoff. Enrollment in extended algebra reduced the
probability that students at the margin of the cutoff completed, with a B or better, elementary
algebra by 8-10 percentage points, and the probability that they progressed on to pass
EXTENDING TIME IN DEVELOPMENTAL MATH 37
intermediate algebra by about 5-6 percentage points. While there are largely no differences in
credit attainment one year after assessment, students in extended algebra appeared to have
completed fewer degree-applicable credits than students placed in the one-semester course. This
corroborates the “diversion” hypothesis outlined by Scott-Clayton and Rodriguez (2015) –
students in remediation earn credits at about the same rate as students in higher-level courses, but
not ones that count towards degrees progress. Although we cannot specifically tease out whether
this is directly attributable to increasing the number of exit points, or just increasing the overall
duration of the sequence, or a combination of both, the practice of extending math courses in this
way reduces the likelihood that students will complete the developmental math sequence and
persist in college towards credential attainment.
This study contributes to the math education literature because increasing time in math
has been found to be beneficial for student achievement in K-12 settings (Cortes et al., 2015;
Taylor, 2014). We provide evidence from California community colleges suggesting that this
may not necessarily hold in postsecondary contexts where students have more autonomy in
making enrollment and persistence decisions. Attrition from developmental math sequences, for
example, may largely be the result of the freedom afforded within them. Models that add exit
points where students must make additional enrollment decisions may therefore not be helpful
for community college students since they can exacerbate the unstructured nature of community
college experiences. Our findings suggest that guided pathway models that are more structured
may be beneficial for developmental math students (Bailey, Jaggars, & Jenkins, 2015).
Similarly, models that extend time and increase costs can also deter or discourage
students from continuing on in their college careers. Reforms in developmental education should
thus consider the influence of time and cost on students’ enrollment decisions as well as the
EXTENDING TIME IN DEVELOPMENTAL MATH 38
structure of academic experiences. The opportunity costs to students associated with any reform
should not outweigh the reforms’ potential academic benefits. A promising approach may be
acceleration models, which combine courses and shorten the overall time spent in remediation
(Hern, 2012; Jaggars, Hodara, Cho, & Xu, 2013; Moltz, 2010; Rutschow & Schneider, 2011).
However, aside from a few studies highlighting the promise of acceleration models, there is still
limited evidence of the effectiveness of this approach.
Although our research design accounts for unobservable factors related to student
success, we note that faculty, instruction, and other institutional factors may play a large role in
explaining differences between the outcomes of students placed in alternative models of delivery
such as extended courses. Further research should examine the nature of classroom and
institutional environments, the stigma of extended time in remediation, and how these may
influence developmental math outcomes.
Nevertheless, the results of the study provide rigorous and important evidence on the
impact of extending time in math remediation in community college settings. Despite the
potential benefits to learning of increasing time in math, extending math over two semesters
inherently increases the need for students to make persistence decisions at exit points and
increases the costs associated with completing math remediation. Delivering developmental math
in this way may ultimately offset the benefits that additional time spent studying math may yield
and, as our study shows, prevent students from attaining their educational goals.
EXTENDING TIME IN DEVELOPMENTAL MATH 39
References
Aronson, J., Zimmerman, J., & Carlos, L. (1999). Improving student achievement by extending
school: Is it just a matter of time? Retrieved from http://files.eric.ed.gov
/fulltext/ED435127.pdf
Bahr, P. R. (2012). Deconstructing remediation in community colleges: Exploring associations
between course-taking patterns, course outcomes, and attrition from the remedial math
and remedial writing sequences. Research in Higher Education, 53, 661–693.
Bailey, T. R., Jaggars, S. S., & Jenkins, D. (2015). Redesigning America's community colleges:
A clearer path to student success. Cambridge, MA: Harvard University Press.
Bailey, T., Jeong, D. W., & Cho, S. W. (2010). Referral, enrollment, and completion in
developmental education sequences in community colleges. Economics of Education
Review, 29, 255-270.
Bandura, A. (1995). Self-efficacy in changing societies. New York: Cambridge University Press.
Barnett, E. A. (2011). Validation experiences and persistence among community college
students. The Review of Higher Education, 34(2), 193-230.
Becker, G. S. (1964). Human capital: A theoretical analysis with special reference to
education. National Bureau for Economic Research.
Bettinger, E., Boatman, A., & Long, B. (2013). Student supports: Developmental education and
other academic programs. The Future of Children, 23, 93–115.
Bourdieu, P. (1986). The forms of capital. In J. G. Richardson (Ed.), Handbook of Theory and
Research for the Sociology of Education (pp. 241–258). New York: Greenwood Press.
EXTENDING TIME IN DEVELOPMENTAL MATH 40
Burdman, P. (2012). Where to begin? The evolving role of placement exams for students starting
college. Boston, MA: Jobs for the Future.
Burdman, P. (2015). Degrees of freedom: Probing math placement policies at California
colleges and universities (No. 3). Stanford, CA: Policy Analysis for California Education.
Cortes, K. E., Goodman, J., & Nomi, T. (2015). Intensive math instruction and educational
attainment: Long-run impacts of double-dose algebra. Journal of Human Resources,
50(1), 108-158.
Crisp, G., & Delgado, C. (2014). The impact of developmental education on community college
persistence and vertical transfer. Community College Review, 42(2), 99-117.
Crisp, G., & Nora, A. (2009). Hispanic student success: Factors influencing the persistence and
transfer decisions of Latino community college students enrolled in developmental
education. Research in Higher Education, 51, 175–194.
Deil-Amen, R., & Tevis, T. L. (2010). Circumscribed agency: The relevance of standardized
college entrance exams for low SES high school students. The Review of Higher
Education, 33(2), 141-175.
Edgecombe, N. D. (2011). Accelerating the academic achievement of students referred to
developmental education. Retrieved from
http://academiccommons.columbia.edu.libproxy.usc.edu/catalog/ac:146646
Edgecombe, N., Cormier, M. S., Bickerstaff, S., & Barragan, M. (2013). Strengthening
developmental education reforms: Evidence on implementation efforts from the scaling
innovation project. New York, N.Y: Columbia University, Teachers College, Community
College Research Center.
EXTENDING TIME IN DEVELOPMENTAL MATH 41
Fain, P. (2013, July 19). Free courses for a big problem. Inside Higher Ed. Retrieved from
http://www.insidehighered.com/news/2013/07/19/two-year-colleges-go-open-source-
seek-fix-remediation.
Fong, K. E., Melguizo, T., & Prather, G. (2015). Increasing success rates in developmental
math: The complementary role of individual and institutional characteristics. Research in
Higher Education. DOI: 10.1007/s11162-015-9368-9.
Grubb, N. (1999). Honored but invisible: An inside look at America’s community colleges. New
York: Routledge.
Hern, K. (2012). Acceleration across California: Shorter pathways in developmental English and
math. Change: The Magazine of Higher Learning, 44(3), 60-68.
Hughes, K. L., & Scott-Clayton, J. (2011). Assessing developmental assessment in community
colleges. Community College Review, 39(4), 327-351.
Imbens, G., & Kalyanaraman, K. (2011). Optimal bandwidth choice for the regression
discontinuity estimator. The Review of Economic Studies, 79(3), 933-959.
Jaggars, S. S., Hodara, M., Cho, S. W., & Xu, D. (2014). Three accelerated developmental
education programs: Features, student outcomes, and implications. Community College
Review, 43(1), 3-26.
Kosiewicz, H., Ngo, F., & Fong, K. E. (2014). Alternative models of delivering developmental
math: Issues of use and student access. Working paper. University of Southern
California.
Lee, D. S., & Lemieux, T. (2010). Regression discontinuity designs in economics. Journal of
Economic Literature, 48(2), 281-355.
EXTENDING TIME IN DEVELOPMENTAL MATH 42
Manski, C. F., &Wise, D. (1983). College choice in America. Cambridge, MA: Harvard
University Press.
Martorell, P., & McFarlin Jr, I., & Xue, Y. (2015). Does failing a placement exam discourage
underprepared students from enrolling college? Education Finance and Policy, 10(1), 46-
80.
McCrary, J. (2008). Manipulation of the running variable in the regression discontinuity design:
A density test. Journal of Econometrics, 142(2), 698-714.
Melguizo, T. (2011). A review of the theories developed to describe the process of college
persistence and attainment. In Higher education: Handbook of theory and research (pp.
395-424). Springer Netherlands.
Melguizo, T., Kosiewicz, H., Prather, G., & Bos, H. (2014). How are community college
students assessed and placed in developmental math? Grounding our understanding in
reality. Journal of Higher Education, 85(5), 691-722.
Melguizo, T., Hagedorn, L. S., & Cypers, S. (2008). Remedial/developmental education and the
cost of community college transfer: A Los Angeles County sample. The Review of Higher
Education, 31(4), 401-431.
Moltz, D. (2010, July 6). Picking up the pace. Inside Higher Ed. Retrieved from
http://www.insidehighered.com/news/2010/07/06/accelerated.
Murnane, R. J., & Willett, J. B. (2010). Methods matter: Improving causal inference in
educational and social science research. Oxford University Press.
National Center for Education Statistics. (2014). Digest of Education Statistics 2014. The U.S.
Department of Education. Washington D.C.
EXTENDING TIME IN DEVELOPMENTAL MATH 43
Nomi, T. and E. Allensworth (2009). “Double-dose” algebra as an alternative strategy to
remediation: Effects on students’ academic outcomes. Journal of Research on
Educational Effectiveness, 2(2), 111–148.
Oreopoulos, P., & Petronijevic, U. (2013). Making college worth it: A review of research on the
returns to higher education. The Future of Children, 23(1), 41-65.
Pascarella, E. T., Salisbury, M. H., & Blaich, C. (2011). Exposure to effective instruction and
college student persistence: A multi-institutional replication and extension. Journal of
College Student Development, 52(1), 4-19.
Patall, E. A., Cooper, H., & Allen, A. B. (2010). Extending the school day or school year: A
systematic review of research (1985–2009). Review of Educational Research, 80(3), 401-
436.
Paulsen, M. B. (2001). The economics of human capital and investment in higher education. In
M. B. Paulsen (Ed.), The Finance of higher education: Theory, research, policy, and
practice, 55-94. Agathon Press.
Perry, M., Bahr, P. M., Rosin, M., & Woodward, K. M. (2010). Course-taking patterns, policies,
and practices in developmental education in the California Community Colleges.
Mountain View, CA: EdSource. Retrieved from
http://www.edsource.org/assets/files/ccstudy/FULL-CC-DevelopmentalCoursetaking.pdf
Rendon, L. I. (1994). Validating culturally diverse students: Toward a new model of learning
and student development. Innovative Higher Education, 19(1), 33-51.
Rutschow, E. Z., & Schneider, E. (2011). Unlocking the gate: What we know about improving
developmental education. New York, NY: MDRC.
EXTENDING TIME IN DEVELOPMENTAL MATH 44
Scaling Innovation (2012). Doing developmental education differently: Scaling innovation.
Retrieved from http://www.scalinginnovation.org/doing-developmental-education-
differently/
Scott-Clayton, J., Crosta, P., & Belfield, C. (2014). Improving the targeting of treatment:
Evidence from college remediation. Educational Evaluation and Policy Analysis, 36(3),
371-393.
Scott-Clayton, J., & Rodriguez, O. (2015). Development, discouragement, or diversion? New
evidence on the effects of college remediation. Education Finance and Policy, 10(1), 4-
45.
Stein, M. K., Kaufman, J. H., Sherman, M., & Hillen, A. F. (2011). Algebra: A challenge at the
crossroads of policy and practice. Review of Educational Research, 81(4), 453-492.
Taylor, E. (2014). Spending more of the school day in math class: Evidence from a regression
discontinuity in middle school. Journal of Labor Economics, 117, 162-181.
Tinto, V. (1993). Leaving college: Rethinking the causes and cures of student attrition. Chicago:
University of Chicago Press.
Zachry, E. M., & Schneider, E. (2008). Promising instructional reforms in developmental
education: A case study of three Achieving the Dream Colleges. New York, NY: MDRC.
EXTENDING TIME IN DEVELOPMENTAL MATH 45
Appendix
Figure A1. Trends in Covariates, Local Linear Regression
EXTENDING TIME IN DEVELOPMENTAL MATH 46
Figure A2. McCrary Density Tests, 4 Colleges
College A
Log Discontinuity Estimate: 0.092
Standard Error: 0.055
College B
Log Discontinuity Estimate: 0.12
Standard Error: 0.10
College C
Log Discontinuity Estimate: -0.052
Standard Error: 0.057
College D
Log Discontinuity Estimate: 0.22
Standard Error: 0.109
EXTENDING TIME IN DEVELOPMENTAL MATH 47
Table A1. Test of discontinuities in covariates, Seemingly Unrelated Regressions
All 1 SD 0.5 SD 0.33 SD 0.25 SD
Age at assessment 1.38*** .473 .539 .330 .976*
.203 .244 .318 .376 -.030
Female -.0002 -.023 -.035 -.050 .029
.014 .017 .022 .026 .022
Asian -.009 .003 .002 .000 -.012
.008 .010 .012 .014 .016
Black -.018 .014 -.016 -.014 .009
.010 .012 .016 .019 .021
Hispanic -.025 -.022 -.022 .001 -.005
.014 .017 .022 .026 .029
English not primary language -.007 .002 -.002 .005 -.016
.012 .015 .019 .023 .026
Permanent Resident -.002 .006 .006 .004 -.003
.008 .010 .013 .015 .017
Other Visas -.012 -.005 -.084 .012 .012
.008 .009 .039 .014 .016
Joint Significance Test
Chi2 54.10 7.31 6.86 5.8 8.7
Prob>Chi2 0.00 0.5 0.552 0.67 0.369
N 11906 11425 7545 5387 4114
Note: Each covariate is the dependent variable in a Seemingly Unrelated Regression using a regression discontinuity design, and the reported
estimate is of the discontinuity in the covariate at the placement cutoff. ***p<0.001, **p<0.01, *p<0.05
Abstract (if available)
Linked assets
University of Southern California Dissertations and Theses
Conceptually similar
PDF
Math and the making of college opportunity: persistent problems and possibilities for reform
PDF
Reforming developmental education in math: exploring the promise of self-placement and alternative delivery models
PDF
Developmental math in California community colleges and the delay to academic success
PDF
Three essays on the high school to community college STEM pathway
PDF
Ready or not? Unprepared for community college mathematics: an exploration into the impact remedial mathematics has on preparation, persistence and educational goal attainment for first-time Cali...
PDF
Unprepared for college mathematics: an investigation into the attainment of best practices to increase preparation and persistence for first-time California community college freshmen in remedial...
PDF
To what extent does being a former high school English learner predict success in college mathematics? Evidence of Latinx students’ duality as math achievers
PDF
The impact of remedial mathematics on the success of African American and Latino male community college students
PDF
A multi-perspective examination of developmental education: student progression, institutional assessment and placement policies, and statewide regulations
PDF
Essays on education: from Indonesia to Los Angeles, CA
PDF
Mathematics identity and sense of belonging in mathematics of successful African-American students in community college developmental mathematics courses
PDF
Motivational, parental, and cultural influences on achievement and persistence in basic skills mathematics at the community college
PDF
The effects of a math summer bridge program on college self-efficacy and other student success measures in community college students
PDF
The perceptions and attitudes of “low-router” students in developmental math
PDF
Essays on economics of education
PDF
Oppression of remedial reading community college students and their academic success rates: student perspectives of the unquantified challenges faced
PDF
Measuring the alignment of high school and community college math assessments
PDF
Relationships between a community college student’s sense of belonging and student services engagement with completion of transfer gateway courses and persistence
PDF
Community college education for the incarcerated: the provision of access, persistence and social capital
PDF
Assessing persistence for low-income students at community colleges: the impact of student-parent relationships
Asset Metadata
Creator
Kosiewicz, Holly
(author),
Ngo, Federick J.
(author)
Core Title
How extending time in developmental math impacts persistence and success: evidence from a regression discontinuity in community colleges
School
College of Letters, Arts and Sciences
Degree
Master of Arts
Degree Program
Economics
Publication Date
09/17/2015
Defense Date
09/16/2015
Publisher
University of Southern California
(original),
University of Southern California. Libraries
(digital)
Tag
Community Colleges,developmental/remedial education,math education,OAI-PMH Harvest,regression discontinuity
Format
application/pdf
(imt)
Language
English
Contributor
Electronically uploaded by the author
(provenance)
Advisor
Nugent, Jeff (
committee chair
), Melguizo, Tatiana (
committee member
), Nyshadham, Anant (
committee member
)
Creator Email
federicn@usc.edu,mrfngo@gmail.com
Permanent Link (DOI)
https://doi.org/10.25549/usctheses-c40-182375
Unique identifier
UC11276375
Identifier
etd-NgoFederic-3922.pdf (filename),usctheses-c40-182375 (legacy record id)
Legacy Identifier
etd-NgoFederic-3922.pdf
Dmrecord
182375
Document Type
Thesis
Format
application/pdf (imt)
Rights
Ngo, Federick J.; Kosiewicz, Holly
Type
texts
Source
University of Southern California
(contributing entity),
University of Southern California Dissertations and Theses
(collection)
Access Conditions
The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the a...
Repository Name
University of Southern California Digital Library
Repository Location
USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA
Tags
developmental/remedial education
math education
regression discontinuity