Close
Home
Collections
Login
USC Login
Register
0
Selected
Invert selection
Deselect all
Deselect all
Click here to refresh results
Click here to refresh results
USC
/
Digital Library
/
University of Southern California Dissertations and Theses
/
Utilizing analytics to evaluate the San Diego Housing Commission's approach to maximizing housing efficiency: Moving to Work, Path to Success, and the Achievement Academy
(USC Thesis Other)
Utilizing analytics to evaluate the San Diego Housing Commission's approach to maximizing housing efficiency: Moving to Work, Path to Success, and the Achievement Academy
PDF
Download
Share
Open document
Flip pages
Contact Us
Contact Us
Copy asset link
Request this asset
Transcript (if available)
Content
UTILIZING ANALYTICS TO EVALUATE THE SAN DIEGO HOUSING COMMISSION’S
APPROACH TO MAXIMIZING HOUSING EFFICIENCY: MOVING TO WORK, PATH TO
SUCCESS, AND THE ACHIEVEMENT ACADEMY
by
Timothy Sean Walsh
A Dissertation Presented to the
FACULTY OF THE USC SOL PRICE SCHOOL OF PUBLIC POLICY
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulfillment of the
Requirements for the Degree
DOCTOR OF POLICY, PLANNING, AND DEVELOPMENT
December 2022
Copyright 2022 Timothy Sean Walsh
MAXIMIZING HOUSING EFFICIENCY
ii
Acknowledgments
I greatly appreciate Dr. T.J. McCarthy (Dissertation Committee Chair), Dr. Pamela
McCann, and the Honorable Richard Gentry for their mentorship, professionalism, and expertise
within their respective fields. Specifically, Dr. McCarthy’s unwavering support, perfectly
balanced guidance, and commitment to my research and academic growth were essential to
completing this research. He is a model for the consummate mentor and pedagogical
professional. I have rarely encountered someone of his caliber in all my professional and
academic endeavors.
I want to express gratitude for the support of the entire San Diego Housing Commission
Team. While there are too many individuals to mention by name, I consistently encountered
professional, diligent, and considerate public servants at the Commission. This included every
interaction from the CEO to the Information Technology Team. Their sincere dedication to the
public housing residents was apparent in each communication. Further, the challenges they are
working to address are significant; nonetheless, I feel confidently optimistic after interacting
with them.
Importantly, I thank my family and friends for their love and support throughout my
graduate training. I will never be able to fully express my gratitude and admiration to both of my
sets of parents, Bill and Penny Walsh and Dr. Jose Escobar and Miriam Aguilar-Escobar. Their
support and inspiration have allowed me to pursue paths that would otherwise be impossible. For
my daughter, Catalina Walsh, thank you for providing unlimited hope and motivation. Finally, I
thank my wife, Dr. Natalia Escobar Walsh, who has consistently been my greatest supporter. Her
encouragement and sacrifice have made this daunting task attainable. She is the very best life
partner, friend, and mother of our child, for which I could have ever hoped.
MAXIMIZING HOUSING EFFICIENCY
iii
Table of Contents
Acknowledgments .......................................................................................................................... ii
List of Tables .................................................................................................................................. vi
List of Figures ............................................................................................................................... vii
Abstract ........................................................................................................................................ viii
Chapter 1: Introduction: California’s Affordable Housing Shortage .............................................. 1
Statement of the Problem ............................................................................................................ 2
Definition and History of Terms ................................................................................................. 4
Housing Efficiency .................................................................................................................. 4
Positive Moveouts ................................................................................................................... 4
Positive Outcomes ................................................................................................................... 4
The Quality Housing and Work Responsibility Act (QHWRA) of 1998 ................................ 4
Moving to Work ...................................................................................................................... 5
San Diego Housing Commission and Residents ................................................................... 10
SDHC’s Path to Success Program and Achievement Academy ............................................ 10
Family Self-Sufficiency Programs ........................................................................................ 14
Research Question ..................................................................................................................... 16
Statement of Hypothesis ............................................................................................................ 16
Research Purpose – Labor Participation Programs and the Need for More Research .............. 17
Research Problem ...................................................................................................................... 19
Review of the Related Literature ............................................................................................... 20
Evaluations of Local MTW Self-Sufficiency Programs ........................................................ 21
Evaluations of FSS ................................................................................................................ 21
Chapter 2: Review of the Related Literature ................................................................................. 24
Comparable Evaluations of Two Categories of Public Housing Agency Programs ................. 25
Evaluations of Local MTW Self-Sufficiency Programs ............................................................ 26
Charlotte Housing Authority ................................................................................................. 26
Housing Authority of Champaign County ............................................................................. 26
Family Self-Sufficiency Programs ............................................................................................ 27
Lynn and Cambridge, Massachusetts .................................................................................... 27
NYC-Work Rewards demonstration ..................................................................................... 28
The Youngs Lake Commons Program .................................................................................. 29
Rockford Housing Authority ................................................................................................. 30
Behavioral Economics Research on Reinforcement Mechanisms ............................................ 31
MAXIMIZING HOUSING EFFICIENCY
iv
Summary .................................................................................................................................... 35
Chapter 3: Methodology ................................................................................................................ 36
Re-Statement of Hypothesis ...................................................................................................... 37
Subjects ...................................................................................................................................... 38
Data Collection and Instrumentation ......................................................................................... 39
Data Sources .......................................................................................................................... 40
Procedure/Design ....................................................................................................................... 46
Data Analytic Approach ........................................................................................................ 46
Descriptive Analysis .................................................................................................................. 47
Summary .................................................................................................................................... 49
Chapter 4: Results .......................................................................................................................... 50
Population Demographics Differences ...................................................................................... 50
The Difference in Earnings Between PTS and AA – Real Income ....................................... 51
Accounting for Extreme Values ............................................................................................ 54
The Difference in Earnings Between PTS and AA – Income Growth .................................. 57
Regression - Income .............................................................................................................. 64
Summary of Income Results .................................................................................................. 67
Days Occupied Results .............................................................................................................. 68
The Difference in Days Occupied Between PTS and AA ..................................................... 68
Moveout Reasons ...................................................................................................................... 69
Differences in Moveout Reasons Between AA and PTS ...................................................... 69
Summary .................................................................................................................................... 69
Chapter 5: Limitations, Recommendations, Future Research Opportunities ................................ 71
Limitations ................................................................................................................................. 71
Test Group Selection Bias ..................................................................................................... 71
Challenges with SDHC 50058 Data ...................................................................................... 72
Achievement Academy Tracking .......................................................................................... 74
Days Occupied and Reasons for Moveout Data .................................................................... 74
Recommendations ..................................................................................................................... 75
Days Occupied and Reasons for Moveout Data .................................................................... 75
Unintended Effects of Incentives and Sanctions ................................................................... 76
SDHC 50058 Data Collection ............................................................................................... 77
Program Tracking .................................................................................................................. 79
Real-Time Data Gathering ..................................................................................................... 80
Behavioral Science Consultation ........................................................................................... 80
Future Research Opportunities .................................................................................................. 81
Additional Analysis of the Achievement Academy .............................................................. 81
MAXIMIZING HOUSING EFFICIENCY
v
Analysis of Additional Programs .......................................................................................... 81
Cost-Benefit Analysis ............................................................................................................ 83
Relevant Research on Work Requirements, Time Limits, and Tiered Rent Programs ......... 83
Conclusions ............................................................................................................................... 83
References ..................................................................................................................................... 85
MAXIMIZING HOUSING EFFICIENCY
vi
List of Tables
Table 1 PTS Tiered Rent Table (Truncated) ………………...…………………………………. 11
Table 2 Target Variable Descriptions from 50058 Data …………...…………...…………..….. 41
Table 3 Target Variable Descriptions from Demographic and Moveout Data ……...……….… 43
Table 4 Moveout Reason Grouping ……...……..…..………………………………...………... 45
Table 5 Summary Statistics 1: San Diego Housing Commission PTS Heads of Households …. 48
Table 6 Summary Statistics 2: San Diego Housing Commission PTS Heads of Households …. 48
Table 7 Head of Household Resident Participation …………...……………………………….. 49
Table 8 Summary Statistics 1: AA and PTS Work-able Heads of Households ……………...… 51
Table 9 Summary Statistics 2: AA and PTS Work-able Heads of Households ………...…....… 51
Table 10 Mean Income AA and PTS Work-able Households ……….….……..……..………... 52
Table 11 Mean Income with Minimums and Maximums: AA and PTS Work-able ……..….… 53
Table 12 Real Income: Untrimmed, 1/99% Trimmed, and 5/95% Trimmed .............................. 56
Table 13 Percent Annual Income Growth AA and PTS Work-able Households ……….……... 58
Table 14 Percent Income Change: Untrimmed, 1/99% Trimmed, and 5/95% Trimmed …....… 60
Table 15 Real Income Regression ……..………………....………..…………….……..……… 66
Table 16 Income Growth Regression ……..……………….…….…..………………………… 67
MAXIMIZING HOUSING EFFICIENCY
vii
List of Figures
Figure 1 Moving To Work (MTW) – Participating Agencies ………………...………………… 7
Figure 2 Average Income Under Path to Success ………………......………………….………. 12
Figure 3 Relational Representation of SDHC Program Structure ......…………………….…… 14
Figure 4 Gross Annual Income Boxplots …………………………………………………….… 62
Figure 5 Income Growth Boxplots …………………………...……………………………...… 63
MAXIMIZING HOUSING EFFICIENCY
viii
Abstract
This is an evaluation of the San Diego Housing Commission (SDHC), specifically whether two
programs offered, using the latitude created by the Moving to Work (MTW) demonstration,
differ in the outcomes produced for residents. The programs commonly designed by Moving to
Work agencies often include programs that attempt to increase income or labor participation;
therefore, similar programs created by analogous agencies are discussed and highlighted to
support this evaluation. A criterion is highlighted to categorize these programs for any future
analysis. Two candidate groups within the SDHC are compared, and outcomes created by each
are evaluated. Specifically, the results of this evaluation demonstrated a statistically significant
difference in the income growth rate between the groups. Existing limitations are discussed, and
recommendations are made.
Keywords: Moving to Work, San Diego Housing Commission, Path to Success, Achievement
Academy, and Housing Efficiency
MAXIMIZING HOUSING EFFICIENCY
1
Chapter 1: Introduction: California’s Affordable Housing Shortage
Many urban areas in the United States are facing an affordable housing crisis. In
California alone, “there is a shortfall of more than one million rental homes affordable to
extremely- and very low-income households, and California’s homeownership rate has declined
to the lowest rate since the 1940s” (Bates et al., 2018, p. 5). The Golden State will have to
produce over 1.8 million more homes to meet expected demand by 2025 (Bates et al., 2018). To
put this into perspective, “California has averaged less than 80,000 new homes annually” from
2007 to 2017 (Bates et al., 2018, p. 6). According to The Joint Center for Housing Studies in The
State of the Nation’s Housing 2022, “renters were particularly hard-pressed, with 46 percent at
least moderately cost burdened and 24 percent severely burdened” (p. 37). Stated bluntly,
“millions of US households are unable to afford their housing” (Joint Center for Housing
Studies, 2022, p. 37).
There are two commonly proposed methods to address the shortage of affordable
housing. The first is to create additional affordable housing and make this housing available to
those in need. However, it is unlikely that building new housing alone will address this problem,
at least not quickly. Providing additional housing will not address all the societal causes leading
someone to be unhoused. Additionally, some of the programs currently in place that attempt to
help individuals in subsidized housing are controversial, surrounded by partisan disagreement,
and plagued by a discriminatory past. Further, the rising cost of constructing affordable housing
is compounding this wicked problem (Bates et al., 2018). Lastly, exacerbating this challenge of
creating and allocating housing assistance is that “only about a fourth of eligible households
receive it, and there are long waiting lists for assistance in most communities” (Riccio, 2007, p.
7).
MAXIMIZING HOUSING EFFICIENCY
2
The second method to address the affordable housing shortage is to help work-able
residents in existing affordable housing increase self-sufficiency and then transition to market-
rate housing or eventually purchase their own home, making existing affordable housing units
available for other qualified residents. The term "Housing Efficiency" is used to refer to this
method. Both efforts create additional space for those in need of affordable housing; therefore,
the most prudent approach is to pursue both simultaneously. It should be noted that while this
research may help determine the effectiveness of a particular agency’s programmatic efforts,
increasing housing efficiency alone will not sufficiently address the current crisis.
In an acknowledgment of the current crisis, Public Housing Agencies (PHAs) are
increasingly taking on the long-term commitment of attempting to increase the self-sufficiency
of residents ( by enhancing their income and labor participation) in the hopes that residents will
eventually make a “positive moveout” to market-rate housing. This transitional perspective from
transitory to more long-term housing accompanied the federal cutbacks in the 1980s and a shift
in the public’s view that some were taking advantage of existing support programs (Rohe &
Kleit, 1999). This evolution coincided with the creation of self-sufficiency programs to aid in the
process of helping residents move on from public-supported housing and the recognition by
policymakers that “the delivery of housing services must be accompanied by other services that
would eventually foster family economic self-sufficiency” (Shlay, 1993, p. 457).
Statement of the Problem
While the State of California attempts to increase its supply of affordable housing, it must
also maximize each existing affordable unit. Therefore, to address the current lack of affordable
housing, it is essential to build new housing units while simultaneously using existing housing
efficiently by helping current work-able residents increase their income and graduate to market-
MAXIMIZING HOUSING EFFICIENCY
3
rate housing. These positive moveouts are vital to make space for others in need of affordable
housing and allow for more efficient use of federal resources by increasing the occupancy of
existing affordable housing.
This study evaluates one agency's approach to maximizing existing housing: the San
Diego Housing Commission's (SDHC) Path to Success (PTS) program and its affiliated
Achievement Academy (AA). To better understand these programs, this evaluation reviews the
United States Department of Housing and Urban Development's (HUD's) Moving to Work
(MTW) demonstration and provides a brief history of programs attempting to increase resident’s
income and labor participation (these are referred to as “labor participation programs” for
simplicity). Much of the research on MTW agencies and, precisely, on these types of programs
operated by MTW agencies calls for additional research to determine the impact of these
innovative efforts. Similar programs operated by other housing agencies are discussed to provide
further context for the SDHC’s PTS and AA programs. While some evaluation of the labor
participation programs of individual housing agencies exists, the SDHC and its PTS program
have undergone little analysis by an independent evaluator before this effort, and the public
availability of internal research, if it exists, is understandably limited due to privacy concerns.
The existing research offers no uniformly accepted metric of comparison between other
existing agencies or their programmatic efforts. Therefore, this analysis contributes to a clear gap
in existing research of MTW agencies by not only adding an evaluation of one of the longest and
most established MTW agencies while the MTW program is undergoing a significant expansion
but also by providing one evaluation framework for future evaluations of other MTW agencies.
This research offers a model of categorizing the labor participation programs these agencies
create. Lastly, this research provides clear recommendations for specific data collection by HUD
MAXIMIZING HOUSING EFFICIENCY
4
and PHAs to make future evaluations as impactful as possible and eliminate impediments to an
effective analysis by suboptimal data collection/handling.
Definition and History of Terms
Housing Efficiency
Housing Efficiency, as used in this analysis, refers to the length of time work-able
residents remain in affordable housing and the reasons for their moveout. The faster residents
move from supportive or affordable housing to market-rate or unsupported housing, or if the
moveout from supportive or affordable housing is due to increased self-sufficiency, the more
efficient that unit is. A necessary step to increasing housing efficiency is increasing resident
income.
Positive Moveouts
Positive Moveouts are exits resulting from favorable outcomes.
Positive Outcomes
Positive Outcomes are defined as an increase in income as reported on the United States
Department of Housing and Urban Development Form HUD-50058 MTW Family Report (HUD
Family Report, 2013) or a positive reason for moveout as recorded by the SDHC.
The Quality Housing and Work Responsibility Act (QHWRA) of 1998
The Quality Housing and Work Responsibility Act (QHWRA) of 1998 spawned the
creation of a "variety of self-sufficiency innovations and housing policy reforms” (Riccio, 2007,
p. iii). While policymakers have proposed that the housing subsidy system in the United States
should play a more significant role in helping residents strive for self-sufficiency, this legislation
once again clarified this approach. The programs created by the QHWRA are not the focus of
this study and evidence of each program’s impacts vary, nonetheless, they are important to
MAXIMIZING HOUSING EFFICIENCY
5
highlight as evidence of efforts by policymakers to find successful self-sufficiency approaches
with residents of public housing. This research focuses on one such program.
Jobs-Plus Community Revitalization Initiative for Public Housing Families
The Jobs-Plus Community Revitalization Initiative for Public Housing Families (Jobs-
Plus) began with a demonstration targeting the residents of housing developments in six cities:
Baltimore, Chattanooga, Dayton, Los Angeles, St. Paul, and Seattle. The program aims “to
increase earnings and advance employment outcomes” (HUD Jobs Plus, 2021, para. 1). This
well-researched and successful program began in the mid-1990s and is now funded by HUD
through annual grants (Bloom et al., 2005; Riccio, 2007). To this author's knowledge, all
previous research to date on Jobs Plus tested its rent policy alterations along with other
incentives and services offered to residents; therefore, “it is impossible to know for sure how
much of the earnings effect [increase] was attributable to that feature of the program” (Riccio et
al., 2017, p. 4). This research tests the SDHC’s rent policy (PTS) with and without incentives
(AA) to better understand the impact of both. However, a comprehensive comparison of PTS’s
effects is impossible without a comparable control group. Nonetheless, the incremental effects of
AA are clearly understood.
Moving to Work
Moving to Work (MTW) is a demonstration program launched in 1997 that allows
authorized housing authorities “to design and test innovative, locally-designed strategies in
pursuit of the national goal of delivering rental assistance more efficiently” (Cadik & Nogic,
2010, p. 10; Riccio, 2007). Moving to Work agencies can receive an exemption from many of
the rules in the United States Housing Act of 1937 to reduce costs, give incentives to families so
MAXIMIZING HOUSING EFFICIENCY
6
they may become economically self-sufficient, and increase housing choices for qualifying
families (HUD MTW Demonstration Program, 2022).
The programs created by MTW agencies to achieve the strategies mentioned above create
self-sufficiency programs that, in some instances, include the “imposition of work requirements,
which require work-able public housing residents to work a minimum number of hours per
week” (Rohe et al., 2016, p. 910). Other programs may involve professional training, schooling,
rent reform, time limits, or receiving cash-based incentives. HUD has designated 109 PHAs
(roughly three percent of the national total) as MTW agencies, with plans to expand to another
estimated 30 PHAs (HUD MTW, 2022).
MAXIMIZING HOUSING EFFICIENCY
7
Figure 1
Moving To Work (MTW) – Participating Agencies
Note. United States Department of Housing and Urban Development (2022). Moving to Work Demonstration
Program – Participating Agencies. Retrieved June 23, 2022, https://www.hud.gov/program_offices/public_
indian_housing/programs/ph/mtw/mtwagencies.
The MTW demonstration, and the programs its flexibility has allowed, have become the
subject of debate among policymakers and researchers. Stater (2018) argued that "the MTW
program allows agencies to best assess their community’s needs and distribute housing
assistance where it is most needed” (p. 11). However, because of the program’s flexibility, “each
MTW agency implements a distinct set of activities with unique outcome measures, making the
effects of MTW difficult to generalize” (Cadik & Nogic, 2010, p. 3). Congresswoman Maxine
Waters, a critic of the program, argued in a 2015 letter about the shortcomings of the MTW
MAXIMIZING HOUSING EFFICIENCY
8
program. She continues to call for reforms to address an “issue of great concern…that the
demonstration program allows participating agencies to impose rent increases, work
requirements, time limits, and other policy changes that risk serious hardship for recipients”
(Waters, 2015, p. 2). Similarly, "social justice advocates argue that safe and affordable housing
should be a basic human right and that policies that interfere with this right—such as a work
requirement—are unjust" (Rohe et al., 2016, p. 912).
Following the criticism of the MTW program and the proposals created by individual
MTW agencies, the United States Congress passed the 2016 Consolidated Appropriations Act,
which authorized the expansion of the MTW demonstration program (mentioned above) in five
policy targeted cohorts: a) Flexibility for Smaller PHAs Cohort, b) Stepped and Tiered Rent
Cohort, c) Work Requirement Cohort, d) Landlord Incentive Cohort, and e) Asset Building
Cohort. According to HUD, “the vision for the MTW expansion is to learn from MTW
interventions in order to improve the delivery of federally assisted housing and promote self-
sufficiency for low-income families across the nation” (HUD, MTW Expansion, 2022, para. 4).
Evaluation of these cohorts is intended to address the consistent past criticism that "no systemic
evaluation or attempt has been made to analyze what the impact has been on residents' work
engagement, incomes, or housing stability or on agency administrative cost" (Levy et al., 2018,
p. 2). Restating, current evaluations are insufficient to determine whether these initiatives
accomplish their goals.
In a 2018 Report to the Ranking Member, Committee on Financial Services, the United
States General Accountability Office (GAO) made multiple recommendations to HUD to
improve its oversight and analysis of the MTW program (GAO, 2018). HUD plans to use these
cohort evaluations to institute some recommendations and address criticisms. HUD’s analysis
MAXIMIZING HOUSING EFFICIENCY
9
“will be overseen by a research advisory committee to ensure the demonstrations are evaluated
with rigorous research protocols, quantitative analysis, and comparisons to control groups”
(National Low Income Housing Coalition, 2021, para. 7). On June 17, 2021, HUD rescinded its
application request under the MTW program for the Work Requirement Cohort; however, the
Landlord Incentive Cohort remained in place, and the Asset Building Cohort, was later added
(HUD Notice PIH-2021-18, 2021).
This evaluation of the SDHC adds to the research (mentioned above) by analyzing the
incremental effects of a self-sufficiency program within a long-standing MTW agency that, to
date, has received little external academic programmatic evaluation. Specifically, additional
research is required to determine if the incentives, such as those offered by Jobs Plus, are
impactful in the programs offered by MTW agencies. Further, this research provides additional
information for policymakers to consider when evaluating rent reform initiatives and the
outcomes produced absent and with incentives. Lastly, reviewing the research to date, one
proposed model for assessing these agencies’ innovative programs is offered, as is the
opportunity for HUD and PHAs to eliminate impediments to an effective analysis by suboptimal
data collection/handling. This will assist those evaluations underway and those yet to come and
guide policymakers while the MTW program is undergoing a significant expansion. The lessons
learned from this analysis benefits future researchers as, should other agencies engage in similar
data practices, the steps highlighted here to improve those practices will serve as a guide.
MAXIMIZING HOUSING EFFICIENCY
10
San Diego Housing Commission and Residents
The San Diego Housing Commission (SDHC) is one of the earliest approved
1
MTW
agencies. It has received local and national recognition for its innovative solutions to address
housing affordability, including its Path to Success program (San Diego Housing Commission,
2019). Nevertheless, the SDHC programs have undergone little analysis by an independent
evaluator before this effort. If it exists, the public availability of internal research is
understandably limited due to privacy concerns.
The San Diego Housing Commission (SDHC) is the public housing agency providing
programs and housing opportunities “for low-income and homeless individuals and families in
the City of San Diego” (San Diego Housing Commission, About Us, 2020, p. 1). The SDHC is
also a significant participant in the City of San Diego’s efforts to address homelessness and
preserve and create new affordable housing and, towards this effort, provides rental housing to
over 16,000 households. The data used in this analysis includes nearly all 16,000 SDHC resident
households over the last five years.
SDHC’s Path to Success Program and Achievement Academy
The SDHC Path to Success (PTS) program is “a comprehensive rent reform measure
designed to motivate and benefit Work-Able rental assistance participants while remaining
impact neutral to Elderly/Disabled households” (Dayal & Davis, 2018, p. 17). Approximately
6,500 of the SDHC’s 16,000 households participate in the Path to Success program (Gentry,
2018). The program attempts to incentivize work-able households to increase their income by
progressively instituting minimum rents (Dayal & Davis, 2018). While this may be the intent of
1
SDHC gave up its original 1998 MTW designation to regain it in 2009 (Khadduri,
Vandawalker, Cohen, Lubell, Buron, Freiman & Kean, 2014).
MAXIMIZING HOUSING EFFICIENCY
11
increasing minimum rent with income, it is not clear, based on the research on other means-
tested programs, that there is a reasonable basis for assuming that progressive minimum rents
incentivize households to increase their incomes.
Table 1
PTS Tiered Rent Table (Truncated)
Note. SDHC (2021). Moving Forward. Moving to Work Program: Annual Plan for Fiscal Year 2019. https://www.
hud.gov/sites/dfiles/PIH/document/SanDiegoFY19Plan.pdf. Truncated for simplicity and only includes years 8+,
one work-able household minimum rents.
A truncated version of the most recent SDHC Tiered Rent Table is included above.
Minimum rents are set for work-able families, and these rents increase as a household increases
its income. The rent payment amount is “either the minimum monthly rent payment amount or a
Annual Income Bands 0 – 6 Beds
$0 - $4,999 $400
$5,000 - $9,999 $400
$10,000 - $14,999 $400
$15,000 - $19,999 $400
$20,000 - $24,999 $500
$25,000 - $29,999 $625
$30,000 - $34,999 $750
$35,000 - $39,999 $875
$40,000 - $44,999 $1,000
$45,000 - $49,999 $1,125
$50,000 - $54,999 $1,250
$55,000 - $59,999 $1,375
$60,000 - $64,999 $1,500
$65,000 - $69,999 $1,625
$70,000 - $74,999 $1,750
$75,000 - $79,999 $1,875
$80,000 - $84,999 $2,000
$85,000 - $89,999 $2,125
$90,000 - $94,999 $2,250
$95,000 - $99,999 $2,375
$100,000 - $104,999 $2,500
$105,000 - $109,999 $2,625
MAXIMIZING HOUSING EFFICIENCY
12
predetermined amount of rent that is based on the family’s annual income, whichever is greater”
(San Diego Housing Commission, Path to Success, 2021, para. 7). As can be seen, the income
ranges that trigger rent increases are segmented into $5,000 ranges so that, according to the
SDHC, “Work-Able families [may] increase their income without being penalized” (San Diego
Housing Commission, Path to Success, 2021, para. 11). SDHC Section 8 Housing Choice
Voucher rental assistance families are enrolled in the PTS program; however, “there is no
minimum monthly rent payment amount for Elderly/Disabled families, which are households in
which all adult family members are 55 or older, disabled, or a full-time student age 18 to 23”
(San Diego Housing Commission, Path to Success, 2021, para. 5).
Figure 2
Average Income Under Path to Success
Note. MTW Collaborative. Moving to Work: Innovation and Flexibility to Address America’s Affordable Housing
Challenge (December 2020). https://65bcdb2c-4e6f-44c9-ae9b-f9100a326749.filesusr.com/ugd/15cee7_dbc3631b7
2b a44d399271512ebf96091.pdf.
MAXIMIZING HOUSING EFFICIENCY
13
According to the San Diego Union Tribune, “the [PTS] program has been successful,
with about two-thirds of people finding jobs and work-able households increasing their income
by an average of 13 percent” (Warth, SDUT, 2017). Self-reported data from a December 2020
report states, “the average annual income of households identified as able to work (Work-Able)
[had] increased 44 percent since the Path to Success initiative was implemented, beginning on
July 1, 2013” (MTW Collaborative, 2020, p. 12).
All residents in PTS receive access to the SDHC Achievement Academy (AA), “a
learning and resource center and computer lab located at SDHC’s headquarters in Downtown
San Diego” (San Diego Housing Commission, 2020, p. 1). This program provides “career
planning, job skills, and personal financial education” at no cost (San Diego Housing
Commission, 2020, p. 1). These resources are essential for any effort to increase access to
employment as “HUD households report that not having enough education or training makes it
difficult to find a job or find a better job” (Mazzara & Sard, 2018, p. 11). Under the umbrella of
the AA is the SDHC’s Family Self-Sufficiency (FSS) program. Like other FSS programs, this
program is supported by HUD and follows a federal model discussed in the following section.
Participation in AA or FSS is voluntary.
MAXIMIZING HOUSING EFFICIENCY
14
Figure 3
Relational Representation of SDHC Program Structure
Note. Created by Timothy Walsh based on multiple reports written by Dayal and Davis (2018 – 2020) and other
SDHC public information.
Family Self-Sufficiency Programs
As previously discussed, the United States Department of Housing and Urban
Development (HUD) has created various programs to increase employment among its federal
housing residents. To fairly evaluate the SDHC, this proposal reviews literature highlighting
those housing agencies that operate similar programs. While not always a component of an
MTW agency, Family Self-Sufficiency (FSS) programs are a standard part of many agencies’
labor participation strategy. Therefore, FSS and the literature evaluating it deserve a brief
overview.
FSS is a well-established self-sufficiency program originating with “Project Self-
Sufficiency (PSS), launched in 1984; Operation Bootstrap, launched in 1989; and FSS, which
San Diego
Housing
Commission
Path to Success
Achievement
Academy
SDHC's Family
Self Sufficiency
MAXIMIZING HOUSING EFFICIENCY
15
began in 1991 and continues today” (Riccio, 2007, p. 14). FSS serves families in the Housing
Choice Voucher Program, families living in public housing, and families living in HUD-assisted
multifamily developments. FSS is a voluntary program whereby enrolled families volunteer for
“a 5-year contract of participation (COP) with the PHA that specifies the steps both the family
and the PHA will take to move the family toward economic independence” (de Silva et al., 2011,
p. vii).
Once participants agree to their personalized self-sufficiency plan, they receive a case
manager, help with a variety of supportive services, and an interest-bearing escrow account that
collects some portion of their rent payment that would have been due to the PHA. Upon
completion of the program, the family receives the escrow funds with the accrued interest. These
funds are a substantial benefit to the graduates. The average escrow savings upon completing the
program is over $5,000 (de Silva et al., 2011, p. vii). SDHC FSS program participants can earn
up to $10,000 (San Diego Housing Commission, 2020, p. 1). While FSS is a stand-alone
program, “several [MTW] housing authorities have merged their public housing and HCV
[Housing Choice Voucher] FSS programs and altered the time frames associated with FSS”
(Webb et al., 2015, p. 3). Many “have merged separate self-sufficiency programs into an ‘MTW
FSS’ that allows the agency to streamline operations and alter procedures governing such
programs, like time limits” (Webb et al., 2015, p. 19). While the MTW program requires further
research to demonstrate its outcomes, FSS has undergone substantially more analysis, including
a systematic examination by HUD, which performed a retrospective analysis comparing FSS
participants to non-participants and found that FSS participants did much better financially
(Ficke & Piesse, 2004). While the current evaluation focuses on the MTW program and precisely
one housing agency’s programmatic efforts, FSS plays an essential role in creating positive
MAXIMIZING HOUSING EFFICIENCY
16
outcomes for residents and is often affiliated with MTW efforts. Therefore, analysis of FSS self-
sufficiency incentives may provide beneficial insights into other self-sufficiency programs'
potential outcomes and effectiveness.
Whereas the current literature demonstrates that Jobs Plus is associated with increased
positive outcomes, the analysis does not separate the impact of the tiered rent program from the
incentives provided. Similarly, the existing literature on FSS demonstrates that it is also
associated with increases in positive outcomes. However, FSS does not include a tiered rent
program, as with the SDHC PTS program. Using the AA (which includes the SDHC FSS) and
the PTS programs as case studies, this research will be able to measure the incremental impacts
an incentive program (AA) has compared to the tiered rents of PTS on their own.
Research Question
One crucial question must be addressed to evaluate the SDHC’s efforts, among SDHC
residents in the PTS program, do outcomes differ for those who are also in AA? Every housing
agency must report similar data to HUD. Therefore, analyzing these data will create a model for
similar analyses other agencies can perform with their self-reported data.
Statement of Hypothesis
Based on the following review, this research hypothesizes that if AA has the intended
impact, there should be improved outcomes (increased income, reduced days occupied, and
increased self-sufficiency as defined by moveout reasons) as compared to other PTS work-able
residents. With better outcomes, AA resident housing units should experience more housing
efficiency compared to the housing units of work-able residents in the PTS program and not in
AA. A core component of this comparison is that it separates outcomes produced by programs
that attempt to influence residents’ behavior in very different ways. AA is a service-based
MAXIMIZING HOUSING EFFICIENCY
17
program that provides rewards for participation. The PTS program “was designed to motivate
families to increase earnings” and encourage employment by progressively increasing rents
based on earnings (San Diego Housing Commission, Path to Success, 2021, para. 3). While AA
represents the classic incentive or “carrot” approach, the PTS program attempts to motivate
residents by making strategic rental changes based on earnings for work-able residents. While
these rent increases may be determined to be a disincentive for labor participation, they are not
the classic “stick,” as are the traditional work requirement programs.
Research Purpose – Labor Participation Programs and the Need for More Research
Congress has introduced an assortment of proposals and reforms to address a common
criticism of the public housing programs in the United States, “that they actively discourage
employment among program participants” (Rohe et al., 2016, p. 909). Generally, these proposals
and reforms assume that if public housing participants become employed or increase
employment and earnings, they are more likely to successfully exit public housing and reduce
their dependency on federal assistance.
One method of these reforms has been the creation of labor participation programs. Some
of these programs, specifically work requirements, have a troubled history. Past work
requirements for welfare programs were rife with bias and discrimination (Minoff, 2020).
Additionally, evidence of the success of these programs in increasing employment and wages is
limited. Scally et al. (2018) stated, "even though housing authorities have been experimenting for
many years with work requirements, minimum rents, and time limits little evidence is available
on how these programs have affected assisted households, housing agencies, participating
properties, and communities" (p. vii). Further, with HUD designating new MTW agencies, more
MAXIMIZING HOUSING EFFICIENCY
18
agencies may adopt new programs; therefore, "policymakers must be able to access more and
better information to guide policy in this area." (Levy et al., 2018, p. 9).
Below are four categories of commonly instituted programs at MTW agencies. Clearly
labeling these programs (even though there will be some variation) will assist future researchers
in evaluating outcomes.
1. Work Requirement Programs: These programs commonly “require work-able tenants
to work a certain number of hours or face sanctions, including loss of housing subsidy
and eviction” (Webb et al., 2015, p. 21). This approach makes engagement in work or
work activities a condition of benefits (McCarty et al., 2016).
2. Time Limit Programs: These programs limit eligibility to a specific duration and, in
some cases, include other requirements, such as employment. Agencies argue that a
time limit may incentivize participants to enter the workforce (McCarty et al., 2016).
3. Tiered Rent/Adjusted Rent or Rent Reform Programs: These programs are a
“commonly utilized MTW flexibility, with participating agencies adjusting rent
calculations to increase revenues, simplify administrative procedures, and/or promote
self-sufficiency” (Webb et al., 2015, p. 13). The rent adjustments are often structured
to allow some increased earnings and require a minimum rent from work-able
households without harming elderly/disabled households.
4. Incentive and Support/Resource-Based Programs: These programs include income
subsidies (like escrow accounts), direct payments, or other income/employment
supporting activities (vocational, educational, public health initiatives) meant to
facilitate employment. Incentive and Support/Resource-Based Programs are
structured to lessen the disincentives in means-tested programs or reward
MAXIMIZING HOUSING EFFICIENCY
19
employment (McCarty et al., 2016). These programs are often referred to as a
housing-plus approach. They operate from the belief “that housing should play an
important role in alleviating poverty and should help residents address other issues
limiting their social and economic opportunities” (Bratt, 2008, p. 100). This involves
providing housing “but also services to support resident families as well as initiatives
focused on the neighborhoods in which the housing is located” (Bratt, 2008, p. 100).
Research Problem
As demonstrated above, all programs attempting to increase earnings and labor
participation are not work requirement programs, and all programs are not equal in their intent or
outcomes. Therefore, the above descriptions/categorization will hopefully assist with some
standard of measurement for these programs’ outcomes and may help policymakers and assist
individual agencies in assessing and comparing their programs.
Addressing the previously discussed research question highlights two critical points of
evaluation and comparison:
1. Establish the rate of positive outcomes (increase in earnings and positive moveouts,
and potential reduction in length of stay) for a majority of the work-able SDHC
population (PTS participants) and AA participants exclusively. Creating clear
baselines provide a criterion for future internal comparison and, assuming that the
sub-populations served are adequate for such a comparison, may allow for
comparisons between the SDHC and other housing agencies.
2. Once baselines are established for primary outcomes, AA participants (those
receiving incentive-based reinforcement) in the PTS program will be compared to
PTS participants not enrolled in AA to determine whether there are statistically
MAXIMIZING HOUSING EFFICIENCY
20
significant different outcomes between the two populations. Should that be the case,
additional research may be warranted to determine the potential factors influencing
these various outcomes.
Addressing these critical points of evaluation addresses a knowledge gap for policymakers and a
current gap in the public housing literature on this subject.
Review of the Related Literature
This analysis reviews evaluations of a select group of public housing agency self-
sufficiency programs. It clarifies where the PTS and AA programs fall among these other
programs. Much of the research on MTW programs and, precisely, on self-sufficiency programs,
including those operated by MTW agencies, calls for additional research to better determine the
impact of these innovative efforts (Cadik & Nogic, 2010; Lee & McNamara, 2018; Levy et al.,
2018; McNamara et al., 2017; Rohe et al., 2016; Scally et al., 2018; Webb et al., 2016, Walter et
al., 2020; Riccio, 2007; Riccio et al., 2017). While some analysis of MTW programs exists, the
current evaluations are inconsistent in their use of measurement. The literature surveyed attempts
to build on these evaluations by focusing on comparable (MTW designated or FSS operating),
recent (published post-2000, although the data analyzed may be older) analyses of PHAs, and
their local programs. These parameters yield two categories of research, including recent and
important works that suggest some agencies demonstrate promising results. The two categories
include literature evaluating local MTW agency self-sufficiency programs and individual FSS
programs. This literature is discussed in greater detail in the accompanying literature review.
However, a summary is warranted here.
MAXIMIZING HOUSING EFFICIENCY
21
Evaluations of Local MTW Self-Sufficiency Programs
An evaluation by Rohe et al. (2016) of a local MTW work requirement program is a
foundational piece of literature in this area. Their assessment of the Charlotte Housing Authority
(CHA) work requirement program "presents the first empirical analysis of the impacts of a
public housing work requirement on employment” (p. 910). The authors concluded that there is
an increase in employment when work requirement enforcement, supportive services, and case
management are combined (Rohe et al., 2016, p. 922). Rohe et al. are at the forefront of this area
of research.
A rigorous evaluation of the Housing Authority of Champaign County (HACC) by
McNamara et al. (2017) "generate[s] lessons of how a local housing authority can promote
greater economic self-sufficiency via federal deregulations permitted under the Moving to Work
(MTW) Designation program" (p. 62). The authors concluded that participants in the HACC
Local Self-sufficiency (LSS) program, an incentive-based program, "experienced a substantial
increase in earnings and employment," and their results mirror those of Rohe et al. (2016)
(McNamara et al., 2017, p. 62).
Evaluations of FSS
Geyer et al. (2017), under the umbrella of Abt Associates, provided a report, funded by
the HUD Office of Policy Development and Research and the Oak Foundation, evaluating the
FSS programs in Lynn and Cambridge, Massachusetts. These programs are administered by
Compass Working Capital (Compass) in partnership with public housing agencies in those cities
(Geyer et al., 2017). The authors found that the participants of the local FSS "performed
substantially better than the applicable comparison groups in terms of (a) growth in earnings, (b)
reductions in welfare income, (c) growth in FICO Scores; and (d) reductions in credit card and
MAXIMIZING HOUSING EFFICIENCY
22
derogatory debt" (Geyer et al., 2017, p. v). While Lynn is not an MTW agency, Cambridge
Housing Authority is. Therefore, its inclusion is warranted.
The interim findings of an MDRC report on the NYC-Work Rewards demonstration
evaluated the effectiveness of its FSS by evaluating three groups: its FSS participants, its FSS
participants as well as an enhanced version of the program with special cash work incentives
(identified as FSS+), and a final group offered work incentives alone without participation in
FSS (Nuñez et al., 2015, p. ES-2). The report’s findings showed “that the FSS + incentives
produced large and statistically significant impacts on employment and earnings for voucher
holders who were not already working when they entered the program” (Nuñez et al., 2015, p.
ES-3).
Kleit (2004) compared the Youngs Lake Commons program operated by the King
County Housing Authority (KCHA) in Washington with other FSS programs. The Young Lake
Commons program assisted public housing residents in enhancing their income, skills, and job
retention and, eventually, “encourage[d] low-income families to move out of public housing into
the private housing market” (Kleit, 2004, p. 365). Results showed that participants were more
likely to increase their income than the comparison group and eventually exit public housing.
The Rockford Housing Authority FSS is one of the oldest in the country, established in
1992. An evaluation of the Rockford Housing Authority self-sufficiency program demonstrated
that graduates of the program showed “significant economic and housing benefits” (Anthony,
2005, p. 65). This evaluation is another piece of literature highlighting the benefits of FSS to
public housing residents if the program is completed.
These two categories of literature offer a model to evaluate the efforts of other programs
created by MTW agencies. These evaluations also show that there is currently no standard to
MAXIMIZING HOUSING EFFICIENCY
23
judge MTW agencies or self-sufficiency programs. While each review looks at some mixture of
employment and earnings, the empirical approach (and resulting comparability of evidence)
differs across studies, making comparisons challenging. While the research on FSS is far more
robust and demonstrates that well-structured programs can demonstrate a statistically significant
impact, the research on MTW programs is scant and less conclusive.
The final section of the accompanying review of relevant literature highlights
reinforcement mechanisms' impact on behaviors. This is an important topic to review as the two
programs offered by the SDHC, and evaluated by the Research Question at hand, are examples
of efforts to alter behavior. This section also highlights literature on programs with similar
incentive mechanisms that have been demonstrated to discourage work participation (not just
positive/negative incentives in general).
MAXIMIZING HOUSING EFFICIENCY
24
Chapter 2: Review of the Related Literature
This chapter provides a review of the literature in the affordable and public housing
sectors in the context of its contribution to the research question being studied: among San Diego
Housing Commission (SDHC) work-able residents in the Path to Success (PTS) program, do
outcomes differ for those who are also in the Achievement Academy (AA)?
The previous chapter briefly reviewed the literature establishing the current California
affordable housing shortage and the commonly proposed methods to address this shortage. It also
examined the relevant literature defining and evaluating Moving to Work (MTW) agencies, the
specific institution under evaluation (the SDHC and its PTS and AA programs), and the Family
Self-Sufficiency Program (FSS). Reviewing the limited literature in each category is essential to
place the research question being studied into the appropriate context. Specifically, an
understanding of these interrelated topics is necessary to demonstrate the existing gap in the
literature. For example, the FSS is a federally funded program that may be used by MTW
agencies and public housing agencies (PHAs) alike; however, for MTW agencies, it is often
operated in conjunction with the agency’s unique programs. Non-Moving to Work PHAs offer
FSS as a stand-alone incentive-based program.
This chapter reviews two categories of literature surveying similar housing agency
programs in the United States. Specifically, it includes only housing agencies with an MTW
designation, those operating some form of FSS, or both. This chapter also discusses behavioral
economics and provides a short review of relevant literature on positive versus negative
reinforcers' impact on behaviors. This is an important topic to review as the two programs
offered by the SDHC (evaluated by the research question) are examples of an incentive and
disincentive-based reinforcement mechanism. This section also highlights literature on the
MAXIMIZING HOUSING EFFICIENCY
25
outcomes of programs with similar incentive mechanisms that have been demonstrated to impact
participant behavior by discouraging work participation.
Comparable Evaluations of Two Categories of Public Housing Agency Programs
This review includes literature highlighting a select group of public housing agency
programs. It will clarify, for future analysis, how the outcomes and design of the SDHC's
programs, specifically the PTS and AA, compare to these programs. Much of the research on the
MTW program and, specifically, on self-sufficiency programs operated by MTW agencies calls
for additional research to better determine the impact of these innovative efforts (Cadik & Nogic,
2010; Lee & McNamara, 2018; Levy et al., 2018; McNamara et al., 2017; Rohe et al., 2016;
Scally et al., 2018; Webb et al., 2016; Walter et al., 2020). While some analyses of the programs
created by MTW agencies exist, the previous evaluations vary in their use of measurement and
outcomes. This literature review attempts to execute a well-defined comparison group.
Highlighting the current diversity in the analysis method between the comparable
literature demonstrates the lack of a uniform empirical approach in this field. While it is not
problematic per se that different studies use different empirical approaches, the existing
literature’s variety of methods might confuse those who want to conduct new evaluations
(potentially of their agencies). There is value in establishing a standard approach that can be
utilized across agencies. The literature surveyed focuses on comparable (MTW designated or
FSS operating), recent (published post-2000, although the data may be older) analyses of
housing agencies and their self-sufficiency programs. These parameters yield two categories of
recent and significant works suggesting that some agencies demonstrate promising results.
MAXIMIZING HOUSING EFFICIENCY
26
Evaluations of Local MTW Self-Sufficiency Programs
Charlotte Housing Authority
Rohe et al. (2016), in their evaluation of the Charlotte Housing Authority (CHA) work
requirement program, presented “the first empirical analysis of the impacts of a public housing
work requirement on employment and evictions” (p. 910). This evaluation of a work requirement
program is a foundational piece of literature in this area. The authors concluded that there is an
increase in employment when work requirement enforcement, supportive services, and case
management are combined. In contrast, these components alone do not produce similar results
(Rohe et al., 2016). In addition, there is some evidence that enforcing work requirements
increases positive moveouts while not increasing evictions (Rohe et al., 2016). Rohe et al. used
CHA administrative data, end-of-the-month data compiled by case managers, two population
surveys, and 43 in-person interviews between the treatment and control groups to reach their
results. Their analysis included linear regression testing the difference-in-difference and
difference-of-means to demonstrate the statistical difference in outcomes between the treatment
and control groups. Rohe et al. argued that the United States Department of Housing and Urban
Development (HUD) “should mandate” that current MTW agencies evaluate how their
innovative programs, like work requirements, impact their residents and that Congress should
pay for it (p. 924).
Housing Authority of Champaign County
A rigorous evaluation of the Housing Authority of Champaign County (HACC) MTW
program by McNamara et al. (2017) "generate lessons of how a local housing authority can
promote greater economic self-sufficiency via federal deregulations permitted under the Moving
to Work (MTW) Designation program" (p. 62). The authors concluded that participants in the
MAXIMIZING HOUSING EFFICIENCY
27
HACC Local Self-sufficiency (LSS) program "experienced a substantial increase in earnings and
employment," and their results are like those of Rohe et al. (2016) (McNamara et al., 2017, p.
62). However, households also “tended to suffer more from mental illnesses such as depression
and anxiety” (McNamara et al., 2017, p. 63). The HACC also “enforces participation in the
Family Self-Sufficiency program” for those in LSS and “noncompliant with the HACC work
requirement” (McNamara et al., 2017, p. 52).
McNamara et al. (2017) used Picture of Subsidized Households (PSH) data along with
HUD Form 50058 Family Report to compare and examine “longitudinal change in assisted
households’ employment status and earnings compared with similar variations of households” (p.
52). This data and social survey data “collected in collaboration with HACC and a comparison
PHA” established the treatment and control group. The authors used quantitative and qualitative
data to evaluate annual total income and employment changes over time.
Family Self-Sufficiency Programs
Lynn and Cambridge, Massachusetts
As discussed earlier, Geyer et al. (2017), under the umbrella of Abt Associates, provided
a report funded by the HUD Office of Policy Development and Research and the Oak
Foundation, evaluating the FSS programs in Lynn and Cambridge, Massachusetts. These
programs are administered by Compass Working Capital (Compass) in partnership with public
housing agencies in those cities (Geyer et al., 2017). The authors found that the participants of
the local FSS programs "performed substantially better than the applicable comparison groups in
terms of (a) growth in earnings, (b) reductions in welfare income, (c) growth in FICO
®
scores;
and (d) reductions in credit card and derogatory debt" (Geyer et al., 2017, p. v). While Lynn is
not an MTW agency, Cambridge Housing Authority is. This report is “the first evaluation of a
MAXIMIZING HOUSING EFFICIENCY
28
full FSS program to find statistically significant differences between the performance of FSS
participants and an applicable comparison group” (Geyer et al., 2017, p. v).
Using HUD Form 50058, the authors performed a quasi-experimental impact analysis by
comparing the experiences of the Compass FSS program to those in a control group made up of
similar voucher holders in Massachusetts, Connecticut, and Rhode Island (Geyer et al., 2017).
The authors also used data provided by Compass and the Experian Credit Bureau to track
changes in FICO scores and debt for the participants. Performing basic descriptive statistical
analysis measuring change over time in earnings, welfare and social security income, FICO
score, and debt levels, the authors quantified the outcomes of the FSS program and demonstrated
that participants performed better than the comparison group on these specific measures (Geyer
et al., 2017).
NYC-Work Rewards demonstration
The interim findings of the nonprofit and nonpartisan MDRC (known initially as
Manpower Demonstration Research Corporation) report on the City of New York Work Rewards
demonstration program (Opportunity NYC) evaluated the effectiveness of its FSS program by
assessing three groups: its FSS participants, its FSS participants as well as an enhanced version
of the program with special cash work incentives (called FSS+), and a group offered work
incentives alone without participation in FSS (Nuñez et al., 2015). The report’s findings showed
“that the FSS + incentives intervention produced large and statistically significant impacts on
employment and earnings for voucher holders who were not already working when they entered
the program” (Nuñez et al., 2015, p. ES-3). The group receiving incentives alone produced no
statistically significant effect. Further, the results “suggest that FSS by itself, at least in the first
New York City test, is not effective in improving employment, earnings, or aspects of material
MAXIMIZING HOUSING EFFICIENCY
29
well-being” (Nuñez et al., 2015, 137). However, the authors admit that the reader should be
cautious with their conclusions due to the small sample sizes used (Nuñez et al., 2015).
The comparison groups consisted of voucher holders in the New York City Department
of Housing Preservation and Development (HPD) and the New York City Housing Authority
(NYCHA). Participants were randomly assigned to the control and treatment groups using
various data sources from these two agencies and the program providers. Nuñez et al. (2015)
performed two parallel, randomized controlled trials measuring the impacts of FSS and the
increased incentives on participants. Outcome levels for all groups and subgroups were
calculated via the difference in mean and basic statistical analysis.
The Youngs Lake Commons Program
Kleit (2004) evaluated the Youngs Lake Commons Program operated by the King County
Housing Authority (KCHA) in Washington. The program assists public housing residents in
enhancing their income, skills, and job retention and, eventually, “encourage[s] low-income
families to move out of public housing into the private housing market” (Kleit, 2004, p. 364-
365). Kleit’s analysis compared Youngs Lake with other FSS programs through focus groups
and analysis of administrative data. King County eventually entered the MTW program in 2003,
but the period studied was before 2003. Some results showed the “program helped those who
were potentially the most likely to be able to leave public housing to do so, it also frustrated
some who found the goal [of leaving public housing] unattainable within … 3 years,” which was
the time allotted by the program (Kleit, 2004, p. 383). The program self-nominated its
participants engaging, in what the author labeled, "creaming,” or selecting participants that are
among those residents most likely to succeed in the program (Kleit, 2004, p. 383). While this
may produce biased estimates of the program’s impact, it is understandable that a public housing
MAXIMIZING HOUSING EFFICIENCY
30
agency may engage in such a selection process if it has the potential to create greater housing
efficiency. Further, the “on-site, intensive case management was crucial for many residents in
maintaining their motivation throughout the 3 years” (Kleit, 2004, p. 387). Lastly, participants
were more likely to increase their income than the comparison group, and most eventually did
leave public housing.
Rockford Housing Authority
The Rockford Housing Authority FSS is one of the oldest in the country, established in
1992. An evaluation of the Rockford self-sufficiency program demonstrated that graduates of the
program showed “significant economic and housing benefits” (Anthony, 2005, p. 65). The author
examined data on 135 successful and unsuccessful participants and explored factors affecting
program completion. Initial analysis of program data was that “there appeared to be no
significant differences between successful and unsuccessful [participants] based on race, age,
number of children, and marital status” (Anthony, 2005, p. 79). However, after regression
analysis, "five factors seem to have had significant effects on success”—younger age at entry,
being single at entry, having a high school diploma at entry, having a higher household income at
admission, and acquiring more skills during the program (Anthony, 2005, p. 81-84). This
evaluation is another piece of literature highlighting the benefits of completing an FSS program
for public housing residents.
The literature presented offers models to evaluate the efforts of MTW self-sufficiency
programs. These evaluations also make it apparent that no clear standard exists to judge MTW
programs and their associated self-sufficiency efforts. While most evaluation looks at some
mixture of employment and earnings, the empirical approach (and resulting comparability of
evidence) differs across studies, making comparisons challenging. However, each of these
MAXIMIZING HOUSING EFFICIENCY
31
studies calls for additional research on similar programs to better determine what program
characteristics are most likely to produce positive outcomes.
Behavioral Economics Research on Reinforcement Mechanisms
The impact on behavior of positive versus negative reinforcers has been heavily studied
in Economics, Psychology, and Policy. Therefore, it is essential to provide a short review of
relevant literature on the topic as PTS and the AA are examples of an effort to alter participant
behavior. Specific mention will be made of research in public housing and public subsidies.
Balch (1980) analyzed “four strategies that a government agency might use to change the
behavior of individuals” using two foundational theories in behavioral economics: Operant
Behavior Theory (B.F. Skinner) and Utility Theory (Charles Schultze) (p. 36). While Balch is
nuanced in his approach and acknowledged the role regulation and punishment could play in
behavior change, he highlighted the benefits of incentives and, with some appropriate
qualifications, stated that “incentives are the basis of the most reliable, efficient strategies” (p.
35).
Highlighting the benefits of incentives over disincentives, De Geest and Dari-Mattiacci
(2013) commented on the increasing tendency in the legal system to use carrots. The authors
justified this phenomenon by arguing the specific instances where carrots are more effective.
While the authors focused on legal systems, their arguments are structured for the lawmaker or
policymaker. De Geest and Dari-Mattiacci are also balanced in their analysis, stating that “in
simple settings, sticks are superior to carrots” and offering “a word of caution… ‘carrots versus
sticks’ is a complex problem that remains understudied” (p. 391-392). However, in complex
scenarios, “when the lawmaker faces specification problems, which means that she does not
know what to expect from each individual citizen” or “when the lawmaker needs to require
MAXIMIZING HOUSING EFFICIENCY
32
significantly higher efforts from some citizens than from others,” carrots are superior (De Geest
& Dari-Mattiacci, 2013, p. 345).
While the literature demonstrates the benefits of incentives to influence behavior change, the
specifics of the incentive and in what form they are provided may significantly impact the
effectiveness. Research evaluating incentives to work in France has produced additional analysis
suggesting housing benefits may create disincentives to work (Ferey, 2018). Similarly, standard
incentives, like money, sometimes fail. Kamenica (2012), to better understand the “anomalous
impacts of incentives,” offered “a coherent set of principles that can help improve the design of
incentive structures in a variety of settings” (p. 428). While the French tax/transfer system is,
obviously, not the same as that in the United States, and all monetary incentives are not equal, a
structural review analyzing the current incentives provided by AA and the increased rents required by
the PTS program would be beneficial to fully understand how income increases may be undermined
by increases in rents or may lead to a disincentive to increase participation in the workforce. For
example, recipients must clearly understand incentives (Eisenberger et al., 1999). Incentives
must be close in time to the desired behavior to be effective (Skinner, 1974), and trust in the
provider is a potential factor influencing the recipient's motivation (Benabou & Tirole, 2003). In
short, careful analysis should accompany any decision regarding including carrots or sticks in a
program meant to further public policy aims by impacting behaviors.
A clear example of careless policy implementation is included in the ample literature on
programs with incentive/sanction mechanisms that have unambiguously been demonstrated to
discourage work participation. The PTS element that increases rent as a function of income (with
the intent of encouraging employment) is analogous to a tax on earnings with a potential work-
discouraging effect. There is a strong parallel in the design of PTS to efforts with sanctions
MAXIMIZING HOUSING EFFICIENCY
33
applied to traditional welfare programs. Hasenfeld et al. (2004) argued that “the 1996 welfare
reform legislation expanded the use of sanctions under the assumption that welfare recipients can
comply with work requirements and that they can calculate costs and benefits of compliance” (p.
304). Hasenfeld et al., through a record- and survey-based study, found that sanctioned recipients
“face greater barriers to meeting the work requirements” (p. 304) as compared to their
nonsanctioned recipients. Most of the sanctioned recipients were not even aware they were
sanctioned, challenging the idea of these sanctions as valuable mechanisms for behavioral
change.
Similarly, Fording et al. (2013) compared the growth in earnings of non-sanctioned and
sanctioned TANF clients. The authors used difference-in-difference propensity score matching to
demonstrate “that sanctioning has a statistically significant negative effect on earnings among
TANF clients” (Fording et al., 2013, p. 641). These findings “suggest that sanctioning may serve
to undermine TANF’s goals of reducing welfare use and improving earnings in severely
disadvantaged families” (Fording et al., 2013, p. 641).
While sanctions in the PTS program may have other benefits for the agency, i.e., as a
revenue generator, similar programs have been shown to establish a work or earnings
disincentive. Several studies support “the hypothesis that housing subsidies do indeed lead to a
reduction in work effort” (Riccio et al., 2017, p. 3). This is highlighted in the MDRC report
Reducing Work Disincentives in the Housing Choice Voucher Program: Rent Reform
Demonstration Baseline Report (Carlson et al., 2012; Jacob & Ludwig, 2012; Mills et al., 2006;
Riccio et al., 2017; Shroder 2010). As mentioned in the report, “the disincentive effect can occur
because the benefit reduction acts as an implicit tax on earnings, so that every extra dollar earned
yields less than a dollar of extra income” (Riccio et al., 2017, p. 4). Therefore, individuals may
MAXIMIZING HOUSING EFFICIENCY
34
decide to forego work as the increase in income is not worth the additional labor. This is often
the argument for the inclusion of work requirements and work incentives: ‘“work requirements’
counteract the work disincentive of government benefits by requiring a recipient to engage in a
labor market-related activity—rather than a nonmarket activity” (McCarty et al., 2016, p. 2). A
different approach, “‘work incentives’ are a set of policies designed to reduce the high implicit
‘tax rate’ on earnings inherent in low-income assistance programs” (McCarty et al., 2016, p. 3).
A new and highly acclaimed approach beyond the classic incentive/disincentive paradigm
exists in recent research. Thaler and Sunstein (2009) highlighted how one could unconsciously
make better decisions by altering the decision-making environment or providing slight nudges or
cues to push one to decide without force. Societies and large institutions can use these nudges to
improve our communities and advance the public good. Thaler and Sunstein emphasized “choice
architecture” and its profound impact on our decision-making (p. 255). According to Thaler and
Sunstein, “choice architects can make major impacts to the lives of others by designing user-
friendly environments” ( p. 11). In short, effective programs plan well beyond carrot and stick
approaches and pursue more strategic approaches to designing decision-making processes that
encourage positive choices.
Thaler (2015) focused on the history of behavioral economics by retracing the
foundational research in the field and offered his perspective on the growth and evolution of
economic theory in decision-making, explicitly discussing implications for the field of finance
and applying behavioral insights to practical situations. While Thaler provided many examples
of errors in decision-making, whether at the University of Chicago or by the average individual
attempting to increase retirement savings, he focused on the concept of liberal paternalism and
the awareness that the average person cannot be counted on to make an optimal decision.
MAXIMIZING HOUSING EFFICIENCY
35
Therefore, while he acknowledged that providing nudges cannot solve all problems, it does help
individuals solve their problems more effectively (Thaler, 2015).
Summary
This chapter includes a review and comparison of the literature related to an evaluation of
a Moving to Work designated housing agency and, specifically, the impact of two programs on
housing efficiency and positive outcomes. This review contains a diverse survey of existing
literature, including federal agency-produced reports, government and nonprofit sponsored
analysis, and peer-reviewed journals. With the existing literature as a guide, this review identifies
a current gap in the research highlighted by two categories of the literature evaluating programs
comparable to the SDHC’s PTS and AA programs (Geyer et al., 2017; McNamara et al., 2017;
Nuñez et al., 2015; Rohe et al., 2016). Lastly, a summary of existing behavioral economic
research is included to frame the research question within the appropriate academic context.
MAXIMIZING HOUSING EFFICIENCY
36
Chapter 3: Methodology
This section describes the methodology whereby the researcher tests the previously
discussed hypothesis through a quantitative study. This study evaluates one agency's approach to
maximizing housing efficiency: the San Diego Housing Commission's (SDHC) Path to Success
(PTS) program and the incremental effects of the affiliated Achievement Academy (AA) in
conjunction with PTS. Much of the previous research on MTW (Moving to Work) agencies and,
precisely, on these types of programs operated by MTW agencies calls for additional research to
determine the impact of these innovative efforts.
While some evaluation of the labor participation programs of individual housing agencies
exists, the SDHC and its PTS program have undergone little analysis by an independent
evaluator before this effort, and the public availability of internal research, if it exists, is
understandably limited due to privacy concerns. Further, the previous review of the related
literature highlights that the existing research offers no uniformly accepted comparison metric
between other agencies or their programmatic efforts. Therefore, this analysis contributes to a
clear gap in existing research of MTW agencies by not only adding an evaluation of one of the
longest and most established MTW agencies while the MTW program is undergoing a
significant expansion but also by providing one evaluation framework for future evaluations of
other MTW agencies. Similarly, this evaluation identifies strategies for the SDHC, the United
States Department of Housing and Urban Development (HUD), and other Public Housing
Agencies (PHAs) to eliminate impediments to an effective analysis by suboptimal data
collection/handling, as identified here.
While previous evaluations look at some mixture of employment and earnings, the
empirical approach (and resulting comparability of evidence) differs across studies, making
MAXIMIZING HOUSING EFFICIENCY
37
comparisons challenging. While the research on Family Self-Sufficiency (FSS) programs is more
robust and demonstrates that well-structured programs can make a statistically significant
impact, the research on MTW programs is less conclusive.
Re-Statement of Hypothesis
This research hypothesizes that if AA has the intended impact, outcomes for AA
households should be improved (as defined by increased income, reduced days occupied, and
increased self-sufficiency via moveout reasons) compared to other PTS work-able resident
households. With improved outcomes, housing units occupied by these residents should
experience greater housing efficiency. As every housing agency must report similar data to
HUD, the analysis of these data will create a model for similar analysis other agencies can
perform with their self-reported data. Self-sufficiency programs, like AA, are becoming
increasingly more common among PHAs, especially MTW agencies. This research highlights
SDHC’s efforts toward greater housing efficiency by targeting the incremental impacts of AA
participation. The lessons learned from this analysis benefits future researchers as, should other
agencies engage in similar data practices, the steps highlighted here to improve those practices
will serve as a guide.
As other MTW agencies have analogous subprograms, this research could be adopted and
adapted to assist policymakers in reviewing these agencies/programs. For example, if incentives
produce positive outcomes at a greater rate than other programs, policymakers may have the
opportunity to redirect funding to a more productive approach. Should other analogous agencies
perform similar research, a potential metric for evaluation will allow the SDHC and other
agencies to compare outcomes while accounting for preexisting differences in the populations
MAXIMIZING HOUSING EFFICIENCY
38
they serve. The subjects, data collection/instruments, procedure/design, and data analysis of this
research methodology are discussed in this chapter.
Subjects
To find the best population to participate in this analysis, the author selected one of the
longest participating MTW agencies in a large urban area with a multi-year data tracking
program. The comparison population is composed of two groups a) SDHC work-able residents
that participate in PTS and b) those residents that participate in PTS plus the AA. The work-able
PTS residents included are five years (7/2016 to 9/2021) of residents in SDHC supervised
properties. By measuring the effects of AA on actual participants, this research informs how
participation in AA may affect the outcomes of other PTS participants or other work-able public
housing residents. This author does not presume the generalizability of these findings to all PTS
participants or all public housing agency residents at large but instead is providing easily
understandable and comparable information for future researchers and policymakers should the
specific comparison be warranted in some instances.
As stated by Mills and Gay (2019), “if the sample is well selected, the results of a study
testing that sample should be generalizable to the population” (p. 147). While the results of this
evaluation are demonstrable in the population studied, at best, they can only be considered as one
case study of two programs at one housing agency. The selection of these two groups was a
significant decision point in this analysis. Suppose these two groups, or more broadly, the
participants of SDHC programs, are not representative of other public housing agencies or
similar programs in those agencies. In that case, the stated results may not be generalizable.
While the optimal choice would be to obtain the total population of all public housing
participants throughout the United States in similar programs, this would be difficult, time-
MAXIMIZING HOUSING EFFICIENCY
39
consuming, and expensive. Therefore, this research offers a more feasible approach: to study the
population and programs within one housing agency and then use these results as a launching
point to expand similar research to other agencies and programs. This evaluation may also
contribute to the cohort evaluation currently taking place by HUD, adding additional evidence of
outcomes for consideration with one well-established agency.
This research begins with a simple descriptive analysis of the participants in the studied
programs (PTS and AA) while identifying any existing associations or relationships. Then, using
additional analysis (via linear regression) progresses toward something that plausibly reflects a
causal impact. By exploring various non-causal explanations for any observed difference in
mean, this research moves towards causality as much as possible within the limitations of data
and analytics methods. It is impossible to eliminate all potential causal factors, but the author
eliminates many factors to move towards causality. Policymakers and researchers require this
level of analysis to move forward with testing public policy solutions to address potential causal
impacts.
Data Collection and Instrumentation
The data management and secondary data analyses have been carried out using STATA
version 16.1. The author employed restricted-use administrative data (HUD-50058 MTW Report
and self-collected SDHC reporting data) to examine change in assisted households’ earnings,
days occupied comparisons, and moveout reasons within two overlapping SDHC programs,
specifically between those enrolled in the AA plus PTS and those work-able residents only
enrolled in PTS.
MAXIMIZING HOUSING EFFICIENCY
40
Data Sources
There are three separate data sources for this research. These data sources were provided
independently, downloaded to an external drive, and viewed securely at the SDHC headquarters
in downtown San Diego. This process ensured the appropriate protection of any personally
identifiable resident data. The data collection and analysis process required many site visits to
the SDHC as all data analysis was required to take place in a secure location under the review of
an SDHC staff member. The data were first evaluated to gain a descriptive understanding of the
data and its existing format. Then any inaccessible data were placed in a useable format, and a
data summary was produced and analyzed. Based on this review, multiple clarifying questions
were submitted to the SDHC to ensure a complete understanding of the data and its
representation.
The United States Department of Housing and Urban Development Form HUD-50058
MTW Family Report data is the first and most comprehensive data source (HUD MTW Family
Report, 2013). The SDHC maintains this outcome-based data as required by HUD. The data is
compiled in YARDI (a housing software program) and then transmitted to HUD as a monthly
submission.
The outcome-based data was included in 63 CSV files representing all the monthly
available data beginning in July 2016 and continuing until September 2021. Each CSV file
included at least 175 columns of data for every resident in SDHC housing. Each column
represented a potential variable to track on each CSV file and target for analysis. After careful
review, the following variables (as seen in Table 2) were identified as potential targets. It is
important to note that variables containing personally identifiable information were used solely
as descriptive variables allowing precise tracking of variables across multiple data sources.
MAXIMIZING HOUSING EFFICIENCY
41
Table 2
Target Variable Descriptions from 50058 Data
Variable Definition Variable Type
Descriptive Analysis
SSN Social security number X
First Name Resident first name X
Last Name Resident last name X
Relationship Head of household identifier X
Race Resident race X
Gender Resident gender X
Head Identity Unique number for each household X
Admission Date Date of admission into SDHC housing X
Zip Code Zip code of resident home X X
Age Resident age X
Family Size Number in family X
Reexamination Month Month of resident examination X
FSS Start Date Date of entry into FSS X
FSS Escrow Account Amount in the resident escrow account X
Program Name Enrolled program: ex. PTS, elderly /disabled, etc. X
Census Tract Census tract of resident home X X
Veteran Status Whether the resident is a veteran or not X
Annual Income Total annual household income X
AA Entry AA entry date X
AA Exit Date the resident left the AA X
Note. All variables are contained in 50058 data. However, the names used above are the author’s selection for ease
of understanding.
None of the output created by the data collected and included in the research contains
personally identifiable information. For example, the software used the social security number
and name of the resident to cross-check resident comparisons when a valid Head Identity
Number was missing. Similarly, variables containing information on resident race, location, or
MAXIMIZING HOUSING EFFICIENCY
42
gender are used as descriptive variables to understand the SDHC population being studied at
large and allow for evaluation of non-causal explanations. The researcher contacted no resident,
and none of the resident data evaluated was for the purpose of demonstrating the applicability of
these findings to an individual resident or a particular demographic group outside of what is
expressly stated herein.
The data clean-up process required significant time and review. Some of the variables
were inaccurate and were discarded, the format of other variables required manipulation, and
some data were missing and had to be re-requested. For example, the reporting format of the
50058 downloads (Excel reports) changed twice during the analysis period. These changes were
minor, with the addition of two unrelated variables (caseworker [11-2017] and homelessness
[09-2018] labels); however, these alterations required clarification and consideration during the
analysis process. Additionally, the collected data reported information from multiple SDHC
departments and personnel. Therefore, gaining a greater understanding of one variable often
required facilitated exchanges with various personnel at the SDHC. Lastly, as all data were
required to remain at the SDHC and a staff member was required to be present during the review,
each task required careful planning to ensure the limited time was used efficiently. In short, the
process spanned months of visits and repeated questions and clarifications through different
departments of the SDHC. While no SDHC staff or SDHC resources were expended
in collecting or analyzing this data beyond those that are de minimis, the SDHC staff are to be
commended for their kindness and efficiency.
It was discovered during the evaluation process that the SDHC 50058 data provided did
not have a comprehensive list of all AA participants. The total number of participants within the
date ranges of provided 50058 data files was 1183, while public data clearly showed AA
MAXIMIZING HOUSING EFFICIENCY
43
participants closer to 3000. Once aware of this gap, SDHC staff shared the existence of
additional data sources containing the same inputs. This additional data ensured accurate
tracking of AA participants, avoiding what would be a critical gap dis-allowing for an accurate
comparison, and providing additional data on resident reasons for moveout, which was limited in
the previous databases. The data were provided in two separate excel files: a “Demographics
Report” (demographic) and a “Moveout Report” (moveout).
As all participants are assigned a Head Identity Number, the following additional
variables were used to ensure all AA participants were accurately tracked while allowing for
further analysis on moveout comparisons between work-able PTS and AA populations. Below is
a list of the other target variables evaluated along with their definition:
Table 3
Target Variable Descriptions from Demographic and Moveout Data
Variable Definition Definition
Descriptive Analysis
Exit Violation Description of resident violation, if any X X
Exit Reason Reason for resident exit X X
Moveout Date Date of resident exit X X
Reason for Exit Reason for exit X X
Other Exit Reason Additional exit reason X X
Note. All variables are contained in demographic and moveout data; however, the names used above are the author’s
selection for ease of understanding.
An important point regarding the 50058, demographic, and moveout data is that individuals can
enroll, exit, and re-enroll in the AA. A new enrollment eliminates the old enrollment in the data,
and multiple different enrollment dates are possible with only the most recent being represented.
Therefore, for this research, participation in AA is solely measured by entrance into the AA and
not based on length of time or any other participatory measure.
MAXIMIZING HOUSING EFFICIENCY
44
Using the previously mentioned variables, the author created new variables necessary for
the proposed analysis and accurately defined outcomes. The new variables created include the
annual income for each resident, annual percent income change, and the number of days
residents occupied SDHC housing. All income measurements are inflation-adjusted as “real”
2021 dollars using CPI values.
Notably, all income variables excluded any work-able resident outcomes that contained
zero-income results. The SDHC informed the author that zero-income results often do not
represent a work-able resident’s actual income. Upon evaluation of a sample of annual reports of
zero-income for work-able residents, the SDHC conveyed that many zero-income reports were
likely to be data errors (e.g., mistakenly coded as zero instead of blank in years for which income
was not recorded) or not representative as a resident may have had a zero-income during the
reporting month, but shortly after that began earning income. While residents may have zero-
income for a period, it is unlikely the resident would be able to pay their portion of rent with
years of no income. Excluding zero-income data had a minimal impact on annual income or
annual percent income change for work-able residents. The impact of extreme values on the
income results is highlighted in the trimming process displayed in Ch. 4.
The demographic and moveout data include moveout information on over 4000 residents.
These outputs were categorized into positive reasons/increasing self-sufficiency, neutral reasons,
and negative reasons, as seen below:
MAXIMIZING HOUSING EFFICIENCY
45
Table 4
Moveout Reason Grouping
Positive Reason Negative Reasons Neutral Reason
Self-Sufficiency/Zero HAP Criminal activity Blank
Damage to unit Multiple subsidy
Debts owed Skip
Eviction Unauthorized household member
Failed to provide Unreported income
Lease violation Deceased
Missed inspection SBS-Eligible for Moving-on
Violent criminal activity SBS-other type of assistance
SBS-Received Section 8 voucher
VASH-Unable to locate/VA
Voluntary surrender
Voucher Expiration-Exhausted
extension
Voucher Expiration-No
RFTA/Response
Note. All moveout variables are contained in demographic and moveout data.
These three categories are based on the definitions provided by SDHC and, according to SDHC
staff, are influenced by the HUDUser Glossary (HUD.Gov, 2022). For example, while “eviction”
contains a specified HUD definition, “voluntary surrender” does not. According to SDHC staff,
“voluntary surrender” is an individual that willingly surrenders their Section 8 voucher.
Unfortunately, for these unspecified outcomes, such as voluntary surrender, the reasons behind
the surrender are not included in the data. For example, the surrender might be that the resident
has achieved self-sufficiency or moved in with family elsewhere. Hence, it is difficult to
categorize this outcome as positive, negative, or neutral. Analyzing these results based on AA
MAXIMIZING HOUSING EFFICIENCY
46
participation can provide additional insight into the impact of AA and its outcomes compared to
the PTS program. Nevertheless, upon detailed review, the SDHC data on moveout reasons does
not lend itself to analysis. This is discussed further in Chapters 4 and 5.
These data sources required substantial clean-up and multiple rounds of email, Zoom, and
in-person exchanges with SDHC staff to understand and clarify the shared data. Once this
process was complete, the researcher compiled each data source into one complete database.
This complete database simplified the process of performing analysis on the data to evaluate the
hypothesis presented.
Procedure/Design
The comprehensive file contains all target inputs organized to simplify analysis. The
annual income variable was collected each year at the Admission Date and annually after that at
the Reexamination Month. The demographic and moveout variables were added using the Head
Identity Number to match residents between the different data sources. Then the database was
coded to label work-able PTS and PTS plus AA populations. With the target populations labeled,
all AA income was organized to label income for each AA participant’s pre-enrollment in the
program and then listed annually.
Data Analytic Approach
With the data organized and coded, the researcher began the analysis process. The first
step was to produce full summary statistics for the SDHC population. Then analysis was
performed to describe the average SDHC resident and the average resident in the work-able PTS
and PTS plus AA programs. Moving on to comparisons between the two target populations,
annualized percentage change in income and average days occupied for all PTS residents in the
50058 files were calculated and compared. Any difference was highlighted, and, within existing
MAXIMIZING HOUSING EFFICIENCY
47
data limitations, fine grain comparisons were analyzed to understand those differences better.
Lastly, the researcher performed linear regression analysis to help determine if difference-in-
means for relevant characteristics potentially explained any discrepancies between the variables
examined (Anthony, 2005, p.79). This analysis highlights differences between the two
populations and begins to address the exact impact of participation in AA.
Descriptive Analysis
2
In total, the descriptive data covered 1,576 AA participating households (any household
with any participant in AA was considered an AA participating household), 10,034 PTS
elderly/disabled, and 6,714 PTS work-able head of household participants. The demographic and
household composition of SDHC heads of households vary. As presented in Tables 5 and 6, most
heads of household in the data studied are female, elderly/disabled, and more than 50% percent
of heads of households are White. Table 5 displays that the average age of all participants in the
sample was approximately 55 years, and the average number in the family was 2.47.
2
As stated previously, all data presented here includes all residents identified as “Head” of household in 50058 data
from 07/2016 to 09/2021.
MAXIMIZING HOUSING EFFICIENCY
48
Table 5
Summary Statistics 1: San Diego Housing Commission PTS Head of Households
Variable Obs. Mean S.D.
Age 18,324 55.09 15.42
Number in Family 18,324 2.47 1.76
Income 2016 6,781 24,455.54 15,137.90
Income 2017 13,084 21,480.96 13,776
Income 2018 13,040 21,982.89 14,909.44
Income 2019 12,832 22,136.81 15,731.02
Income 2020 13,936 22,162.52 15,100.59
Income 2021 10,496 21,050.95 14,367.67
Days Occupied 2,879 4775.10 2842.81
Note. This chart includes all PTS residents, including non work-able residents. Therefore, zero-income outputs are
included. Zero-income results are not included in any of the additional analyses of this research as those
comparisons include only work-able populations.
Table 6
Summary Statistics 2: San Diego Housing Commission PTS Head of Households
Variable Number Percent
Elderly 5,289 25.4%
Disabled 9,792 47.0%
Male 7,251 34.8%
White 11,434 54.9%
Race
Hispanic/Latino 6,509 31.25%
Black 6,354 30.5%
Asian 2,774 13.3%
American Indian/Alaskan 420 2.0%
Native Hawaiian 147 0.7%
Note. N=20,830.
To understand the size and distribution of the PTS population, Table 7 below presents the
participation for each PTS grouping in SDHC housing and its percentage.
MAXIMIZING HOUSING EFFICIENCY
49
Table 7
Head of Household Resident Participation
Note. The descriptive summary includes all residents identified as “Head” of household in 50058 data from 07/2016
to 09/2021. All categories above are exclusive of one another.
Summary
In this chapter, the researcher explained the data management required to get the datasets
and variables needed into the form required for analyses. A description of the population, target
variables, procedure/design, analytical methods followed, and a short descriptive summary of the
SDHC PTS population is provided. Any methodological issues are addressed in the following
results section to demonstrate validity.
PTS Grouping Resident Participants Per Year
2016 2017 2018 2019 2020 2021
Elderly/Disabled 2,374 7,133 7,236 7,222 7,921 6,348
34.8% 53.9% 54.6% 55.2% 55.2% 58.6%
Work-Able 3,662 4,947 4,830 4,691 5,113 3,554
53.6% 37.4% 36.4% 35.9% 35.6% 32.8%
AA 795 1,151 1,188 1,169 1,313 927
11.6% 8.7% 9.0% 8.9% 9.2% 8.6%
Total 6,831 13,231 13,254 13,082 14,347 10,829
MAXIMIZING HOUSING EFFICIENCY
50
Chapter 4: Results
This study provides an evaluation of one agency's approach to maximizing housing
efficiency: the San Diego Housing Commission's (SDHC) Path to Success (PTS) program and its
affiliated Achievement Academy (AA). The following research question is addressed to evaluate
the SDHC’s efforts: when compared to all SDHC work-able residents in PTS, do outcomes differ
among those who are also in AA? For the sake of analysis, the null hypothesis is that there is no
relationship (zero effect) corresponding to participation in PTS and AA versus PTS alone for any
outcome seen in the work-able population. In short, participation in AA would not produce better
outcomes. This research hypothesizes that if AA has the intended impact, there should be better
outcomes (i.e., increased income, reduced days occupied, increased self-sufficiency via
moveout) compared to other PTS work-able residents. With better outcomes, housing units
occupied by AA households should experience greater housing efficiency.
Population Demographics Differences
During the study period, an estimated 54% percent of SDHC PTS heads of households
were classified as Elderly/Disabled, 37% as work-able, and 9% percent of households enrolled in
AA (these percentages are exclusive of one another). Across work-able households,
approximately 24% of those enrolled in PTS also enrolled in AA. The annual distribution of the
PTS grouping participation is presented in Table 7 in the previous chapter.
Significant demographic differences between work-able and AA heads of households are
listed in Tables 8 and 9 below. AA heads of households are more likely than work-able
households to be younger (42.2 and 46.1, respectively). They are also more likely to be Hispanic
(49% and 43%), African American (38% and 33%), and American Indian (3% and 1%). AA
MAXIMIZING HOUSING EFFICIENCY
51
heads of households are less likely to be Asian (6% and 15%) and male (12% and 19%)
compared to work-able heads of households.
Table 8
Summary Statistics 1: AA and PTS Work-able Head of Households
Variable Mean
AA PTS-WA Diff
Age 42.2 46.1 -3.9***
Number in Family 3.78 3.76 0.02
Note. All totals are exclusive of one another.
∗
p < 0.1,
∗∗
p < 0.05,
∗∗∗
p < 0.01.
Table 9
Summary Statistics 2: AA and PTS Work-able Head of Households
Variable Percent
AA PTS-WA Diff
Male 12% 19% -7%***
Race
White
54%
51%
3%***
Hispanic/Latino 49% 43% 6%***
African American 38% 33% 5%***
Asian 6% 15% -9%***
American Indian/Alaskan 3% 1% 2%***
Native Hawaiian 1% 1% 0%
Note.
∗
p < 0.1,
∗∗
p < 0.05,
∗∗∗
p < 0.01.
The Difference in Earnings Between PTS and AA – Real Income
The analysis period (July 2016 to September 2021) was selected as July 2016 was the
earliest date the SDHC began collecting data via the 50058 data on AA participation. September
2021 was the first opportunity this author had access to the data. In the current study, AA
households are the treatment group, and PTS work-able households not enrolled in AA are the
MAXIMIZING HOUSING EFFICIENCY
52
comparison group. The impacts of receiving housing-plus incentives through AA participation
(e.g., job training, resume assistance, financial education) is measured by evaluating the mean
difference in earnings of the AA population compared to PTS work-able households not enrolled
in AA. It is important to note that, since the total annual income could contain various income
sources, including an escrow account due to participation in the Family Self-Sufficiency (FSS)
program, this increased income does not necessarily indicate household gains in employment-
based earnings. As previously stated, all income results are inflation-adjusted for real 2021
dollars per the CPI. The mean real income of PTS and AA populations during the period studied
is reflected in the table below.
Table 10
3
Mean Income AA and PTS Work-able Households
Year of Income AA PTS-WA Diff
2016
$27,508
($561)
$28,925
($273)
-$1,417**
2017
$27,513
($491)
$27,853
($232)
-$340
2018
$28,769
($511)
$29,100
($256)
-$331
2019
$29,635
($516)
$29,234
($289)
$401
2020
$29,833
($514)
$29,176
($253)
$657
2021
$29,857
($646)
$27,945
($288)
$1,912***
Note. (S.E.),
∗
p < 0.1,
∗∗
p < 0.05,
∗∗∗
p < 0.01.
3
As mentioned in Ch. 3, these income totals as well as those discussed throughout this analysis omit zero-income
observations.
MAXIMIZING HOUSING EFFICIENCY
53
While PTS work-able residents appear to have a higher initial mean income at the onset
of the study, the AA population’s mean income is higher over time. However, this approach to
measurement is problematic. First, this measurement may not account for the extreme values
within the two populations that alter the outcomes, as shown by the minimum and maximum
incomes listed in Table 11. These extreme values are especially concerning as many likely
represent data errors and/or otherwise nonrepresentative values.
Table 11
4
Mean Income with Minimums and Maximums: AA and PTS Work-able
Year of Income AA PTS-WA
Mean Min Max Mean Min Max
2016
$27,508
($561)
$185 $99,870
$28,925
($273)
$265 $144,617
2017
$27,513
($491)
$178 $129,820
$27,853
($232)
$531 $128,839
2018
$28,769
($511)
$174 $120,506
$29,100
($256)
$518 $156,638
2019
$29,635
($516)
$432 $130,455
$29,234
($289)
$220 $668,320
2020
$29,833
($514)
$427 $161,070
$29,176
($253)
$419 $198,175
2021
$29,857
($646)
$1,860 $148,693
$27,945
($288)
$456 $217,113
Note. (S.E.).
Second, measuring the difference in real income of the two populations produces a result that is a
difference of income amongst different residents across the years analyzed. Therefore, it may not
indicate real income growth for any household.
4
As mentioned in Ch. 3, these income totals as well as those discussed throughout this analysis omit zero-income
observations.
MAXIMIZING HOUSING EFFICIENCY
54
Accounting for Extreme Values
While the results above appear illustrative and have been publicly represented similarly
(Dayal & Davis, 2018; Warth, 2017), the two problems mentioned (extreme values and the
limitations of real income as a measurement) must be addressed. A preliminary data review was
performed to understand the extreme values better, and the results showed that many of the
extreme values were likely data errors. For example, one resident had income outputs for every
year from 2016 to 2021; however, 2016 income was reported as nearly $144,617, while income
did not exceed $12,000 annually for the following five years. Upon further evaluation with
SDHC staff, the 2016 reported income was an error.
To determine the frequency and scope of these errors, the researcher produced a sample
of the most extreme outputs and evaluated each with SDHC staff. Based on this review, three
common scenarios appear to be present in the data: a) some of the reported extreme values are
data errors, b) some are individuals that have achieved self-sufficiency but have yet to exit
SDHC housing, and c) some are very different from others (e.g., a resident that received a one-
time inheritance or households with a considerably greater number of work-able adults). While
extreme values are not always problematic, they are a concern when their presence in the data
substantially affects the results. Ideally, all errors would be removed from the existing data, but it
was infeasible to identify all erroneous values. Moreover, the primary purpose of the research is
to provide a guide for other housing agencies that may face similar data challenges. Therefore, a
practical and easily reproduced approach was selected after spot-checking for diagnostic
purposes, and confirming errors were present. This research addresses these extreme values by
trimming the data. Trimming the data is accomplished by eliminating observations above and
MAXIMIZING HOUSING EFFICIENCY
55
below the selected percentiles within each group by year. As shown in Table 12, the results
include the full population and trimmed outcomes at the 1st/99th and 5th/95th percentiles.
MAXIMIZING HOUSING EFFICIENCY
56
Table 12
Real Income: Untrimmed, 1/99% Trimmed, and 5/95% Trimmed
Untrimmed 1/99% 5/95%
AA PTS-WA Diff AA PTS-WA Diff AA PTS-WA Diff
2016 $27,508 $28,925 -$1,417** $27,262 $28,610 -$1,348** $26,622 $27,989 -$1,367**
($561) ($273) ($535) ($255) ($473) ($227)
[786] [3,651] [772] [3,579] [708] [3,289]
2017 $27,513 $27,853 -$340 $27,112 $27,478 -$366 $26,491 $26,784 -$293
($491) ($232) ($454) ($215) ($401) ($189)
[1,131] [4,931] [1,110] [4,833] [1,019] [4,441]
2018 $28,769 $29,100 -$331 $28,377 $28,599 -$222 $27,475 $27,628 -$153
($511) ($256) ($476) ($232) ($414) ($201)
[1,166] [4,807] [1,144] [4,711] [1,053] [4,358]
2019 $29,635 $29,234 $401 $29,228 $28,656 $572 $28,434 $27,856 $578
($516) ($289) ($476) ($232) ($415) ($200)
[1,136] [4,661] [1,114] [4,569] [1,027] [4,195]
2020 $29,833 $29,176 $657 $29,344 $28,675 $669 $28,549 $27,834 $715
($514) ($253) ($466) ($229) ($407) ($199)
[1,268] [5,054] [1,244] [4,954] [1,142] [4,550]
2021 $29,857 $27,945 $1,912*** $29,307 $27,507 $1,800*** $28,306 $26,730 $1,576
($646) ($288) ($585) ($266) ($500) ($233)
[886] [3,489] [870] [3,426] [798] [3,141]
Note. (S.E.),
∗
p < 0.1,
∗∗
p < 0.05,
∗∗∗
p < 0.01.
MAXIMIZING HOUSING EFFICIENCY
57
Viewing trimmed real income demonstrates that the mean income differences do not
appear highly sensitive to trimming. This is not particularly surprising, as the trimmed values of
real income are relatively few in number and not so exceptionally far from the typical range of
values that their presence substantially and differentially alters the group means. For example,
one person’s higher income has a negligible effect on the mean of a large group when evaluated
by real income. However, the second problematic issue raised earlier must still be addressed; a
better measure is required than real income.
The Difference in Earnings Between PTS and AA – Income Growth
The following analysis includes the year-on-year percent change in income for each
household, reflecting income growth over time. Table 13 illustrates the mean of household-level
percent change and the difference in means between AA and PTS work-able households. As
shown below, income for the AA population grew at annual average rates from 25-39%
throughout the period studied, while the PTS work-able population grew at annual average rates
from 14-25% (adjusting all income for inflation via CPI). Participation in AA was associated
with higher mean rates of household income growth than participation in PTS alone. However,
the large standard errors reported below in the years 2018-2019 and 2020-2021 and the
significant range of minimum and maximum income outputs in Table 11 highlight the need to
address extreme outputs in these years.
MAXIMIZING HOUSING EFFICIENCY
58
Table 13
Percent Annual Income Growth AA and PTS Work-able Households
Year of Income AA PTS-WA Diff
2016-2017 25.92% 20.97% 4.95%
(3.99%) (1.91%)
2017-2018 30.00% 25.00% 5.00%
(4.39%) (2.77%)
2018-2019 37.27% 17.78% 19.49%**
(12.90%) (2.60%)
2019-2020 25.33% 16.37% 8.96%**
(3.99%) (1.92%)
2020-2021 38.72% 13.79% 24.93%**
(18.36%) (2.80%)
Note. (S.E.),
∗
p < 0.1,
∗∗
p < 0.05,
∗∗∗
p < 0.01.
A percentage growth of income measure will be more susceptible to extreme values as
percentage growth is a relative measure. While the sample size is only slightly smaller, when the
average growth of a group is only a few percent, one extreme value with an increase of more
than 100% can make a significant impact. Therefore, viewing the result of the trimming process
by percentage income change is likely to demonstrate a more pronounced effect of these extreme
values.
By viewing Table 14, the impact can especially be seen in the percent income change in
two years. In 2019, the full population output (untrimmed) presented a difference in income
growth of 19.49%, but with the 1/99% level trimmed, the difference is reduced to 7.02%.
Similarly, the 2021 output of the full population presents a difference in income growth of
24.93% but trimmed at the 1/99% level, the difference is reduced to 6.77%. Income growth, as it
compares residents to themselves over time, is a better measure of the impact of AA and, as seen
MAXIMIZING HOUSING EFFICIENCY
59
below, presents a significant difference between the groups. This outcome would have been
missed without this approach.
MAXIMIZING HOUSING EFFICIENCY
60
Table 14
Percent Income Change: Untrimmed, 1/99% Trimmed, and 5/95% Trimmed
Untrimmed 1/99% 5/95%
AA PTS-WA Est. Diff. AA PTS-WA Est. Diff AA PTS-WA Est. Diff.
2017
25.92% 20.97% 4.95% 19.42% 13.94% 5.48%** 11.31% 7.66% 3.65%**
(3.99%) (1.91%) (2.60%) (0.99%) (1.52%) (0.58%)
[732] [3,310] [718] [3,244] [660] [2,980]
2018
30.00% 25.00% 5.00% 21.15% 14.38% 6.77%*** 11.39% 8.36% 3.03%**
(4.39%) (2.77%) (2.48%) (0.85%) (1.35%) (0.49%)
[1,045] [4,404] [1,025] [4,316] [941] [3,964]
2019
37.27% 17.78% 19.49%** 17.56% 10.54% 7.02%*** 10.22% 5.88% 4.34%***
(12.90%) (2.60%) (2.08%) (0.72%) (1.23%) (0.43%)
[1,051] [4,350] [1,031] [4,264] [947] [3,916]
2020
25.33% 16.37% 8.96%** 16.85% 9.42% 7.43%*** 7.88% 4.58% 3.30%***
(3.99%) (1.92%) (2.33%) (0.76%) (1.22%) (0.43%)
[1,056] [4,359] [1,036] [4,273] [952] [3,925]
2021
38.72% 13.79% 24.93%** 12.14% 5.37% 6.77%*** 4.90% 0.50% 4.40%***
(18.36%) (2.80%) (2.33%) (0.86%) (1.35%) (0.51%)
[811] [3,164] [795] [3,102] [731] [2,848]
Note. (S.E.),
∗
p < 0.1,
∗∗
p < 0.05,
∗∗∗
p < 0. 01.
MAXIMIZING HOUSING EFFICIENCY
61
While the differences between the groups sometimes change noticeably, the within-group
means themselves are affected by trimming (even in cases where the differences between groups
did not significantly change). This demonstrates that both groups were similarly and dramatically
affected by the trimming. The results from 2020 in Table 14 are a representation of this impact.
At the 1/99% timed level, the difference between AA and PTS is only reduced by 1.53%, but the
difference within each group is reduced by over 8.48% and 6.95%, respectively. These are
significant reductions.
This is a major contribution to this research and demonstrates two strengths of this
approach. First, it indicates that, while household participation in AA is associated with higher
income growth rates compared to PTS work-able households not enrolled in AA, the growth may
be less than the previous analysis suggests. Second, this approach addresses the data challenges
that are likely inherent in other sizeable public housing agencies by addressing the issue of
extreme values relatively simply, thereby increasing the feasibility and generalizability of this
method. In short, housing agencies can easily replicate this approach to reduce the impact of
extreme values without having to expend substantial effort and time addressing existing data
limitations. Third, as previously mentioned, this approach presents a better measurement of the
difference in income growth between the comparison groups. Measuring the rate of income
growth accounts for individual income growth. In contrast, the real income measure represents
the mean difference between two groups whose compositions change from year to year (i.e.,
mean difference between non-fixed groups). Therefore, real income is a less informative measure
and, as can be seen in Table 12, presents no meaningful difference between the two populations
evaluated. Income growth provides a more useful measure by enabling households to be
MAXIMIZING HOUSING EFFICIENCY
62
compared to themselves and, as seen in Table 14, demonstrates a significant difference between
the two populations.
The two boxplots below visually demonstrate the distribution of incomes and income
growth of the two populations. These boxplots display the typical representation of the normal
set of values. As viewed below, the growth remains, but, again, they are far less than they
initially appear when presented in real dollar amounts. The representation below provides an
additional indicator that income growth is a more volatile representation of the associated impact
of AA on income.
Figure 4
Gross Annual Income Boxplots
Note. All original income data are contained in 50058. Income is adjusted to reflect CPI 2021 “Real” income.
MAXIMIZING HOUSING EFFICIENCY
63
Figure 5
Income Growth Boxplots
Note. All original income data are contained in 50058. Income is adjusted to reflect CPI 2021 “Real” income.
The boxplots above represent the data's 25
th
percentile, median, and 75
th
percentile. As
shown in Figure 4, the median is easily observed near the center of each boxplot, while in Figure
5, the median is not visible or near the 25
th
percentile of the boxplot. This eliminates the distinct
left half of the box. The whiskers of each boxplot extend to the highest/lowest observations.
While the gross annual income boxplot represents a similar distribution between AA and
PTS incomes, the income growth boxplot highlights the difference between the two populations
in the 75
th
percentile. The 25
th
percentiles of the two boxplots are similar, but the 75
th
percentiles
of the AA population extend beyond the PTS work-able population every year presented in the
income growth boxplot. This raises some valid questions, such as whether the upper half of AA
residents demonstrate higher income growth compared to other PTS work-able residents or
-40% -20% 0% 20% 40% 60%
Real Income Growth (%)
2021
2020
2019
2018
2017
PTS+AA
PTS Work-Able
PTS+AA
PTS Work-Able
PTS+AA
PTS Work-Able
PTS+AA
PTS Work-Able
PTS+AA
PTS Work-Able
excludes outside values
MAXIMIZING HOUSING EFFICIENCY
64
whether there is a take-up issue for those in AA that remain in the program but do not improve.
Again, the visual representations above indicate that additional analysis is warranted.
Additional analysis (via linear regression) is necessary as there exist limitations in the
research done so far. Specifically, the descriptive analysis does not account for differences
between the PTS work-able and AA populations. The previous analysis may reflect pre-existing
differences rather than causal effects of AA participation. Regression analysis addresses these
limitations by controlling for differences between the groups regarding the specified covariates.
Regression analysis allows for the opportunity for income growth demonstrated above to be
analyzed in a manner that, should it continue to be associated with participation in AA,
progresses toward something that plausibly reflects a causal impact of AA.
Regression - Income
Regression analysis models the relationship between independent (predictor) variables
and dependent (outcome) variables, which makes it possible to make predictions about the
impact that predictor variables have on the outcome. For this reason, a linear regression was
estimated to demonstrate the statistical difference in estimated outcomes between AA
participants and other PTS work-able households. The independent variable of interest is
participation in AA. The outcome variables are income throughout each year of the study (2016 -
2021). The control variables are age, age squared, sex, ethnicity, race (White, Hispanic/Latino,
African American, Asian, American Indian/Alaskan, and Native Hawaiian), and number in
family. Notably, the number in family variable is treated as categorical (i.e., included
nonparametrically, without imposing any functional form on the relationship between family size
and income).
5
5
As a robustness check, all regression models were re-estimated with zip code as an additional categorical regressor.
In no case were the results meaningfully affected by this addition.
MAXIMIZING HOUSING EFFICIENCY
65
For the analysis performed here, six regression models were estimated. The six models
use two different outcomes under three different approaches. Each model uses Ordinary Least
Squares regression (OLS). OLS is a common technique for estimating coefficients of linear
regression equations, which describe the relationship between one or more independent variables
and a dependent variable. Regression allows the comparisons between these two groups to be
adjusted for other differences that might otherwise bias the results.
The two outcomes are real income and the year-over-year rate of income growth as a
percent. Each of these outcomes is presented with three specifications:
1) The full population (untrimmed)
2) 1/99% Trimming
3) 5/95% Trimming
As a reminder, trimming the data is accomplished by producing mean income results for the
populations and then trimming (eliminating) observations outside some specified percentiles
within each group by year (for example, below 1% and above 99%).
As seen below, Table 15 presents the linear regression results investigating whether there
is a significant relationship between participation in AA and residents’ real income while
controlling for age, sex, race, number in family, and year of income. When evaluating the real
income, AA participation is associated with approximately $300-$400 higher income regardless
of the degree of trimming applied. However, this estimated difference in none of the
specifications is statistically significant (p =0.286 to 0.348). In other words, after controlling for
factors such as age, sex, race and ethnicity, and the number of individuals in the family, AA
participants have approximately $300 higher real income on average than other PTS work-able
households.
MAXIMIZING HOUSING EFFICIENCY
66
Table 15
Real Income Regression
Full Population 1/99% Trimmed 5/95% Trimmed
AA Participation
379.30
397.78
326.90
(403.90)
(373.18)
(323.65)
[32,966]
[32,326]
[29,721]
Note. Coefficients, (S.E.), and [N].
∗
p < 0.1,
∗∗
p < 0.05,
∗∗∗
p < 0.01.
Table 16 presents the results of a linear regression investigating whether there is a
significant relationship between participation in the AA and residents’ income growth (as a
percent) while controlling for age, sex, race, number in the family, and year of income. When
evaluating the income growth, AA participation is associated with a 10.58 percentage point
higher income growth rate at the full population level. As the level of trimming increases to
1/99% and 5/95%, estimated differences are slightly less at 5.53 percentage points and 3.18
percentage points, respectively. While the magnitude decreases, the finding of a positive
coefficient holds. Further, the statistical significance improves when moving from the full
sample to the trimmed samples; and the standard error decreases dramatically with trimming,
which is not surprising as the AA coefficient is estimated far more precisely when observations
with extreme values are omitted. These models show AA participation to be a robustly
significant predictor of income growth.
While other controls like sex had a significant relationship with income, other racial
identifications, age, and number in the family did not.
MAXIMIZING HOUSING EFFICIENCY
67
Table 16
Income Growth Regression
Full Population 1/99% Trimmed 5/95% Trimmed
AA Participation
0.1058** 0.0553*** 0.0318***
(0.0484) (0.0100) (0.0058)
[24,282] [23,804] [21,864]
Note. Coefficients, (S.E.), and [N].
∗
p < 0.1,
∗∗
p < 0.05,
∗∗∗
p < 0.01.
Summary of Income Results
Accounting for age, sex, race, year of income, and number in family, income growth
among AA participants is significantly higher compared to other PTS work-able households in
the analysis summarized in Table 16. Evaluating the outcomes above and comparing the results
presented by real income and income growth, there are clear reasons to view the income growth
regressions as the most compelling analysis. Real income, as it represents an average difference
between groups with changing membership, is the lesser of the two approaches. As stated
previously, analyzing income growth compares residents to themselves over time, which is a
better measure of the impact of AA and a significant difference compared to the models
previously presented. Whether or not a difference is seen between the group means, being in AA
is indicative of higher income growth. This conclusion would have been missed by an analysis
that restricted its attention to mean income levels by group.
Although demographic differences exist between the AA and PTS work-able groups, the
regression analyses demonstrate that these differences cannot explain the higher rate of income
growth observed for AA participants. These results warrant additional research on AA and
incentive-based programs to evaluate whether participation in incentive-based programs (often
referred to as Housing Plus) positively impacts income growth. Including results from trimming
assists with extreme value sensitivity and approximates what cleaning the data would achieve,
MAXIMIZING HOUSING EFFICIENCY
68
albeit in a manner that is more feasible given the typical resources of a public housing agency. If
data issues were addressed, trimming would be superfluous. Trimming is easy to replicate and is
an efficient way for the SDHC and other public housing agencies with similar data challenges to
approximate results without extensive data clean-up efforts.
During the review of residents’ incomes, a possible trend was identified. While
attempting to understand extreme values within the data, there were a few residents who, upon
gaining an increase in income (leading them to no longer qualify for a rental subsidy), soon after
(within the allowed 6-month grace period) demonstrated a reduction in household income to a
level that continues to qualify them for supported housing. A similar trend is seen where an
increase occurs in household income due to additional adult members in the household.
Similarly, upon required disclosure, those adults exit the property so that the Head of Household
will not incur a rental increase due to an associated rise in household wage. This trend is
discussed further in Ch. 5.
Days Occupied Results
The Difference in Days Occupied Between PTS and AA
To understand the impact of AA more clearly, the researcher attempted to compare the
change in days occupied by AA households to SDHC PTS work-able households not enrolled in
AA. With such a comparison, it would be possible to determine if resident households enrolled
in AA demonstrated a significant difference in their length of stay in SDHC housing. This
outcome would be an additional variable to evaluate if participation in AA impacted housing
efficiency. Unfortunately, upon review of the existing data, it was clear the data offers limited
value and should be expanded before any analysis. Specifically, days occupied are only
calculated for individuals who have moved out of SDHC housing, and those totals likely omit
MAXIMIZING HOUSING EFFICIENCY
69
any past stays in SDHC housing if they have taken place. In effect, there are unknown values for
any households still in SDHC, which is a substantial portion of the sample. Therefore, days
occupied is a retrospective variable and not immediately useful for this analysis.
Moveout Reasons
Differences in Moveout Reasons Between AA and PTS
Like the days occupied analysis, data on moveout reasons would highlight significant
differences in residents’ reasons for exiting the program, producing an additional variable to
evaluate when considering if participation in AA impacted housing efficiency. Unfortunately, the
SDHC data on moveout reasons is incomplete and does not lend itself to analysis; 96% of the
outputs lack any description, and many descriptors lack clarity in the exact reason for the
moveout. Therefore, it is often impossible to know whether a reported move was for increased,
decreased, or neutral self-sufficiency reasons. Nonetheless, the current study has identified a gap
that can inform future data collection efforts that will allow for analysis. These findings provide
an opportunity to give targeted recommendations to the United States Department of Housing
and Urban Development (HUD) and the SDHC on better tracking the reason for moveouts.
Summary
These quantitative results demonstrate that when evaluated via regression and
considering income growth, AA program participants are associated with higher growth in
earnings. However, the growth in earnings is only seen when accounting for extreme values and
measuring the rate of income change over the same population year-on-year. Notably, the growth
in earnings is far less than would be represented if trimming was not performed.
This approach is easily generalizable to other housing agencies as it is feasible to
accomplish whether the agency has staff with a high level of statistical or quantitative training or
MAXIMIZING HOUSING EFFICIENCY
70
not. In short, housing agencies can easily replicate this analysis method to reduce the impact of
extreme values without having to expend substantial effort and time addressing existing data
limitations. No meaningful change in days occupied or moveout reasons were observed as the
existing data provided limited use for evaluation and, therefore, is not sufficient for the current
analysis.
MAXIMIZING HOUSING EFFICIENCY
71
Chapter 5: Limitations, Recommendations, Future Research Opportunities
Public Housing Agencies (PHAs) are increasingly adopting programs that focus on
enhancing residents’ self-sufficiency, hopefully increasing the efficiency of existing affordable
housing. These programs attempt to improve resident household income to increase positive
moveouts to market-rate housing. While positive moveouts are a measure of success for the
resident, reflecting newfound self-sufficiency, they are also vital to making space in existing
housing units for others in need of affordable housing, thereby improving housing efficiency.
This study consisted of an evaluation of one agency's approach to maximizing its housing
efficiency: the San Diego Housing Commission's (SDHC) Path to Success (PTS) program and its
affiliated Achievement Academy (AA) by testing the hypothesis that if AA has the impact
intended, there should be significant measurable income growth among the AA participant
households as compared to other PTS work-able residents. As previously mentioned, increasing
resident income is a necessary component of achieving self-sufficiency, which is essential for
residents to positively exit affordable housing to a market rate home. As demonstrated by the
quantitative results in Chapter 4, AA participation is associated with higher income growth rates
than the SDHC work-able population (PTS alone).
Limitations
Despite the strength of the evaluation, there are limitations to the presented findings.
Test Group Selection Bias
A point of caution with the results obtained in this research is whether a selection bias
occurs when AA participants are compared to other members of the SDHC PTS program.
Individuals who self-select to enter AA may already be more likely to have positive outcomes
than other residents of SDHC programs. If this is the case, participation in AA may be correlated
MAXIMIZING HOUSING EFFICIENCY
72
with positive outcomes; however, the type of person that chooses to enroll in the AA may be part
of the reason for this correlation.
The process of self-selection into AA, while ultimately up to the resident, may create a
similar “creaming” process as seen in Kleit’s (2004) evaluation of the Youngs Lake Commons
Program (2004). However, the Young Lake Commons Program self-nominated its participants
targeting those that are among those residents most likely to succeed in the program, and the
SDHC does not (Kleit, 2004, p. 383). Nonetheless, some informal targeting may occur as
successful residents refer other successful residents. Also, SDHC staff may substantially
influence resident participation more than they intend in their interactions with residents.
Notwithstanding, the linear regression analysis discussed previously attempts to address this
concern but only diminishes the potential problem of selection bias rather than eliminating it.
While the regression analysis controlled for age, age squared, sex, ethnicity, race, and number in
family, the AA-PTS outcome difference is thus not attributable to differences in these
characteristics. The remaining limitation is that the better outcomes of AA participants may be
attributable to differences in additional unobserved characteristics.
Challenges with SDHC 50058 Data
A consistent challenge faced by this researcher is the SDHC's data collection and
management process. Additionally, the purpose for requesting each data point is unclear, and
inconsistency and error-prone data entry procedures limit usability. Below are a few examples of
the challenges with SDHC data:
• Inconsistent Variable Format: Data and variable formats are incompatible in
challenging ways. For example, some date information is via a drop-down menu
MAXIMIZING HOUSING EFFICIENCY
73
while other data appears to be manually inputted. Further, the drop-down menu
produces outputs with unnecessary and often unused data.
• Unclear Variable Values: Some data, such as the census tract data that “on-the-
ground” inspectors’ input, contains meaningful outputs to the inspector; however,
they are not reflected in the data. For example, some inspectors enter a census
tract result that does not exist to represent another value, such as “unknown.”
• Variable Input Errors: Randomly, the data include extra spaces or zeros in the
variable input.
• Variable Definition: Many of the variables lack clear definitions. While some are
self-explanatory and other definitions are contained in HUD reports, many of the
variables are not listed formally, and definitions are only known by SDHC staff,
but not all staff.
• More Than One Variable Output: Some outputs have more than one variable to
record a specific output. These multiple variables are sometimes different and
sometimes conflicting. This is the case even though one variable or column would
clearly describe the entry being measured.
• Variable Accessibility/Usability: While many of the variables collected offer
valid data, the outputs selected and how they are collected obscure information.
For example, the 50058 data produce preselected variables defining relationship
roles, such as head of household, spouse, student, and other adult, but it does not
include a column stating the exact number in the family. Therefore, evaluating
income requires additional effort to gather family size, and the existing 50058
data makes it challenging to gather individual-level income data. However, the
MAXIMIZING HOUSING EFFICIENCY
74
SDHC can collect this information separately, as demonstrated by the income
report. This is an important limitation as household size may limit the availability
to engage in AA and impact the results of AA participation.
• Missing Employment Data: While the 50058 MTW (Moving to Work) form
allows for the collection of employment information on hours worked (it is
grouped into three categories: full-time, part-time, and not employed), this
information was not included with the 50058 data files.
While the researcher was able to perform analysis and highlight significant income
growth results, the current availability of the data does not allow for the detailed analysis needed
to evaluate fully the SDHC’s efforts toward greater housing efficiency.
Achievement Academy Tracking
Individuals can enroll, exit, and re-enroll in the AA during their tenure in SDHC housing,
sometimes more than once. When this occurs, a new enrollment eliminates the old enrollment in
the 50058 data, and multiple different enrollment dates are possible over numerous years, with
only the most recent being represented. Therefore, for this research, participation in AA is solely
measured by current participation in the AA and not based on length of time or any other
participatory measure.
Days Occupied and Reasons for Moveout Data
While this study demonstrated that participation in AA is associated with significant
income growth compared to other PTS work-able participants, additional metrics such as days
occupied or increased self-sufficiency via moveout reasons were not useful. Unfortunately, it
was impossible to gain certainty on the SDHC population exits and their characteristics based on
MAXIMIZING HOUSING EFFICIENCY
75
the data provided. Due to its construction, the days occupied variable only considers those who
have already left SDHC housing, ignoring current residents.
The tracking of the moveout data performed by SDHC includes the labels in Table 4. The
reasoning behind these specific labels is unclear. Categorizing these labels is a significant
assumption in this research and highlights an important limitation. The current tags relating to
the violation/exit reason lack easily understood descriptions that assist in understanding whether
the exit was for a positive, neutral, or negative reason. For example, “voluntary surrender” means
the resident voluntarily surrendered the unit, but that may be because they are now self-
sufficient, moving out of state, or any combination of negative outcomes. Many of the outcomes
recorded do not help determine the reason for the exit, while some of the labels may represent
both positive and negative outcomes. Additionally, the label “self-sufficiency” may be
problematic as it is not an intuitive label.
Recommendations
Based on the evaluation performed, including the review of the relevant literature, this
research offers the following recommendations. Each recommendation requires varying levels of
resources, and as PHAs face significant resource constraints, implementing all the following
recommendations immediately may be challenging. Therefore, the SDHC and HUD will have to
weigh the impacts versus the investment required.
Days Occupied and Reasons for Moveout Data
Adequate metrics such as days occupied or increased self-sufficiency via moveout
reasons were not available to be measured during this evaluation. Ideally, this researcher would
have had access to a complete list of all SDHC residents and their entry and exit date. A
complete list is recommended to be maintained, including a precise entry and exit date for each
MAXIMIZING HOUSING EFFICIENCY
76
resident. This includes collecting move-in and moveout dates for all residents audited or
overseen by SDHC and accounting for residents that may move from one SDHC supervised
property to another. Similarly, the SDHC should collect the reason for moveout for all SDHC
housing residents. The tracking performed by SDHC includes the labels in Table 4. The
reasoning behind these specific labels is unclear. To better track residents' progress towards self-
sufficiency, it is recommended that specific labels be revised or added to the moveout reasons.
Knowing the resident’s rationale behind a voluntary exit would assist the SDHC in
understanding if the move is related to an improvement or reduction in circumstances and then
allow for an analysis of the contributing causes.
Unintended Effects of Incentives and Sanctions
As mentioned in Ch. 4, a review of resident incomes indicates a possible trend of
residents gaining an increase in income, leading them to no longer qualify for a rental subsidy
and then a reduction in household income shortly after that (before the expiration of a 6-month
grace period) to a level that continues to qualify them for supported housing. Ch. 2 discussed the
ample literature on programs with incentive/sanction mechanisms that have unambiguously been
demonstrated to discourage work participation (ex., cash welfare as opposed to the earned
income tax credit). These programs are analogous to a tax on earnings with a potential work-
discouraging effect. Several studies support “the hypothesis that housing subsidies do indeed
lead to a reduction in work effort” (Riccio et al., 2017, p. 3). Therefore, individuals may decide
to forego work as the increase in income is not worth the additional labor.
While more research is needed to determine the frequency and actual causes of the above
scenario in SDHC housing, some residents may achieve an increase in income but then choose to
reduce their wages to ensure they still qualify for supported housing. The fear for these residents
MAXIMIZING HOUSING EFFICIENCY
77
may be that, should they lose their Section 8 qualification, they may end up in a difficult
situation later if their income falls and they must wait 12 years on a waiting list to get Section 8
support again. Should this be the case, the SDHC would be well-served by programmatic
alterations designed to address the aspects of the program driving these residents’ concerns, as
the existing requirements may have work discouraging effects.
SDHC 50058 Data Collection
The data collection processes used by the SDHC are designed to facilitate operations and
the required reporting to HUD; however, there are many ways in which they could be improved
to reduce inconsistency and eliminate error-prone data entry procedures. Below are
recommendations to improve some of the challenges with SDHC 50058 data:
• Inconsistent Variable Format: Reliably input all data in a consistent format.
Establish clear protocols to eliminate the creation of unnecessary spaces and zeros
during the data entry process. Additionally, eliminate unnecessary outputs such as
the time entry that accompanies the drop-down menu when a date is entered.
• Unclear Variable Values and Definitions: Ensure that all variable values are
documented transparently and reliably in a manner that does not rely on
institutional memory to be understood.
• More Than One Variable Output: Whenever possible, the SDHC should avoid
having responses fragmented in a manner that requires pulling together multiple
variables/columns to make them coherent. When this is impossible, review entries
to ensure variable selection does not conflict with previous variables selected.
• Data Resolution/Detail: The outputs selected and how they are collected should
be improved. All information should be collected for everyone individually and
MAXIMIZING HOUSING EFFICIENCY
78
aggregated by household. This will enable individual and household-level
information most suitable for any given analysis. One way to do this would be to
have an individual ID formatted so that a part of it identifies the household.
• Employment/Income Data: The employment and income data would benefit from
some additions. All employment information from the 50058 form should be
included in the data. Further, employment information should specifically include
hours worked per individual in the household. A more precise measure of hours
worked will provide an improved understanding of income outcomes.
The data practices of the SDHC would benefit from the expertise of a professional
program analyst or similar consultant. The current data collection process contains deficiencies
that impede analysis. Other PHAs may engage in similar data practices, and the SDHC has the
potential to be a leader for other agencies that may face similar challenges.
Additionally, it is recommended that the SDHC make sure every resident file is easily
linked to a file containing everyone who has ever been in SDHC supervised housing. Accessing
a comprehensive list of everyone who has been a resident of SDHC housing, including all
individual-level data, will allow for more functional analysis and comparison. This will enable
the agency to highlight its successes and quickly target areas for improvement. Ideally, HUD will
find a way to compile all agency data and make it available to researchers in a usable format.
The current availability is less than ideal and does not allow analysis to demonstrate whether
agencies are meeting their MTW objectives.
Currently, HUD has an MTW Research Advisory Committee evaluating the outcomes of
the HUD Cohorts and any additional policy proposals and research or evaluation methods that
may positively impact HUD policy (discussed in Ch. 1). As a part of this evaluation, a new
MAXIMIZING HOUSING EFFICIENCY
79
50058 reporting form called the HUD – 50058 MTW Expansion is being used for newly
admitted MTW agencies (HUD, MTW Expansion, 2022). This evaluation highlights how
existing MTW agencies could similarly benefit from an updated 50058 form.
The advisory committee, which includes a mixture of qualified practitioners,
policymakers, and academics, should recommend an update to the current 50058 form to
enhance its usability for data gathering. The 50058 form should collect additional data points that
allow for better measurement and offer guidance on the required implementation of data
practices that will enable easy access.
Program Tracking
The data collected on AA participation should be improved
6
. It is recommended that the
SDHC improve data practices allowing for detailed analysis of participant use of the AA.
Tracking more clearly entries and exits of AA participants, frequency of use, and specific
milestones throughout the participation process may lead to a better understanding of why the
AA might improve income growth and potentially housing efficiency. Additionally, tracking
when individuals enter and exit the AA program will allow for a precise measurement of the
length of time in AA and the frequency of enrollment/exit. While some of this information may
be tracked across different departments, it is not uniformly housed in one location and is not
combined with the 50058 data.
Similarly, developing a consistent AA completion variable will allow for a better
understanding of the impact of the AA curriculum, for example, understanding whether there is a
significant difference between the outcomes of those that enroll in AA and those who complete
6
This recommendation may be valid for additional SDHC programs; however, other program data was not
evaluated.
MAXIMIZING HOUSING EFFICIENCY
80
AA. Lastly, specific moveout reasons may be justified for households enrolled in AA to
understand the program's impacts better.
Real-Time Data Gathering
The usefulness of SDHC reporting data would be enhanced by efficiently tracking
resident self-sufficiency gains/losses and reasons for participation/nonparticipation. Currently, in
the 50058 reporting process, resident information is updated annually at the earliest. While this
collection period satisfies HUD requirements, any possible reduction in gaps between collection
times will assist in understanding the impact of existing programs or changes and allow for
immediate, more targeted delivery of incentives when required.
It is recommended that the SDHC pursue mechanisms triggering updates from residents
whenever noteworthy changes occur, such as employment or income changes. This will enhance
the usability of SDHC data and allow for the possibility of better demonstrating the achievement
of its MTW objectives.
Behavioral Science Consultation
The existing programs operated by the SDHC appear to be structured through a
traditional lens of carrots versus sticks or incentives versus disincentives. Modern behavioral
science research has demonstrated that better decision-making can be facilitated with a different
approach that does not fit into traditional programmatic understanding. It is recommended that
the SDHC engage a behavioral science consultant to evaluate ways that improved outcomes may
be achieved by altering programmatic processes and approaches. It is beyond the scope of this
evaluation to discuss all the possible processes. However, one area that this author recommends
is the process by which residents are transitioned out of SDHC housing upon achieving self-
sufficiency.
MAXIMIZING HOUSING EFFICIENCY
81
As previously mentioned, these recommendations require varying levels of resources.
While adding a program label or correcting a common data input error may require minimal
investment, implementing the real-time data gathering or hiring a behavioral science consultant
may require substantially more resources. Unfortunately, PHAs must weigh these costs and
potential benefits while dealing with existing resource constraints.
Future Research Opportunities
Based on the analysis of the data and a review of the relevant literature, the following is a
list of future research that may be warranted. These items were discovered during the research
process and are beyond the scope of what was proposed in this analysis but would be a natural
extension of the analysis performed here and benefit the field of study by addressing existing
gaps in the current literature.
Additional Analysis of the Achievement Academy
Additional research on the long-term employment outcomes of AA participants should be
undertaken. An exit from supported housing is an arbitrary deadline. Other impacts of AA may
be seen once the reporting relationship with the SDHC is terminated.
Analysis of Additional Programs
The need exists for high-quality evaluation of additional programs at the SDHC. AA is
only one program within the SDHC, and it was only compared to the broader PTS program and
not to a “no intervention” option. For example, while participation in AA is associated with a
higher income growth rate, this finding is of no use to ascertain whether the SDHC assumptions
about the benefits of PTS on a work-able population are correct, given that the analysis did not
include comparisons to any non-PTS households. It is recommended that the SDHC undertake a
MAXIMIZING HOUSING EFFICIENCY
82
study to evaluate its additional programs and their outcomes, like the impact of PTS alone, to
determine if the program requirements of PTS achieve what they intend.
Having demonstrated steps toward better evaluations pertinent to other
agencies/programs, a similar analysis of the incentive programs at other agencies may be
warranted. Should other MTW agencies have analogous subprograms, this research could be
adopted and adapted to assist policymakers in reviewing these agencies’ programs. Suppose a
similar analysis to that achieved here is performed. In that case, a potential metric for evaluation
will allow the SDHC and other agencies to compare outcomes while accounting for preexisting
differences in the populations they serve.
While this research creates a foundation upon which other agencies may build, this author
does not presume the generalizability of these findings to all PTS participants or all public
housing agency residents. There are potentially distinctive features of the SDHC population. The
SDHC population is majority White and female. Additionally, residents of affordable housing in
San Diego, California, may be distinctly different compared to residents of affordable housing in
other locations. Similarly, the evaluation period and the sub-program population (AA) studied
might add to the limited generalizability of these findings. Further, the resulting estimates of the
regression analysis performed, while accounting for demographic differences between the AA
and other PTS work-able groups, are applicable to this specific population and may not inform
the outcomes of a similar program elsewhere. Nonetheless, to the extent that the effects of these
specific programs or populations are not generalizable, separate evaluations of other agency
programs are still warranted. Lastly, while the conclusions of this research may not be
generalizable to all populations, the methods presented here will translate in addressing
challenges likely present in other public housing data.
MAXIMIZING HOUSING EFFICIENCY
83
Cost-Benefit Analysis
The success of the SDHC programs requires resources and expertise. A study analyzing
the precise cost-benefit analysis of these two competing economic impacts is essential to fully
understand the impact of a social services program that is measured by its impact on federal
resources, such as the MTW demonstration. Additional research on self-sufficiency programs
providing a precise cost-benefit analysis of the program's effects, or other housing plus efforts,
would benefit this field of study.
Relevant Research on Work Requirements, Time Limits, and Tiered Rent Programs
This research evaluated the incremental impacts of an incentive and support program.
This is one of the four categories of common programs instituted at MTW agencies (as discussed
in Ch. 1). It is worthwhile for future research to evaluate the other three types of programs listed
(work requirements, tiered rents, and time limit programs) to determine if they are helpful
components to achieving self-sufficiency gains. As previously mentioned, further evaluation of
the SDHC PTS program would add an evaluation of a current tiered rent program; however,
others could also be evaluated. Additional evaluations (perhaps building on the foundation
presented here) that examine outcomes of programs with and without the four categories (work
requirements, tiered rents, time limits, and incentive/support) will contribute to the resolution of
this question. The existing HUD Cohort evaluation endeavors to test these program components
on new MTW agencies; however, additional research on existing MTW agencies with these
program components will also benefit this field of study.
Conclusions
A comparison of AA participants and other work-able PTS participants demonstrated that
AA participation is associated with a significantly higher income growth rate than PTS alone.
MAXIMIZING HOUSING EFFICIENCY
84
This robust finding of a significantly higher income growth rate for AA participants remains
even after accounting for underlying differences between the two groups. The evaluation
presented here falls short of a compelling causal impact estimate due to the limitations of the
data. More research is required to conclude whether participation in AA improves housing
efficiency for work-able adults compared to PTS alone. Other outcomes are of interest and await
evaluation upon obtaining currently-unavailable data (analysis on the reason for moveouts and
days occupied).
This evaluation of the SDHC adds to an existing gap in the research of the MTW
designation and labor participation programs by analyzing the incremental effect of a self-
sufficiency program within a long-standing MTW agency. This research provides additional
information for policymakers to consider when evaluating MTW agencies and the outcomes
produced absent and with incentives. One proposed model for assessing public agencies’
innovative programs is offered, as is the opportunity for HUD and PHAs to eliminate
impediments to analysis caused by suboptimal data collection and handling. While the
conclusions of this research may not be generalizable to all populations, the methods presented
here will translate in addressing challenges likely present in other public housing data. This
research may assist those evaluations underway and those yet to come and help guide
policymakers while the MTW program is undergoing a significant expansion.
MAXIMIZING HOUSING EFFICIENCY
85
References
Anthony, J. (2005). Family self-sufficiency programs: An evaluation of program benefits and
factors affecting participants’ success. Urban Affairs Review, 41(1), 65-92.
Balch, G. I. (1980). The stick, the carrot, and other strategies: A theoretical analysis of
governmental intervention. Law & Policy, 2, 35-60. https://doi.org/10.1111/j.1467-
9930.1980.tb00203.
Bates, L., Seegar, L., Kirkeby, M., Coy, M., & Anixter, H. (2018). California’s housing future:
challenges and opportunities final statewide housing assessment 2025. Prepared for
California Department of Housing and Community Development, State of California.
Business, Consumer Services, and Housing Agency. Sacramento, CA.
Benabou, R., & Tirole, J. (2003). Intrinsic and extrinsic motivation. The Review of Economic
Studies, 70(3), 489-520.
Bloom, H., Riccio, J., Verma, N., & Walter, J. (2005) Promoting work in public housing. The
effectiveness of Jobs-Plus. Final report. MDRC. www.mdrc.org.
Bratt, R. G. (2008). Viewing housing holistically: The resident-focused component of the
housing-plus agenda. American Planning Association. Journal of the American Planning
Association, 74(1), 100-110.
Buron, L., Vandawalker, M., & Morril, T. (2017). Testing performance measures for the MTW
Program. Abt Associates. Cambridge, MA, and Bethesda, MD. Sponsored by The Public
and Affordable Housing Research Corporation at HAI Group. Cheshire, CT.
Cadik, E., & Nogic, A. (2010). Moving to work: Interim policy applications and the future of the
demonstration. Prepared for the United States Congress. United States Department of
Housing and Urban Development, Office of Policy Development and Research.
Washington, DC.
Carlson, D., Haveman, R., Kaplan, T., & Wolfe, B. (2012) Long-term earnings and
employment effects of housing voucher receipt. Journal of Urban Economics, 71(1),
128–150.
Creswell, J. W., & Creswell, J. D. (2018). Research design: qualitative, quantitative, and mixed
methods approaches 5
th
Edition. Los Angeles, CA: SAGE Publications, Inc.
Dayal, S., & Davis, J. (2018). Item 102: Fiscal year 2019 Moving to Work annual plan approval
(Report No: HCR 18-040). Prepared for San Diego Housing Commission. San Diego,
CA.
MAXIMIZING HOUSING EFFICIENCY
86
De Geest, G., & Dari-Mattiacci, G. (2013). The rise of carrots and the decline of sticks. The University
of Chicago Law Review, 80(1), 341-393. http://www.Jstor.org /stable/41825878.
de Silva, L., Wijewardena, I., Wood, M., & Kaul, B. (2011). Evaluation of Family Self-
Sufficiency Program: Prospective study. Report prepared for United States Department of
Housing and Urban Development, Office of Policy Development and Research.
Washington, DC. https://www.huduser.gov/portal//Publications/pdf/
FamilySelfSufficiency.pdf.
Eisenberger, R., Pierce, W. D., & Cameron, J. (1999). Effects of reward on intrinsic
motivation—Negative, neutral, and positive: Comment on Deci, Koestner, and Ryan
(1999).
Ferey A. (2018) Housing benefits and monetary incentives to work: Simulations for France.
In: Economie et Statistique / Economics and Statistics, n°503-504, p. 37-59.
Ficke, R. & Piesse, A. (2004). Evaluation of the Family Self-Sufficiency Program. Retrospective
Analysis. 1996-2000. U.S. Department of Housing and Urban Development, Office of
Policy Development and Research. Westat. Rockville, MD.
Flanigan, S. T. (2018). Achievement Academy – San Diego Housing Commission report.
Internal report for Achievement Academy and Access to Opportunity Team/Funders.
Flanigan, S. T. (2019) Municipal diverging from “bureaucracy:” A case study of
organizational image in housing services. Journal of Public Management & Social
Policy: Vol. 26, No. 1, Article 2. https://digitalscholarship. tsu.edu/jpmsp/vol26/iss1/2.
Fording, R. C., Schram, S. F., & Soss, J. (2013). Do welfare sanctions help or hurt the poor?
Estimating the causal effect of sanctioning on client earnings. Social Service
Review, 87(4), 641-676.
Gentry, R. (2018). Presentation to US House of Representatives Committee on Financial
Services Subcommittee on Housing and Insurance. Hearing on HUD’s role in rental
assistance: An oversight and review of legislative proposals on rent reform.
https://www.sdhc.org/wpcontent/uploads/2018/04/SDHC_CongressionalTestimony
_RichardCGentry_042518.pdf.
Geyer, J., Freiman, L., Lubell, J., & Villarreal, M. (2017). Evaluation of the Compass Family
Self-Sufficiency (FSS) Programs administered in partnership with public housing
agencies in Lynn and Cambridge, Massachusetts. Prepared by Abt Associates. Prepared
for: Compass Working Capital and United States Department of Housing and Urban
Development. Bethesda, MD. http://abtassociates.com/sites/default/files/2 018-
06/Compass%20FSS%20Evaluation%20Report_09082017.pdf.
Hasenfeld, Y., Ghose, T., & Larson, K. (2004). The logic of sanctioning welfare recipients: An
empirical assessment. Social Service Review, 78(2), 304-319.
MAXIMIZING HOUSING EFFICIENCY
87
HUD.GOV. U.S. Department of Housing and Urban Development. (2022). HUDUser Glossary/
Archives. https://archives.huduser.gov/portal/glossary/glossary.html.
Jacob, B. A., & Ludwig, J. (2012). The effects of housing assistance on labor supply:
evidence from a voucher lottery. American Economic Review 102(1), 272–304.
Joint Center for Housing Studies of Harvard University (2022). The state of the nation’s housing
2022. Joint Center for Housing Studies of Harvard University. Harvard Graduate School
of Design and Harvard Kennedy School. https://www.jchs.harvard.edu/sites/default/files
/reports/files/Harvard_JCHS_State_Nations_Housing_2022.pdf.
Kamenica, E. (2012). Behavioral economics and psychology of incentives. Annual Review of
Economics 2012 4:1, 427-452. https://doi.org/10.1146/annure-economics-080511-
110909.
Khadduri, J., Vandawalker, M., Cohen, R., Lubell, J., Buron, L., Freiman, L., & Kean, E. (2014).
Innovations in the Moving to Work Demonstration. Abt Associates. Sponsored by: Public
and Affordable Housing Research Corporation, a HAI Group company. Cambridge, MA,
and Bethesda, MD. https://www.abtassociates.com/sites/default/files/migrated_files/
b8bd4434-4303-408e-b696-874f821e66ea.pdf.
Kleit, R. G. (2004). Designing and managing public housing self-sufficiency programs: the
Youngs Lake Commons Program. Evaluation Review, 28(5), 363-395.
Lee, H. B., & McNamara P.E. (2018). Achieving economic self-sufficiency through housing
assistance: An assessment of a self-sufficiency program of the Housing Authority of
Champaign County, Illinois. Housing Policy Debate. 28(6), 879-900.
Levy, D., Edmonds, L., & Simington, J. (2018). Work requirements in public housing
authorities: Experiences to data and knowledge gaps. Washington, DC: Urban Institute.
Mazzara, A., & Sard, B. (2018). Chart Book: Employment and earnings for households receiving
federal rental assistance. Center on Budget and Policy Priorities. Washington, DC.
McCarty, M., Aussenberg, R. A., Falk, G. (2016). Work requirements, time limits, and work
incentives in TANF, SNAP, and housing assistance. Congressional Research Service.
McNamara, P. E., Strick, C. and Lee, H.B. (2015). Evaluation of the Moving to Work Program
in Champaign County. Progress Report to the Housing Authority of Champaign County.
Champaign, IL.
McNamara, P. E., Lee, H. B., & Strick, C. (2017). Promoting economic self-sufficiency via
HUD’s Moving to Work Program: Evidence from the Housing Authority of Champaign
County. Illinois Municipal Policy Journal, 2(1), 49-67.
MAXIMIZING HOUSING EFFICIENCY
88
Miller, R., Riccio, J., Verma, N., Nuñez, S., Dechausay, N., &Yang, E. (2015). Testing a
conditional cash transfer program in the U.S.: the effects of the family rewards program
in New York City. IZA J Labor Policy 4, 11 (2015). https://doi.org/10.1186/s40173-015-
0037-6.
Mills, G., Gubits, D., Orr, L., Long, D., Feins, J., Kaul, B., Wood, M., , Jones, A., & Associates,
Cloudburst Consulting, and the QED Group. (2006). Effects of housing vouchers on
welfare families: Final report. Prepared for the U.S. Department of Housing and Urban
Development, Office of Policy Development and Research. Cambridge, MA: Abt
Associates, Inc.
Mills, G. E., & Gay, L.R. (2019). Educational research: Competencies for analysis and
applications. Twelfth Edition. Pearson.
Minoff, E. (2020). The racist roots of work requirements. The Center for the Study of Social
Policy: Ideas into action February 2020. https://cssp.org/resource/racist-roots-of-work-
requirements/.
MTW Collaborative. Moving to Work: Innovation and flexibility to address America’s
affordable housing challenge (December 2020). https://65bcdb2c-4e6f-44c9-ae9b-
f9100a326749.filesusr.com/ugd/15cee7_dbc3631b72ba 44d399271512ebf96091.pdf.
National Low Income Housing Coalition. HUD PIH withdrawals MTW Work Requirement
Cohort (June 01, 2021). https://nlihc.org/resource/hud-pih-withdraws-mtw-work-
requirement-cohort.
Nuñez, S., Verma, N., & Yang, E. (2015). Building self-sufficiency for housing voucher
recipients: interim findings from the Work Rewards Demonstration in New York City.
New York: MDRC June.
Riccio, J. (2007). An MDRC working paper. Subsidized housing and employment; building
evidence about what works to improve self-sufficiency. Prepared for Revisiting Rental
Housing: A National Policy Summit. A symposium organized by the Harvard Joint
Center for Housing Studies and Supported by the MacArthur Foundation November 14
and 15, 2006. March 2007.
Riccio, J. (2010). Sustained earnings gains for residents in a public housing jobs program:
Seven-year findings from the Jobs-Plus Demonstration. New York: MDRC.
Riccio, J., Deith, V., & Verma, N. (2017). Reducing work disincentives in the Housing Choice
Voucher Program: Rent reform demonstration baseline report. Prepared for the U.S.
Department of Housing and Urban Development Office of Policy Development
Research. MDRC.
Rohe, W. M., Webb, M. D., & Frescoln, K. P. (2016). Work requirements in public housing:
Impacts on tenant employment and evictions. Housing Policy Debate, 26(6), 909-927.
MAXIMIZING HOUSING EFFICIENCY
89
Rohe, W. M., & Kleit, R. G. (1999) Housing, welfare reform, and self‐sufficiency: An
assessment of the Family Self‐Sufficiency Program, Housing Policy Debate, 10:2, 333-
369, DOI: 10.1080/10511482.1999.9521335.
San Diego Housing Commission. (2020). About us. https://www.sdhc.org/about-us/.
San Diego Housing Commission. (2021). Moving Forward. Moving To Work Program. Annual
plan for fiscal year 2022. https://www.sdhc.org/wp-content/uploads/2021/02/FY2022-
MTW-Annual-Plan-HUD-Approved.pdf.
San Diego Housing Commission. (2021). Path to Success. https://www.sdhc.org/housing-
opportunities/path-to-success/.
San Diego Housing Commission. (2019). San Diego Housing Commission initiatives and
partnership developments receive national and local honors. https://www.sdhc.org/news-
release/sdhc-initiatives-and-partnership-developments-receive-national-and-local-
honors/.
San Diego Housing Commission. (2020). San Diego Achievement Academy: We’re about the
people. https://www.sdhc.org/achievement-academy/.
Scally, C. P., Batko, S., Popkin, S. J., and DuBois, N. (2018). The case for more, not less:
shortfalls in federal housing assistance and gaps in evidence for proposed policy changes.
Washington, DC: Urban Institute. https://www.urban.org/sites/default/files/publication
/95616/case_for_more_not_less.pdf.
Shlay, A. B. (1993). Family self-sufficiency and housing. Housing Policy Debate 4(3):457–95.
Shroder, M. D. (2010). Housing Subsidies and Work Incentives. https://ssrn.com/abstract
=1691112 or http://dx.doi.org/10.2139/ssrn.1691112.
Skinner, B. F. (1974). About behaviorism. Random House. New York.
Stater, K. (2018). Local decision-making in the Moving to Work (MTW) Program. Public and
Affordable Housing Research Corporation (PAHRC) at HAI Group. https://www.
housingcenter.com/wp-content/uploads/2018/04/MTW-White-Paper.pdf.
Thaler, R. H. (2015). Misbehaving: The making of behavioral economics. W Norton & Co.
Thaler, R. H., & Sunstein, C. R. (2009). Nudge: improving decisions about health, wealth, and
happiness. Rev. and expanded ed. New York: Penguin Books.
United States Department of Housing and Urban Development (2013). Office of Public and
Indian Housing. MTW Family Report. Form HUD-50058 MTW, Family Report.
11/2013. Washington, DC.
MAXIMIZING HOUSING EFFICIENCY
90
United States Department of Housing and Urban Development (2013). Office of Public and
Indian Housing. Moving forward. Moving to Work Program: Annual plan for fiscal year
2019. https://www.hud.gov/sites/dfiles/PIH/documents/SanDiegoFY19Plan.pdf.
United States Department of Housing and Urban Development (2019). Module 1.2: Introduction
to the FSS program and FSS training. https://www.hudexchange.info/trainings/fss-
program-online-training/1.2-what-is-fss.html.
United States Department of Housing and Urban Development (2022). Moving to Work (MTW)
expansion. https://www.hud.gov/program_offices/public_indian_housing/programs/ph/
mtw/expansion.
United States Department of Housing and Urban Development (2021). Office of Public and
Indian Housing. Jobs Plus Initiative Program. https:// www.hud.gov/program_
offices/public_indian_housing/jpi.
United States Department of Housing and Urban Development (2022). Office of Public and
Indian Housing. Moving to Work Demonstration Program – Participating agencies.
https://www.hud.gov/program_offices/public_indian_housing/programs/ph/
mtwagencies.
United States Department of Housing and Urban Development (2021). Office of Public and
Indian Housing. Notice PIH-2021-18. RESCINDING of request for applications under
the Moving to Work Demonstration Program – Work Requirements Cohort. https://www.
hud.gov/program_offices/publicindian_housing/programs/ph/mtw/expansion/cohort3.
United States General Accountability Office (2018). Report to the Ranking Member Committee
on Financial Services, House of Representatives. Rental Housing: Improvements needed
to better monitor the moving to Work Demonstration, including effects on tenants.
January 2018. Washington, DC.
Walter, R.J., Colburn, G., Yerena, A., Pederson, M., Fyall, R. & Crowder K. (2020) Constraints
and opportunities for innovation in the Moving to Work Demonstration Program.
Housing and Society, 47:1, 1-21. https://doi.org/10.1080/08882746.2019.1706067.
Warth, G. (January 2, 2017). After wait of 11 years, renter gets voucher. San Diego Union
Tribune. January 2, 2017.
Waters, M. Member, United States House of Representatives (2015, July 14). Letter to
Appropriations Committee Leadership regarding conference for FY 2016 THUD
Appropriation Bill.
Weaver, R. K. (2015). Getting people to behave: Research lessons for policy makers. Public
Admin Rev, 75: 806-816. https://doi.org/10.1111/puar.12412.
MAXIMIZING HOUSING EFFICIENCY
91
Webb, M. D., Frescoln, K. P., & Rohe, W. M. (2015). Innovation in public housing: The Moving
to Work Demonstration. Prepared under a contract with the Charlotte Housing Authority.
Center for Urban and Regional Studies, University of North Carolina at Chapel Hill.
January 2015.
Webb, M. D., Frescoln, K. P., & Rohe, W. M. (2016). Innovation in US public housing: A
critique of the moving to work demonstration. International Journal of Housing
Policy, 16(1), 111-124.
Abstract (if available)
Linked assets
University of Southern California Dissertations and Theses
Conceptually similar
PDF
Philanthropic foundations and social impact bonds: understanding impact investment approaches among philanthropic foundations
PDF
Three essays on housing demographics: depressed housing access amid crisis of housing shortage
PDF
A framework for evaluating urban policy and its impact on social determinants of health (SDoH)
PDF
A better method for measuring housing affordability and the role that affordability played in the mobility outcomes of Latino-immigrants following the Great Recession
PDF
Lessons from TAP implementation: obstacles and solutions to improve the transit users experience
PDF
A stakeholder approach to reimagining private departments of public safety: the implementation of a community advisory board recommendation report
PDF
Planning care with the patient in the room: a patient-focused approach to reducing heart failure readmissions
PDF
The interplay between green space and transit: a case study of revitalization within Atlanta’s Adair Park and its potential to improve future policy outcomes
PDF
Affordable south Los Angeles: survival, support, and different futures
PDF
A miracle or a mirage? A study to evaluate the impacts of microfinance
PDF
Three essays on aging, wealth, and housing tenure transitions
PDF
A strategic talent management retention model: an effective way to shape the United States Space Force
PDF
Resilient and equitable urbanism by design: insights from the collaborative process to reimagine the SF Bay Area
PDF
Insights into residential mobility and pricing of rental housing: the role of gentrification, home-ownership barriers, and market concentrations in low-income household welfare
PDF
How does collaborative governance work? The experience of collaborative community-building practices in Korea
PDF
The role of leadership and collaboration as a catalyst for regional economic development: a grounded theory study
PDF
Local votes and outside money: campaign contribution geographic origins and their impact on Los Angeles City Council election outcomes
PDF
Evergreen economies: institutions, industries and issues in the green economy
PDF
Municipal employee reactions to city council incivility: an exploratory data analysis
PDF
Outcomes-based contracting through impact bonds: ties to social innovation, systems change, and international development
Asset Metadata
Creator
Walsh, Timothy Sean
(author)
Core Title
Utilizing analytics to evaluate the San Diego Housing Commission's approach to maximizing housing efficiency: Moving to Work, Path to Success, and the Achievement Academy
School
School of Policy, Planning and Development
Degree
Doctor of Policy, Planning & Development
Degree Program
Planning and Development,Policy
Degree Conferral Date
2022-12
Publication Date
08/18/2022
Defense Date
07/19/2022
Publisher
University of Southern California
(original),
University of Southern California. Libraries
(digital)
Tag
Achievement Academy,housing efficiency,Moving to Work,OAI-PMH Harvest,Path to Success,San Diego Housing Commission
Format
application/pdf
(imt)
Language
English
Contributor
Electronically uploaded by the author
(provenance)
Advisor
McCarthy, T. J. (
committee chair
), Gentry, Richard (
committee member
), McCann, Pamela (
committee member
)
Creator Email
tim_s_walsh@yahoo.com,twalsh@usc.edu
Permanent Link (DOI)
https://doi.org/10.25549/usctheses-oUC111376961
Unique identifier
UC111376961
Legacy Identifier
etd-WalshTimot-11144
Document Type
Dissertation
Format
application/pdf (imt)
Rights
Walsh, Timothy Sean
Type
texts
Source
20220819-usctheses-batch-974
(batch),
University of Southern California
(contributing entity),
University of Southern California Dissertations and Theses
(collection)
Access Conditions
The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the author, as the original true and official version of the work, but does not grant the reader permission to use the work if the desired use is covered by copyright. It is the author, as rights holder, who must provide use permission if such use is covered by copyright. The original signature page accompanying the original submission of the work to the USC Libraries is retained by the USC Libraries and a copy of it may be obtained by authorized requesters contacting the repository e-mail address given.
Repository Name
University of Southern California Digital Library
Repository Location
USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA
Repository Email
cisadmin@lib.usc.edu
Tags
Achievement Academy
housing efficiency
Moving to Work
Path to Success
San Diego Housing Commission