Close
About
FAQ
Home
Collections
Login
USC Login
Register
0
Selected
Invert selection
Deselect all
Deselect all
Click here to refresh results
Click here to refresh results
USC
/
Digital Library
/
University of Southern California Dissertations and Theses
/
A framework for evaluating urban policy and its impact on social determinants of health (SDoH)
(USC Thesis Other)
A framework for evaluating urban policy and its impact on social determinants of health (SDoH)
PDF
Download
Share
Open document
Flip pages
Contact Us
Contact Us
Copy asset link
Request this asset
Transcript (if available)
Content
A FRAMEWORK FOR EVALUATING URBAN POLICY AND ITS IMPACT ON
SOCIAL DETERMINANTS OF HEALTH (SDOH)
By
Gia E. Grant-Rutledge
A Dissertation Presented to the
FACULTY OF THE USC SOL PRICE SCHOOL OF PUBLIC POLICY
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulfillment of the
Requirements for the Degree
DOCTOR OF POLICY, PLANNING AND DEVELOPMENT
AUGUST 2022
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
ii
Acknowledgements
Proverbs 16:3
The Message
“
Put GOD in charge of your work,
then what you have planned will take place.”
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
iii
TABLE OF CONTENTS
Acknowledgements................................................................................................... ii
List of Tables............................................................................................................. v
List of Figures........................................................................................................... vi
Abstract..................................................................................................................... vii
Chapter 1: Introduction ............................................................................................ 1
Background................................................................................................... 2
Urban Policy and Policy Evaluation............................................................. 2
Social Determinants of Health (SDoH)......................................................... 3
Political Determinants of Health (SDoH)..................................................... 4
Research Question………………………..................................................... 6
Research Purpose ......................................................................................... 6
Contribution to Practice................................................................................ 7
Need for the Study........................................................................................ 8
Gaps in the Literature.................................................................................... 10
Methods......................................................................................................... 14
Summary....................................................................................................... 15
Chapter 2: Literature Review.................................................................................... 16
The History of Urban Policy......................................................................... 17
The Housing Acts of 1937, 1949 & 1954 ………………………………… 18
The Grey Areas Project................................................................................. 20
The Model Cities Program/War on Poverty………….................................. 21
Community Development Block Grant…………………………………… 23
A New Partnership to Conserve America’s Communities………………… 25
State Enterprise Zones (Reagan and Bush) .................................................. 26
Empowerment Zones and Enterprise Communities...................................... 30
Promise Zones............................................................................................... 33
Opportunity Zones……………………………………................................ 39
Policy Evaluation Methods and Framework……………............................. 41
Summary……………………………………............................................... 48
Chapter 3: Methodology, Data Analysis, and Outcomes.......................................... 49
Study Purpose………………….................................................................... 50
Analytic Framework...................................................................................... 50
Urban Policy and SDoH Collective Impact Evaluation Framework and
Shared Measurements Development Approach............................................
53
Step 1: Literature Review of Urban Policies……………............................. 55
Step 2: Data Abstraction of Literature Review to Identify SDoH
Measures…...................................................................................................
56
Step 3: Environmental Scan of Existing Databases and Data Sources to
Identify Key SDoH Measures.......................................................................
57
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
iv
Summary and Limitations............................................................................. 58
Chapter 4: Data Reporting and Analysis................................................................... 59
Step 1: Literature Review of Urban Policies……………............................. 60
Step 2: Data Abstraction of Literature Review to Identify SDoH
Measures…...................................................................................................
60
Step 3: Environmental Scan of Existing Databases and Data Sources to
Identify Key SDoH Measures.......................................................................
85
Summary………………............................................................................... 91
Chapter 5: Discussion, Implications, and Conclusion.............................................. 92
A Collective Impact Approach to Urban Policy and the SDOH................... 93
An Urban Policy and SDOH Collective Impact Evaluation Framework...... 97
Implications for Public Health 99
A Policy Bubble for Social Determinants of Health 100
Summary……………………………………………………………........... 102
References…............................................................................................................. 108
Appendices................................................................................................................ 121
Appendix A: Urban Policy SDoH Logic Model........................................... 122
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
v
List of Tables
Table 1.1 The Five Conditions of Collective Impact Framework…………………. 10
Table 4.1 Enterprise Zones (EZ) Evaluation Questions, Measures/Outcomes …… 64
Table 4.2: Empowerment Zones/Enterprise Communities/Renewal Communities
(EZ/EC/RC) SDoH Evaluation Questions, and Measures/Outcomes……………...
68
Table 4.3 Promise Zones (PZ) SDoH Evaluation Questions, and
Measures/Outcomes………………………………………………………………...
75
Table 4.4 Opportunity Zones (OZ) SDoH Evaluation Questions, and
Measures/Outcomes…...............................................................................................
80
Table 4.5 Urban Policy Focus Areas and Social Determinants of Health
Domains…………………………………………………………………………….
85
Table. 4.6 Shared Measures/Outcomes, Evaluation Questions and Available Data
Sources by Social Determinants of Health and Urban Policy Domains……………
88
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
vi
List of Figures
Figure 2.1 The War on Poverty Tour……………………………………………… 22
Figure 2.2. President Obama’s Administration’s Model for Intergovernmental
Urban Policy Making………………………………………………………………
35
Figure 2.3 Three Rounds of Promise Zones……………………………………….. 37
Figure 3.1 A Framework of Performance Measurement and Evaluation of
Collective Impact Efforts…………………………………………………………..
52
Figure 3.2. Urban Policy SDoH Collective Impact Logic Model Components…… 53
Figure 5.1 Collective Impact Framework for Public Policy ……………………… 94
Figure 5.2: Summary of Urban Policy and SDoH Evaluation Framework
Development Process………………………………………………………………
97
Figure 5.3 Urban Policy and SDoH Collective Impact Evaluation Framework…... 99
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
vii
Abstract
Urban Policy impacts the day-to-day lives of urban communities, and no research has
been done to evaluate the impact on the SDoH of people who live in the communities targeted by
urban policy. Over time, little policy evaluation has been done systematically to evaluate the
social impact of urban policy on communities collectively to reduce poverty and improve the
economic, social, housing conditions, and/or health in urban communities. The most recent urban
policy, the Tax and Jobs Act signed into law created over 8,700 “qualified opportunity zones
(OZ)”. In 2019, HUD released a report providing recommendation for future Opportunity Zone
(OZ) evaluations based off lessons learned from previous place-based incentive program
evaluations (i.e., Enterprise Zones, Empowerment Zones, and Enterprise Communities). Two
major tasks identified from this HUD (2019) report were: (1) identify a comparison set of census
tracks; and (2) identify data that can be used to track neighborhood change.
A GAO (2020) report highlights that Congress did not designate an agency with the
responsibility and authority to collect data, evaluate, and report on OZ performance and
concluded that “there are insufficient data available to evaluate OZ performance”. The GAO
recommended Congress: (1) “consider providing Treasury with authority and responsibility to
collect data and report on OZ’s performance, in collaboration with other agencies and (2)
consider identifying questions about OZ’s effects that it wants Treasury to address in order to
help guide data collection and reporting of performance, including outcomes” (GAO, 2020). In
2021, Congress requested the GAO conduct an evaluation of the OZ policy, due to the
magnitude (impacting over 9,000 census tracts) and the extended time for the OZ tax benefit (tax
benefits for this policy do not expire until 2047).
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
viii
Also in 2020, Congress acknowledged and supported public health’s role in “working
across sectors on social determinants of health” and passed the Social Determinants Accelerator
Act of 2019. There was continued support for this bill in 2021 with the reintroduction of the
Improving Social Determinants of Health Act of 2021 (H.R.379), and a final approved FY2022
appropriation bill with an increase to $5 million to address SDoH. There were two primary goals
proposed in this bill: “1.) improve health outcomes and reduce health inequities through the
coordination of SDOH activities across the CDC; and 2.) improve public health agencies’ and
community organizations’ capacity to address SDOHs in communities” (Brown, 2021).
The purpose of this study is to develop an evaluation framework and core set of SDoH
measures to assess the impact of urban policy at the local level on the social determinants of
health (SDoH) in urban communities. The Collective Impact Evaluation Framework was used
for the study design. A Collective Impact Evaluation Framework is used when the desire is to
achieve large-scale, sustainable change (Preskill, Parkhurst, & Juster, 2014). It supports the
complex implementation of urban policy and allows for a mixed methods research design to
assess the implementation, effectiveness, and impact of the policy. A literature review and
environmental scan of past urban policies provided insight on what key evaluation measures and
outcomes should be used to evaluate the impact of urban policy on SDoH. A qualitative analysis
of guidance documents, evaluation reports, administrative documents, and government reports
was also conducted to identify key measures, partnerships, programs, activities, and policies that
were employed to support implementation and evaluation of past urban policy in targeted
communities. SDoH measures identified were then used to develop an evaluation framework that
can be used to assess if SDoH improve for the people in communities that have been targeted by
urban policies over time.
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
ix
The Urban Policy SDoH Collective Impact Evaluation Framework includes short-term
and intermediate changes in systems and policies, and long-term changes in behavioral outcomes
in targeted communities. Impacts evaluated include long-term changes in key SDoH indicators in
targeted communities. The Urban Policy SDoH Evaluation Framework can be used to assess the
impact of urban policy on SDoH at the local level and allow for the study of two timely topics
(urban policy and measuring SDoH). In addition, the White House Opportunity Zone
Implementation Plan has identified the need to develop robust reporting and analytics to measure
the impact of the urban policy in designated Opportunity Zones (White House Opportunity and
Revitalization Council, 2019). The results of this study can fill the need to develop shared
measurement system to assess the social determinants of health of urban policy, provide an
evaluation framework, social determinants of health measures, and data sources to measure the
impact of the urban policy. The Urban Policy SDoH Evaluation Framework can be applied at the
federal and local level by providing an evaluation framework, social determinants of health
measures, and data sources to measure the impact of the urban policy on the SDoH on people
who live in communities targeted by urban policy.
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
1
Chapter 1: Introduction
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
2
Background
We will never move forward significantly with improving Social Determinants of Health
(SDoH) without adequately understanding the role of urban policy where people live, work, play,
and worship. SDoH are the conditions in which we are born, live, learn, work, play, worship, and
age that affect a wide range of health, functioning, quality-of-life outcomes, and risks
(Healthypeople.gov, 2020). Urban policy is defined as a cluster of policies aimed at influencing
the development of urban areas and urban lives (Wang, 2018). The concept of place-based
policy is one form of urban policy that has been implemented by several presidents. While many
presidents failed at changing health or tax policies, urban policy is one of the only policies that
every administration was able change. The history of urban policies includes over ten different
policy initiatives from 1937 to 2020.
Urban Policy and Policy Evaluation
Two thirds of the world’s population is projected to live in urban areas by the middle of
this century (OECD/UN-HABITAT, 2018). As the proportion of the population living in urban
areas grow, it is important to understand the social impact of federal urban policies on the people
who live in these communities (Barrow, 2002). Social impact is defined as “a significant or
lasting change in people’s lives brought about by a given action or actions” (Barrow, 2002).
Policy impact evaluation looks at how important indicators have changed after a policy was
implemented and how much of that change can be attributed to the policy (CDC, 2013).
In 1978, the Carter Administration proposed that all government policies be evaluated
prior to implementation to assess possible impacts on urban communities and that evaluation
should be conducted in alignment with the administration’s urban policy (Clark, 1981). With
each new administration, there is the development of a new federal urban policy that typically
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
3
focuses on four types of programs: place-targeted urban programs in functional areas, federal
intergovernmental grants, people-oriented social programs, and economic programs (Glickman,
1980). Warren (1990) states that after a half of century, the lack of formal urban policy and wave
of national programs, have left an endemic of problems in urban communities such as “poverty,
racial segregation, political marginality, inadequate housing, environmental degradation, and
uneven intra- and inter metropolitan development. Over time urban policies have not been
systematically evaluated and have left a legacy of unresolved issues from the shift of one
administration to the next administration (Rich & Stoker, 2014)
Social Determinants of Health (SDoH)
SDoH are shaped by “distributions of money, power, and resources at global, national,
and community levels” (Gomez, et.al, 2021). SDoH are in large part influenced by social policy
(Braveman & Gottlieb, 2014). The social determinants include factors such as social, economic,
physical/environmental, and access to services in a community. Healthy People 2030 identified
five goals across 5 SDoH domains:
1. Economic Stability: Help people earn steady incomes that allow them to meet their health
needs.
2. Education Access and Quality: Increase educational opportunities and help children and
adolescents do well in school.
3. Health care Access and Quality: Increase access to comprehensive, high-quality health
care services.
4. Neighborhood and Built Environment: Create neighborhoods and environments that
promote health and safety.
5. Social and Community Context: Increase social and community support (DHHS, 2030).
As we look upstream, the question arises, what shapes SDoH and recent literature points to
political determinants of health (Mishori, 2019; Dawes, 2020).
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
4
Political Determinants of Health (PDoH)
Mishori (2019) states that the root causes of health and health disparities are driven by
policy and politics. Political determinants of health (PDoH) are factors, systems, or structures
(laws and policies) that create, perpetuate, or exacerbate these differences (Dawes, 2020). This
“involves the systematic process of structuring relationships, distributing resources, and
administering power, operating simultaneously in ways that mutually reinforce or influence one
another to shape opportunities that either advance health equity or exacerbate health inequities”
(Dawes, 2020). Dawes (2020) highlights multiple attempts, successes, and failures of political
determinants of health that either aimed to improve health equity or caused health inequities. For
example, the Act for the Relief of Sick and Disabled Men (1798) created the Marine Hospital
Service which eventually became the US Public Health Service. It created a mandatory maritime
tax paid by captains of each ship to develop federal hospitals to treat employed sailors. Another
example is the Freedmen’s Bureau Bill (1865) which was originally intended to provide
“protection, support, and access to healthcare, education, and employment for newly freed
people” (Dawes, 2020). Instead, it was passed without the healthcare provisions and was not
limited to only benefit the former enslaved but was inclusive for former Confederate soldiers and
low-income whites. The purpose of the revised bill was “to provide food and emergency relief to
freed people, to administer justice, to manage land confiscated during the war, and to establish
schools for the freed people” (Dawes, 2020). The Freemen’s Bureau was shut down in 1872,
impacting generations of African Americans and poor whites, leaving them unable to access care
and treatment.
One of the first bills to include anti-discrimination language was the Hill-Burton Act
(1946) which prohibited health care discrimination and funded governments to resolve major
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
5
determinants of health, access to nutritious foods (i.e., National School Lunch Act) and funding
to address old health care facilities (Dawes, 2020). However, President Reagan eliminated this
program in 1981. The Mental Health Act (1946) provided funding for research into the causes,
prevention, and treatment of mental illness, led to the creation of the National Institute on Mental
Health (NIMH), and led to the integration of mental health treatment in the community rather
than psychiatric hospital (Dawes, 2020). The Medicaid Act (1965) established the Medicare and
Medicaid programs which increased access to health care for vulnerable populations and
prohibited racial segregation in the health care system (Dawes, 2020). President Nixon defunded
and destabilized many of the policies under this Act when he was in office.
The Mental Health System Act (1980) was introduced by President Carter, and it required
states to “establish community mental health centers, providing inpatient, outpatient, and
emergency services, as well as consultation and education” (Dawes, 2020). President Reagan
repealed and rendered the provisions of this bill when he was elected to office in 1981 and ended
community treatment for the mentally ill. Lastly, the Affordable Care Act (2010) was the most
comprehensive health law that created a more “accessible, equitable, and inclusive health
system” (Dawes, 2020). The history of health policy shows that politics often drive access to
health care and the health outcomes of the citizens it serves, however as administrations change,
so to do the policies that impact its citizens. It also shows that “successes and failures in
advancing health-equity focused policies and rights to resources” are in the hands of the
legislative, executive, and judicial branches (Dawes, 2020).
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
6
Research Question
• Although federal urban policy has been passed over the past 46 years, the desired
social impact of these policies has not been evaluated to assess their overall
effectiveness or impact they have had on poverty or on the health of communities.
What common core measures should be used to assess the impact of urban
policies on the social determinants of health?
Research Purpose
The purpose of this study is to develop an evaluation framework and core set of SDoH
measures to assess the impact of urban policy at the local level on the social determinants of
health (SDoH) in urban communities. The Collective Impact Evaluation Framework was used
for the study design. A Collective Impact Evaluation Framework is used when the desire is to
achieve large-scale, sustainable change (Preskill, Parkhurst, & Juster, 2014). It supports the
complex implementation of urban policy and allows for a mixed methods research design to
assess the implementation, effectiveness, and impact of the policy. A literature review and
environmental scan of past urban policies provided insight on what key evaluation measures and
outcomes should be used to evaluate the impact of urban policy on SDoH. A qualitative analysis
of guidance documents, evaluation reports, administrative documents, and government reports
was also conducted to identify key measures, partnerships, programs, activities, and policies that
were employed to support implementation and evaluation of past urban policy in targeted
communities. SDoH measures identified were then used to develop an evaluation framework that
can be used to assess if SDoH improve for the people in communities that have been targeted by
urban policies over time.
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
7
Contribution to Practice
A recent GAO (2020) report highlights that Congress did not designate an agency with
the responsibility and authority to collect data, evaluate, and report on OZ performance and
concluded that “there are insufficient data available to evaluate OZ performance”. The GAO
recommended Congress: (1) “consider providing Treasury with authority and responsibility to
collect data and report on OZ’s performance, in collaboration with other agencies and (2)
consider identifying questions about OZ’s effects that it wants Treasury to address in order to
help guide data collection and reporting of performance, including outcomes” (GAO, 2020).
Also in 2020, Congress acknowledged and supported public health’s role in “working across
sectors on social determinants of health” and passed the Social Determinants Accelerator Act of
2019. There was continued support for this bill in 2021 with the reintroduction of the Improving
Social Determinants of Health Act of 2021 (H.R.379), and a final an approved FY2022
appropriation bill with an increase to $5 million to address SDoH. There were two primary goals
proposed in this bill: “1.) improve health outcomes and reduce health inequities through the
coordination of SDOH activities across the CDC; and 2.) improve public health agencies’ and
community organizations’ capacity to address SDOHs in communities” (Brown, 2021). The
contribution of my dissertation can be applied at the federal and local levels to address two of the
proposed SDoH activities by providing an evaluation framework, social determinants of health
measures, and data sources to measure the impact of urban policy on the SDoH in communities
targeted by urban policy. These two activities included: 1). coordinate, support, and align CDC
SDOH activities with other Federal agencies and 2). collect and analyze data related to SDOHs
(Brown, 2021).
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
8
Need for the Study
When the most recent urban policy, the Tax and Jobs Act was signed into law creating
over 8,700 “qualified opportunity zones,” there was little coordination between Federal agencies
to implement or evaluate the new policy and there was no framework, implementation plan or
guidance to achieve the goals. Over a year later, the White House Opportunity and Revitalization
Council was established by the President to coordinate across multiple agencies and improve the
grant application processes, identify program gaps in knowledge, and to effectively use existing
resources (White House Opportunity and Revitalization Council, 2019). This included
developing an Implementation Plan, using language in existing federal programs and grants to
target funding, streamline processes, and coordinate existing programs in Opportunity Zones
(White House Opportunity and Revitalization Council, 2019).
The White House Opportunity and Revitalization Council Implementation Plan and the
federal Opportunity Zone provision both required measurement and analysis, but did not provide
specific data, metrics and methodologies needed to measure the effectiveness and impact (social
and economic) of the policy, nor was an agency designated to monitor the impact of this policy.
In 2019, the HUD Office of Strategic Partnership: International and Philanthropic Affairs Division
(IPAD) worked with three non-governmental entities, the U.S. Impact Investing Alliance, the
Beeck Center for Social Impact + Innovation at Georgetown University, and the Federal Reserve
Bank of New York to develop an Opportunity Zone Framework to address this gap (The U.S.
Impact Investing Alliance and the Beeck Center, 2019). The goal of the framework was to provide
investors with guidance and a sample reporting tool to collect market and impact data to analyze
the long-term outcomes of the Opportunity Zone policy (The U.S. Impact Investing Alliance and
the Beeck Center, 2019). While this framework provided some guidance, there were no specific
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
9
performance measures that were required to be reported on which would allow a specific federal
agency to evaluate this policy at a national, state, or local level.
The Collective Impact Framework (Figure 1,1) includes “a centralized infrastructure, a
dedicated staff, and a structured process that leads to a common agenda, shared measurement,
continuous communication, and mutually reinforcing activities among all participants”
(Hanleybrown, Kania, & Kramer, 2012). Using the collective impact framework includes the tools
needed to develop a shared measurement system that can assess the impact of urban policy on the
SDoH. By using a shared measurement system with the same SDoH measures across workstreams
ensures that all efforts remain aligned, and all levels of government, community-based
organizations, private investors, and public-private partners can be held accountable for collective
impact (Hanleybrown, Kania, & Kramer, 2012). A shared measurement system increases
efficiency, reduces cost, improves the quality and credibility of data collection, increases
effectiveness and tracks progress towards the common agenda (Hanleybrown, Kania, & Kramer,
2012). It includes collecting data on shared measures, a set of early performance indicators agreed
upon by the collective to ensure progress can be tracked continuously.
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
10
Table 1.1 The Five Conditions of Collective Impact Framework
Common Agenda All participants have a shared vision for change including a
common understanding of the problem and a joint
approach to solving it through agreed upon actions.
Shared Measurement Collecting data and measuring results consistently across
all participants ensures efforts remain aligned and
participants hold each other accountable.
Mutually Reinforcing
Activities
Participant activities must be differentiated while still
being coordinated through a mutually reinforcing plan of
action.
Continuous Communication Consistent and open communication is needed across the
many players to build trust, assure mutual objectives, and
create common motivation.
Backbone Support Creating and managing collective impact requires a
separate organization(s) with staff and a specific set of
skills to serve as the backbone for the entire initiative and
coordinate participating organizations and agencies
Source: Hanleybrown, Kania, & Kramer, 2012, p. 1
In 2019, HUD released a report providing recommendation for future Opportunity Zone
evaluations based off lessons learned from previous place-based incentive program evaluations
(i.e., Enterprise Zones, Empowerment Zones, and Enterprise Communities). Two major tasks
identified from this HUD (2019) report were: (1) identify a comparison set of census tracks; and
(2) identify data that can be used to track neighborhood change. In addition, a GAO (2021)
evaluation was requested by Congress due to the magnitude (impacting over 9,000 census tracts)
and the extended time for the OZ tax benefit (tax benefits for this policy do not expire until
2047). This study will expand on this guidance and study the impact of the urban policy on the
SDoH at the local level and identify a core set of SDoH measures that can be aggregated and
used to evaluate the urban policy at the national level.
Gaps in the Literature
Urban policy can be defined as a “cluster of policies that are aimed at influencing the
development of urban areas and urban lives (Wang, 2018). Over time, little policy evaluation has
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
11
been done systematically to evaluate the social impact of urban policy on communities
collectively to reduce poverty and improve the economic, social, housing conditions, and/or
health in urban communities. Policy evaluation should be conducted systematically and
comprehensively on all urban policies “to acquire insight into the consequences that are
attributed to certain policies” (Bartels, et. al, 1982).
A series of papers provide an overview of national urban policies and programs
developed and implemented from 1960 to 1994 (Kaplan, 1995; Scruggs, 1995; Warren, 1990;
Logan, 1983). For each federal program or policy, the authors provided an overview of the
historical and political context of the policy, the administration the policy or program was
established under, any bias in the policy development, and the reasons why the policy or
program failed. The authors also provide recommendations on principles and practices that
should guide future urban policy makers.
A policy analysis of the Bush and Reagan administration’s urban policies, assessed the
similarities and differences and found that both administrations’ urban policies were grounded in
economic policy, budget cuts and no new taxes (Barnes, 1990). Although Bush’s urban policies
seemed to be a continuation of the Reagan administrations urban policy, he was known to
support the federal government, not be against it (Barnes, 1990). Bush also included non-
traditional initiatives in his urban policy such as drugs, clean air, child-care, education, and
savings and loans for housing. No quantitative data was provided to describe the impact of the
policy. The author of this study recommended the need for analytic and political frameworks that
can be applied to assess the impact of urban policy and that incorporate federalism and
capitalism, and public and private factors that can create sustainable urban solutions, however no
metrics were recommended (Barnes, 1990).
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
12
Lessons learned from implementation and evaluation of the Clinton Administration’s
Empowerment Zone policy concluded that although no two Empowerment Zone programs were
alike, the evaluation design included standard evaluation criteria, using mixed methods
comparing changes over time in jobs, poverty, unemployment, housing investment, and business
investments (Cooper-McCann, 2014). The quantitative analysis used propensity score matching
procedures to compare census tracts within each enterprise zone and comparable census tracts
outside each enterprise zone. The qualitative analysis was designed to understand why positive
effects occurred in some cities but not in others. The results of the qualitative analysis indicated
that success depended on good local governance, including: the capacity to act, community
participation, and program integrity (Cooper-McCann, 2014). The results of this evaluation also
indicate that federally funded comprehensive community initiatives can expand economic
opportunity and foster sustainable community development (Cooper-McCann, 2014). The author
suggests this model from this rigorous evaluation be considered for future policy analysis of
urban policy (Cooper-McCann, 2014).
An evaluation of the Promise Zone Initiative provides an overview of the disparities
minorities face in the Promise Zone neighborhoods and highlights that data should not be the
sole source for observing these communities when making decisions for revitalization (Corona,
2016). The author suggests utilizing oral histories that provide political, cultural, and social
context and suggests that the cultural assets that neighborhoods possess are assets that planners
and policymakers should use as a road map for revitalization planning (Corona, 2016). The
author states that the federal government can better understand communities by assessing the
impact of revitalization policy on low-income communities of color (Corona, 2016). The author
concludes by recommending the federal government include the following criteria in future
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
13
Promise Zone initiatives: (1) more points for highest-need applicants, (2) capacity building by
the federal government in highest-need communities, (3) award applicants who provide plans for
preserving and creating affordable housing, and (4) provide asset-based economic development
plans for the communities they serve (Corona, 2016).
A GAO (2021) evaluation on OZ tax incentives was requested by Congress because of
the magnitude of the number of communities that will be impacted (impacting over 9,000 census
tracts) and the extended time for the OZ tax benefit (tax benefits for this policy do not expire
until 2047). Considering past urban policies were only in place for one administration, 2-10 years
if fully enacted during term and not terminated by successor, and fewer zone designations
enacted by Congress (22-3,091 census tracks), the OZ tax incentive currently in place can have a
significant impact on urban communities. A study conducted on 3,091 Enterprise Zones enacted
during the Bush and Reagan administration concluded that effects of the policies and programs
(primarily tax benefits) by both administrations benefited higher incomes and resulted in
suburbanization (Barnes, 1990). Although over thirty years ago Barnes (1990) recommended the
need for analytic and political frameworks that could be applied to assess the impact of urban
policies, these type of tax incentivized policies are still being passed without considering the
impact they have had and will continue to have on urban communities and their SDoH.
Overall, national urban policies from the 1950s to present have not been systematically
evaluated. Most of the literature on urban policy provide the historical and political context of
the policy, the administration the policy or program was established under, bias in the policy
development, and some include the reasons why the policy or program failed. While the authors
provide detailed historical and political context for urban policy or program and their challenges,
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
14
there is little to no data on the social impact of the policies provided to support claims of success
or failure to meet the original intent of the policies passed.
Methods
The Collective Impact Evaluation Framework was used for the study design. A
Collective Impact Evaluation Framework is used when the desire is to achieve large-scale,
sustainable change (Preskill, Parkhurst, & Juster, 2014). It supports the complex implementation
of urban policy and allows for a mixed methods research design to assess the implementation,
effectiveness, and impact of the policy. A mixed method design allows for a logical approach to
the research design. It includes multiple methods, different assumptions and allows for diverse
forms of data collection and analysis. The Collective Impact Evaluation Framework consist of
three key components of the Collective Impact change process: socio-political context,
implementation, and outcomes (Figure 1.1). This framework also allows for you to assess the
policy outcomes over the lifetime of the policy.
A mixed method design allows for a logical approach to the research design. The focus of
the collective impact evaluation is on short-term, intermediate, and long-term outcome and
impacts. The Urban Policy SDoH Collective Impact Evaluation Framework includes short-term
and intermediate changes in systems and policies, and long-term changes in behavioral outcomes
in targeted communities. Impacts evaluated includes long-term changes in key SDoH indicators
in targeted communities. A cross walk of the Healthy People 2030 Social Determinants of Health
Domains (SDoH) and common urban policy domains was conducted to identify the best and
most comprehensive set of SDoH measures to assess the impact of urban policy on the social
determinants of health in targeted communities.
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
15
Summary:
Urban Policy impacts the day-to-day lives of urban communities, and no research has
been done to evaluate the impact on the SDoH of people who live in the communities targeted by
urban policy. The Urban Policy SDoH Evaluation Framework can be used to assess the impact of
urban policy on SDoH at the local level and allow for the study of two timely topics (urban
policy and measuring SDoH). In addition, the White House Opportunity Zone Implementation
Plan has identified the need to develop robust reporting and analytics to measure the impact of
the urban policy in designated Opportunity Zones (White House Opportunity and Revitalization
Council, 2019). The results of this study can fill the need to develop shared measurement system
to assess the social determinants of health of urban policy, provide an evaluation framework,
social determinants of health measures, and data sources to measure the impact of the urban
policy.
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
16
Chapter 2: Literature Review
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
17
Chapter 2 provides a review of literature on urban policy, policy evaluation methods and
frameworks, and the linkage between urban policy and the social determinants of health. A
review of the literature will provide context for the research question: What common core
measures should be used to assess the impact of urban policies on the social determinants of
health? For the purpose of this literature review we will examine urban policy enacted from
1937-2019.
The History of Urban Policy
Urban policy can be defined as a “cluster of policies that are aimed at influencing the
development of urban areas and urban lives (Wang, 2018). A series of papers provide an
overview of national urban policies and programs developed and implemented from 1960 to
1994 (Kaplan, 1995; Scruggs, 1995; Warren, 1990; Logan, 1983). For each federal program or
policy, the authors provided an overview of the historical and political context of the policy, the
administration the policy or program was established under, any bias in the policy development,
and the reasons why the policy or program failed. The authors also provide recommendations on
principles and practices that should guide future urban policy makers. While the authors do an
excellent job laying the found for the historical and political context of each urban policy or
program and its challenges, there is limited data provided in these articles to support claims of
success or failure to meet the original intent of the policies described. Studies described by
Warren (1990) describe the impact of past national urban policies as having consequence for the
life of cities and people who live in them. Warren (1990) described some of these as severe, as
being more implicit effects than explicit, and that they “re-enforced market process that led to
disparities in the life chances of different individuals and communities by spatially segregating
races and classes and by clustering jobs in some areas but not others”.
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
18
The Housing Acts of 1937, 1949 & 1954 (Roosevelt, Truman, Eisenhower)
Urban policy initially began with the Housing Act of 1937 prior to World War II,
however low levels of public housing were being built and after the war Congress reenacted the
Housing Act of 1949 and continued to update it in 1954 with over 54 additional pieces of
legislation, Executive orders, and Government committee reports that augmented, changed,
and/or refined renewal policy (Scruggs, 1995; Schill & Wachter, 1995). This series of legislation
provided for slum clearance, urban redevelopment, residential renewal, and downtown central
business district revitalization. The assumption was made that Federal intervention in urban
communities would spur economic growth and produce a “trickle-down effect that would
ultimately include most marginal and nonparticipating members of society in its benefits”
(Scruggs, 1995). However, from 1950 to 1960 while income rose, relative quality of life declined
for the poor and for minorities and the twelve largest cities saw a decrease in white households
and an increase in minority households (Scruggs, 1995). Schill & Wachter (1995) found that “the
concentration of public housing in inner cities contributed both directly and indirectly to
poverty”.
Urban renewal also came under fire for discriminating against minorities, in that it often
resulted in minority-heavy substandard housing (slums) being destroyed and replaced with more
expensive housing or non-residential public works that were not accommodating to the original
community members. The Housing Act of 1937 mandated that “substandard housing be
eliminated for each unit of public housing constructed”, this was called the “equivalent
elimination requirement” (Schill & Wachter, 1995). People displaced by slum clearance were
given priority to public housing. By the time public housing that was approved from 1950-1960s,
out of the thirty-three-housing development completed, thirty-two were in communities that were
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
19
at least 84% Black, and twenty-six housing developments were in communities that were at least
95% Black (Hirsch, 1983).
The Housing Act of 1949 & 1954 also encouraged segregation and white movement to
the suburbs through unconstitutional racial zoning ordinances enacted by city governments.
When the civilian housing industry picked up in the 1950s, the federal government subsidized
mass production builders to create suburbs on conditions that those homes in the suburbs be sold
only to whites. No African Americans were permitted to buy them, and the FHA often added an
additional condition requiring that every deed in a home in those subdivisions prohibit resale to
African Americans.
One major consequence of the Housing Act of 1949 policy was that it created the largest
wealth gap in America, that we still have today. Over the next several generations, those homes
would sell for seven-to-eight times the median national income – unaffordable to working-class
families or African Americans. The public segregation that took place when the homes were first
built created a permanent system that locked minorities out of it as appreciation grew. White
families gained in home equity, in wealth, from the appreciation of their homes while minorities
who were forced to live in apartments and as non-homeowners gained none of that appreciation.
Another major consequence of this policy was property values decreased in urban communities,
business investments decreased in urban communities, and poverty increased in urban
communities.
The Housing Act of 1954 shifted urban policy focus from building public housing to
commercial revitalization (Flanagan, 1997). Flanagan (1997) states that the shift was due to
interactions between state institutions and special interest groups about their ideas about what
policy should include. These special interest groups included mayors, developers, and downtown
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
20
businesspeople who collectively brought together their local power to influence policy in
Washington, DC (Flanagan, 1997). As a result, the number of public housing units being
constructed decreased from 52,000 per year in 1952 to 24,000 per year in 1964 (Flanagan, 1997).
Data also shows that the number of urban renewal projects increased from 260 projects in 1953
to 1,210 in 1962 (Flanagan, 1997). The Housing Act of 1954 biggest policy impact was the shift
of national urban policy from public housing to urban renewal (Flanagan, 1997).
The Grey Areas Project
In the mid-1950s, the Ford Foundation and HUD established the Grey Areas Project to
develop a new formula for equalizing the competitive disadvantage for the urban areas
comprised of the displaced, the relocated, migrants, racial minorities, and economically and
socially marginalized people (Scruggs, 1995). The Grey Areas Project introduced the concepts of
institutional change, social engineering, housing, and redevelopment projects, and utilized the
expertise of social analysts, citizens, and service-delivery practitioners. The Grey Areas project
was made up of 18 communities: Syracuse and Onondaga County, New York; Houston, Texas;
St. Louis, Missouri; Minneapolis, Minnesota; Los Angeles, California; Harlem and New York
City, New York; San Francisco, California; Chicago, Illinois; Charleston, West Virginia;
Cleveland, Ohio; Providence, Rhode Island; Eugene, Oregon; New Haven, Connecticut; Boston,
Massachusetts; Washington, D.C.; and the State of North Carolina (Kaplan, 1995). Collectively
they were able to develop community action plans that changed the way communities felt,
behaved, looked physically, and functioned economically (Scruggs, 1995).
The Grey Areas Project led to much of the language in the Economic Opportunity Act of
1964 and Title II which included provisions for community action plans and a comprehensive,
coordinated approach for improving the quality of life in “pockets of poverty” in the cities,
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
21
modeled after the Grey Areas Projects. However, “local white politicians were agitated and
became hysterical when the federal government permitted, and often encouraged, its new
apparatus of local agencies to put pressure on municipal services themselves—pressure to get
more for blacks” (Scruggs, 1995). By 1967, the Grey Areas Project was shifted to the Model
Cities Project to address President Johnson’s War on Poverty.
The Model Cities Program/War on Poverty (Johnson)
Under the Economic Opportunity Act of 1964 (The War on Poverty) and the
Demonstrations Cities and Metropolitan Development Act of 1966, President Johnson formed a
private, nongovernmental taskforce, Task Force on Urban Problems to provide him with
guidance and recommendations to his national urban policy. James (1972) describes three
problems this task force was to address from past urban policy challenges:
“1. dissatisfaction with urban renewal programs and related efforts to deal with urban slums; and
2. difficulty in reconciling citizen participation and decentralization of government programs
with the interests of established bureaucracies and officials; and 3. fragmentation and lack of
coordination of national grant-in-aid programs.”
The Model Cities Program (1966-1972) was developed as a part of this “place-based”
solution, 150 cities were chosen as “research laboratories for the war against poverty”, and
legislation allocated $924 million dollars to the program (Scruggs, 1995; Schechter, 2011). The
aim of the program was “to expand housing, job, and income opportunities; to reduce
dependence on welfare payments; to improve educational facilities and programs; to combat
disease and ill health; to reduce the incidence of crime and delinquency; to enhance recreational
and cultural opportunities; to establish better access between houses and jobs; and generally, to
improve living conditions” (Schechter, 2011).
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
22
The Model Cities Project also shifted the tone, image, and marketing of who President
Johnson was aiming his war on poverty to help (NC DNCR, 2020). Figure 2.1 illustrates his tour
called the “Poverty Tour” to gain support and shift the narrative of his War on Poverty from
giving more to poor Black communities to giving more to poor white communities (NC DNCR,
2020). The administration gained approval for the War on Poverty to expand the welfare
program, expand the social security programs, and to create new Medicare and Medicaid
programs. It also included a large housing package and civil rights legislation to reduce housing
and employee discrimination. However, the welfare programs were designed to provide income
to the poor and services, rather than programs to help people out of poverty (Flanagan, 1997). A
consequence of the War on Poverty included an increase in the number of blue-collar and middle
-class Black and Hispanics moving out of inner-city neighborhoods, leaving many poor
minorities in deteriorating inner city neighborhoods (Kaplan, 1995).
Figure 2.1 The War on Poverty Tour
Source: NC DNCR, 2020
Schechter (2011) conducted an empirical evaluation to assess the impact of the Model
Cities program and found it had no impact on many of its targeted cities compared to cities that
did not receive funding. Additional, Schechter (2011) found that on average there were larger
increase in poverty rates in Model Cities than in comparison group cities. Hetzel’s (1994)
evaluation concluded that increases in the numbers of persons employed in the area, number of
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
23
businesses in the area, and in living conditions relating to crime, education, housing, and other
commercial and social services determine if the effort have been successful. By 1973, President
Nixon took office and dismantled the Model Cities Program, combining Model Cities Programs,
Urban Renewal and HUD Community development programs into Community Development
Block Grants (CDBG) (Schechter, 2011).
Community Development Block Grant (Nixon and Reagan)
By 1969, President Nixon took office and dismantled the Model Cities Project before it
could be fully implemented. In 1970 Nixon established a short-lived Urban Affairs Council that
was replaced with a new Department of Community Development. Congress created the
Community Development Block Grant (CDBG) program under the Housing and Community
Development Act of 1974 that fully replaced the remaining Urban Renewal, Neighborhood
Development, and Model Cities programs. Community Development Block Grants consolidated
low-rent public housing, water and sewer facilities grants, open-space land grants, and public
facility loans. These formula -based grants gave the mayor broad authority and flexibility to
determine the operations the community would undertake. President Nixon had many attempts of
getting Congress to approve multiple block grants but was only successful at getting two block
grants enacted, CDBGs and the Comprehensive Employment and Training Act (CETA) (Conlan,
1984).
President Nixon’s programs also took more of a person-focused approach, than a place-
based approach. Rather than targeting specific communities, he often focused on programs that
supported his suburban constituents, urban growth, and reform, rather than urban distress. His
support for fair housing programs and rental assistance programs for leased housing also
strengthened his support for people vs. place-based urban policy strategies (Kaplan, 1995).
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
24
Although there was no formal evaluation conducted, data indicated that from 1970-1975, the
U.S. population changed from urban to suburban during his administration, more Black people
were in cities, and unemployment was also increasing in cities (Scruggs, 1995).
The Reagan administration was more successful with enacting block grant initiatives,
creating nine new block grants in The Omnibus Budget Reconciliation Act of 1981, however it
consolidated seventy-seven programs and terminated sixty-two programs (Conlan, 1984). The
nine block grants included four in health services, three in social services, one in education, and
one in community development (Morgan & England, 1984). President Reagan focused on a
market approach that put more emphasis on national economic performance and less focus on the
federal role towards urban areas (Wolman, 1986). Nathan (1982) stated that “The Omnibus
Budget Reconciliation Act of 1981 is the single most important piece of social legislation
enacted in the United States since the Social Security Act of 1935”. This legislation “did not just
cut spending, it fundamentally changed social programs” (Nathan, 1982).
A study done by Wolman (1986) found that Reagan’s urban policy drastically cut
funding to urban programs. President Reagan decreased funding to urban areas in his first year,
resulting in cumulative reductions in CDBGs (-11%), Urban Development Action Grant
(UDAG) (-33.6%), Mass transit (-30%), CETA (-100%); Wastewater treatment (-32.2%),
Economic Development Administration (EDA) (-67.5%), Compensatory Education (-21.9%),
Social Services Block Grant (-21.3%), Community Service Block Grants (-40.1%), and
Subsidized Housing (-42.8%) for a total of a -66.3% reduction in urban programs (Wolman,
1986). As the economy recovered from the recession, from 1983 to 1984 unemployment rates for
the largest metropolitan areas decreased from 8.6% to 6.8%. However, they did not decrease
equally, for example Pittsburg had an unemployment rate of 10.7% compared to areas like
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
25
Boston, Dallas, and Washington, DC which have unemployment rates less than 4% (Wolman,
1986). There was no data available at the central-city resident level to show these disparities for
all cities examined. Wolman (1986) states that “national economic growth cannot necessarily be
relied upon to reduce these disparities”. Under President Carter and President Ford
administrations no new block grants were approved by Congress (Conlan, 1984).
A New Partnership to Conserve America’s Communities (Carter)
In 1977, President Carter signed an Executive Order to establish the Urban and Regional
Policy Group (URPG), a White House working group on urban redevelopment.
His goal was to develop a coherent national urban policy that was “consistent, compassionate,
realistic, and that reflected the decency and good sense of the American people” (Scruggs, 1995).
After over 1,000 meetings with community members, private industry, local and state
governments, and civil rights organizations, A New Partnership to Conserve America’s
Communities: A National Urban Policy was announced. President Carter “urged the necessity of
resolving domestic problems through "shared responsibility" among public and private partners
and all levels of government (Barnes, 2005). He believed a healthy city must have healthy
businesses to provide jobs and revenues (Barnes, 2005).
The Carter administration’s urban policy proposed to address the following problems:
“1). the longer-term problems are often those that do not make headlines and are unlikely to be
resolved within a four-year electoral cycle; 2). federal action is needed on those issues that
reflect more than one set of problems, thereby requiring more than a single set of programs and
agencies to address them; and 3). substantial cooperation among jurisdictions” (Houstoun, 1979).
President Carter wanted to address the problems of all cities through “seven governing
principles, four goals, ten policies, and thirty-eight strategies” (Kaplan, 1995). This included
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
26
economic development, job creation, revitalization of distressed neighborhoods, sprawl and
growth, improvements of services in distressed urban areas, increased mobility for poor people,
environmental concerns, deteriorating housing stock and physical plan (Kaplan, 1995).
State Enterprise Zones (Reagan and Bush)
State enterprise zones were established during the Reagan Administration under the
Urban Jobs and Enterprise Act in 1984. The urban policy included experimental programs
designed to stimulate economic development and revitalization in distressed communities. The
concept was developed as an approach to reduce Black unemployment and encourage growth
and retention of Black businesses (Jones, 1987). There were no requirements in the federal
policy to engage the Black community or encourage access to capital markets (Jones, 1987).
Rather, there was an assumption that upon implementation of the policy, local governments
would upon their own will include Black business and residents in development of these zones
(Jones, 1987).
The eligibility criteria were designed based on the Urban Development Action Grant
(UDAG) program (Mounts, 1981). The zone had to be “located in a UDAG-eligible area, be
contiguous, and in metropolitan areas that have at least 4,000 inhabitants; outside of SMSAs the
minimum population requirement is 2,500; must qualify as a low-income area on the basis of at
least one of several optional criteria measuring income, unemployment or population loss; must
be a city, or a state with the approval of the city or town where the zone will be located; must
include a formal expression of commitment (i.e. lower taxes and fees in the zone, development
requirements, or building standards) to the zone in the form of its own contributions for
economic development incentives” (Mounts, 1981).
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
27
In a study aimed to examine state enterprise zones and whether development would have
occurred in these areas without the policy or programs the author conducted a case study on one
opportunity zone, Decatur, Illinois (Jones, 1987). It was selected as a case study because it
closely modeled the original concept of the policy, to stimulate economic development and
neighborhood revitalization, as well as to provide incentives to grow and develop enterprise
zones. Results from this case study indicate that development and investments increased after
enterprise zone designation, however the initial increase began before designation. The results
also state that due to the decline in interest rates during this time, it is likely that the same rate of
increase in development and investment may have occurred. Jones (1987) notes that without
comparable data for analysis, the conclusion is tentative. Ultimately, the author concluded that
enterprise programs had minimal effect on neighborhood revitalization and business growth.
This study supports other studies that control cities should be considered for analysis purposes.
Title VII of Public Law 100-242, the Housing and Community Development Act of
1987, was the only law that was enacted to establish an enterprise zone program. It provided the
Department of Housing and Urban Development (HUD) authority to designate up to one
hundred enterprise zones and offer incentives (such as waivers on some HUD regulations that
otherwise would apply) to businesses that were in the enterprise zone designated areas. The
number of zones established ranged for each state from 1 (in Michigan) to 1,625 (in Louisiana).
There were also four states whose state programs ended at the time of the HUD update (Wilder
& Rubin, 1996). There was no explanation provided on why the programs ended or why the
number of zones varied by state. Congress did not enact EZ and EC legislation until 1993,
thirteen years after the first enterprise zone bill had been introduced in Congress (Hyman, 1998).
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
28
In 1988, the GAO conducted a study on Enterprise Zones-Lessons from the Maryland
Experience. The GAO (1988) defined Enterprise Zones are economically distressed areas
designated for preferential governmental treatment to promote investment and job creation by
private industry. Through this policy, incentives to businesses that hire employees or invest
within enterprise zones could receive substantial tax incentives and regulatory relief. The
rationale behind enterprise zones was that reducing governmental burdens on industry (such as
taxes and regulation) would compensate for costs associated with operating businesses in
distressed areas (such as high crime and untrained labor). The GAO (1988) Enterprise Zones-
Lessons from the Maryland Experience had the following objectives:
1. “Examine federal agencies’ assumptions about the performance of a federal enterprise
zone program, particularly as they relate to program cost offsets,
2. measure employment growth in state-designated enterprise zones,
3. measure offsets to the cost of an enterprise zone program resulting from program effects
on workers,
4. measure reductions in welfare dependence associated with an enterprise zone program,
and
5. assess the relative effectiveness of tax incentives and other local development strategies”
The GAO (1988) found in the Maryland Experience that enterprise zones were not
effective. GAO'S results suggest that a modest demonstration program might be more useful than
the large-scale efforts proposed in several bills and the policy should be designed to evaluate
different program options and to allow sound evaluation of demonstration results. In 1982 prior
to this study, the GAO conducted a study to determine the effectiveness of the proposed state
enterprise zone concept as an innovative approach to revitalize distressed areas by providing tax
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
29
relief and regulatory reform to stimulate business development and create jobs. As a result of this
assessment the GAO recommended Congress “require the administering federal agency to
establish program effectiveness criteria supported by a systematic data collection and evaluation
effort to analyze the benefits and costs of the program if enterprise zone legislation is enacted”
(GAO, 1982). However, the status of this recommendation by GAO, is documented as not
implemented (GAO, 2021).
Bush established Enterprise Zones that were designed to attract private investments and
new small businesses to the inner city via tax relief and reduction of government regulations; to
replace abandoned commercial spaces with new businesses; and to increase in jobs for local
residents. Congress would not approve the “no tax incentive” so Enterprise Zones were not able
to be fully implemented or evaluated during his term. During the Bush Administrations, direct
aid to cities decreased by almost one-third and federal aid from the federal government to states
and noncity localities, such as counties, suburbs, and rural districts, increased almost 100 percent
in that same period, from $69 billion to over $114 billion (Scruggs, 1995).
A policy analysis of the Bush and Reagan administration’s urban policies examined the
similarities and differences and found that both administrations’ urban policies were grounded in
economic policy, budget cuts and no new taxes (Barnes, 1990). Although Bush’s urban policies
seemed to be a continuation of the Reagan administrations urban policy, he was known to
support the federal government, not against it. Bush also included non-traditional initiatives in
his urban policy such as drugs, clean air, child-care, education, and savings and loans for
housing. The author concludes that effects of the policies and programs by both administrations
benefited higher incomes and resulted in suburbanization (Barnes, 1990).
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
30
Empowerment Zones and Enterprise Communities
President Clinton’s urban policy established Empowerment Zones, Enterprise
Communities, and Renewal Community (EZ/EC/RC) programs in 1993 and in subsequent
legislation in 1997, 1999, and 2000, to reduce unemployment and generate economic
growth in selected Census tracts (GAO, 2012). Urban and rural communities designated as
EZs, ECs, or RCs received grants, tax incentives, or a combination of both to stimulate
community development and business activity. EZ, EC, and RC programs focused on four
key principles: economic opportunity, community-based partnerships, sustainable community
development and a strategic vision for change (Herbert, 2001). Although no two Empowerment
Zone programs were alike, the national evaluation design included standard evaluation criteria,
using mixed methods comparing changes over time in jobs, poverty, unemployment, housing
investment, and business investments. A qualitative analysis was also conducted to understand
why positive effects occurred in some cities but not in others (Cooper-McCann, 2014).). The
results of the qualitative analysis indicated that the success of enterprise zones depended on good
local governance: the capacity to act, community participation, and program integrity (Cooper-
McCann, 2014).
A study aimed to evaluate whether the urban policy, Empowerment Zones alleviated
poverty and improved socioeconomic conditions within the enterprise zone itself and their
adjacent neighborhoods included analysis of 1990 and 2000 census data on population size, level
of poverty, percent unemployment, percent homeownership, percent vacancy, percent living in
the same household for the last five years, household income (adjusted for inflation), percent
non-Hispanic white, percent non-Hispanic black, and percent Hispanic (Oakley, D., & Tsao, H.,
2007). Results from this study were mixed and indicate that the zone area and the adjacent
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
31
neighborhoods experienced significant socioeconomic gains in the form of poverty and
unemployment reductions. However, there were no gains in the other socioeconomic variables.
The authors of this study conclude with two final questions to consider: (1) Can socioeconomic
progress within and around the zone area be attributed to this single policy initiative? (2) When
multiple public policies and/or private programs are occurring in the same zone at the same time,
is it possible to isolate the effect of one initiative (Oakley, D., & Tsao, H., 2007)?
An overview of lessons learned from implementation and evaluation of the Clinton
Administration’s urban policy; Empowerment Zones found that no two Empowerment Zone
programs were alike (Cooper-McCann, 2014). The evaluation design included standard
evaluation criteria, using mixed methods comparing changes over time in jobs, poverty,
unemployment, housing investment, and business investments (Rich & Stoker, 2014). The
quantitative analysis used propensity score matching procedures to compare census tracts within
each enterprise zone and comparable census tracts outside each enterprise zone. The qualitative
analysis was designed to understand why positive effects occurred in some cities but not in
others. The results of the qualitative analysis indicated that success depended on good local
governance: the capacity to act, community participation, and program integrity (Cooper-
McCann, 2014).). The results of this evaluation also indicate that federally funded
comprehensive community initiatives can expand economic opportunity and foster sustainable
community development. Rich and Stoker (2014) suggest the model from this rigorous
evaluation be considered for future policy analysis of urban policy.
Busso, Gregory, and Kline (2010) conducted an empirical study with a focus on
quantifying the effects of the Empowerment Zones (EZ) Program of the 1990s on job creation
and wages. They use Census data and information from the Longitudinal Business Database and
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
32
a difference-in-differences (DID) strategy to compare funded empowerment zones to areas that
were nominated to the program but did not receive EZ grants. Busso, Gregory, and Kline (2010)
found that the empowerment zone programs increased employment and generated wage
increases for residents of empowerment zones. Busso and Kline (2008) used propensity score
matching techniques to assess the causal, direct effects on a variety of neighborhood
characteristics and concluded that there are significant, robust, and generally positive effects on
neighborhood racial makeup, employment, educational attainment, poverty, and home values.
A special report on community financing in eighteen Empowerment Zone and Enterprise
Community (EZ/EC) sites found that residential loan activity was more prevalent in metropolitan
statistical areas (MSA) and EZ/EC sites by 2:1 ratio. For the total MSAs, the proportion of loans
originated by low-income, working-class, and moderate-income individuals was also higher than
the proportion of loans originated by residents of the EZ/EC sites (Nelson A. Rockefeller
Institute of Government, 1997). The quantity of SBA-related lending per qualified firm was
higher in the EZ/EC than in the MSA in nearly half of the sites (Nelson A. Rockefeller Institute
of Government, 1997).
There were multiple studies conducted by the GAO and HUD that concluded that there
was either not enough data to evaluate EZ, EC and RC programs or that the previous studies had
not produced definitive results about the effectiveness of Empowerment Zone tax incentives
(GAO, 2012; GAO, 2004; GAO, 2006, Nelson, 1997; HUD, 1997; HUD; 1998; HUD, 1999). In
2012, the GAO found that there was a 730-million-dollar loss of revenue in 2010 in
Empowerment Zones and Renewal Communities (GAO, 2012). The GAO (2012) also identified
a methodological challenge that prevented them from demonstrating a causal relationship
between community development efforts and economic growth in specific communities (GAO,
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
33
2012). In a 2004, GAO recommended that HUD, USDA, and IRS collaborate to “(1) identify the
data needed to assess the use of the tax benefits and the various means of collecting such data;
(2) determine the cost-effectiveness of collecting these data; (3) document the findings of their
analysis; and, if necessary, (4) seek the authority to collect the data, if a cost-effective means was
available” (GAO, 2012).
The GAO (2004) concluded that the EZ and RC programs had been implemented but that
limited data tax benefits presented several challenges to evaluating the programs and that
acquiring additional data that could attribute the use of the tax benefits to EZs and RCs would be
needed to audit EZ and RC programs (GAO, 2004). In addition, the GAO concluded that
additional tax data would be necessary to evaluate certain aspects of the programs, such as the
use of the tax benefits (GAO, 2004). The GAO stated that without utilization data, EZs and RCs
“could not reliably report on how local businesses used program funds, limiting the ability of
GAO and other researchers to determine the programs’ impact on designated communities”
(GAO, 2010). Overall, the lack of data collected by HHS, USDA, HUD and IRS, lack of
reporting requirements, or clear guidance on how to monitor program grant funds limited
program oversight and GAO’s ability to assess the effect of the EZ, EC and RC programs (GAO,
2004; GAO, 2006; GAO, 2010; GAO, 2012).
Promise Zones
In 2013, President Obama announced the establishment of the Promise Zones Initiative to
“partner with high-poverty communities across the country to create jobs, increase economic
security, expand educational opportunities, increase access to quality, affordable housing, and
improve public safety” (HUD Exchange, 2020). Previous urban policies in other administrations
have been less holistic, with movements towards tax relief to ensure economic development but
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
34
with little social investment (OECD, 2017). Promise Zones were place-based designations in
President Obama’s federal urban policy. They are defined as “high poverty communities where
the federal government partners with local leaders to increase economic activity, improve
educational opportunities, leverage private investment, reduce violent crime, enhance public
health and address other priorities identified by the community” (HUD Exchange, 2020).
The goal of the Promise Zone policy was to change the way that the federal government
interacted with local communities and to ensure alignment of federal funding streams for
comprehensive neighborhood revitalization. The initiatives within the Promise Zone policy were
designed to build upon the pre-existing Neighborhood Revitalization Plan which included
technical planning assistance to support local strategic planning. The Obama administration's
Neighborhood Rehabilitation Initiative (NRI) included the Choice Neighborhood and Byrne
Criminal Justice Innovation grant programs, which were anticipated to constitute the backbone
of revitalization activities in Promise Zones (Stoker & Rich, 2020).
The Obama administration’s federal urban policy aimed to address two persistent
problems of governance: resource scarcity and policy coordination (Stoker & Rich, 2020). To
address this problem, the administration designed a strategy that was comprised of
“interconnected solutions” to “interconnected problems” through interagency collaboration
(Stoker & Rich, 2020). Their approach was “interdisciplinary, place-based, locally-led, data and
results-driven, and flexible” comprised of “community-owned revitalization initiatives” (Stoker
& Rich, 2020). These initiatives were locally planned, comprehensive, and integrated local
initiatives (Stoker & Rich, 2020). The federal urban policy also included coordinated and pooled
federal resources together in the same community across multiple policy domains (Stoker &
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
35
Rich, 2020). Figure 2.2 illustrates President Obama’s Administration’s Model for
Intergovernmental Urban Policy Making.
Figure 2.2. President Obama’s Administration’s Model for Intergovernmental Urban
Policy Making
Source: Stoker & Rich, 2020, p.1610. The Obama administration’s model of
intergovernmental urban policy making.
Obama’s urban strategy can be traced to an “urban policy working group” within the
White House Office of Urban Affairs that promoted the Neighborhood Revitalization Initiative
(NRI) to create “neighborhoods of opportunity.” In the early rounds initiatives within the
Promise Zone policy were designed to build upon a pre-existing Neighborhood Revitalization
Plan which included technical planning assistance to support local strategic planning. HUD was
the federal lead for the fourteen urban designees and USDA served as the lead federal partner to
the tribal and rural Promise Zones. The core components and benefits of being a Promise Zone
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
36
designee included a assigned a federal staff to help navigate the array of Federal assistance and
programs already available to them. In addition, eligible applicants in Promise Zones received
preference points for certain competitive Federal programs and technical assistance. The original
policy also included language of a tax incentive subject to enactment by Congress, which would
incentivize businesses investing in Promise Zones or hiring residents of Promise Zones. Each
urban, rural, and tribal Promise Zone applicant was asked to put together a clear description of
how the Promise Zone designation would accelerate and strengthen the community’s own efforts
at comprehensive community revitalization. Each Promise Zone was coordinated by a lead
community-based organization in partnership with federal agencies.
Over three years, 22 Promise Zones were selected, 13 Promise Zones invested in
strategies with proven effectiveness by leveraging millions in additional funding, creating new
jobs, training for teachers and parents, creating social enterprises, providing workforce training,
reducing violent crimes, and increasing graduation rates (White House, 2016). Figure 4.0
illustrates the location of each round of Promise Zone designations. There were three rounds of
Promise Zone designations, each round increased in number of Promise Zone designees. In the
first round five Promise Zone communities were designated. The Promise Zone designation has a
term of 10 years, and the tax incentives portion was enacted during the Trump Administration.
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
37
Figure 2.3 Three Rounds of Promise Zones
Source: HUD, 2020
In an overview of Obama Administration’s urban policy initiative including Promise
Zones, Choice Neighborhood Grants and Partners for Sustainable Communities Grants one study
described key components of the policy and identified a gap in President Obama’s urban policy,
lack of a small business lending component (Hyra, 2015). The author provides evidence on the
importance of place-based small business lending and shares a case study of a small business in
Washington, DC that was displaced because of urban revitalization (Hyra, 2015). The author also
suggests that preventing displacement may also reduce social tensions and stabilize emerging
mixed-income and mixed-race neighborhoods (Hyra, 2015). Lastly, the author provides policy
recommendations on how this might be done using existing federal programs such as the Small
Business Administration’s 504 Program. The 504 programs could be used to increase loan
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
38
guarantee amounts for federal guaranteed small business loans to existing small business in the
designated Promise Zones (Hyra, 2015).
Historically, a new administration dismantles the previous administrations urban policy
programs. However, self-selected Promise Zone leaders formed the National Coalition of
Promise Zones (NCPZ) and were able to sustain the Promise Zone designations and incentives.
The NCPZ was able to mobilize and use their collective impact to frame the Promise Zones as
high capacity, low-cost initiatives that promoted collaborative local problem-solving through
public-private partnerships that stimulated market activity to encourage business investment and
job creation. They were successful with this approach because the new Trump administration did
not take steps to undermine or dismantle the program or terminate their 10-year designation or
benefits. Rather, the Tax Cuts and Jobs Act of 2017 expanded the Promise Zones benefits by
including Promise Zones as beneficiaries to the same Opportunity Zone tax benefits. The
coalition was also able to persuade states and locals to designate the Promise Zones as
Opportunity Zones. Other benefits the Promise Zone designees were able to take advantage of
included the 2018 Omnibus Spending Bill which reaffirmed Promise Zones designations and
agreements with HUD and gave Promise Zones more extensive grantmaking preferences.
The Obama Administration was successful at implementing and sustaining Promise
Zones due to the formation of National Coalition of Promise Zones and the data collection and
evaluation requirements. The coalition’s formation and overall success was due to mobilization
and policy maintenance. They were able to use data collectively to maintain and expand the
Promise Zone designation and incentives. The policy empowered communities to collectively
implement, evaluate, and maintain their programs. By enhancing local capacity, encouraging
collaboration, and supporting local initiatives through the coordinated use of existing federal
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
39
resources, and requiring programs to collect data, and evaluate their programs, the Obama
Administration was successful at empowering Promise Zones to collectively sustain Promise
Zone designations and incentives in their communities. Although no new Promise Zone
communities were established after round 3 (Figure 2.3), existing Promise Zones were able to
keep designations and qualify for some of the same Opportunity Zone benefits (i.e., extra points
on federal grants). In February 2019 HUD released an initial assessment report of findings based
on the experiences of three initial urban cities, but the report was not intended to evaluate the
overall effectiveness of the Promise Zone initiative or quantify the program’s impact on
neighborhoods and residents (HUD, 2019).
Opportunity Zones
In 2017, President Donald Trump signed into law a bill, the Tax and Jobs Act, which
included a federal tax incentive to promote long-term investments in low-income communities
designated as “qualified opportunity zones” (White House, 2018). Over 8,700 census tracts were
designated as Opportunity Zones across all fifty states, the District of Columbia and five U.S.
territories (White House, 2018). Opportunity Zones are disproportionately located in
communities of color (Green, et. al., 2018). Most households in Opportunity Zones are African
American and Latino; have higher poverty rates; and lower rates of public and private investment
than the national average (Green, et. al., 2018).
Through the Opportunity Zone policy, public and private investment resources can be
leveraged to address housing insecurity and economic inequality in the communities designated
as “opportunity zones” (White House, 2018). The intent of the Opportunity Zones legislation
was to bring ‘‘the best possible mix of investments in new and expanding businesses,
infrastructure and energy projects, commercial real estate, affordable housing and more’’ (Joint
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
40
Economic Committee, 2018). The main goals of Opportunity Zone policy were to “stimulate
economic opportunity and mobility, encourage entrepreneurship, expand quality educational
opportunities, develop and rehabilitate quality housing stock, promote workforce development,
and promote safety and prevent crime in urban and economically distressed communities”
(White House, 2018).
The White House Opportunity and Revitalization Council established by the President in
2018 to solve the problem of each Federal agency “independently targeting public investment
programs according to many different priorities and requirements” (White House Opportunity
and Revitalization Council, 2019). The Council was established to solve these problems by
“targeting, streamlining, and coordinating existing Federal programs to Opportunity Zones and
other economically distressed communities” (White House Opportunity and Revitalization
Council, 2019). The Council’s Implementation Plan require measurement and analysis, but did
not provide specific data, metrics, and methodologies requirement to measure the effectiveness
and impact (social and economic) of public-private investments in opportunity zones.
Three non-governmental entities, the U.S. Impact Investing Alliance, the Beeck Center
for Social Impact + Innovation at Georgetown University, and the Federal Reserve Bank of New
York developed an Opportunity Zone Framework (The U.S. Impact Investing Alliance and the
Beeck Center, 2019). The framework was developed using the information gathered from
roundtables with community development investors, researchers, policy makers and philanthropy
(The U.S. Impact Investing Alliance and the Beeck Center, 2019). The goal of the framework
was to provide investors with guidance and a sample reporting tool to collect market and impact
data to enable independent evaluators and researchers to analyze the long-term outcomes of the
opportunity zone policy (The U.S. Impact Investing Alliance and the Beeck Center, 2019). The
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
41
framework includes guiding principles, a reporting framework, and a shared goal to measure
outcomes (The U.S. Impact Investing Alliance and the Beeck Center, 2019). However, there is
no literature available to support the implementation of this framework.
In one evaluation of the Trump Administration’s Opportunity Zone policy the author
used GIS to illustrate the percent vacant business addresses using natural breaks classification
overlaid with opportunity zones (Din, 2018). The author recommended monitoring the business
vacancy rate in Opportunity Zones to measure community and economic development and
hypothesizes that as Opportunity Funds are created, new businesses will develop within the
Opportunity Zones and attract other businesses that are not part of Opportunity Fund (Din,
2018). The author also hypothesizes that workers will be recruited from local communities and
thus job creation will also increase (Din, 2018). The limitation of this study is that it only
provides a GIS map of one point in time. It would be helpful to see a comparison map overtime.
A recent GAO (2020) report highlights that Congress did not designate an agency with
the responsibility and authority to collect data, evaluate, and report on OZ performance and
concluded that “there are insufficient data available to evaluate OZ performance”. The GAO
recommended Congress: (1) “consider providing Treasury with authority and responsibility to
collect data and report on OZ’s performance, in collaboration with other agencies and (2)
consider identifying questions about OZ’s effects that it wants Treasury to address in order to
help guide data collection and reporting of performance, including outcomes” (GAO, 2020).
Policy Evaluation Methods and Frameworks
The purpose of this study is to develop an evaluation framework and core set of SDoH
measures to assess the impact of urban policy at the local level on the social determinants of
health (SDoH) in urban communities. In order to develop an evaluation framework, it is
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
42
important to provide a clear understanding of policy evaluation, methods, and current
frameworks used in policy evaluation and measuring social impact. It is also important to
provide literature on the collective impact evaluation framework and its core components.
Policy Evaluation
An analysis of case studies conducted to systematically study policy theories, describes
the pitfalls of the case studies, provides recommendation on how to build better theory, and
highlights some of the significant characteristics of public policy itself. Greenberg, et. al, (1977)
describes the following challenges with public policy research: “(1) the policy process takes
place over time, involving several decision points and several predictive values that are likely to
change with the process; (2) the complexity and multiplicity of all aspects of the policy make
them difficult to categorize; (3) the large number of participants makes the process of policy
complicated (subjectively and objectively); and (4) the policy cannot be described by single
additive models”. Although the authors highlight the challenges with researching policy, there
are also recommendations provided for conducting a policy study that include: “(1) identify a
point of last significant controversy and a point of first significant controversy; (2) do not
attempt to predict characteristics of the policy process by variables whose value or score for a
given case is not known until too late in the process itself, or by predictive variables that are
characteristics of outputs” (Greenberg, et. al, 1977). In conclusion, the author suggests
“measuring success by comparing the output of the process with the intentions of the relevant
actors” (Greenberg, et. al, 1977).
The purpose for policy evaluation is “to acquire insight into the consequences that are
attributed to certain policies” (Bartels, et. al, 1982). These consequences can be intentional or
unintentional. Bartels, et. al (1982) describes the importance of policy evaluation and how the
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
43
consequences of the results can be interpreted, used in political debates, and the importance of
objective insights and the challenges with using certain methodologies for assessing policy
impact at the regional level. There are a wide range of considerations that should be examined to
conduct a comprehensive assessment of policy impact. This includes direct, primary effects;
indirect secondary effects; negative and positive effects; and the location and time dimension of
these effects due to the multiple stages of policy implementation (Bartels, et. al, 1982). It is also
important to select the appropriate research design to ensure internal and external validity of the
results and being aware of the limitation of each of the approaches (Bartels, et. al, 1982).
It is also important to ensure methods used to evaluate regional policy include a control
group to identify what would have happened in the absence of the policy. Isserman & Merrifield,
(1982) bring up the point that without a control group it is difficult to know if the effect in the
community was due to the policy or extraneous effects. They describe a shift-share analysis
techniques and provide guidance on how to select a control area and what characteristics to
consider (Isserman & Merrifield, 1982).
Reno (2007) re-examines Edward C. Banfield’s work The Unheavenly City and the
Horizon Theory and describes Banfield’s approach to evaluating social policy using culture as
his level of analysis. He further describes the horizon theory of human behavior as drawing from
economic, sociological, and psychological research and states that Banfield’s declaration “that
policy must be designed with an understanding of the relationship between the source of the
problem and human behavior” (Reno, 2007). Reno (2007) describes several errors that may lead
one to reach uncertain conclusions regarding public policy and recommends rethinking the
horizon theory, pay attention to the design of institutions, political leadership, and networks of
participation for individuals and organized interests and determine how each can produce policy
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
44
that adapts to normative expectations. In Banfield’s (1974) explanation of analyzing public
policy, he describes horizon theory “to be that of psychological orientation toward the future”.
Lastly, Reno (2007) recommends reconsidering the standards that guide policy evaluation.
Urban Impact Analysis
Urban Impact Analysis (UIA) were designed to evaluate federal policy in urban areas
during the Carter Administration. In 1978, it was proposed that all government policies be
evaluated ahead of time with respect to possible urban impacts and that the evaluation should be
conducted in alignment with the administration’s urban policy (Clark, 1981). However, there
were no evaluation studies conducted during the Carter Administration using UIA in the
literature.
Measuring Social Impact
A ‘social impact’ is defined as “a significant or lasting change in people’s lives brought
about by a given action or actions” (Barrow, 2002). A social impact assessment (SIA) is a
systematic evaluative process that uses descriptive and analytical tools to understand, plan, and
assess policy changes (Barrow, 2002). SIAs are often implemented along with other tools (i.e.,
environmental impact assessments) and program or policy evaluation to identify potential
impacts that can be avoided or mitigated (Barrow, 2002). It is recommended that SIAs be
conducted prior to the implementation of a program or policy (Barrow, 2002). However, they
can be implemented retrospectively to assess the experience and understand how change takes
place (Barrow, 2002). As the proportion of the world’s population living in urban areas grow,
SIAs can be used to assess the publics’ reactions, adaptability for various social, ethnic and
gender groups where social development is occurring in diverse areas (Barrow, 2002).
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
45
SIA are used around the world “to understand the way human communities change as a
result of either an intended or unintended action; to expand the knowledge base on a particular
community or project level assessments when there is no baseline data; as ‘social follow-ups’ to
existing projects or policies; to collect quantitative data to identify social impacts and share
information to build community awareness about a particular project or policy; and to evaluate
policies and programs” (Burdge, 2003). A major benefit of a SIA is engaging the affected
populations in the SIA and allowing them to understand the proposed action or policy (Burdge,
2003).
The World Economic Forum (2020) provided a vision for 2020 on how social impact
should be utilized to drive systemic change. They suggested strengthening ecosystems that
support the society via partnership and collaboration to jointly run interventions (World
Economic Forum, 2020). They also recommended emphasis on the quality of social impact
measurements and a call for alignment of social impact measures used to get an understanding of
what is and is not working to address complex challenges. Lastly, they suggest public and private
sectors use the United Nation’s 17 Sustainable Development Goals (SDGs) to measure the exact
goals that their strategies align with (World Economic Forum, 2020).
The Principles and Guidelines for Social Impact Assessment in the United States are
organized by six principles: “1) understanding of local and regional settings; 2) dealing with the
key elements of the human environment; 3) using appropriate methods and assumptions; 4)
providing quality information for decision making; 5) ensuring that environmental justice issues
are addressed; and 6) establishing mechanisms for evaluation/monitoring and mitigation”
(Downs, et. al, 2003). There are also three proposed SIA models based on the study type:
comparison, impact, or control (Downs et. al., 2003). It is important to assess social impacts at
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
46
all stages of a policy to ensure all community concerns are assessed (Downs, et. al, 2003).
Downs et. al (2003) provides a matrix of variables, a process for selecting the right variables
based off the stage of the policy, and a process for identifying the data requirements needed for a
preliminary assessment. The authors concluded that the U.S. has not done a good job of
integrating social impact assessments into the federal agency decision making process (Downs,
et. al, 2003).
It is important to maintain the technical methodology of a SIA when integrating it with
participatory and political processes. When conducting a SIA, social impact cannot be
considered only by the analyst, but should be considered by the range of groups that will be
impacted by the outcome of the policy (Lane et al, 1997). This requires the data to be viewed by
diverse groups with different values and perspectives. The Blishen-Lock Hart Model includes
three broad indicators (social validity, economic validity, and political efficacy) used in cross-
cultural context to understand a community’s structure and process and can aid in the
identification of the value system of the community and identify which indicators should be
prioritized to measure social impact based on what the stakeholders in the community value
(Lane et al, 1997).
Collective Impact Evaluation
The five key conditions of Collective Impact Framework include a common agenda,
shared measurement systems, mutually reinforcing activities, continuous communication, and the
presence of a backbone organization. There are also three conditions that must be in place before
launching a collective impact initiative: (1) an influential champion, (2) adequate financial
resources, and (3) a sense of urgency for change (HanleyBrown, Kania & Krammer, 2012). The
Collective Impact Framework also includes a shared measurement system that can be used to
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
47
increase efficiency, reduce cost, improve the quality and credibility of data collection, increase
effectiveness and track progress towards the common agenda (Kania & Kramer, 2011). By using
a shared measurement system with the same indicators across all public-private partners and
workstreams ensures that all efforts remain aligned, and partners can be held accountable for
collective impact (Kania & Kramer, 2011).
Preskill (2014) developed a guide that provides a more detailed description of
performance measurement and evaluation activities within the Collective Impact Framework that
can be implemented to assess an initiative’s progress, effectiveness, and impact. The guide
provides sample evaluation questions, measures/outcomes that can be used to assess
implementation of the collective impact initiative at various stages of implementation (early
years, middle years, and later years). It also provides a framework for performance measurement
and evaluation during the process of implementing the collective impact initiative over time that
fosters continuous learning. Preskill (2014) suggests using data and insight from a variety of data
collection activities to inform decision making and clearly distinguishes the use of performance
measurement versus the use of evaluation.
Evaluation is used for informing learning, decision making, and actions needed (Preskill,
2014). Performance measurement is used to monitor accountability and progress toward the
desired outcomes of the initiative (Preskill, 2014). Outcomes are changes that occur because of
activities and outputs (U.S EPA, 2007). Short-term outcomes are the result of changes in
learning, knowledge, attitude, skills, or understanding (U.S. EPA, 2007). Changes in behavior,
practice, or decisions are the result of intermediate outcomes and changes in condition occur
because of long-term outcomes (U.S. EPA, 2007). Indicators show whether a given condition
exists or whether certain outcomes have been reached (Horsch, 1997). Decision-makers can use
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
48
indicators to track progress toward achieving desired outputs, outcomes, goals, and objectives
(Horsch, 1997).
Summary
A literature review was conducted to understand the historical context and identify key
components of urban policies. This literature review provided the information necessary to
understand the themes across urban policy enacted from 1937 (1937 Housing Act) to 2019
(Opportunity Zones). In summary, it has been difficult to evaluate this series of urban policies,
due to the multi-factorial nature of urban activities, the diversity of program setup, and the lack
of quality data to support such an evaluation (Zhang, 2019). Therefore, it is important to develop
an evaluation framework designed to measure the impact of urban policies and programs that
have a desire to achieve large-scale, sustainable change (Preskill, Parkhurst, & Juster, 2014).
Hence, a collective impact evaluation framework was selected to develop a framework for
evaluating urban policy and its impact on the SDoH because it supports the complex
implementation of urban policy and allows for a mixed methods research design to assess the
implementation, effectiveness, and impact of the policy on targeted communities.
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
49
Chapter 3: Methodology, Data Analysis, and Outcomes
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
50
Study Purpose
The purpose of my research is to develop an evaluation framework and core set of SDoH
measures to assess the impact of federal urban policy at the local level on achieving the desire
outcomes of reducing poverty and the impact on social determinants of health (SDoH) in urban
communities designated as “zones.” A qualitative analysis of manuscripts, grey literature, white
papers, and government reports was conducted to identify the key components of the logic
model: inputs (Acts, policies, programs, funding allocated, etc.); activities (mutually reinforcing
strategies); outputs (capacity); desired outcomes, impact, and evaluation recommendations from
past urban policies. SDoH measures were also identified to develop an evaluation framework
that can be used to assess if social conditions improve for the people who live in communities
that have been targeted by urban policies over time. A literature review and environmental scan
were conducted to provide context on the positive or negative impacts urban policy on the SDoH
and provide insight on what key evaluation questions, shared measures/outcomes should be used
to evaluate the impact of urban policy on SDoH.
Analytic Framework
Preskill (2014) defines evaluation as “the systematic collection of information about the
activities, effects, influence, and impacts of programs or initiatives to inform learning, decision
making, and action”. The first step in developing an evaluation is to engage stakeholders
(Milstein & Wetterhall, 1999). For the purpose of developing this evaluation framework,
stakeholders are people involved in implementing the program or conducting the evaluation, who
will use the results of the evaluation, and/or agencies who fund or authorize the continuation or
expansion of the program (Milstein & Wetterhall, 1999). The second step is to describe the
program which is often illustrated using a logic model (Milstein & Wetterhall, 1999). A logic
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
51
model illustrates the if-then sequence of events, or a logical flow of how a program theoretically
leads to outcomes and/or impact (Savaya, et. al. 2005). A logic model consists of inputs,
activities, outputs, outcomes, and impact. Savaya, et. al. (2005) defines inputs as “the human,
financial, organizational, and community resources that need to be invested in a program so that
it will be able to perform its planned activities”. Activities are defined as “what the program does
with the inputs (i.e., the processes, events, and actions that are an intentional part of the program
or policy implementation.” Outcomes are changes that occur because of activities and outputs
(U.S EPA, 2007). Outputs are the direct products of program activities, measured in terms of the
volume of work accomplished and the number of people reached (Savaya, et. al. 2005). Impacts
can be defined as long-term changes in indicators (CDC, 2013).
The Collective Impact Evaluation Framework will be used as the foundation for
developing a SDoH Urban Policy Evaluation Framework (Figure 3.1). A Collective Impact
Evaluation Framework is used when the desire is to achieve large-scale, sustainable change
(Preskill, Parkhurst, & Juster, 2014). It supports the complex implementation of urban policy and
allows for a mixed methods research design to assess the implementation, effectiveness, and
impact of the policy. A mixed method design includes multiple methods, different assumptions
and allows for diverse forms of data collection and analysis. The Collective Impact Evaluation
Framework consist of three key components of the Collective Impact change process: socio-
political context, implementation, and outcomes. This framework enables an assessment of the
policy outcomes over the lifetime of the policy.
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
52
Figure 3.1 A Framework of Performance Measurement and Evaluation of Collective
Impact Efforts
Source: Preskill, et al. (2014, p.12). Guide to evaluating collective impact: Assessing
progress and impact. Aspen Institute Forum for Community Solutions.
A mixed method design allows for a logical approach to the research design. The
development of the Urban Policy SDoH Collective Impact Evaluation Framework was adapted
using the Preskill (2014) Collective Impact Evaluation Framework and will align social-political-
economic context with urban policy inputs and activities and SDoH short-term, intermediate, and
long-term outcome and impacts illustrated in Figure 3.2. Outcomes included in this evaluation
framework will measure short-term and intermediate changes in behavior, awareness, and or
knowledge in targeted communities’ residents. Impacts evaluated will include long-term changes
in key SDoH indicators in targeted communities.
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
53
Figure 3.2. Urban Policy SDoH Collective Impact Logic Model
Source: Adapted from Preskill, et al. (2014). Guide to evaluating collective impact:
Assessing progress and impact. Aspen Institute Forum for Community Solutions.
Urban Policy and SDoH Collective Impact Evaluation Framework and Shared
Measurements Development Approach
A three-step approach was implemented to develop the Urban Policy and SDoH
Evaluation Framework and Performance Measures. A qualitative analysis of guidance
documents, evaluation reports, administrative documents, and government reports was conducted
to identify key inputs (Acts, policies, programs, funding allocated, etc.); activities (mutually
reinforcing strategies); outputs (capacity); desired outcomes, impact, and evaluation
recommendations to support urban policy implementation and evaluation, and gaps in the
evaluation of past urban policies (Step 1: Literature Review of Urban Policies). SDoH
measures/outcomes identified in the qualitative analysis were used to develop an evaluation
framework that can be used to assess if SDoH indicators improved for target population in
INPUTS
• Common
Agenda
• Backbone
Infrastructure
ACTIVITIES
• Mutually
Reinforcing
Activities
OUTPUTS
• CI Capacity
SHORT-TERM/
INTERMEDIATE
OUTCOMES
• CI Systems: Policy
• CI Systems: Funding
LONG-
TERM
OUTCOMES
• CI Behavior
Change
COLLECTIVE
IMPACT
CONTINUOUS COMMUNICATION:
Backbone Organization, Federal, State, Local, Zone, CBOs, and Private Entities
Social-Political- Economic
Context
Collective Impact Design and
Implementation
Outcomes
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
54
communities that have been targeted by urban policies over time (Step 2: Data Abstraction from
Literature to Identify SDoH Measures for Urban Policies and Programs). Quantitative data
indicators from multiple data sources were identified in an environmental scan to assess if they
meet the criteria to evaluate the overall impact of urban policy on the SDoH (Step 3:
Environmental Scan of Existing Data Sources and Databases). Variables used will not be pulled
for statistical analysis, only identified using a set of criteria to determine if they will be included
in the framework. SDoH indicators and data sources that met this set of criteria were included in
the Urban Policy SDoH Collective Impact Evaluation Framework.
Criteria was established based on recommendations and limitation in access to data
identified from the literature in Step 2: Data Abstraction to Identify SDoH Measures. For
example, previous Enterprise Zone studies recommended use of spatial units of zip codes for
economic data and census tracts for demographic data. (Zhang, 2019). Zhang (2019) also
recommended “decomposing economic indicators by industry type, size, and/or business status.
Freedman, et. al (2021) described the ability to access American Community Survey (ACS) data
at a regional data center to conduct analysis and data availability every 5-years as a limitation. In
addition, the GAO (2020) recommended the following questions be used to evaluate the
Opportunity Zone’s tax incentive’s performance: “(1) How have Opportunity Zones’
characteristics changed, for example with regard to poverty, income, unemployment, education
levels, race, affordable housing, and displacement?; and (2) What are the characteristics of
businesses in Opportunity Zones, such as location, business type, number of employees,
finances, and residential units (if applicable)?”. Therefore, inclusion criteria included national
data, open-source data, public availability, annual data collection, data available at the census
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
55
track or county level, data that can be stratified by race/ethnicity, age, gender, education, or
income level, and by business or industry type, size, and/or business status when applicable.
Step 1: Literature Review of Urban Policies
To identify the key components of urban policy logic model from manuscripts, grey
literature, white papers, and government reports on urban policies enacted from 1937 (1937
Housing Act) to 2019 (Opportunity Zones). The literature review was conducted via Google
Scholar, GAO Reports, Government Program websites. The Google Scholar search was used to
build a preliminary list. A snowball sampling technique was used to analyze the references of
manuscripts that met the search criteria and further papers were added to the initial list (Zhang,
2019). This process was completed until all key components for each policy were abstracted
(Zhang, 2019). To be eligible for inclusion in this review manuscripts, grey literature, white
papers, and government reports had to (1) include information on at least one of the key
components of the logic model (input, activities, outputs, outcomes, and impact) and/or
recommendations; (2) be an urban policy or program enacted by Congress, (3) include place-
based criteria for eligibility, (4) and be enacted between 1937-2019. Non-publicly available
manuscripts or books, literature that did not contain any of the key component (inputs, activities,
outputs, outcome, or impact) and urban policies/programs that were not enacted by Congress
were excluded from the search. Search criteria included the following search terms:
• Housing Act of 1937
• Housing Act of 1949
• Housing Act of 1954
• America’s Communities: A National Urban Policy
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
56
• Enterprise Zones
• Grey Areas Project
• Economic Opportunity Act of 1964: The Model Cities Program
• Community Development Block Grant (CDBG) Program
• A New Partnership to Conserve
• Urban Jobs and Enterprise Act 1984: State Enterprise Zones
• Empowerment Zones/Enterprise Communities
• Promise Zones
• Opportunity Zones
Step 2: Data Abstraction of Literature Review to Identify SDoH Measures
To identify and align common urban policy strategies with key evaluation questions,
shared measures/outcomes, and data sources from past urban policies and programs, information
was abstracted from the literature review for urban policies listed in Step 1. An abstraction table
was created with the following components: Evaluation Questions, Shared Measures/Outcomes,
Challenges with Measures, Methods, Data Sources to ensure the development of the framework
used evidence from the literature to develop evaluation questions, identify validated measures
and outcomes from multiple studies, to identify challenges with specific measures and methods,
and to identify existing data sources that have been validated in other studies. Data was
abstracted from urban policy manuscripts, grey literature, white papers, and government reports
on urban policies enacted from 1984 (State Enterprise Zones) to 2019 (Opportunity Zones). This
review only included Enterprise Zones, Empowerment Zones, Promise Zones, and Opportunity
Zones studies because they were place-based policies that provided criteria for specific census
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
57
tracks and/or community data as a part of the selection criteria. Policies that did not have any
literature with outcome measures or evaluation questions were also excluded
Step 3: Environmental Scan of Existing Databases and Data Sources to Identify Key SDoH
Measures
To assess existing data sources and to identify readily available public data with no
personal identifiable information (PII) on SDoH indicators an environmental scan was conducted
via Google, literature identified in Steps 1 and 2, and government program websites. Search
terms included SDoH indicators, SDOH database, SDoH measures, social impact indicators,
social impact databases, social impact measures. Data sources and databases that required
payment were not included or that included PII due to access to the data limitation identified in
previous studies (Freedman, Khanna, & Neumark, 2021). Data sources and database criteria
included:
• National data
• Open-source data
• Public availability
• Annual data collection
• Data available at the census track or county level
• Data that can be stratified by race/ethnicity, age, gender, education, or income level, and
by business or industry type, size, and/or business status when applicable
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
58
Summary and Limitations
In summary the methodological steps described include using urban policy historical
context and urban policy literature to develop a conceptual framework to describe the key
components of an urban policy and identify urban policy SDOH shared measures/outcomes,
searching the literature, abstracting the data, and analyzing the results to develop an evidence-
based Urban Policy SDoH Collective Impact Evaluation Framework (Sipe, et. al., 2012). In
addition, applicability of results, barriers to data collection, and evaluation gaps are described.
One study limitation was limited access to hardbound books or government reports located in
libraries due to COVID 19, specifically for urban policies prior to 1980s that were not available
via Google Books or accessible online.
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
59
Chapter 4. Data Reporting and Analysis
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
60
Chapter 4 provides a summary of findings from the methods steps described in Chapter 3.
This includes the first section of this chapter summarizing findings from Step 1: Literature
Review of Urban Policies using logic models for each urban policy enacted from 1937-2019.
The second section provides an overview of results from Step 2: Literature Review to Identify
SDoH Shared Measures for Urban Policies and Programs. This includes a summary of common
urban policy strategies with key evaluation questions, measures/outcomes, and data sources from
past urban policies and programs. The third section summarizes Step 3: Environmental Scan of
Existing Databases to Identify Key SDoH Measures by bridging urban policy, evaluation, the
social determinants of health to identify key measurement indicators, assessments, and data
sources that can be used to evaluate urban policy.
Step 1: Literature Review of Urban Policies
Appendix A. illustrates an Urban Policy SDoH Logic model based on information
abstracted from the literature review in Step1. Information abstracted from multiple data sources
was used to develop a comprehensive logic model of federal urban policies passed from 1937-
2019. Core components of the logic model include inputs, activities, outputs, outcomes, and
impact of the urban policy on the social determinants of health (SDoH).
Step 2: Literature Review to Identify SDoH Shared Measures for Urban Policies and
Programs
To identify and align common urban policy strategies with key evaluation questions,
measures/outcomes, and data sources from past urban policies and programs, information was
abstracted from the literature review for urban policies listed in Step 1. In addition, challenges
with measures, methods identified in the literature on Enterprise Zones, Empowerment Zones,
Promise Zones, Opportunity Zones, and any recommendations for future studies were also
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
61
abstracted and used to provide guidance in development of the Urban Policy SDoH Collective
Impact Evaluation Framework. Tables 4.1-4.4 illustrate the evaluation questions, shared
measures/outcomes by Enterprise Zones, Empowerment Zones/Enterprise Communities,
Renewal Communities, Promise Zones, and Opportunity Zones policy.
Enterprise Zones
Two studies looked at number and the value of properties on building permits issued in
state EZs in Illinois after zone EZ designation and found no significant difference in building
permit activity within EZs, but rather found more activity in non-EZs (Jones, 1985; Jones, 1987).
This data point was not a good indicator to measure forms of new investment or employment
because there was a drop in interest rates during this time, therefore development and investment
are likely to have increased at the same rate. Jones (1987) recommended that future studies
should include control cities for comparative analysis purposes. HUD also conducted a study of
nine state EZs and found that 263 zone firms were responsible for over $147 million in new
investment, and 7,348 new or retained jobs (Wilder & Rubin, 1996). However, there was a
limitation in the HUD study. This limitation included reliability of data because the data were
collected from interviews with zone officials who were more likely to attribute any growth
within the zones to EZ designation. Although there was an overall increase in new investment
and employment across nine EZs, some zones showed net loss (Wilder & Rubin, 1996).
In another study conducted in California which included ten state-EZs, total employment
increased across the 10 EZs, however, no causal link was found between employment changes
and zone program (Dowall, D., et al., 1994). An additional study in Indiana conducted on 12 EZs
was able to assess if direct and indirect costs or burdens were attributable to enterprise zones.
The Indiana EZ study found the total direct program costs were $20.6 million, total direct
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
62
benefits were $199 million in investment, and there was $272 million in new state/local tax
revenues (Rubin, et al., 1992). From 1989-1990, the were also 2,024 new jobs created by firms in
these twelve zones (Rubin, et al., 1992).
State EZ success was strongly associated with the degree of program management, array
of services or incentives, and differences in state and local economic conditions (Sheldon &
Elling, 1989). Several weaknesses were also identified in the legislation of State EZs, including
the failure to include planning process requirements in state legislation (Wilder & Rubin, 1996).
The State EZ legislation did not require an action plan that linked to specific desired outcomes
for the designated communities (Wilder & Rubin, 1996).
Although there were several studies conducted on the impact of State EZs, there were no
consistent indicators used across multiple State EZs or even within one state which limited the
ability to make causative conclusions about the impact of state-EZs because implementation of
the policy and outcome varied drastically by zone (Wilder & Rubin, 1996; HUD, 1986; Jones,
1985; Jones, 1985; Dowall, D., et al., 1994). Study design, population, timeline, indicators,
results, and outcomes also varied for each zone even when answering the same questions.
However, one consistent finding was that zone program and tax incentives were less influential
in determining whether to start or expand a business in a EZ (Litster, 1990; Papke, 1988; Papke,
1989; Staley, 1988).
A review of Enterprise Zone literature found via descriptive and correlation analyses
“that outcome measure, spatial unit, post-program time span, and the choice of an endogeneity
control method have significant impacts on program effectiveness studies” (Zhang, 2019). Out of
127 Enterprise Zone studies, eighty-one studies used non-economic indicators (demographics
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
63
and housing) and fifty-one of those studies showed no impact from the Enterprise Program on
non-economic indicators (Zhang, 2019).
There were several challenges with data collection and analysis identified in EZ studies
(Table 4.1). Challenges included: verifying actual program effects and programs impact on
reducing unemployment; variation among state EZ programs strategies selected, goals,
objectives, types of incentives, regulatory relief; and role of EZ associations (Jones, 1985 and
Jones, 1987). Wilder and Rubin, (1996) describes the methods used by Jones (1985) to look at
property transfers and building permits-used as a limitation in analyses of zone effects, because
they did not include such forms of new investment or employment as capital equipment
purchases or added work-shifts.
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
64
Table 4.1 Enterprise Zones (EZ) Evaluation Questions, Measures/Outcomes
Reference Enterprise Zones (EZs)
Evaluation Questions
EZ Measures /Outcomes
GAO,
1982
- Will the zones attract substantial
private investments, create jobs,
and/or result in adverse effects?
- Have enterprise zones succeeded
in stimulating business
development and at what cost? If
successful, to what degree and
what types of businesses have
enterprise zones attracted? If
certain types of businesses have
not responded or overall
business response has been
limited, what additional action is
needed and by whom?
- Have enterprise zones succeeded
in creating jobs, particularly for
the disadvantaged? If so, to what
degree? What types of jobs have
been created and for what types
of workers? Is additional job
creation action needed and if so,
by whom?
- What revenue losses have
resulted from employment
incentives?
- What effect have enterprise
zones had on distressed areas?
Have there been unintended
effects, such as:
o (1) unfair competition for
businesses not in a zone,
o (2) business relocations
rather than new, net
economic activity, and
o (3) residential
displacement? If
problems are encountered
in these areas, what
actions are needed to
mitigate their effects?
- What has the program cost in
lost Federal revenues? Have
Economic and Small Business
Development:
- stimulate business development
- create jobs
Funding/Tax Incentives:
- quality and strength of state and local
incentive contributions
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
65
Reference Enterprise Zones (EZs)
Evaluation Questions
EZ Measures /Outcomes
these costs been offset by
benefits such as newly generated
revenues or reduced Government
expenditures and, if so, to what
extent?
Jones,
1985;
Jones,
1987
- What are EZ program effects
and programs impact on
reducing unemployment?
- Whether development would
have occurred in these areas
without the policy or programs?
- CT: Determine if there was a
significant difference in
revitalization activity after zone
designation?
- IL: Determine if an increase in
development activity occurred
after zone designation, and if an
increase occurred can it be
attributed to the program.
-
Economic and Small Business
Development:
- stimulate economic development and
revitalization in distressed communities
- reduce Black unemployment
- growth and retention of Black
businesses
- number of jobs in place/progress
- number of new jobs created or retained
- number of new businesses started
operations
- number of firms expanded
- number of firms were retained
- number of new employees hired filed
on tax returns
Investments:
- number and the value of building
permits issued in state EZs after zone
EZ designation
- dollar amount investment in new
construction/expansion/renovation
- dollar amount in capital investments
retained in EZs
- number and the value of building
permits issued in state EZs
GAO,
1988
- Assessed federal agencies’
assumptions about the
performance of a federal
enterprise zone program,
particularly as they relate to
program cost offsets,
- Assessed the relative
effectiveness of tax incentives
and other local development
strategies
Economic and Small Business
Development:
- employment growth
- reduction in welfare dependance
- measure employment growth in state-
designated enterprise zones
Funding/Tax Incentives:
- program cost offsets
- measure offsets to the cost of an
enterprise zone program resulting from
program effects on workers
Wilder &
Rubin,
1988;
- Can observed changes in
economic activity be directly
attributed to enterprise zones?
Economic and Small Business
Development:
- number of new or retained jobs
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
66
Reference Enterprise Zones (EZs)
Evaluation Questions
EZ Measures /Outcomes
Wilder &
Rubin,
1996
- Whether enterprise zones are
more or less cost effective than
alternative initiatives for
community revitalization?
- number of new businesses created in
the EZ
- number of existing businesses
expanded
- number of new full-time jobs created
- percent of new jobs to zone residents
Investments:
- number of and dollar amount in new
investment
Rubin, et
al., 1992
- What factors explain the
variations in zone effects?
Funding/Tax Incentives:
- dollar amount in new state/local tax
revenues
Economic Development:
- number of new jobs created by firms in
zones
Dowall,
D., et al.,
1994)
- Identify the underlying causes of
the employment and business
growth
- Assess if direct and indirect
costs or burdens are attributable
to enterprise zones?
Economic and Small Business
Development:
- total employment increased across the
EZs
Funding/Tax Incentives:
- dollar amount of total direct program
costs
- dollar amount of total direct benefits
Zhang,
2019
- Whether Enterprise Zone
programs can stimulate growth
and improve the economic
environment in poor
neighborhoods?
Economic and Small Business
Development (Aggregated):
- total employment
- total residence employment
- workers income
- total investment or inventory
- number of establishments or number of
firms
- household or personal family income
- unemployment reduction/disparity
- poverty reduction
income disparity
Economic and Small Business
Development (Disaggregated):
- indicators by industry
- industry by size of firms
- industry by firms’ business cycle
Demographics of Zone:
- total population
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
67
Zhang (2019) makes several evaluation recommendations including: decomposing
economic indicators by industry type; size and or business status. Previous EZ studies use spatial
units of zip codes for economic data and census tracts for demographic data due to data
availability; use a difference-in-difference approach (DID) to assess the changes of growth
before or after EZ program; and select control groups based on direct comparison (i.e., socio-
demographic characteristics). Zhang (2019) concluded “a good research design should select an
outcome measure based on the analysis of the program design, proper geographic scale, and
allow time for the program to show its impacts.” Lastly, Barnes (1990) recommended the need
for analytic and political frameworks that can be applied to assess the impact of urban policy and
that incorporate federalism, capitalism, and public and private factors that can create sustainable
urban solutions.
Empowerment Zones/Enterprise Communities/Renewal Communities (EZ/EC/RC)
Table 4.2 highlights several EZ/EC/RC studies that have mixed results on the impact of
these zones on economic and small business development outcomes. Although one of the largest
EZ/EC/RC studies reported that EZ/EC/RCs had large effects on job creation, with increases in
local payrolls about 10% (Herbert, 2001), Busso and Kline (2008) identified weaknesses in the
Herbert (2001) study methodology. The methodology limitations included reliability upon within
city comparisons of census tracts, limitations in the matching algorithm used to match control
census tracks, and concerns with data quality (Busso & Kline, 2008).
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
68
Table 4.2: Empowerment Zones/Enterprise Communities/Renewal Communities
(EZ/EC/RC) SDoH Evaluation Questions, and Measures/Outcomes
Reference EZ/EC/RC Evaluation Questions EZ/EC/RC Measures /Outcomes
Hetzel, 1994 - Whether the incentives
provided will lead to successful
economic development?
Economic and Small Business
Development:
- numbers of residents obtaining jobs
- attraction of business to the area
- extent to which the poor will be
empowered to become economically
self-sufficient as a result of economic
development activities in EZEC
- increases in employment of residents
in zones
- increases in income of residents in
zones
Housing and Safe Neighborhoods:
- better housing and other
infrastructure changes
- improvements in housing and
commercial facilities in
neighborhoods
Investments:
- infrastructure investment
- private sector
participation/investment
Funding/Tax Incentives:
- priority in other funding decisions
- waivers for flexibility with federal
funding
- state and local commitments of
resources to zones
- public/private partnerships in zones
- capital formation incentives
- write-offs for equipment purchases
- state abatements and tax forgiveness
- tax increment financing (i.e., section
108 loan guarantees)
- the extent that funds were siphoned
from other areas of funding
- funds leveraged
Nelson A.
Rockefeller
Institute of
Government,
1997
- “Provides a measure of the
relative access to capital for
community development
financing in EZ/EC sites and
Housing and Safe Neighborhoods
(residential lending):
- total loan activity in each EZ/EC and
corresponding MSA (the relative
share of all loans either originated,
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
69
Reference EZ/EC/RC Evaluation Questions EZ/EC/RC Measures /Outcomes
their respective metropolitan
areas.
- Explores the degree to which
community development
financing was raised as an issue
during the strategic planning
process, and the context within
which the topic was raised.
- Explicates the themes
underlying community
development financing
activities designed by the
framers of EZ/EC strategies in
these sites.
- Outlines the content of
community development
financing activities and their
evolution.
- Examines the degree of
progress that has been made in
implementation through July
1997.
- Assesses barriers that have
arisen and how they have been
met.
- Describes the nature and extent
of community participation in
the design, review/approval,
and operation of community
development financing
initiatives.
- Characterizes how the
community development
financing activities created
through the EZ/EC Initiative
differ from other such
mechanisms.
- Offers concluding thoughts on
common challenges and best
practices among these
programs.”
meaning approved and made
unaccepted, meaning approved by the
institution but declined by the
applicant, or denied, referring to loan
applications turned down by a
financial institution).
- rate of loan originations and denials
per 1,000 dwellings in EZ/EC sites
and their corresponding MSAs
- the value of originated and denied
loans in the EZIEC sites
- the proportion of loans originated by
low-income, working-class, and
moderate-income individuals
Economic and Small Business
Development:
Commercial Lending:
- average lending amounts for each
EZ/EC commercial site
- quantity of SBA-related lending per
qualified firm
Federal Funding:
- number and amount of EDI
- number and amount of BEDI
- number of and amount of CDBGs
- number and amount of EDA Public
Works and Economic Development
Grants
- number and amount of Historic
Rehabilitation Tax Credits
- number and amount of HOME
Investment Partnerships (HOME)
grants
- number and amount of Low-Income
Housing Tax Credits
- number and amount of New Markets
Tax Credit
State Funding:
- number and amount of Housing Trust
Funds
- number and amount of Tax Increment
Financing (TIF)
- number and amount of State Tax
Credit Programs
Private Fund:
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
70
Reference EZ/EC/RC Evaluation Questions EZ/EC/RC Measures /Outcomes
- developer fees
- private loans
- grants
- land donations
- individual financial contributions to
Section 108 and other federal housing
and community and economic
development projects
Herbert,
2001
- To what extent does the EZ/EC
program create jobs and
economic opportunity within
the zone?
- How have the economic and
social well-being of zone
residents been affected?
- Has the zone become a more
attractive place to live, work
and do business?
- To what extent does the zone
program connect zone residents
to opportunities in the
metropolitan area as a whole?
- Has the EZ/EC program
fundamentally changed the way
local, state, and Federal
agencies “do business”?
- Have residents of the zone been
empowered to take greater
control over their own lives and
the future of their community?
- At what costs are these
outcomes being achieved?
Economic and Small Business
Development:
- poverty rate: the percentage of the
area’s Census tracts with a poverty
rate of less than 20 percent, 20 to 25
percent, 25 to 35 percent, and 35
percent or higher
- adult employment rate: the
percentage of the adult population
that is employed
- racial/ethnic composition: the
percentage of the area’s population
that is white and non-Hispanic
- income: median household income in
the area
- producer-oriented employment: the
percentage of the area’s employment
that is producer-related, versus
consumer-related or public sector
related
Busso &
Kline (2008)
- Whether the jobs being created
in EZs were staffed by local
residents?
- Whether the neighborhood
composition of EZ residents
changed?
- Whether poverty,
unemployment, or the local
housing market responded to
the treatment?
Economic and Small Business
Development:
- decrease in the unemployment
- decrease in poverty headcount
- increase in employment
- wage value of the number of jobs
created
Housing and Safe Neighborhoods:
- average change in annual track wide
rent
- total increase in annual rents inside
EZ
- present value
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
71
Reference EZ/EC/RC Evaluation Questions EZ/EC/RC Measures /Outcomes
- average change in track-wide owner-
occupied housing value
- total increase in owner-occupied
housing value inside EZ
- total increase of value of EZ housing
units
Busso,
Gregory, &
Kline (2010)
- Assess the incidence and
efficiency of Round I of the
federal urban Empowerment
Zone (EZ) program
Economic and Small Business
Development:
- increase employment for residents of
empowerment zones
- wage increases for residents of
empowerment zones
Demographic:
- neighborhood racial makeup,
employment, educational attainment,
poverty, and home values
Oakley &
Tsao, 2006
- Can socioeconomic progress
within and around the zone
area be attributed to this single
policy initiative?
- When multiple public policies
and/or private programs are
occurring in the same zone at
the same time, is it possible to
isolate the effect of one
initiative?
- How effective zone programs
were at improving the
socioeconomic status of the
zone itself as well as that of the
neighboring areas?
- How effective Chicago’s
Empowerment zone programs
were at alleviating poverty,
decreasing unemployment, and
increasing income, as well as
improvement across a number
of other socioeconomic
indicators within the zone
itself?
- Whether socioeconomic gains
observed within the zone
spilled over to adjacent areas as
was intended in Chicago’s
original zone plan?
Economic and Small Business
Development:
- level of poverty (HUD definition)
- percent unemployment
Demographics of Zone:
- population size
- percent non-Hispanic white
- percent non-Hispanic black
- percent Hispanic
Housing and Safe Neighborhoods:
- percent homeownership
- percent vacancy
- percent living in the same household
for the last five years
- household income (adjusted for
inflation)
- residential stability - percent living in
the same household for the last five
years
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
72
GAO (2007, 2010, 2012) conducted several studies to evaluate EZ/EC/RC. One study
concluded that due to data and methodological limitations, previous studies have not produced
definitive results about the effectiveness of the New Markets Tax Credit, Empowerment Zone
tax incentives, historic rehabilitation tax credits, and tax aid for certain disaster areas. An
additional limitation was the inability for the IRS to separate the total credits claimed to show
how much was claimed for specific EZ communities (GAO, 2010). Although HUD collects
community level information for some EZ-related tax expenditures, they are unable to validate
the information on the use of some of these tax expenditures and they only track a portion of the
EZ employment credits (GAO, 2010).
Reference EZ/EC/RC Evaluation Questions EZ/EC/RC Measures /Outcomes
GAO, 2007 - Describe how federal funds
may have been or could be
leveraged in the Section 108
program?
Funding: Leveraged funds with Section
108 funds by type
- other federal funding sources
- state or local funding sources
- private funding sources
GAO, 2010
GAO, 2012
- Identify community
development tax expenditures
and potential overlap and
interactions among them
- Assess the data and
performance measures
available and used to assess
their performance
- Determined what previous
studies have found about
selected tax expenditures’
performance
Tax Incentives:
- utilization of the New Markets Tax
Credit
- utilization Empowerment Zone tax
incentives
- utilization historic rehabilitation tax
credits
- utilization of facility bonds in EZs
- utilization of the CRD in RCs
- utilization EZ/RC employment credit
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
73
Promise Zones
Table 4.3 highlights evaluation questions and measures identified in a review of Promise
Zone literature. All communities that received a Promise Zone designation were required to
collect data and information related to the progress of their efforts and HUD developed a Data
Collection and Evaluation Framework for Urban Promise Zones to support these requirements
(HUD, 2018). Each Promise Zone was required to evaluate their programs using the HUD
developed evaluation framework. The HUD Evaluation Framework includes detailed guidance
on evaluating the Promise Zone initiative including how to develop a theory of change and logic
model; how to develop an evaluation strategy; how to develop data collection tools/surveys; how
to analyze data; how to create data visualization and dissemination products (HUD, 2018). The
framework also included a matrix for key measures and potential indicators by Promise Zone
goals, outcomes, data sources, availability by geographic level, accessibility, and additional
information specific to each measure (HUD, 2018). The framework included key quantitative
metrics required to be reported across eight domains: 1). employment and asset building; 2).
investment and business growth; 3). education; 4). public safety; 5). housing; 6). health; 7).
community infrastructure; and 8). civic engagement (HUD, 2018). There was also an additional
set of recommended indicators linked to each policy domain, based on common goals across
sites, and categorized into four groups (HUD, 2018). Although there was a matrix of data to be
collected for evaluation, the partner organizations described difficulty in trying to figure out what
indicators should be used to evaluate Promise Zone work and absence of guidance on how to
define or collect measures. A limitation of the HUD evaluation framework was that not all
Promise Zones were required to report on the same measures, which did not allow for data to be
aggregated across multiple PZs.
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
74
Although there were no standardized required measures to report across Promise Zones,
each program was able to report on individual success of their initiatives. The IndyEast Promise
Zone secured almost $10 million in federal grants across seven different agencies (HUD, 2020).
This support has helped create 110 jobs, provided life-changing workforce training to hundreds
of formerly incarcerated individuals, and developed a new energy efficient apartment building
for low-income senior citizens (HUD, 2020). The Southeastern KY Promise Zone developed a
new educational framework and with support from AmeriCorps VISTA members helped lift the
graduation rate at Leslie County High School from 67% to 98% (HUD, 2020). The Choctaw
Nation secured $21 million in New Markets Tax Credits to build an environmentally sustainable
steel manufacturing facility and the mill was able to support three hundred new jobs in the region
(HUD, 2020). The Los Angeles Promise Zone was awarded 67 Federal Grants for over $310
million in awards from 14 Federal Agencies, enabling them to fund twenty-five different LA
Promise Zone Partners (HUD, 2020).
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
75
Table 4.3 Promise Zones (PZ) SDoH Evaluation Questions, and Measures/Outcomes
Reference PZ Evaluation Questions PZ Measures/Outcomes
HUD,
2020
White
House,
2016
Hyra,
2015
Program Vision, Organization, and
Management:
- What was the original
government vision for the
Promise Zone program?
- What were the primary
motivations for communities to
participate in the Promise Zone
program?
- What were the biggest
perceived barriers to launching
the program?
- What did it mean to be a
Promise Zone before the
official designations went out
in 2016?
- Did Round 1 receive promised
benefits? When and how?
- What types and how many
federal staff were made
available to help Promise
Zones navigate federal
resources?
- When and how many
AmeriCorps VISTAs
(Volunteers in Service to
America) were obtained?
- What were the original
reporting requirements, and
what data was received by
HUD?
- What activities are currently
ongoing to enable Promise
Zones to provide the required
information?
- What are the greatest realized
benefits of program
participation?
- What are the recommendations
for improving the program
structure?
- Has participation in the
program created perceived
Funding (leveraged):
- number applications
- number grants awarded
- number grants awarded with
preference
- dollar amount of funding awarded
- receive priority consideration
through additional preference
points on competitive federal grant
applications
Economic and Small Business
Development:
- poverty rate
- median household income
- unemployment rate
- increase economic activity:
o private and public
investments
o financing initiatives
o improved security for small
businesses
o employing a business
consultant to develop an
economic development plan
that will enhance investment
among business owners and
government contractor
- leverage private investment:
o use of committed and
anticipated investments to
spur economic development
and create job opportunities
for residents
o pledging 50 percent of
existing business assistance
grants to promise zone
enterprises and developing a
capacity building and
sustainability plan for the
area’s nonprofits
- create jobs:
o providing incentives that
will attract small businesses,
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
76
Reference PZ Evaluation Questions PZ Measures/Outcomes
effects on the behavior of local
partners? On the ways local
entities view and organize their
partnerships?
private companies, and
institutions to the
community
o providing job skills for area
residents
o includes partnering with
education and training
providers to provide career
training programs in local,
high-demand occupations
o prioritizing the city’s
funding for non-profits that
provide employment
services
Education:
- educational attainment (less than
HS degree)
- increasing access and enrollment to
pre-kindergarten programs
- installing a science, technology,
engineering, and mathematics
(stem) program, expanding
enrollment in early college
programs
- increasing adult education
opportunities by partnering with a
local college
- improving school readiness for pre-
school-aged children
- increasing during and after school
programs to promote kindergarten
through 12th grade (k-12) success
- improving school attendance
through parent engagement and
wrap-around services
- increasing college readiness and
enrollment
Critical Community Improvements:
- reduce violent crime:
o includes efforts to improve
neighborhood conditions by
removing blight and
maintaining vacant lots
o includes increasing
proactive policing activities,
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
77
Reference PZ Evaluation Questions PZ Measures/Outcomes
improving lighting, cleaning
up vacant lots, and razing
abandoned buildings
- address other priorities identified
by the community
- engage in joint local planning
- improved access to Volunteers in
Service to America (VISTA)
personnel
- promoting local assets—including
parks, landmarks, and
transportation networks
- ensuring quality transportation
access
- focusing resources on commercial
districts, parks, and public spaces
- make neighborhoods safe:
o improving public safety
service delivery
o providing gang prevention
o intervention services for
families
o promoting safe routes to
school for children and
parents
Quality, Affordable Health Care:
- enhance public health
Housing and Safe Neighborhoods:
- promote more sustainable and
livable communities
o preserving affordable and
rent-controlled housing
o supporting homelessness
prevention programs
Programmatic:
- improved access to federal staff
from multiple agencies
- take advantage of the provision of
technical assistance
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
78
With the support and technical assistance from HUD in the development and
implementation of their Promise Zone Data Collection and Evaluation Frameworks and technical
assistance with data collection from the National Neighborhood Indicators Partnership (NNIP),
federal agencies supported local grantees and helped them to use data in all phases of their work
(situation analysis, policy analysis and planning, performance management, case management,
evaluation, and sustainability). This support may have contributed to each community being able
to report on outcomes specific to their community that addressed local needs. The Obama
administration’s approach to urban policy showed that spatial and temporal coordination across
federal agencies at the local level is needed. This includes federal technical support at the local
level to coordinate and integrate local interventions across multiple policy domains so that
multiple projects are implemented at the same time and place for a collective impact (Stoker &
Rich, 2019).
PZ designees made several recommendations including “establishing a memorandum of
understanding (MOU) or interagency agreement (IAA) among partnering federal agencies to
ensure continuity over terms beyond administrations; ensure that there is high-level buy-in (local
and federal); have a dedicated project director at headquarters; keep sites and stakeholders
involved including ongoing communication, both from federal headquarters to the partner
agencies and between promise zone designees; have a clear organizational structure; provide
technical assistance for on-the-ground partners including a matrix of data to be collected for
evaluation and “mini-grants” to organizations for collective impact training, a media kit for
guidance in communications (Zapolsky, et al, 2019).
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
79
Opportunity Zones
There were several manuscripts and/or reports analyzed for the Opportunity Zone (OZ)
urban policy (Table 4.4). Din (2018) used GIS to illustrate the percent vacant business addresses
using natural breaks classification overlaid with opportunity zones and recommends monitoring
the business vacancy rate in OZs to measure community and economic development. The author
hypothesizes that as Opportunity Funds are created, new businesses will develop within the
Opportunity Zones and attract other businesses that are not part of Opportunity Fund (Din,
2018). The author also hypothesizes that workers will be recruited from local communities and
thus job creation will also increase (Din, 2018). The limitation of this article is that it only
provides a GIS map of one point in time.
The GAO (2021) report (Opportunity Zones- Census Tract Designations, Investment
Activities, and IRS Challenges Ensuring Taxpayer Compliance) stands out the most because it
highlights the IRS’s inability to effectively monitor compliance of OZ tax incentives or the
impact of the policies in the designated census tracks (communities). This GAO (2021)
evaluation was requested by Congress because of the magnitude (impacting over 9,000 census
tracts) and the extended time for the OZ tax benefit (tax benefits for this policy do not expire
until 2047). The GAO (2021) report (1) describes the process for designating census tracts as
OZs and compares select demographic characteristics of designated and non-designated tracts;
(2) describes Qualified Opportunity Funds’ experiences with and use of the OZ tax incentive; (3)
describes states’ experiences with the incentive; (4) describes the effect of the Coronavirus
Disease 2019 (COVID-19) pandemic on investment activity in OZs; (5) analyzed available
taxpayer data about Qualified Opportunity Funds and their investors; and (6) evaluated Internal
Revenue Service.
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
80
Table 4.4 Opportunity Zones (OZ) SDoH Evaluation Questions, and Measures/Outcomes
Reference OZ Evaluation Questions OZ Measures/Outcomes
The U.S.
Impact
Investing
Alliance and
the Beeck
Center.
(2019). The
Opportunity
Zone
Framework:
Prioritizing
and
Achieving
Impact in
Opportunity
Zones.
- Understand the impact of
investors and enable
independent evaluators
and researchers to
analyze the long-term
outcomes of the overall
policy more deeply
- Measure and track
progress against specific
impact objectives,
identifying key outcome
measures and allow for
continuous improvement
Investments:
- geographic focus (state, zip code, urban
focus, rural focus, etc.)
- intended investment focus (housing,
small business, growth business, etc.)
- target investment size
- investment mission statement or impact
objective
- size of the fund
- total assets
- eligible deferred gain asset
- type of investor (corporate, individual
filer)
- tax-paying residence of investors
(aggregated by state and portion of fund
assets)
- structure of fund (single, multi-asset)
- location of capital deployment (census
tracts)
- type of qualifying property or business
- GP demographics (race and gender
composition)
Transaction Data Reporting
- size of investment
- location of investment (census tract or
address)
- NAICS code of operating business
- type of qualifying property
Housing and Safe Neighborhoods:
- number of affordable units created as %
of development
- net new number of affordable units
- number of net additional individuals
housed based on development
- percent of units that are affordable
- number of affordable units renovated
- square footage of real estate
- commercial
- residential
- infrastructure improvements
Economic and Small Business Development:
- job retained
Education
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
81
Reference OZ Evaluation Questions OZ Measures/Outcomes
- high school graduation rates
- school retention rates
Economic and Small Business Development:
- number of jobs retained within
community
- percent of salary increase for lowest paid
employees
- number of employees (permanent,
seasonal, construction)
- net new jobs created
- number of employees from LMI
communities
- employment of targeted disadvantaged
groups (i.e., returning citizens, veterans,
etc.)
- entrepreneurship (if applicable)
- NAICS code of commercial tenants (real
estate)
- percentage of woman- or minority-owned
enterprises
- percentage of first-time business owners
- number of new businesses started,
number of female and minority owned
businesses created
- access to healthy food
- environment impact
Critical Community Improvements:
- method of community engagement
- type of feedback incorporated
- partnerships with local organizations and
type of partnership
- community needs assessment or
alignment with established community
priorities
- how investment is aligned with
local/regional economic development
strategies
Post-Exit Evaluation
- community impact beyond hold period
- variation from original intention
Freedman,
M., Khanna,
S., &
Neumark, D.
(2021).
- Early impacts of the
Opportunity Zone
program on residents of
targeted areas particular
Economic and Small Business Development:
- increase in employment rates of residents
- reductions in poverty rate
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
82
Reference OZ Evaluation Questions OZ Measures/Outcomes
on employment,
earnings, and poverty
Din, A.
(2018).
- Who will benefit from
investments into
Opportunity Zones?
- One measure to
community and
economic development is
to monitor the business
vacancy rate in
Opportunity Zones
Economic and Small Business Development:
- the percent vacant business addresses in
OZs
(GAO, 2020) - What are the
characteristics of
businesses in Zones, such
as location, business
type, number of
employees, finances, and
residential units (if
applicable)?
- What are the
characteristics of
Qualified Opportunity
Funds, particularly in
regard to the dollar
amount of assets held and
types of investments?
Investments:
- number of Qualified Opportunity Fund
investments and amount
- amounts of short-term and long-term
gains invested and the dates of those
investments,
- census tract locations of investments,
- descriptions of investments and dates
sold
Economic and Small Business Development:
- the numbers of jobs by type
- the type of business and number of
employees
Housing:
- numbers of rental and for-sale housing
unis
- the square footage of real property and
number (if any) of residential units
Critical Community Improvements:
- the capacity of educational, childcare,
and health care facilities
GAO, 2021 - How have Opportunity
Zones’ characteristics
changed, for example
with regard to poverty,
income, unemployment,
education levels, race,
affordable housing, and
displacement?
- What are the
characteristics of
businesses in
Opportunity Zones, such
as location, business
Economic Development:
- median household income (dollars)
- average poverty rate (percent)
- average unemployment rate (percent)
Demographic data:
- percent of population non-white
(race/ethnicity
make-up of census track)
- foreign born
- living in non-English speaking household
- high school education or less
- a bachelor’s degree or higher
Investment (type and amount):
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
83
Reference OZ Evaluation Questions OZ Measures/Outcomes
type, number of
employees, finances, and
residential units (if
applicable)?
- What are the
characteristics of
Qualified Opportunity
Funds, particularly with
regard to the dollar
amount of assets held and
types of investments?
- residential
- mixed-use
- commercial real estate
- operating business
Incentives (state-level):
- offering additional incentives to invest in
opportunity zones
- greater consideration to Opportunity
Zones projects if those projects also
apply for state economic development
grants or other programs (i.e., points)
- layer other state or local incentives,
including the Investment Tax Credit,
Enterprise Zone Incentive Leverage Act,
and tax increment financing
Programmatic:
- state promotion activities:
- providing information and education on
oz tax benefit
- providing technical assistance to
communities
- working with non-profits in public-
private partnerships
- connecting investors with investment
opportunities
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
84
Another GAO (2020) report on OZs concluded that data collected by the IRS can be
useful in evaluating outcomes however there are challenges with their usefulness and usability
due to IRS constraints, data availability, taxpayer privacy protections, and limited collection of
data specific to the economic and other societal effects of OZ investments. GAO (2020) also
provided two recommendations for congressional consideration, “1). that Congress consider
providing Treasury with authority and responsibility to collect data and report on OZ’s
performance, in collaboration with other agencies; and 2). Congress should also consider
identifying questions about OZ’s effects that it wants Treasury to address in order to help guide
data collection and reporting of performance, including outcomes.” Freedmen (2021) studied
four main outcome measures: (1) overall employment among residents, (2) the employment-to-
population ratio for residents, (3) average earnings of employed residents, and (4) the poverty
rate for residents. Freedman (2021) found limited evidence of any impacts of zone investment to
date on zone resident outcomes.
In 2020, the GAO (2020) recommended the following questions that could be used to
evaluate the Opportunity Zone’s tax incentive’s performance: “(1) How have Opportunity
Zones’ characteristics changed, for example with regard to poverty, income, unemployment,
education levels, race, affordable housing, and displacement?; (2) What are the characteristics of
businesses in Opportunity Zones, such as location, business type, number of employees,
finances, and residential units (if applicable)?; and (3) What are the characteristics of Qualified
Opportunity Funds, particularly with regard to the dollar amount of assets held and types of
investments?” However, no legislation to address these recommendations have been introduced
as of September 2021 (GAO, 2021).
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
85
Step 3: Environmental Scan of Existing Databases to Identify Key SDoH Measures
Table 4.6 highlights shared measures collected or recommended by past urban policy
evaluations and/or government reports that were identified in Step 2. The SDoH measures are
organized by urban policy domain using a cross walk of Healthy People 2030 social
determinants of health domains Table 4.5. highlights a cross walk of the Healthy People 2030
Social Determinants of Health Domains (SDoH) and common urban policy domains to identify
the best and most comprehensive set of SDoH measures to assess the impact of urban policy on
the social determinants of health in targeted communities (DHHS, 2020).
Table 4.5. Urban Policy Focus Areas and Social Determinants of Health Domains
HP 2030 Social Determinants of Health
Domains
Common Urban Policy Domains
Economic Stability Economic & Small Business Development
Neighborhood and Built Environment Housing & Safe Neighborhoods
Education Access and Quality Education and Workforce Development
Health Care Access and Quality Quality, Affordable Health Care
Social and Community Context Critical Community Improvements
Data sources summarized in Table 4.6 met the inclusion criteria described in Step 2:
national data, open-source data, public availability, annual data collection, data available at the
census track or county level, data that can be stratified by race/ethnicity, age, gender, education,
or income level, and by business or industry type, size, and/or business status when applicable,
with the exception of IRS data forms (Form 8949, Form 8997, Form 1099-B, and Form 8996).
Although, the IRS collects data explicitly for tax administration and compliance purposes, these
data, such as investment amounts, can be used to evaluate outcomes (GAO, 2021). The data
sources described below are all collected, analyzed, and maintained by federal agencies.
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
86
The United States Census Bureau develops the American Community Survey (ACS)
Data Profiles and summarizes the most used ACS data requested across four themes: social,
economic, housing, and demographic data (U.S. Census Bureau, 2020). The ACS Data Profiles
summarize data in both numbers and percentages for a single geographic area, covering
information across all four themes (U.S. Census Bureau, 2020). The Small Business
Administration’s (SBA) Office of Advocacy develops Small Business Profiles at the
Congressional Districts level (SBA, 2021). The Small Business Profile provides a snapshot of
436 congressional districts’ small business statistics (SBA, 2021). These statistics include data
on the total number of small employers and their industry breakout, the number of workers
employed by small businesses, and the number and distribution of self-employed workers across
the district (SBA, 2021).
The Internal Revenue Service (IRS) administers the Opportunity Zones tax incentive as a
component of the Internal Revenue Code to ensure that taxpayers follow the regulations and
conditions (GAO, 2021). There are four forms required by the IRS used by investors and
qualified opportunity funds to certify opportunity funds, report deferred gains, and to report
disposition of interests in funds (GAO, 2021). These four forms include key indicators that can
be used in OZ evaluations. However, there a limitation to use and reporting with these forms. For
example, qualified businesses that receive Qualified Opportunity funds are not required to
provide data on those investments to the IRS (GAO, 2021). The HUD Integrated Disbursement
and Information System (IDIS) collects information annually on five federal grant programs:
Community Development Block Grant (CDBG), HOME Investment Partnerships
(HOME), Housing Trust Fund (HTF), Emergency Solutions Grants (ESG), and Housing
Opportunities for Persons With AIDS (HOPWA). The Section 108 Loan Guarantee Program
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
87
(Section 108) enables CDBG recipients to access low-cost, flexible financing for economic
development, housing, public facility, and infrastructure projects by leveraging their annual grant
allotment. The HUD IDIS database also requires grant recipients to provide demographic and
income information about the persons that benefited from a community’s activities.
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
88
Table 4.6 Shared Measures/Outcomes, Evaluation Questions and Available Data Sources
by Social Determinants of Health and Urban Policy Domains
Shared Measures /Outcomes (within the
zone)
Data Sources
Economic Stability/Economic and Small Business Development
Evaluation Questions:
- Have urban policies/programs succeeded in stimulating business development and at
what cost? If successful, to what degree and what types of businesses have enterprise
zones attracted? If certain types of businesses have not responded or overall business
response has been limited, what additional action is needed and by whom?
- Have urban policies/programs succeeded in creating jobs, particularly for the
disadvantaged? If so, to what degree? What types of jobs have been created and for
what types of workers? Is additional job creation action needed and if so, by whom?
- What effect have urban policies/programs had on distressed areas?
- Have there been unintended effects, such as: business relocations rather than new, net
economic activity?
- How have communities targeted by urban policies/programs changed, for example
with regard to poverty, income, and unemployment?
- What are the characteristics of businesses in Zones, such as location, business type,
number of employees, finances, and residential units?
- What are the characteristics of Qualified Opportunity Funds, particularly in regard to
the dollar amount of assets held and types of investments?
Poverty rate (family/persons) ACS
unemployment rate ACS
median household income/median family
income (dollars)
ACS
dollar amount invested in Opportunity and
Promise Zones
IRS
- Investors: Form 8949 and Form
8997
- Qualifies Opportunity Funds: Form
1099-B and Form 8996
number of new jobs created or retained IRS (corporate income tax credits)
number of employees by congressional district
SBA: Small Business Profiles for
Congressional Districts
number of new small businesses (with
employees) by congressional district
SBA: Small Business Profiles for
Congressional Districts
total payroll of employees of small businesses
by congressional district (dollar amount)
SBA: Small Business Profiles for
Congressional Districts
types of investment projects in the zones (i.e.,
residential real estate development,
commercial real estate development,
IRS
- Investors: Form 8949 and Form
8997
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
89
Shared Measures /Outcomes (within the
zone)
Data Sources
multifamily housing development, agricultural
land development, renewable energy
businesses, hotel development, etc.)
- Qualified Opportunity Funds: Form
1099-B and Form 8996
economic development loans
HUD Integrated Disbursement and
Information System (IDIS)
small business loans approved U.S. Small Business Administration’s
primary programs (7A and 504) data
residential lending Home Mortgage Disclosure Act Data
funding leveraged for economic development
programs (federal, state, local grants)
- number of applications from PZ/OZs
- number of grants awarded in PZ/OZs
- number of grants awarded with preference
points in PZ/OZs
- amount of funding awarded in PZ/OZs
Federal agency program websites (i.e.,
SBA, EDA, Department of Treasury,
etc.)
Neighborhood and Built Environment/Housing & Safe Neighborhoods
Evaluation Questions:
- Whether the neighborhood composition of urban policies/programs residents
changed?
- Whether the local housing market responded to the policy or program in the
designated zone?
- How have communities targeted by urban policies’/programs’ characteristics
changed, for example with regard to race, affordable housing, and displacement?
- What effect have urban policies/programs had on distressed areas? Have there been
unintended effects, such as:
o business relocations rather than new, net economic activity, and
o residential displacement? If problems are encountered in these areas, what
actions are needed to mitigate their effects?
Racial composition ACS
median gross rent ACS
median value (owner occupied) ACS
total housing units ACS
occupied housing units ACS
vacant housing units ACS
homeowner vacancy rate ACS
rental vacancy rate ACS
year householder moved into unit (2019 or
later)
ACS
housing and physical infrastructure loans
- number and dollar amount of housing
loans
- number and dollar amount of physical
infrastructure loans
HUD Integrated Disbursement and
Information System (IDIS)
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
90
Shared Measures /Outcomes (within the
zone)
Data Sources
Education Access and Quality/Education and Workforce Development
Evaluation Questions:
- How have communities targeted by urban policies’/programs’ characteristics
changed, for example in regard to education levels?
High school graduation rates ACS
educational attainment (less than HS degree) ACS
the population 18 to 24 years enrolled in
college or graduate school
ACS
Population 3 years and older enrolled in pre-
school
ACS
funding leveraged for economic development
programs (federal, state, local grants)
- number of applications from PZ/OZs
- number of grants awarded in PZ/OZs
- number of grants awarded with preference
points in PZ/OZs
amount of funding awarded in PZ/OZs
Federal agency program websites (i.e.,
DOE, DOJ, etc.)
Health Care Access and Quality/Quality, Affordable Health Care
Evaluation Questions:
- There were no evaluation questions identified in STEP 2 of analysis for this domain.
with health insurance coverage ACS
with no health insurance coverage ACS
Social and Community Context/Critical Community Improvements
funding leveraged for social programs (federal,
state, local grants)
- number of applications from PZ/OZs
- number of grants awarded in PZ/OZs
- number of grants awarded with preference
points in PZ/OZs
amount of funding awarded in PZ/OZs
Federal agency program websites (i.e.,
HHS, CMS, etc.)
total household with a computer ACS
total household with broadband internet
subscription
ACS
ACS – American Community Survey: Data Profiles | American Community Survey | U.S.
Census Bureau
SBA – Small Business Administration: 2021 Small Business Profiles for Congressional Districts
– SBA's Office of Advocacy
HUD – IDIS: Integrated Disbursement and Information System - HUD Exchange
IRS – Form 8997, Initial and Annual Statement of Qualified Opportunity Fund (QOF)
Investments
IRS: Form 8949, Sales and other Dispositions of Capital Assets
IRS: Form 8996, Qualified Opportunity Fund
IRS: Form 1099-B Proceeds from Broker and Barter Exchange Transactions
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
91
Summary
There were several limitations to identifying national data sources that met the inclusion
criteria. Overall, identifying this data at the national level required using data sources developed,
collected, and maintained at the national level by federal agencies. The majority of data sources
included in Table 4.6 met the criteria with the exception of IRS data forms (Form 8949, Form
8997, Form 1099-B, and Form 8996). Although, the IRS collects data explicitly for tax
administration and compliance purposes, these data, such as investment amounts, can be used to
evaluate outcomes (GAO, 2021). Due to limited reporting by the IRS on performance, additional
data collection and reporting on the incentive are necessary to evaluate outcomes of the OZ tax
incentives (GAO, 2021). GAO (2020) asked Congress to consider requiring more reporting on
OZ tax incentives, but as of September 2021, no legislation was passed. One additional
limitation was the frequency of the ACS Profiles; although data is collected annually, summary
reports are only developed by ACS every 5 years covering a 5-year period of time. To conduct
data analysis on annual data, a researcher would have to gain access to data and conduct analysis
at a Federal Data Research Center.
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
92
Chapter 5: Discussion, Implications, and Conclusion
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
93
A Collective Impact Approach to Urban Policy and the SDOH
In order to effectively implement and evaluate urban policy, an evidence-based
approached should be used to develop urban policies and programs. Lewis, et. al. (2011) state
that “achieving policy change does not necessarily insure effective policy implementation”. A
study done in 1995 estimated since the War on Poverty, U.S. taxpayers have spent over $22
trillion on anti-poverty programs, but no significant improvement in poverty had been achieved
in urban communities (Scruggs, 1995). Lessons learned from the individual success of the
Promise Zone initiatives illustrate the capacity for communities to make strides in achieving
local level outcomes they set out to achieve and the ability to leverage existing federal resources,
without pouring $1 billion additional taxpayers’ dollars into programs. These lessons can be used
for policy expansion (a larger public sector role) and recalibration (reforming the existing policy)
of current Promise Zones and Opportunity Zone policies and programs. The recalibration would
include developing a rigorous evaluation that would aggregate data across the Urban Policy
SDoH domains (mutually reinforcing strategies); incorporate solutions to address challenges
encountered by the designated communities (challenges identified in a policy evaluation);
require specific performance measures across the Urban Policy SDoH domains (via shared
measurement system of Urban Policy SDoH CI Evaluation Framework); require specific number
of preference points be given in notice of funding opportunities (NOFOs) by each agency for
consistency (i.e. 30 additional points in the needs category for designated communities)each
agency would need to align their timeline of release of grants to ensure the programs can be
implemented collectively in the same designated communities at the same time.
A collective impact framework should serve as the framework for developing urban
policy that addresses the SDoH. Figure 5.1, Collective Impact Framework for Public Policy.
Figure 5.1 illustrates how public policy can allow or incentivize collective impact (Ferber &
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
94
White, 2014). In the grant application process key components in the Collective Impact
Framework can be required and/or considered in the design of the grant. For example, to ensure
there is a common agenda developed that engages stakeholders (i.e., multi-sectoral partners), a
planning grant can be released to provide funding to convene grant recipients and their
stakeholders and provide funding support to develop a common agenda (i.e., Choice
Neighborhood Transformation Plan). The Collective Impact Framework includes “a centralized
infrastructure, a dedicated staff, and a structured process that leads to a common agenda, shared
measurement, continuous communication, and mutually reinforcing activities among all
participants” (Kania, & Kramer, 2011). The collective impact framework includes the tools
needed to develop a shared measurement system that would align with the multi-agency
approach to addressing poverty and the SDoH via urban policy.
Figure 5.1 Collective Impact Framework for Public Policy
Source: Ferber & White, 2014, p.23. Public Policies that Encourage Collective Impact
The Collective Impact for Public Policy was used as an evidence-based model to develop
the Urban Policy and SDOH Collective Impact Evaluation Framework, the first imperative was
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
95
to understand needs and opportunities in communities designated by urban policies as “zones” in
Step 1 (Literature Review) of my methods which also aligned with developing a common
agenda. The second component of the collective impact framework was to develop shared
measurement system. This included identifying a common set of measures abstracted from urban
policy literature readily available data sources for tracking progress towards a common agenda.
Using lessons learned from Step 2 (Literature Review to Identify SDoH Measures for Urban
Policies and Programs), I drafted a comprehensive set of measures abstracted from past urban
policies that could be collected for each SDoH Domain.5.3 Data sources for measures that are
reported at the local level should be included in reporting requirements and data sharing
agreements in contracts, grants and public-private partnership agreements for entities that seek to
do work in the communities designated by the urban policy (i.e., Promise Zones and Opportunity
Zones). This data should also be collected and stratified by race/ethnicity, age, and gender to
ensure equity can be assessed.
The third component of the collective impact framework was to develop mutually
reinforcing strategies or activities. For public policy this includes creating blended funding
streams and allowances for tailoring to local conditions (Ferber & White, 2014). One example
was found in the Opportunity Zone Implementation Plan which included goals, activities, action
items, and a plan to coordinate across agencies and engage with community leaders,
entrepreneurs, and investors. This also included creating blended funding streams and
allowances via the White House Office of Management and Budget rule allowing private
organizations that receive money from more than one agency to consolidate their reporting
(White House Office of Management and Budget, 2013). The Opportunity Zone Reporting
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
96
Framework also allowed for one report with shared measurement across the multiple work
streams.
The fourth component of the Collect Impact Framework is continuous communication,
which includes keeping an open dialogue between stakeholders to build trust, reaffirm shared
goals and objectives, and recognize the shared motivation of their divergent efforts. The
Collective Impact Framework for Public Policy includes ensuring there are requirements for
documenting the process of collaboration in grants and contracts; and allowing for adjustment in
plans to support emergence or innovation (Ferber & White, 2014). In addition to collecting data
for shared measurement, the evaluation plan should include data collection on community
engagement via qualitative data collection. This includes collecting data to assess if community
feedback was incorporated into the strategic plan; to assess partnerships with local organizations
and type of partnership; to assess if the community needs assessments aligns with established
community priorities; and to assess how investment is aligned with local/regional economic
development strategies (The U.S. Impact Investing Alliance and the Beeck Center, 2019).
The fifth component of the collective impact framework is ensuring there is a backbone
agency. At the federal level, the President often designated the Department of Housing and
Urban Development (HUD) as the backbone agency for past urban policies. At the local level,
the Mayor’s Office often serves as the backbone agency for providing leadership to mobilize for
collective action in the designated opportunity zones. The Mayor’s Office also leads the
development of the infrastructure, establishes multi-sector partnerships (including public-private
partnerships), measures and monitors continuous improvement, and shares outcome measures
across stakeholders internally and externally (public).
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
97
An Urban Policy and SDOH Collective Impact Evaluation Framework
Figure 5.2 summarizes the steps used to develop the Urban Policy and SDOH
Collective Impact Evaluation Framework. Step 1, the Literature Review of Urban Policies led
to the identification of the logic model components: inputs, activities, and outputs. Step 2, the
Data Abstraction from Urban Policy Literature led to the identification of additional logic model
components: outcomes and measures. Step 3, the Environmental Scan of Existing Data Sources
and Databases led to a set of SDoH measures and data sources. Ultimately all this information
was used to develop the Urban Policy and SDoH Collective Impact Evaluation Framework
illustrated in Figure 5.3.
Figure 5.2: Summary of Urban Policy and SDoH Evaluation Framework Development
Process
Figure 5.3 expands upon the initial framework adapted from Preskill, et al. (2014) in
Figure 3.2. It synthesizes information abstracted in Steps 1-3 to provide details in the Urban
Policy and SDoH Collective Impact Evaluation Framework along the continuum of the social
political context, collective impact design and implementation, and outcomes. The Collective
Impact (CI) common agenda (inputs) includes the information pertaining to the urban policy
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
98
including grant programs, the budget amount appropriated, and the lead agencies responsible
overseeing implementation of the policy and programs. The CI mutually reenforcing strategies
include key activities included in the urban policy legislation or programs, for example those
listed as Urban Policy Focus Areas and Social Determinants of Health Domains in Table 4.5.
As we across the logic model (framework), the CI Capacity (outputs), CI Systems
(short/intermediate outcome), and CI Behavioral Change (long-term outcomes) are key
components of the framework that evaluation of policy should occur. The CI Capacity of a
community designated as a zone plays a vital role on if the policies and programs made available
had an impact on SDoH outcomes. Assessing the capacity of the collective can be done through
evaluating the outputs of urban policy and programs. An evaluation question to consider
includes: What difference has the urban policies programs made on its stakeholders and their
capacity to address complex problems? Capacity is also dependent on the leadership of the CI
initiative, which should consist of stakeholders from multiple sectors that are able to shift both
public and private funds (Preskill, Parkhurst, & Juster, 2001). The CI Systems component
includes policy and funding outcomes and can be used to assess if the overall public funding
(federal, state, or local government) appropriated for policy implementation for the targeted issue
area or system has increased and if the desired short and intermediate outcomes have increased.
Lastly, the CI Behavior Change component of the framework includes long-term outcome used
to assess behavior change in the people who live in communities impacted by urban policy.
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
99
Figure 5.3 Urban Policy and SDoH Collective Impact Evaluation Framework
Implications for Public Health
Mishori (2019) states that the root causes of health and health disparities are driven by
policy and politics. Political determinants of health (PDoH) are factors, systems, or structures
(laws and policies) that create, perpetuate, or exacerbate these differences (Dawes, 2020). As
urban policies and programs are developed that target specific communities and people who live
in these communities, we need to ensure that they are evaluated collectively to assess if they are
achieving the intended outcomes. Public and private entities can no longer work in silos or assess
the impact of policies or programs in silos without assessing the impact on the SDoH of the
people. It is imperative that as policies and programs are implemented, that they are also
evaluated using common measures that can aggregated at the local, state, and national level in
order to assess the effectiveness of the policies and programs.
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
100
A Policy Bubble for Social Determinants of Health
Maor (2014) describes a policy bubble “as a situation in which the support of the general
public for a certain public policy increases significantly over a brief period of time and is
therefore suspected of being vulnerable to an equally rapid collapse of support”. In 2020,
Congress acknowledged and supported public health’s role in working across sectors on social
determinants of health and passed the Social Determinants Accelerator Act of 2019. Through this
act a SDOH pilot program was appropriated with a $3 million budget to support the development
of SDOH Accelerator Plans to address health and social outcomes for specified target
populations (Brown, 2021). There was continued support in 2021 with the reintroduction of the
Improving Social Determinants of Health Act of 2021 (H.R.379), by the House of
Representatives who requested $50 million be appropriated to fund the program, with an
additional request to increase appropriation to $153 million in President Biden’s FY2022
Discretionary Request (Brown, 2021). The final approved FY2022 appropriation was approved
at $5 million. There were two primary goals of the program: “1.) improve health outcomes and
reduce health inequities through the coordination of SDOH activities across the CDC; and 2.)
improve public health agencies’ and community organizations’ capacity to address SDOHs in
communities” (Brown, 2021, p.1). Activities in the bill to be implemented by the program
included:
1. “Ensure that CDC programs consider and incorporate SDOHs in grant awards and
other activities.
2. Award grants to local, State, territorial, and Tribal health agencies and
organizations, and other eligible entities, to address SDOHs in target communities.
3. Award grants to nonprofit organizations and public or nonprofit institutions of
higher education to conduct research on best practices; provide technical
assistance, training, and evaluation assistance; and share best practices with
grantees.
4. Coordinate, support, and align CDC SDOH activities with other Federal agencies
(e.g., HHS and CMS).
5. Collect and analyze data related to SDOHs” (Brown, 2021, p.1).
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
101
There are a number of SDoH activities being implemented to build the capacity to collect
SDoH data and to integrated SDoH measurement activities into public health data modernization
efforts. For example, the Gravity Project is national public collaborative that develops
consensus-based data standards to improve how we use and share information on social
determinants of health (SDOH) (The Gravity Project, 2022). CDC’s Clinical and Community
Data Initiative (CODI) engages people and develops processes and technology to improve data
accessibility and use for understanding health and latest updates to CODI will enable assessment
of SDOH (CDC, 2022). The United States Core Data for Interoperability (USCDI) specifies
requirements for standards-based health data classes and constituent data elements whose use are
mandated by Office of the National Coordinator for Health Information Technology (ONC) and
Center for Medicaid Services (CMS) to ensure improved healthcare information interoperability
(ONC, 2022). Currently version three is open for public comment and includes SDoH goals, and
a SDoH assessment (ONC, 2022). Lastly, the Assistant Secretary for Planning and Evaluation
(ASPE) which advises the Secretary of the U.S. Department of Health and Human Services
(HHS) on policy development in health, disability, human services, data, and science; and
provides advice and analysis on economic policy released documents that: “(1) explain why
HHS has identified addressing SDOH as a major priority for the Department; (2) describe HHS’
strategic approach, and specific actions the Department is taking to address SDOH; and (3) link
how these actions build on evidence in the field on what works” (ASPE, 2022).
The contribution of my dissertation will provide an SDoH evaluation framework for
urban policy, a comprehensive set of SDoH measures, and SDoH data sources to measure the
impact of the urban policy on local communities targeted by urban policy. This comprehensive
set of SDoH measures can be integrated into future grants and contracts to ensure that urban
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
102
policy is evaluated across the five social determinants of health domains (economic stability,
neighborhood and built environment; education access and quality; health care access and
quality; and social and community context).
Summary
Brewer (1974) suggested the need for a more systematic process for policy decision
making over time. This process included six phases: invention/initiation, estimation, selection,
implementation, evaluation, and termination (Brewer, 1974). The invention/initiation phase
included recognizing the problem and identifying a range of solutions (Brewer, 1974). The
estimation phase included conducting systematic literature reviews, outcomes analysis, risk
assessments, and cost/benefit analysis for each of the possible solutions (Brewer, 1974). The
selection phase includes identifying the best possible solution, as determined by the decision
maker (Brewer, 1974). Brewer (1974) describes the implementation phase as understanding how
a policy or program is implemented, what is happening, and how implementation differed from
how the policy or program was intended. Today, this is considered implementation science.
(Glasgow, Eckstein, El Zarrad & Khair (2013) define implementation science as “the application
and integration of research evidence in practice and policy”.
The fifth phase of the policy decision process described by Brewer (1974) is evaluation.
Brewer (1974) described evaluation as looking back at the practice and assessing the success of
the program. It is amazing that almost 50 years ago, Brewer (1974) describes evaluation as
essential in determining whether a policy is terminated, and over the course of history policy
evaluation was mostly done via economic evaluations and cost-benefit analysis, not social
impact. The professional field of program evaluation was not established until the 1970s (Hogan,
2007) and has not been widely implemented across all government agencies.
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
103
The last phase described by Brewer (1974) is the termination phase. In the termination
phase, evaluation results should be used to adjust or terminate a policy or program that is
“dysfunctional, redundant, outmoded, and unnecessary” (Brewer, 1974, p. 241). This phase is
often not executed and results in the continuation of programs or policies that affect people’s
lives. This phase should be a part of the evaluation phase (i.e., not a separate sixth phase) and
should include recommendations made by program evaluators that provide qualitative and
quantitative outcomes of the policy or program’s impact. Programs that do not have a positive
impact should be evaluated more to determine if they can be improved, including
recommendations from the people impacted by the policy or program on how it can be improved,
and/or what unintended consequences need to be addressed if the program is terminated.
The challenges with the policy process that Brewer mention in 1974 are still issues today
and the decision-making process for policy development, selection, implementation, evaluation,
and termination, are not being applied to the creation or termination of a policy. Over the last one
hundred years every president has changed urban policy and not one president has gone through
the policy decision-making process described by Brewer (1974). Moreover, almost every
president has terminated the previous administrations’ urban policy and there is limited literature
that demonstrates an evaluation was conducted to determine the success of the policy or the
unintended consequences of terminating the policy on the communities where the policy was
implemented.
Policy evaluation should be conducted systematically and comprehensively on all urban
policies “to acquire insight into the consequences that are attributed to certain policies” (Bartels,
et. al, 1982). Most of the literature on urban policy provides the historical and political context of
the policy, the administration the policy or program was established under, any bias in the policy
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
104
development, and some include the reasons why the policy or program failed. However, there is
little to no data on the social impact of the policies provided to support claims of success or
failure to meet the original intent of the policies passed. But there is literature available now to
demonstrate the impact national urban initiatives such as the New Deal (federal home lending
and large-scale public housing), the War on Poverty (slum clearance and other forms of urban
“renewal”), and the Model Cities have had on cities and their residents, including racism,
welfarism, sprawl, and hubristic top-down planning (Hendrickson, 2004). The concept of the
trickledown effect via providing tax incentives to businesses in urban communities to solve the
poverty problem and influence investment or business location decisions, has yet to show
effectiveness. (Harrison & Kanter 1978; Peirce, Vaughn 1979; Jacobs & Wasylenko 1981;
Schmenner 1982; Hawkins 1984; Rubin & Zorn 1985).
Wheatley (1992) states that “no problem can be solved from the same consciousness that
created it.” There needs to be a paradigm shift in urban policy and programming to help the field
understand the whole of the community and the problems it faces as it relates to poverty,
affordable housing, economic development, education, and workforce development There needs
to be a shift from focusing on one approach to focusing on holism, a collective impact. This
paradigm shift requires assessing the community and understanding the historical context of the
community, current data related to poverty (crime, education, access to jobs, past policies,
programs, community sentiments, political, context, etc.), and understanding the community as
“a system and giving primary value to the relationships that exist among seemingly discrete
parts” (Wheatley, 1992). Karakas (2009) bases a new paradigm in organizational development
on a) understanding the characteristics of social systems, b) understanding the hows and whys of
organizational change, and c) understanding the role of third-party change agents.
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
105
Although urban policy stakeholders’ primary interest and role are to provide affordable
housing and jobs, they are only plugging a few holes in a boat that has multiple holes, therefore
the boat will still sink unless collectively all holes are filled. To plug all the holes in the boat,
there needs to be a paradigm shift to a collective impact approach with multi-sectoral
partnerships and an understanding that social determinants of health do not exist independently.
There is also a need for social innovation from within the community and at the state and federal
levels to support this type of innovation to address the social determinants of health across
sectors. Nichollas, et. al (2015) defines social innovation as “a complex process of introducing
new products, processes or programs that profoundly change the basic routines, resource and
authority flows, or beliefs of the social system in which the innovation occurs”. This approach to
problem solving has resilience and broad impact (Nichollas, et al., 2015).
I propose an approach to addressing the social determinants of health through urban
policy that supports the notion “you can’t just do one thing, that everything is connected to
everything else” (Sterman, 1994). Only addressing affordable housing would not create or
provide jobs for people to maintain and sustain the housing but might also contribute to the
system of poverty and poor health outcomes. Providing only low-wage jobs does not provide an
income for a livable wage, and without access to technical and employment development
programs it is difficult to have a sustainable workforce in a community. Without providing small
business develop programs and access to capital, the community will not be able to attract and
sustain small businesses. Without access to healthy food and places to participate in physical
activity, one cannot meet the recommendations of eating healthy and exercising 150 minutes per
week.
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
106
This paradigm shift completely changes the trajectory of how we approach urban policy
and the players at the table. Meadows (2008) states that “the concept of feedback opens up the
idea that a system can cause its own behavior”. A collective impact approach is the best and only
approach to solving the complex problem of poverty in the historical context of urban
communities. This can benefit from design thinking to consider each community and innovative
solutions to address the specific social needs of the community. Tracy & Scott (2017) defines
design thinkers as persons who “look for work-arounds and improvise solutions and find ways to
incorporate those into the offerings they create". Design thinkers “consider the edges, the places
where “extreme” people live differently, think differently, and consume differently” (Tracy &
Scott, 2017). We need a trigger and the design activist at all levels (federal, state, local, and
community-based) to ensure social innovation in urban communities can be developed, funded,
implemented, evaluated, sustained, and scaled with the appropriate public-private partnerships.
Manzini (2014) describes the trigger as the person who starts the new social conversations. The
design activist is described as the person who proactively launches socially meaningful design
initiatives (Manzini, 2014). A key role as a design activist includes shaping dynamic social
conversations amongst developers, investors, political leaders, advocates, and community
members about what to do and how (Boal & Schultz, 2007).
In conclusion, the data gathered from systematically implementing the Urban Policy
SDoH Collective Impact Evaluation Framework across multiple federal agencies, would provide
insight on specific changes to improve the current urban policies and place-based programs,
while still using our current federal, state, and local programs, funding, and data collection
systems. In addition, by scaling the Promise Zone urban policy strategy of braiding resources,
using existing categorical funding from multiple federal agencies, Congress would be more
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
107
supportive because it will not increase funding, but rather direct funding to areas of highest need
(by funding programs based on need via significant bonus points). Lastly, by incorporating an
Urban Policy SDoH Shared Measurement System, federal agencies can use data for program
monitoring and improvement and identify best practices to implement and scale in other
communities with similar demographics and socio-cultural context.
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
108
References:
Banfield, E. C. (1974). The unheavenly city revisited. Little, Brown.
Barnes, W. R. (1990). Urban policies and urban impacts after Reagan. Urban Affairs
Quarterly, 25(4), 562-573.
Barnes, W. R. (2005). Beyond federal urban policy. Urban Affairs Review, 40(5), 575-589.
Barrow, C.J. (2002). Evaluating the Social Impacts of Environmental Change and the
Environmental Impacts of Social Change: An Introductory Review of Social Impact
Assessment, International Journal of Environmental Studies, 59:2, 185-195.
Bartels, C. P., Nicol, W. R., & Van Duijn, J. J. (1982). Estimating the impact of regional policy:
A review of applied research methods. Regional Science and Urban Economics, 12(1), 3-
41.
Burdge, R.J. (2003). Benefiting from the practice of social impact assessment, Impact
Assessment and Project Appraisal, 21:3, 225-229.
Busso, M., & Kline, P. (2008). Do local economic development programs work? Evidence from
the federal empowerment zone program.
Busso, M., Gregory, J., & Kline, P. (2013). Assessing the incidence and efficiency of a
prominent place based policy. American Economic Review, 103(2), 897-947.
Braveman, P., & Gottlieb, L. (2014). The social determinants of health: it's time to consider the
causes of the causes. Public health reports, 129(1_suppl2), 19-31.
Brewer, Garry D. 1974. “The Policy Sciences Emerge: To Nurture and Structure a Discipline.”
Policy Sciences 5(3): 239-244. Lindblom, Charles. 1959. “The Science of Muddling
Through.” Public Administration Review 19(2): 79-88.
Brown, Z. (2021). Social Determinants of Health Legislation: Opportunities for a New Future.
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
109
National Health Law Program. Retrieved from https://healthlaw.org/social-determinants-
of-health-legislation-opportunities-for-a-new-future/
Brudney, J. L., & England, R. E. (1982). Urban policy making and subjective service
evaluations: Are they compatible? Public Administration Review, 127-135.
Centers for Disease Control and Prevention. (2013). Step by step—evaluating violence
and injury prevention policies: Brief 1: Overview of policy evaluation.
Centers for Disease Control and Prevention. (2022). CODI Clinical and Community Data
Initiative. Retrieved from https://www.cdc.gov/obesity/initiatives/codi/community-and-
clinical-data-initiative.html
Centers for Disease Control and Prevention. (2022). Introducing the Gravity Project. Retrieved
from https://thegravityproject.net/
Clark, G.L. (1981). Law, the State, and the Spatial Integration of the United States. Environment
and Planning A: Economy and Space 13:10, pages 1197-1232.
Compson, M. (1993). The Revenue Reconciliation Act of 1993: Tax Provisions of Significance
to Farmers and Rural America. US Department of Agriculture, Economic Research
Service, Agriculture and Rural Economy Division.; The Revenue Reconciliation Act of
1993: Tax Provisions of Significance to - Michael Compson - Google Books
Cooper-McCann, P. (2014). A Review of “Collaborative Governance for Urban Revitalization:
Lessons from Empowerment Zones”: Michael J. Rich and Robert P. Stoker. (2014).
Ithaca, NY: Cornell University Press. 296 pages. Journal of the American Planning
Association, 80(3), 279–279.
Corona, C. (2016). Examining Promise Zones: Prioritizing Affordable Housing during
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
110
Revitalization. Harvard Journal of Hispanic Policy, 28, 44–62. Retrieved from
http://search.proquest.com/docview/1832211759/
Dawes, D. E. (2018). The future of health equity in America: addressing the legal and political
determinants of health. The Journal of Law, Medicine & Ethics, 46(4), 838-840.
Dawes, D. E. (2020). The political determinants of health. Johns Hopkins University Press.
Din, A. (2018). Intersecting Opportunity Zones with Vacant Business Addresses. Cityscape,
20(3), 277–279. Retrieved from http://search.proquest.com/docview/2174191237/
Dowall, David E., Marc Beyeler, and Chun-Cheung S. Wong. 1994. Evaluation of California’s
Enterprise Zone and Employment and Economic Incentive Programs. California Policy
Seminar, University of California. Berkeley.
Downs, M., Stoffle, R., Burdge, R. J., Charnley, S., Finsterbusch, K., Freudenburg, B., ... &
Kragh, B. C. (2003). Principles and guidelines for social impact assessment in the USA:
the interorganizational committee on principles and guidelines for social impact
assessment. Impact Assess. Project Apprais, 21(3), 231-250.
Ferber, T., & White, E. (2014). Making public policy collective impact friendly. Stanf Soc Innov
Rev.
Flanagan R. The Housing Act of 1954: The Sea Change in National Urban Policy. Urban affairs
review (Thousand Oaks, Calif). 1997;33(2):265-286.
Freedman, M., Khanna, S., & Neumark, D. (2021). The Impacts of Opportunity Zones on Zone
Residents (No. w28573). National Bureau of Economic Research.
GAO. (1982). Revitalizing Distressed Areas Through Enterprise Zones: Many Uncertainties
Exist. CED-82-78. (Washington, DC.: Jul 15, 1982).
GAO. (1988). Enterprise Zones: Lessons from the Maryland Experience. PEMD-89-2.
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
111
(Washington, DC.: Dec 15, 1988).
GAO. (1998). Community Development: Progress on Economic Development Activities Varies
Among the Empowerment Zones. RCED-99-29. (Washington, DC.: Nov 25, 1998).
GAO. (2004). Community Development: Federal Revitalization Programs Are Being
Implemented, but Data on the Use of Tax Benefits Are Limited. GAO-04-306.
(Washington, DC.: Mar 05, 2004).
GAO. (2006). Empowerment Zone and Enterprise Community Program: Improvements
Occurred in Communities, but the Effect of the Program Is Unclear. GAO-06-727.
(Washington, DC.: Sep 22, 2006).
GAO (2010), Revitalization Programs: Empowerment Zones, Enterprise Communities, and
Renewal Communities, GAO-10-464R (Washington, D.C.: Mar. 12, 2010).
GAO. (2020). Opportunity Zones: Improved Oversight Needed to Evaluate Tax Expenditure
Performance. GAO-21-30 (Washington, DC.: Oct 08, 2020).
GAO. (2021) Census Tract Designations, Investment Activities, and IRS Challenges
Ensuring Taxpayer Compliance. (Washington, DC.: Oct 07, 2021).
Glasgow RE, Eckstein ET, Elzarrad MK. Implementation science perspectives and opportunities
for HIV/AIDS research: integrating science, practice, and policy. J Acquir Immune Defic
Syndr. 2013;63 Suppl 1: S26-S31.
Glickman, N. (Ed.) (1980) The Urban Impacts of Federal Policies. Baltimore, MD: Johns
Hopkins University Press.
Greenberg, G., Miller, J., Mohr, L., & Vladeck, B. (1977). Developing Public Policy Theory:
Perspectives from Empirical Research. The American Political Science Review, 71(4),
1532-1543.
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
112
Greene, S., Theodos, B., Reynolds, K., Meixell, B., Fedorowicz, M. (2020). The Urban Institute.
Opportunity Zone Community Impact Assessment Tool User Guide.
Gómez, C. A., Kleinman, D. V., Pronk, N., Gordon, G. L. W., Ochiai, E., Blakey, C., ... &
Brewer, K. H. (2021). Practice Full Report: Addressing Health Equity and Social
Determinants of Health Through Healthy People 2030. Journal of Public Health
Management and Practice, 27(6), S249.
Hanleybrown, F., Kania, J., & Kramer, M. (2012). Channeling change: Making collective impact
work (pp. 56-78). FSG.
Hanson, R. (1983). Rethinking urban policy. Urban development in an advanced economy,
National Academy Press, 59.
Hebert, S., Vidal, A., Mills, G., James, F., & Gruenstein, D. (2001). Interim assessment of the
Empowerment Zones and Enterprise Communities (EZ/EC) Program: A progress
report. Office of Policy Development and Research. Retrieved from
https://www.huduser.gov/Publications/pdf/ezec_report.pdf
Hetzel, O. J. 1994. "Some Historical Lessons for Implementing the Clinton Administrations
Empowerment Zones and Enterprise Communities Program - Experiences from the
Model-Cities Program." Urban Lawyer, 26(1), pp. 63-81.
Hirsch, A. R. (2009). Making the second ghetto: Race and housing in Chicago 1940-1960.
University of Chicago Press.
Hogan, R. L. (2007). The historical development of program evaluation: Exploring past and
present. Online Journal for Workforce Education and Development, 2(4), 5.
Hood, C. M., Gennuso, K. P., Swain, G. R., & Catlin, B. B. (2016). County health rankings:
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
113
relationships between determinant factors and health outcomes. American journal of
preventive medicine, 50(2), 129-135.
Horsch, Karen. (1997). Indicators: Definition and Use in a Results-Based Accountability System.
Harvard Family Research Project. Retrieved from http://www.hfrp.org/publications-
resources/browse-our-publications/indicators-definitionand-use-in-a-results-based-
accountability-system
Houstoun Jr, L. O. (1979). The Carter Urban Policy a Year Later. The Antioch Review, 134-147.
Hyra, D. S. (2015). The Obama Administration’s Place-Based Initiatives: Why Not Include
Small Business Lending Components? Journal of Urban Affairs, 37:1, 66-69.
Isserman, A., & Merrifield, J. (1982). The use of control groups in evaluating regional economic
policy. Regional Science and Urban Economics, 12(1), 43–58.
James, J. L. (1972). Federalism and the model cities experiment. Publius, 2(1), 69-94.
Joint Economic Committee. (2018). The promise of opportunity zones: hearing before the Joint
Economic Committee, Congress of the United States, One Hundred Fifteenth Congress,
second session, May 17, 2018. Washington: U.S. Government Publishing Office.
Jones, E. R. (1987). Enterprise zones for the Black community - promise or product: A case
study. The Western Journal of Black Studies, 11(1), 1. Retrieved from
http://libproxy.usc.edu/login?url=https://search.proquest.com/docview/1311810755?acco
untid=14749
Kaplan, M. (1995). Urban policy: an uneven past, an uncertain future. Urban Affairs Quarterly,
30(5), 662-680.
Krupka, D. J., & Noonan, D. S. (2009). Empowerment Zones, neighborhood change and owner-
occupied housing. Regional Science and Urban Economics, 39(4), 386-396.
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
114
Lane, M. B., Ross, H., & Dale, A. P. (1997). Social impact research: integrating the technical,
political, and planning paradigms. Human organization, 302-310.
Lewis, L. B., Galloway-Gilliam, L., Flynn, G., Nomachi, J., Keener, L. C., & Sloane, D. C.
(2011). Transforming the urban food desert from the grassroots up: A model for
community change. Family and Community Health, S92-S101.
Levitan, S. A., & Miller, E. I. (1992). Enterprise Zones: A Promise Based on Rhetoric.
Occasional Paper 1992-1.
Logan, J.R. (1983). The disappearance of communities from urban policy. Urban Affairs Q. 19
(September): 75-90.
Kania, J. & Kramer, M. (2011). Collective impact.
Karakas, F. (2009). New paradigms in OD: Positivity, spirituality, and complexity.
Organizational Development Journal, 27: 11-26
Manzini, E. (2014). Making things happen: Social innovation and design. Design Issues, 30(1):
57-66.
Maor, M. (2014). Policy bubbles: Policy overreaction and positive feedback. Governance, 27(3),
469-487.
Meadows, D. H. (2008). Introduction: The Systems Lens, and Chapter 1 – The Basics, from
Thinking in Systems: A Primer.
Milstein, B., & Wetterhall, S. F. (1999). Framework for program evaluation in public health.
Mishori, R. (2019). The social determinants of health? Time to focus on the political
determinants of health. Medical care, 57(7), 491-493.
Morgan, D. R., & England, R. E. (1984). The small cities block grant program: an assessment of
programmatic change under state control. Public Administration Review, 477-482.
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
115
Mounts, R., & Analyst, P. (1981). Urban Enterprise Zones: Will They Work? The
American Review of Public Administration, 15(1), 86-96
Nelson A. Rockefeller Institute of Government. 1997. Investing in a New Future: Special Report
on Community Development Financing in Selected Empowerment Zone/Enterprise
Community Sites. Report prepared for the U.S. Department of Housing and Urban
Development. Albany, NY:
Oakley, D., & Tsao, H. (2007). Socioeconomic gains and spillover effects of geographically
targeted initiatives to combat economic distress: An examination of Chicago’s
Empowerment Zone. Cities, 24(1), 43–59.
Office of the National Coordination for Health Information Technology (ONC). (2022). United
States Core Data for Interoperability (USCDI) Interoperability Standards Advisory (ISA).
Available from: https://www.healthit.gov/isa/united-states-core-data-interoperability-
uscdi
Organization for Economic Co-operation and Development (OECD). (2017). The State of
National Urban Policy in the United States. Available from:
https://www.oecd.org/regional/regional-policy/national-urban-policy-United-States.pdf
Parkes, Rhae et al. (2012). Opportunity Chicago: 2006–2010. Improving Access to Employment
for Public Housing Residents in Chicago. Evaluation report. Chicago: Opportunity
Chicago.
Parsons R. (2019). Forces for change in social impact assessment, Impact Assessment and
Project Appraisal.
Preskill, H. (2014). Guide to evaluating collective impact: Assessing progress and impact. Aspen
Institute Forum for Community Solutions.
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
116
Preskill, H. (2014). Guide to evaluating collective impact: Learning and evaluation in the
collective impact context. Aspen Institute Forum for Community Solutions.
Preskill, H. (2014). Guide to Evaluating Collective Impact Supplement: Sample Questions,
Outcomes, and Indicators. Aspen Institute Forum for Community Solutions.
Preskill, H., Parkhurst, M., & Juster, J. S. (2001). Learning and evaluation in the collective
impact context. Guide to evaluating collective impact. Available from:
https://collectiveimpactforum. org/sites/default/files/Guide% 20to% 20Evaluating% 20C.
Reno, B. J. (2007). Rethinking horizon theory: Culture vs. nature. Perspectives on Political
Science, 36(2), 84-90.
Revenue Reconciliation Act of 1993, Title XIII of the Omnibus Reconciliation Act of 1993, Pub.
L. No. 103-66, 13301, et seq. (enacted August 6, 1993), adding to or amending various
sections of the Internal Revenue Code, 38(c)
Rich, M. J., & Stoker, R. P. (2010). Rethinking Empowerment: Evidence from Local
Empowerment Zone Programs. Urban Affairs Review, 45(6), 775–796.
Rich, M. J., & Stoker, R. P. (2014). Collaborative governance for urban revitalization. Cornell
University Press.
Robey, J. (1982). Urban Policy Analysis: Directions for Future Research. Edited By Terry
Nichols Clark. (Beverly Hills, Calif.: Sage Publications, 1981. Pp. i 296. $22.50, cloth;
$9.95, paper.). American Political Science Review, 76(4), 895-895.
Robinson, M. W. (1995). Empowerment zones and enterprise communities under the omnibus
budget and reconciliation act of 1993: A promising concept with some modifications. JL
& Pol., 11, 345.
Rubin, Marilyn M., Steven A. Brooks, and Richard W. Bubaum. 1992. Indiana Enterprise Zones:
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
117
A Program Evaluation for 1989 and 1990. Report prepared for the Indiana Dept. of
Commerce. October
Savaya, R., & Waysman, M. (2005). The logic model: A tool for incorporating theory in
development and evaluation of programs. Administration in Social Work, 29(2), 85-103.
Schechter, J. (2011). An empirical evaluation of the Model Cities program (Doctoral
dissertation).
Schill, M., & Wachter, S. M. (1994). Spatial Bias of Federal Housing Law and Policy:
Concentrated Poverty in Urban America. U. Pa. L. Rev., 143, 1285.
Scruggs, Y. (1995). HUD's stewardship of national urban policy: A retrospective
view. Cityscape, 33-68.
Seidman, K. F. (2004). Inner City Commercial Revitalization: A Literature Review. Fannie Mae
Foundation.
Sipe, T. A., Chin, H. B., Elder, R., Mercer, S. L., Chattopadhyay, S. K., Jacob, V., & Community
Preventive Services Task Force. (2012). Methods for conducting community guide
systematic reviews of evidence on effectiveness and economic efficiency of group-based
behavioral interventions to prevent adolescent pregnancy, human immunodeficiency
virus, and other sexually transmitted infections: Comprehensive risk reduction and
abstinence education. American Journal of Preventive Medicine, 42(3), 295-303.
Small Business Administration (SBA) Office of Advocacy. (2021). State Profiles. 2021 Small
Business Profiles for Congressional Districts. 2021 Small Business Profiles for
Congressional Districts – SBA's Office of Advocacy
Sterman, J. D. (1994). Learning in and about complex systems. System Dynamics Review, 10:
291-330.
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
118
Stoker, R. P., & Rich, M. J. (2020). Old policies and new presidents: Promise Zones and the
Trump administration. In Urban Affairs Forum.
The U.S. Impact Investing Alliance and the Beeck Center. (2019). The Opportunity Zone
Framework: Prioritizing and Achieving Impact in Opportunity Zones. Available from:
https://ozframework.org/
Tracey, P. and Stott, N. (2017). Social innovation: A window on alternative ways of organizing
and innovating. Innovation: Organization & Management, 19(1): 51-60.
The White House Office of the Press Secretary. (2016). Obama Administration Announces Final
Round of Promise Zone Designations to Expand Access to Opportunity in Urban, Rural
and Tribal Communities.
United States Census Bureau. (2020). American Community Survey: Data Profiles Available
from: https://www.census.gov/acs/www/data/data-tables-and-tools/data-profiles/
United States Department of Health and Human Services. (DHHS) (2020). Healthy People 2030.
Social Determinants of Health. Retrieved from www.healthypeople.gov/2020/topics-
objectives/topic/socialdeterminants-of-health
United States Department of Housing and Urban Development (HUD) (2019). Place-Based Tax
Incentives for Community Development. HUD’s Office of Policy Development and
Research periodical Evidence Matters, Spring/Summer 2019. Retrieved from
https://www.huduser.gov/portal/periodicals/em/SpringSummer19/highlight1.html
United States Department of Housing and Urban Development (HUD) (2020). HUD Exchange.
Promise Zones. Retrieved from https://www.hudexchange.info/programs/promise-zones/
United States Department of Housing and Urban Development (HUD) (2020). HUD Exchange.
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
119
Available from: https://www.hudexchange.info/programs/promise-zones/promise-zones-
designees-finalists-and-applicants/
United States Environmental Protection Agency. “Program Evaluation Glossary.” (November 1,
2007).
United States. President's Commission for a National Agenda for the Eighties. (1980). A
National Agenda for the Eighties: Report of the President's Commission for a National
Agenda for the Eighties. The Commission. A National Agenda for the Eighties: Report of
the President's Commission for - United States. President's Commission for a National
Agenda for the Eighties. Available from:
https://books.google.com/books?hl=en&lr=&id=3RcaAAAAIAAJ&oi=fnd&pg=PA1&d
q=President%E2%80%99s+Commission+on+a+National+Agenda+for+the+Eighties&ots
=gdp20zNX47&sig=3e4_D5yN29kfuYOqBx8YOYXbYKU#v=onepage&q=focus%20o
n%20people&f=false
Wang J. (2018) Urban Policy. In: Farazmand A. (eds) Global Encyclopedia of Public
Administration, Public Policy, and Governance. Springer, Cham. Available from:
https://doi.org/10.1007/978-3-319-20928-9_226
Warren, R. (1990). National Urban Policy and the Local State: Paradoxes of Meaning, Action,
and Consequences. Urban Affairs Quarterly, 25(4), 541–561. Available from:
https://doi.org/10.1177/004208169002500402
Wheatley, M. J. (1992). Preface, Introduction – Searching for a Simpler Way to Lead, and
Chapter 1 – Discovering an Orderly World, from Leadership and the New Science:
Learning about Organization from an Orderly Universe
White House Executive Order 13853 (2018). Executive Order on Establishing the White House
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
120
Opportunity and Revitalization Council. Available from:
https://www.whitehouse.gov/presidential-actions/executive-order-establishing-white-
house-opportunity-revitalization-council/
White House Opportunity and Revitalization Council (2019). Implementation Plan for The
White House Opportunity and Revitalization Council. Available from:
https://www.hud.gov/sites/dfiles/Main/documents/WHORC-Implementation-Plan.pdf
White House Office of Management and Budget. (2013). Uniform Administrative Requirements,
Cost Principles, and Audit Requirements for Federal Awards. Washington D.C.
Wilder, Margaret G., and Barry M. Rubin 1988. Targeted Redevelopment Through Urban
Enterprise Zones. Journal of Urban Affairs 10, l: 1-17.
Wilder, M. G., & Rubin, B. M. (1996). Rhetoric versus reality: A review of studies on state
enterprise zone programs. Journal of the American Planning Association, 62(4), 473-491.
Williamson, R. S. (1982). Community Development Block Grants. Urb. Law., 14, 283.
Wolman, H. (1986). The Reagan urban policy and its impacts. Urban Affairs Quarterly, 21(3),
311-335.
World Economic Forum. (2020). How to effect real social impact in 2020. Cape Town:
SyndiGate Media Inc. Available from: http://libproxy.usc.edu/login?url=https://search-
proquest-com.libproxy2.usc.edu/docview/2350487312?accountid=14749
Zapolsky, S., Ault, M., Brock, J., Rekhi, J., & Sweeney, D. (2019). Promise Zones: Initial
Implementation Assessment Report February 2019. US Department of Housing and
Urban Development Office of Policy Development and Research.
Zhang, S. (2019). Rethinking US enterprise zones: The role of research design in program
evaluation. Local Economy, 34(6), 545-571.
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
121
Appendix A. Urban Policy SDoH Logic Model
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
122
References: Scruggs, 1995; Schill and Wachter, 1995; Flanagan, 1997; Hirsch, 1983
Housing Act of 1937, 1949, 1954
Presidents: Franklin D. Roosevelt (1933-1941); Harry S. Truman (1941-1945; Dwight D. Eisenhower
(1953-1961)
INPUTS:
Act/Policies/
Program/ Funding
allocated
ACTIVITIES:
Mutually
Reinforcing
Strategies/Goals/A
im/ Objectives
OUTPUTS:
CI Capacity
OUTCOMES:
CI Systems,
Policies, Funding,
Behavior
IMPACT:
- Urban
Renewal
Program
- Housing Act
of 1937
- Housing Act
of 1949
- Housing act of
1954
- Over 54
additional
pieces of
legislation,
Executive
orders, and
Government
committee
reports
augmented,
changed,
and/or refined
renewal policy
Strategies:
- Slum
clearance
- Urban
redevelopment
- Residential
renewal
- Downtown
central
business
district
revitalization
Criteria:
- No African
Americans
were
permitted to
buy federal
government
subsidized
homes in the
suburbs
- FHA often
added an
additional
condition
requiring that
every deed in
a home in
those
subdivisions
prohibit resale
to Black
people
- “Equivalent
elimination
requirement” -
mandated
substandard
housing be
eliminated for
each unit of
public housing
constructed
- The Housing
Act of 1949:
approved
construction
for 135,000
public housing
units per year
for six years.
- The Housing
Act of 1954:
the number of
public housing
constructed
decrease from
52,000 per year
in 1952 to
24,000 per year
in 1964
- The number of
urban renewal
projects
increased from
260 projects in
1953 to 1,210
in 1962
- Poverty
increased in
inner cities
- Income rose
nationally
- Relative
quality of
life declined
for the poor
and for
minorities
- 12 largest
cities saw a
decrease in
white
households
and an
increase in
minority
households
- Created the
largest
wealth gap
in America,
that we still
have today
Black
people
displaced
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
123
References: Williamson, 1982; Conlan, 1984; Scruggs, 1995
Model Cities Program – The War of Poverty
President: Lyndon B. Johnson (1963-1969)
INPUTS:
Act/Policies/
Program/ Funding
allocated
ACTIVITIES:
Mutually Reinforcing
Strategies/Goals/Aim
/ Objectives
OUTPUTS:
CI Capacity
OUTCOMES:
CI Systems, Policies,
Funding, Behavior
IMPACT:
- Economic
Opportunity
Act of 1964
(The War on
Poverty) - $500
million
- The
Demonstrations
Cities and
Metropolitan
Development
Act of 1966 –
$924 million
- 5 years
- 400 Categorical
Programs or
programs for
every category
of urban and
rural problem
- Enactment of
Medicaid and
Medicare
- Medicaid and
Medicare,
HUD housing
programs, Head
Start, Job
Corps,
Highway
Safety Act,
Water Quality
and Clean Air
Acts
- Created the
Task Force on
Urban
Problems
- Lead Agency:
HUD
Aims:
- Expand housing,
job, and income
opportunities
- Reduce
dependence on
welfare payments
- Improve
educational
facilities and
programs
- Combat disease
and ill health
- Reduce the
incidence of
crime and
delinquency
- Enhance
recreational and
cultural
opportunities
- Establish better
access between
houses and jobs
- Improve living
conditions
Objectives:
- Improve the
concentration of
available
resources in
high-need areas
- Coordination
social service
efforts across
agencies and
levels of
government
- Mobilization of
citizens and local
political leaders
in the planning
process
- 150
Model
Cities
selected
Selection
Criteria:
less than
15,000
people or
ten percent
of the
population,
whichever
was larger
- Increase in the
number of blue-
collar and middle
class black and
Hispanics leaving
inner-city
communities,
leaving many
disadvantaged
minorities in
deteriorating
neighborhoods
- Expanded the
welfare program
- Expanded social
security programs
- New Medicare and
Medicaid
programs created
- Added a large
housing package
and civil rights
legislation to
reduce housing
and employee
discrimination
- Model cities or as
a tract in a model
neighborhood had
worse outcomes in
the 12 variables
intended to
measure success in
achieving program
- Between
1970 and
2000, model
cities did not
see greater
improvement
, compared to
matched
cities, along
any of the
twelve
variables
analyzed
- When
comparing
changes in
poverty rates,
model cities
fared worse
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
124
References: Williamson, 1982; Conlan, 1984; Scruggs, 1995
Community Development Block Grants
President: Richard M. Nixon (1969-1974)
INPUTS:
Act/Policies/
Program/ Funding
allocated
ACTIVITIES:
Mutually
Reinforcing
Strategies/Goals/A
im/ Objectives
OUTPUTS:
CI Capacity
OUTCOMES:
CI Systems,
Policies, Funding,
Behavior
IMPACT:
- Title I of The
Housing and
Community
Development
Act of 1974
- Community
Development
Block Grants
(CBDGs) -
$14 billion
over 3 years
- Comprehensiv
e Employment
and Training
Act (CETA)
- Lead Agency:
Department of
Community
Development
Created: Urban
Affairs Council
that was replaced
with a new
Department of
Community
Development
- 80% of funds
- Entitlement
cities, cities
with
populations
over 50,000
and urban
counties with
populations
over 200,000,
directly
receive in the
form of
entitlement
grants without
having to go
through state
governments
- 20 % of funds
Discretionary
funds
available to
those units of
government
that did not
automatically
qualify for
entitlements,
namely, non-
metropolitan
and rural
areas; small
communities
had to
compete for
these funds
- No reporting
required
- From 1970-
1975, the
U.S.
population
changed
from urban
to suburban
- More blacks
were in
cities
- Unemploym
ent
increased in
cities
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
125
A New Partnership to Conserve America’s Communities: A National Urban Policy
President: Jimmy Carter (1977-1981)
INPUTS:
Act/Policies/
Program/ Funding
allocated
ACTIVITIES:
Mutually
Reinforcing
Strategies/Goals/A
im/ Objectives
OUTPUTS:
CI Capacity
OUTCOMES:
CI Systems,
Policies,
Funding,
Behavior
IMPACT:
- HUDs Urban
Development
Action Grant
and Loan - $2-
3 billion budget
submitted, but
program –
never reached 1
billion annually
- Urban Parks
Program - 5
years/ $150
million-dollar
- Anti-Recession
Fiscal
Assistance Act
1977 – counter
cycle revenue
sharing
program
- Public Works
Employment
Act of 1976 –
local public
works program
- Expansion of
CETA Title II
and VI
- Executive
Order to
establish the
Urban and
Regional
Policy Group
(URPG)
- Expansion of
the Economic
Development
Administration
(EDA) as an
urban agency
- Reauthorizatio
n of the
Community
Strategies:
- economic
development
- job creation
- revitalization
of distressed
neighborhoods
- sprawl and
growth
- improvements
of services in
distressed
urban areas
- increased
mobility for
poor people
- address
environmental
concerns
- deteriorating
housing stock
and physical
plan
Selection Criteria:
- Focused
primarily on
central cities and
their poorest
residents
- Jobs and
economic
development
- Partnerships-
shared federal-
state-local
responsibilities
- Participation of
private
businesses,
neighborhood
groups, and
others with
emphasis on
conservation
- Required the
coordination of
programs, impact
analyses, and
targeting of
procurement and
facility siting.
- Requirement to
assess existing
programs to
review the
impacts of their
programs for
negative effects
on cities
- No new
CDBGs
- Increase in
federal aid
to cities
because of
the Carter
administrati
on’s fiscal
stabilization
policy
- No impact
reported
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
126
References: Houstoun, 1979; Barnes, 2005; Kaplan, 1995
Development
Block Grant
(CDBG)
- Antirecession
package that
put several
billion new
federal dollars
into municipal
coffers
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
127
Community Development Block Grants (CDBGs)
President: Ronald Reagan (1981-1989)
INPUTS:
Act/Policies/
Program/ Funding
allocated
ACTIVITIES:
Mutually
Reinforcing
Strategies/Goals/A
im/ Objectives
OUTPUTS:
CI Capacity
OUTCOMES:
CI Systems,
Policies, Funding,
Behavior
IMPACT:
- Omnibus
Budget
Reconciliation
Act of 1981
- The Small
Cities CDBG -
$1.1 billion
(FY 1981 and
FY 1982)
Lead Agency:
National League
of Cities Institute
Objectives of
funds:
- Benefit low-
and moderate-
income
families
- Eradicate or
prevent slums
or blight
- Address
urgent needs
affecting the
health and
welfare of the
community for
which no
other
resources are
available
- States were
required to
submit annual
reports
detailing the
use of funds
and how they
met the stated
objectives.
Economic
Outcomes from
1980-1982:
- Real median
income (prior
to taxes)
decreased by
4.9% vs.
nationally 1.6%
- real median
income
decreased by
5.0% in
metropolitan
areas and 5.4%
inside central
cities of
metropolitan
areas
- Central Cities:
unemploymen
t rate increased
from 10.6% to
13.8% (2.7%
points)
- Central Cities:
the number of
individuals
with income
below the
poverty level
rose from
14.5% to 17.2
% (2.7%
points)
Housing Outcomes
(HUD):
- Elimination of
all new low-
income
construction
program
(1990);
- Total of
66.3%
reduction in
Urban
programs
- Intergovern
mental aid
to cities
decreased
from 14
billion
dollars in
1980 to 9
billion
dollars in
1986
- Cumulative
reductions
in CDBGs
(11%)
- UDAG
(33.6%)
- Mass transit
(30%)
- CETA
(100%)
- Wastewater
treatment
(32.2%)
- EDA
(67.5%)
- Compensato
ry Education
(21.9%)
- Social
Services
Block Grant
(21.3%)
- Community
Service
Block
Grants
(40.1%)
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
128
References: Williamson, 1982; Conlan, 1984; Kaplan, 1995
compared to
125,000 (1980)
- Net additions
to rental
assistance/vouc
her rolls
averaged fewer
than 84,000
households per
year between
fiscal years
1979-1990
- Subsidized
Housing
(42.8%)
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
129
Enterprise Zones (Experimental) and Enterprise Zone (State Legislation)
President: Ronald Reagan (1981-1989)
INPUTS:
Act/Policies/
Program/ Funding
allocated
ACTIVITIES:
Mutually
Reinforcing
Strategies/Goals/A
im/ Objectives
OUTPUTS:
CI Capacity
OUTCOMES:
CI Systems,
Policies, Funding,
Behavior
IMPACT:
- 1982 Report
to Congress –
Experimental
Basis of
Enterprise
Zones
- Urban Jobs
and Enterprise
Act in 1984
Objectives:
- To attract
businesses and
jobs to
enterprise
zones through
Federal tax,
regulatory,
and other
governmental
incentives
- Stimulate
business
development
and create
jobs
- Included tax
incentives
designed to
encourage (1)
the hiring of
disadvantaged
workers, (2)
the creation of
jobs for
workers of all
types, and (3)
workers to
accept
employment
in an
enterprise
zone
- States required
to provide
Economic
development
incentives:
(i.e.,
development
requirements,
building
standards,
lower taxes,
- 37 states and
the District of
Columbia
enacted
enterprise
zone
legislation
- 1987 Congress
authorized 100
EZs
- Established
the Urban
Development
Action Grant
(UDAG)
Program
Selection criteria:
- Located in a
UDAG-
eligible area,
- Contiguous
- In
metropolitan
areas that have
at least 4,000
inhabitants
- Outside of
SMSAs the
minimum
population
requirement is
2,500
- Must qualify
as a low-
income area
based on at
least one of
several
optional
criteria
measuring
income,
unemploymen
- 1983 (n = 9
states with EZ
legislation):
Reported
21,271 jobs in
place/progress;
450 million in
new
construction/ex
pansion/renova
tion
- 1984 (n = 23
states with
programs):
- 37,00 new jobs
created or
retained; 358
new businesses
started
operations; 72
firms
expanded, and
578 firms were
retained
- 1985 (n =
1,300 zones;
600
jurisdictions):
75,000 jobs
created or
retained; 2.5
billion in
capital
investments
retained in EZs
1985 (n = 26
states): 84,000
jobs created
and retained;
3.19 billion
dollars invested
in the zones
- Tax
incentives
alone are
unlikely to
be sufficient
motivators
to attract
businesses
to EZs
unless a
municipal or
county
government
can ensure
other factors
are
addressed
(i.e., crime
rate,
transportatio
n, and utility
access,
building
security,
lighting,
code
enforcement
, and
adequacy of
the labor
force)
- A review of
the
experiences
of five
states–
Michigan,
Virginia,
Maryland,
Indiana, and
New Jersey
concluded
that states
were ill
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
130
References: GAO, 1982; Mounts, 1981; Levitan, 1992; Mounts, 1981
and fees in the
zone)
t or population
lost
- Must be a city,
or a state with
the approval
of the city or
town where
the zone will
be located
- Must include a
formal
expression of
commitment
(i.e., lower
taxes and fees
in the zone,
development
requirements,
or building
standards) to
the zone in the
form of its
own
contributions
for economic
development
incentives
equipped to
revitalize
blighted
areas alone
- GAO (1982)
determined
that there
was
unknown
the
effectivenes
s of the
proposed
incentives
and their
cost
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
131
Enterprise Zones
President: George Bush (1989-1993)
INPUTS:
Act/Policies/
Program/ Funding
allocated
ACTIVITIES:
Mutually
Reinforcing
Strategies/Goals/A
im/ Objectives
OUTPUTS:
CI Capacity
OUTCOMES:
CI Systems,
Policies, Funding,
Behavior
IMPACT
- Urban Jobs
and Enterprise
Act in 1984
- Title VII of
Public Law
100-242, the
Housing and
Community
Development
Act of 1987
Aim:
- An approach
to reduce
Black
unemploymen
t and
encourage
growth and
retention of
Black
businesses
Strategies:
- State tax
incentives, no
federal tax
incentives
- Offered
incentives
(such as
waivers on
some HUD
regulations
that otherwise
would apply)
to businesses
that were
located in the
enterprise
zone
designated
areas
- Federal
funding for
programs:
- to weed out
criminals from
inner cities
- reform welfare
and improve
job training
- job training
programs
- The GAO
(1988) defined
Enterprise
Zones are
economically
distressed
areas
designated for
preferential
governmental
treatment to
promote
investment
and job
creation by
private
industry
- Development
and
investments
increased, but
may have been
the result of
other factors
- GAO (1988)
found in the
Maryland
Experience
that
enterprise
zones were
not
effective.
- Effects of
the policies
and
programs by
both Reagan
and Bush
administrati
ons
benefited
higher
incomes and
resulted in
suburbanizat
ion
- Direct aid to
cities
decreased
by almost
one-third
- Federal aid
to states and
noncity
localities,
such as
counties,
suburbs, and
rural
districts
increased
from $69
billion to
over $114
billion
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
132
References: Jones, 1987; GAO, 1988; Barnes, 1990; Hendrickson, 2004
- funding to
increase police
patrols
- funding for
street
improvements
- rental
initiatives
- community
service
requirements
for public
housing
residents
- drug
rehabilitation
programs
- marriage
counseling
- work
programs
- mentoring
services
- prison release
programs
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
133
Empowerment Zones, Enterprise Communities, Rural Development Investment Areas
President: Bill Clinton (1993-1997)
INPUTS:
Act/Policies/
Program/ Funding
allocated
ACTIVITIES:
Mutually
Reinforcing
Strategies/Goals/A
im/ Objectives
OUTPUTS:
CI Capacity
OUTCOMES:
CI Systems,
Policies, Funding,
Behavior
IMPACT:
- Title XIII,
Chapter I,
Subchapter C,
Part I of the
Omnibus
Budget
Reconciliation
Act of 1993:
- Round I:
Omnibus
Budget
Reconciliation
Act 1993
- Round II:
Taxpayer
Relief Act of
1997;
Omnibus
Consolidated
and
Emergency
Supplemental
Appropriation
s Act of 1999
- Round III:
Community
Renewal Tax
Relief Act of
2000;
American Jobs
Creation Act
of 2004
- Federal
Assistance:
- Title XX
Social
Services
Block Grant
(SSBG) Funds
- Designation:
10 years
- Lead
Agencies:
Economic
Development
Strategies:
- the creation of
new
institutions or
mechanisms to
provide
commercial
capital
- the creation of
new or
demonstrably
tailored
lending
programs
provided
through
existing
lenders,
private and
public
- the
establishment
of "one-stop
capital shops"
or other,
related means
by which
capital and
technical
assistance
were to be
pulled
together and
made more
accessible
- mixed-use or
flagship
projects
the creation of
incubators or
industrial
parks
- 600 state
authorized
EZs
- 95 Enterprise
Communities
Selection Criteria:
- Jointly
nominated by
the local and
state
governments
in which they
are located
- Urban zones
must be in
Metropolitan
Statistical
Areas (MSAs)
and between
200,000 and
50,000 in
population, or
ten percent of
the population
of the most
populous city
in the
nominating
area; less than
twenty square
miles
- Rural zones
must be
outside MSAs
and with
populations of
no more than
30,000; rural
are to be less
than 1,000
square miles
of the six
urban zones,
- 18 of the
EZ/EC
locations that
were initially
supported
showed
significant
increases in
economic
growth and job
creation can be
seen
empowerment
zone programs
increased
employment
and generated
wage increases
for residents of
empowerment
zones
- Neighborho
ods
receiving
EZ
designation
experienced
substantial
improvemen
ts in labor
market
conditions
and
moderate
increases in
rents
relative to
rejected and
future zones
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
134
References: Hetzel, 1994; Nelson A. Rockefeller Institute of Government, 1997; Herbert, 2001; Cooper-
McCann, 2014; Busso and Kline, 2008; Rich and Stoker, 2014
USDA &
HUD
- Empowerment
Zone
Employment
Credits
- Increased
Section 179
Expensing
- Tax-Exempt
Bond
Financing
- Employment
Tax Credits
- Title XX
Social
Services
Block Grant
(SSBG) Funds
- $100 million
- Economic
Development
Initiative
(EDI) Grants
- Enterprise
Zone Facility
Bonds
- Increased
Section 179
Expensing
- Regulatory
Waivers/
Priority in
Other Federal
Programs
Housing
Strategies:
- encouraging
home
ownership
- improving the
housing stock
through
rehabilitation
and new
construction
of owner-
occupied and
rental housing,
- code
enforcement
strengthening
producers of
housing such
as
intermediary
organizations;
and improving
public housing
at least one
must be in a
city of less
than 500,000
and another is
to have
population of
50,000 or less
drawn from
two
contiguous
states
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
135
Promise Zones
President: Barack Obama (2009-2017)
INPUTS:
Act/Policies/
Program/ Funding
allocated
ACTIVITIES:
Mutually
Reinforcing
Strategies/Goals/A
im/ Objectives
OUTPUTS:
CI Capacity
OUTCOMES:
CI Systems,
Policies, Funding,
Behavior
IMPACT:
- Choice
Neighborhood
Grants
- Partners for
Sustainable
Communities
Grants
- Byrne
Criminal
Justice
Innovation
Grant
Programs
- Neighborhood
Revitalization
Initiative
(NRI) in 2010
- Agencies: The
White House
Domestic
Policy Council
(DPC), White
House Office
of Urban
Affairs
(WHOUA),
HUD, ED,
DOJ, HHS),
(DOTR)—
- Lead agencies:
HUD the
federal lead
for the 14
urban
designees
- USDA the
lead federal
partner to the
tribal and rural
Promise Zones
Strategy Domains:
- Employment
and asset
building
- Investment
and business
growth
- Education
- Public safety
- Housing
- Health
- Community
infrastructure
- Civic
engagement
- Braiding
resources:
multiple
sources of
financial
support
(private,
public and
philanthropies
) and
coordinating
the use of
resources
across
multiple
policy
domains to
create a
synergistic
local initiative
(a collective
impact),
implementing
initiatives at
the same and
same place.
- Assigned a
federal staff to
help navigate
Selection Criteria:
- 1
st
round:
limited to
communities
with a Choice
Neighborhood
s or Promise
Neighborhood
s
Implementatio
n grant, or a
Byrne
Criminal
Justice
Innovation
grant; rural
and tribal
eligibility was
limited to
communities
with a
Stronger
Economies
Together,
Sustainable
Communities,
Promise
Neighborhood
s
Implementatio
n, or Rural
Jobs
Accelerator
grant
- 2
nd
round had
to meet the
previous
round’s
specified
eligibility
criteria
without regard
to their prior
selection for
- 1
st
round: 5
Promise Zone
communities
were
designated
- 2
nd
round: 6
selected from
123
applications
submitted: 97
urban
applicants, 19
rural
applicants, and
7 tribal
applicants
- 3
rd
round: 9
selected from
82 applications
from 38 states
and Puerto
Rico
- number of PZ
cities winning
federal grants:
(fiscal years
2017 ad 2018
- number of
grants provided
by Promise
Neighborhood
- Choice
Neighborhood
- Byrne Criminal
Justice
Innovation
grant programs
- 3 out of 14
Promise Zone
cities received
support from
one of the core
NRI programs
(Byrne
- Over three
years, 22
Promise
Zones were
selected, 13
Promise
Zones
invested in
strategies
with proven
effectivenes
s by
leveraging
millions in
additional
funding,
creating new
jobs,
training for
teachers and
parents,
creating
social
enterprise,
providing
workforce
training,
reducing
violent
crime, and
increasing
graduation
rates
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
136
References: Corona, 2016; Hyra, 2015; HUD, 2020; White House, 2016
the array of
Federal
assistance and
programs
Received any
available (a)
preference for
certain
competitive
Federal programs
and (b) technical
assistance.
receipt of
federal grants
3
rd
round had to
meet the previous
round’s specified
eligibility criteria
and 13 additional
questions specific
to implementation,
communication,
stakeholder
engagement, data
collection,
research, and
evaluation to the
application
process
Criminal
Justice
Innovation
grants) to
implement
projects
(Indianapolis,
Los Angeles,
and St. Louis)
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
137
Opportunity Zones
President: Donald J. Trump (2017-2021)
INPUTS:
Act/Policies/
Program/
Funding allocated
ACTIVITIES:
Mutually
Reinforcing
Strategies/Goals/Ai
m/ Objectives
OUTPUTS:
CI Capacity
OUTCOMES:
CI Systems,
Policies,
Funding,
Behavior
IMPACT:
- The Tax and
Jobs Act
- White House
Opportunity
and
Revitalizatio
n Council
- The U.S.
Impact
Investing
Alliance, the
Beeck Center
for Social
Impact +
Innovation at
Georgetown
University
- The Federal
Reserve
Bank of New
York
- Opportunity
Zone
Framework
Economic
Development
- Leverage federal
grants and loans
in a more
integrated way
to develop
dilapidated
properties and
provide basic
infrastructure
and financial
tools to attract
private
investment
Entrepreneurship
- Leverage
government
lending and
grants to
stimulate access
to private capital
and promote
programs that
assist
entrepreneurs.
Safe
Neighborhoods
- Combat drug
addiction and
the opioid crisis,
reduce
crime/enhance
public safety,
and address
environmental
contamination
obstacles to
development.
Education and
Workforce
Development
- Improve the
efficacy of K-12
- 8,764 census
tracts—8,566
LIC tracts and
198 contiguous
tracts
- Populations in
selected tracts are
Black (21%) or
Hispanic (33%)
compared to all
tracts (12% Black
and 19%
Hispanic) and
tracts that were
eligible, but not
selected (16%
Black and 23%
Hispanic)
- Population was
foreign born
(10%) and have
lower educational
attainment (53%
high school
education or less)
- OZ the
designated census
tracts have lower
incomes (average
median family
income in a
designated tract is
just over $38,000
compared to
$66,000 across
all tracks), higher
poverty (25%
compared to 13%
nationally), and
higher
unemployment.
- The poverty in
the designated
tracts averages
- There are
insufficient
data
available to
evaluate
OZ
performanc
e,
including
outcomes
- limited
evidence of
any
impacts of
zone
investment
to date on
zone
residents
- positive
impacts on
employme
nt growth
in the
zones
- little effect
on job
postings or
salaries
- no impact
on
investment
- no impact
on
residential
property
prices
- positive
impacts on
some
commercia
l property
values
positive
- 76% of OZ
census tracks
were in urban
communities,
with the
potential to
impact over
26,358,483
people, 61% of
the population in
the selected
census tracks are
non-white
- Out of 18 case
studies of
Qualified
Opportunity
Funds, their
projects
principally
focused on real
estate
development
- Including:
commercial real
estate
development,
multifamily
housing
development,
agricultural land
development,
renewable
energy
businesses, and
hotel
development.”
- OZ is a
potentially
costly tax
expenditure with
few limits on
types of
projects, but as
URBAN POLICY AND SDOH COLLECTIVE IMPACT EVALUATION FRAMEWORK
138
Reference: White House, 2018; GAO, 2020; GAO, 2021; Freedman, 2021; Arefeva et al., 2021; Atkins et
al., 2021; Corinth and Feldman, 2021; Chen et al., 2020; Sage et al., 2019); Frank et al., 2020
and community
college career
and technical
education and
workforce
development
programs to
better prepare
workers in
distressed
communities for
jobs
Measurement
- Develop robust
reporting and
analytics to
measure impact
of Opportunity
Zones.
nearly 25 percent
while the national
average is just
over 13 percent
Selection Criteria:
- Tracts were
eligible for
designation based
principally on
income and
poverty rates
- Census tracts
needed to be low-
income
communities
(LIC) or non-LIC
contiguous tracts.
- Poverty rate at
least 20 percent
- Tracts in which
the median
family income
does not exceed
80 percent of
statewide median
family income if
located outside a
metropolitan area
if located within
a metropolitan
area, tracts in
which the median
family income
does not exceed
80 percent of the
statewide median
family income or
the metropolitan
area median
family income,
whichever is
higher
impacts on
commercia
l real estate
activity
it is currently
being managed
- Limited
evidence that
Opportunity
Zone
designation has
positive effects
on the economic
conditions of
local residents.
Abstract (if available)
Abstract
Urban Policy impacts the day-to-day lives of urban communities, and no research has been done to evaluate the impact on the SDoH of people who live in the communities targeted by urban policy. Over time, little policy evaluation has been done systematically to evaluate the social impact of urban policy on communities collectively to reduce poverty and improve the economic, social, housing conditions, and/or health in urban communities. The most recent urban policy, the Tax and Jobs Act signed into law created over 8,700 “qualified opportunity zones (OZ)”. In 2019, HUD released a report providing recommendation for future Opportunity Zone (OZ) evaluations based off lessons learned from previous place-based incentive program evaluations (i.e., Enterprise Zones, Empowerment Zones, and Enterprise Communities). Two major tasks identified from this HUD (2019) report were: (1) identify a comparison set of census tracks; and (2) identify data that can be used to track neighborhood change.
A GAO (2020) report highlights that Congress did not designate an agency with the responsibility and authority to collect data, evaluate, and report on OZ performance and concluded that “there are insufficient data available to evaluate OZ performance”. The GAO recommended Congress: (1) “consider providing Treasury with authority and responsibility to collect data and report on OZ’s performance, in collaboration with other agencies and (2) consider identifying questions about OZ’s effects that it wants Treasury to address in order to help guide data collection and reporting of performance, including outcomes” (GAO, 2020). In 2021, Congress requested the GAO conduct an evaluation of the OZ policy, due to the magnitude (impacting over 9,000 census tracts) and the extended time for the OZ tax benefit (tax benefits for this policy do not expire until 2047).
Also in 2020, Congress acknowledged and supported public health’s role in “working across sectors on social determinants of health” and passed the Social Determinants Accelerator Act of 2019. There was continued support for this bill in 2021 with the reintroduction of the Improving Social Determinants of Health Act of 2021 (H.R.379), and a final approved FY2022 appropriation bill with an increase to $5 million to address SDoH. There were two primary goals proposed in this bill: “1.) improve health outcomes and reduce health inequities through the coordination of SDOH activities across the CDC; and 2.) improve public health agencies’ and community organizations’ capacity to address SDOHs in communities” (Brown, 2021).
The purpose of this study is to develop an evaluation framework and core set of SDoH measures to assess the impact of urban policy at the local level on the social determinants of health (SDoH) in urban communities. The Collective Impact Evaluation Framework was used for the study design. A Collective Impact Evaluation Framework is used when the desire is to achieve large-scale, sustainable change (Preskill, Parkhurst, & Juster, 2014). It supports the complex implementation of urban policy and allows for a mixed methods research design to assess the implementation, effectiveness, and impact of the policy. A literature review and environmental scan of past urban policies provided insight on what key evaluation measures and outcomes should be used to evaluate the impact of urban policy on SDoH. A qualitative analysis of guidance documents, evaluation reports, administrative documents, and government reports was also conducted to identify key measures, partnerships, programs, activities, and policies that were employed to support implementation and evaluation of past urban policy in targeted communities. SDoH measures identified were then used to develop an evaluation framework that can be used to assess if SDoH improve for the people in communities that have been targeted by urban policies over time.
The Urban Policy SDoH Collective Impact Evaluation Framework includes short-term and intermediate changes in systems and policies, and long-term changes in behavioral outcomes in targeted communities. Impacts evaluated include long-term changes in key SDoH indicators in targeted communities. The Urban Policy SDoH Evaluation Framework can be used to assess the impact of urban policy on SDoH at the local level and allow for the study of two timely topics (urban policy and measuring SDoH). In addition, the White House Opportunity Zone Implementation Plan has identified the need to develop robust reporting and analytics to measure the impact of the urban policy in designated Opportunity Zones (White House Opportunity and Revitalization Council, 2019). The results of this study can fill the need to develop shared measurement system to assess the social determinants of health of urban policy, provide an evaluation framework, social determinants of health measures, and data sources to measure the impact of the urban policy. The Urban Policy SDoH Evaluation Framework can be applied at the federal and local level by providing an evaluation framework, social determinants of health measures, and data sources to measure the impact of the urban policy on the SDoH on people who live in communities targeted by urban policy.
Linked assets
University of Southern California Dissertations and Theses
Conceptually similar
PDF
Health impact assessment, the concept, science, and application in China
PDF
Social Determinants of Health: working with social workers and social work managers to build capacity to screen and refer in the medical setting
PDF
Global health diplomacy: a new era of health in U.S. foreign policy
PDF
Exploring the social determinants of health in a population with similar access to healthcare: experiences from United States active-duty army wives
PDF
Urban universities' campus expansion projects in the 21st century: a case study of the University of Southern Calfornia's "Village at USC" project and its potential economic and social impacts on...
PDF
Social capital and community philanthropy: the impact of social trust and social networks on individual charitable behavior and community foundation development
PDF
Environmental justice in real estate, public services, and policy
PDF
Outcomes-based contracting through impact bonds: ties to social innovation, systems change, and international development
PDF
Critical factors in evaluating compliance to United Nations Security Council Resolution 1540: developing a methodology for compliance evaluation
PDF
Collective Impact: a framework to advance health promotion in higher education
PDF
Healthy mobility: untangling the relationships between the built environment, travel behavior, and environmental health
PDF
On planning, place, and race: how trauma imaginaries reveal communal trauma and impact collective well-being
PDF
Advances and applications for economic evaluation methods in health technology assessment (HTA)
PDF
Avoiding middle-class planning 2.0: media arts and the future of urban planning
PDF
The role of healthcare volunteers in addressing social determinants of health
PDF
Supporting a high value maternity system of care: prioritizing resilience of and relationships with mothers to improve maternal and child health
PDF
Evaluating art for social change: the Anti-Eviction Mapping Project and the Los Angeles Poverty Department in relationship to Tania Bruguera’s concept of Arte Útil; Evaluating art for social cha...
PDF
Social determinants of physiological health and mortality in China
PDF
Institute on social practice integration research and education (INSPIRE): a workforce development solution to close the health gap
PDF
Teaching academic and behavioral skills through arts-based programming: an evaluation study
Asset Metadata
Creator
Grant-Rutledge, Gia Elizabeth
(author)
Core Title
A framework for evaluating urban policy and its impact on social determinants of health (SDoH)
School
School of Policy, Planning and Development
Degree
Doctor of Policy, Planning & Development
Degree Program
Planning and Development,Policy
Degree Conferral Date
2022-08
Publication Date
08/03/2022
Defense Date
05/11/2022
Publisher
University of Southern California
(original),
University of Southern California. Libraries
(digital)
Tag
collective impact,collective impact evaluation,Collective Impact Framework,conceptual framework,empowerment zone,enterprise communities,enterprise zone,evaluation,indicators,logic model,OAI-PMH Harvest,opportunity zone,performance measurement,Policy,policy evaluation,promise zone,qualitative analysis,SDoH,social determinants of health,social impact,social impact measures,state enterprise zone,urban policy
Format
application/pdf
(imt)
Language
English
Contributor
Electronically uploaded by the author
(provenance)
Advisor
Lewis, LaVonna (
committee chair
), Jack, Leonard (
committee member
), Peter, Robertson (
committee member
)
Creator Email
giagrant@usc.edu,grgrants@gmail.com
Permanent Link (DOI)
https://doi.org/10.25549/usctheses-oUC111375988
Unique identifier
UC111375988
Legacy Identifier
etd-GrantRutle-11078
Document Type
Dissertation
Format
application/pdf (imt)
Rights
Grant-Rutledge, Gia Elizabeth
Type
texts
Source
20220803-usctheses-batch-967
(batch),
University of Southern California
(contributing entity),
University of Southern California Dissertations and Theses
(collection)
Access Conditions
The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the author, as the original true and official version of the work, but does not grant the reader permission to use the work if the desired use is covered by copyright. It is the author, as rights holder, who must provide use permission if such use is covered by copyright. The original signature page accompanying the original submission of the work to the USC Libraries is retained by the USC Libraries and a copy of it may be obtained by authorized requesters contacting the repository e-mail address given.
Repository Name
University of Southern California Digital Library
Repository Location
USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA
Repository Email
cisadmin@lib.usc.edu
Tags
collective impact
collective impact evaluation
Collective Impact Framework
conceptual framework
empowerment zone
enterprise communities
enterprise zone
evaluation
indicators
logic model
opportunity zone
performance measurement
policy evaluation
promise zone
qualitative analysis
SDoH
social determinants of health
social impact
social impact measures
state enterprise zone
urban policy