University of Southern California Dissertations and Theses

All hands on deck! Collective impact for systemic change

(USC Thesis Other)

All hands on deck! Collective impact for systemic change

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content All Hands on Deck! Collective Impact for Systemic Change
by
Rhonda Jeanne Beasley
Rossier School of Education
University of Southern California
A dissertation submitted to the faculty
in partial fulfillment of the requirements for the degree of
Doctor of Education
December 2021

© Copyright by Rhonda Jeanne Beasley 2021
All Rights Reserved

The Committee for Rhonda Jeanne Beasley certifies the approval of this Dissertation

Paul Gothold
Courtney Malloy
Darline Robles, Committee Chair

Rossier School of Education
University of Southern California
2021

iv
Abstract
The purpose of this qualitative study was to evaluate the degree to which a cross-sector
collaborative was meeting its goal to use the collective impact framework to streamline and
improve service delivery to improve outcomes for children, youth, and families. The
stakeholders for this study were the members of a steering committee of a cross-agency
collaborative located in a local county government setting in the Western United States. Clark
and Estes’s (2008) gap analysis framework guided the study to assess the knowledge,
motivation, and organizational influences related to achieving the organizational goal. Literature
helped to define the assumed knowledge, motivation, and organizational influences assessed.
Data was collected through individual interviews, meeting observations, and document analysis.
The study found that the KMO influences were assets that could be leveraged to better prepare
for engaging in collective impact. The study offers recommendations through an implementation
and evaluation plan aligned to the New World Kirkpatrick Model (Kirkpatrick & Kirkpatrick,
2016).

Keywords: collective impact, intergenerational cycle of disadvantage, cross-sector
collaboration

v
Dedication
To my sons Albert James (AJ) and Jadon Charles, you inspire me every day. This dissertation is
for you. I hope you always know that you are wonderfully made in our Lord’s image. Thank you
for your patience and understanding over the last three years of this journey. I love you both all
day, every day.
To my mom who has made my every dream a reality. Thank you for all of your sacrifices. I love
you. This is every bit your celebration, as it is mine.
To Grandma Eileen, I hope you know how much your sacrifices, all those years ago, have
inspired me to work towards this moment.
To my loved ones who have passed on: Granny, Ma, Aunty Amoy, Grandma Jeanne, and Daddy.
I will never forget and I remain forever thankful for your influence on my life. You may not have
lived to see this moment, but it would not have happened without any of you. I love you and
miss you all.
To my late grandfather James O’Neil Lewis. Though you have passed, I hope you are looking
down with pride. This was a dream you had for me since I was a young girl. On your visits home
to Trinidad from Washington DC, you would dress me in your own PhD graduation gown and
take a picture and tell me one day I will achieve this too. Well, I did. Thank you and I love you.
To all who have loved me, believed in me, and supported me throughout the years, thank you.
This is your celebration too. I am forever grateful.

vi
Acknowledgements
I begin with acknowledging my committee. I am forever thankful for your time,
guidance, and support. Dr. Robles, thank you for not giving up on me. You told me every time
we spoke that I would successfully complete this. You were right. Thank you!
Thank you to my husband, Kenn, for your love and support. Thank you for encouraging
me to pursue my dream and giving me the space to grow into a better person. I love you!
This moment would be impossible were it not for my entire family from Guyana who
made the sacrifice to immigrate to America in hopes of a better future for the generations to
come. I am a product of your sacrifice and hard work. I hope you know your sacrifices are
appreciated and never forgotten. One love!
A very special thank you to my Trinidad family, my original collective impact. You all
had a common agenda to love and care for the original four (Susan, Sam, Justin, and me), and
eventually Jesse, Marc, and Jabari. You leveraged each of your strengths to ensure we had
fulfilling childhoods. We wanted for nothing because we were surrounded by such immense
love. You are the best prayer warriors. No request is ever too big. I love you all so much and I
am forever in your debt for helping to get me to where I am today.
My celebration of success is also shared with my Beasley family. Thank you for your
love and support. Thank you for forming a collective impact around my sons. You are the best
village. I am so blessed to have you all in my life. I love each of you!
I am blessed to have true sister-friends, Dionne, Rebecca, Christina, Felicia, Leilah, and
Moe. Your support kept me moving forward to the finish line.
Special acknowledgment to Tia M., thank you for sharing your knowledge with me and
being there whenever I needed you. I am so honored to call you my friend.

vii
Table of Contents
Abstract .......................................................................................................................................... iv
Dedication ........................................................................................................................................v
Acknowledgements ........................................................................................................................ vi
List of Tables ...................................................................................................................................x
List of Figures ................................................................................................................................ xi
List of Abbreviations .................................................................................................................... xii
Chapter One: Overview of the Study ...............................................................................................1
Organizational Context and Mission ...................................................................................2
Organizational Performance Goal........................................................................................3
Related Literature.................................................................................................................3
Importance of the Evaluation ...............................................................................................5
Description of Stakeholder Groups ......................................................................................6
Stakeholder for the Study and Stakeholder Performance Goal ...........................................7
Purpose of the Project and Questions ..................................................................................8
Methodological Framework .................................................................................................8
Definitions............................................................................................................................9
Organization of the Study ..................................................................................................12
Chapter Two: Review of the Literature .........................................................................................14
Ecological and Contextual Factors in the Life Course: The Lived Experiences of Black
Boys and Men Across Generations ..............................................................................15
Declining Employment and Earnings Among Black Men ................................................16
The Rise of Mass Incarceration of Black Men ..................................................................17
The Increase of Disadvantaged Families ...........................................................................19
Black Boys in Disadvantaged Families .............................................................................20

viii
Responding to the Problem ................................................................................................26
Collective Impact ...............................................................................................................27
Conceptual Framework ......................................................................................................33
Stakeholder Knowledge, Motivation, and Organizational Influences ...............................33
Organizational Factors .......................................................................................................46
Conceptual and Methodological Framework .................................................................................51
Assessment of Performance Influences .........................................................................................53
Participating Stakeholders and Sample Selection ..............................................................62
Instrumentation ..................................................................................................................62
Data Collection ..................................................................................................................63
Data Analysis .....................................................................................................................65
Trustworthiness of Data .....................................................................................................65
Role of Investigator........................................................................................................................66
Limitations .....................................................................................................................................66
Chapter Four: Results and Findings ...............................................................................................68
Participating Stakeholders .................................................................................................69
Determination of Assets and Needs ...................................................................................69
Research Question One: KMO Influences .........................................................................70
Research Question Two: Organizational Culture and Context ..........................................95
Research Question Three: Readiness for Collective Impact ............................................104
Summary of Validated Influences ...................................................................................119
Chapter Five: Recommendations and Evaluation ........................................................................124
Organizational Context and Mission ...............................................................................125
Organizational Performance Goal....................................................................................125
Description of Stakeholder Group for the Study .............................................................126

ix
Goal of the Stakeholder Group for the Study ..................................................................126
Recommendations to Address Knowledge, Motivation, and Organization Influences ...127
Implementation and Evaluation Plan ...............................................................................141
Limitations and Delimitations..........................................................................................158
Recommendations for Future Research ...........................................................................159
Conclusion .......................................................................................................................161
References ....................................................................................................................................162
Appendix A: Interview Protocol ..................................................................................................184
Appendix B: Qualitative Research Documents Analysis Protocol ..............................................188
Appendix C: Information Sheet for Exempt Research ................................................................189
Appendix D: Recruitment Email .................................................................................................193

x
List of Tables
Table 1: Organizational Mission, Global Goal, and Stakeholder Goals 7
Table 2: Summary of Assumed Knowledge Influences on Stakeholders Ability to Achieve the
Performance Goal 40
Table 3: Summary of Assumed Motivation Influences on Stakeholders’ Ability to Achieve the
Performance Goal 45
Table 4: Summary of Assumed Organization Influences on Stakeholders Ability to Achieve the
Performance Goal 49
Table 5: Summary of Knowledge Influences and Method of Assessment 56
Table 6: Summary of Motivation Influences and Method of Assessment 59
Table 7: Summary of Organization Influences and Method of Assessment 61
Table 8: Summary of Findings for Research Question One 120
Table 9: Summary of Findings for Research Question Two 122
Table 10: Summary of Findings for Research Question Three 123
Table 11: Summary of Knowledge Influences and Recommendations 128
Table 12: Summary of Motivation Influences and Recommendations 134
Table 13: Summary of Organizational Influences and Recommendations 138
Table 14: Outcomes, Metrics, and Methods for External and Internal Outcomes 143
Table 15: Critical Behaviors, Metrics, Methods, and Timing for Evaluation 146
Table 16: Required Drivers to Support Critical Behaviors 149
Table 17: Evaluation of the Components of Learning for the Program. 154
Table 18: Components to Measure Reactions to the Program. 155

xi
List of Figures
Figure 1: The Collective Impact Model 30
Figure 2: Gap Analysis Process 53

xii

List of Abbreviations
IMTSS Integrated Multi-tiered System of Support
SACT Siloed Agencies Collaboration Team
SC Steering Committee
SSC Siloed Services County

1

Chapter One: Overview of the Study
Disproportionate exclusionary discipline rates and the racialized academic achievement
gap (U.S. Department of Education [USDOE], 2021b) are examples of systemic inequities in
public schools across the United States. There is increasing evidence that these inequities most
negatively affect Black boys living in poverty (Creamer, 2020), who are most likely to have low
academic outcomes, experience higher rates of exclusionary discipline, live in single-parent
homes (USDOE, 2021a), and have an incarcerated parent (Glaze & Maruschak, 2016). As these
boys become men, they are also most likely to experience incarceration and fewer job
opportunities, resulting in social and economic outcomes that contribute to the intergenerational
cycle of disadvantage and creating a series of complex problems (Aizer & Doyle, 2015; Cheng et
al., 2019; Fabelo et al., 2011; Grodsky & Pager, 2001; Hirsch & Winters, 2014; Pettit &
Western, 2004; Reardon & Bischoff, 2011; Sakamoto et al., 2018; Tonry, 2011). In response, in
2009, a county board of supervisors in the Western Region of the United States established the
Siloed-Agency Collaboration Team (SACT), a cross-system multi-stakeholder collaborative
of child- and family-serving departments and organizations from throughout Siloed Services
County. The team’s goal was to address racial disparities among children and families in SSC.
Without a coordinated service delivery model, there was a plethora of duplicated school-based
initiatives from multiple agencies that still left many children and families unserved.
In 2017, the SACT prevention working group, concerned about Black boys’ high
suspension rates in local schools, learned about the socio-ecological factors contributing to the
school-to-prison pipeline. They noted a common focus of dismantling the school-to-prison
pipeline by focusing on a multi-tiered framework to organize county programs and initiatives to
quickly match these resources to identified needs. In 2018, the SACT steering committee (SC)
2

addressed the school-to-prison pipeline problem by focusing on upstream prevention to address
the social and economic barriers that, when coupled with systemic inequities in the school
setting, perpetuate the cycle of disadvantage. The upstream prevention model aims to improve a
population’s overall health, well-being, and quality of life by decreasing and removing barriers
by improving social and economic structures (Butterfield, 2017; National Collaborating Centre
for Determinants of Health [NCCDH], 2021). The committee selected the collective impact
model to remove silos and systemically work together in trusting relationships towards a
common agenda (Kania & Kramer, 2011).
With a focus on cross-sector collaboration to streamline and improve service delivery
across the county, the committee also wants to ensure each member is ready to fully commit to
collective impact. Therefore, the problem of practice is to assess the committee’s readiness to
engage in collective impact to achieve the larger organizational goal. Collective impact is a
commitment of a group of people from different sectors to address a complex social issue
through structured cross-sector collaboration. The outcome is the collective impact. The structure
is defined by the five conditions of collective impact: a common agenda, shared measurement,
continuous communication, mutually reinforcing activities, and a backbone support organization
(Kania & Kramer, 2011). Literature on collective impact efforts points to the role of the SACT
SC in moving the initiative towards desired outcomes (Hanleybrown et al., 2012; Parkhurst &
Preskill, 2014). For this reason, this study explored how members of the SACT SC work together
towards collective impact.
Organizational Context and Mission
The SACT is a cross-system multi-stakeholder collaborative of child- and family-serving
departments and organizations throughout Siloed Services County (SSC, a pseudonym). Located
3

in the western United States, SSC has a reputation as an innovation hub and a significant
contributor to the world’s technological advances. The county has an estimated population of
about 1.9 million, of whom about 600,000 are younger than 24. Despite its premier status for
technology, significant racial disparities impact African American youth aged 24 and younger in
SSC. Although they account for only 2% of the population, African American youth make up
19% of all youth living in poverty, compared to Latinx (18%), Asian (5%), and White (4%). In
addition to the disproportionate poverty rates, 2018–19 data show that African American youth
in SSC also experience persistent disparities in educational attainment, suspension and
expulsion, placement in foster care, and juvenile arrests. African American families also
experience persistent disparities in median income and infant mortality rates.
Organizational Performance Goal
The SACT’s goal is to use the collective impact framework to design and implement an
integrated multi-tiered system of support (IMTSS) across all child- and family-serving agencies
by June 30, 2022, to streamline and improve the quality of services provided to the community.
The IMTSS is intended to focus collective efforts on prevention and intervention using the
collective impact framework with an ultimate goal of dismantling the school-to-prison pipeline
and eliminating the intergenerational cycle of disadvantage. As noted earlier, collective impact is
an intentional way of working together towards a common purpose (Kania & Kramer, 2011). To
that end, the IMTSS spans multiple agencies.
Related Literature
As noted earlier, exclusionary practices yield negative consequences for Black boys and
men. These include a widening academic gap, increased dropout rates, and a pathway into the
criminal justice system (Fabelo et al., 2011; Wood et al., 2018). In addition to the in-school
4

factors that contribute to this phenomenon, there are larger societal factors that also create
barriers that affect Black men later in life and affect the next generation of Black boys
(Bradshaw et al., 2010; Bush & Bush, 2013; Cheng et al., 2019; Chetty et al., 2018; Grodsky &
Pager, 2001; Hirsch & Winters, 2014; Kaufman et al., 2010; National Research Council, 2014;
Okonofua & Eberhardt, 2015; Orrock & Clark, 2018; Philpart & Bell, 2015; Reardon &
Bischoff, 2011; Sakamoto et al., 2018; Skiba et al., 2002, 2011; Todd et al., 2016; Wilbur et al.,
2007). One philosophical approach to addressing this issue is to apply upstream thinking. The
term “upstream thinking” was originally used by J.B. McKinlay in 1979 to stress the importance
of prevention. McKinlay described a health care system more focused on caring for those already
afflicted with illness rather than investigating the root causes of the illness to prevent them from
happening. The idea of moving prevention further upstream, or upstream prevention, has moved
beyond a healthcare phenomenon to a theory of prevention that is applicable in many different
sectors. The idea is to address the larger societal factors or root causes to prevent negative
outcomes further downstream. This theory of upstream prevention served as the foundation for
past cross-sector collaboration efforts to improve outcomes for children, youth, and families.
Cross-sector collaboration to improve educational systems and eventual student outcomes
is not new. Henig et al. (2015) conducted a large-scale review of the history of cross-sector
collaborations focused on improving education and off-setting poverty’s impact on children.
Initial efforts were community-based and did not include the school system as a part of the
process. Later efforts brought many services into the schools, placing schools at the center of
these efforts (Rogers, 1998). However, a lack of cohesion among the schools, social service
providers, and community members led to divergent views on what should be prioritized. This
resulted in failed cross-sector initiatives and collaboration. In 1966, the Equality of Educational
5

Opportunity report, also commonly known as the Coleman Report (Coleman, 1966), identified
the integration of in-school and out-of-school factors as key to determining successful outcomes
for students in school and later in life. In later reports, Coleman (1987) continued to push for
coordination among agencies to provide comprehensive services to support students in schools
and their families at home. This led to more focused efforts to create cross-sector collaborations
that did not prioritize in-school factors over social factors or vice versa but, instead, sought to
address these simultaneously.
Challenges with cross-sector collaboration include funding, merging programs,
prioritizing community needs, and navigating the political landscape (Crowson & Boyd, 1993,
1996; Jolin et al., 2012; Kubisch et al., 2010; Riehl & Lyon, 2017). Collective impact offers a
systemic approach to cross-sector collaboration. There are three phases of implementation
(Hanleybrown et al., 2012). The first is to initiate action. During this phase, key partners and
existing initiatives in the specific policy area are identified. An initial governance structure in the
form of a steering committee is also organized. During Phase Two, the five conditions of
collective impact are established. The steering committee is also responsible for directing this
phase of implementation. During the final phase of sustainability and measuring impact, data are
reviewed to make continuous improvement and refinement.
Importance of the Evaluation
This evaluation study of the SACT’s goal of using the collective impact framework to
implement the IMTSS was important for a number of reasons. Past cross-collaboration efforts
between schools and social service agencies have faced challenges related to funding, merging
programs, prioritizing community needs, and navigating the political landscape. In the 12 years
since its creation, SACT has increased opportunities for coordinated services; however, many
6

agencies continue to work in isolation, focusing on their specific agency goals, thus leaving
many families underserved. Therefore, this study can offer insight for other multi-sector
practitioners seeking a successful approach to systemic collaboration to accomplish their larger
organizational goals. Additionally, the underlying goal of the SACT’s cross-sector collaboration
and IMTSS is to dismantle the school-to-prison pipeline by eliminating the intergeneration cycle
of disadvantage. Successful progression towards this goal depends on the coordinated efforts of
system agencies and community-based organizations. Regarding the impact and effectiveness of
collective impact on institutions, Stachowiak and Gase (2018) found that strong implementation
contributed to meaningful changes for the target population.
Description of Stakeholder Groups
Successful implementation of the five conditions of collective impact requires the support
of a number of stakeholder groups. The SACT SC is comprised of the executive-level directors
from the county’s executive office and child and family-serving agencies. This group coordinates
implementation of the overall vision and joint approach to solutions, defines roles for SACT, and
plans for SACT meetings and retreats. Middle managers at multiple departments in these
agencies make up the SACT working groups. Their role is to carry out the specific tasks assigned
to the working groups and report to the SACT SC and the larger SACT. Finally, the SSC’s chief
executive’s office (CEO) agreed to serve as the backbone organization.

7

Table 1
Organizational Mission, Global Goal, and Stakeholder Goals
Organizational mission
To expand efforts for collaborative work across child and family-serving departments to
improve outcomes for children, youth, and families
Organizational performance goal
By June 2022, the SACT will use the collective impact framework to fully implement the
Integrated Multi-tiered System of Support (IMTSS) across all child- and family-serving
agencies to streamline and improve service delivery across child- and family-serving
departments, agencies, and the community.
Stakeholder 1: SACT steering
committee

By June 2021, the SACT
steering committee will
establish an implementation
plan for the IMTSS using
the conditions of collective
impact as measured by an
internal audit.
Stakeholder 2: SACT working
groups

By August 2021, 100% of the
SACT working groups will
identify objectives for their
specific groups.
Stakeholder 3: Backbone
organization

By December 2020, the
Backbone Support
Organization will develop
a theory of change focused
on upstream prevention.

Stakeholder for the Study and Stakeholder Performance Goal
At the core of successful collective impact efforts are the agreement and alignment of
stakeholders from multiple sectors. While the joint efforts of all stakeholders will contribute to
the full implementation of the five conditions of collective impact, the SACT SC is responsible
for the oversight of the implementation of four of the five conditions. The SSC County
Executive’s Office (SSC-CEO) assumed the role of the backbone agency, the fifth collective
impact condition. Hanleybrown et al. (2012) found that many successful collective impact efforts
begin with an oversight group or steering committee. As such, it is important to evaluate the
SACT SC’s collective efforts to measure overall progress towards the organizational
8

performance goal. Therefore, the stakeholder of focus for this study will be all of the members of
the SACT SC.
The stakeholder goal is directly linked to the organizational goal of using the collective
impact framework to implement the IMTSS across all child- and family-serving agencies to
streamline and improve service delivery across child- and family-serving departments, agencies,
and the community. Therefore, failure to implement the collective impact conditions could
compromise the overall effectiveness of the SACT and have detrimental consequences on SSC’s
children and families most in need of county services.
Purpose of the Project and Questions
The purpose of this project was to evaluate the degree to which the SACT was meeting
its goal of fully implementing the IMTSS using the collective impact framework. The analysis
focused on the areas of knowledge and skills, motivation, and organizational influences related
to achieving the organizational goal. While a complete performance evaluation would focus on
all stakeholders, for practical purposes, the stakeholder of focus on in this analysis was the
SACT SC. As such, three questions guided this study:
1. What are the committee members’ knowledge, motivation, and organizational resources
related to creating a plan to implement IMTSS?
2. What is the interaction among organizational culture and context and the committee
members’ knowledge and motivation?
3. In what way is the SACT SC positioned to engage in collective impact?
Methodological Framework
Clark and Estes’s (2008) gap analysis framework aims to identify the knowledge,
motivation, and organizational (KMO) influences and then identify performance gaps related to
9

the achievement of the goal. Related literature was used to identify the KMO influences for the
SACT SC, followed by a qualitative data gathering and analysis to evaluate the committee’s
current performance in relation to their performance goal in the areas of KMO resources. The
committee members’ current performance was evaluated by using interviews, artifact review,
meeting observations, literature review, and content analysis.
Definitions
• Adverse Childhood Experiences (ACEs): Potentially traumatic events that occur in
childhood (0–17 years). Also included are aspects of the child’s environment that can
undermine their sense of safety, stability, and bonding, such as growing up in a
household with substance abuse, mental health problems, and instability due to parental
separation or household members being in jail or prison (CDC, 2020).
• Adaptive or Complex Problem: A problem where the answer is not known and no single
entity has the resources or authority to bring about necessary change (Kania & Kramer,
2011).
• Causal Density: Refers to the tangled web of interrelated causes; causes co-vary with one
another, making it difficult to determine one single cause. Everything is related to
everything else, which makes siloed approaches ineffectual (personal communication,
Martinez, April 5, 2020).
• Collective Impact: A commitment of a group of important actors from across different
sectors to focus on a common agenda to address a specific complex problem in a
structured way. There are five conditions that guide the work of collective impact (Kania
& Kramer, 2011). The first is a common agenda, which is a shared vision for change,
including a common understanding of the problem and a joint approach to solving it
10

through agreed-upon actions (Kania & Kramer, 2011). The second is shared
measurement systems, meaning consistent data collection and agreed-upon measures of
success and how they will be reported to all partners (Kania & Kramer, 2011). The third
is continuous communication, meaning consistent and open communication is needed
across the many players to build trust, assure mutual objectives, and create common
motivation (Kania & Kramer, 2011). The fourth is mutually reinforcing activities,
meaning the coordination of the differentiated activities across partners through a
combined efforts plan of action (Kania & Kramer, 2011). The fifth is a backbone support
organization: A separate organization and staff to manage and facilitate the collective
impact by coordinating participating organizations and agencies (Kania & Kramer, 2011).
• Disadvantaged Family: A single-mother-led family living in poverty.
• Downstream Interventions: interventions and strategies that focus on providing equitable
access to care and services to mitigate the negative impacts of an already occurred crisis
(NCCDH, 2021).
• Educational Equity: Educational equity means that each child receives what they need to
develop to their full academic and social potential (National Equity Project, 2020).
• Equity: Fairness achieved through systematically assessing disparities in opportunities
and outcomes and addressing disparities through targeted actions (Urban Strategies
Council, 2020).
• Exclusionary Discipline or Suspension: The removal or exclusion of a student from his or
her customary educational learning environment as a form of punishment. During a
suspension, a student is not allowed to attend school or attend school activities for a set
length of time (National Clearinghouse on Supportive School Discipline, n.d.).
11

• Housing Insecurity: An umbrella term to describe the housing problems many experience
due to affordability, safety, quality, and loss of housing (U.S. Department of Housing and
Urban Development, 2018).
• Income Segregation: An unbalanced distribution of families into income-based
neighborhoods and communities, leading to concentrations of disadvantaged families in
certain communities (Reardon & Bischoff, 2011).
• Intergenerational Cycle of Disadvantage: Poverty and the disadvantages as a result of
poverty that transfer from one generation to the next (Georgia Center for Opportunity,
n.d.).
• Isolated Impact: An approach oriented toward finding and funding a solution embodied
within a single organization, combined with the hope that the most effective
organizations will grow or replicate to extend their impact more widely (Kania &
Kramer, 2011).
• Multi-Stakeholder Initiative: A voluntary activity by stakeholders from different sectors
around a common theme. This type of initiative tends to lack any shared measurement of
impact and the supporting infrastructure to forge any true alignment of efforts or
accountability for results (Kania & Kramer, 2011).
• Poverty: A child who qualifies for free and reduced lunch programs at school (de Brey et
al., 2019).
• School-to-Prison Pipeline (STPP): A construct that describes the policies and practices in
the public schools and juvenile justice system that decrease the probability of school
success for children and youth and increase the probability of negative life outcomes,
particularly through involvement in the juvenile justice system (Skiba et al., 2014).
12

• Socioeconomically Disadvantaged Students: Students who are eligible for free or
reduced-price meals or have parents/ guardians who did not receive a high school
diploma (National Institutes of Health, 2019).
• Technical Problem: A problem that is well defined with an answer that is known in
advance, and one or a few organizations have the ability to implement the solution (Kania
& Kramer, 2011).
• Upstream Prevention: Interventions and strategies that focus on improving social and
economic structures in order to decrease barriers and improve support before a crisis
occurs (NCCDH, 2021).
Organization of the Study
Five chapters are used to organize this study. This chapter provided the background on
the problem, describes the organization, including mission and goals and stakeholders of interest.
It also presented the key concepts and terminology commonly found in a discussion about the
school-to-prison pipeline, the intergenerational cycle of disadvantage, cross-sector collaboration,
and collective impact. Initial concepts of gap analysis were also introduced. Chapter Two
provides a review of related literature on the subject of the study. Topics of the relationship
between exclusionary school discipline and the school-to-prison pipeline, a sociological view of
the influencing factors of the life course framework, and the efficacy of collective impact will be
addressed. Chapter Two also utilizes Clark and Estes’s (2008) gap analysis to present the
evaluation of the KMO factors that could influence the SACT SC’s performance. Chapter Three
details the methodology when it comes to the choice of participants, data collection, and
analysis. In Chapter Four, the data and results are assessed and analyzed. Finally, Chapter Five
13

provides solutions, based on data and literature, for closing the perceived gaps as well as
recommendations for an implementation and evaluation plan for the solutions.

14

Chapter Two: Review of the Literature
Children do not develop in isolation from their communities or larger society. Rather,
multiple factors influence children’s development, including their home, family, education
system, and larger system policies and practices (Bronfenbrenner, 1977, 1979). As agencies and
policy makers respond to the call to improve outcomes for Black boys, it is important to consider
the ecological contexts and experiences in these boys’ daily lives and the impact of race, racism,
classism, and sexism on their life transitions (Bush & Bush, 2013; Elder, 1998; Philpart & Bell,
2015). There is a plethora of data on how racial and economic disparities negatively affect
generations of Black boys (Chetty et al., 2018; National Research Council, 2014; Orrock &
Clark, 2018; Wilbur et al., 2007). As such, this literature review examines the socio-ecological
influences on the intergenerational cycle of disadvantage. Given the evidence that Black men’s
experiences impact Black boys’ development and contribute to the intergenerational cycle of
disadvantage (Barbarin, 2010; Dumas & Nelson, 2016; Howard, 2013; Orrock & Clark, 2018;
Wilbur et al., 2007), this literature review will also examine the potential efficacy of an IMTSS
consisting of multiple county agencies focused on the common goal of dismantling the school-
to-prison pipeline by eliminating the intergenerational cycle of disadvantage.
This review begins with an overview of larger societal policy and structural changes that
produced intergenerational cycles of disadvantage that significantly negatively impact Black
boys. The review of literature on the collective impact framework highlights a cross-agency
solution (Kania & Kramer, 2011) to address complex problems. Finally, utilizing Clark and
Estes’s (2008) gap analysis framework, the review explores the KMO influences on the
committee members’ readiness to engage in collective impact through working together to
accomplish the organizational goal.
15

Ecological and Contextual Factors in the Life Course: The Lived Experiences of Black
Boys and Men Across Generations
The U.S. Department of Education (2021a) reported that in 2019, Black children under
the age of 18 continued to experience the highest rate of poverty (30%), were least likely to live
with married parents (34%), and were more likely to live in a female-headed single-parent home
(55%) showing racial gaps that exposed these children to higher rates of disadvantage. In
addition to racial gaps, gender gaps were also found in high school dropout rates, postsecondary
education rates, and rates related to education outcomes like employment and salary (Buchmann
& DiPrete, 2006; DiPrete & Buchman, 2006; Geller et al., 2011; Haskins, 2016; Murray &
Farrington, 2005). The USDOE reported that the 2018 high school dropout rate among Black
boys was 7.0% and 4.4% for Black girls. The data also show the gender gap for college
completion rate was the widest among Black students (44% for females and 34% for males)
when compared to other racial groups. Additionally, the labor force participation rate for Black
men has been on the decline since 1972, while for Black women, it has held steady since
reaching a peak in 2016 (U.S. Bureau of Labor Statistics, 2020). These data are important to
acknowledge because they reveal the negative trends for Black males in key opportunity
outcomes relative to Black females.
In addition to opportunity trend data, data on incarceration also show a racial gap.
According to the U.S. Bureau of Justice Statistics (2020), although state and federal
imprisonment rates have declined since 2008 to the lowest rates in 2018, there still remains a
race and gender gap with particular negative outcomes for Black men. According to this report,
Black men were imprisoned 5.8 times more than White men, and almost double the rate of White
women. These data show disparities embedded in societal structures that negatively affect Black
16

men based on race and gender and create adverse consequences for the next generation.
Therefore, to understand Black boys’ experiences, it is necessary to study the experiences of the
Black men who influence their lives.
Declining Employment and Earnings Among Black Men
The labor market conditions in the United States reflect racial inequalities against Black
men in job opportunities as well as income (Cheng et al., 2019; Grodsky & Pager, 2001; Hirsch
& Winters, 2014; Reardon & Bischoff, 2011; Sakamoto et al., 2018). Hirsch and Winters (2014)
reported that there has been an overall withdrawal of Black men from the labor market. In a
review of decennial census data, they found that between 1950 and 2010, long-term joblessness
(no employment for 12 or more months) among Black men increased from 13% to 42% (Hirsch
& Winters, 2014). The authors also examined the connection between education level (i.e., no
high school diploma, high school graduate, some college, and college graduate) and employment
rates. They found that while Black men at all education levels had the highest rates of all
unemployed men, the data were most detrimental for Black men who did not complete college.
In addition, when employed, Black men earn lower incomes than other races across all
levels of occupational mobility (Grodsky & Pager, 2001). PEW Charitable Trusts and the
Economic Mobility Project (2010) reported that while both Black and White men experienced a
decline in incomes between 1974 and 2004, Black men were earning less in 2004 than in 1974.
Again, lower education levels resulted in lower long-term earnings rates (Cheng et al., 2019;
Donovan & Bradley, 2019) and career instability (Sakamoto et al., 2018). Low-income career
trajectories contribute to income segregation, which refers to an unbalanced distribution of
families into income-based neighborhoods and communities, leading to concentrations of
disadvantaged families in certain communities (Reardon & Bischoff, 2011). These downward
17

economic trends create instability for the next generation since parental income is a strong
indicator of a child’s upward economic mobility into adulthood (PEW Charitable Trusts with the
Economic Mobility Project 2010).
The Rise of Mass Incarceration of Black Men
In the 50 years prior to 1973, the United States experienced stable incarceration rates
between 100 to 200 per 100,000 residents (U.S. Bureau of Justice Statistics, 2020). From 1973 to
2010, incarceration rates in the U.S. increased 430%, reaching a peak of 767 per 100,000
residents in 2007. In 2012, there were 2.23 million people held in jails and prisons in the United
States, which was almost seven times the incarcerated population in 1972. Despite the U.S. being
just 5% of the world’s population, the U.S. incarcerated population represented almost 25% of
the world’s prisoners, the highest rate of adult incarceration in the world. This rise in
incarceration disproportionately affects Black men (Pettit & Western, 2004; Tonry, 2011), who
are 5.8 times more likely than White men to be incarcerated (U.S. Bureau of Justice Statistics,
2020). Similar to employment rates, incarceration rates were more heavily concentrated among
less-educated men. Western and Pettit (2010) reported that 68% of Black men in the United
States who had dropped out of high school went to prison in their lifetime compared to 21% of
Black men who had graduated from high school or had earned a GED and 7% of Black men who
had attended college.
Although class and race disparities among incarcerated male populations were found to
be standard in the data reporting, when these inequities were coupled with low educational level,
the results were especially harmful to Black men aged 20 to 34 with no college and particularly
those who had dropped out of high school (Ewert et al., 2014; Western & Pettit, 2010). In an
investigation of the data for the 2010 prison boom, the National Research Council (2014)
18

reported that 68% of Black men who had not completed high school had been incarcerated in
either the state or federal penitentiary system. Specifically, the data showed that there were more
Black men aged 20 to 34 incarcerated than employed (Western & Pettit, 2010). Age is a
significant factor here because Martinez et al. (2012) reported that about 65% of Black fathers
have their first child before age 25, and about 85% have their first child by age 30. These fathers
have another child within 5 years of the birth of the first child (Martinez et al., 2012). These data
show that the next generation of Black children is born to a large population of Black men who
are around 35 years old. Given the data presented regarding incarceration and employment, the
next generation of Black boys is most likely to be born into a home with an incarcerated father or
a father who is relegated to poverty due to a felony conviction that allows for legal
discrimination in housing, education, and employment.
The intentional targeting of Black men has had the greatest impact on the most
disadvantaged communities (National Research Council, 2014), which tend to have high
unemployment rates and concentrated long-term low-income earnings (Bloome, 2014; Reardon
& Bischoff, 2011; Sakamoto et al., 2018). This connection between unemployment rates and
incarceration rates demonstrates the feedback loop formed between mass incarceration and
unemployment and declining earnings (Chandler, 2020). While the relationship between
unemployment and crime is complex (Andresen, 2012; Phillips & Land, 2012) and beyond the
scope of this review, there is a more direct connection between a felony conviction and few, low-
paying job opportunities (Ewert et al., 2014; Shannon et al., 2017; Sykes & Maroto, 2016;
Wakefield & Uggen, 2010; Western, 2002; Western & Pettit, 2010) leading to structural shifts in
affected families, particularly Black families (Wakefield & Uggen, 2010). These shifts directly
contribute to hardships that extend to the social and economic outcomes of the families of the
19

formerly incarcerated (Foster & Hagan, 2015; Wakefield & Uggen, 2010; Western & Pettit,
2010).
The Increase of Disadvantaged Families
Black men’s increasing joblessness, declining earnings, and mass incarceration have
driven the growth of disadvantaged families. For the purpose of this review, a disadvantaged
family is defined as a family headed by a single mother living in poverty. Poverty is defined as
qualifying for free or reduced-price lunch programs at school (de Brey et al., 2019). As reported
earlier, Black children have maintained the highest rates of poverty. de Brey et al. (2019)
reported that Black children’s poverty rates remained steady between 32% and 40% from 2001
to 2018. This translates to about one-third of all Black children living in poverty. The data for
Black children under age 18 living in single-mother families increased from 20% in the late
1950s to 64% in 2019 (The Annie E. Casey Foundation, 2020). The high rates of poverty and
single-mother families for Black children are directly connected to the rates of joblessness,
earnings, and mass incarceration of Black men.
The U.S. Bureau of Justice Statistics (2020) reported that about 2.7 million children had
an incarcerated parent. Since more men are incarcerated than women, more children experience
paternal incarceration than maternal incarceration. The racial and economic disparities of the
incarcerated population can also be found in the families most impacted. Black children were
reported to be 7.5 times more likely than White children to have a parent in prison (Glaze &
Maruschak, 2016) and 10 times more likely to have an incarcerated father (PEW Charitable
Trusts, 2010). This racial disparity is important when considering the research on poor outcomes
for children with an incarcerated parent.
20

Black men’s increased incarceration rates create a void in the family structure and
economic hardship for the family, contributing to an intergenerational cycle of disadvantage
(Martin, 2017). The loss of a parent to incarceration reduces the family’s income, which
increases the risks of poverty and housing insecurity (Geller et al., 2011; PEW Charitable Trusts,
2010; Sykes & Pettit, 2014; Wildeman, 2014). Housing insecurity increases the likelihood of the
family’s reliance on government programs (Sugie, 2012; Sykes & Pettit, 2014; Sykes & Maroto,
2016), as well as the family’s likelihood of living in communities with concentrated
disadvantage which often have limited community resources (Reardon & Bischoff, 2011) and
higher levels of crime and drugs (Orrock & Clark, 2018). Concentrated incarceration and reentry
rates also destabilize the larger community as there is a constant flow of men in and out of the
community due to incarceration (Chung & McFadden, 2010). Therefore, communities with
concentrated disadvantage that are under-resourced and highly criminogenic increase the
likelihood of poor outcomes for the children in those communities.
Black Boys in Disadvantaged Families
Parental incarceration has been shown to produce negative outcomes for children (Arditti,
2005; Arditti et al., 2003; Foster & Hagan, 2015; Geller et al., 2009; Johnson & Easterling, 2012;
NRC, 2014). The rise of single-parent homes has gender-specific effects of family disadvantage
on educational outcomes (Battle & Scott, 2000; Geller et al., 2011; Haskins, 2016; Haskins et al.,
2018; Murray & Farrington, 2005). Geller et al. (2011) found that while aggressive behavior
increased in both boys and girls after a father was incarcerated, aggression increased most among
boys as young as 5. For these young children, their increased aggression often conflicts with the
behavioral demands of school. Bertrand and Pan (2013) found that boys from single-parent
families exhibit more disruptive behavior. They also found that the deficit in behavior skills was
21

not due to biological influences but was more strongly linked to the home environment. In a
study of one million children in Florida, Autor et al. (2019) reported that boys, more than girls,
who were born into disadvantaged families experienced higher rates of behavior problems and
lower academic outcomes, making them less prepared for kindergarten and increasing their
likelihood of not completing high school.
The effects of growing up in a single-parent family extend beyond the in-school
experience. The absence of a father or father figure was also found to increase delinquent
behavior in adolescent boys and young adult males (Cobb-Clark & Tekin, 2013; Swisher &
Shaw-Smith, 2015). Additionally, college-going rates were higher for females than males from
single-parent homes (Buchmann & DiPrete, 2006). This section of the review examines specific
influences on Black boys’ experiences in disadvantaged families.
Poverty Implications
Given the high rate of Black children exposed to poverty, it is important to consider the
implications for these children. While much of the research points to causes and potential
impacts for the family experiencing poverty, this section identifies specific implications for the
children living in poverty. Poverty has a cumulative effect on children’s growth and development
(Yoshikawa et al., 2012), meaning that children living in poverty often experience immediate
negative outcomes as well as outcomes that accumulate over time. Children living in poverty
experience social inequities that start very early in life (Brooks-Gunn & Duncan, 1997; Chung et
al., 2016) and contribute to negative outcomes related to physical and behavioral health,
cognitive development, academic achievement, and educational attainment (Aber et al., 1997;
Brooks-Gunn & Duncan, 1997; Eamon, 2001; Engle & Black, 2008; Mallett, 2017; Scharf et al.,
22

2016; Wolf et al., 2017; Yoshikawa et al., 2012) which ultimately impacts their life course
trajectory.
Physical and Behavioral Health
The reduction of available resources due to poverty increases the likelihood of food and
housing insecurity and exposure to inferior and toxic housing conditions and decreases access to
regular medical and dental care. As such, childhood obesity, chronic asthma, and food insecurity
are common among children living in poverty (Chaudry & Wilmer, 2016). In addition to
physical health, children living in poverty experience increased risk for mental, emotional, and
behavioral disorders (National Research Council & Institute of Medicine, 2009). Researchers
have linked these increases to the parental stress due to living in poverty, creating a shared
family experience with poverty for both the child and the parents (Aber et al., 2012; Byrnes &
Miller, 2012; Engle & Black, 2008; Yoshikawa et al., 2012). As children experience the direct
impacts of poverty such as hunger reduced physical health, they also experience the indirect
impacts of the stress of the adults in the home, disrupting daily routines critical to development,
such as mealtimes, family activities, and regular bedtime routines (Aber et al., 2012; Dorsey et
al., 2007; Yoshikawa et al., 2012). Engle and Black (2008) identified this as the mediated effects
of poverty, meaning that overwhelmed parents may be unable to meet the child’s emotional
needs.
Cognitive Development
Poverty also impairs a child’s cognitive development. The first 6 years of a child’s life
represent a critical period of development for the human brain (Loughan & Perna, 2012). Many
direct and indirect factors contribute to brain development. While it is easier to connect proper
nutrition and physical health to healthy cognitive development, many aspects of a child’s
23

environment contribute to the development and ultimate brain functioning. As previously noted,
adults’ stress lowers their ability to meet a child’s emotional needs. The same connection can be
made between the stress experienced as a result of living in poverty and parenting practices
(Engle & Black, 2008; Gutman et al., 2005). Poverty can disrupt early language development,
thus contributing to delayed cognitive development. Hart and Risley (1995) found that by age 3,
children living in poverty were exposed to fewer than 25% of the words to which children not
living in poverty were exposed, representing a 32-million-word gap. In addition to the
quantitative word gap, Hart and Risley also found a qualitative word gap between children who
live in poverty and those who do not. By age 4, children living in poverty heard 125,000 more
words of discouragement than encouragement compared to children not living in poverty who
heard 560,000 more words of encouragement than discouragement (Hart & Risley, 1995). As
noted earlier, parents experiencing stress caused by poverty may be unable to meet their
children’s cognitive needs, and the stress is likely to impair their parenting practices (Engle &
Black, 2008; Gutman et al., 2005).
Academic Achievement and Educational Attainment
The cumulative effect of poverty and lack of resources is well documented in delayed
early language and cognitive development and eventual low academic achievement. The
evidence highlights that many children living in poverty enter school with significantly smaller
vocabularies and overall early literacy skills than children from higher socioeconomic status
(SES) homes (Barbarin & Aikens, 2015; Fernald et al., 2013; Hart & Risley, 1995). As already
noted, Hart and Risley (1995) identified a 32-million-word gap between children in high-SES
homes and children living in poverty. Building on this study, Fernald et al. (2013) found that as
early as 18 months, there were already significant differences in vocabulary and language-
24

processing efficiency between infants from high- and low-SES families. Additionally, the home-
literacy environment is also a predictor of language and reading development. High-SES families
have access to a broader range of literature, which increases word-level literacy and overall
reading comprehension (Hamilton et al., 2016).
In addition to reduced access to educational toys and books, many families living in
poverty do not have access to high-quality early care and education (Chaudry & Wilmer, 2016),
which has been shown to reduce achievement gaps (Chung et al., 2016; Friedman-Krauss et al.,
2016). There is data to support that early achievement gaps widen as children enter school.
According to a report on fourth-grade reading by the National Assessment of Educational
Progress (NAEP, 2019), 74% of fourth-grade students identified as living in poverty were not
proficient in reading (The Annie E. Casey Foundation, 2020; USDOE, 2021a). The Children’s
Defense Fund (2021) also reported on the poverty effects on children’s educational outcomes.
They reported that almost 75% of fourth- and eighth-grade students from low-income homes
could not read or compute at grade level.
Communities with Concentrated Disadvantage
The effects of poverty extend to and from the neighborhood and home environments. The
neighborhood environment has an impact on a child’s life. Chetty et al. (2016) found that
moving into a lower-poverty neighborhood before age 13 increases college attendance and
subsequent earnings and reduces single-parent rates. However, financial hardship constrains
many single-mother families to high-poverty communities with concentrated disadvantage.
These communities are often characterized by higher crime and incarceration rates, high levels of
violence, increased drug use, and lack of resources like parks, libraries, and children’s programs
(Aber et al., 2012; Manduca & Sampson, 2019; Orrock & Clark, 2018). Thus, children in these
25

communities experience a cumulative impact of negative outcomes, some of which are
experienced immediately, such as those related to education and health, and some experienced
later in adulthood and possibly passed on to the next generation (Chaudry & Wilmer, 2016).
Schools in these communities also tend to be underfunded and offer fewer advanced courses, so
students remain economically and racially segregated, with high concentrations of them living in
poverty (Barbarin, 2010; Sakamoto et al., 2018).
Adverse Childhood Experiences and Trauma
Adverse childhood experiences (ACEs) are traumatic events that occur in a child’s life
and can create poor outcomes throughout the child-to-adult life course (Centers for Disease
Control and Prevention [CDC], 2010). Children living in poverty and those also living in
communities with concentrated disadvantage are more likely to experience trauma due to
exposure to violence and crime in the neighborhood as well as disruptive and high-stress home
environments. Poverty, having an incarcerated parent, living in a single-parent home,
homelessness, and exposure to violence are examples of ACEs. This review of the literature has
examined studies on Black boys’ disproportionately higher vulnerability to ACEs. ACEs can
also have an intergenerational effect (Strompolis et al., 2019; Woods-Jaeger et al., 2018).
Children who have experienced ACEs suffer long-lasting effects that follow them into
adulthood, including an increased risk for depression, anxiety, substance abuse, and suicide
(Jones et al., 2018). These effects contribute to traumatic experiences for their children, leading a
new generation to their own ACEs.
There is much research on how exposure to ACEs alters the developing brain and results
in impaired long-term cognitive and behavioral development from childhood into adolescence
(Burke et al., 2011; Choi et al., 2019; Sacks & Murphey, 2018). Early childhood exposure to
26

ACEs results in below-average language, literacy, and math skills and creates attention
problems, social problems, and increased aggression in children as young as age 5 (Jimenez et
al., 2016; Mazza et al., 2017). The effects of ACEs exposure are also seen in middle childhood
(ages 6 to 9), as children who experienced trauma are also likely to exhibit both externalizing
behaviors (i.e., physical aggression, verbal bullying, relational aggression, defiance, theft, and
vandalism) and internalizing behaviors (i.e., depressive disorders, anxiety disorders, suicidal
thoughts) (Hunt et al., 2017). The impact of ACEs on a child’s academic and social-emotional
readiness for school must be taken into account, as externalizing behaviors contradicts school
discipline policies.
Responding to the Problem
Federal and state responses to disproportionate exclusionary discipline rates have largely
stayed at the school level. The Supportive School Discipline Initiative (U.S. Department of
Education & U.S. Department of Justice, 2011), coordinated federal efforts to support states and
schools with school climate and discipline. Several states and the District of Columbia revised
their education laws to limit exclusionary discipline practices and provide behavioral
interventions (Steinberg & Lacoe, 2017). In response to changes in education laws, districts and
schools implement program-based initiatives to address school culture and build the adults’ skills
in managing and addressing behavior more positively (Welsh & Little, 2018). In addition to
program-based initiatives, districts and schools have also changed policies to guide how
personnel responds to student behaviors (Steinberg & Lacoe, 2017; Welsh & Little, 2018).
As noted earlier in this review, societal and structural factors influence Black boys’ in-
school experiences. These boys’ early life experiences impact their later life’s outcomes, which
shape their children’s early life experiences, thus creating or continuing an intergenerational
27

cycle of disadvantage. Therefore, fixing one part of the life course continuum will not make a
difference unless all other of its parts are simultaneously improved (Kania & Kramer, 2011).
This creates a need for upstream prevention to address the family and child’s needs in the context
of the neighborhood and community. The complexity of this problem means no particular agency
or organization holds the solution. Rather, a collective effort across agencies and sectors to
address the whole family's needs will benefit Black boys’ in-school experiences and begin to
dismantle the school-to-prison pipeline.
Collective Impact
Kania and Kramer (2011) introduced collective impact to describe a framework for
addressing large-scale and complex social problems. In this framework, cross-sector
collaboration is built around a centralized infrastructure and a highly structured process for
accomplishing a goal. The result of this collaboration is a collective impact. Cross-sector
collaboration and partnerships have long been embraced by local governments and communities
to bring about change in education, public health, human and social services, natural resources,
and other systems (LeChasseur, 2016).
Past Collaboration Reform Efforts
Cross-sector collaboration can be traced to the 19th century (Henig et al., 2015). Early
efforts to address poverty’s impact on children through a community-centered approach include
urban settlement houses (Kagan & Neville, 1993) and school-based community centers (Rogers,
1998), both of which were neighborhood centers aimed at addressing the needs of the local
community by merging educational services with social services. Political and larger societal
shifts changed the local coordination of programs to address poverty through education and
social initiatives. What emerged was a funding bureaucracy that complicated the ability to work
28

across sectors to address children’s and families’ needs. New agencies were created to address
specific needs, creating silos and competition among agencies (Henig et al., 2015).
Renewed efforts in the late 20th century led to school-based initiatives that placed
schools at the center and created partnerships between them and service providers (Henig et al.,
2015). The goal was an integrated services approach, but implementation challenges of past
efforts resurfaced, including silos and turf wars that perpetuated leadership and communication
gaps (Driscoll et al., 1998). Another challenge with this approach was the lack of understanding
of the school governance culture, including funding, board governance, district leadership, site
leadership, teaching staff, and parental involvement that contributed to multiple failed attempts at
collaboration between schools and local social service agencies (Henig et al., 2015). This pushed
many collaborative efforts away from school-centered models that sought to bring health and
social services to the school toward community-centered initiatives. Although there are examples
of successful community-centered initiatives, the challenge of breaking down structural and
funding silos failed to create a truly integrated collaboration (Henig et al., 2015).
When Kania and Kramer (2011) introduced collective impact, they brought attention to
the few collaborative initiatives that were achieving their goals despite the challenges of cross-
sector collaboration. In their examination of these initiatives, they found similar characteristics
that aided in their success.
The Collective Impact Difference
There are some distinct differences between collective impact efforts and other change
collaboration efforts. Collective impact has been described as “a new operating system for
community change” (Cabaj & Weaver, 2016, p. 1), with the result of this new type of
collaboration being large-scale collective and sustainable change. All partners in the collective
29

implement the changes in the cultural context of the individual organizations and agencies. This
is a marked difference from isolated impact, a term used to describe the approach of
implementing the solution in a single organization (Kania & Kramer, 2011). Isolated impact
results from individual organizations attempting to tackle complex problems individually, often
resulting in the duplication of efforts, lack of communication across agencies and organizations,
and increased competition to attract funders (Kania & Kramer, 2011). This approach has led to
fragmented results that fail to address the complexity of the problem (Cabaj & Weaver, 2016;
Wright, 2019) and increase frustration among policymakers and end-users (Kania & Kramer,
2011).
As a relatively new formalized framework, evidence of collective impact’s efficacy is
still emerging. Cross-sector collaboration is not new, however, and Hanleybrown et al. (2012)
argued that collective impact is set apart by a highly disciplined approach to large-scale social
change. Kania and Kramer (2011) identified the five conditions that are common across
successful collective impact efforts, such as Strive Partnership, an educational initiative in
Cincinnati; the Elizabeth River clean-up, which was a local government environmental initiative
in Virginia; and Shape Up Somerville, a health initiative in Massachusetts. The five conditions
are a common agenda, shared measurement, mutually reinforcing activities, continuous
communication, and backbone support. Hanleybrown et al. (2012) noted the necessary
preconditions and phases of implementation that increase the likelihood of successful collective
impact. Figure 1 details the five conditions, the three preconditions, and the three phases.

30

Figure 1
The Collective Impact Model

Note. Adapted from “Channeling Change: Making Collective Impact Work” by F. Hanleybrown,
J. Kania, J. & M. Kramer, 2012. FSG. (https://mappofskp.net/wp-content/uploads/2015/05/SSIR-
Collective-Impact-2.pdf)

Kania and Kramer (2013) provided more insight into how collective impact addresses
complexity. Collective impact works best on complex problems for which there are no
31

predetermined solutions or solutions are unknown. The authors argued that even with the five
conditions, three pre-conditions, and three phases, the solution emerges through an ever-evolving
process that involves continuous learning and feedback loops, as well as a shared urgency to
action that is shared across all participating member organizations of the collective effort (Kania
& Kramer, 2013). The journey through and to collective impact is uncertain, but it leads to a
collective voice and an innovative approach to systemic change (Kania & Kramer, 2013).
Making Collective Impact Work
Additional research into collective impact’s successful implementation and sustainability
yielded the following recommendations (Hanleybrown et al., 2012). First, a collective impact
effort should build on an existing collaborative effort to address the issue of focus. In already
established collaborative efforts, there is an existing structure as well as current efforts that can
be utilized to work toward a solution. There are also existing relationships that can be used to
move the work. Trusting relationships, the second recommendation, were also identified as
central to successful collective impact efforts. The third recommendation was to create an
oversight group or steering committee, separate from the backbone support organization. The
steering committee should consist of executive-level leaders representing the agencies or
organizations that engage with the issue being addressed. The steering committee’s role is to
create the common agenda, determine the strategic action framework, and then meet regularly to
provide oversight of the initiative (Hanleybrown et al., 2012). Finally, in addition to trusting
relationships, a culture of learning was also identified as an important ingredient for the steering
committee’s success.
Other researchers have advocated for applying an equity lens that uses targeted actions to
assess and address opportunity and outcome disparities throughout all five conditions of
32

collective impact. Williams and Marxer (2014) argued that the steps to applying an equity lens
are identifying existing disparities among different groups, considering these disparities’
institutional and structural sources, and applying explicit steps to improve outcomes for the
people most impacted by the inequities. Social inequities shape the complex problems that
collective impact efforts address (LeChasseur, 2016; Williams & Marxer, 2014). Therefore, it is
imperative to infuse equity throughout the processes (McAfee et al., 2015; Williams & Marxer,
2014). Some recommendations for equity emerged from the literature. First, the community that
will be impacted should be represented in the planning of the common agenda. Second, the
community should also have a say as to what data matter to them and what characteristics of data
will be examined (e.g., disaggregation by race, gender, income level, etc.). The third
recommendation is that mutually reinforcing activities should be reviewed to determine which
are or are not engaging. The fourth recommendation is that communication should be considerate
of the community served. Therefore, communication needs include translation into the languages
represented in the community. Methods of communication should also be considered based on
the community context. Finally, the backbone support organization should reflect the diversity of
the community.
The Promise of Collective Impact to Address this Problem
The problem addressed in this study connects Black boys’ in-school experiences to their
out-of-school experiences. As noted earlier, past education reform efforts have either been
school- or community-centered but have not led to systemic changes (Henig et al., 2015). Unique
to collective impact is a comprehensive approach to address the overlapping complex problems
experienced by children living in poverty. Community schools, the Harlem Children’s Zone,
Promise Neighborhoods, Say Yes to Education, and Strive Together are examples of collective
33

impact efforts that integrate schools and community-based social services to address the needs of
both the child and family (Henig et al., 2015; Kania & Kramer, 2013). These demonstrate that
collective impact has the potential to address key gaps between school and social factors.
Conceptual Framework
Clark and Estes’s (2008) gap analysis framework provides a systematic problem-solving
process to investigate the root causes of organizational performance goals. The root causes are
organized into three categories of influences: the three KMO factors, also known as KMO. This
gap analysis framework begins with clearly defined goals, followed by an analysis to determine
the gaps between the goals and the current performance (Clark & Estes, 2008). After
hypothesizing about possible causes for the gap, those causes are validated and prioritized. The
gap analysis ends with developing solutions (Rueda, 2011). In addition to identifying the
barriers, this process also highlights an organization’s strengths.
In this study, Clark and Estes’s (2008) framework was adapted to evaluate the SACT
SC’s performance toward achieving its goal. The purpose of this analysis was to understand the
committee’s goals and identify assumed performance influences based on the KMO factors,
context-specific literature, and an understanding of the SACT and SSC. This section of the
review will explore the assumed KMO influences on performance specifically related to the
SACT SC.
Stakeholder Knowledge, Motivation, and Organizational Influences
Knowledge and Skills
Clark and Estes’s (2008) gap analysis includes an examination of the knowledge and
skills of the people involved in the work required to achieve the goal by asking what it is that one
needs to be able to know and do to achieve the desired results (Rueda, 2011). There are four
34

types of knowledge: factual, conceptual, procedural, and metacognitive (Krathwohl, 2002).
Factual knowledge includes all the basic facts, information, specific details, and terminology
related to a particular topic (Krathwohl, 2002; Rueda, 2011). Conceptual knowledge is
understanding the factual knowledge elements’ interconnection (Krathwohl, 2002), such as
categories, classifications, principles, generalizations, theories, models, or structures (Rueda,
2011). While the factual and conceptual knowledge types identify what people must know,
procedural knowledge refers to what people need to be able to do, or the necessary skills, based
on their factual and conceptual knowledge (Clark & Estes, 2008; Krathwohl, 2002; Rueda,
2011). Procedural knowledge includes algorithms, techniques, and methods, as well as
knowledge of which procedure is most appropriate in varied contexts (Krathwohl, 2002). The
fourth knowledge type, metacognition, is the awareness, thinking, and knowledge about one’s
own thinking (Baker, 2005; Flavell, 1979; Krathwohl, 2002).
Factual Knowledge Influence
Factual knowledge refers to the required basic facts, information, specific details, and
terminology to understand a particular topic (Krathwohl, 2002; Rueda, 2011). In the context of
creating a collective impact effort, to be successful, members of the SACT SC need to know the
mission of the SACT, the SACT theory of change, and the IMTSS. Therefore, an assumed
influence that can prevent the committee from achieving its goal is a lack of knowledge of these
components. Interviews with individual members, document analysis, including meeting agendas
and the SACT theory of change document, and meeting observation served to examine this
assumption.
The assumed influence evaluated was that the committee members know the mission of
the SACT and the SACT theory of change and can define the IMTSS framework. The SACT SC
35

oversees the direction of the SACT, which was created in response to the SSC supervisors’
priority for coordinated prevention. The backbone organization, the county’s chief executive
officer’s office, developed a theory of change to align with the SACT’s mission and the board’s
prevention priority. To begin the collective impact effort, the members of the SACT SC need to
use their prior knowledge of the SACT’s mission, the theory of change, and similar past efforts
to establish a common agenda. This prior knowledge (Ambrose et al., 2010) helps define the
work’s boundaries (Hanleybrown et al., 2012).
Knowledge of the mission, priorities, and theory of change will enable members to
identify current collaboration efforts currently underway to avoid duplication. An important
ingredient for success is to build on existing collaborative efforts to address the same area of the
collective impact effort’s focus (Hanleybrown et al., 2012; Kania & Kramer, 2013). The SACT
SC agreed to use an IMTSS framework to guide collective efforts. Working towards a common
agenda requires a common understanding of what members have agreed to do together (Kania &
Kramer, 2011) and can explain why they did so (Born, 2017).
Conceptual Knowledge Influence
Conceptual knowledge is a more complex form of knowledge than declarative factual
(Krathwohl, 2002). While declarative factual knowledge is focused on facts, conceptual
knowledge demonstrates a deeper understanding of how the declarative knowledge elements
interconnect (Krathwohl, 2002; Rueda, 2011). Simply, it is the knowledge of how to classify and
differentiate between entities (Rueda, 2011). Ambrose et al. (2010) suggested that the
organization of knowledge can influence ongoing learning and application of the knowledge. In
the context of overseeing a collective impact effort, to be successful, the committee members
need to understand how the member agencies and partners will contribute to achieving the goal.
36

Therefore, an assumed influence that can act as a barrier is not understanding why cross-sector
collaboration is necessary. Interviews with individual members regarding how their agencies
contribute to the IMTSS collective impact effort served to evaluate this assumption.
The assumed conceptual knowledge influence examined here was that committee
members know why cross-sector collaboration is needed to achieve their goal and know the role
of the SACT member agencies and partners in achieving the SACT’s top priorities. Collective
impact efforts represent a shift from a single organization working for social change to a highly
structured coalition across multiple sectors (Kania & Kramer, 2011, 2013). The committee must
align the SACT’s mission, the priorities of the board of supervisors and county executives’
office, the theory of change, and the collective impact effort to demonstrate a sense of urgency
for change, which is one of the three preconditions of collective impact (Hanleybrown et al.,
2012). The urgency for change brings member agencies together. In multi-agency initiatives,
when partner members do not fully understand why working together is important and how
working together will benefit the community, they are less likely to be committed to the process
of collaboration (Townsley et al., 2004). Working together does not mean that all member
agencies do the same thing. Instead, the IMTSS collective impact approach requires agencies to
leverage their resources toward the common agenda. Steering committee members should
demonstrate systems-level thinking by articulating and clarifying the role, responsibilities, and
contributions of the collective impact effort agency partners (Townsley et al., 2004; Weaver,
2016).
Procedural Knowledge Influence
Factual and conceptual types of knowledge do not answer the question of how.
Procedural knowledge is the application of knowledge (Krathwohl, 2002) and is knowing how
37

and when to apply procedures, methods, and theories (Ambrose et al., 2010). In the context of
creating an evaluation plan, to be successful, the SACT SC members would need to know how to
communicate across agencies. Therefore, an assumed influence that can act as a barrier is not
knowing how to communicate across agencies about activities related to the IMTSS collective
impact effort. Interviews responses regarding communicating across agencies, document
analysis, and meeting observations served to assess this assumption.
The assumed procedural knowledge influence examined here was that members know
how to communicate across agencies. Continuous communication is one of the five conditions of
collective impact (Kania & Kramer, 2011). However, across a multi-agency collaborative,
communication is not always easy. Nonetheless, successful collaboration depends on successful
communication (Gilliam et al., 2016; Hanleybrown et al., 2012; Kania & Kramer, 2011, 2013;
Townsley et al., 2004; Weaver, 2016). Regular inter-agency meetings facilitate communication
opportunities (Moran et al., 2007) as members determine new and inventive ways of responding
to problems (Hicks et al., 2008). In addition to regular meetings, structured processes for
communication will also help to set the expectations for how communication should occur
(Gilliam et al., 2016) as well as what should be communicated (Kania et al., 2014).
Metacognitive Knowledge Influence
Metacognition refers to the self-directed awareness, thinking, and knowledge about one’s
cognition (Ambrose et al., 2010; Baker, 2005; Flavell, 1979; Krathwohl, 2002). This reflection
process aids with problem solving, knowing when and why to take specific actions (Rueda,
2011). In the context of overseeing a collective impact effort, to be successful, the SACT SC
members would need to reflect on how they can work differently. Additionally, to ensure an
equity focus, they also need to ensure community representation in the collective impact effort’s
38

design, implementation, and evaluation (Williams & Marxer, 2014). Therefore, an assumed
influence that can act as a barrier in achieving the goal is an inability to reflect on and learn from
past collaboration efforts to determine how to work together towards the goal. Additionally,
addressing the systemic inequities that community members experience when interacting with
different agencies requires their involvement. This cause was assessed through interviews and
document analysis to determine how committee members work differently and keep the
community at the center of the work. Finally, meeting observations also helped to assess this
cause. This research assessed committee members' ability to explain how they could collaborate
differently and describe a vision for a collective impact effort in which the community is at the
center to evaluate this assumption.
The metacognitive influence examined here was that SACT SC members think about
how they can collaborate differently under the IMTSS. Collective impact efforts begin with a
desire to solve a complex social problem. The complexity of these problems means that there are
no predetermined solutions (Kania & Kramer, 2013). As agencies work together to generate new
and innovative ideas, they need to agree on how they will collaborate differently with a focus on
the community. Steering committee members must also operate with the understanding that any
strategic plan utilized in the collective impact effort is not static but must be continuously
adapted based on shared data reflecting the needs of the community (Hanleybrown et al., 2012).
Steering committee members need to examine cross-agency results to look for patterns, make
meaning of those patterns in the context of the problem being addressed, and then brainstorm
possible solutions to implement (Kania & Kramer, 2011, 2013). Steering committee members
who embrace an iterative strategic planning process will enhance their own learning and
39

understanding, as well as that of the collective group (Kania et al., 2014). Table 2 shows the
stakeholder’s influences and the related literature.

40

Table 2
Summary of Assumed Knowledge Influences on Stakeholders Ability to Achieve the Performance
Goal
Assumed knowledge influences Type of knowledge Literature
Steering committee members
know the mission of the
SACT and the SACT theory
of change and can define the
IMTSS framework.
Factual:
terms, facts, concepts
Ambrose et al., 2010; Born,
2017; Clark & Estes, 2008;
Hanleybrown et al., 2011;
Kania & Kramer, 2011, 2013;
Krathwohl, 2002; Rueda, 2011
Steering committee members
know why cross-sector
collaboration is needed to
achieve their goal.
Conceptual:
categories, process
models, principles,
relationships
Ambrose et al., 2010; Barberg,
2015; Hanleybrown et al.,
2012; Kania & Kramer, 2011,
2013; Krathwohl, 2002; Rueda,
2011; Townsley et al., 2004;
Weaver, 2016
Steering committee members
know the role of the SACT
member agencies and the
partners in the IMTSS.
Conceptual:
categories, process
models, principles,
relationships
Ambrose et al., 2010; Barberg,
2015; Hanleybrown et al.,
2012; Kania & Kramer, 2011,
2013; Kania et al., 2014;
Krathwohl, 2002; Rueda, 2011;
Townsley et al., 2004; Weaver,
2016
Steering committee members
know how to communicate
with each other across
agencies.
Procedural:
categories, process
models, principles,
relationships
Ambrose et al., 2010; Gilliam,
Counts, & Garstka, 2014;
Hanleybrown et al., 2012;
Hicks et al., 2008; Kania &
Kramer, 2011, 2013;
Krathwohl, 2002; Moran et al.,
2007; Rueda, 2011; Townsley
et al., 2004; Weaver, 2016
Steering committee members
think about how they can
collaborate differently under
the IMTSS collective impact
effort.
Metacognitive Ambrose et al., 2010; Clark &
Estes, 2008; Kania & Kramer,
2011; Krathwohl, 2002; Rueda,
2011

41

Motivation
Motivation has been identified as another probable root cause of organizational problems
(Clark & Estes, 2008). Learning and motivation are reciprocal. While knowledge represents an
understanding of what and how to accomplish a given task, motivation is the internal will that
pushes an individual or group to take the necessary action to accomplish the given task (Pintrich,
2003; Rueda, 2011). After a goal has been established and knowledge of the stakeholders has
been determined, the next step is to assess stakeholder motivation.
General Theory
Three indices of motivational behavior have been identified to determine a motivation-
performance connection: the active choice to pursue a goal or accomplish a task, the persistence
through distractions or challenges, and the mental effort to learn new skills and maintain
confidence while working on the task or pursuing the goal (Clark & Estes, 2008; Mayer, 2011;
Rueda, 2011). Procrastination, avoidance, being easily distracted away from the task, under-
confidence, and overconfidence all contribute to motivation problems in organizations (Clark &
Estes, 2008; Rueda, 2011). Motivation has both internal, or cognitive, influences and external, or
sociocultural, influences like self-efficacy, attribution, and value (Rueda, 2011).
Stakeholder/Topic-Specific Factors
In relation to a collective impact effort, if SACT SC members enter into the collective
impact collaboration with a preconceived solution, they are not likely to seek out the new
knowledge required to generate new and innovative ideas (Kania et al., 2014). This lack of
mental effort could indicate a motivational gap (Clark & Estes, 2008; Rueda, 2011). Steering
committee members may exhibit active choice when they regularly meet to determine the
common agenda, choose how to measure progress, and decide collectively what activities their
42

individual agencies will lead (Kania & Kramer, 2011). When the members procrastinate, argue
against, or even avoid any of these choices, the motivational problem may be associated with
active choice (Clark & Estes, 2008). Collective impact requires a long-term commitment (Kania
et al., 2014). Steering committee members need to be confident that their choice to collaborate
across agencies will help them to achieve their goal. A lack of confidence in the cross-agency
collaborative will indicate a motivation gap in persistence (Clark & Estes, 2008).
Value
One motivational variable is value, which refers to the perceived importance, enjoyment,
usefulness, and cost associated with a task (Rueda, 2011) that influence an individual’s choice to
pursue and persist through a task (Clark & Estes, 2008). Value serves as a motivational influence
because the higher the perceived value, the higher the chance of one starting and persisting
through a task (Rueda, 2011). Clark and Estes (2008) identified three types of value: interest
value, relating to interest in a particular task; skill value, tasks that challenge an individual’s
special skills; and utility value, a focus on the benefits of the end-results.To be successful, SACT
SC members must value the alignment between the SACT’s common agenda and their individual
agency goals. The assumed influence of value was examined through interview questions on how
the SACT’s common agenda benefits individual agencies. Explanations regarding how agency
goals align with the SACT’s common agenda served to evaluate this assumption.
The value-related influence examined here was that, individually and collectively,
members need to see the value of the alignment between the SACT’s common agenda and their
agency goals. Collective impact efforts bring together leaders from across multiple agencies to
find new ways to work together towards a common agenda, a shared vision, aimed at improving
outcomes (Kania & Kramer, 2011, 2013). The collaboration that occurs in collective impact
43

efforts does not require all participants to do the same thing, but it does require participants to
maximize and leverage their individual agency’s strengths to support the collective goal through
mutually reinforcing activities, the third condition of collective impact (Barberg, 2015; Kania &
Kramer, 2011). When there is no alignment, participants have to balance working towards their
agency goals while also working towards the goals of the collective impact effort, creating a
motivational barrier to accomplishing the goal (Gilliam et al., 2016; Moran et al., 2007).
Therefore, to increase the success of mutually reinforcing activities, the SACT SC works to
ensure that there is alignment between each participating agency’s goals and the goals of the
IMTSS (Barberg, 2015; Kania et al., 2014; Kania & Kramer, 2011).
Self-Efficacy
Self-efficacy refers to a person’s belief in their own capabilities (Bandura, 1997).
Bandura (2012) identified four influences that shape a person’s self-efficacy: mastery
experiences that build resilience, social mastery that comes from seeing others succeed, social
persuasions that build resolve and belief in one’s ability to succeed, and how one responds to and
manages stress. Self-efficacy beliefs influence motivation and perseverance through the journey
to accomplishing set goals (Bandura, 2012). Similarly, collective efficacy refers to a group’s
confidence in its ability to perform successfully (Bandura, 1977). In the context of implementing
and overseeing a collective impact effort, members must have individual and collective
confidence that they can effectively work together on the SACT’s common agenda. These
assumed influences were examined via interviewees’ responses regarding their individual
motivation to work towards the SACT’s goals and their collective belief in their ability to
succeed. Participants’ ability to explain how their work on the SACT common agenda enhances
their agency’s work served to evaluate this influence.
44

The efficacy-related influenced assessed in this study was that, individually and
collectively, SACT SC members need to be confident that they can effectively work together to
accomplish the goal. Collective impact efforts begin with cross-agency collaboration around a
complex social need. After participants agree on the common agenda, or shared vision, they
work on shared measurements, mutually reinforcing activities, and continuous communication
begins (Kania & Kramer, 2011). Individual participants need to be confident that their attention
to the common agenda does not negatively impact the purpose of their individual agencies or
diminish their professional identities and contributions (Moran et al., 2007). As already noted, as
a part of the value influence, members need to ensure that the activities of the collective impact
effort reinforce and fit with the activities of the participating agencies (Kania et al., 2014).
Building trusting relationships is a critical component of successful collective impact efforts
(Hanleybrown et al., 2012; Kania et al., 2014). Steering committee members must believe that all
members have equal influence over the initiative (Hicks et al., 2008). Gilliam et al. (2016)
suggested that participants can build trusting relationships with each other by adopting a growth
mindset about their abilities to work together successfully.
The SACT SC members also need to believe in their ability to be collectively successful.
To accomplish this, the members need to stay focused on the whole system while seeking
opportunities for alignment between the IMTSS collective impact effort and the individual
agencies (Weaver, 2016). As noted earlier in this review, collective impact efforts are created to
solve complex social problems (Kania & Kramer, 2011). The members must believe that their
collective commitment to the common agenda of the IMTSS collective impact effort does not
minimize the importance of the work and purpose of their agencies but demonstrates that isolated
impact is not sufficient to address the complex social problem (Gilliam et al., 2016; Kania et al.,
45

2014). The nature of a collective impact effort calls for cross-agency collaboration toward the
common agenda, as well as coordination of the individual agencies’ activities that address both
the goals of the agency and the initiative (Kania & Kramer, 2011). Table 3 shows the
stakeholder’s influences and the related literature.

Table 3
Summary of Assumed Motivation Influences on Stakeholders’ Ability to Achieve the Performance
Goal
Assumed motivation influences Motivation constructs Literature
Individually and collectively,
steering committee members
see the value of the
alignment between the
SACT’s common agenda and
their agency goals.
Utility value Barberg, 2012; Clark & Estes,
2008; Eccles, 2006; Gilliam et
al., 2016; Kania & Kramer,
2011, 2013; Kania et al., 2014;
Moran et al., 2007; Pintrich,
2003; Rueda, 2011
Individually, steering
committee members are
confident that they can
effectively work together to
accomplish the SACT’s
stated goal.
Self-Efficacy Bandura, 1977; 1997; 2012;
Gilliam et al., 2016;
Hanleybrown et al., 2012;
Hicks, 2008; Kania & Kramer,
2011; Kania et al., 2014; Moran
et al., 2007; Weaver, 2016
Collectively, steering
committee members are
confident that the group’s
collective efforts will be
effective in accomplishing
the SACT goals.
Collective efficacy Bandura, 1977; 1997; 2012;
Gilliam et al., 2016;
Hanleybrown et al., 2012;
Hicks, 2008; Kania & Kramer,
2011; Kania et al., 2014; Moran
et al., 2007; Weaver, 2016

46

Organizational Factors
An organization can still experience a performance gap relative to its goal even with
people who are knowledgeable and motivated. For this reason, the third component of Clark and
Estes’s (2008) gap analysis framework addresses the organizational factors that may prevent the
achievement of the performance goals. Organizational performance gaps can surface due to
inefficient and ineffective material resources and work processes (Clark & Estes, 2008). In
addition to material resources and work processes, an organization’s performance is influenced
by the organizational culture, specifically the cultural model and the cultural settings. The
cultural model refers to the shared beliefs, values, and customs for how things work within an
organization (Bolman & Deal, 2017; Deal & Kennedy, 1983; Gallimore & Goldenberg, 2001;
Schein, 1992), while the cultural settings offer insight into why the organization functions the
way it does (Gallimore & Goldenberg, 2001).
Resources
Resources within an organization refer to the tangible supplies, materials, tools, and
equipment needed to achieve the organizational goal (Clark & Estes, 2008). In the context of
implementing the IMTSS collective impact effort, the SACT SC members need the time for
planning and coaching support. They also need the funding to acquire the tools and materials
needed to accomplish the goal. Validation of the assumed influence of resources occurs through
interviews and document analysis. Interviews and analysis of budgets and meeting notes
identified the allocation of resources to evaluate this assumption.
The assumed influence related to resources evaluated in this study was that the
organization provides the SACT SC with the funding and time to implement collective impact to
achieve its priorities. Successful collective impact efforts must meet three pre-conditions before
47

beginning the implementation process (Hanleybrown et al., 2012). One of these preconditions is
having adequate financial resources to last for 2 to 3 years at a minimum. Having a backbone
support organization is one of the five conditions of collective impact (Kania & Kramer, 2011).
The backbone support organization provides the project manager, data manager, and facilitator to
guide the members through the highly structured collective impact processes (Kania & Kramer,
2011). The SSC-CEO has assumed the role of backbone support. The SACT SC overseeing the
collective impact effort also requires time to meet regularly to establish a common agenda, agree
upon shared measurement, build a commitment to mutually reinforcing activities, and establish
the structure for continuous communication (Gilliam et al., 2016; Kania & Kramer, 2011, 2013;
Townsley et al., 2004). To reduce the anxiety that occurs with change, the members must have
the time to learn about the steps needed to implement IMTSS collective impact effort (Schein &
Schein, 2017).
Cultural Setting
Culture in an organization refers to the core values, beliefs, emotions, and processes that
individuals learn and develop over time (Clark & Estes, 2008). The cultural setting refers to the
place or social contexts in the organization in which these cultural models develop (Gallimore &
Goldenberg, 2001; Rueda, 2011). In the context of launching the IMTSS collective impact effort,
SSC needs to provide regular opportunities for the SACT SC members to collaborate around the
initiative. Validation of the assumed influence of cultural setting occurred through interviews.
The cultural setting influence examined herein was that the organization provides the
SACT SC the resources to implement collective impact to achieve its priorities. Collective
impact efforts are highly structured collaborative efforts that bring together people from multiple
agencies and organizations working toward a common agenda (Kania & Kramer, 2011). To
48

achieve large-scale impact, members of the collective initiative need protected time to
collaborate to develop the strategic action framework (Gilliam et al., 2016; Hanleybrown et al.,
2012; Townsley et al., 2004) based on the common agenda of the collective impact effort. After
determining the strategic action framework, working groups operationalize the strategic action
plan (Hanleybrown et al., 2012). The SSC needs to commit to allowing staff from the various
agencies and organizations to participate in these working groups.
Cultural Models
Cultural models such as values, practices, and policies develop over time (Bolman &
Deal, 2017) and help to shape the structure of the organization (Rueda, 2011). In the context of
launching the IMTSS collective impact effort, the SACT SC members need the support of the
Siloed Services Agency to utilize innovation and creativity as they work to address a complex
problem through their collective efforts. Interviews with individual members served to validate
or invalidate this assumption.
The influence related to cultural models evaluated in this study was that the organization
promotes innovation and creativity to solve problems. Unlike technical problems that tend to be
more easily solved, solutions to complex problems are not predetermined (Kania & Kramer,
2011). In an assessment of successful collective impact efforts, Kania and Kramer (2013) found
that each successful initiative discovered a new approach to solve their complex problem.
Working together toward the common agenda, members of the collective can find innovative and
creative solutions to collectively address the problem (Born, 2017; Kania et al., 2014). Table 4
shows the stakeholder’s influences and the related literature.

49

Table 4
Summary of Assumed Organization Influences on Stakeholders Ability to Achieve the
Performance Goal
Assumed organization
influences
Organizational factors Literature

The organization provides the
steering committee the
funding and time to
implement the IMTSS to
achieve its priorities.
Resources Clark & Estes, 2008;
Hanleybrown et al., 2012;
Gilliam et al., 2016; Kania &
Kramer, 2011; Schein &
Schein, 2017; Townsley et al.,
2004
The organization provides the
steering committee the
resources to develop the
implementation plan for the
IMTSS.
Cultural setting Clark & Estes, 2008; Gallimore
& Goldenberg, 2001;
Hanleybrown et al., 2012;
Kania & Kramer, 2011; Kania
& Kramer, 2013; Kania et al.,
2014; Rueda, 2011
The organization promotes
innovation and creativity to
solve problems.

Cultural model

Bolman & Deal, 2017; Born,
2017; Clark & Estes, 2008;
Gallimore & Goldenberg,
2001; Hanleybrown et al.,
2012; Kania & Kramer, 2011;
Kania & Kramer, 2013; Kania
et al., 2014

This literature review serves two purposes. The first is to identify the root causes of
exclusionary disciplinary practices against Black boys that ultimately contribute to the school-to-
prison pipeline and the intergenerational cycle of disadvantage. Second, the literature review
presented literature about collective impact and how collective impact collaboration can address
this complex social problem. The evaluation is organized using Clark and Estes’s (2008) gap
50

analysis framework. Performance gaps will be identified by analyzing the SACT SC members’
KMO factors as described in Chapter Three.

51

Chapter Three: Methodology
The purpose of this project was to evaluate the degree to which the SACT was meeting
its goal of fully implementing the IMTSS using the collective impact framework. The IMTSS is
a collective impact effort aimed at dismantling the school-to-prison pipeline by eliminating the
intergenerational cycle of disadvantage. The study focused on the SACT SC members’ readiness
to engage in collective impact by evaluating KMO influences related to achieving the
organizational goal. While a complete performance evaluation would focus on all stakeholders,
for practical purposes, the stakeholder of focus in this analysis was the SACT SC. Three
questions guided this study:
1. What are the SACT steering committee members’ knowledge, motivation, and
organizational resources related to creating a plan to implement the IMTSS?
2. What is the interaction between organizational culture and context and the steering
committee members’ knowledge and motivation?
3. In what way is the SACT steering committee positioned to engage in collective impact?
Conceptual and Methodological Framework
The conceptual and methodological framework used to guide this evaluation was Clark
and Estes’s (2008) gap analysis model. This improvement model provides a comprehensive
approach to evaluating performance gaps by evaluating the KMO factors within organizations.
The first step of this model is to identify the gap by comparing the organization’s goal to its
current status. Next, an evaluation of the gap occurs by analyzing the KMO factors to identify
the root causes of the gap (Clark & Estes, 2008). Based on the identified KMO causes, solutions
are recommended to close the gaps. The final step of the model is to implement and evaluate the
52

recommended solutions. This study will not include the implementation or evaluation of the
recommended solutions.
As this study aimed to evaluate the SACT SC’s progress toward its goal, this gap analysis
model is well suited to determine the gap and evaluate the KMO root causes. The committee
members’ knowledge was evaluated to determine what they know about the overall mission of
the SACT and about the cross-agency collective impact effort. Since achieving SACT’s goal
relies on authentic cross-agency collaboration, the SACT SC members’ motivation to work
together was also evaluated. Finally, the information gathered regarding knowledge and
motivation was juxtaposed with organizational factors that contribute to the performance gap.
This evaluation helped to identify solutions related to the committee members’ knowledge of
what they need to do, motivation to remain committed to accomplishing the overall goal, and the
removal of organizational barriers to cross-collaboration work. Clark and Estes’s (2008) gap
analysis process, illustrated in Figure 2, represents a roadmap for organizational change based on
identifying root causes related to KMO gaps that leads to the generation of effective solutions for
performance improvement.

53

Figure 2
Gap Analysis Process

Assessment of Performance Influences
Clark and Estes’s (2008) gap analysis process begins by identifying the root causes of
performance gaps by analyzing the contributing KMO factors. In relation to this evaluation
study, it is important to examine these factors to determine if the SACT SC members have the
KMO support to accomplish their goal. The information gathered through this analysis was used
to develop an evaluation plan with recommended solutions to decrease the performance gap.
Knowledge Assessment
Clark and Estes’s (2008) gap analysis process includes examining the knowledge and
skills of the people involved in the work towards the desired goal. Krathwohl (2002) identified
four types of knowledge: factual, conceptual, procedural, and metacognitive. Each of these
knowledge types was analyzed to determine the specific knowledge influences affecting the
54

members' understanding of the purpose of the cross-collaboration collective impact effort. These
influences are addressed in the evaluation plan.
Factual Knowledge
Factual knowledge refers to the basic facts and information of a topic (Krathwohl, 2002;
Rueda, 2011). Therefore, to assess this type of knowledge, information was gathered through
questions, document analysis, and meeting observations so that the study participants
demonstrated their ability to recall certain facts related to the initiative and their understanding of
these facts (Anderson & Krathwohl, 2001). Collective impact efforts begin with the development
of a common agenda. To work towards this common agenda, the committee members need to
know the SACT’s mission and theory of change and define the IMTSS framework implemented.
Conceptual Knowledge
Building on the foundation of factual knowledge, conceptual knowledge is the knowledge
of classification (Krathwohl, 2002; Rueda, 2011). Therefore, to assess this type of knowledge
and building on the assessment of declarative factual knowledge, information was gathered
through interviews to examine participants’ understanding of the interconnected functionality of
the basic factual elements that exist in a larger structure (Anderson & Krathwohl, 2001). As the
committee oversee the IMTSS collective impact effort, it is important that its members
understand why cross-sector collaboration is needed and the roles of each member agency in its
implementation.
Procedural Knowledge
The application of knowledge is procedural knowledge (Ambrose et al., 2010;
Krathwohl, 2002). Therefore, to assess this type of knowledge, information was gathered through
questions and document analysis to examine study participants’ knowledge of how to do
55

something (Anderson & Krathwohl, 2001). One of the five conditions of collective impact
efforts is continuous communication. As such, the SACT SC members need to know how to
communicate across agencies.
Metacognitive Knowledge
Metacognitive knowledge is an awareness of one’s thinking (Ambrose et al., 2010;
Baker, 2005; Flavell, 1979; Krathwohl, 2002). Therefore, to assess this type of knowledge,
information was gathered through interviews, document analysis, and meeting observations to
evaluate participants’ understanding and awareness of their cognition (Anderson & Krathwohl,
2001). The members will need to reflect on past collaboration efforts to determine how they can
work together more successfully towards the goal. Table 5 provides an overview of the methods
and interview questions used and the artifacts reviewed to gather information.

56

Table 5
Summary of Knowledge Influences and Method of Assessment
Assumed knowledge influences Interview item Documents
Factual
Steering committee members know
the SACT’s mission and theory
of change and can define the
IMTSS framework.
What is the mission of SACT?
What is the SACT theory of
change?
Please describe the IMTSS
initiative.
SACT steering
committee meeting
agendas and notes;
SACT theory of
change document;
Conceptual
Steering committee members know
why cross-sector collaboration is
needed to achieve their goal.
Why is collaborating across
agencies necessary for
building out the IMTSS?

Steering committee members know
the role of the SACT member
agencies and the partners in the
IMTSS.
How would you describe the
role of the various cross-
agency partners in designing
and implementing the
IMTSS?

Procedural
Steering committee members know
how to communicate with each
other across agencies.
Please describe how your
agency currently
communicates with the other
agencies to establish
partnerships.
SACT steering
committee meeting
agenda and notes
Metacognitive
Steering committee members think
about how they can collaborate
differently under the IMTSS
collective impact effort.
Considering past collaboration
efforts, how would you think
about working differently
moving forward?
SACT steering
committee meeting
agenda and notes

57

Motivation Assessment
After assessing to identify knowledge gaps, Clark and Estes (2008) suggested that the
next step is to assess motivation gaps. Learning and motivation are reciprocal. While knowledge
represents an understanding of what and how to accomplish a given task, motivation is the
internal will that pushes an individual to start a task and persist through barriers to accomplish
the goal (Pintrich, 2003; Rueda, 2011). Three key variables shape motivation: value, self-
efficacy, and attribution (Rueda, 2011). This section reviews the assessment of value and self-
and collective efficacy to identify the motivational influences on the SACT SC members to
engage in the IMTSS collective impact effort.
Value
There are three types of value: interest, skill, and utility (Clark & Estes, 2008). Interest
value refers to an individual’s intrinsic motivation to increase their expertise through task
completion or skill mastery. Motivation to complete a task also comes from skill value, which
allows an individual to demonstrate specialized skills. The third type of value is utility value
which describes when an individual is motivated by the benefits of task completion. Variables
that influence value include perceived importance, enjoyment, usefulness, and cost associated
with a task (Rueda, 2011). Stakeholders were assessed on their individual and collective utility
value. To assess utility value, they were asked to describe how their work on the IMTSS
collective impact effort aligns with the work they perform in their agencies. They were also
asked to describe how the goal of the IMTSS collective impact effort aligns with their agency’s
goals.

58

Self-Efficacy
Self-efficacy refers to a person’s belief in their capabilities (Bandura, 1997). Similarly,
collective efficacy is a group’s confidence in its ability to accomplish a task (Bandura, 1977). In
the context of motivation, self- and collective efficacy have a direct influence on motivation and
perseverance (Bandura, 2012). Stakeholders were assessed on their individual and collective
efficacy. To assess self-efficacy, interviewees were asked to share their confidence in partnering
with another agency to fulfill a community need. To assess collective efficacy, they were asked
to describe how confident they are that the SACT member agencies can stay focused on the
common agenda.
Table 6 provides an overview of the methods, interview questions, and artifacts reviewed
to gather information related to each motivation type reviewed in this section.

59

Table 6
Summary of Motivation Influences and Method of Assessment
Assumed motivation influences Interview item
Utility value
Individually and collectively,
steering committee members see
the value of the alignment
between the SACT’s common
agenda and their agency goals.

How does your work on IMTSS align with the work
you do in your agency?

How will the integrated multi-tiered system of support
benefit the work of your agency?
Self- and collective efficacy
Individually, steering committee
members are confident that they
can effectively work together
with other partner agencies.
Tell me about a time when you felt confident
partnering with another agency to fill a need in the
community.
Collectively, steering committee
members are confident that the
group’s collective efforts will be
effective in accomplishing the
SACT goals.
On a scale of 1 to 5, with 1 being no confidence at all
and 5 being the highest confidence, how confident are
you that this group can stay focused on the common
agenda?

Organization/Culture/Context Assessment
An organization’s problems and performance gaps can also have root causes related to
organizational factors such as inefficient and ineffective material resources, organizational
cultural model, and cultural setting (Clark & Estes, 2008). Therefore, a comprehensive gap
analysis plan includes assessing the organizational factors that contribute to the problem. When
the organizational barriers are due to material resources, the organization can implement
technical solutions to address the gap. However, when the organizational barriers have root
causes tied to the organizational culture, an in-depth analysis is needed to better understand the
60

cultural setting and the cultural models that influence the overall organizational culture (Bolman
& Deal, 2017; Deal & Kennedy, 1983; Gallimore & Goldenberg, 2001). This section reviews the
assessment of the organizational influences of resources, cultural settings, and cultural models
discussed in Chapter Two.
Resource
An organization’s tangible supplies, materials, tools, and equipment can impact overall
performance (Clark & Estes, 2008). Assessing organizational resources requires that the
stakeholders are aware of and know how to access these resources. Therefore, to assess the
resources, stakeholders were asked to quantify the amount of time they spent working on the
IMTSS collective impact outside of the SC meetings. Additionally, meeting agendas and notes
were analyzed to identify funding allocations and commitment.
Cultural Settings
The cultural setting refers to the place or social contexts within an organization in which
the cultural models develop (Gallimore & Goldenberg, 2001; Rueda, 2011). In addition to time
to meet and plan, SACT SC members need to feel empowered to ask for additional support. To
assess the organization's cultural settings, the participants were asked to identify what additional
support they needed to fulfill their role in implementing the IMTSS.
Cultural Models
Cultural models include values, practices, and policies that develop over time (Bolman &
Deal, 2017) and shape the overall structure of an organization (Rueda, 2011). The complexity of
the social problem of the school-to-prison pipeline that contributes to the intergenerational cycle
of disadvantage requires innovative and creative solutions. While the IMTSS represents
innovation and creativity, collective impact will also require innovation and creativity. To assess
61

the organization’s cultural models, interviewees were asked how innovation and creativity are
encouraged for problem solving. Table 7 provides an overview of the methods and survey
questions used to gather information related to each organizational influence reviewed in this
section.

Table 7
Summary of Organization Influences and Method of Assessment
Assumed organization influences Interview item Document
Resources
The organization provides the
steering committee the funding and
time to implement the IMTSS to
achieve its priorities.
Outside of the scheduled
steering committee
meetings, how much time
do you spend on this
initiative?
SACT meeting agendas
and notes
Cultural Setting
The organization provides the
steering committee the resources to
develop the implementation plan
for the IMTSS.
Thinking about your role as
a steering committee
member, what additional
resources do you need?

Cultural Model
The organization promotes
innovation and creativity to solve
problems.

How supportive is county
leadership of using
innovation and creativity
to solve problems?

62

Participating Stakeholders and Sample Selection
The stakeholder group of focus for this paper was the SACT SC. Thirteen members
represent each sector of specific areas of the child- and family-serving systems, including a
grassroots community organization. As each member has an alternate member, the alternate was
only included in the study if the core member was not available to participate.
Sampling
The sampling strategy for this study was purposeful and nonrandom to maximize the
examination of the stakeholder knowledge and motivation and the organizational culture and
group dynamics (Creswell, 2014; Patton, 2015). I specifically chose the SACT SC members
because of their leadership role in the implementation of IMTSS using collective impact and
their leadership roles at their agencies.
Recruitment
The sampling strategy sought out all 13 members. The recruitment process included an
individual email from me to each member to explain the purpose of the study with a solicitation
request for each member's participation in the study. Each participant was offered a list of dates
and times from which they selected a time for the interview. As I am a member of the SACT and
also served as a lead for one of the SACT working groups, there was a peer relationship between
the participants and me.
Instrumentation
Interview Protocol
The design for this study (Appendix A) provided individual interviews with the members.
As this was a qualitative case study, interviews were selected to conduct an in-depth analysis of
the participants’ knowledge, motivation, and understanding of the organizational factors
63

influencing their readiness to engage in collective impact (Clark & Estes, 2008; Merriam &
Tisdell, 2016).
Observation Checklist
I, as a member of the SACT, assumed a full participant-observer stance (Merriam &
Tisdell, 2016). Standard qualitative field notes were collected as I overtly conducted the
observations. As this was a qualitative case study, observations were selected to observe the
members’ interactions in their regular monthly meetings.
Document Analysis Protocol
Document analysis was used to systematically review and evaluate documents to gain
understanding (Bowen, 2009). Document analysis contributed to measuring the participants’
KMO factors in the context of implementing the IMTSS. Document analysis included a review
of meetings and meeting notes, the SACT organizational chart, the SACT theory of change, and
meeting artifacts describing the IMTSS.
Data Collection
Following the University of Southern California Institutional Review Board’s (IRB)
approval, participants were solicited via a group email advising of the observation data that
would be collected. Participants were solicited via direct and individual email communication
from me to participate in the study. All participants were provided with an informed consent
form explaining the procedures and purpose of the study. Once consent was given, the interview
was scheduled. A request for pertinent documents was submitted detailing how data would be
collected and shared. To answer the research questions, data were collected from in-depth
individual interviews, meeting observations, and documents. In addition to providing
information on the knowledge and motivational influences, data also provided information on
64

readiness to engage in collective impact, which addresses the third research question. This
section provides more details on the methods used to collect data for this study.
Interviews
The interview protocol was created to measure the SACT SC members’ KMO factors.
All current members were invited to participate in the interviews. The interviews were conducted
through an online meeting platform due to current social distancing guidelines resulting from the
public health crisis. I conducted the interviews and obtained consent for recording each interview
prior to the interview. The interview recordings and my notes were used for data analysis.
Observations
I observed three meetings and one SACT meeting using an online platform. Data related
to participation, activities, interactions, and conversation were recorded (Merriam & Tisdell,
2016). Prior to the observations, members were invited to participate via a group email. The
email invitation was sent 2 weeks prior to each observation. As the meetings occurred virtually, I
was visible to all participants on the computer screen. Each meeting lasted between 1 and 2
hours.
Document Analysis
I analyzed documents to measure the KMO influences on members when developing a
plan to implement the IMTSS utilizing collective impact. The documents included SACT and SC
committee meeting agendas and subsequent meeting notes, SACT organizational chart, the
SACT theory of change, and meeting artifacts describing the IMTSS. I requested access to the
documents from the SACT coordination manager. All protocols for requesting public documents
were followed.

65

Data Analysis
Qualitative data analysis is an iterative process (Merriam & Tisdell, 2016). In this
qualitative evaluation study, analysis occurred using the data gathered from in-depth individual
interviews, meeting observations, and documents. Prior to the interviews, I developed a
codebook using Google sheets. Within the codebook, I generated a color-coded list of a priori
codes related to each assessed influence from Clark and Estes’ (2008) KMO gap analysis
framework and the four assessed collective impact conditions. I added open codes, which
emerged during the interviews, to the codebook. The codebook was loaded into ATLAS.ti to
conduct qualitative analyses. As I conducted interviews were, I reviewed each anonymized
transcript to ensure transcription accuracy and then uploaded it into ATLAS.ti. Documents and
observation notes were also uploaded into ATLAS.ti.
I reviewed each transcript a minimum of three times. During the first review, I focused
on identifying themes related to the KMO influences. For the second review, I focused on
identifying themes related to the four assessed collective impact conditions. During the third
review, I identified axial codes to organize the data into themes to reveal assets or needs related
to the KMO influences and collective impact conditions. I cross-referenced the patterns and
trends identified in the interview data with the observation data as well as the documents and
artifacts. This analysis aimed to identify and classify gaps related to the KMO influences and to
examine how the KMO assets could be leveraged to increase readiness for collective impact.
Trustworthiness of Data
Throughout this study, many deliberate steps were taken to ensure that the findings were
credible and trustworthy. Triangulation was used to validate the findings. Triangulation uses
various methods to collect information from a variety of sources (Maxwell, 2012) to justify the
66

themes presented in the findings (Creswell, 2014). For this study, data analysis included
triangulating interview and observation data and data from the secondary sources of the SACT,
including monthly SACT SC meetings and meeting notes, the SACT organizational chart, the
SACT theory of change, and meeting artifacts describing the IMTSS.
To maximize validity and reliability, I used member checks and peer review (Merriam &
Tisdell, 2016). For member checks, I solicited feedback on some emerging findings with some of
the interview participants to ensure that their responses were accurately represented in the
findings. For peer review, I asked peers familiar with collective impact to ensure that the data
collected provided feedback on the four conditions of collective impact that were studied as a
part of this evaluation study.
Role of Investigator
I served as one of the two co-leads for one of the SACT working groups and did not
supervise any of the study participants. My role in the prevention working group provided her
with a valuable perspective. I purposefully selected the sample for the study, gathered SACT
artifacts and documents, collected informed consent, and conducted the interviews. I am the
main point of contact for the study and the person who reported the data analysis and findings in
Chapter Four.
Limitations
The limitations of this study were sample size and instrumentation. There are only 13
members, and 12 agreed to participate. A small study population can impact the validity of the
conclusions. The current public health crisis, COVID-19, shifted all meetings into a virtual
setting. Observation of interactions among members was limited in this setting. Another
limitation was the sole focus on the members. A more comprehensive study would include
67

information from community members, community-based organizations, the larger SACT group,
and the board of supervisors to gain more insight into the need for an IMTSS and collective
impact for its implementation.
68

Chapter Four: Results and Findings
The purpose of this project was to evaluate the degree to which the SACT was meeting
its goal of fully implementing the IMTSS using the collective impact framework. The IMTSS is
a collective impact effort aimed at dismantling the school-to-prison pipeline by eliminating the
intergenerational cycle of disadvantage. The study focused on the SACT SC members’ readiness
to engage in collective impact through an evaluation of their knowledge and motivation and the
organizational influences related to achieving the larger organizational goal of streamlining and
improving the quality of services provided to the community. Qualitative data were collected to
validate the assumed KMO influences identified in Chapters Two and Three. Specifically,
interview, observation, and artifact data were collected to understand the KMO challenges the
members encounter as they prepare to implement the IMTSS collective impact effort.
Data collection consisted of qualitative interviews. Interviews were conducted
individually using the Zoom video conferencing format. Each interview lasted from 35 to 60
minutes and utilized a semi-structured interview protocol. Three SACT SC meetings were also
observed. Two meetings were adhoc planning meetings for the upcoming launch of the
demonstration IMTSS sites, and one was the regular monthly meeting. SACT artifacts, including
past meeting agendas and meeting notes and the SACT theory of change, were also reviewed to
identify evidence of the assumed influences.
This chapter is organized by the following study questions that also guided data
collection:
1. What are the SACT steering committee members’ knowledge, motivation, and
organizational resources related to creating a plan to implement the IMTSS?
69

2. What is the interaction between organizational culture and context and the steering
committee members’ knowledge and motivation?
3. In what way is the SACT steering committee positioned to engage in collective impact?
Participating Stakeholders
I invited each of the 13 steering committee members to participate. One declined to
participate. Each of the remaining members (N = 12) agreed to participate and completed a 45- to
60- minute interview. They also agreed to be observed in three of their meetings and a larger
SACT meeting for a total of four meeting observations. The one member who did not want to be
included in the meeting observations turned their camera off during the meetings. Three
members were male-identified, and nine were female-identified. Each participant had been in
their field for over 10 years, but their years of service to SSC ranged from 3 to 22. The 12
participants represented nine different agencies or organizations. Two of the participants worked
together at one agency, and three other participants worked at another agency in different
divisions. Data on age and race were not collected.
Due to the small stakeholder group, additional information is withheld to maintain
anonymity. For increased anonymity and confidentiality, participants’ agencies will not be
named or described, but each provides direct services to children and families. A pseudonym was
created for participants to further protect their identities. In the analyses that follow, study
participants are referred to by the following pseudonyms: Andrew, Beth, Cynthia, Diane, Eric,
Francine, Gail, Hannah, Irene, Jack, Kathy, and Lucy.
Determination of Assets and Needs
The determination of assets and needs materialized from the analysis of the interviews,
documents, and meeting observations. To validate the assets and needs, I relied on data
70

triangulation. The first component was the extensive literature review presented in Chapter Two
to identify the knowledge and motivation related to implementing the collective impact effort
and the organizational factors that can impact readiness for collective impact. The second
component was the individual interviews. The third component consisted of document analysis
and meeting observations.
The literature review occurred prior to the data collection and analysis. From the
literature review, I aligned the emerging themes with the KMO influences (Clark & Estes, 2008).
I then used the emerging themes to create a codebook of a priori codes aligned to each of the
KMO influences and the four assessed conditions of collective impact. I included additional axial
and open codes when analyzing the interview transcripts, documents, and meeting observation
notes. Saturation was reached after all 12 participants were interviewed. Across all interviews,
several common themes emerged. The document analysis and meeting observations supported
the emerging themes and provided clarity on the history of the use of the IMTSS and collective
impact in SSC.
Research Question One: KMO Influences
This section presents findings related to the first research question. I assessed
participants’ presumed KMO influences related to implementing the IMTSS collective impact
effort through interviews, document analysis, and meeting observations. The following sections
present the findings from the interviews, document analysis, and meeting observations for each
assumed cause within the categories of factual knowledge, conceptual knowledge, procedural
knowledge, metacognitive knowledge, utility value, self-efficacy, and collective efficacy.
Findings are also presented for the organizational categories of resources, cultural setting, and
cultural model. A minimum of seven interviewees, or at least 55%, was needed to demonstrate a
71

lack of knowledge, motivation, or organizational factors in order to validate the influence as a
gap.
Results and Findings for Factual Knowledge
Interviews and document analysis were used to assess the SC’s factual knowledge. Based
on the protocols listed in Chapter Three, meeting observations were not conducted to assess this
influence. Interviewees were asked about the SACT’s mission and theory of change and the
IMTSS framework. The findings have been organized and evaluated to assess if there is a gap
regarding the assumed causes.
The factual knowledge influence was that SACT SC members know the SACT’s mission
and theory of change and can define the IMTSS framework.
Interview Findings
To use collective impact to implement the IMTSS, SACT SC members must demonstrate
knowledge of the core elements needed to establish the collective impact conditions of a
common agenda and mutually reinforcing activities. Therefore, interviewees were asked to
describe the SACT’s mission and theory of change and to define the IMTSS framework. Overall,
all 12 participants demonstrated knowledge of the SACT mission and theory of change and could
define the IMTSS.
When describing the mission of SACT, all interviewees described SACT as a multi-
agency partnership and collaboration focused on improving services to the community.
Collaborative and community-focused were themes in the interviews. Irene’s description
captured the broader themes across all responses:
Our job in SACT is to keep people out of the system. Our work is not about the
institution. It’s about the community. Responding to their needs. And those needs are
72

ever-changing, so I see that SACT as a collective or a collaborative where all of us. These
agencies and entities that serve community come together and say, “Okay, we have this
new need, so how do we work together to address this need?”
In addition to SC members’ knowledge of the mission of the SACT, eight SC members were
able to speak to how their understanding of the mission evolved. They spoke about how SACT
has shifted from talking about collaboration to actually defining how SACT member agencies
will collaborate. Beth mentioned that SACT began as a “think tank for identifying challenges
and creating principled approaches to challenges and more recently has shifted to
operationalizing these approaches.” In addition to Beth’s observation, seven other participants
specifically spoke to how SACT operates differently now, compared to when it was created 12
years ago and as recently as 2 years ago. Cynthia spoke to the evolution of the mission of SACT
from its inception to today:
It has evolved. Back when we started, the mission was to really do that cross-system
partnership and collaboration in service delivery for the benefit of the family and
community to make sure that we focused on bringing barriers down and developing
opportunities for better customer service and collaboration of those services. Now, as part
of the IMTSS, it’s really building upon that. You know, building a system that is
integrated, coordinated, and multi-tiered for better services and better coordination for a
seamless no wrong door service delivery model for the county.
These sample statements show how the interviewees are more focused on taking coordinated and
collective action toward a common focus. In addition to more focused and systemic
collaboration, the SACT mission has become more focused on upstream prevention, which has
73

helped the interviewees to think about the role of prevention within their own agencies. Eric
spoke to what this shift means for his agency. He shared,
We wanted to shift from waiting for the badness to happen, which is one of the downfalls
of child welfare, but really focus on prevention. What would real meaningful prevention
look like? We wanted to be thought leaders around the prevention piece. Our guiding
questions: How do we push prevention? How do we ultimately inform how child welfare
works?
The participants agreed that the SACT’s mission has shifted over time from bringing agencies
together in the form of a think tank to agencies actively engaged in working towards a common
goal of upstream prevention. This shift towards prevention will be useful in developing the
common agenda and establishing mutually reinforcing activities.
When describing the SACT theory of change, interviewees described it in relation to the
SACT mission and the IMTSS framework. Responses from nine members included descriptions
of the theory as the landscape or roadmap of the elements being addressed in the SACT
collaborative, while the remaining three members described it as an outline of “what we are
trying to do and the outcomes we are expecting.” Specifically, Hannah described it as “people
from the different sectors and with different resources coming together and focusing on doing
our work better together. It really describes our mutually reinforcing activities.” Diane added,
It shows those elements that we are trying to address through our partnership, those larger
buckets. Then it lays out what are those things we need to do or milestones we need to
reach. It also shows those outcomes we are hoping to achieve and, of course, the long
term impact we are trying to have.
74

The SC members agree that the theory of change can be used to guide their collective work. This
can be leveraged when they work on determining their mutually reinforcing activities.
When speaking about the IMTSS framework, all SC members emphasized the focus on
community and described the IMTSS as a structure to help SACT organize to achieve its goals.
Andrew described it as “a client-centered service-oriented framework that meets clients where
they are and provides what they need.” Diane added, “What we are trying to create is a system
that is able to flexibly customize a service plan that meets their [families’ and children’s] needs
and addresses their issue and empowers them and allows them to go to the next level.” Despite
the SC members’ understanding of the IMTSS framework, there was some initial confusion
around what was being designed. Hannah spoke about her initial confusion:
This was a new term that was thrown out there, and it took me a while to really
understand what we were trying to do. I kept thinking, “What is it? How is it different
from anything we had been doing before?” But I finally understand that it’s providing a
framework for how we work together. It’s not a what. It’s a how. You know, partnerships
and collaboration. These are not new for us. I have been in this county a long time, and I
have been a part of some wonderful partnerships. But I often wonder what happens when
these great people leave. So, the framework. It will allow us to keep this work going
because it will be the way we work.
The SACT SC is responsible for managing the implementation of the IMTSS. Their
common understanding of the IMTSS will be crucial as they develop an implementation plan
using the collective impact framework. They can leverage their knowledge of the mission and
the theory of change, along with their understanding of the IMTSS, to establish a common
75

agenda and define the mutually reinforcing activities that will help them meet the organizational
performance goal.
Document Analysis
A review of SACT meeting agendas showed the IMTSS was introduced in August 2020
and has remained on the agenda at each monthly meeting since then. In October 2020, the SC
meeting focused on defining the IMTSS and connecting it to the theory of change. The SACT
mission was listed at the top of each agenda from May 2019 to June 2021. The theory of change
was introduced in September 2019 and was a part of the monthly agenda from September 2019
to June 2021.
Summary
The assumed influences were that SC members know the SACT’s mission and theory of
change and can define the IMTSS framework. According to the interview findings, this influence
was validated as an asset. All interviewees described the SACT mission and theory of change
and defined the IMTSS. Responses included similar themes of a framework that organizes the
SACT agencies to address the community’s needs of focused through an upstream prevention
lens. Knowledge of the SACT mission can be used to determine the common agenda. Similarly,
knowledge of the SACT theory of change can be used to establish the mutually reinforcing
activities needed to implement the IMTSS. SC members’ ability to define the IMTSS will ensure
that their implementation plan reinforces the overall objective of the IMTSS, which is to
streamline and improve outcomes for children, youth, and families.
Results and Findings for Conceptual Knowledge
Interviews were used to assess conceptual knowledge. Based on the protocols listed in
Chapter Three, meeting observations and document analysis were not conducted to evaluate this
76

influence. I asked interviewees about the importance of cross-sector collaboration in achieving
their goal and the role of cross-agency partners in designing and implementing the IMTSS. The
findings were organized and evaluated to assess if there was a gap.
Conceptual Knowledge Influence
The influences examined herein were that SACT SC members know the role of the
SACT member agencies in achieving the top priorities of the SACT and that they know why
cross-sector collaboration is needed to achieve their goal. The IMTSS requires SACT agencies to
work across agencies toward a common agenda. Therefore, interviewees were asked to explain
the role of each agency in designing and implementing the IMTSS. Interviewees spoke about
how the existing silos led to multiple duplicated initiatives and programs. They spoke about how
each agency can work to remove silos to fully implement the IMTSS. Jack provided further
insight into the role of each agency:
Our role is to break down the silos in order to avoid duplication of effort. In doing this,
we will increase access for children and families, and that’s what we are trying to do
under the integrated multi-tiered system of support.
Francine added, “By working together, each agency is taking the step to de-silo our work so that
we don’t have any duplications. This takes real partnerships.” The participants shared that
removing silos will not help prevent duplicated efforts and allow the SACT agencies to be more
effective and efficient in their service delivery. In addition to sharing about the importance of
removing the silos, Andrew added,
We each come into this with the unique role each of our agencies hold. By coming
together, we are acknowledging that by working together we will accomplish a lot more
than if we continued to work in isolation. So, when I think of our role, we come together
77

to do what we do best. That is the unique work of our agency in partnership with others
because, in the end, we are serving the same families.
Interviewees understand there is a need to work together to remove silos to better serve the
community and achieve SACT’s top priorities.
I also asked interviewees to explain why collaborating across agencies was necessary for
building out an IMTSS. All 12 spoke about the importance of collaborating in order to remove
silos and to stop the duplication of initiatives across the county. Ten spoke about the genuine
interest in cross-agency collaboration from all layers of county leadership. Despite this desire to
collaborate, there has not been a systematic approach to collaboration. Nine spoke about the need
for a systemic approach to cross-system collaboration across the county. Kathy noted that
collaboration across agencies was necessary because “we tend to be person-based, not system-
based. The problem with this is when people move on, we lose those connections, and the work
suffers.” She provided examples of prior partnerships between her agency and another that were
paused or discontinued when her contact at the partner agency moved into a different role, and
she once again had to work on rebuilding relationships with the new lead to continue the
partnership:
That’s why this integrated multi-tiered system of support is so critical. It’s about building
the infrastructure for systemic partnership and establishing how we work together. But
even more important is that we get to design it. We get to collaborate to build it, like
authentic collaboration to establish systematic collaboration.
Kathy was not the only one to experience a setback when leaders moved on to other positions.
Other interviewees shared examples of how their initiatives either stopped or were delayed due
to changes in leadership in other agencies.
78

In addition to creating systemic processes for sustainability, participants shared that
cross-agency collaboration benefits the community. Seven spoke about how the current lack of
collaboration among agencies impacts the services that the community receives. They spoke
about how the current siloed service delivery model forces the community to visit multiple
agencies for services forcing them to retell their story to each agency they go to for assistance.
Francine spoke to the importance of cross-agency collaboration in reducing trauma:
When I think about who we are trying to help, the community most in need, I am aware
of the trauma they may have already experienced before they even come into one of our
agencies to get assistance. And then when we duplicate our efforts because we function
in silos then all we have done is added more trauma as they have to repeat their stories to
every agency they walk into. So, you know, if we really are committed to improving our
service, then it's important that we work in collaboration, in partnership, and in unison to
address and have at the heart of all this work, the family in mind that we're talking about.
In addition to working together to help the larger community, cross-agency collaboration will
also help build relationships among agency partners. Nine interviewees spoke about how a
systemic approach to cross-sector collaboration will help SACT agency leaders improve their
working relationships. Hannah highlighted how, in working to serve the larger community, the
agencies will themselves become a community, and that will improve how they engage in cross-
sector collaboration. She shared,
So, when I think of our work and how focused we are on working with the community,
I’m reminded that we, the SACT, are also a community, and that means we need to be in
community with each other and collaborate with each other. It’s only when we are fully
engaged with each other that we will be able to achieve our goal. It’s like when you are
79

on the airplane, and you are told to put your oxygen mask on first then help others. Well,
we can’t really help, I mean truly help, community if we don’t even know how we work
together. Without this, we will continue to work in our silos.
The interview data suggest that participants know each SACT must actively work to remove
existing silos. They also know that a systemic approach to cross-sector collaboration will
improve how agencies work together, and that will lead to improved services for the community.
Summary. The assumed influences are that SC members know the role of the SACT
member agencies in achieving the top priorities of the SACT. SC members also know why cross-
sector collaboration is needed to achieve their goal. According to the interview findings, this
influence is validated as an asset for the SC members. All interviewees could explain why cross-
sector collaboration is necessary as they design and implement the IMTSS.
Results and Findings for Procedural Knowledge
Interviews and document analysis were used to assess procedural knowledge. Based on
the protocols listed in Chapter Three, meeting observations were not conducted to evaluate this
influence. I asked interviewees to describe how they communicate with other agencies to
establish partnerships. The findings have been organized and evaluated to assess if there is a gap
regarding the assumed causes.
Procedural Knowledge Influence
Continuous communication is one of the collective impact conditions. As SC members
establish an implementation plan for the IMTSS, they must establish how agencies will
communicate with each other and with the larger SACT. Therefore, an assumed influence was
that they know how to communicate with each other across agencies.
80

Interview Findings. Interviewees were asked to describe how their agencies currently
communicate with other agencies to establish partnerships. All 12 SC interviewees spoke about
the SACT meetings and the SACT SC meetings as a venue for open communication. They
agreed that they could easily add items to the agenda and feel comfortable sharing information
and asking questions in the meetings. Five members spoke about how the meetings have changed
and how those changes have improved communication. Kathy shared how the improved
communication has improved her SC meeting experience.
These meetings used to just be a sit and get with some random activities, but now when I
attend these meetings, I feel like I am walking away with more awareness of what’s
happening in all of these agencies across the county, and that helps me know how my
agency can support the others. We still have work to do, but these meetings have helped
with communication.
In reflecting on what has led to this change, interviewees mentioned the common focus
on the IMTSS has helped to streamline and improve communication at the SC meetings. They
identified their work on developing the implementation plan as helping to identify what is shared
at the meetings. Beth shared her observation about improved communication: “I feel as though
because we now know our common focus, our communication is around our IMTSS and helping
community, and we are all sharing more about our common work in these meetings.” In addition
to improved communication, members also spoke about how they reach out to other agencies to
establish partnerships. They expressed the collaborative nature of their relationships with each
other. However, none of them could describe a formal process for communicating with each
other without leveraging their personal connections. Diane said, “When I see a need, I just call
up so and so, and say, ‘Hey, here’s an opportunity to work together to solve a particular
81

problem.’ I’m not shy about reaching out for help.” Eric also spoke to the importance of being
able to easily reach out to others to collaborate:
You know, great collaborators, we pick up the phone, anytime, day or night, and just say,
“You know, I've got this going on. What do you think?” You know, just to even do that
kind of work that's pretty unprecedented, and that makes a huge difference to make this
collaborative thing work because otherwise, you get in a room together with people with
their arms crossed and saying, “why am I here?’ and ‘I can’t wait to get out of here
because this is really uncomfortable.” And so much of what we do is based on
relationships.
Participants agreed that the SACT and SACT SC meetings increase communication among
partner agencies. They also agreed that they are comfortable reaching out directly to other
leaders to establish partnerships. While members are comfortable reaching out to each other to
collaborate, the limitations of this approach were raised when they talked about how their
reliance on personal relationships to engage in collaboration created problems when their
personal contacts moved on to different positions or opportunities. This highlights the need for a
systemic approach to communication regarding cross-agency collaboration.
Document Analysis. SACT SC meeting agendas and minutes from May 2019 to June
2021 were reviewed. I observed that meeting agendas are emailed to SC members no later than a
week before the meeting, and meeting minutes are sent within a week after the meeting. All
members are included in the email distribution list. A review of the meeting minutes indicated
that collaboration opportunities through grants or other partnerships are regularly shared among
SC members. The meeting minutes also show that the meetings are used to plan the agenda for
the larger SACT monthly meetings. The larger SACT group also meets once per month. The
82

agenda for those meetings are also sent no later than 48 hours before the meeting. This group has
a much larger membership, and the SACT SC members also attend these meetings. An external
consultant serves as the facilitator for those meetings. The larger SACT meeting minutes show
that while the IMTSS has been discussed, the agenda focuses on other presentations that do not
always connect back to the IMTSS.
Summary. The assumed influence that SC members know how to communicate with
each other across agencies was validated as a partial asset. Across the SACT agencies, SC
members use the SACT meetings to communicate with each other. Additionally, they
comfortably communicate with each other in order to partner. However, a partial gap exists in
the lack of a systemic process for collaboration. Additionally, the agenda for the larger SACT
meeting is not always connected to the IMTSS, creating a missed opportunity to build capacity
and understanding around the IMTSS.
Results and Findings for Metacognitive Knowledge
Based on the protocols listed in Chapter Three, interviews, documents, and meeting
observations were used to assess the SC’s metacognitive knowledge. I asked interviewees to
reflect on how they could work differently under the IMTSS.
Metacognitive Knowledge Influence
Collective impact is not just about working together, but it is more focused on how
partners work together. Therefore, SACT SC members need to reflect on how they can
collaborate differently under the IMTSS collective impact effort.
Interview Findings. I asked interviewees to reflect on how they could work differently
under the IMTSS. Two larger themes emerged. SC members identified the need to remove silos,
83

especially given the interconnectedness of their work. Additionally, SC members identified
being client-centered as another way of working differently.
All 12 interviewees reflected on the silos that exist and how important removing them is
to stop duplicating efforts. Five members shared that they are unsure if everyone really
understands why removing the silos is important. They expressed that while many will
acknowledge there are silos, not many truly understand how those silos lead to duplicated
services. Reflecting on the current siloed approach, Andrew shared,
Let's get an agreement upfront that we commit to not duplicate services. I think
everybody would agree with that. Right. But the harder statement is let's agree that we
have duplicative services. And to be efficient and effective, we need to remove the silos.
Seven members spoke about how removing the silos is important given the interconnectedness of
their work. Diane shared, “If we really stop to think about how all of this work is connected, we
will really begin to streamline and improve how we serve the community.” Others added that the
interconnectedness of the work could be leveraged to prioritize resources based on the
community’s needs. Lucy spoke to the prioritization of resources in relation to the reality of a
budget deficit. She shared,
Over the years, we have successfully built relationships across the system that have led us
to this common goal setting. We are heading for a significant budget deficit over the next
few years, due in part to COVID, and the foundation we have built with those
relationships will be critical because of the interdependence of the work we do.
Everything is interconnected. So, if we don’t have a discussion about the impact of
reduction in one area of the system, then it impacts the other parts of the system. We have
to make decisions about the prioritization of resources.
84

In addition to reflecting on the work’s interconnectedness, interviewees also reflected on
the need to be more client-centered as they implement the IMTSS. Seven specifically spoke
about the need to be “equity-focused.” They talked about the need to increase engagement with
the community to ensure that the IMTSS addresses the community’s needs and that the
community determines what resources they need to address their needs. Jack described being
equity-focused as listening to the community and not pushing a predetermined menu of services:
To be equity-focused means we are listening to the community and addressing their
needs rather than telling them what we think they need. Far too often, we try to tell them
they need this service or that support, but they know what they need. We need to listen,
and that’s equity: giving everyone what they need.
In moving toward a more client-centered approach, six SC members talked about being
innovative in thinking about the experience of the end-user. Cynthia added, “We need to ask why
do things work this way, instead of just doing things because that’s the way we have always done
them.” Andrew also spoke about the need to think differently about how to be client-centered.
He added,
But I think the single most important thing to me would be to free ourselves of our real or
perceived constraints, envision the desired outcome, and then work backwards from there
as you overcome the constraints that are legitimate. So, what I mean by that is, again, too
often, instead, we say, I have these constraints, so we just keep designing and
implementing to avoid, you know, not being in compliance. And you end up with a
system that is anything but client-centered. And I'm saying, start with the client-centered
desired outcome. And as you try to build to that vision, you can legitimately say, well,
wait a minute, this funding stream can only be used for that or whatever the issue is, and
85

then be creative to find ways around that. Instead of being creative as to what the
outcome is going to be because we've seen too often, the outcome is as a consequence of
bending toward other needs. And it's anything but client-centered, and that's all that we're
here to do.
Lucy, along with five other interviewees, talked about changing the system to be more client-
centered rather than creating temporary solutions to navigate the system’s complexity. Lucy
provided an example:
We need to think about how to make this very complicated system much easier for our
customers to navigate. And it’s not about adding navigators who are supposed to help our
customers navigate this system. I’m saying let’s examine why the system is so hard to
navigate to begin with. For example, if an application for assistance is 18 pages long, we
don’t need a navigator to help them fill out 18 pages. No, we need to figure out why the
application is 18 pages to begin with.
Beth also spoke about addressing the complexity of the current service delivery model and the
need to change it to be more centered around the client's needs:
We have so many programs. It’s like a house that we just keep adding rooms to. The
client doesn’t need all of these rooms, but we all want our room in this house. And often,
I hear my colleagues say, “What we need is case navigators.” I'm like, “No, that's the
wrong answer.” We can't afford to have a case navigator. We need to simplify the system
so that I could say your room is room 17 down this hallway to the right, and there is
signage, and they can go. That's what we need, not go find your room out of the 101
rooms in this house, and by the way, you have to figure out which one has your name on
86

it. But wait, we will have someone navigate to get you there. How does that empower our
clients? All it does is make us the one who holds the power.
As they design and implement the IMTSS, SC members reflect on the interconnectedness of
their work, the need to be equity-focused, and how they can utilize the IMTSS to move client
needs to the forefront.
Document Analysis. In a review of the SC meeting notes from May 2019, the committee
agreed to utilize an existing structure to demonstrate what would become the IMTSS collective
impact effort, but it was not revisited until October 2020, when the members met to plan for the
IMTSS. In meeting agendas from October 2020 to June 2021, the SC members planned the
launch of the IMTSS demonstration site.
Meeting Observations. I observed two adhoc SC planning meetings in May and June
2021 and one regular meeting in June 2021. In all meetings, the members were observed
planning for the launch of their demonstration sites. In all three meetings, there were
conversations around improving the customer experience, representing community voice, and
shifting from doing to community to doing with community.
Summary. The assumed influence that SC members need to reflect about how they can
collaborate differently under the IMTSS collective impact initiative was validated as an asset.
Participants regularly reflect on how to improve the experience for their customers and how they
can work together differently as they design and implement the IMTSS.
Results and Findings for Motivation: Utility Value
Interviews with the SC members were used to assess the SC’s individual and collective
value. Based on the protocols listed in Chapter Three, meeting observations and document
analysis were not conducted to assess this influence. Interviewees were asked to explain how
87

their work on the IMTSS aligns with the work of their agency. SC members were also asked to
explain how the IMTSS will benefit the work of their individual agencies.
Value Influences
The influence assessed was that, individually and collectively, SC members see the value
of the alignment between the SACT’s common agenda and their agency goals.
Interview Findings. Interviewees were asked to explain how their work on the IMTSS
aligns with the work of their agency. All 12 responded that there was “a direct alignment”
between the two. Cynthia explained, “Part of the integrated multi-tiered system of support is
building that partnership and aligning the services that we provide with the other partners, so the
families don't have to come into our systems.” Diane described it simply by stating, “This is our
work.”
I also asked interviewees to explain how the IMTSS will benefit the work of their
individual agencies. All responses included a general theme of “improved collaboration.” Each
member spoke about the beneficial outcome between their agency and the IMTSS. Lucy spoke
about the interconnectedness of the work of her agency and the IMTSS: “The IMTSS will create
opportunities to bridge a connection to avoid duplicating work happening in another part of the
system. That will help my team in that we focus on what’s not being addressed.” Eric shared
more on how the IMTSS will specifically benefit his agency:
We provide services that cover the youngest of children to the elderly. The IMTSS
supports our continuum of care and also helps to point us toward upstream prevention.
And we know that we can’t do this alone. The family we may be supporting will have
kids in school, with a family member who is incarcerated, and some medical needs. That
88

can’t all fall to us alone. We need to work with all these other agencies, or we run the risk
of families falling through the cracks.
The interview data suggest that 100% of the SC members see the value of the alignment between
the SACT common agenda and their agency goals. In addition to valuing how the IMTSS will
align services across agencies, SC members also value how their specific agencies will benefit
from the IMTSS.
Summary. The assumed influence that, individually and collectively, SC members need
to see the value of the alignment between the SACT’s common agenda and their agency goals
was validated as an asset. Interviewees not only see the value in how the IMTSS aligns to their
individual work, but they also find value in the beneficial outcomes between their agencies and
the IMTSS.
Results and Findings for Motivation: Self-Efficacy
Interviews were used to assess the SC’s individual and collective efficacy. Based on the
protocols listed in Chapter Three, meeting observations and document analysis were not
conducted to evaluate this influence. I asked interviewees to share about a time when they felt
confident partnering with another agency. I also asked them to rate their confidence level on the
members’ ability to stay focused on the common agenda. The findings have been organized and
evaluated to assess if there is a gap regarding the assumed causes.
Efficacy Influence
Individually, participants are confident that they can effectively work with other partner
agencies. Collectively, they are confident that the group's collective efforts will be effective in
accomplishing SACT’s goals.
89

Interview Findings. I asked participants to share about a time when they felt confident
partnering with another agency. Every interviewee was able to provide at least one example of a
successful partnership between their agency and a partner agency. Despite the positive
partnership experiences, members acknowledged that their confidence was rooted in people
instead of the system. Gail shared,
I think of over the years how I have just called up Kathy or Irene or even Diane. In each
of those instances, I reached out because I saw a need and felt confident that I could reach
out to those leaders, and they would not hesitate to work with me to solve the particular
problem I wanted to address. Where I am not so confident is that if those people were not
there (Kathy, Irene, and Diane), would that partnership even happen? So, much of this is
based on the personal relationships we have with each other. Would it work with
different people, maybe? But what I want to see is the expectation that this is how our
county functions so that every leader, old and new, expects that they will be asked to
collaborate.
Interviewees were also asked to rate their confidence in the committee’s ability to stay
focused on the common agenda. Six responded with a confidence score of 5, four responded with
a confidence score of 4, and the remaining two responded with a confidence score of 3. Kathy
explained that her confidence level of 5 was due to the shift she saw in the SACT SC meetings:
“As I shared before, these meetings seemed to lack direction, and it was just a sit and get. I have
seen the shift, and I feel like we are finally moving towards meaningful change.” Beth added,
“Our common focus or common agenda is shaping our direction.” Irene shared, “The IMTSS
framework is helping us to build the infrastructure we need for collaboration and
communication. The momentum is there; I feel it, and it feels different.” Hannah, who rated her
90

confidence level at a 3, explained that although she believes in the committee’s ability, she
worries about the amount of sustained attention this effort will need given the “attention deficit
disorder world” that we live in. She added,
I do think it's kind of right there in the middle. I think there are really good people at the
table working really hard to create impact. But I think that we are a county where we talk
a lot about collaboration, we do a lot, we like each other, and we want to move forward.
But despite all of that, we stay at the superficial level. I think about this work, and this
work requires us to go deep, and we're moving into the space of really having to roll up
our sleeves to really have to get down to the work. So, I think of this like a puzzle. We
have the IMTSS that’s the frame or the outside, and now we all need to add our pieces to
complete this puzzle, and I think there are some people who are not laying down all of
their puzzle pieces. They are holding on to something. Maybe it’s trust or uncertainty. I
don’t know. But we are not all fully present.
Jack shared that he, too, has concerns about the ability to stay focused. He shared, “I really
believe that everyone here is committed, but I know that this work takes time, and I just worry
that when some of these leaders move on, so will this work.” Lucy added,
I think we generally can work together, but I don’t know that we are fully trusting each
other. I hope this IMTSS can help to work through that, but then I’m not sure we can
really create the IMTSS without fully trusting each other. It’s like a catch 22.
Even with the higher confidence ratings of 5, other SC members expressed similar sentiments to
those of Hannah, Jack, and Lucy, indicating that while they were confident that they could work
together, there is lower confidence in the ability to stay focused on the common agenda for the
duration of creating, implementing, and sustaining the IMTSS.
91

Summary. The assumed influences for self and collective efficacy were validated as
assets. Individually, participants were confident that they could effectively work together with
other partner agencies to accomplish the goal. Collectively, they were confident that the group's
collective efforts would be effective in meeting SACT’s goals. Interviewees not only had a long
history of collaborating to accomplish a task or goal, but there was also an overall high
confidence rating in their belief that they could stay focused on the common agenda.
Results and Findings for Organizational Influences: Resources
Based on the protocols listed in Chapter Three, interviews and document analysis were
used to assess to what extent the SC members felt their organization provided them the time and
funding to meet the organizational goal. Meeting observations were not conducted to assess this
influence. The organizational resources that were evaluated included the funding to engage in
partnerships and the time to implement collective impact.
Resources Influence
The resources influence analyzed was that the organization provides the S SACT SC the
funding and time to implement collective impact to achieve its priorities.
Interview Findings
Interviewees were asked how much time they spent on the IMTSS collective impact
effort outside of the SACT SC meetings. The average amount of time was about 30%. All 12
interviewees reported being able to be a part of this work as it connected to their work at their
own agencies. In terms of funding, they talked about the complication of government funding,
which tends to be reimbursement funding and often categorical, meaning that there are
restrictions on how the money can be spent. They tend to rely on grant funding and other sources
92

that can be leveraged to support this work. Despite these restrictions, Diane talked about trying
to move the group in the direction of being creative with how to leverage the funds. She shared,
Sometimes, we can get so caught up in which funding stream to use for this activity and
which funding stream to use for that activity. I try to remind everyone at the table that
there are multiple funding streams. Let’s just focus on designing the service, and then we
can figure out the funding.
Andrew, Kathy, and Lucy also echoed Diane’s sentiment. They agreed that funding remains an
issue because of the “categorical restrictions,” but they were the only four members who spoke
of how they could navigate around the funding restrictions to still engage in cross-sector
collaboration. Andrew added, “I see my role as going to the board and advocating for the
funding. I want this group to focus on the end goal.” Five members mentioned the board of
supervisors as an asset. Eric also added about the support from the board. He shared,
The board is very supportive. They take action on items and fund projects without us
having to ask. This is very unique as this can typically take many years of lobbying and
securing funding. At times we get funding without even asking for it.
Document Analysis. SACT SC meeting agendas and minutes from May 2019 to June
2021 were reviewed. Analysis revealed that the SACT finance committee had approved funding
for the consultant to develop the IMTSS and for the SACT facilitator, indicating a financial
commitment to the SACT work. Professional development and training support were also
scheduled and offered in August and October 2020. Additional funding has been approved to
support this work into June 2023.
Summary. The assumed influence of organizational resources is a partial asset and partial
gap. The interview data highlights that the SACT SC members receive time and funding to
93

support this work. The document analysis also confirmed time and funding as resources.
Additionally, the document analysis found that members have professional development
opportunities and consultants to support meeting facilitation and the design and implementation
of the IMTSS. These are all assets. However, the funding can be a partial gap given the
restrictions on funding activities and the reliance on government funding.
Results and Findings for Cultural Models and Cultural Setting
Based on the protocols listed in Chapter Three, interviews were used to assess cultural
models and cultural setting. Document analysis and meeting observations were not conducted to
evaluate this influence. I asked interviewees about the promotion of innovation and creativity in
problem solving as well as any additional resources they would need to engage in this work.
Cultural Models and Cultural Setting Influences
The influences analyzed were that the organization promotes the use of innovation and
creativity to solve problems and that it provides the SACT SC the resources to implement
collective impact to achieve its priorities.
Interview Findings. I asked interviewees to reflect on how supportive county leadership
is of using creativity and innovation to solve problems. Given the complexity of the problem that
the SACT is working to address, five SC members talked about how supportive the board of
supervisors is of their work. Irene shared,
The board [of supervisors] is very supportive. They want to see us working together, and
they also recognize that this is a long-term commitment. This is important because since
they are elected into their positions, time is not a luxury they have. So, for them to
recognize that this innovative solution will be long-term is a huge win for us.
94

The support of the board of supervisors is important in securing funding and navigating the
political climate. In addition to the board of supervisors, seven SC members described the
county’s executive leadership, which serves as the backbone organization, as being very
supportive. Andrew also added that county leadership is very supportive and has been
encouraging the use of creativity to find new ways to work together and leverage resources to
better serve clients. Lucy shared about the added pressure this can create:
We tend to be a county that is viewed as innovative, and what tends to happen is that we
think we have to do everything, all at once, and do it quickly. Not that this is necessarily
a bad thing, but the overwhelming demand impacts the work. We are constantly being
given new responsibilities. You think you have an idea of how to move forward, then five
new things get added, and you have to figure out again how to move forward.
SC members were also asked to reflect on what else they need to engage in this work.
While support at the board and executive levels is important in helping the members to stay
focused on their and the larger organizational goals, the added demand and pressure led all 12
participants to identify time and staffing as resources still needed. Diane summed up the
sentiments shared by all members:
Staff and time. Of course, we could always use more money. But the staff is important.
This is a lot of work and takes quite a bit of time. But we can’t let the work stop. We
have to keep going. But we need more staff. And that’s hard when we look ahead to a
couple years of a budget crisis. But then I think this is how we will see what the priorities
are. Will we continue to get the support in terms of funding to do this work we all agree
and believe needs to get done? I think now more than ever, we will see this board
95

continue to fund this work because it is really the only way to increase our effectiveness
and efficiency.
Summary. The assumed influences of cultural settings and cultural models are assets.
Steering committee members have the time to engage in this work. Additionally, there is support
from the senior leadership, and innovation and creativity are encouraged while problem solving.
Summary of Research Question One: KMO Influences
Interviewees’ presumed KMO influences related to implementing the IMTSS collective
impact effort were assessed through interviews, document analysis, and meeting observations.
Findings are presented for each assumed cause within the categories of factual knowledge,
conceptual knowledge, procedural knowledge, metacognitive knowledge, utility value, self-
efficacy, and collective efficacy. Findings were also presented for the organizational categories
of resources, cultural setting, and cultural model. Each KMO influence was found to be an asset
with the exception of procedural knowledge and organizational resources, which were identified
as partial assets. While SC members know how to leverage their personal relationships with each
other to communicate across agencies, they could not identify a formalized communication
process to establish cross-agency collaborations. Additionally, government funding restrictions
created a partial gap as only three interviewees identified ways to navigate around the restrictions
of the funding to engage in cross-agency collaboration.
Research Question Two: Organizational Culture and Context
As mentioned in Chapter Two, an organization can still experience a performance gap
even with people who are knowledgeable and motivated. Therefore, to understand the
participants’ readiness to work together under the collective impact framework, it is important to
study and investigate the interaction between the organizational culture and context and the
96

interviewees’ knowledge and motivation already presented earlier in this chapter. This was the
focus of the second research question. Specifically, this section will juxtapose organizational
culture and context with the participants’ knowledge and motivation already presented as part of
the findings for the first research question.
Results and Findings of Organizational Influences: Context and Setting
Steering committee members participated in interviews to assess their perception of the
larger SACT organizational context. Interviews were used to establish the organizational context
in which the SACT members work together. SACT documents were also analyzed to provide
evidence of the larger organizational context. One SACT SC meeting and the subsequent general
SACT meeting were observed to establish the connection between the two meetings.
Interview Findings
Interviewees talked about their overall organizational structure. Currently, SACT’s
organization includes the steering committee and the larger SACT membership, which consists
of five working groups. Only two members could explain the process for inviting member
agencies to participate in SACT. The remaining 10 interviewees did not know how agencies
became SACT members. Eight members expressed that SACT felt like a “secret society where
no one knows how to join.” Hannah added,
I sort of pushed my way in. I think that is a downfall in that there doesn’t seem to be a
systematic way for how to add people to SACT. Who should really have a seat at the
table? It makes sense in terms of agencies, but in terms of CBOs [community-based
organizations]-, who decides who joins? It’s just not clear. So, often, it’s me inviting
people saying, “You should come to this meeting. I think you should be there.” But I
know I have missed so many people. And flip it, sometimes I have invited people who
97

really should not have been there. We need a process for how people are invited to be a
part of SACT.
In addition to not having a documented process for inviting agencies to be part of SACT, there
was also concern about how large SACT has become. Beth discussed SACT’s growth:
So, you have all these people who are a part of SACT. Some people attend, some don’t,
but the membership roster is quite large. And I think that’s part of the challenge. This
should be about coming to this table to get involved in the work. So, many people
complain and say, “Oh, we’re not moving fast enough. All we do is sit and talk,” and yet
not everyone’s involved in the working groups, which is where the work is. I feel like we
need to have some expectations laid out. You want to be a part of SACT? Great, but
you’re coming here to work.
When sharing about the size and structure of SACT, seven members also talked about the current
structure of SACT meetings. Kathy shared,
I have mentioned that I have seen a shift in our steering committee meetings, which I
appreciate. But I haven’t really seen that shift in the larger SACT meetings. There doesn’t
always seem to be a connection to the work we are trying to do in the steering committee
meetings. At the steering committee meetings, we are talking about the IMTSS and
building out the demonstration sites, but that doesn’t translate over to the SACT
meetings. So my reps who attend the SACT meetings have expressed frustration over
coming to those “sit and get” meetings. And I understand their frustration. I don’t think
they walk away seeing what they can do in partnership with others. I feel like we all leave
those meetings and return to our corners of the county.
98

Eight additional SC members also shared this overall frustration with the lack of
connection between the SC meeting and the larger SACT meeting. They talked about how the
changes they see in the SC meetings are not happening in the larger SACT meetings. Jack
shared, “Sometimes I don’t even know what we are doing in the larger group. It’s such a missed
opportunity to connect the work for the larger group.” Cynthia also shared that she has noticed a
slow shift in the SACT meetings:
I think what’s happening is we are trying to bring all of the steering committee members
along in this work. And we are trying to help them connect what we are doing here [in
the steering committee meetings] to the larger SACT. To make this shift, I have
advocated to have the workgroups share out the status of what they are working on in the
larger SACT meetings. And you see that we just started this. When we first did it in May,
the facilitator had the workgroup leads add their updates in the chat. That was a start, but
it was lost in the larger agenda. At our meeting in June, we had everyone share out. I
think that made a difference because I saw people making connections in the chat. It’s a
start. It can happen. It just takes time.
While SC members were accepting of the fact that systemic change will require more time, they
found the lack of alignment as one obstacle to implementing the changes in the SACT meetings.
Cynthia acknowledged that more alignment is needed.
At last week’s SACT meeting, when [the guest speaker] shared, there was no
connection to the IMTSS. But the opportunity was there for the connection to be made.
When she asked us to talk about what resources exist to address the problem, she
presented I thought we already have this resource, but here we were again recreating the
wheel. And that’s why people get frustrated. I just don’t know why we aren’t being more
99

clear with the connection. You see that people left as we were moving into the first
breakout. [The facilitator] kept adding time for [the guest speaker], and what that shows
people is that we don’t have a clearly developed agenda, and we are not respecting their
time. It’s like the steering committee members all seem to get it, but then we don’t bring
that to the larger SACT.
Through the interviews, participants identified some gaps in the organizational context regarding
how potential members are recruited to SACT as well as how the communication flows between
the committee and the larger SACT group.
Document Analysis
The SACT and SACT SC meeting agendas and minutes from May 2019 to June 2021
were analyzed to better understand how the SACT SC meetings informed the agenda for the
SACT meetings. There was evidence to support that these meetings included an agenda item to
plan for the SACT meetings. The IMTSS was added to the general SACT agenda in August 2020
in reference to a training that was being planned. Subsequent agendas for September and October
2020 show that an update on the IMTSS was included as an agenda item. Specifically, updates
on the August training and October training were shared. There were no general SACT meetings
in November and December 2020. In January through April 2021, there was a standing agenda
item to provide an update on the IMTSS and the potential demonstration sites. In May and June
2021, an additional agenda item was included to allow the SACT working group leads to provide
an update to the larger SACT membership. Additional agenda items were not explicitly
connected back to the IMTSS.

100

Meeting Observations
Two meetings were observed as part of the investigation on organizational context and
setting. The first meeting was the SACT SC meeting. During this meeting, members were
observed contributing to the SACT agenda for the meeting to be held the following week. They
agreed to have the leads for each working group give an update. For the five working groups,
they allocated 10 minutes with the expectation that each update would be about two minutes.
There was no discussion as to what working groups should speak about when they provided their
updates. The rest of the agenda was created with presentations that some SC members thought
could be useful for others to hear. There was no connection between those agenda items and the
IMTSS. None of the members asked for any explicit connections between the suggested agenda
items and the IMTSS.
The second meeting observed was the general SACT meeting. During this meeting, the
working group leads provided an update to the larger SACT. They did not exceed the allowed
time, and each update provided insight into the current tasks of each working group and some
potential next steps. No questions were asked of the working group leaders. A presentation
connected to prevention, but not affiliated with the prevention workgroup, was shared.
Additional time was given to this agenda item to allow for breakout room discussion. Attendance
declined by 50% as participants were pushed into the breakout rooms. After the presentation, one
member asked the presenter what support was needed from the SACT and if the presenter
wanted to work directly with the prevention working group. The presenter said they were just
presenting an informational item and did not have a specific request of the larger group at that
time.
101

Summary. Participants expressed a disconnect between the SACT SC and the larger
SACT. While the participants understand and are motivated to work on the IMTSS, the larger
shifts are not occurring within the larger SACT group, which is responsible for operationalizing
the IMTSS. This represents a gap.
Results and Findings of Organizational Culture
Steering committee members participated in interviews to assess their perception of
overall organizational culture. Interviews were used to learn more about the organizational
culture in which the SACT members work together. SACT artifacts and meeting observations
were not used as a part of this assessment.
Interview Findings
Nine interviewees talked about the challenges of trying to collaborate when there is
overall competition among members as to whose agency takes priority. Diane provided an
example:
Let’s say you have a family with grandma needing senior services, mom needs mental
health support, dad has recently been released from custody and is on probation, and
three minor children who have been referred to [child welfare]. What do you expect the
family to do first? Well, child safety is the first thing. And, so, in essence, what are the
things that need to be put in place to support and that should be the first thing. And then
sequentially those other responsibilities that come from the other entities that may get
into empowerment, self-improvement, etc. They should be sequenced accordingly after
you deal with the primary effort to stabilize the family. But instead, what happens is
probation will say, well, they need to respond to me first. Then child welfare will argue
that safety comes first, and so that creates a battle with child welfare and everyone else.
102

So, there becomes this in-fighting because there is no coordination of how to put the
needs of the family first. It’s not about which agency takes priority. What about the
family’s priorities? What has the family identified that’s most important to them?
The desire to put the needs of the community ahead of the organizational needs was a resounding
theme throughout the interviews. All participants demonstrated a commitment to the community.
Andrew described the good intentions among agencies to serve the community that leads to this
layer of competition among agencies. He added,
So, you know, for better or worse, we are organized in about 70 different departments
across the county, and unfortunately, they get siloed. And through the goodness of their
hearts. They all want to serve, and they want to solve the problem completely on their
own. And you know I sometimes feel that all that is done and again with best of
intentions, but it's neither efficient nor effective. In fact, it's counterproductive, right? It
gets to the point where the client doesn't even know where to turn to next in this maze of
services that may or may not be what they need.
The siloed approach leads to duplicative work and increases the number of meetings
these leaders attend, which requires more time from them. The average amount of time spent on
committee duties, in addition to their regular work duties, was about 30%. As reported earlier,
the participants identified time and staffing as the resources they wanted more of as they engaged
in this work. As they reflected on how they spent their time, they shared how much time they
spent in meetings, usually about the same thing and with the same people. Some participants
wondered if the time spent in meetings was efficient or effective. Jack shared,
I see the same faces in meeting after meeting. I sometimes wonder if we can just stay in
one location but change the agenda. Although now with Zoom, I guess we can. But it was
103

a real challenge when we were meeting in person. We used to joke that we could just start
a meeting carpool because it was almost the same group of people attending the same
meetings. So we could just all travel together to save on gas. I mean, we can laugh now,
but it’s really not the best use of our time to just go from one meeting to the next with the
same people talking about almost the same thing. The only difference is some catchy new
acronym for a name, and a different agency is the host.
Kathy added,
I have found that the more meetings I attend, the more I get invited to. I’m not sure why
this work seems to be synonymous with more meetings. I want to send more of my senior
directors, but I try to be more protective of their time.
As the leaders spoke about attending all of the different meetings, they realized that they were
not building the capacity of other leaders in their organizations. Hannah wondered if this was
because of the hierarchical structure of SSC:
I think of Diane and all of the meetings she attends. Well, she has Cynthia, who also
attends, but she is just as busy. They are attending all of these meetings. When does the
work happen? But Diane also has [two other senior leaders]. How can we build their
capacity so that we know when they are attending these meetings, they fully understand
the work that is being done, and they can go back and communicate within their agency
what is happening? So, Diane doesn’t feel like she is missing out when she is not there.
This can then free up some of her time so she can move the work in her agency and
across agencies.
Given the number of meetings and the time spent in these meetings, participants were concerned
about building capacity and being able to do the work at their own agencies.
104

Summary. The SACT is very committed to serving the community. Not only is
collaboration a part of the organizational culture, but SC members have the knowledge and
motivation to collaborate. However, duplicative projects and meetings, as well as inefficient
communication within and across agencies, often stymied their collaborative efforts. This
represents a gap.
Summary of Research Question Two: Organizational Culture and Context
The juxtaposition between the organizational culture and context and the interviewees’
knowledge and motivation was used to further understand the potential gaps and assets that can
impact the SACT SC members’ readiness to engage in collective impact. The findings show a
gap in cultural context because there is no documented process for inviting agencies to
participate in the SACT. Interviewees also shared that there is a disconnect in communication
between the SC and the larger SACT. This is a gap because the SC is responsible for leading the
larger SACT. There is also an identified gap in organizational culture. The larger organizational
culture leads to duplicative projects and meetings and inefficient communication within and
across agencies. The organizational setting was identified as an asset. The interviewees
expressed gratitude for the support they received from the board of supervisors and the county’s
executive leadership in the form of funding and time.
Research Question Three: Readiness for Collective Impact
The purpose of this project was to evaluate the degree to which the SACT was meeting
its goal of fully implementing its goal to fully implement the IMTSS using the collective impact
framework. The study focused on the readiness to engage in collective impact by evaluating
KMO influences related to achieving a larger organizational goal. When launching a collective
impact effort, in addition to focusing on the design and implementation, the SACT SC must also
105

focus on the necessary changes in the overall context and culture (Preskill & Splansky, n.d.). The
findings presented earlier focused on the interviewees’ readiness based on their knowledge,
motivation, and the organizational context and culture. The findings in this section are intended
to provide further insight into their readiness in terms of the five conditions of collective impact
and how those were established in the IMTSS collective impact initiative. Hanleybrown et al.
(2012) identified the three phases as initiating action, organizing for impact, and sustaining
action and impact. The findings presented below are organized into Phase I and Phase II,
including the core tenets or conditions of collective impact to show progress in each phase and to
highlight the opportunities for improvement based on the data gathered from the interviews.
Phase III was not included in this study. Interviews, artifact analysis, and meeting observations
were used to provide evidence of the conditions of collective impact.
Phase I: Initiating Action
This phase requires an initial governance structure that includes strong and credible
champions (Hanleybrown et al., 2012). The SACT SC represents this governance structure
within a cross-agency collaborative, and it is for this reason that this group was chosen as the
stakeholder group for this study. These members are executive-level leaders who are also
champions of this work. As previously shared, interviewees provided examples of how they have
collaborated with other committee members. There is mutual respect among these colleagues and
confidence among certain members that they can successfully collaborate with other members.
Another requirement for this phase is to use data to establish the need for this level of
collaboration. Steering committee meetings minutes show that the SACT SC has reviewed data
to establish the need for this work. There have been three guest speaker presentations featuring
nationally renowned experts on the school-to-prison pipeline as well as the intergenerational
106

cycle of disadvantage. Local data and an argument for a collective impact approach were shared
in the context of these presentations.
Finally, a sense of urgency helps to initiate action. All interviewees expressed an
understanding of the urgency of their work. Kathy shared her motivation around the urgency in
addressing the racial inequities perpetuated in the current design of the system:
I have been involved in [my kind of work] since [the late 1980s]. And as a county, I can
agree that we have done some good work over the years. We can brag that we have
reduced kids in jail, and reduced the number of families needing assistance, and reduced
the number of teen pregnancies or any other social problem. But what I know is that
when we disaggregate the data, we see that it is our Black and Brown communities that
continue to be disadvantaged.
Four SC members specifically spoke about the urgency to rally around the renewed focus on
upstream prevention. Gail added,
As we come together in this work, we need to be the loudest advocates for prevention.
We need to know the data and be able to present the data around why prevention is the
best investment of our county dollars.
Based on the interviews and document analysis, the participants demonstrated that they met the
criteria of this phase.
Phase II: Organizing for Impact
The five conditions of collective impact are established during Phase II by building off
the governance structure and understanding of the need based on the data. Additionally, during
this stage, an infrastructure for collaboration is created. The following sections present the
107

findings for four of the five conditions. The fifth condition, the backbone organization, was not
assessed in this study.
Common Agenda
A common agenda under collective impact requires a shared understanding of the
problem among participants and an agreed-upon approach for solving it (Kania & Kramer,
2011). The 12 interviewees demonstrated a shared understanding of the problem. Hannah shared
how she came to better understand the problem:
I had heard about the school-to-prison pipeline. In 2017, I invited [this expert speaker] to
speak at our steering committee meeting. She was a keynote at a local summit earlier that
year and had shared how this problem extends beyond the schools and is really a result of
and contributes to the intergenerational cycle of disadvantaged families. When we heard
her present, we realized that the more we worked in isolation, the more we were all
perpetuating the school-to-prison pipeline. We knew we needed to do more. So SACT
became the perfect structure to utilize to do this work.
Cynthia added,
SACT was already bringing together all of the agencies. And we had all the leaders of
these agencies sitting on the steering committee. We were always focused on prevention,
but the understanding of the intergenerational cycle of poverty and disadvantage and how
it was connected to the school-to-prison pipeline really gave us the momentum to say,
“We can work together to solve this. We can actually do this.” It’s like we finally got a
tangible problem to solve. This helped us to identify the essential elements as we were
working through our theory of change and beginning to develop a strategic plan.
108

Identifying a common problem helped the participants think of how to focus their efforts
to solve the problem. The IMTSS was introduced as a framework for how they could work
together to solve this problem. Diane shared, “We needed to have a framework that would add
structure and processes to the work. We needed to create an infrastructure that would help us
create sustainable change.” The IMTSS is closely aligned with the three SACT goals.
An analysis of SC meeting notes showed that in August 2020, SC members participated
in a half-day retreat focused on the IMTSS framework. The IMTSS has been an agenda item for
every SACT SC meeting since August 2020. There was also documentation to support that the
working groups are meeting regularly to establish their role in the IMTSS.
Based on the interviews and document analysis, the participants demonstrated a shared
understanding of the problem of the school-to-prison pipeline and its connection to the
intergenerational cycle of disadvantage. The SC members also have agreed to design and
implement the IMTSS as a solution to this problem. Therefore, the SC has a clearly established
common agenda.
Shared Measurement
As the SC members engage in this collective impact effort, they will need to identify and
agree to the data that will be used to measure impact. With a well-established common agenda,
the SC members can develop a shared measurement system. For SACT SC, the shared
measurement system will allow them to identify common metrics that can be used to track their
progress toward the common agenda (Kania et al., 2014). As the SC members build out their
shared measurement system, they will identify common metrics and relevant data for consistent
measurement of results to encourage partners to hold each other accountable and ensure
alignment of the work across agencies (Kania & Kramer, 2011).
109

The SACT theory of change was reviewed as part of the document analysis. While
individual and family outcomes were identified, there were no specific metrics tied to these
outcomes, indicating that there were no clear indicators for measurement. Beth provided some
additional context for this. She shared,
The theory of change offers a great starting point. But what’s missing is really what is
important to move this work? How are we measuring our effectiveness? What data
should we even collect? I think this is an important step for us to make sure we get to.
Otherwise, we continue to do what we have always done, which is to say that we are
working together, but we have no common data points to measure our work. One agency
collects the data, and it ends up looking like it was a single agency effort.
Despite not having clear metrics for the theory of change outcomes, using data to establish
partnerships was a strength for this group. Interviewees shared that data had been used to initiate
past collaborative efforts. Jack stated,
We use data to determine where to focus our efforts. In our [zip code] project, we used
data that showed, in [a specific zip code], Black and Brown juvenile offenders had the
highest rates of recidivism. So, we brought together all supporting agencies with a focus
on reducing crime. The data showed that in addition to reducing crime at the local high
school, we also reduced crime in that local neighborhood. The data helped to show our
progress and make the case for similar programs to be created and implemented in other
neighborhoods. What’s even more exciting is that we also saw reduced probation
violations among Black youth offenders. We made a difference.
Despite data being used to initiate partnerships, the examples shared reinforced what Beth
mentioned about single agency data collection. The impact on past collaborations was measured
110

through a single agency or a single data point. Under collective impact, patterns and potential
solutions are more easily found when looking at data across multiple organizations (Kania &
Kramer, 2011).
One roadblock identified by all interviewees was the lack of a shared data system across
the county and among agencies. Lucy shared,
The problem we have is we don’t have a shared data system. This makes it so difficult to
know which families have received services or what services a family has received. So,
we find that we end up having to start all over with a family as they move from one
agency to the next. This frustrates our families because they are having to retell their
story to each agency. So, we end up duplicating our efforts with some families and then
completely missing other families.
The lack of a shared data system involves the legality of what can be shared. Eric clarified,
I have been trying to get us to a shared data system. The issue lies with county counsel.
They are always concerned about what can be shared and what can’t. And I get it. They
need to think about how to protect the data of the families we serve. The problem is it can
hinder some of our work. But we will need to be creative with how we share data that
still meets the standards of county counsel.
As illustrated by these two comments, the lack of a shared data system creates a barrier for cross-
agency shared measurement. There are other concerns with sharing data. Francine provided
clarity:
A shared data system is ideal, but what we need to be mindful of is that we can cause
more harm and trauma if we are not careful. For example, I can think of a former client of
ours. She was in a DV [domestic violence] situation, and she needed to find new housing
111

quickly. When she went to one of our agencies, as she was providing her information
with the [clerk], it was the quick thinking of the [clerk] who purposefully did not enter
her information into one particular system because if the client’s alleged abuser had gone
into another agency and that agency had searched this system, then our client’s location
and status would have been revealed. And that’s just one example. We really have to be
careful that the data we collect is not used to create more harm.
Another theme that arose during the interviews was the need to be sure to apply an equity
lens to the data. Gail shared,
We need to measure the impact we will have on the most vulnerable populations. It’s not
just about reducing the number of families needing assistance or the number of youth
being incarcerated or whatever it is we are measuring. We need to be sure that in addition
to reducing the number of incarcerations that we are seeing a reduction for all groups.
Otherwise, we end up with our Black and Brown communities continuing to be
disproportionately impacted.
In the February, March, and April 2021 SC meetings, the members started to explore how
to move forward with shared measurement through a data and evaluation working group. A
review of meeting notes showed this group was assigned to develop a comprehensive evaluation
plan, including how current programs would be measured and evaluated for success and how
new programs would be developed. The task of establishing metrics to measure the established
individual and family outcomes identified in the theory of change was assigned to the prevention
working group. According to the May SC meeting, the prevention working group will work with
the data and evaluation workgroup to determine the metrics.
112

Based on the interviews and document analysis, the participants demonstrated awareness
of the lack of a shared measurement system, and they were working to determine how to
establish this in light of legal limitations. While this is a gap, it is one that they are actively
working to fix.
Mutually Reinforcing Activities
A common agenda and shared measurement system can be used to inform the mutually
reinforcing activities, which are coordinated differentiated activities tied to a comprehensive
action plan (Kania & Kramer, 2011). The SC members all identified the IMTSS as their
comprehensive action plan. SC members also agreed that their ability to work together was
important to addressing their common agenda. Cynthia shared,
In order for us to build a collective impact on outcomes, we need to have that partnership.
We need to address collectively all the issues that might come up upstream because you
know that saying, “it takes a village to raise a child.” You know, it's not one entity that
can hold everything in place for this family, so it takes that partnership and collaboration
to do that.
Another theme that emerged was the desire to take action. Every SC member spoke about a
desire to move from talking about collaboration and collective impact to taking action. Irene
explained, “We don’t have time to waste. We need to take action. The board wants action.”
Hannah added,
I think it’s important that we begin to operationalize everything we have been building.
We need to just try it out. It won’t be perfect, but we need to start doing something so we
can learn and improve as we go.
113

In sharing how they can collaborate differently, interviewees identified some of the barriers to
determining mutually reinforcing activities. Andrew described the partners as despite having the
best intentions of wanting to work together, they have still functioned in silos. He added,
Instead of these agencies really working together, we end up with these separate
programs or duplicated initiatives being created because everyone wants to solve the
problem on their own. It's almost like a maze. I've termed it, and I'm leading discussions
about how even with the best of intentions, we built this maze without having an
architect, you know. We just kept building and building and building to the point where
we’ve built this huge house with 106 rooms and, you know, 17 or 18 bathrooms, or
whatever. And you know what happens is everyone says, “Look, I built a great kitchen.”
And I'm like, “Yes, you did, but you know what? You also need a place to rest. You also
need a bathroom.” So, then some will say, “Look. I have a beautiful bathroom. Let's just
invest and make another bathroom.” It's like, “No, you need it all.” And I feel like we
need to conceptually raise that house, design a better one. So we can use the IMTSS as
our blueprint for how we work together.
This siloed approach led to multiple initiatives, with some that are duplicated. However, the
number of initiatives can be daunting and complicate coordination. Eric commented,
You know, I always say, and kind of half joke that I learn something new here every
day. And that's because you turn a corner or turn over another rock, and you discover
somebody in one of our partner agencies, or even right in my own agency doing some
new kind of thing that you hadn't heard of before. Yeah, that's amazing, but it's also
daunting in the sense of how do you coordinate that in a way that is truly effective
114

because, as we said, it's part of the challenge of all the work we do is that it can be quite
siloed and not really necessarily strategic.
In addition to the coordination challenge, another emerging theme was the misconception of the
expectations of mutually reinforcing activities. Gail provided some insight:
I think some members think, “Oh, now we all need to do the same thing. Then what is the
value of my agency if that group over there can solve the problem.” But that’s not what
this is about. We all get to do what we do best, and we bring that to the collective. If we
can’t agree to that, then we will continue to try to push our individual work ahead of the
needs of the community, which is why we are doing this work to begin with. In my past
partnerships, I didn’t stop what I was doing, nor did I ask any of my partners to stop their
work. What we did was focus our efforts so that the actions we took were aimed at
achieving our common focus. That’s what we can do, collectively.
Gail’s comments were echoed in Hannah’s earlier analogy of a puzzle and her wondering
whether a lack of trust prevented true collaboration among these agency partners. Irene added,
I think we are trying to build this integrated multi-tiered system of support, and we don’t
really know what this will be. It’s like we are building the plane while flying it. That’s
hard because there are so many unknowns. So, I think we know what we are working to
address. What is still unclear is how we will do it. And without that clear pathway, people
will just go back to what is comfortable and safe. They will do what they know works
because they don’t trust something they are not yet confident about. As the steering
committee, we need to be very confident about what we are doing so that confidence
spreads throughout the entire system. That’s important.
115

While many SC members identified some of the barriers that prevent the establishment of the
mutually reinforcing activities, Diane shared a vision for what it could become:
I think so many of us think the mutually reinforcing activities mean we Are all doing the
same thing. The truth is there will be many commonalities across the system. But the core
of what each agency or community-based organization does will still be unique to them.
But when we come together, we have to look at what we are doing together. So, take, for
example, at the larger SACT meeting that you observed, we had a presentation that
seemed to be another duplication. If we are really ready to be coordinated, we would take
that issue, and every leader would then say what their agency could do. So [Agency 1]
would take that issue back to think about it from their perspective. [Agency 2] would do
the same. [Agency 3], [Agency 4], [Agency 5], and so on. Everyone would be working to
address the same issue from what they do. And in that, we may find that [Agency 5] and
[Agency 2] need to work together or whoever else would need to work together. But we
are not all doing the same thing. We are all still needed and relevant. We also won’t
engage at the same time. We may have to prioritize what comes first, but it doesn’t make
one group more important to the system than another.
Based on the interviews and meeting observation, the participants demonstrated they had
not yet agreed on the mutually reinforcing activities. While they were utilizing the IMTSS as
their comprehensive action plan, there was still a gap in how the partners work together.
Continuous Communication
According to Kania and Kramer (2011), continuous communication in collective impact
is important for developing and building trust among partners. The SACT represents an existing
structure that can be utilized to communicate across agencies. The SACT and the SACT SC meet
116

regularly. Each group meets 11 times a year, with additional meetings added as needed. Agendas
for each meeting are sent ahead of the meeting, and meeting notes are sent after each meeting.
During the pandemic, the meetings shifted to an online format. Further document analysis
showed that the SACT working groups have started to meet regularly. Each working group was
given time at the SACT retreat to establish a team charter and to set a regular meeting schedule.
The leads of each working group shared the progress of their work at the SACT SC and general
SACT meetings in May and June. Cynthia celebrated this as progress. She said,
Before, we had these workgroups on paper, but the only workgroup that did anything
was the prevention workgroup, and so a lot of the work fell to that group. There really
was very little structure as there was just so much to do. Now we have these different
workgroups that are all responsible for their part. Finance, data and evaluation, standards
of excellence, community engagement, and prevention. So, the prevention workgroup
gets to focus on prevention. And the other groups get to build out what their work looks
like. And that’s exciting because it took us a long time to get here. So now the next step
is to make sure these groups communicate with each other, and the work is connected.
Lucy also expressed some concern about the working groups. She added,
I think we have made so much progress when it comes to the workgroups. But I don’t
think that many people understand how the groups will work together. So, like in the
prevention workgroup meetings, I’ll hear members there talking about making sure to
include community voice and getting community to the table. This is all fine and good,
but we have a community workgroup that can go out to the community and get the
feedback from the community, so the prevention workgroup does not need to do this. It’s
hard to get people to see how we all work together to accomplish the larger task.
117

In addition to concerns about how the working groups communicate with each other and
the larger SACT, SC members also expressed concern about a lack of communication about
existing or potential collaboration efforts. Jack shared that this lack of communication “about
collaboration across agencies continues to contribute to the siloed services.” While each
participant spoke about the many times they have partnered with other leaders across agencies,
there were many partnerships they did not know about. Jack added, “I think that’s what SACT is
supposed to facilitate, an open space for this work to occur, but I don’t see it yet.”
Another concern that emerged was communicating the work throughout each
agency. Ten of the SC members shared this as a concern. Francine noted that while they can
utilize the SACT SC meeting structure and the SACT general group meeting structure, she was
not very confident that any members were updating their agencies about the direction of the
IMTSS. She shared,
I worry that the work just lives at the steering committee level but really isn’t getting
communicated throughout each agency. It is important that the ‘boots on the ground’
workers really know what this is about. They are the ones who are closest to our clients.
They won’t know how to work differently if we don’t communicate. Our agencies won’t
make the necessary changes if we don’t communicate. Nothing will really change if we
don’t communicate.
Beth shared some of the same concerns. She shared,
It’s so frustrating at times. I have started to take this message back to my team. I am
communicating with my team to let them know what we are doing and how we are
moving the work. I just don’t think all of us are doing that. How will anything change if
we are not taking the message back to our teams? The work does not stop with us.
118

A review of the SC meeting notes showed that the SC members have talked about how to
utilize the SACT meetings to better communicate with the larger SACT group. As previously
noted, in the May and June 2021 SC meetings, the leads of the working groups were invited to
provide an update about the progress of each working group. According to the SC meeting
minutes from April 2021, the purpose of inviting these leaders was to improve communication
from the working groups back to the SC and the larger SACT group. However, there was no
record of how the working groups communicated with each other.
Based on the interviews and document analysis, the participants demonstrated they have
an existing meeting structure for communication. They also demonstrated they are working on
improving communication from the working groups to the SC and the larger SACT group.
However, they identified a communication gap in the sharing of new initiatives. Another
communication gap was in communicating the work on the IMTSS to the member agencies.
Summary of Research Question Three: Readiness for Collective Impact
The findings for the third research question were intended to provide further insight into
the interviewees’ readiness in terms of the five conditions of collective impact and how those
were established in the IMTSS collective impact initiative. The findings were organized into two
of the three phases of collective impact implementation. The interview and document analysis
data demonstrated that the SC members had met the criteria of Phase I, initiating action.
Participants were part of an initial governance structure and were all strong and credible
champions. They understood the urgency of engaging in the collective impact effort. The data
collected for Phase II, organizing for impact, identified which conditions were established and
which were not. The participants had a clearly defined common agenda. They were working on
establishing a formal continuous communication plan. The shared measurement and mutually
119

reinforcing activities had not yet been established. Nonetheless, they can leverage their KMO
assets to implement the collective impact conditions.
Summary of Validated Influences
This chapter utilized a qualitative approach, utilizing interview data, meeting
observations, and artifact analysis to assess the three study questions:
1. What are the SACT steering committee members’ knowledge, motivation, and
organizational resources related to creating a plan to implement the IMTSS?
2. What is the interaction between organizational culture and context and the steering
committee members’ knowledge and motivation?
3. In what way is the SACT steering committee positioned to engage in collective impact?
Tables 8, 9, and 10 summarize the findings for each research question, including whether
the KMO influences were determined to be an asset or a need. Recommendations for each
demonstrated need will be addressed in Chapter Five.

120

Table 8
Summary of Findings for Research Question One
Assumed KMO influence Asset or need
Knowledge: factual
Steering committee members
know the SACT’s mission
and theory of change and can
define the IMTSS
framework.
Asset
Knowledge: conceptual
Steering committee members
know why cross-sector
collaboration is needed to
achieve their goal.
Asset

Steering committee members
know the role of the SACT
member agencies and the
partners in the IMTSS.
Asset
Knowledge: procedural
Steering committee members
know how to communicate
with each other across
agencies.
Partial need (partial need is indicated because there is a
lack of a systemic process for collaboration.
Additionally, the agenda for the larger SACT meeting
is not always connected to the IMTSS, creating a
missed opportunity to build capacity and
understanding around the IMTSS.)
Knowledge: metacognitive
Steering Committee members
think about how they can
collaborate differently under
the IMTSS collective impact
effort.
Asset

121

Assumed KMO influence Asset or need
Motivation: utility valu
Individually and collectively, steering committee
members need to see the value of the alignment
between the SACT’s common agenda and their
agency goals.
Asset
Motivation: efficacy
Individually, steering committee members are
confident that they can effectively work together
with other partner agencies.
Collectively, steering committee members are
confident that the group’s collective efforts will
be effective in accomplishing the SACT goals.
Asset

Asset
Organization: resources
The organization provides the SACT steering
committee with the funding and time to
implement collective impact in order to achieve
its priorities.
Partial need indicated given the
restrictions of categorical funding
and the reliance on government
funding.
Organization: cultural setting
The organization provides the SACT steering
committee the resources to implement collective
impact to achieve its priorities.
Asset
Organization: cultural models
The organization promotes the use of innovation
and creativity to solve problems.
Asset

122

Table 9
Summary of Findings for Research Question Two
Organizational Context Asset or need
SACT steering committee members expressed a disconnect between the
SACT steering committee and the larger SACT. While the SACT
steering committee members understand and are motivated to work on
the IMTSS, the larger shifts are not occurring within the larger SACT
group, which is responsible for operationalizing the IMTSS.
Need
Organizational Setting Asset or need
Steering committee members expressed gratitude for the support they
received from both the county’s board of supervisors and executive
officers. Support is provided through funding as well as the time to
engage in this multi-year effort.
Asset
Organizational Culture
Steering committee members’ collaborative efforts are often stymied by
duplicative projects and meetings, as well as inefficient communication
within and across agencies.
Need

The SACT SC has made progress in working through both phases of collective impact
implementation. Phase I is fully established. The findings for the collective impact conditions
assessed in this study are presented below.

123

Table 10
Summary of Findings for Research Question Three
Collective impact condition Implementation status
Common agenda Established: A common agenda has been established.
Shared measurement Not established: While data is used to establish partnership,
the lack of a shared data system creates additional barriers to
collaboration. Additionally, metrics to measure progress
toward individual and family outcomes have not been
established.
Mutually reinforcing activities Not established: While the members utilize the IMTSS as a
comprehensive action plan, there is still a gap in how the
partners actually work together.
Continuous communication In progress: There is a communication gap regarding new
initiatives. A gap also exists in communicating the work of
the IMTSS throughout their agencies.

Based on the findings presented in this chapter, Chapter Five will offer empirical
evidence-based recommendations for closing the gaps in the validated and partially validated
influences, as well as the organizational supports necessary to support the SACT SC as it
progresses in implementing their collective impact effort.

124

Chapter Five: Recommendations and Evaluation
The study findings presented in Chapter Four were organized according to the three study
questions. Clark and Estes’s (2008) framework was utilized to evaluate the degree to which the
SACT SC was meeting its goal of fully implementing the IMTSS using the collective impact
framework. The study focused on the SACT SC’s readiness to engage in collective impact by
evaluating KMO influences related to achieving the larger organizational goal. In addition to the
confirmed assets and gaps related to the KMO influences, Chapter Four also presented findings
related to the implementation status of each of the four collective impact conditions evaluated for
this study. The findings suggest that while the SACT SC is well-positioned to implement the
IMTSS using collective impact based on the KMO assets, there is also a need to address the
validated needs in the identified KMO influences to better position the SC engage in collective
impact initiative. The findings suggest more assets than gaps. The findings also suggest that the
KMO assets can be leveraged to increase progress toward establishing the collective impact
conditions, including a common agenda, shared measurement system, mutually reinforcing
activities, and continuous communication.
The intent of this chapter is to offer recommendations to address the validated KMO
needs and leverage the KMO assets to implement the collective impact conditions. First, Chapter
Five will review some foundational elements of this study, including the organization and
stakeholder group of focus. Next, the chapter will use the results and findings to recommend
evidence-based solutions and recommendations. These recommendations will be organized
according to the KMO influences and linked to specific collective impact conditions. Then, an
integrated implementation and evaluation plan, based on the New World Kirkpatrick Model
125

(Kirkpatrick & Kirkpatrick, 2016). Finally, study limitations will be discussed, and suggestions
for future research will be presented.
Organizational Context and Mission
Siloed Services County’s vision is that all children and families are safe, healthy, and
nurtured, and thrive in education, career, and life. Despite its premier status for technology,
significant racial disparities impact the African American youth (ages 0–24) population of SSC.
In addition to the disproportionate poverty rates, 2018–2019 data show that African American
youth in SSC also experience persistent disparities in educational attainment, suspension and
expulsion, placement in foster care, and juvenile arrests. African American families in SSC also
experience persistent disparities in median income and infant mortality rates.
In 2009, the SSC Board of Supervisors established the SACT, a cross-system, multi-
stakeholder initiative to bring together many leaders of child- and family- serving departments
and organizations to address racial and economic disparities that harm many children and
families. The purpose of SACT is to create and leverage opportunities for cross-system
coordination to achieve the SSC mission of providing services and opportunities to enhance the
quality of life for individuals, families, and their children. In the 12 years since its creation,
SACT has increased opportunities for coordinated services, but agencies continue to work in
isolation, focused on their specific agency goals, leaving many families underserved.
Organizational Performance Goal
The SACT’s performance goal is to use the collective impact framework to design and
implement an IMTSS across all child- and family-serving agencies by June 30, 2022, to
streamline and improve the quality of services provided to the community. The IMTSS is
intended to focus collective efforts on prevention and intervention using the collective impact
126

framework with an ultimate goal of dismantling the school-to-prison pipeline by eliminating the
intergenerational cycle of disadvantage.
Description of Stakeholder Group for the Study
The stakeholders at the center of this study were members of the SACT SC. They
coordinate the implementation of the SACT’s overall vision. In addition to planning the meetings
for the larger SACT body, they also define and steer the SACT group’s work.
Goal of the Stakeholder Group for the Study
The SACT SC’s goal is to use the conditions of collective impact to establish an
implementation plan for the IMTSS. This goal is directly linked to the larger organizational goal
of using the collective impact framework to implement the IMTSS to streamline and improve
service delivery across child- and family-serving departments and agencies and the broader
community.
Purpose of the Project and Questions
The purpose of this study was to evaluate the degree to which the SACT was meeting its
goal of fully implementing the IMTSS using the collective impact framework. The IMTSS is
aimed at dismantling the school-to-prison pipeline by eliminating the intergenerational cycle of
disadvantage. The study focused on the participants’ readiness to engage in collective impact by
evaluating their KMO influences related to achieving the larger organizational goal. The
following questions guided this study:
1. What are the SACT steering committee members’ knowledge, motivation, and
organizational resources related to creating a plan to implement the IMTSS?
2. What is the interaction between organizational culture and context and the steering
committee members’ knowledge and motivation?
127

3. In what way is the SACT steering committee positioned to engage in collective impact?
Recommendations to Address Knowledge, Motivation, and Organization Influences
This section will present specific recommendations related to the validated influences
presented in Chapter Four. These recommendations will be leveraged into recommendations to
establish and strengthen progress with developing four of the collective impact conditions. The
fifth condition, backbone organization, was not included as part of this study. Therefore, the
recommendations focus on the remaining four conditions.
Knowledge Recommendations
The knowledge influences assessed in this study were factual, conceptual, procedural,
and metacognitive. Influences were validated through individual interviews, document analysis,
and meeting observations. The research findings validate that the factual, conceptual, and
metacognitive knowledge were assets for the participants. These assets can be used to strengthen
the SC members' progress in developing the collective impact conditions. There was a partial
need identified in procedural knowledge. Table 11 displays a complete list of the knowledge
influences with priority level and evidence-based principles to support the recommendations.
The sections following the table provide a detailed discussion for each high-priority influence
and its connection to the collective impact conditions, the associated principle, and the specific
recommendations based on the supporting literature.

128

Table 11
Summary of Knowledge Influences and Recommendations
Assumed
knowledge
influence
Asset
or
need
Priority
(high,
low)
Principle and citation Context-Specific
recommendation
Factual knowledge
Steering
committee
members know
the SACT’s
mission and
theory of change
and can define
the IMTSS
framework.

Asset Low Information learned
meaningfully and
connected with prior
knowledge is stored
more quickly and
remembered more
accurately because it
is elaborated with
prior learning (Schraw
& McCrudden, 2006)
Reinforce the common
agenda through explicit
connections
between what SC
members already know
about the SACT’s
mission and theory of
change and the IMTSS.

Establish mutually
reinforcing activities
through explicit
connections to the
SACT theory of
change.
Conceptual knowledge
Steering
committee
members know
why cross-sector
collaboration is
needed to
achieve their
goal.
Asset High Creating schemata helps
organize DK in a
domain (Schraw et al.,
2009)

Managing intrinsic load
by segmenting into
simpler parts and pre-
training, among other
strategies, enables
learning to be
enhanced (Kirschner
et al., 2009)
To determine mutually
reinforcing activities,
provide a job aid that
details how each SC
partner agency
contributes to the
collective effort and
how their engagement
in this effort
streamlines services for
the community

Steering
committee
members know
the role of the
SACT member
agencies and the
Asset High Managing intrinsic load
by segmenting
complex material into
simpler parts and pre-
training, among other
strategies, enables
To establish the need for
a formalized
communication loop
for continuous
communication,
present a job aid that
129

partners in the
IMTSS.
learning to be
enhanced (Kirschner
et al., 2009)
details how the
working groups will
work together to
advance the collective
impact effort focused
on the IMTSS.
Procedural knowledge
Steering
Committee
members know
how to
communicate
with each other
across agencies.
Partial
need
High To develop mastery,
individuals must
acquire component
skills, practice
integrating them, and
know when to apply
what they have
learned (Schraw &
McCrudden, 2006)

Continued practice
promotes automaticity
and takes less capacity
in working memory
(Schraw &
McCrudden, 2006)
Use case studies to have
SC members practice
communicating with
each other around a
focused need in order
to demonstrate the
importance of
establishing a process
for continuous
communication across
agencies.

Metacognitive knowledge
Steering
Committee
members think
about how they
can collaborate
differently under
the IMTSS.
Asset High The use of
metacognitive
strategies facilitates
learning (Baker, 2005)
Learning and motivation
are enhanced when
learners set goals,
monitor their
performance, and
evaluate their progress
towards achieving
their goals (Ambrose
et al., 2010)
Engage the SC members
in regular data talks at
their monthly meetings
(common agenda, shared
measurement system,
mutually reinforcing
activities, continuous
communication)

130

Factual Knowledge
The assumed influences to assess factual knowledge were that the SACT SC members
knew the SACT’s mission and theory of change and could define the IMTSS framework. Based
on the findings, factual knowledge was determined to be an asset. Although this was not
identified as a high priority, it is important that members leverage this asset as they engage in
their collective impact initiative. Schraw and McCrudden (2006) found that when prior
knowledge is meaningfully connected to new learning, the information is stored more quickly
and remembered more accurately. The recommendation is to make explicit connections between
what SC members already know about the SACT’s mission and theory of change and the IMTSS
to connect to their common agenda, mutually reinforcing activities, and a shared measurement
system. The findings also show that a common agenda was established. Therefore, the SC
members’ prior knowledge can be used to refine their common agenda as they begin their
collaboration towards developing the implementation plan for the IMTSS. The SC members can
also use the common agenda to establish a common understanding of what they have agreed to
do together (Kania & Kramer, 2011). This will begin their work towards defining their mutually
reinforcing activities.
Conceptual Knowledge
The assumed influence to assess conceptual knowledge was that the SACT SC members
knew why cross-sector collaboration was needed to achieve their goal. Although this was an
asset, it was identified as a high need to support participants’ engagement with collective impact.
Townsley et al. (2004) found that knowledge of the importance of working together increases
commitment to collaboration. Therefore, this asset can be leveraged towards determining their
mutually reinforcing activities. Kirschner et al. (2009) recommended that managing intrinsic
131

load by segmenting into simpler parts and pre-training to enhance learning. Additionally, Schraw
et al. (2009) found that creating schemata helps to organize knowledge in a domain. Therefore,
the recommendation is to provide a job aid that details how each SC partner agency contributes
to the collective effort and how their engagement streamlines services for the community.
Another influence used to assess conceptual knowledge was that the SACT SC members
knew the role of the SACT member agencies and the partners in the IMTSS. This was identified
as both an asset and a high need for collective impact efforts. Collective impact is not the simple
act of bringing together multiple agency partners to work together. Instead, collective impact
builds and leverages the relationships among organizations focused on the common agenda
towards systemic change (Kania & Kramer, 2011). To increase understanding of the role of
the member agencies and partners, the SC members must continually communicate with each
other as they work to determine their mutually reinforcing activities. As already mentioned,
Kirschner et al. (2009) found that learning is enhanced when the learner’s working memory is
not overloaded and recommended segmenting complex material into simpler parts. Therefore,
the recommendation is to provide a job aid that details how the working groups will work
together to advance the collective impact effort focused on the IMTSS.
Procedural Knowledge
The assumed influence to assess procedural knowledge was that the SC members knew
how to communicate with each other across agencies. This was identified as a partial need.
While members indicated comfort with communicating with each other across agencies, they
also indicated that there was not a systemic process for collaboration. They also identified a need
to improve communication to the larger SACT, specifically around the IMTSS. Continuous
communication is a core tenet of collective impact (Kania & Kramer, 2011). Successful
132

communication across agencies in a multi-agency collaboration is challenging, yet it is
successful communication that contributes to the overall success of the collaboration (Gilliam et
al., 2016; Hanleybrown et al., 2012; Kania & Kramer, 2011, 2013; Townsley et al., 2004;
Weaver, 2016). As presented in Chapter Two, regular inter-agency meetings facilitate
communication opportunities (Kania & Kramer, 2011; Moran et al., 2007) when determining
new and inventive ways of responding to problems (Hicks et al., 2008).
Schraw and McCrudden (2006) found that to develop mastery, individuals must acquire
component skills, practice integrating them, and know when to apply what they have learned.
Further, they also found that continued practice promotes automaticity and takes less capacity in
working memory, thus leading to mastery. Therefore, a recommendation is to use case studies to
have SC members practice communicating with each other around a focused need to demonstrate
the importance of establishing a process for continuous communication across agencies. In
addition to regular meetings, structured processes for communication help set expectations for
how communication should occur (Gilliam et al., 2016) as well as what should be communicated
(Kania et al., 2014). As such, another recommendation is to utilize the larger SACT meetings to
provide regular updates on the IMTSS with specific tasks for members to take back to their
agencies. Remaining consistent with monthly meetings and will provide the consistent practice
needed to build automaticity.
Metacognitive Knowledge
The assumed influence to assess metacognitive knowledge was that the steering
committee members thought about how they could collaborate differently under the IMTSS
collective impact effort. The findings support that the SC members had sufficient metacognitive
knowledge as they regularly reflected on working more efficiently to increase their effectiveness.
133

In fact, their direction towards collective impact resulted from their reflection that they needed to
have a more structured way to collaborate. As they implement the IMTSS, they need to clearly
define how they will work together and serve the community. As they work to ensure a feedback
loop among agencies as well as between the county agencies and the community, they can
leverage this asset to improve the IMTSS through their common agenda, continuous
communication, shared measurement, and mutually reinforcing activities. Baker (2005) found
that the use of metacognitive strategies facilitates learning. Learning and motivation are
enhanced when learners set goals, monitor their performance, and evaluate their progress
(Ambrose et al., 2010). A recommendation to leverage metacognitive knowledge is for the SC to
engage in regular data talks to ensure efforts are aligned and to learn from each other.
Motivation Recommendations
The findings from this study validated the motivation influences within utility value and
efficacy. Influences were validated through individual interviews and meeting observation. The
research findings validate that both utility value and efficacy were assets. These assets can be
used to strengthen progress in developing the collective impact conditions. Table 12 displays a
complete list of the motivation influences with priority level and evidence-based principles to
support the recommendations. The sections following the table discuss each high priority
influence, its connection to the collective impact conditions, the associated principle, and the
specific recommendations based on the supporting literature.

134

Table 12
Summary of Motivation Influences and Recommendations
Assumed motivation
influence
Asset
or
need
Priority
(high,
low)
Principle and
citation
Context-Specific
recommendation
Motivation: utility value
Individually and
collectively,
steering
committee
members need to
see the value of
the alignment
between the
SACT’s common
agenda and their
agency goals.
Asset High Learning and
motivation are
enhanced if the
learner values the
task (Eccles,
2006).

Rationales that
include a
discussion of the
importance and
utility value of
the work or
learning can help
learners develop
positive values
(Eccles, 2006;
Pintrich, 2003).

Providing tasks,
materials, and
activities that are
relevant and
useful to learners
enhance their
value for the
work (Rueda,
2011)
Engage in a simulation
activity that allows for SC
members to identify specific
examples of how the
common agenda aligns with
their agency goals and
reinforces why cross-agency
collaboration, through
mutually reinforcing
activities and continuous
communication, is
necessary to achieve the
goal.

135

Assumed motivation
influence
Asset
or
need
Priority
(high,
low)
Principle and
citation
Context-Specific
recommendation
Motivation: efficacy
Individually, steering
committee
members are
confident that they
can effectively
work together to
accomplish the
stated goal.

Collectively, steering
committee
members are
confident that the
group’s collective
efforts will be
effective in
accomplishing the
SACT goals.
Asset High High self-efficacy
can positively
influence
motivation
(Pajares, 2003).

Learning and
motivation are
enhanced when
learners have
positive
expectancies for
success (Pajares,
2003)

When individuals
exercise agency at
the group level
and perceive
collective effort,
motivation is
enhanced through
resilience and
performance
(Bandura, 2012)
As a part of the simulation
activity noted above, group
agencies together to define
specific actions they can
take in order to address
that part of the prevention
continuum as outlined in
their theory of change (ex.
addressing readiness for
school will involve
schools, medical health
services, behavioral health
services, child and family
support services, etc.).
This will be used to define
their mutually reinforcing
activities.

Utility Value
The assumed influence to assess utility value was that individually and collectively,
steering committee members needed to see the value of the alignment between the SACT’s
common agenda and their agency goals. Eccles (2006) found that learning and motivation are
enhanced if the learner values the task. The findings showed that participants valued the
alignment of the common agenda and their agency goals, making this influence an asset. When
136

there is no alignment, participants have to balance working towards their agency goals while also
working towards the goals of the collective impact effort, creating a motivational barrier to
accomplishing the goal (Gilliam et al., 2016; Moran et al., 2007). As SC members design and
implement the IMTSS, they can leverage this asset to establish mutually reinforcing activities
and the shared measurement system, as well as encourage engagement through continuous
communication. Eccles (2006) and Pintrich (2003) found that when rationales include a
discussion of the importance and utility value of the work or learning, learners develop positive
values. To increase or enhance utility value, Pintrich (2003) also found that activities should be
relevant and useful to the learners, connected to their interests, and based on real-world tasks.
Additional research on motivation points to the importance of task value. Rueda (2011) found
that providing tasks, materials, and activities that are relevant and useful to learners enhances
their value for the work. Therefore, a recommendation is to provide training that includes
engaging participants in a real-world simulation activity that pushes SC members to identify
specific examples of how the common agenda aligns with their agency goals and reinforces why
cross-agency collaboration is necessary to achieve the goal.
Efficacy
The assumed influence to assess self-efficacy was that the steering committee members
were confident that they could effectively work together to accomplish the stated goal. Pajares
(2003) found that high self-efficacy can positively influence motivation which is enhanced when
learners have positive expectancies for success. As a group, SC members understand that
individual agencies will be unable to accomplish the goal in isolation and that a collective effort
is needed. The assumed collective efficacy influence was that, collectively, SC members were
confident that the group’s collective efforts would be effective in accomplishing the SACT goals.
137

The findings indicate participants had a clear understanding of why they needed to work together
to accomplish the goal and were generally confident that they could stay focused on the common
agenda. Bandura (2012) found that exercising agency at the group level and perceiving collective
effort enhance individuals’ motivation through resilience and performance.
Building on the momentum from past successful collaborations, SC members recognized
the need to systematize collaboration across agencies. While both self and collective efficacy
were assets for the SC members, they can use their motivation to work together to develop a
shared measurement system to monitor their progress and use their monthly meetings to foster
continuous communication. As a continuation of the simulation activity to remind members of
the value of the collective impact, they can establish mutually reinforcing activities by discussing
how they would work together to address the needs of the family in the simulation along
different parts of their prevention continuum.
Organizational Recommendations
The findings from this study validated the organizational influences of resources, cultural
settings, and cultural models as they relate to the problem of practice. Influences were validated
through individual interviews and document analysis. The research findings validate that cultural
setting was an asset while resources and cultural models were partial assets. The partial gaps for
both resources and cultural models were tied to reliance on government funding and restrictions
on funding that affect staffing. These are not gaps that can be addressed in this study, as reliance
on government funding will not change. Table 13 displays a complete list of the organizational
influences with priority level and evidence-based principles to support the recommendations.
The sections following the table discuss each high priority influence, its connection to the
138

collective impact conditions, the associated principle, and the specific recommendations based
on the supporting literature.

Table 13
Summary of Organizational Influences and Recommendations
Assumed
organizational
influence
Asset
or
need
Priority
(high, low)
Principle and citation Context-Specific
recommendation
Organization: resources
The organization
provides the
SACT steering
committee with
the funding and
time to
implement
collective
impact in order
to achieve its
priorities.
Need Low*
This is a low
priority due
to the
limitations
of this
study
Effective change
efforts ensure that
everyone has the
resources needed to
do their job and that
if there are resource
shortages, then
resources are
aligned with
organizational
priorities (Clark &
Estes, 2008)
Allocate funding to
support training and
coaching for the
steering committee
members to engage
in the collective
impact initiative.
Organization: cultural setting
The organization
provides the
SACT steering
committee with
the needed time
and staff to
implement
collective
impact in order
to achieve its
priorities.
Asset High Effective change
efforts are
communicated
regularly and
frequently to all key
stakeholders (Clark
& Estes, 2008).
Create a formal
communication and
feedback mechanism
to engage all
stakeholders,
including steering
committee members
and SACT working
group members.
(continuous
communication)

139

Assumed
organizational
influence
Asset
or
need
Priority
(high,
low)
Principle and citation Context-Specific
recommendation
Organization: cultural models
The organization
promotes the
use of
innovation and
creativity to
solve
problems.
Asset High Effective change
efforts utilize
feedback to
determine when/ if
improvement is
happening (Clark
& Estes, 2008)
Effective change
efforts use
evidence-based
solutions and adapt
them, where
necessary, to the
organization’s
culture (Clark &
Estes, 2008).
Foster continuous
communication by creating
feedback loops, including (a)
a designated time in SC
meetings for the working
groups to provide updates on
their progress and receive
feedback for improvements;
(b) annual SC retreats to
allow SC members to reflect
on the work accomplished
that year and to plan for the
upcoming year; and (c)
individual coaching support
for each SC member and
working group leader to
increase leadership skills.

Resources
The findings validate that the assumed influence of resources was a partial need.
Effective change efforts ensure that everyone has the resources needed to do their job and that,
when there are shortages, the remaining resources align with organizational priorities (Clark &
Estes, 2008). Interviewees shared that they receive the time to commit to this work. However,
the partial need was identified based on the instability of government funding, especially in light
of the current public health crisis and its economic impact. The unstable funding makes this a
high priority. One of the collective impact preconditions includes having adequate financial
resources to last for two to three years, at a minimum (Kania & Kramer, 2011).
Recommendations to resolve funding needs are beyond the scope of this study and are the
140

responsibility of the backbone organization, which was not included in this evaluation. Those
overseeing the collective impact initiative must have time, space, and coaching as they learn the
steps to implement IMTSS collective impact initiative (Schein & Schein, 2017). Therefore, one
recommendation is to allocate funding to support training and coaching for the SACT SC to
engage in the collective impact initiative.
Cultural Settings
The findings validated the assumed influence of cultural settings as an asset, showing that
the organization provides the resources to implement collective impact to achieve its priorities.
Participants reported that they have ample time to engage in this work during their regular work
time. They also staff the working groups with other staff from throughout their respective
agencies to participate in the working groups. Although this was an asset, the interviewees
expressed concern about having their staff participate in the larger SACT meetings that seemed
disconnected from the IMTSS work planned in the committee meetings. Clark and Estes (2008)
found that effective change efforts are communicated regularly and frequently to all key
stakeholders. Therefore, a recommendation to address this gap will be to create a formal
communication and feedback mechanism to engage all stakeholders.
Cultural Models
The findings validated the assumed influence of cultural models as an asset, showing that
the organization promotes innovation and creativity to solve problems. Unlike technical
problems that tend to be more easily solved, solutions to complex problems are not
predetermined (Kania & Kramer, 2011). In assessing successful collective impact initiatives,
Kania and Kramer (2013) found that each successful initiative discovered a new approach to
141

solve a complex problem. The following recommendations are intended to strengthen readiness
for collective impact to leverage this asset.
Clark and Estes (2008) wrote that effective change efforts use evidence-based solutions
and adapt them, where necessary, to the organization’s culture. A recommendation of annual
retreats intends to provide a dedicated time and space to explore innovative solutions and adapt
to the SACT culture. The SACT SC members should also participate in individual coaching to
improve leadership skills as they will be required to lead the collective impact initiative and
implement the IMTSS throughout their organizations.
Effective change efforts also utilize feedback to determine when/if improvement is
happening (Clark & Estes, 2008). The importance of continuous communication has already
been addressed in earlier sections of this study. The recommendation to continue to designate
time in SC meetings for the working groups to provide updates on their progress and receive
feedback for improvements is intended to increase communication and allow for informal
assessment of the overall SACT progress toward implementing the IMTSS.
Implementation and Evaluation Plan
The SACT is working to dismantle the school-to-prison pipeline and eliminate the
intergenerational cycle of disadvantage. To accomplish this mission, the SACT SC established a
goal of implementing the IMTSS through collective impact. This study sought to evaluate its
readiness to do so. The previous section offered recommendations to address the KMO gaps to
bolster this readiness. The following section offers an implementation and evaluation plan with
recommendations to advance the collective impact effort.
The model that guided this implementation and evaluation plan is the New World
Kirkpatrick Model (Kirkpatrick & Kirkpatrick, 2016). This model offers four levels of training
142

evaluation and begins with a focus on the desired results (Level 4) and works backward to
reaction (Level 1). Level 4 focuses evaluation on the results: the desired outcome. Level 3
measures the application of the learning represented in a change of the participants’ behavior as a
direct result of the training. Level 2 focuses on learning and measures how much knowledge was
acquired as a result of the training. Level 1 measures the participant’s reaction to the training.
While the New World Kirkpatrick Model includes the same four levels as the historical
Kirkpatrick model, the new model is designed in reverse (Kirkpatrick & Kirkpatrick, 2016). The
intent of the backward design is to develop a plan centered around the leading indicators for
results and the desired outcomes of learning (Kirkpatrick & Kirkpatrick, 2016). With this new
approach, the four levels are presented from Level 4 to Level 1.
Level 4: Results and Leading Indicators
Level 4 of the New World Kirkpatrick Model (Kirkpatrick & Kirkpatrick, 2016) begins
with the end, or desired outcome, in mind. The goal is to ensure that all training is designed with
a focus on the desired outcome (Kirkpatrick & Kirkpatrick, 2016). Results, as defined for this
specific organizational context, refer to the SACT mission. Table 14 shows the results and
leading indicators in the form of outcomes, metrics, and methods for external and internal
outcomes. The external outcomes refer to the long-term family and youth well-being outcomes
connected to the overall mission to eliminate the intergenerational cycle of disadvantage
(Kirkpatrick & Kirkpatrick, 2016). The internal outcomes refer to the organizational outcomes
that can be measured throughout departments and teams to indicate progress toward the desired
results.

143

Table 14
Outcomes, Metrics, and Methods for External and Internal Outcomes
Outcome Metric(s) Method(s)
External outcomes
Comprehensive coordinated
service delivery
Number of partners who commit to
supporting the IMTSS
Signed
commitment
forms
Families linking and engaging
with services and resources
Number of families accessing
services at the community-based
demonstration sites
User data collected
at demonstration
sites
Tiered interventions to access
services and supports
Database of tiered services across all
partnering agencies
Completed
database
Internal Outcomes
Common agenda established by
the steering committee
Number of SACT members and
agency staff who can identify the
common agenda of the IMTSS
SACT member and
staff surveys

Number of SACT members and
agency staff who agree that the
common agenda is relevant to the
mission of their organization/
department
SACT member and
staff surveys

Number of community members
involved in determining and
designing the agreed-upon
solutions
Feedback from
community
Data showing
community
involvement
Shared measurement system
created by the steering
committee
SC agreement on five key indicators
or data points that will be
collected to measure progress
SC meeting
minutes
SC survey

Number of staff who know these
identified indicators or data points
Staff survey

Number of agencies who commit to
sharing their data as per the
identified indicators
Agency survey

144

Outcome Metric(s) Method(s)
Internal Outcomes
Shared measurement system
created by the steering
committee
Amount of time allocated at each SC
meeting for the sharing,
discussion, and reflection on data
SC meeting
minutes
SACT agency activities are
aligned to and mutually
reinforcing of the IMTSS
Number of meetings of each SACT
working group carrying out the
actions of the IMTSS action plan
Working groups
meeting minutes

Amount of time allocated at the SC
meetings for working group leads
to provide updates on the status of
each working group
SC meeting
minutes

Number of agency partners who can
identify what they contribute to
the IMTSS
SACT member
survey

Number of agency/ department
leaders who agree that their
agency/ department activities are
aligned to the IMTSS
Staff survey
Strategies and formal
mechanisms are utilized
across SACT agencies to
support continuous
communication
Monthly SC meetings that include
time for updates from working
groups
SC meeting
minutes
Meeting attendance

Monthly working group meetings Working group
team charter
Working group
meeting minutes
Meeting attendance

Monthly SACT meetings SACT meeting
minutes
Meeting attendance

145

Level 3: Critical Behaviors
As described earlier, Level 3 refers to the application of learning as evidenced by the
behaviors of the SC members, the department/unit leaders, and agency staff. Despite agreeing to
utilize the collective impact framework in 2019, the IMTSS initiative is still in the early stages of
implementation. Hanleybrown et al. (2012) identified the three phases as (a) initiate action, (b)
organize for impact, and (c) sustain action and impact. Based on the findings, the SACT SC has
completed Phase 1 and is in the middle of Phase 2. As such, this section provides suggestions for
critical behaviors to address implementation gaps in Phase 2 to move to Phase 3. Table 15
presents the critical behaviors along with a metric, method for implementation, and timeline.
Since there is already an established common agenda, the first critical behavior is to revisit the
agenda regularly to ensure that boundaries are set and adhered to. The second critical behavior is
to engage in a participatory process involving community members to identify a common set of
indicators and data collection methods to determine progress toward upstream prevention. The
third critical behavior is that SC members establish an action plan that details how agencies will
work together toward the common agenda. The fourth critical behavior is that the SC members
foster continuous communication through regular meetings with opportunities to provide updates
from the working groups, the community, as well as to learn more about best practices that could
be applied to this work. Table 15 includes specific metrics, methods, and timing for their critical
behaviors.

146

Table 15
Critical Behaviors, Metrics, Methods, and Timing for Evaluation
Critical Behavior Metric(s) Method(s) Timing
SC members will revisit
the established
common agenda
Number of
community
members
engaged

SC will utilize the
community
engagement working
group to gather input
from the community
on the common
agenda

SC members will
review the IMTSS to
ensure alignment to
the common agenda
Year 1, since the
common agenda
is already
established, and
again annually
SC members will engage
in a participatory
process to identify a
common set of
indicators and data
collection methods
that determine
progress toward
upstream prevention
Number of SC
members and
staff who can
identify the key
performance
indicators

Developed plan
for how to
collect data

Number of
community
members
engaged

SC will utilize the data
and evaluation
working group and
the prevention
working group to
determine the key
performance
indicators

The data and evaluation
working group in
conjunction with the
community
engagement working
group will determine
the data collection
methods

The community
engagement working
group will gather
input from the
community on how
progress should be
measured and what
data should be
collected
Year 1

Year 1 (after the
key
performance
indicators have
been identified)

Year 1 and
ongoing

147

Critical Behavior Metric(s) Method(s) Timing
SC members will create a
plan detailing which
agencies will work
together to address the
identified common
indicators
Number of agency
partners identified
in the plan

Number of planning
meetings

Number of signed
commitment
forms from agency
partners
SC members will identify
if/how their agency can
contribute to each of the
common indicators

SC members will commit
resources to address the
common indicators

Year 1

Year 2
SC members foster
continuous communication
throughout the county
Communication plan

SC members will meet to
share information and
provide updates

SC members will meet with
leaders within their
individual agencies to
share information related
to the IMTSS

SC members will
communicate with other
stakeholders (i.e., board
of supervisors, backbone
organization,
community)
Monthly

Monthly

Monthly

Required Drivers
Level 3 seeks to evaluate what participants are doing differently as a result of the
training. The findings showed that there were few KMO gaps. However, the assets can be
leveraged to support the SC’s readiness for collective impact. As such, the critical behaviors
identified above align with the four assessed collective impact conditions. These behaviors can
be supported by drivers, including processes and systems where performance is reinforced,
monitored, encouraged, and rewarded (Kirkpatrick & Kirkpatrick, 2016). A job aid will reinforce
148

the structure of the SACT and how members will work together and communicate with each
other. Restructured meetings to include updates and data talks will foster continuous
communication and reliance on data to make informed decisions. Coaching support and
opportunities for members to reflect on how their commitment to the IMTSS collective impact
benefits and reinforces the work in their agencies will continue to foster motivation. Public
recognition by the board of supervisors and the county executive’s office will incentivize them to
keep moving forward. Finally, annual empathy interviews will allow them to safely share their
perspectives on overall progress. Table 16 shows the recommended drivers to support the critical
behaviors.

149

Table 16
Required Drivers to Support Critical Behaviors
Method(s) Timing
Critical
behaviors
supported
1, 2, 3 Etc.
Reinforcing

Use a job aid that shows the structure of SACT, including
the roles of the SC, larger SACT, and workgroups and
the communication/ feedback loop among these groups
Year 1 and
ongoing
1, 3, 4
Restructure monthly SC meetings to include updates
from each SC member as well as the working group
leads.
Ongoing 1, 2, 3, 4
Engage the SC members in regular data talks at their
monthly meetings
Ongoing 1, 2, 3, 4
Encouraging

Opportunities during SC meetings for members to reflect
on past collaboration efforts to identify how they are
working differently under the IMTSS
Year 1 and
quarterly
1, 2, 3, 4
Opportunities during SC meetings for members to reflect
and discuss how their work is connected to the IMTSS
Quarterly 1, 2, 3
Opportunities for SC members to receive individual
coaching to support their leadership in the
implementation of the IMTSS
Monthly during
Year 1 and then
as needed
1, 2, 3, 4
Rewarding

Public acknowledgment from the Board of Supervisors
and the County Executive’s Office after progress is
reported
Quarterly 1, 2, 3, 4
Monitoring

Empathy interviews with SC members to determine
progress in addressing the KMO gaps and progress on
each of the four collective impact conditions
Annually 1, 2, 3, 4

150

Organizational Support
Organizational support, in the form of processes and resources, is necessary for the
critical behaviors and required drivers to take hold. The organization will need to ensure a
culture of continuous improvement where learning and adaptability are encouraged. In their
ongoing research about making collective impact work, Hanleybrown et al. (2012) found that the
common agenda needs to be revisited often to ensure alignment. While the SACT includes
agencies that directly serve children and families, there are other agencies whose work impacts
the same families served through the collective impact model. Disadvantaged families may
encounter difficulties with transportation to work, school, or other important services if the roads
are damaged or if there is not enough public transportation available. Children in communities
with concentrated disadvantage may not have access to parks or walking trails for outdoor
exercise or access to grocery stores with healthy food options. Therefore, it is important that the
organization communicates that the larger organizational mission and goal are relevant to the
work of all agencies throughout the county. Agencies may have direct impact or indirect impact,
but all agencies play a role in eliminating the intergenerational cycle of disadvantage.
Funding was identified as a barrier. Although there were no recommendations presented
to address this barrier given the limitations of this study, there are some ways the organization
can support the SC. The county CEO’s office can leverage funding so that strict funding
guidelines are followed while still providing the freedom to continue work across agencies.
Additionally, the CEO’s office can work to secure grant funding so that individual agencies are
not left to apply for grants separately.

151

Level 2: Learning
Continuing to work backward, after the required drivers to support the critical behaviors
are decided, the next step is to determine what learning must happen to lead to those desired
behaviors. Level 2 of the New World Kirkpatrick Model (Kirkpatrick & Kirkpatrick, 2016)
measures learning that is defined as the extent to which participants acquire the intended
knowledge, skills, and attitudes as a result of the training.
Learning Goals
For the SC members to perform the critical behaviors identified in Level 3, they must
first be able to
• Explain the role of their specific agency in the IMTSS (conceptual knowledge),
• Communicate effectively across agencies (procedural knowledge),
• Utilize data as a tool for reflection for how they are working together more effectively
and efficiently (metacognitive knowledge), and
• Identify how their agency contributes to the IMTSS collective impact initiative (utility
value).
Program
To achieve the learning goals, participants will engage in three types of training
opportunities. The first type will be an annual retreat. The second will be a multi-day training on
the IMTSS to build capacity on the framework and develop an implementation plan. The third
type will be ongoing coaching support.
The training program will begin with an annual retreat for SC members to focus more on
building community. For the inaugural retreat, they will revisit the common agenda and reflect
on how their agency contributes to the IMTSS and how the IMTSS integrates with the work of
152

the individual agencies. They will revisit their team norms for how they will work together. They
will also revisit their team charter that details their mission and purpose and establishes a
recommitment from each member to this work. They will also review how trauma and racial
inequities impact the community they are working to serve. They will learn more about the
behaviors needed to achieve the goal. Due to the amount of content, this can be spread over three
to four days at 3 hours each if led in a virtual setting. If held in person, then 2 days at 6 hours
may be sufficient. At the annual retreats, SC members will reflect on the work accomplished over
the past year and will establish the goals and objectives for the upcoming year.
The second training opportunity will be a multi-day training on the IMTSS intended to
focus on the technical part of the implementation plan. This training will take place after the
inaugural retreat, and the content will be developed in part from the feedback from that retreat.
Specific content will include a review of the collective impact conditions, reviewing data to
determine 4 to 5 common data points/indicators to measure the efficiency and effectiveness of
the IMTSS, developing a communication and feedback loop through a formalized
communication plan, and how members can begin to prepare their agencies to engage in this
work.
To prepare their agencies to engage in this work, SC members will learn about
assembling the right team, the elements of effective meetings (e.g., meeting agendas, taking
notes, assigning roles, adhering to time), reviewing the IMTSS and how their agency contributes
to the common agenda, and communicating this work throughout the agency. Finally, the third
training opportunity will be focused on coaching support. Members will be invited to participate
in one-to-one coaching to provide a confidential space for discussing challenges they may face as
they work on building out the IMTSS while still leading the work of their individual agencies.
153

Coaching support will also be provided during the SC meetings. A coach will help to develop the
agenda with the SACT manager ahead of the meeting. The coach will support the SACT
manager while the manager facilitates the meeting and provide feedback after the meeting.
The comprehensive training program is intended to address gaps and leverage assets to
increase readiness to engage in the collective impact work.
Evaluation of the Components of Learning
With the learning goals and learning plan established, Kirkpatrick and Kirkpatrick (2016)
suggest determining which tools will be used to evaluate learning components (i.e., knowledge,
skills, attitudes, confidence, and commitment). Table 17 presents the evaluation tools for the
learning program described earlier. Some evaluation tools include assessments, group
discussions, individual reflection, group activities, and simulations.

154

Table 17
Evaluation of the Components of Learning for the Program.
Methods or activities Timing
Declarative Knowledge “I know it.”

Knowledge checks through breakout room/ small-group
discussions and individually sharing comments using the
chat feature
During each session
Discussion about how trauma and racial inequities have
created a need for this work and how the IMTSS will
address these
During the inaugural annual
SC retreat
Procedural Skills “I can do it right now.”

Simulation activity to see how SC members will respond to
community need
At the beginning of and
revisited throughout the
multi-day training
Demonstration of effective meetings During each monthly SC
meeting
Attitude “I believe this is worthwhile.”

Facilitator’s observations of SC members in the sessions (do
they stay for the whole meeting? so they keep their
webcam on? do they contribute to the conversation?)
During the session
Feedback at the end of the session based on the question:
“What about today’s session was valuable?”
End of each session
Reflection on what worked and what could be improved
about the monthly meeting structure.
End of each session
Confidence “I think I can do it on the job.”

Feedback via a survey asking SC members to rate their
confidence level that they can apply their new learning.
End of the multi-day training
event
Group discussions about how SC members are integrating
the work of their agencies with the IMTSS.
During the monthly meetings
Commitment “I will do it on the job.”

Goal setting based on a progress evaluation At the end of each annual
retreat

155

Level 1: Reaction
Level 1 of the New World Kirkpatrick Model (Kirkpatrick & Kirkpatrick, 2016)
evaluates how the participants react to the recommended training program. There are dimensions
of reaction: satisfaction; engagement, or the degree to which participants actively participate;
and, relevance which is the degree to which participants will be able to apply what they have
learned to their jobs. Data will be collected through formal surveys and check-ins but also
through informal observations during the training events. Table 18 presents the components to
measure Level 1 reaction to the learning program.
Table 18
Components to Measure Reactions to the Program.
Method(s) or tool(s) Timing
Engagement

Facilitator observations During each session
Attendance at each training session

During each session
Attendance for the duration of the
training
During each session
Workshop evaluation End of each session
Relevance

Check-ins with SC members Quarterly
Check-ins with SACT manager Bi-weekly
Customer Satisfaction

Program evaluation Annually (at the end of each fiscal year)
Check-ins with SC members Quarterly
Check-ins with SACT manager Bi-weekly

156

Evaluation Tools
Under the New World Kirkpatrick Model (Kirkpatrick & Kirkpatrick, 2016), Level 1
measures reaction to the training event, and Level 2 evaluates the amount of learning resulting
from the training event. Given the proximity of these two levels to the training event, they are
assessed simultaneously and immediately after the training program, using both formative and
summative evaluation tools. Level 3 and Level 4 evaluate the effectiveness of the training by
looking at the behavior and ultimate progress toward the goal. These are both evaluated after the
training and then again at scheduled intervals to measure long-term impact. This section presents
tools to be used during these two evaluation periods: immediately after the training and delayed
for a period after the training.
Immediately Following the Program Implementation
The training program is designed to be interactive. As one way to measure reaction) the
facilitator will observe the members in real time on a single screen. Additionally, at the end of
each session, members will share feedback about the value of the training event through an
evaluation form that blends Level 1 and Level 2. They will be asked to share their feedback on
what they learned, their engagement during the session, the relevance of the content presented to
their work, their confidence in engaging in the work, and their willingness to continue to engage
in the work
Delayed for a Period After the Program Implementation
After the annual retreats, members will be asked to complete self-evaluations in 30-day,
60-day, and 90-day intervals. The self-evaluation will focus on assessing progress and rate how
they are working together to accomplish the goal. About 3 months after the multi-day training,
they will be asked to complete a survey that measures all four levels. Level 1 will be measured
157

by asking about their satisfaction with and the relevance of the content shared during the multi-
day training event. For Level 2, they will be asked about their confidence level and their
commitment to applying what they learned. They will be asked to reflect on how their behavior
is changing based on what they are learning. Finally, for Level 4, they will be asked about how
their training has prepared them to implement a collective impact initiative. They will also
participate in one-on-one coaching to increase their leadership skills.
Data Analysis and Reporting
Preskill and Splansky (n.d.) suggested that each of the three developmental stages in the
collective impact change process needs a different evaluation approach. The three stages are
early years (years 1–3), middle years (years 3–7), and later years (years 7–10). The IMTSS is in
the early years stage. During this stage, the focus is on increasing understanding of the design
and implementation of the initiative, including the five conditions of collective impact, and a
coordinated theory of change (Preskill & Splansky, n.d.). Results related to dismantling the
school-to-prison pipeline by eliminating the intergenerational cycle of disadvantage will not be
measured until the collective impact effort advances to the middle and later years. The data
collected from the evaluation tools for all four levels will be reported to the SACT SC and the
county executive’s office. Data will also be shared with the larger SACT group to show the
progress of the collective impact effort.
Summary of the Implementation and Evaluation
The New World Kirkpatrick Model (Kirkpatrick & Kirkpatrick, 2016) was used as a
framework to develop, implement, and evaluate the training activities based on the
aforementioned recommendations to measure the SC’s readiness to implement a collective
impact effort. As this collective impact initiative was in the early years of development, the
158

outcomes identified at Level 4 are focused on the SC members’ progress toward the four
assessed collective impact conditions. In order to achieve the Level 4 outcomes, there are critical
behaviors required of the SC members (Level 3). The recommendations at the beginning of this
chapter are based on the findings reported in Chapter Four. Those recommendations were used to
develop the required drivers to support the identified Level 3 critical behaviors. In addition to
measuring knowledge at Level 2, there are also evaluation components to measure the SC
members’ attitude, confidence, and commitment. As working towards a collective impact is
iterative, continuous learning is critical to its success and sustainability (Preskill & Splansky,
n.d.).
Limitations and Delimitations
Limitations refer to influences beyond the control of the researcher. In this study, there
were three limitations. The first was the relatively small sample. The SACT SC was the
stakeholder group chosen for this study. There were 13 members of this group, and 12 agreed to
participate. The next limitation was the amount of empirical research on collective impact. While
there was some literature about collective impact, empirical research on the use of collective
impact in a similar setting to this study was very limited. The third limitation was the prevalence
of the COVID-19 global pandemic during the data gathering phase of this project. As the
members are all executive-level leaders within their organizations, they were very focused on
addressing COVID-related organizational needs, which limited their availability and led to
several rescheduled meetings.
Delimitations refer to choices made by the researcher. In this study, there are two
delimitations worth mentioning. First, the study focused on a single stakeholder group to gain a
better understanding of the collective impact initiative and how the partners worked together.
159

Focusing on a single stakeholder group was necessary for the study, given this group’s
importance in a collective impact initiative. However, hearing from middle-managers, line staff,
and even community members would have provided a more comprehensive evaluation of the
overall collective impact effort. Second, given the specificity of this study, there are limits to the
use of the recommendations in other organizations without consideration of organizational
context.
Recommendations for Future Research
The SACT was created to address racial disparities among children and families in SSC.
In the 12 years since its creation, SACT has increased opportunities for coordinated services.
However, there are still silos across organizations that force agencies into isolation, lead to
duplicated services, and leave many families underserved. The SC engaged in collective impact
to organize around a common agenda to create systemic change. Chapter Two presented research
to support an upstream prevention approach to address this problem. The SACT SC used the
collective impact framework with a common agenda focused on dismantling the school-to-prison
pipeline by eliminating the cycle of disadvantage. The study focused on the readiness to engage
in collective impact by identifying the KMO influences. Collective impact is still fairly new, and
more empirical research is needed to measure its efficacy. Future studies can include a broader
stakeholder group; evaluating the role of the backbone organization in the early years of
development; evaluating how community voice is included in the early years of development;
evaluating the role of working groups in executing the collective impact conditions (common
agenda, mutually reinforcing activities, shared measurement system, and continuous
communication); and following the SC members as they take the work back to their agencies.
160

Expanding the study to include a broader stakeholder group would increase the
population and would provide a broader perspective of those involved in the collective impact
effort. This would provide deeper insight into gaps beyond the SC. The backbone organization
was not evaluated in this study. However, given its importance in supporting the collective
impact effort, it would be useful to study how the chief executive's office navigates the multiple
agency partners and the political sphere to support this effort. In the interviews, participants
spoke about the importance of engaging in this work in partnership with the community. Despite
literature recommendations to include the community in all stages of developing a collective
impact effort (McAfee et al., 2015; Williams & Marxer, 2014), there is still limited research to
show how this is initiated and sustained. Therefore, another recommendation for future research
is to focus on recruiting and engaging the community in effort from inception to implementation
and beyond.
Working groups are another critical component of a collective impact effort. Future
research could focus on how working groups progress the collective impact effort. Several
studies may emerge to focus on working groups’ progress with each of the collective impact
conditions. Finally, a study on how the SACT SC members transfer their learning about the
collective impact efforts into their organizations is worth exploring. The actual collective impact
effort does not live with the executive leadership but depends on middle managers and line staff
to commit to the mutually reinforcing activities, gather and report the agreed-upon data, and
engage in continuous communication. To sustain the effort, it will be important to examine how
executive leaders bring their organizations along.
161

Conclusion
The purpose of this evaluation study was to identify the KMO influences that contribute
to the readiness of the SACT SC to engage in collective impact. The ultimate goal of the
collective effort is to dismantle the school-to-prison pipeline by eliminating the cycle of
disadvantage. Given the complexity of this problem, the solution lies with multiple stakeholders
working together in trusting relationships towards the common goal. The SC had utilized the
collective impact framework to address this complex problem. Collective impact is a
commitment of a group of people from different sectors to address a complex social issue in a
structured cross-sector collaboration (Kania & Kraamer, 2011). There are five conditions of
collective impact, and this study focused on four of them.
The SACT SC was the stakeholder of focus because of its important role in leading t the
collective impact effort. As executive-level leaders at their organizations, the members lead the
effort and operationalize the mutually reinforcing activities in their organizations. Clark and
Estes’s (2008) gap analysis framework was used to identify their KMO influences. Data were
gathered from interviews, documents, and meeting observations. Based on the findings,
recommendations were presented to address gaps and leverage assets to increase readiness to
engage in collective impact.

162

References
Aber, J. L., Bennett, N. G., Conley, D. C., & Li, J. (1997). The effects of poverty on child health
and development. Annual Review of Public Health, 18(1), 463–483.
https://doi.org/10.1146/annurev.publhealth.18.1.463
Aber, L., Morris, P., & Raver, C. (2012). Children, families and poverty: Definitions, trends,
emerging science and implications for policy and commentaries. Social Policy Report,
26(3), 1–29. https://doi.org/10.1002/j.2379-3988.2012.tb00072.x
Aizer, A., & Doyle, J. J., Jr. (2015). Juvenile incarceration, human capital, and future crime:
Evidence from randomly assigned judges. The Quarterly Journal of Economics, 130(2),
759–803. https://doi.org/10.1093/qje/qjv003
Ambrose, S. A., Bridges, M. W., DiPietro, M., Lovett, M. C., & Norman, M. K. (2010). How
learning works: Seven research-based principles for smart teaching. John Wiley & Sons.
Anderson, L., & Krathwohl, D. (2001). A taxonomy for learning, teaching, and assessing: A
revision of bloom’s taxonomy of educational objectives. Longman.
Andresen, M. (2012). Unemployment and crime: A neighborhood level panel data approach.
Social Science Research, 41(6), 1615–1628.
https://doi.org/10.1016/j.ssresearch.2012.07.003
The Annie E. Casey Foundation. (2020). Children in single-parent families by race in the United
States. https://datacenter.kidscount.org/data/tables/107-children-in-single-parent-families-
by-race#detailed/1/any/false/1729,37,871,870,573,869,36,868,867,133/10,11,9,12,1,185,
13/432,431
The Annie E. Casey Foundation. (2021b). Fourth graders who scored below proficient reading
level by school income in the United States. https://datacenter.kidscount.org/data/tables/
163

5112-fourth-graders-who-scored-below-proficient-reading-level-by-school-
income#detailed/1/any/false/1729,871,573,36,867,38/1165,1166/11552
Arditti, J. (2005). Families and incarceration: An ecological approach. Families in Society, 86(2),
251–260. https://doi.org/10.1606/1044-3894.2460
Arditti, J. A., Lambert-Shute, J., & Joest, K. (2003). Saturday morning at the jail: Implications of
incarceration for families and children. Family Relations, 52(3), 195–204.
https://doi.org/10.1111/j.1741-3729.2003.00195.x
Autor, D., Figlio, D., Karbownik, K., Roth, J., & Wasserman, M. (2019). Family disadvantage
and the gender gap in behavioral and educational outcomes. American Economic Journal.
Applied Economics, 11(3), 338–381. https://doi.org/10.1257/app.20170571
Baker, L. (2005). Developmental differences in metacognition: Implications for metacognitively
oriented reading instruction. In S. E. Israel, C. C. Block, K. L. Bauserman, & K.
Kinnucan-Welsch (Eds.), Metacognition in literacy learning: Theory, assessment,
instruction, and professional development (pp. 61–79). Lawrence Erlbaum Associates.
Bandura, A. (1977). Self-efficacy: Toward a unifying theory of behavioral change.
Psychological Review, 84(2), 191–215. https://doi.org/10.1037/0033-295X.84.2.191
Bandura, A. (1997). Self-efficacy: The exercise of control. W. H. Freeman.
Bandura, A. (2012). On the functional properties of perceived self-efficacy revisited. Journal of
Management, 38(1), 9–44. https://doi.org/10.1177/0149206311410606
Barbarin, O. A. (2010). Halting African American boys’ progression from pre-K to prison: What
families, schools, and communities can do! The American Journal of Orthopsychiatry,
80(1), 81–88. https://doi.org/10.1111/j.1939-0025.2010.01009.x
164

Barbarin, O. A., & Aikens, N. (2015). Overcoming the educational disadvantages of poor
children: How much do teacher preparation, workload, and expectations matter. The
American Journal of Orthopsychiatry, 85(2), 101–105.
https://doi.org/10.1037/ort0000060
Barberg, B. (2015). Harnessing the power source for collective impact: Mutually reinforcing
activities. https://www.collectiveimpactforum.org/sites/default/files/Harnessing%20
the%20Power%20Source%20for%20Collective%20Impact%20v4.pdf
Battle, J., & Scott, B. M. (2000). Mother-only versus father-only households: Educational
outcomes for African American males. Journal of African American Men, 5(2), 93–116.
https://doi.org/10.1007/s12111-000-1009-4
Bertrand, M., & Pan, J. (2013). The trouble with boys: Social influences and the gender gap in
disruptive behavior. American Economic Journal. Applied Economics, 5(1), 32–64.
https://doi.org/10.1257/app.5.1.32
Bloome, D. (2014). Racial inequality trends and the intergenerational persistence of income and
family structure. American Sociological Review, 79(6), 1196–1225.
https://doi.org/10.1177/0003122414554947
Bolman, L. G., & Deal, T. E. (2017). Reframing organizations. Jossey-Bass.
https://doi.org/10.1002/9781119281856
Born, P. (2017). How to develop a common agenda for a collective impact. Tamarack Institute.
https://www.tamarackcommunity.ca/hubfs/Resources/Publications/CommonAgendaforC
ollectiveImpact.pdf
Bowen, G. A. (2009). Document analysis as a qualitative research method. Qualitative Research
Journal, 9(2), 27–40. https://doi.org/10.3316/QRJ0902027
165

Bradshaw, C. P., Mitchell, M. M., O’Brennan, L. M., & Leaf, P. J. (2010). Multilevel
exploration of factors contributing to the overrepresentation of black students in office
disciplinary referrals. Journal of Educational Psychology, 102(2), 508–520.
libproxy2.usc.edu/. https://doi.org/10.1037/a0018450
Bronfenbrenner, U. (1977). Toward an experimental ecology of human development. The
American Psychologist, 32(7), 513–531. https://doi.org/10.1037/0003-066X.32.7.513
Bronfenbrenner, U. (1979). The ecology of human development: Experiments by nature and
design. Harvard University Press.
Brooks-Gunn, J., & Duncan, G. J. (1997). The effects of poverty on children. The Future of
Children, 7(2), 55–71. https://doi.org/10.2307/1602387
Buchmann, C., & DiPrete, T. A. (2006). The growing female advantage in college completion:
The role of family background and academic achievement. American Sociological
Review, 71(4), 515–541. https://doi.org/10.1177/000312240607100401
Burke, N. J., Hellman, J. L., Scott, B. G., Weems, C. F., & Carrion, V. G. (2011). The impact of
adverse childhood experiences on an urban pediatric population. Child Abuse & Neglect,
35(6), 408–413. https://doi.org/10.1016/j.chiabu.2011.02.006
Bush, V. L., & Bush, E. C. (2013). Introducing African American male theory (AAMT). Journal
of African American Males in Education, 4(1), 18434.
Butterfield, P. G. (2017). Thinking upstream: A 25-year retrospective and conceptual model
aimed at reducing health inequities. Advances in Nursing Science, 40(1), 2–11.
https://doi.org/10.1097/ANS.0000000000000161
166

Byrnes, H., & Miller, B. (2012). The relationship between neighborhood characteristics and
effective parenting behaviors: The role of social support. Journal of Family Issues,
33(12), 1658–1687. https://doi.org/10.1177/0192513X12437693
Cabaj, M., & Weaver, L. (2016). Collective impact 3.0: An evolving framework for community
change. Tamarack Institute. http://www.unitedwayglv.org/UnitedWay/media/PDFs/
Collective_Impact_3-0_FINAL_PDF.pdf
Centers for Disease Control and Prevention. (2010). The adverse childhood experiences study.
http://www.cdc.gov/ace/about.htm
Centers for Disease Control and Prevention. (2020). Preventing adverse childhood experiences.
https://www.cdc.gov/violenceprevention/childabuseandneglect/aces/fastfact.html
Chandler, A. (2020, August 5). A Lifecourse framework for boys and men of color [webinar
presentation].
Chaudry, A., & Wilmer, C. (2016). Poverty is not just an indicator: The relationship between
income, poverty, and child well-being. Ambulatory Pediatrics: The Official Journal of
the Ambulatory Pediatric Association, 16(3S), S23.
Cheng, S., Tamborini, C., Kim, C., & Sakamoto, A. (2019). Educational variations in cohort
trends in the Black-White earnings gap among men: Evidence from administrative
earnings data. Demography, 56(6), 2253–2277. https://doi.org/10.1007/s13524-019-
00827-w
Chetty, R., Hendren, N., Jones, M., & Porter, S. (2018). Race and economic opportunity in the
United States: An intergenerational perspective. National Bureau of Economic Research.
https://doi.org/10.3386/w24441
167

Chetty, R., Hendren, N., & Katz, L. (2016). The effects of exposure to better neighborhoods on
children: New evidence from the moving to opportunity experiment. The American
Economic Review, 106(4), 855–902. https://doi.org/10.1257/aer.20150572
Children’s Defense Fund. (2021). The state of America’s children 2021.
https://www.childrensdefense.org/state-of-americas-children/
Choi, J.-K., Wang, D., & Jackson, A. P. (2019). Adverse experiences in early childhood and their
longitudinal impact on later behavioral problems of children living in poverty. Child
Abuse & Neglect, 98, Article 104181 https://doi.org/10.1016/j.chiabu.2019.104181
Chung, E. K., Siegel, B. S., Garg, A., Conroy, K., Gross, R. S., Long, D. A., Lewis, G., Osman,
C. J., Jo Messito, M., Wade, R., Jr., Shonna Yin, H., Cox, J., & Fierman, A. H. (2016).
Screening for social determinants of health among children and families living in
poverty: A guide for clinicians. Current Problems in Pediatric and Adolescent Health
Care, 46(5), 135–153. https://doi.org/10.1016/j.cppeds.2016.02.004
Chung, H. L., & McFadden, D. (2010). The effects of incarceration on neighborhoods and
communities. In Y. R. Harris, J. A. Graham, & G. J. Oliver Carpenter (Eds.), Children of
incarcerated parents: Theoretical, developmental and clinical issues (pp. 105–126).
Springer.
Clark, R. E., & Estes, F. (2008). Turning research into results: A guide to selecting the right
performance solutions. CEP Press.
Cobb-Clark, D., & Tekin, E. (2013). Fathers and youths’ delinquent behavior. Review of
Economics of the Household, 12(2), 327–358. https://doi.org/10.1007/s11150-013-9194-9
Coleman, J. S. (1966). Equality of educational opportunity. U.S. Dept. of Health, Education, and
Welfare, Office of Education.
168

Coleman, J. S. (1987). Families and schools. Educational Researcher, 16(6), 32–38.
https://doi.org/10.3102/0013189X016006032
Creamer, J. (2020). Inequalities persist despite decline in poverty for all major race and
Hispanic origin groups. US Census Bureau. https://www.census.gov/library/stories/2020/
09/poverty-rates-for-blacks-and-hispanics-reached-historic-lows-in-2019.html
Creswell, J. W. (2014). Research design: Qualitative, quantitative, and mixed methods
approaches (4th ed.). SAGE Publications.
Crowson, R., & Boyd, W. (1993). Coordinated services for children: Designing arks for storms
and seas unknown. American Journal of Education, 101(2), 140–179.
https://doi.org/10.1086/444037
Crowson, R. L., & Boyd, W. L. (1996). Achieving coordinated school-linked services:
Facilitating utilization of the emerging knowledge base. Educational Policy, 10(2), 253–
272. https://doi.org/10.1177/0895904896010002007
Deal, T. E., & Kennedy, A. A. (1983). Culture: A new look through old lenses. The Journal of
Applied Behavioral Science, 19(4), 498–505.
https://doi.org/10.1177/002188638301900411
de Brey, C., Musu, L., McFarland, J., Wilkinson-Flicker, S., Diliberti, M., Zhang, A.,
Branstetter, C., & Wang, X. (2019). Status and trends in the education of racial and
ethnic groups 2018 (NCES 2019-038). U.S. Department of Education.
https://nces.ed.gov/pubs2019/2019038.pdf
DiPrete, T., & Buchmann, C. (2006). Gender-specific trends in the value of education and the
emerging gender GAP in college completion. Demography, 43(1), 1–24.
https://doi.org/10.1353/dem.2006.0003
169

Donovan, S. A., & Bradley, D. H. (2019). Real wage trends, 1979–2018. Congressional
Research Service.
Dorsey, S., Forehand, R., & Brody, G. (2007). Coparenting conflict and parenting behavior in
economically disadvantaged single parent African American families: The role of
maternal psychological distress. Journal of Family Violence, 22(7), 621–630.
https://doi.org/10.1007/s10896-007-9114-y
Driscoll, M. E., Boyd, W., & Crowson, R. (1998). Collaborative services initiatives: A report of
a national survey of programs. Laboratory for Student Success, Mid-Atlantic Regional
Educational Laboratory at Temple University.
Dumas, M. J., & Nelson, J. D. (2016). (Re)imagining Black boyhood: Toward a critical
framework for educational research. Harvard Educational Review, 86(1), 27–47.
https://doi.org/10.17763/0017-8055.86.1.27
Eamon, M. K. (2001). The effects of poverty on children’s socioemotional development: An
ecological systems analysis. Social Work, 46(3), 256–266.
https://doi.org/10.1093/sw/46.3.256
Eccles, J. (2006). Expectancy value motivational theory. http://www.education.com/reference/
article/expectancy-value-motivational-theory/
Elder, G., Jr. (1998). The life course as developmental theory. Child Development, 69(1), 1–12.
https://doi.org/10.1111/j.1467-8624.1998.tb06128.x
Engle, P. L., & Black, M. M. (2008). The effect of poverty on child development and educational
outcomes. Annals of the New York Academy of Sciences, 1136(1), 243–256.
https://doi.org/10.1196/annals.1425.023
170

Ewert, S., Sykes, B. L., & Pettit, B. (2014). The degree of disadvantage: Incarceration and
inequality in education. The Annals of the American Academy of Political and Social
Science, 651(1), 24–43. https://doi.org/10.1177/0002716213503100
Fabelo, T., Thompson, M. D., Plotkin, M., Carmichael, D., Marchbanks, M. P., & Booth, E. A.
(2011). Breaking schools’ rules: A statewide study of how school discipline relates to
students’ success and juvenile justice involvement. Council of State Governments Justice
Center.
Fernald, A., Marchman, V. A., & Weisleder, A. (2013). SES differences in language processing
skill and vocabulary are evident at 18 Months. Developmental Science, 16(2), 234–248.
libproxy2.usc.edu/. https://doi.org/10.1111/desc.12019
Flavell, J. H. (1979). Metacognition and cognitive monitoring: A new area of cognitive–
developmental inquiry. The American Psychologist, 34(10), 906–911.
https://doi.org/10.1037/0003-066X.34.10.906
Foster, H., & Hagan, J. (2015). Punishment regimes and the multilevel effects of parental
incarceration: Intergenerational, intersectional, and interinstitutional models of social
inequality and systemic exclusion. Annual Review of Sociology, 41(1), 135–158.
https://doi.org/10.1146/annurev-soc-073014-112437
Friedman-Krauss, A., Barnett, W. S., & Nores, M. (2016). How much can high-quality universal
pre-K reduce achievement gaps? Center for American Progress.
Gallimore, R., & Goldenberg, C. (2001). Analyzing cultural models and settings to connect
minority achievement and school improvement research. Educational Psychologist,
36(1), 45–56. https://doi.org/10.1207/S15326985EP3601_5
171

Geller, A., Cooper, C., Garfinkel, I., Schwartz-Soicher, O., & Mincy, R. (2011). Beyond
absenteeism: Father incarceration and child development. Demography, 49(1), 49–76.
https://doi.org/10.1007/s13524-011-0081-9
Geller, A., Garfinkel, I., Cooper, C., & Mincy, R. (2009). Parental incarceration and child well-
being: Implications for urban families. Social Science Quarterly, 90(5), 1186–1202.
https://doi.org/10.1111/j.1540-6237.2009.00653.x
Georgia Center for Opportunity. (n.d.). Understanding intergenerational poverty.
https://georgiaopportunity.org/intergenerational-poverty-study/
Gilliam, W. S., Maupin, A. N., Reyes, C. R., Accavitti, M., & Shic, F. (2016). Do early
educators’ implicit biases regarding sex and race relate to behavior expectations and
recommendations of preschool expulsions and suspensions. Yale Child Study Center.
Glaze, L. E., & Maruschak, L. M. (2016). Parents in prison and their minor children.
https://biblioteca.cejamericas.org/bitstream/handle/2015/2976/Parents_Prison_Minor_Chi
ldren.pdf?sequence=1&isAllowed=y
Grodsky, E., & Pager, D. (2001). The structure of disadvantage: Individual and occupational
determinants of the Black-White wage gap. American Sociological Review, 66(4), 542–
567. https://doi.org/10.2307/3088922
Gutman, L. M., McLoyd, V. C., & Tokoyawa, T. (2005). Financial strain, neighborhood stress,
parenting behaviors, and adolescent adjustment in urban African American families.
Journal of Research on Adolescence, 15(4), 425–449. https://doi.org/10.1111/j.1532-
7795.2005.00106.x
Hamilton, L. G., Hayiou-Thomas, M., Hulme, C., & Snowling, M. J. (2016). The home literacy
environment as a predictor of the early literacy development of children at family-risk of
172

dyslexia. Scientific Studies of Reading, 20(5), 401–419.
https://doi.org/10.1080/10888438.2016.1213266
Hanleybrown, F., Kania, J., & Kramer, M. (2012). Channeling change: Making collective impact
work. FSG. https://mappofskp.net/wp-content/uploads/2015/05/SSIR-Collective-Impact-
2.pdf
Hart, B., & Risley, T. R. (1995). Meaningful differences in the everyday experience of young
American children. Brookes Publishing.
Haskins, A. (2016). Beyond boys’ bad behavior: Paternal incarceration and cognitive
development in middle childhood. Social Forces, 95(2), 861–892.
https://doi.org/10.1093/sf/sow066
Haskins, A., Amorim, M., & Mingo, M. (2018). Parental incarceration and child outcomes:
Those at risk, evidence of impacts, methodological insights, and areas of future work.
Sociology Compass, 12(3), e12562–n/a. https://doi.org/10.1111/soc4.12562
Henig, J. R., Riehl, C. J., Rebell, M. A., & Wolff, J. R. (2015). Putting collective impact in
context: A review of the literature on local cross-sector collaboration to improve
education. Teachers College.
Hicks, D., Larson, C., Nelson, C., Olds, D. L., & Johnston, E. (2008). The influence of
collaboration on program outcomes: The Colorado nurse-family partnership. Evaluation
Review, 32(5), 453–477. https://doi.org/10.1177/0193841X08315131
Hirsch, B., & Winters, J. (2014). An anatomy of racial and ethnic trends in male earnings in the
U.S. Review of Income and Wealth, 60(4), 930–947. https://doi.org/10.1111/roiw.12064
173

Howard, T. (2013). How does it feel to be a problem? Black Male students, schools, and learning
in enhancing the knowledge base to disrupt deficit frameworks. Review of Research in
Education, 37(1), 54–86. https://doi.org/10.3102/0091732X12462985
Hunt, T. K. A., Slack, K. S., & Berger, L. M. (2017). Adverse childhood experiences and
behavioral problems in middle childhood. Child Abuse & Neglect: The International
Journal., 67, 391–402.
Jimenez, M. E., Wade, R., Jr., Lin, Y., Morrow, L. M., & Reichman, N. E. (2016). Adverse
experiences in early childhood and kindergarten outcomes. Pediatrics, 137(2),
e20151839–e20151839. https://doi.org/10.1542/peds.2015-1839
Johnson, E. I., & Easterling, B. (2012). Understanding unique effects of parental incarceration on
children: Challenges, progress, and recommendations. Journal of Marriage and Family,
74(2), 342–356. https://doi.org/10.1111/j.1741-3737.2012.00957.x
Jolin, M., Schmitz, P., & Sheldon, W. (2012). Needle-moving community collaboratives: A
promising approach to addressing America's biggest challenges. Bridgespan Group.
Jones, T. M., Nurius, P., Song, C., & Fleming, C. M. (2018). Modeling life course pathways
from adverse childhood experiences to adult mental health. Child Abuse & Neglect, 80,
32–40. https://doi.org/10.1016/j.chiabu.2018.03.005
Kagan, S. L., & Neville, P. (1993). Integrating services for children and families: Understanding
the past to shape the future. Yale University Press.
Kania, J., Hanleybrown, F., & Splansky Juster, J. (2014). Essential mindset shifts for collective
impact. Collective Insights on Collective Impact, 2-5.
https://buildingblocksofpeace.com/wp-content/uploads/2021/03/Fall_2014_Essential_
Mindset_Shifts_for_Collective_Impact.pdf
174

Kania, J., & Kramer, M. (2011). Collective impact. Stanford Social Innovation Review, 35–41.
Kania, J., & Kramer, M. (2013). Embracing emergence: How collective impact addresses
complexity. Stanford Social Innovation Review. https://assets.neindiana.com/resources/
embracing-emergence-how-collective-impact-addresses-complexity.pdf
Kaufman, J., Jaser, S., Vaughan, E., Reynolds, J., Di Donato, J., Bernard, S., & Hernandez-
Brereton, M. (2010). Patterns in office referral data by grade, race/ethnicity, and gender.
Journal of Positive Behavior Interventions, 12(1), 44–54.
https://doi.org/10.1177/1098300708329710
Kirkpatrick, J. D., & Kirkpatrick, W. K. (2016). Kirkpatrick's four levels of training evaluation.
Association for Talent Development.
Kirschner, F., Paas, F., & Kirschner, P. A. (2009). A cognitive load approach to collaborative
learning: United brains for complex tasks. Educational Psychology Review, 21(1), 31–42.
https://doi.org/10.1007/s10648-008-9095-2
Krathwohl, D. R. (2002). A revision of Bloom’s taxonomy: An overview. Theory into Practice,
41(4), 212–218. https://doi.org/10.1207/s15430421tip4104_2
Kubisch, A. C., Auspos, P., Brown, P., & Dewar, T. (2010). Community change initiatives from
1990-2010: Accomplishments and implications for future work. Community Investments,
22(1), 8-12. https://core.ac.uk/download/pdf/6231641.pdf
LeChasseur, K. (2016). Re-examining power and privilege in collective impact. Community
Development, 47(2), 225–240. https://doi.org/10.1080/15575330.2016.1140664
Loughan, A., & Perna, R. (2012). Neurocognitive impacts for children of poverty and neglect.
CYF News. http://www.apa.org/pi/families/resources/newsletter/2012/07/neurocognitive-
impacts
175

Mallett, C. A. (2017). The school-to-prison pipeline: Disproportionate impact on vulnerable
children and adolescents. Education and Urban Society, 49(6), 563–592.
https://doi.org/10.1177/0013124516644053
Manduca, S. (2019). Punishing and toxic neighborhood environments independently predict the
intergenerational social mobility of Black and White children. Proceedings of the
National Academy of Sciences - PNAS, 116(16), 7772–7777.
https://doi.org/10.1073/pnas.1820464116
Martin, E. (2017). Hidden consequences: The impact of incarceration on dependent children.
American Jails, 31(5), 16–22.
Martinez, G., Daniels, K., & Chandra, A. (2012). Fertility of men and women aged 15-44 years
in the United States: National Survey of Family Growth, 2006-2010. National Health
Statistics Reports, 51, 1–28.
Maxwell, J. A. (2012). Qualitative research design. Sage.
Mayer, R. E. (2011). Applying the science of learning. Pearson.
Mazza, J. R. S. E., Lambert, J., Zunzunegui, M. V., Tremblay, R. E., Boivin, M., & Côté, S. M.
(2017). Early adolescence behavior problems and timing of poverty during childhood: A
comparison of lifecourse models. Social Science & Medicine (1982), 177, 35–42.
https://doi.org/10.1016/j.socscimed.2017.01.039
McAfee, M., Blackwell, A. G., & Bell, J. (2015). Equity: The soul of collective impact.
PolicyLink.
McKinlay, J. B. (1979). A case for refocusing upstream: the political economy of illness. In E.
G. Jaco (Ed.), Patients, physicians and illness: A sourcebook in behavioral science and
health (pp. 9–25). Free Press
176

Merriam, S. B., & Tisdell, E. J. (2016). Qualitative research: A guide to design and
implementation (4th ed.). Jossey-Bass.
Moran, P., Jacobs, C., Bunn, A., & Bifulco, A. (2007). Multi-agency working: Implications for
an early-intervention social work team. Child & Family Social Work, 12(2), 143–151.
https://doi.org/10.1111/j.1365-2206.2006.00452.x
Murray, J., & Farrington, D. (2005). Parental imprisonment: Effects on boys’ antisocial
behaviour and delinquency through the life-course. Journal of Child Psychology and
Psychiatry, and Allied Disciplines, 46(12), 1269–1278. https://doi.org/10.1111/j.1469-
7610.2005.01433.x
National Assessment of Educational Progress. (2019). National achievement-level results.
https://www.nationsreportcard.gov/reading/nation/achievement/?grade=4.
National Clearinghouse on Supportive School Discipline. (n.d.). Reference guide: Exclusionary
discipline. National Collaborating Centre for Determinants of Health.
https://nccdh.ca/glossary/entry/upstream-downstream
National Collaborating Centre for Determinants of Health. (2021). Upstream/downstream.
https://nccdh.ca/glossary/entry/upstream-downstream
National Equity Project. (2020). Educational equity. https://www.nationalequityproject.org/
education-equity-definition
National Institutes of Health. (2019). Notice of NIH's interest in diversity.
https://grants.nih.gov/grants/guide/notice-files/NOT-OD-20-031.html
National Research Council. (2014). The growth of incarceration in the United States: Exploring
causes and consequences. The National Academies Press. https://doi.org/10.17226/18613
177

National Research Council & Institute of Medicine. (2009). Preventing mental, emotional, and
behavioral disorders among young people: Progress and possibilities. The National
Academies Press. https://www.ncbi.nlm.nih.gov/books/NBK32775/
Okonofua, J. A., & Eberhardt, J. L. (2015). Two strikes: Race and the disciplining of young
students. Psychological Science, 26(5), 617–624.
https://doi.org/10.1177/0956797615570365
Orrock, J., & Clark, M. A. (2018). Using systems theory to promote academic success for
African American males. Urban Education, 53(8), 1013–1042.
https://doi.org/10.1177/0042085915613546
Pajares, F. (2003). Self-efficacy beliefs, motivation, and achievement in writing: a review of the
literature. Reading & Writing Quarterly, 19(2), 139–158.
https://doi.org/10.1080/10573560308222
Parkhurst, M., & Preskill, H. (2014). Learning in action: Evaluating collective impact. Stanford
Social Innovation Review, 12(4), S17.
Patton, M. Q. (2015). Qualitative research and evaluation methods (4th ed.). Sage.
Pettit, B., & Western, B. (2004). Mass imprisonment and the life course: Race and class
inequality in U.S. incarceration. American Sociological Review, 69(2), 151–169.
https://doi.org/info:doi/ https://doi.org/10.1177/000312240406900201
PEW Charitable Trusts. (2010). Collateral costs: Incarceration’s effect on economic mobility.
Phillips, J., & Land, K. (2012). The link between unemployment and crime rate fluctuations: An
analysis at the county, state, and national levels. Social Science Research, 41(3), 681–
694. https://doi.org/10.1016/j.ssresearch.2012.01.001
178

Philpart, M., & Bell, J. (2015). The Boys and Men of Color Framework: A model for community
and systems change. The Foundation Review, 7(3), 98–. https://doi.org/10.9707/1944-
5660.1257
Pintrich, P. R. (2003). A motivational science perspective on the role of student motivation in
learning and teaching contexts. Journal of Educational Psychology, 95(4), 667–686.
https://doi.org/10.1037/0022-0663.95.4.667
Preskill, P., & Splansky, J. (n.d.). Guide to evaluating collective impact.
https://www.fsg.org/publications/guide-evaluating-collective-impact
Reardon, S., & Bischoff, K. (2011). Income inequality and income segregation. American
Journal of Sociology, 116(4), 1092–1153. https://doi.org/10.1086/657114
Riehl, C., & Lyon, M. A. (2017). Counting on context: Cross-sector collaborations for education
and the legacy of James Coleman’s sociological vision. The Annals of the American
Academy of Political and Social Science, 674(1), 262–280.
https://doi.org/10.1177/0002716217735284
Rogers, J. S. (1998). Community schools: Lessons from the past and present. Unpublished
manuscript. Kirp, DL (2011). Kids first: Five big ideas for transforming children’s lives
and America’s future. Public Affairs.
Rueda, R. (2011). The 3 dimensions of improving student performance: Finding the right
solutions to the right problems. Teachers College Press.
Sacks, V., & Murphey, D. (2018). The prevalence of adverse childhood experiences, nationally,
by state, and by race or ethnicity. Child Trends.
https://www.childtrends.org/publications/prevalence-adverse-childhood-experiences-
nationally-state-race-ethnicity
179

Sakamoto, A., Tamborini, C. R., & Kim, C. (2018). Long-term earnings differentials between
African American and White men by educational level. Population Research and Policy
Review, 37, 91–116. https://doi.org/10.1007/s11113-017-9453-1
Scharf, R. J., Stroustrup, A., Conaway, M. R., & DeBoer, M. D. (2016). Growth and
development in children born very low birthweight. Archives of Disease in Childhood.
Fetal and Neonatal Edition, 101(5), F433–F438. https://doi.org/10.1136/archdischild-
2015-309427
Schein, E. H. (1992). How can organizations learn faster? The problem of entering the Green
Room. Alfred P. Sloan School of Management
Schein, E. H., & Schein, P. (2017). Organizational culture and leadership (Vol. 5). John Wiley
& Sons.
Schraw, G., Veldt, M., & Olafson, L. (2009). Knowledge. Education. com.
Schraw, G., & McCrudden, M. (2006). Information processing theory.
http://www.education.com/reference/article/information-processingtheory/
Shannon, S., Uggen, C., Schnittker, J., Thompson, M., Wakefield, S., & Massoglia, M. (2017).
The growth, scope, and spatial distribution of people with felony records in the United
States, 1948–2010. Demography, 2017(54), 1795–1818.
Skiba, R., Michael, R., Nardo, A., & Peterson, R. (2002). The color of discipline: Sources of
racial and gender disproportionality in school punishment. The Urban Review, 34(4),
317–342. https://doi.org/10.1023/A:1021320817372
Skiba, R. J., Arredondo, M. I., & Williams, N. T. (2014). More than a metaphor: The
contribution of exclusionary discipline to a school-to-prison pipeline. Equity &
Excellence in Education, 47(4), 546–564. https://doi.org/10.1080/10665684.2014.958965
180

Skiba, R. J., Horner, R. H., Chung, C., Rausch, M. K., May, S. L., & Tobin, T. (2011). Race is
not neutral: A national investigation of African American and Latino disproportionality
in school discipline. School Psychology Review, 40(1), 85–107.
https://doi.org/10.1080/02796015.2011.12087730
Stachowiak, S., & Gase, L. (2018). Does collective impact really make an impact? Stanford
Social Innovation Review.
Steinberg, M. P., & Lacoe, J. (2017). What do we know about school discipline reform?
Education Next, 17(1).
Strompolis, M., Tucker, W., Crouch, E., & Radcliff, E. (2019). The intersectionality of adverse
childhood experiences, race/ethnicity, and income: Implications for policy. Journal of
Prevention & Intervention in the Community, 47(4), 310–324.
https://doi.org/10.1080/10852352.2019.1617387
Sugie, N. F. (2012). Punishment and welfare: Paternal incarceration and families’ receipt of
public assistance. Social Forces, 90(4), 1403–1427. https://doi.org/10.1093/sf/sos055
Swisher, R., & Shaw-Smith, U. (2015). Paternal incarceration and adolescent well-being: Life
course contingencies and other moderators. The Journal of Criminal Law & Criminology,
104(4), 929.
Sykes, B., & Pettit, B. (2014). Mass incarceration, family complexity, and the reproduction of
childhood disadvantage. The Annals of the American Academy of Political and Social
Science, 654(1), 127–149. https://doi.org/10.1177/0002716214526345
Sykes, B. L., & Maroto, M. (2016). A wealth of inequalities: Mass incarceration, employment,
and racial disparities in U.S. household wealth, 1996 to 2011. The Russell Sage
181

Foundation Journal of the Social Sciences, 2(6), 129–152.
https://doi.org/10.7758/rsf.2016.2.6.07
Todd, A., Thiem, K., & Neel, R. (2016). Does seeing faces of young Black boys facilitate the
identification of threatening stimuli? Psychological Science, 27(3), 384–393.
https://doi.org/10.1177/0956797615624492
Tonry, M. (2011). Punishing race a continuing American dilemma. Oxford University Press.
Townsley, R., Watson, D., & Abbott, D. (2004). Working partnerships? A critique of the process
of multi-agency working in services to disabled children with complex health care needs.
Journal of Integrated Care, 12(2), 24–34. https://doi.org/10.1108/14769018200400013
U.S. Bureau of Justice Statistics. (2020). National Prisoner Statistics, United States, 1978-2018.
https://doi.org/10.3886/ICPSR37639.v1
U.S. Bureau of Labor Statistics. (2020). Labor market activity of Blacks in the United States.
U.S. Department of Education. (2021a). Characteristics of children’s families.
https://nces.ed.gov/programs/coe/indicator/cce
U.S. Department of Education. (2021b). 2013–14 state and national estimations.
https://ocrdata.ed.gov/estimations/2013-2014
U.S. Department of Education & U.S. Department of Justice. (2011). The Supportive School
Discipline Initiative. https://www2.ed.gov/policy/gen/guid/school-discipline/appendix-3-
overview.pdf
U.S. Department of Housing and Urban Development. (2018).
https://www.huduser.gov/portal/pdredge/pdr-edge-frm-asst-sec-111918.html
Urban Strategies Council. (2020). Equity is… https://urbanstrategies.org/about/equity/#.U-
ULDlvn_IU
182

Wakefield, S., & Uggen, C. (2010). Incarceration and Stratification. Annual Review of Sociology,
36(1), 387–406. https://doi.org/10.1146/annurev.soc.012809.102551
Weaver, L. (2016). Possible: Transformational change in collective impact. Community
Development, 47(2), 274–283. https://doi.org/10.1080/15575330.2016.1138977
Welsh, R. O., & Little, S. (2018). The school discipline dilemma: A comprehensive review of
disparities and alternative approaches. Review of Educational Research, 88(5), 752–794.
https://doi.org/10.3102/0034654318791582
Western, B. (2002). The impact of incarceration on wage mobility and inequality. American
Sociological Review, 67(4), 526–546. https://doi.org/10.2307/3088944
Western, B., & Pettit, B. (2010). Incarceration & social inequality. Daedalus, 139(3), 8–19.
https://doi.org/10.1162/DAED_a_00019
Wilbur, M., Marani, J., Appugliese, D., Woods, R., Siegel, J., Cabral, H., & Frank, D. (2007).
Socioemotional effects of fathers’ incarceration on low-income, urban, school-aged
children. Pediatrics, 120(3), e678–e685. https://doi.org/10.1542/peds.2006-2166
Wildeman, C. (2014). Parental incarceration, child homelessness, and the invisible consequences
of mass imprisonment. The Annals of the American Academy of Political and Social
Science, 651(1), 74–96. https://doi.org/10.1177/0002716213502921
Williams, J., & Marxer, S. (2014). Bringing an equity lens to collective impact. Collective
Impact Forum. https://collectiveimpactforum.org/sites/default/files/Equityand
CollectiveImpact_UrbanStrategiesCouncil.pdf
Wolf, S., Magnuson, K. A., & Kimbro, R. T. (2017). Family poverty and neighborhood poverty:
Links with children’s school readiness before and after the Great Recession. Children
183

and Youth Services Review, 79, 368–384.
https://doi.org/10.1016/j.childyouth.2017.06.040
Wood, J. L., Harris, F., III, & Howard, T. C. (2018). Get Out! Black male suspensions in
California public schools. Community College Equity Assessment Lab and the UCLA
Black Male Institute.
Woods-Jaeger, B. A., Cho, B., Sexton, C. C., Slagel, L., & Goggin, K. (2018). Promoting
resilience: Breaking the intergenerational cycle of adverse childhood experiences. Health
Education & Behavior, 45(5), 772–780. https://doi.org/10.1177/1090198117752785
Wright, U. (2019, July 16). Systems change is a noun and a verb. FSG
https://www.fsg.org/blog/systems-change-noun-and-verb
Yoshikawa, H., Aber, J. L., & Beardslee, W. R. (2012). The effects of poverty on the mental,
emotional, and behavioral health of children and youth: Implications for prevention. The
American Psychologist, 67(4), 272–284. https://doi.org/10.1037/a0028015

184

Appendix A: Interview Protocol
Introduction
Mrs./Mr./Dr. X, Thank you for taking the time to talk with me today. As you know, I am a
doctoral student with the Rossier School of Education, and I am conducting an evaluation study
of the implementation of the SACT integrated multi-tiered system of support collective impact
effort. I am hoping to develop a deeper understanding of how this steering committee, made of
members who come from many different agencies, work together to lead such an important
initiative.
I also want to remind you of the research protocols that I will use to ensure your
confidentiality and comfort. I will be recording today’s interview, and as we chat today I will
also avoid using your real name. Throughout the analysis of the interview data I will use a
pseudonym in place of your real name. The recording will only be listened to by me, and it will
not be linked with your name at all. The purpose of the recording is to allow me to ensure that I
have captured the information accurately. All recorded data and the transcriptions of these
interviews will be kept in password protected files on my computer.
Your participation in this study is completely voluntary, and as such you may choose to
skip any questions today or to withdraw from the interview at any time. Do you have any
questions about these procedures?
So to begin, I would like to start with learning a little more about your role in your agency.
Interview Questions
1. How long have you worked with this agency?
2. What is your role and responsibility in your agency?
Transition: Thank you. Now can you ...
185

3. Please tell me how you came to be involved in the work of SACT.
Transition: Thank you. Now can you ...
4. Please tell me about your role on the steering committee.
Transition: Thank you. Now I will ask you more about what you know about SACT.
5. What is the mission of SACT? (K-F)
Probe: Is this mission still relevant today?
Transition: Ok, so let’s shift to the focus on prevention. (go to question 6)
6. What is the SACT theory of change? (K-F)
Probe: How does this fit with the SACT mission?
Transition: I’d like to shift now to the single system of support initiative that the steering
committee has adopted. (go to question 7)
7. Please describe the integrated multi-tiered system of support initiative. (K-F)
Transition: Based on your understanding of this initiative,… (go to question 8)
8. How would you describe the role of various cross-agency partners in designing and
implementing the single system of support? (K-C)
9. Why is collaborating across agencies necessary for building out the integrated multi-
tiered system of support? (K-C)
Transition: I would like to now focus on how all of these participating agencies work
together. One of the conditions of successful cross-sector collaboration is
communication. (go to question 10)
10. Please describe how your agency currently communicates with other agencies to establish
partnerships. (K-P)
186

11. Please tell me about a time when you felt confident partnering with another agency to fill
a need in the community. (M-SE)
12. Considering past collaboration efforts, how would you think about working differently
moving forward? (K-M)
Transition: Now I would like to shift just a bit to ask you about your work as it relates to
the single system of support initiative. (go to question 13)
13. How does your work on the integrated multi-tiered system of support align with the work
you do in your agency? (M-V)
14. How will the integrated multi-tiered system of support benefit the work of your agency?
(M-V)
15. On a scale of 1 to 5, with 1 being no confidence at all and 5 being the highest confidence,
how confident are you that this group can stay focused on the common agenda? (M-CE)
Transition: Now I would like to ask about how innovation and creativity are embraced
for problem solving (go to question 16)
16. How supportive is county leadership of using innovation and creativity to solve
problems? (O-CM)
17. Outside of the scheduled steering committee meetings, how much time do you spend on
this initiative? (O-R)
Probe: Does that additional time impact your regular work duties?
18. Thinking about your role as a steering committee member, what additional resources do
you need? (O-CS)
Transition: I have one last question. (go to question 19)
187

19. Is there anything else you would like to share that I have not already asked you about
with regards to the integrated multi-tiered system of support collective impact effort?
This concludes our interview. Thank you so much for your time today and for sharing your
insights with me. Do you have any questions for me before we close? (Allow time for questions).
If you think of any (additional) questions after today, please feel free to contact me via email.
Thank you.

188

Appendix B: Qualitative Research Documents Analysis Protocol
Artifact/Document Influence assessed (K-M-O) Data analyzed

189

Appendix C: Information Sheet for Exempt Research
University of Southern California
Rossier School of Education
3470 Trousdale Pkwy, Los Angeles CA, 90089
STUDY TITLE: All Hands on Deck! Dismantling the Prison Pipeline through Collective
Impact
PRINCIPAL INVESTIGATOR: Rhonda Beasley
FACULTY ADVISOR: Dr. Darline Robles
You are invited to participate in a research study. Your participation is voluntary. This document
explains information about this study. You should ask questions about anything that is unclear to
you.
PURPOSE
The purpose of this study is to evaluate the Siloed Agencies Collaboration Team’s (SACT)
progress toward the stated goal of implementing the integrated multi-tiered system of support
collective impact effort to dismantle the school-to-prison pipeline. We hope to learn more about
the gaps in participants’ knowledge, motivation, and organizational culture in implementing this
initiative. You are invited as a possible participant because you currently serve as a steering
committee member.
PARTICIPANT INVOLVEMENT
190

If you decide to take part, you will be asked to participate in an individual interview, conducted
via Zoom due to COVID restrictions. You will also be asked to consent to being observed in at
least two SACT steering committee meetings.
The interview should last about one hour and will be recorded with your consent. The recording
will be used by the researcher for the sole purpose of ensuring that all information is captured
correctly. If you prefer to not have your interview recorded, you may decline recording prior to
or during the interview. Your participation will not be affected.
Two SACT steering committee meetings and one SACT meeting will be observed. Due to
ongoing COVID restrictions, these meetings will occur using an online platform. Data related to
activities and interactions, as well as conversations will be observed but no participant
identifying information will be connected to the data collected. You will be notified two weeks
in advance of the meetings that will be observed. You have the option of declining to participate
in these meeting observations.

191

CONFIDENTIALITY
The members of the research team and the University of Southern California Institutional
Review Board (IRB) may access the data. The IRB reviews and monitors research studies to
protect the rights and welfare of research subjects.
When the results of the research are published or discussed in conferences, no identifiable
information will be used.
Throughout the interview, your real name will not be used. Throughout the analysis of the
interview data a pseudonym will be used in place of your real name. The interview recording
will only be listened to by the principal investigator, and will not be linked with your name. The
purpose of the recording is to ensure that the information has been captured accurately. All
recorded data and the transcriptions of this interview will be kept in password protected files on
the principal investigator’s computer. All data will be destroyed upon the completion of this
study.
INVESTIGATOR CONTACT INFORMATION
If you have any questions about this study, please contact the following individuals:
Principal Investigator
Rhonda Beasley
rjbeasle@usc.edu
Faculty Advisor
Dr. Darline Robles
dprobles@rossier.usc.edu

192

IRB CONTACT INFORMATION
If you have any questions about your rights as a research participant, please contact the
University of Southern California Institutional Review Board at (323) 442-0114 or email
irb@usc.edu.

193

Appendix D: Recruitment Email
Dear (Insert Participant Name),

My name is Rhonda Beasley and I am a third-year doctoral student at the University of
California in the area of Organizational Change and Leadership. I am currently engaged in my
doctoral research which is an evaluation study of the SACT steering committee members’
readiness to engage in a collective impact effort.
The purpose of this study is to evaluate the Siloed Agencies Collaboration Team’s (SACT)
progress toward the stated goal of implementing the integrated multi-tiered system of support
collective impact effort to dismantle the school-to-prison pipeline.
As you are a SACT steering committee member, you provide valuable insight into the work on
SACT and the progress made to date, as well as any obstacles that may prevent successful
implementation of the IMTSS. I am reaching out to see if you would be willing to participate in a
virtual interview that would be approximately 45-60 minutes in length as part of this research
study.
To adhere to statewide requirements for physical distancing, all interviews will be conducted via
Zoom. Interview content will be kept confidential as well as responses being kept anonymous.

I greatly appreciate your consideration of my request to participate. Please feel free to reply to
this email with your response to this invitation.

Thank you for your consideration.

Sincerely,
Rhonda Beasley

Abstract (if available)

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

Workforce system collaboration

PDF

Collective Impact: a framework to advance health promotion in higher education

PDF

An evaluation study of... What do teachers know about gifted students?

PDF

Raising special needs: an evaluation study of respite care for medically fragile children living with autism and other illnesses

PDF

Women of color senior leaders: pathways to increasing representation in higher education

PDF

A customer relationship management approach to improving certificate completion

PDF

Retaining special education staff in a rural consortium setting: an evaluation study

PDF

Understand how leaders use data to measure the effectiveness of leadership development programs

PDF

Closing the completion gap for African American students at California community colleges: a research study

PDF

Reducing suspensions through implementation of schoolwide PBIS

PDF

Avoidance of inpatient medical necessity denials for short-stay admissions: an evaluative study

PDF

The impact of dual enrollment programs on first-year college success for Hispanic students from low-socioeconomic-status communities: a promising practice

PDF

Improving the representation of women in the independent school headship

PDF

Building capacity in homeroom teachers to support English language learners

PDF

Universal algebra: an evaluation

PDF

African American college completion at Hillside College: an evaluation study

PDF

Addiction recovery: success in the recovery process

PDF

Education after incarceration: identifying practices in prison education programs that positively affect the persistence of formerly incarcerated men to continue education

PDF

Calibrating the college preparation bridgeway for independent study program students at Amazing Independent High School District: a multisite evaluation study through the teachers' lens

PDF

Using mastery learning to address gender inequities in the self-efficacy of high school students in math-intensive STEM subjects: an evaluation study

Asset Metadata

Creator Beasley, Rhonda Jeanne (author)

Core Title All hands on deck! Collective impact for systemic change

School Rossier School of Education

Degree Doctor of Education

Degree Program Organizational Change and Leadership (On Line)

Degree Conferral Date 2021-12

Publication Date 09/27/2021

Defense Date 09/02/2021

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag collective impact,cross-agency collaboration,intergenerational cycle of disadvantage,OAI-PMH Harvest,systemic change

Format application/pdf (imt)

Language English

Contributor Electronically uploaded by the author (provenance)

Advisor Robles, Darline (committee chair), Gothold, Paul (committee member), Malloy, Courtney (committee member)

Creator Email rhondajbeasley@gmail.com,rjbeasle@usc.edu

Permanent Link (DOI) https://doi.org/10.25549/usctheses-oUC15965162

Unique identifier UC15965162

Legacy Identifier etd-BeasleyRho-10105

Document Type Dissertation

Format application/pdf (imt)

Rights Beasley, Rhonda Jeanne

Type texts

Source University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Access Conditions The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the author, as the original true and official version of the work, but does not grant the reader permission to use the work if the desired use is covered by copyright. It is the author, as rights holder, who must provide use permission if such use is covered by copyright. The original signature page accompanying the original submission of the work to the USC Libraries is retained by the USC Libraries and a copy of it may be obtained by authorized requesters contacting the repository e-mail address given.

Repository Name University of Southern California Digital Library

Repository Location USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA

Repository Email cisadmin@lib.usc.edu