University of Southern California Dissertations and Theses

Assessing implementation of a child welfare system practice change

(USC Thesis Other)

Assessing implementation of a child welfare system practice change

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content

ASSESSING IMPLEMENTATION OF A
CHILD WELFARE SYSTEM PRACTICE CHANGE

by

Jaymie Lorthridge

A Dissertation Presented to the
FACULTY OF THE USC GRADUATE SCHOOL
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulfillment of the
Requirements for the Degree
DOCTOR OF PHILOSOPHY
(SOCIAL WORK)

May 2014

Copyright 2014 Jaymie Lorthridge

ii

DEDICATION
My words are spent, and thank you is so inadequate. Mom, dad bonus parents (Phil &
Connie), Jaylah, Fab Five +1 I dedicate this journey (as reflected in these pages) to you.
Thanks for your patience, guidance, humor, and endless support. Since my words here are so
brief I commit to showing my thanks to in the years to come.

iii

ACKNOWLEDGEMENTS
This dissertation would not be possible without the critical input of my committee. I
would like to acknowledge your perseverance and understanding. I also must acknowledge the
employees of the Los Angeles County Department of Children and Family Services, who
graciously shared their time with the researcher. Participants reflect an incredible group of staff;
people that rarely receive thanks. I thank you for the work you do on behalf of the most
vulnerable families.

iv

TABLE OF CONTENTS
DEDICATION ...................................................................................................................... ii
ACKNOWLEDGEMENTS................................................................................................... iii
LIST OF TABLES................................................................................................................. viii
LIST OF FIGURES............................................................................................................... x
ABSTRACT .......................................................................................................................... xii
CHAPTER ONE: INTRODUCTION, PROBLEM STATEMENT, AND STUDY
CONTRIBUTION..................................................................................................................1

Complexity of Child Welfare Systems...................................................................... 4
Consequences of Maltreatment.................................................................................. 7
Statement of the Problem: Assessing Implementation of Practice Change…….... 9
Contribution of the Study…………... ...................................................................... 11
Research Questions ................................................................................................... 18
Organization of Dissertation ..................................................................................... 19
CHAPTER TWO: REVIEW OF LITERATURE DESCRIBING THE CHILD WELFARE
SYSTEM AND INDICATORS OF IMPLEMENTATION SUCCESS ............................... 21

Child Welfare Systems……………………………………………. ....................... 21
History……………………………………………....................................... 21
Multiple Levels of Regulation for Child Welfare Systems........................... 23
Child Welfare Systems Structures and Functions.......................................... 26
Levels of intervention: Court ordered, voluntary, and
alternative/differential response.................................................... 27
Front and back end units: Emergency response, family
maintenance/preservation, family reunification, and
permanency planning.………………………………………….. 27
Emergency response……………………………………………...... 28
Family maintenance/preservation………………………………….. 31
Family reunification……………………………………………….. 31
Permanency planning………………………………………….…… 32
Implementing Child Welfare Systems Practice Change………………..…………. 33
What Impacts Implementation of Practice Change?...................................... 34
Organizational influences………………………………………...... 35
Actor level influences……………………………………………… 36
Environmental influences…………………………………………. 37
Study specific assessment of implementation………………………38
Assessing indicators of implementation success…………………... 39
Assessing environmental influences……………………………….. 41

v

Assessing trends in POE targeted outcomes……………………..... 43

CHAPTER THREE: METHODS.......................................................................................... 48
Research Aims and Questions .................................................................................. 48
Research Design – Case Study.................................................................................. 52
Unit of Analysis – Regional Offices…………………………………………..….. 53
Data Sources and Procedures..................................................................................... 55
Question one………………………………………………………………. 55
Question two……………………………………………………………… 58
Question three…………………………………………………………….. 60
Analysis………………………………………………………………..................... 65
Question one………………………………………………………………. 65
Question two: Did the environment influence implementation
success?………………………………………………………………… 68
Question three……………………………………………………………… 68

CHAPTER FOUR: IMPLEMENTATION INDICATORS AND ENVIRONMENTAL
INFLUENCES RESULTS ............................................................................................. 70

Questions One and Two……………………………………………………………. 70
Office excluded in question one and two results…………………………... 72
Low Implementation Office Level Results……………………………………….. 75
Office one…………………………………………………………………. 75
Office two…………………………………………………………………. 78
Office three……………………………………………………………….. 82
Office four…………………………………………………………………. 85
Office five…………………………………………………………………. 87
Office six………………………………………………………………….. 90
Office seven……………………………………………………………….. 93
Medium Implementation Office Level Results………..………………………….. 96
Office eight……………………………………………………………….. 96
Office nine………………………………………………………………… 98
Office ten………………………………………………………………….. 102
Office eleven……………………………………………………………….. 104
Office twelve………………………………………………………………. 106
Office thirteen……………………………………………………………… 108
High Implementation Office Level Results………………………………………... 111
Office fourteen…………………………………………………………….. 111
Office fifteen………………………………………………………………. 113
Summary of Offices by Level of Implementation…………………………………. 117
Low implementation offices………………………………………………. 117
Medium implementation offices…………………………………………... 117
High implementation offices………………………………………………. 118
Continued Presence Indicators in Offices 14 and 15……………………………… 120

CHAPTER FIVE: HIGH IMPLEMENTATION OFFICE OUTCOMES............................ 122

vi

Question Three……………………………………………………………….……. 122
Level of service in offices 14 and 15 in comparison to other offices……… 123
Office 14 and 15 referral characteristics………………………….. 125
Removals………………………………………………………….. 129
Reunifications……………………………………………………… 132
Timeline to reunifications………………………………………… 134
Recurrence of maltreatment……………………………………….. 137

CHAPTER SIX: CONCLUSION……………………………………………………….. 140

Discussion……………… ......................................................................................... 140
Why Did Degree of Implementation Differ?............................................................. 141
What Affected the Presence of Implementation Success Indicators?....................... 144
Early intervention: Recognizing and utilizing family strengths
to avoid removal………………………………………………………. 144
Family collaboration through strengths-based communication
and family involvement in decision making………………………….... 145
Collaboration with community-based organizations through
communication and collegial relationships…………………………….. 147
Leader/Champion………………………………………………………….. 149
Cross unit collaboration through communication and collegial
relationships……………………………………………………………. 151
Resource adjustments: Adequate emergency response and
bridge worker staff…………………………………………………….. 153
How Did the Environment Influence Implementation?............................................. 155
Were Desired Outcomes Observed for Children Served by High
Implementation Offices?....................................................................................... 156
Limitations……………………. ............................................................................... 157
Implications for POE and Child Welfare Practice……………………………….. 159
Implications for Implementation Assessment…………………………………….. 161

References ............................................................................................................................. 163
Appendix A. Federal Child Welfare Structure……………………………………………. 180
Appendix B. Point of Engagement Process Flowchart……….............................................. 181
Appendix C. POE Components ........................................................................................... 182
Appendix D. Informed Consent and Script for 2012 Focus Groups………………………. 185
Appendix E. Template for Questions 1 & 2 2008-2009 Data………………………….….. 189
Appendix F. 2009 Referrals by Office……………………………………………………. 193
Appendix G. 2009 Removals by Office…………………………………………………… 194

vii

Appendix H. 2009 Children in Out-of-Home Care by Office…………………………….. 195

viii

LIST OF TABLES
Table 1. Major POE Components.......................................................................................... 13
Table 2. Overarching Research Question Crosswalk……………........................................ 64
Table 3. Presence of Indicators by Office.............................................................................. 73
Table 4. Office One Community Needs…………………………………………………… 78
Table 5. Office Two Community Needs………………………………................................ 82
Table 6. Office Three Community Needs……….................................................................. 85
Table 7. Office Four Community Needs……........................................................................87
Table 8. Office Five Community Needs …………………….............................................. 89
Table 9. Office Six Community Needs ................................................................................. 93
Table 10.Office Seven Community Needs............................................................................ 95
Table 11. Office Eight Community Needs…………………………………....................... 98
Table 12. Office Nine Community Needs…………………………………………..…….. 101
Table 13. Office Ten Community Needs…………………………………………….…….. 104
Table 14. Office Eleven Community Needs………………………………………………. 106
Table 15. Office Twelve Community Needs………………………………………….…... 108
Table 16. Office Thirteen Community Needs………………………………………….….. 110
Table 17. Office Fourteen Community Needs…………………………..………………..... 113
Table 18. Office Fifteen Community Needs………………………………………………. 116
Table 19. Predominant Characteristics of Referrals for Offices 14 & 15 2009-2012…...… 127
Table 20. 2009-2012 Status of Removals by Open Cases & by Offices………………..…. 130
Table 21. 2009-2012 Reunifications by Office and Countywide…………………………. 133
Table 22. Median Time to Reunification in Days by Office……………………………… 135

ix

Table 23. 2009-2012 Recurrences by Office and Countrywide…………………………… 138
Table 24. Changes in Collaboration with Families................................................................ 146
Table 25. Value of Collaboration with Families…….…………………………………….. 146
Table 26. Attributions for Lack of Collaboration with Families………………………….. 147
Table 27. Facilitators and Barriers to Collaboration with CBOs..………………………… 148
Table 28. Affirmative vs. Unsupportive Climates for Staff Collaboration………………... 152
Table 29. Contributors to Inadequate Staffing…………………………………………….. 155
Table 30. Summary of Safety and Permanency Outcomes …………………….................. 157

x

LIST OF FIGURES
Figure 1. Conceptual Model for Assessing Implementation................................................. 3
Figure 2. Interdependent Relationships between Federal, State, and County
Child Welfare Systems…………..……................................................................. 4
Figure 3. Regulatory Authority………………………………………………………........ 25
Figure 4. Services Provided by DCFS Units in 2009……………………………………… 28
Figure 5. Child Welfare System Decision Points………………………………………….. 30
Figure 6. The Collaboration Loop………………………………………………...……….. 40
Figure 7. Regional Office Staffing Structure………………………………………………. 54
Figure 8. Samples Referrals to Reunifications and used for Question Three……………… 62
Figure 9. Offices Ranked by Service Category……………………………………………. 124
Figure 10. Office 14 2009-2012 Referrals………………………………………………… 125
Figure 11. Office 15 2009-2012 Referrals…………………………………………………. 126
Figure 12. 2009-2012 Referrals by Office & County……………………………………… 126
Figure 13. 2009-2012 Referrals and Dispositions by Office………………………………. 128
Figure 14. 2009-2012 Number of Referrals and Dispositions by Office & County………. 129
Figure 15. Office 14 Number of Removals from 2009-2012……………………………… 130
Figure 16. Office 15 Number of Removals from 2009-2012 ……………………….......... 131
Figure 17. 2009-2012 Number of Removals by Office and County………………………. 132
Figure 18. Office 14 Number of Reunifications from 2009-2012 ………………………… 134
Figure 19. Office 14 Median Number of Days to Reunification from 2009-2012………… 135
Figure 20. Office 15 Median Number of Days to Reunification from 2009-2012………… 136
Figure 21. 2009-2012 Median Time to Reunification in Months by Office & County…… 137
Figure 22. 2009-2012 Number of Recurrences by Office & Countywide………………… 139
Figure 23. Contributors to Degree of Implementation…………………………………… 142

xi

Figure 24. Indicators by Frequency Across Offices……………………………………….. 144
Figure A1. Federal Child Welfare Structure………………………………………………. 180
Figure B1. Point of Engagement Process Flowchart………………………………………. 181
Figure F1. 2009 Referrals by Office………………………………………………………. 193
Figure G1. 2009 Removals by Office……………………………………………………… 194
Figure H1. 2009 Children in Out-of-Home Care by Office……………………………….. 195

xii

ABSTRACT
Meaningful child welfare system practice change is challenging. The complexity of the
child welfare system makes change challenging. When stakeholders are able to wade through
the complexity and change practice, they are under pressure to quickly establish the significance
of the change. The dire consequences of maltreatment and the urgency of family needs demand
immediate results. Stakeholders turn to child outcomes when assessing change impact. Yet the
assessment of change needs to start with an examination of implementation. After successful
implementation is documented then trends in outcomes can be tracked.
This study examined implementation of a practice change at the systems level. Point of
Engagement (POE) was implemented within the Los Angeles County Department of Children
and Family Services (DCFS). POE was a philosophical shift that informed a redesigned case
flow process (Marts, Lee, McRoy & McCroskey, 2008). The change was intended to facilitate
early engagement of families whose children were likely to be placed in foster care, and to
preserve the family by intervening with appropriate services as early possible. DCFS offices
served as the unit of analysis and POE implementation was staggered by office. At the time of
the study, POE had been implemented between two to six years. Literature suggests that
implementation occurs over two to four years (Fixsen et al., 2009), thus regional offices were
informative units of analysis.
Implementation literature has found that organizational and actor level characteristics and
environmental influences affect implementation (Henggeler et al., 2008; Rye & Kimberly, 2007).
There is not yet consensus about which characteristics and influences are most important, nor is
there consensus about operationalizing the characteristics and influences. Study-specific
prioritization and operationalization was necessary. First implementation success was defined.
Six indicators of implementation success were derived from a review of the implementation

xiii

literature and subsequent translation of featured factors into POE specific indicators. Indicators
included early intervention, collaboration with families and community-based organizations,
collaboration amongst office staff in different units, leader/champion, and resource adjustments.
Office-level implementation was judged as successful or not depending on the degree of
implementation. Degree of implementation, high, medium, or low, was defined by the number
of implementation indicators present as reported by regional office staff. A high degree of
implementation, operationalized as the presence of five or more indicators of implementation
success, was observed in two offices (#15 and #14). Six offices (#8-13) had a medium degree of
implementation, operationalized as the presence of three to four indicators of implementation
success. Seven offices (#1-7) had a low degree of implementation, operationalized as the
presence of one to two indicators of implementation success. The influence of the environment
on the presence or absence of indicators was also examined. Finally, trends in child safety and
permanency outcomes for those served by high implementation offices, and where the
environment influenced implementation success, were tracked over three years.
There were two high implementation offices. Office 14 had five out of six indicators of
success and Office 15 had all six. In Office 14, there was a positive climate for cross unit staff
collaboration, collaboration with families and community-based organizations, and for
intervening as early as possible. A practice change champion, in the form of a leader was also
present. Office 15 had all of the same indicators plus the office also made resource adjustments
needed for POE implementation. The environmental assessment included information on
families’ poverty level, levels of education, unemployment, proportion of grandparents as
caregivers and the extent of DCFS services. Identified needs of the communities were assessed
to determine whether the needs corresponded to implementation barriers cited by participants.

xiv

Level of needs did seem to influence staff workload; workload was a collaboration barrier
reported by participants. Observed outcome trends for the high implementation offices suggest
POE may help in reducing the timeline for reunification. During similar time periods the
timelines to reunification did not consistently shorten throughout Los Angeles County.

1

Chapter One: Introduction, Problem Statement, and Study Contribution
Meaningful child welfare practice change is challenging due to the complexity of the
child welfare system. When stakeholders are able to wade through the complexity and change
practice, they are under pressure to quickly establish the significance of the change. They need
to see immediate results because the consequences of maltreatment are so dire. Stakeholders
turn to child outcomes when assessing change impact; yet the assessment of change needs to
start with an examination of implementation. Research documents that implementation, while
challenging, must be included in the assessment of outcomes (Hall & Loucks, 1977; Klein &
Sorra, 1996) because the success of a practice change is partly attributable to implementation
(Dusenbury, Brannigan, Falco & Hansen, 2003).
This study examines implementation of a change that altered practices at the system
level. Meaning the study focuses on the change to, “..a collection of components or parts that are
organized around a common purpose or goal..” (Wulczyn et al, 2010, p. 2). Point of
Engagement (POE) was that systems change process and POE was implemented within the Los
Angeles County Department of Children and Family Services (DCFS). POE was a philosophical
shift that informed a redesigned case flow process (Marts, Lee, McRoy & McCroskey, 2008).
The change was intended to facilitate early engagement of families whose children were likely to
be placed in foster care, and to preserve the family by intervening with appropriate services as
early possible. There were several reasons why this systems level practice change was selected
for study. To further knowledge of child welfare system practice change, it was important to
select a change that was in some ways typical of the field. POE’s catalyst was, and still is, the
most common in child welfare systems. Public outcry led to scrutiny, in this case an audit, and
subsequently new directives or policies were created. Those policies required practice changes.

2

Other frequent catalysts for change include response to reports of child deaths, and scrutiny by
media or elected officials. Though POE was implemented alongside a number of other reforms
1
,
it had a wider (across all offices) and deeper impact then the other reforms. POE affected the
way in which all DCFS regional offices functioned. Studying implementation of wide sweeping
change allowed for observation of the intentional and unintentional consequences of change as
reflected in staff functioning and service delivery. POE was also chosen for study because if its
implementation site. DCFS is one of the largest child welfare systems in the United States, and
it is highly bureaucratic because its services are state regulated and county administered through
regional offices. The volume and diversity of the populations served, the highly differentiated
service environment created in response to service demands, and the multiple layers of oversight
make practice change difficult. It is important to know if implementation can be successful in
one of the most difficult environments.
POE implementation was assessed using study specific “Implementation Success
Indicators,” (see Figure 1). The indicators were derived after a review of the implementation
literature and study specific interpretation of pertinent concepts. The presence or absence of
indicators of implementation success in each regional office was assessed using office staff
interview and focus group data. After assessing the indicators, characteristics of each office’s
environment were assessed. Characteristics were examined to better understand what might
have facilitated or impeded implementation success. The final step in studying implementation
was observation of child safety and permanency trends among the offices where the most
indicators were present, and where the environment facilitated implementation success. Trends
were observed over three years.

1
Multiple reform processes were going on at the same time, most were implemented in a few regional offices.

3

Figure 1. Conceptual Model for Assessing Implementation

Operationalize & Assess for Presence of Indicators of POE
Implementation Success
Organizational - Resource adjustments such that staff were allocated where
needed [ER & ISW/Bridge]
Actor – Climate - changes (collaboration across DCFS units, collaboration
with families, collaboration with community organizations, intervening as
early as possible) perceived as positive
Actor - Leader and/or champion facilitates practice changes
Determine Implementation Factors Relevant to POE Practice
Changes
 Organizational – Resource adjustments
 Actor – climate, leadership/champions
 Environmental – Aspects of the practice context that may impact
implementation (i.e. starting points)
Observe Targeted Safety and Permanency Trends
 Fewer removals from home
 More cases terminated as reunifications
 Reductions in time to reunification
 Lower # of substantiated re- referrals (recurrence)

Operationalize & Assess Environment’s Influence on Implementation
Success
 Historical relationships with community based organizations
 Characteristics of community needs
Step One
Step Two
Step Three
Step Four
Figure 1. Conceptual Model for Assessing Implementation

4

Complexity of Child Welfare Systems
Systemic child welfare change is different than implementing a new program because of
all that the system encompasses. For the purposes of this study, a system is defined as,
“…hierarchies that have their own set of structures, functions, and interdependent relationships,”
(Kreger, Brindis, Manuel, & Sassoubre, 2007, p. 302). Systems by nature are complex, with
complexity defined as the result, “…from the inter-relationship, inter-action, and inter-
connectivity of elements within a system and between a system and its environment,” (Chan,
2001, p.1; see Figure 2).

Figure 2. Interdependent Relationships between Federal, State and County Child Welfare
Systems

5

Child welfare systems are the federal, state, and county organizations whose mission it is to
ensure:
all children are raised in safe, loving families, by—
1. protecting and promoting the welfare of all children;
2. preventing the neglect, abuse, or exploitation of children;
3. supporting at-risk families through services, which allow children, where appropriate,
to remain safely with their families or return to their families in a timely manner;
4. promoting the safety, permanence, and well-being of children in foster care and
adoptive families; and
5. providing training, professional development and support to ensure a well qualified
child welfare workforce. (Social Security Act, Title IV Grants to States for Aid and
Services to Needy Families with Children and for Child Welfare Services, Part B
Child and Family Services, 1967)
and, “..to provide, in appropriate cases, foster care and transitional independent living programs,”
(Social Security Act, Title IV Grants to States for Aid and Services to Needy Families with
Children and for Child Welfare Services, Part E Federal Payments for Foster Care and Adoption
Assistance, 1980).
Child welfare systems are complex because of the multiple levels of structure governing
and organizing the systems, multiple functions (crisis response and continued oversight of
families), numerous policy regulations that connect them, and due to the varying contexts in
which services are delivered. The federal child welfare system started in 1912 with the creation
of the United States Children’s Bureau (Child Welfare League of America, n.d.). Currently the
Administration for Children Youth and Families (ACYF) oversees public child welfare services

6

at the federal level (Child Welfare Information Gateway, 2013a). ACYF is part of the
Administration for Children and Families, located within the Department of Health and Human
Services. ACYF is divided into two bureaus, one of which is the Children’s Bureau. At the
federal level, Congress sets policy, which is monitored largely by the Children’s Bureau (see
Appendix A) for diagram of the federal structure). Federal funding and regulations guide state
level child welfare systems, which may be operated directly by the state or the state may oversee
child welfare services directly provided by the counties (McCroskey, 2001).
The systems also receive their funding from multiple levels of government. Within, and
across the different funding streams comes more variation. For example, federal funding for
child welfare has come from several sources including Title IV-E - Foster Care and Adoption
Assistance, Title IV-B - Child Welfare Services and Family Preservation, Title XX - Social
Services Block Grant, and Title XIX – Medicaid (McGowan, 2005; O’Neill Murray & Gesiriech,
2004), with each funding stream intended for the provision of different types of services,
targeting different sub populations, and requiring different types of reporting and oversight.
Child welfare systems must function within this milieu of policies and funding sources.
Details on the multilayered structure of child welfare systems, and the different types of
services provided, illustrate the challenges of changing practice in such a context, and illustrate
the complexities of the Los Angeles County Department of Children and Family Services. The
child welfare system service array tries to address the urgent needs of diverse families. Work
units designated as Emergency Response, Family Maintenance/Preservation, Family
Reunification, and Permanency Planning provide services in court ordered, voluntary, and
Alternative or Differential Response categories. Services are provided after a series of intricate,

7

and sometimes concurrent decisions are made as to what, if any maltreatment occurred, the
safety needs of the child, and how to best respond to the situation.
In 2009, one of the focal years of the study, LA County DCFS received 157,960 reports
2

of maltreatment and of those reports 134,220 (85%) were investigated (County of Los Angeles
Department of Children and Family Services, 2009). Emergency Response, Family
Maintenance/Preservation, Family Reunification, and Permanency services were required for
32,317 children. Some of these services were provided to children while they were in foster
care. During 2009, 15,816 children were in foster care. Services are expected to be delivered in
an efficient, yet competent manner, under shifting guidelines and cultural norms. Services are
also provided under constant scrutiny because the systems serve the most vulnerable children. It
is the urgent needs of these children, and their families, that creates another challenge for child
welfare practice change. System stakeholders must determine how they can create deliberate
change, and carefully implement change while cognizant that the practice change may not
ameliorate the dire consequences of maltreatment.
Consequences of Maltreatment
Child maltreatment has serious consequences in terms of physical, emotional and
cognitive development, and these effects may lead to long-term disadvantage. Yet the
consequences are amplified when the national scope of maltreatment is considered. In 2009, one
of the focal years of this study, six million children were subjects of reports of maltreatment
nationwide Further investigation of the reports found that 467,161 (23%) of the over two million
reports
3
were valid incidents, or were likely incidents
4
of abuse and/or neglect (U.S. Department

2
A report can include more than one child
3
A report can include more than one child
4
When a report of maltreatment is investigated the investigation may be inconclusive. This means that it could not
be determined if maltreatment occurred yet an occurrence of maltreatment cannot be ruled out. Researchers have

8

of Health & Human Services, 2009. Child maltreatment can be classified as a traumatic event
(Toth & Cicchetti, 2011). Traumatic events are when,
the person experienced, witnessed, or was confronted with an event or events that
involved actual or threatened death or serious injury, or a threat to the physical integrity
of self or others [and] the person's response involved intense fear, helplessness, or horror.
(American Psychiatric Association Diagnostic and Statistical Manual of Mental
Disorders, 1994, p. 427-428).
Exposure to traumatic events contributes to adverse outcomes for children (Toth & Cicchetti,
2011). In reviewing the knowledge about early experiences and brain architecture the National
Council on the Developing Child documented that for young children, “effects of threatening
environmental conditions can cause neural circuits to change in ways that prevent them from
functioning well, or at all, even in a subsequent healthy environment,” (2007, p. 2).
Maltreatment experienced early in life has also been found to be an independent risk factor for
both intentional and unintentional injury death among children five years old and younger
(Putnam Hornstein, 2011).
Some adverse outcomes experienced after childhood trauma endure into adulthood
(Corso, Edwards, Fang & Mercy, 2008). For example, a relationship has been found between
childhood experiences of maltreatment and loss of at least two years of life expectancy (Corso et
al, 2008). A retrospective study on childhood traumatic events, which included but were not
limited to child maltreatment, found that as the number of childhood traumatic events increased
the risks for life threatening health issues in adulthood increased (Felitti et al., 1998). A
longitudinal study on sexual abuse victims found differences between abused and non-abused

found that there may be not be significant differences in maltreatment experienced by children with evidence of
maltreatment and those without evidence (Fluke, Shusterman, Hollinshead & Yaun, 2008).

9

females, and these differences appeared at different developmental stages (Trickett, Noll, &
Putnam, 2011). Though there was some variability in the patterns observed, overall abused
participants were more likely to experience a host of biological (such as increased rates of
obesity and early pubertal onset), cognitive (reduced ability to think abstractly, solve problems,
and apply learning from experiences), mental health (depression and PTSD), and other adverse
outcomes such as teen motherhood, substance dependency, major illnesses, and lower levels of
educational attainment. The children born to mothers in the abused cohort were found to be at
increased risk for maltreatment due to repetitive patterns of maltreatment and familial
dysfunction. A study exploring the economic well-being of adults that experienced child
maltreatment found that the maltreated sample had lower levels of educational attainment, were
less likely to have a skilled job in young adulthood, and experienced more unemployment in
middle adulthood when compared to a demographically matched, non-maltreated sample (Currie
& Widom, 2010). Among the participants who reported their earnings, the maltreated sample
earned approximately $8,000 less a year than their non-maltreated counterparts.
Statement of the Problem: Assessing Implementation of Practice Change
Practice change in child welfare systems is a complicated undertaking and the
consequences of failure can be measured in children’s lives. When changing practice, child
welfare stakeholders typically focus on outcomes; however, there is a need to first focus on
implementation. Implementation can be defined as the, “…process of gaining targeted
organizational members’ appropriate and committed use of an innovation,” (Klein & Sorra,
1996, p.1055). Lack of attention to implementation leads to “process blindness,” (Morgan, 2005,
p. 16). When there is process blindness, there is a possibility that outcomes attributed to the
practice change are instead, attributable to implementation issues (Klein & Sorra, 1996).

10

Some decision makers are turning to research on evidence based practices (EBP) for
support. Evidence based practices are ‘‘interventions for which there is consistent scientific
evidence showing that they improve client outcomes’’ (Drake et al. 2001, p. 180). EBP research,
has helped draw attention to the issues surrounding implementation, and driven the growth of
literature focusing on implementation. In their review of implementation research literature EBP
researchers and scholars from the National Implementation Research Network (NIRN) arrived at
similar findings “...the challenges and complexities of implementation far outweigh the efforts of
developing the practices and programs themselves.” (Fixsen, Naoom, Blase, Friedman, &
Wallace, 2005). Though EBP are not yet the norm in child welfare, and only a small portion of
the implementation literature discusses child welfare settings or features research with
applications to human services, the literature provides a framework that can be used to analyze
the implementation of any child welfare practice (Aarons, Hurlburt & Horwitz, 2011; Fixsen,
Blase, Naoom, & Wallace, 2009).
Implementation science is the body of literature that provides direction for assessment of
implementation. Without this literature policies would be the only guide for implementation
assessment. Relying on policy materials is problematic because most policies do not reflect the
reality that each practice setting is different from the original conditions under which the change
was designed (Fixsen et al., 2005). Implementation science however, accounts for the setting
when recommending methods for assessment. Implementation science typically focuses on
change at a non-systems level. Though this study focuses on systemic change the literature was
useful because it is more developed than the systems change literature that pertains to human
service systems.

11

Contribution of the Study
This study aimed to increase the understanding of implementation of a child welfare
system practice change by focusing on the local level where careful implementation can make
the difference between a good idea that fails and one that improves outcomes for children and
families. Studies of child welfare practice change must give stakeholders enough information to
decide how they should assess implementation. The information must be in-depth and when
possible should focus on practice changes with longitudinal implementation. Implementation
does not occur on the intervention authorization date; rather implementation is thought to
progress over two to four years (Fixsen et al, 2009).
“Point of Engagement,” (POE) is the child welfare system practice change being studied.
Though POE is referred to as a singular practice change it is better conceptualized as a systemic
reform due to the way in which it changed the entire service delivery system and sought to
change the philosophy behind working with families. As a service delivery system POE strived
to address family needs at the first contact with an array of activities. As a philosophy POE
asserted that all families had strengths, all deserved to participate in decision-making about their
lives, and all communities had stakeholders that could help DCFS meet families’ needs (Marts,
Lee, McRoy & McCroskey, 2008). The perceptions of POE, as a service delivery system, and or
philosophy, differed among staff; that is one reason why implementation differed among offices.
POE was utilized within the Los Angeles County Department of Children and Family
Services (DCFS) between 2004 and 2013. In the late 1990’s concerned community members in
the southern portion of Los Angeles County voiced their outrage over the number of foster care
placements among African-American families (Marts, Lee, McRoy & McCroskey, 2008).

12

According to one of the main architects
5
of POE, there was a grand jury inquiry, and then in
1999, the County Board of Supervisors commissioned an audit of DCFS. The audit, conducted
by Price Waterhouse Cooper, revealed that the different sections of the office, known as the front
and back ends, were disconnected, and these silos extended beyond individual offices across the
entire department. For example some programs were functioning in obscurity and were being
underutilized. Price Waterhouse Cooper also found that DCFS was not mitigating risks for
removal, nor was it facilitating permanency as quickly as it should. An administrator was
charged with formulating a response to the audit, and the administrator started with the front-end
maltreatment investigation tasks as they were highlighted in the audit.
Committees were formed and the advice of direct service staff, known as Children’s
Social Workers (CSWs) was sought about the best approaches to connecting interoffice service
delivery tasks. These workgroups created concept papers which were later transformed into
components of POE (see Table 1). The resulting child welfare system practice change, (POE),
featured a redesigned service delivery process that was intended to facilitate engagement of
families at the front-end, particularly African-American families whose children were likely to
be placed in foster care, keep them engaged until they could be connected with services, and
continue to provide services needed to preserve families. POE, in both philosophy and practice
emphasized intervening at the earliest point possible. Early intervention meant leveraging
families’ strengths while providing services responsive to demonstrated safety needs. The
intervention goal was to prevent removal of children from their homes of origin. There were
different types of intervention, with each depending on strengths and safety needs. DCFS
involvement could start and end with a referral to a community-based organization; this type of

5
As reported during a 2008 study interview

13

intervention is known as Alternative or Differential Response
6
(Conley, 2007). The next type of
intervention required a family’s voluntary agreement to child welfare system oversight.
Meaning the family agreed to accept the services being coordinated by DCFS staff. When a
voluntary agreement could not be reached or was inappropriate due to safety issues the Juvenile
Court stepped in and mandated services. In these instances a Juvenile Court judge ordered
specific services that the parent(s) had to complete in order to maintain legal and physical
custody of their child or have their child return back home from foster care (See Appendix B for
Point of Engagement Process Flowchart illustrating the types of intervention).
Table 1

Major POE Components
7

Component Definition Purpose
Structured Decision Making An actuarial safety and risk
assessment tool
Standardize investigation
decision making across the
department

Team Decision Making

A collaborative meeting
between DCFS staff, families,
families support (teachers,
pastors, extended family, etc.),
and community based
organization representatives.

Engage the family in the
decision making and case
planning process, facilitates
strength based approach to
working with families, and
serves as an opportunity to
connect families to services

Concurrent Planning

A policy requiring that
conversations about
permanency occur when a
child is detained.

Prevent delay of permanency
conversations, and ensure
better preparation of
permanency plans ahead of
termination of parental rights

Bridging Staff

A new worker (Intensive
Services Worker) or new

This staff member is supposed
to provide an interoffice

6
This pathway was called Alternative Response in Los Angeles County but nationwide is more commonly referred
to as Differential Response. (see Conley, 2007 for more background on Differential Response).
7
For a full listing of POE components see Appendix C. Two of the components included in the table are prominent
in the study data (Team Decision Making and Bridging Staff).

14

responsibilities for an existing
worker.
connection between the front-
and back-end sections, the
worker connects families to
services as soon as possible.

Alternative/Differential
Response

A service pathway which
workers could utilize, if
referrals met specified criteria,
so that families could be
connected to services without
continued DCFS oversight of
the case.

Facilitate early DCFS
involvement/intervention and
fund services for children, and
their families when the child
was not detained.

POE provided a framework for connecting and leveraging multiple components, some of
which were in place ahead of official POE implementation, with pre-POE implemented
components varying by office. What set POE apart from previous systemic changes was that it
drew all the components together into a cohesive service delivery model and further, the model
was undergirded with a strengths based philosophy (McCroskey, Furman, & Yoo, 2007). The
strengths based focus was paradigm shift, particularly because POE was created in communities
with the highest needs. The model was meant to ensure that every office was using both pre-
POE and POE components collectively to impact service provision, and staff were no longer
acting alone, but across units, and with community-based organizations, and families to make
decisions. Staff was instructed to connect families to services quickly and though the connecting
staff differed among offices (Emergency Response, ISW, or FM/R) this required them to have
knowledge of and healthy relationships with the community-based organizations in their region.
One new POE component was additional staff serving as bridge between the front and back ends
(Intensive Services Worker/ISW), or additional responsibilities for existing staff that would serve
as the bridge worker. The bridging worker was designed to connect the front-end or Emergency
Response (ER) section responsible for maltreatment investigations, to the back-end or case

15

management section of regional offices. The back-end section was also referred to as the Family
Maintenance (FM) and/or the FM and Family Reunification (FR) section.
Offices were to be staffed at 120% before POE was implemented because the new case
flow added responsibilities to ER units. The amount of referrals ER staff had to investigate
under POE increased due to the use of the Alternative/Differential Response pathways and
increases in back-end referrals. Back-end referrals derived their title from the source of the
referring party. These referrals were generated for a family that was already being case managed
by a back-end DCFS staff member. In 2004, POE was piloted in two regional offices and further
implementation was staggered by office, with the last office added in 2007. POE policies were
deliberately left vague so that offices could customize them to the needs of their region, and this
led to differential implementation.
Implementation researchers have stated that future work should investigate change across
multiple sites to help capture the nuances of implementation and, when available, include
multiple respondents from different sections of each site to increase understanding of
implementation (Klein & Sorra, 1996). This study adheres to those recommendations. POE
was differentially implemented in 16 DCFS regional offices. The offices were the units of
analysis. Data from respondents in different positions, and working in different units was used
to assess implementation. DCFS regional offices were informative units of analysis because of
the diverse populations served. Diversity in implementation and service make lessons learned
applicable in some way to other jurisdictions nationwide. Studying DCFS provides critical
insight into implementation because the system is one of the nation’s largest child welfare
service providers due to the size and population of the county it serves (Fox, Frasch, & Berrick,
2000). Los Angeles County has over 9.9 million residents (U.S. Census Bureau, 2013a) living in

16

88 cities (County of Los Angeles, Cities, n.d.). Further, since DCFS is a state regulated and
county administered system, the previously described complexities around structure and
function, exist in the setting.
As mentioned practice change policies alone, especially vague ones are not sufficient
sources of information to use when determining how to assess implementation of the change. In
this study implementation science and practice change details were integrated to develop a study
specific conceptual model. The model had to be flexible enough to capture inevitable alterations
to the new practice, observed during implementation, and aspects of the practice environment
that were critical to implementation. The model guided assessment of implementation success,
and it also incorporated outcomes POE targeted. See Figure 1 for the conceptual model
summarizing the steps in the study.
One of the recommended next steps in furthering implementation science is research that
identifies indicators for implementation success, and then monitors the progress of
implementation on an ongoing basis, or at consistent intervals (Grol & Grimshaw, 2003). Study
specific indicators of implementation success were derived using an iterative process. First the
literature was explored to determine what impacted implementation. Subsequent to review, study
indicators were chosen, and then child welfare system practice change components were used to
operationalize indicators. Next, pertinent aspects of the environment that could influence
implementation success were identified.
Implementation science literature does not provide a consistent, empirically informed
definition of implementation success, nor has the literature definitively identified which
indicators of implementation success are most important. Typically implementation success is
viewed as a binary outcome; full implementation equals success, anything less indicates failure

17

(VanDeusen, Hall & Gerena-Melia, 2010). Practice change architects who have invested time,
and often personal capital, in creating the change would likely agree that their change should be
implemented in full, and anything short of full implementation would be equated to business as
usual, or failure. However, the measurement of implementation success that leads to outcome
decisions can be more nuanced. If measurement is defined as investigating whether or not
intended change is present (VanDeusen, Hall, & Gerena-Melia, 2010), then degrees of success
can be identified. In this study, implementation was measured in degrees of success.
Specifically a high, medium, and low degree of implementation success was operationalized
according to the number of indicators present. The study mines rich qualitative data gathered
from staff in each office during the latter part of the implementation period (2008-2009), and
then during a follow-up period (2012) to determine if the indicators of implementation success
were present or not, and to determine which offices had high implementation and should be
examined further to assess children’s safety and permanency outcomes. When analysis of 2008-
2009 staff level data revealed that there were two high implementation offices, newer (2012)
qualitative data was collected as a means of validating the presence of the indicators found in the
2008-2009 data. The newer data used the component of the practice change that was
implemented differently, the bridging or Intensive Services Worker, in the two offices as a
springboard for further data collection. Data collection focused on the function of the
differentially implemented component because it influences workers’ ability to interact with
families and the differential implementation helped to focus the respondents on POE versus other
changes happening in the office.
Lastly, office level data was reviewed. The data focused on the safety and permanency
trends for children served by the two offices with, according to staff, the highest degree of

18

implementation. The researcher looked for trends in the data, particularly any different trends
between the two offices. The last step spoke to the final goal of practice change, observing
whether the collective impact of the new practice, and all of the other existing efforts in the child
welfare system context, translated into positive trends.
Research Questions
The following overarching questions were answered in the study:
1. What organizational and actor level indicators of implementation success were present in
each office?
2. Did the environment influence implementation success? If so, was it a barrier or
facilitator for success?
3. What were the safety and permanency outcomes for offices with the highest degree of
implementation, and where the environment influenced implementation success?
The synthesis of literature on child welfare systems and implementation research
provides a frame for the study’s questions and it also illustrates the sources of complexity that
make child welfare system practice change challenging. Specifically, the brief historical
overview of public child welfare systems highlights the crisis-oriented, reactive, and somewhat
piecemeal way in which first private organizations and later government became involved in
child safety issues. The sections on types of responses and responding units, along with
demographic patterns of those served, provide information on the multiple types of child welfare
system services and the varied populations who are the main recipients of these services; these
sections provide context for the structures and functions of the Los Angeles County Department
of Children and Family Services. Literature on previously studied implementation factors
informed the decision-making on which factors were most suitable to use in this study. The

19

resulting indicators of implementation success are featured in the conceptual model.
Operationalization of concepts used for assessment of implementation success is described in the
methods chapter.
In summary, child welfare systems serve vulnerable yet varied populations in a complex
and ever-changing context. Service changes are frequently implemented in response to reports
of child deaths, scrutiny by media or elected officials, audits or analyses by outside groups, as in
the case of POE. Though Implementation Science has helped focus the research, there is still
little known about the process surrounding social services delivery (Elliott & Mihalic, 2004),
particularly in the area of child protection services. Even less is known about assessing social
service delivery processes. The purpose of the current study is to add to existing knowledge
through a study of a system level practice change implemented in a large, urban, county operated
child welfare system serving a diverse population. Using Los Angeles County as the site of the
study, and DCFS regional offices as the unit of analysis, this study provides information on
differential implementation of POE. Because the practice change was in place for some time,
there is also some data available to assess progress of over time. Findings may be useful for
change agents, other stakeholders, and researchers planning and studying child welfare system
practice change efforts.
Organization of Dissertation
There are six chapters in the dissertation. Chapter 1 provides the introduction, statement
of the problem, the contribution of the study, and brief description of the chapters to follow.
Chapter 2 reviews literature describing the history, structures, and functions of child welfare
systems, and literature on indicators of implementation success. Chapter 3 lists the research aims
and questions, and presents methods used to answer the research questions. Chapter 4 and 5

20

includes study results. Chapter 6 the final chapter, describes the conclusions of the study and
implications, while also noting its limitations, and providing suggestions for future practice and
research.

21

Chapter Two: Review of Literature Describing the Child Welfare System and Indicators of
Implementation Success
Child Welfare Systems
This section provides an overview of the child welfare system. Review of literature on
the history, structure, and functions of the system, and system regulations facilitates
understanding of why and how the child welfare system is so complex. The review also
illustrates aspects of the system involved in large scale change, and illustrates typical ways in
which a family may encounter the system. Provided details describe the system being studied,
Los Angeles County Department of Children and Family Services. Details will help consumers
determine how the system may or may not be similar to other child welfare systems, and
determine how to apply study implications.
History. Private organizations and subsequently the government were not proactive
about child safety issues. Rather they acted when the issues were thrust upon them. The case of
Mary Ellen Wilson is often credited with starting a chain of events that led to private
organizations responding to maltreatment, and attempting to care for child victims (Watkins,
1990). Mary Ellen was placed with a couple that mistreated her, and in 1874, the nine year old’s
condition came to the attention of Etta Wheeler, a neighborhood missionary. Mrs. Wheeler
explained Mary Ellen’s plight to Henry Bergh, founder of the American Society for Prevention
of Cruelty to Animals. Mr. Bergh tasked his lawyer with finding a legal mechanism for
removing Mary Ellen from her guardians, and the lawyer succeeded. However, Mr. Bergh was
not finished.
Mr. Bergh and his lawyer created the New York Society for the Prevention of Cruelty to
Children (NYSPCC; New York Society for Prevention of Cruelty to Children, 2000). NYSPCC

22

became the first private agency whose mission was child protection, and it was an important
catalyst for development of similar agencies in other cities. States took their cues from private
organizations such as the NYSPCC and began coordinating services, mostly for orphans. The
creation of the first juvenile court, in Chicago 1899 (Myers, 2011), was another important
development in the government’s oversight of child safety. Juvenile courts were given
jurisdiction over child maltreatment cases, and that purview is intact today. Well into the 1900’s
private agencies continued to lead child protection efforts, and government participation stayed
largely at the state level.
Government involvement was graduated, though not necessarily because leaders were
using a comprehensive strategic plan to guide involvement. A nurse, a sociologist, and child
labor advocate conceptualized the federal Children’s Bureau in 1903 (Bradbury & Eliot, 1956).
The idea was discussed at the first White House Conference on the Care of Dependent Children
convened by President Theodore Roosevelt in 1909. In 1912, the Children’s Bureau legislation
passed and its original mission was to provide a federal response to infant mortality,
overcrowded orphanages, child labor, and juvenile crime. The significance of the Bureau went
beyond the scope of its first mission. Creation of the Bureau signified that the federal
government had a responsibility to ensure the welfare of all children (McGowan, 2005). As the
government’s willingness to intervene on behalf of vulnerable children increased, the Children’s
Bureau transitioned from an informative federal department into a national oversight and funding
entity for child welfare services in every state.
Several important pieces of legislation facilitated the transition. The Sheppard Towner
Act, enacted in 1921, gave the Bureau authority for dispensing and regulating maternal and child
health grants to states (McGowan, 2005). This created an early structure that would be

23

crystallized with the passage of the Social Security Act, signed into law in 1935. Title IV
created The Aid to Dependent Children Program, and Title V, Part 3 created the Child Welfare
Services Program. Aid to Dependent Children introduced the concept of federal matching funds.
Under matching funds states were provided federal funds equal to the amounts they spent on
services, in this case pensions for fatherless families. The Children’s Bureau again had
administrative authority over the aid. Title V essentially required each state to create a child
welfare system to coordinate delivery of service, serving not only children cared for by their
families of origin but also children in what would come to be known as foster care. States that
did not have child welfare programs had to create them (Bradury & Eliot, 1956).
The 1961 amendment to the Social Security Act added a foster care component to The
Aid to Families with Dependent Children Program (O’Neill Murray & Gesiriech, 2004). With
the amendment, the Children’s Bureau expanded its financial reach into foster care. States
received matching funds for foster children, adding to the matching funds authorized under the
1935 legislation, allowing expenditures when children remained in their homes. Finally
government involvement in child welfare increased with the 1974 Child Abuse Prevention and
Treatment Act; this act required state level laws mandating report of child maltreatment.
Multiple levels of regulation. The Children’s Bureau funds and regulates state-level
child welfare systems and states in turn may fund and regulate multiple county-level child
welfare systems (McCroskey, 2001). As discussed above, the relationship of the Federal
Children’s Bureau relationship with state Departments of Social Services is operationalized
largely through federal and state legislation. States determine the role and functions of local
jurisdictions through state legislation (see Figure 3). Legislation that is passed and then signed
by the President or a Governor defines the federal or state child welfare system and provides

24

guidelines on functions, requirements, and any funding restrictions. Lower level systems (states
or counties) then decide how to respond to the new legislation, and the higher-level system
(federal or state) monitors responses of the lower level system (see Figure 3).
Though Figure 3 is linear, the Children’s Bureau, state, and county level child welfare
systems are joined in interdependent hierarchies with each level creating and adhering to its own
structures and functions (Kreger et al, 2007). Thus, the daily reality within which the state and
county child welfare systems function is more like concentric Figure 2. These interdependent
relationships are part of what makes changing and studying child welfare systems, particularly at
a local level, challenging. Defining boundaries is problematic because the boundaries are
permeable and changes in one system have a ripple effect. The relational complexity and
boundary issues are compounded by the constant evolution of the systems. For instance,
definitions of maltreatment change not only within the federal, state, and local child welfare
systems but also in adjacent systems such as the legal and law enforcement systems.

25

Figure 3. Regulatory Authority
Adapted from diagram in How federal legislation impacts child welfare service delivery. (Child
Welfare Information Gateway, 2012)

Step 1: Federal or state legislature
passes and President or Governor
signs legislation creating or amending
funding for a child welfare services
Step 3: The Children’s Bureau or state
distributes funds to lower level
systems as authorized by legislation
Step 4: Further statues are enacted by
lower level systems in response to
higher-level legislation, new services
are implemented or reforms are
implemented
Step 5: The Children’s Bureau or state
monitors the lower level system’s
services through data collection &
reviews
Step 6: The Children’s Bureau or state
responds to the legislature’s requests
for performance outcomes
Step 2: The Children’s Bureau or the
state provides guidance to lower level
systems on legislative mandates

26

Child welfare systems structures and functions. All three branches of government,
executive, legislative, and judicial, shape the federal child welfare system. Congress introduces
and passes child welfare legislation, the president either signs the bills into law or rejects the
legislation, and independent administrative agencies oversee implementation of legislation. The
Children’s Bureau is the primary operational administrative arm for child welfare and it oversees
services and research, develops guidelines for funding distribution and oversees legislative and
executive mandates related to child maltreatment (Child Welfare Information Gateway, 2013a).
State level systems supplement Federal legislation and funding, and local child welfare
jurisdictions, which may be run by either state or county government, coordinate responses to
reports of child maltreatment, assess the need for and provide emergency response, family
maintenance, family reunification and other services. At all levels, child welfare systems are
designed to promote the safety, permanency and well-being of children and their families. The
question of child safety directs stakeholders to determine whether a child is protected from initial
or recurrent maltreatment and whether the child can remain in the home without being harmed
(U.S. Department of Health & Human Services, n.d.). Permanency refers to the stabilization of
the child’s living situation such that family ties are maintained or an alternative and permanent
family is found when the family of origin cannot care for the child. Guidelines recommend that
at minimum, safety and permanency services should meet children’s educational, physical and
mental health needs. However, Bryan Samuels, former Commissioner of the Administration on
Children, Youth, and Families, re-emphasized that safety and permanency services are not
enough (Samuels, 2012). The memo, created in response to new research on child development,
stated well-being should be a priority of child welfare systems, and called for well-being services
to enhance the family’s capacity to promote their child’s optimal development.

27

Levels of intervention: Court ordered, voluntary, and alternative/differential response.
The type of child welfare system intervention subsequent to a report of any type of maltreatment
depends on three things, (1) the severity of the reported maltreatment, (2) the imminent, and
continuing risk of danger for the child or children in the situation, and (3) the availability of
services that the child, and or his or her family might need (Child Welfare Information Gateway,
2013a). Intervention can occur through juvenile court ordered actions or through voluntary
commitments between the child welfare system and families. Court supported intervention
occurs when it is determined that the court’s authority is required to keep a child safe. The judge
provides final decisions in interventions supervised by the court. If a child welfare system
representative offers to work with a family on a voluntary basis then both the system
representative and the family have decision-making power. If the family refuses to voluntarily
work with the child welfare system representative, the representative may decide that court
intervention is necessary. Traditionally, court ordered or voluntary interventions are offered
after an investigation into an allegation of child maltreatment. Increasingly, state and county
child welfare systems are using a pathway known as differential response. Under differential
response, child welfare systems typically take more of an assessment versus an investigatory
approach in determining if maltreatment occurred and services are offered to a family on a
voluntary basis (Child Welfare Information Gateway, 2008b).
Front and back end units: Emergency response, family maintenance/preservation,
family reunification, and permanency planning. Different components, or units, of state and
county level child welfare systems provide a continuum of services from investigation of
maltreatment allegations to case management of voluntary and court ordered cases. Each state or
county system may structure its units how it sees fit; meaning the following list of units may not

28

be as distinct as described. In California, the units are Emergency Response, where the focus is
investigation of threats to child safety, Family Maintenance, which focuses on case management
when children can be left in the home, Family Reunification, which focuses on case management
when children are removed from the home and can be safely returned to their parents, and
Permanency Planning, which focuses on finding alternative adoptive or guardianship homes for
children who cannot be safely returned home (Fox, Frasch, & Berrick, 2000). Below is a
snapshot of services provided by DCFS units in 2009, one of the focal years of this study,
starting with the number of referrals received (County of Los Angeles Department of Children
and Family Services, 2009).

Figure 4. Services Provided by DCFS Units in 2009
Emergency response
8
. Employees in Emergency Response units deliver safety-focused
investigation and intervention services 24 hours a day, 7 days a week (Inter-Agency Council on
Child Abuse and Neglect, 2012). These services start when someone reports suspected

8
Figure 5 – Services occur at point of box stating, “CPS Investigates”
Of those investigated, in and out of home services provided to 32,317
cases
882 Received
Continuing Emergency
Response Services
10,847 Received Family
Maintenance Services

8,883 Received Family
Reunification Services
11,705 Received
Permanecy Services
157,960 Referrals Made Due to Suspected Child Abuse or Neglect
23,740 Screened Out 134,220 Investigated

29

maltreatment to a child welfare system (CWS), typically through a hotline (see Figure 5). As
featured in Figure 4, 157,960 referrals, or reports alleging maltreatment were made to LA
County DCFS in 2009. When a referral is made the CWS representative uses the state or
county’s legal definition of maltreatment in deciding whether to further investigate or screen out
the allegations. In some jurisdictions, such as DCFS, hotline workers may do an initial screening
of the allegations and then pass allegations onto Emergency Response units for further
investigation. In 2009, approximately 85%, 134,220 referrals , received further investigation.
Once an investigation is complete, a disposition is recorded.
Each state and county child welfare system has its own disposition categories, but the
following categories are widely utilized. The allegation can be not founded, or unfounded,
meaning the Emergency Response investigator found no evidence to support the allegation
(Child Welfare Information Gateway, 2013a). When an investigation is unfounded, no further
intervention is made. An allegation can be indicated or inconclusive, meaning the investigator
found some, but not enough evidence to support the allegation (Goldman, Salus, Wolcott,
Kennedy, 2003).

30

Figure 5. Child Welfare System Decision Points
Originally titled, “Child Welfare System” and featured in How the child welfare system works. Child
Welfare Information Gateway. (2013a). Washington, DC: U.S. Department of Health and Human
Services, Children’s Bureau. Grey boxes represent decision end points.

31

Another way of characterizing inconclusive investigations is noting that while there is
some amount of risk to the child’s safety, the evidence does not provide a legal mechanism for
child welfare system intervention. When an investigation is inconclusive, a family may be
offered services on a voluntary basis. Finally, an investigation may be founded or substantiated,
meaning that the evidence produced during the investigation supports some aspect of a state or
county’s legal definition of maltreatment. When allegations are substantiated, further
intervention is provided on a voluntary or court ordered basis. After maltreatment is
substantiated, the service focus shifts from safety to either case management or permanency.
Family maintenance/preservation
9
. When a referral is substantiated the referral
becomes a case (Inter-Agency Council on Child Abuse and Neglect, 2012). Family Maintenance
and Preservation units manage the cases in which children are allowed to remain in their home of
origin during child welfare system intervention (Fox, Frasch, & Berrick, 2000). The goal of time
limited in-home services is to maintain the family and prevent out of home placement (Inter-
Agency Council on Child Abuse and Neglect, 2012; U.S.D.H.H.S, 2012a). Case managers
coordinate and ensure completion of services, and services typically target caregivers (Fox,
Frasch, & Berrick, 2000). For example, parenting classes are often a core component of family
maintenance and preservation plans. In 2009 Family Maintenance services were provided for
10,847 Los Angeles County DCFS cases (Inter-Agency Council on Child Abuse and Neglect,
2012). Family Reunification units oversee out-of-home care services (Fox, Frasch, & Berrick,
2000).
Family reunification
10
. Family Reunification units provide services when safety issues
prevent a child from staying within his or her home (Inter-Agency Council on Child Abuse and

9
Figure 5 – Services occur at point of box stating “Child stays with the family, services are provided”
10
Figure 5 – Services occur at point of box stating “Child is placed in out of home care”

32

Neglect, 2012). The service goal is to reunify the family. In 2009, 8,883 DCFS cases received
Family Reunification services (County of Los Angeles Department of Children and Family
Services, 2009). Reunification services are time limited; under court ordered intervention
caregivers have twelve months to make sufficient progress towards case plan goals, though
deadlines can be extended to eighteen months (Fox, Frasch, & Berrick, 2000). Family
reunification services provided on a voluntary basis cannot extend beyond six months. Family
Reunification workers do not wait until service deadlines to begin looking for alternative
families. Child welfare systems utilize Concurrent Planning, which involves simultaneously
developing plans for reunification and plans for alternative legal permanency options such as
adoption or guardianship.
Out of home service is the most intensive type of service provided because it involves 24-
hour substitute care, or foster care (U.S. Department of Health & Human Services, 2012a). Out
of home placements include non-relative foster family homes, therapeutic or medical foster
homes, relative homes known as kin placement, group homes or congregate care, and pre-
adoptive homes (U.S.D.H.H.S, 2012a; Mallon, & McCartt Hess, 2005). Children may move
between these settings during their tenure in foster care, and they exit care when they either
reunify with their parents or caregivers, or they age out of the system.
Permanency planning
11
. When it is determined that it is unlikely for a child to return to
his or her family of origin, the case is transferred to the Permanency Planning unit (Fox, Frasch,
& Berrick, 2000; Inter-Agency Council on Child Abuse and Neglect, 2012). In 2009
Permanency services were provided to 11,705 DCFS cases (County of Los Angeles Department
of Children and Family Services, 2009). The first task of a Permanency Planning worker is to
assess the feasibility of adoption. One criteria for adoption is termination of parental rights,

11
Figure 5 – Services occur at point of box stating “Termination of parental rights”

33

meaning the parents no longer have legal custody over their child. If a parent’s rights are still
intact or there is some other reason preventing adoption, legal guardianship is the next option.
A legal guardian has the authority to make decisions for the child, in the same manner the
child’s caregivers or parents did, but the guardian is not financially responsible for the child, and
the child is not a legal member of the guardian’s family (Fox, Frasch, & Berrick, 2000). In some
systems, once a child obtains a legal guardian the child stops receiving government financial
support. There is growing recognition that subsidized guardianship programs, like California’s
Kin-GAP, are needed (Generations United, 2007). These subsidized programs provide continued
financial support, through government services and or stipends, for children with legal guardians.
If adoption or legal guardianship is not possible, a child will remain in foster care until he or she
ages out of the system (Fox, Frasch, & Berrick, 2000), generally at age 18, though new
legislation authorizes funding for youth remaining in foster care after age 18
12
. Aging out of the
system is called emancipation.
Implementing Child Welfare System Practice Change
While national legislation tends to bring about “sea changes” or major shifts in how child
welfare system services are delivered, state or county level practice changes may or may not
diffuse beyond the edges of the specific region. Legislative and other changes have been
propelled by specific tragedies, sparked by leadership turnover, or created in response to fiscal
issues (Aarons, Hurlburt, & McCue Horwitz, 2011). Practice changes are rarely implemented in
isolation and the overlapping of change efforts often creates competition for attention and
resources. In this way challenges in implementing child welfare system practice changes are
similar to all change or innovation implementation challenges. However, the multiple layers of

12
See https://www.childwelfare.gov/fosteringconnections/ for information on the Fostering Connections to Success
and Increasing Adoptions Act of 2008 (P.L. 110-351)

34

regulations, structures and functions of the systems amplify these challenges. Additionally, child
welfare systems are human service systems and diverse individuals deliver services to unique
constituents (Aarons, Hurlburt, & McCue Horwitz, 2011) in new situations.
Stakeholders, therefore, are turning to a growing body of research on implementation of
human service system practice changes, and extant organizational literature on practice changes
to help guide implementation. Growth in the research on implementation of human service
practice changes, or implementation science, has been furthered by increasing development and
diffusion of evidence based practices (EBP). This study draws from findings, and implications,
in the literature that pinpoint factors that may affect implementation of non-EBP practice
changes. In the next sections, factors affecting implementation of change, as cited in research,
are discussed. The factors are discussed to facilitate an understanding of the research landscape.
Factors pertinent to the study are identified, and an explanation is provided of how
implementation success will be assessed. Finally, the study’s conceptual model is presented.
What impacts implementation of practice changes? Though there are some
inconsistencies implementation research has found that organizational and actor level
characteristics, as well as environmental influences, should be included in assessment of
implementation (Henggeler et al., 2008; Rye & Kimberly, 2007). There is less research to
provide guidance on influences on implementation, such as the importance of the characteristics
of the practice change (Rye & Kimberly, 2007). Organizational characteristics include attributes
such as resources, structure, size, culture, and climate for learning (Rye & Kimberly, 2007).
Actors are those persons that have direct contact with service recipients and are usually
responsible for on the ground implementation of changes (Chaudoir, Dugan, Barr, 2013). Actor
level characteristics include values, and or perceptions.

35

Organizational influences. Organizational literature provides thorough discussion of
structures, and one important part of structure for human services systems, and to
implementation of changes in these systems, is the degree of decentralization. Decentralization
is the sharing of power across all divisions or departments of an organization (Kolehmainen-
Aitken, 2001). Decentralization is seen as a barrier, though not an insurmountable one, to
implementation (Aarons, Hurlburt & McCue Horwitz, 2011) because in decentralized
organizations the power to implement a practice change is diffused. In decentralized
organizations, actor level aspects, like champions, become more important. However,
decentralized organizations also can encourage multi-level decision making, which is thought to
facilitate implementation (Durlak & DuPre, 2008).
Organizational culture also influences practice change implementation. Not to be
confused with climate, an actor level issue, culture is the pervasive or shared assumptions and
behavioral norms within an organizational unit (Glisson & James, 2002). If an organization has
a culture of learning, then there is fertile ground or organizational support for implementation of
changes (Aarons & Swatizky, 2006; Aarons, Sommerfeld & Walrath-Greene, 2009; Durlak &
DuPre, 2008; Klein & Knight, 2005). Defensive organizational cultures tend to promote beliefs
that the status quo must be protected. Implementation of change in defensive cultures is much
harder than implementation in learning cultures.
An organization’s financial resources and its ability to support implementation through
training are two other aspects important for successful implementation (Durlak & DuPre, 2008;
Klein & Knight, 2005). Sometimes these two aspects are interwoven. Since practice change
means a new way of providing services, training is needed, and training must be funded or
otherwise fiscally absorbed (Flynn & Brown, 2008; Gotham 2004; Substance Abuse and Mental

36

Health Services Administration, 2007a; SAMHSA, 2007b). Other fiscally intensive ingredients
for change include additional employees, replacing employees, or moving employees from one
unit to another (Flynn & Brown, 2008; SAMHSA, 2007a; SAMHSA, 2007b). Implementation
may be compromised if organizational resources are not sufficient (Durlak & DuPre, 2008;
Gotham, 2006; Klein & Knight, 2005; Osilla, Hepner, Munoz, Woo, & Watkins, 2009;
SAMHSA 2007a; SAMHSA, 2007b).
Actor level influences. Actor level issues, such as an individual’s perception of the
practice change, and managerial and street level bureaucrat collaboration in the change decision-
making and implementation process may also affect implementation. Climate is a measure of an
individual’s perception of change. Collective perceptions, to the degree that perceptions are
cohesive, create an organizational climate (Glisson & James, 2002; Klein & Sorra, 1996).
Organizational units that have high degrees of collaboration or teamwork may have greater
amounts of shared perceptions. The synergy created by collaboration, may increase the
effectiveness of the change process by producing results that individual parties could not achieve
in isolation (Longoria, 2005). A positive organizational climate, where staff feel implementation
is feasible, supports implementation (Klein & Knight, 2005, Osilla et al., 2009) by increasing the
probability of actors utilizing the practice change (Henggeler et al., 2008). Direct services
employees, or street-level bureaucrats, are important collaborators because of their ability to
facilitate or impede practice changes (Lipsky, 1971). When changes are being planned,
managerial staff should obtain the buy-in, or support, of direct services employees (Lipsky,
1971, Osilla et al., 2009; Osland, 2004), and continue the collaboration, keeping front-line staff
involved in the entire change process. Planning and implementing change gives actors a shared

37

task to engage in, and this shared task facilitates development of common goals and relationships
built on trust (McCroskey, 2003).
Even if implementers and service providers are collaborating, a practice change
champion may still be needed. Champions are people who communicate the importance of the
practice change to others and accumulate buy-in, caring for the practice change almost as a
parent cares for a child (Chakrabarti, 1974) through dedicated attention (Durlak & DuPre, 2008;
Pinnock et al., 2009). Champions have been found to be particularly useful for practice change
implementation in large organizations (Chakrabarti, 1974) where constant changes limit
employees’ attention. The champion can be a manager, but does not have to be. Managers that
were not champions provided important leadership through their support for the change, and by
showing patience during the implementation process (Durlak & Dupre, 2008; Klein & Knight,
2005; Osilla et al., 2009). Conversely, lack of leadership support has been cited as a barrier to
implementation (Fox, Gottfredson, Kumpfer & Beatty, 2004).
Environmental influences. Implementation research suggests that the practice
environment is also influential (Harn, Parisi, Stoolmiller, 2013). For example, external
partnerships have been cited as catalysts for practice change implementation (Durlak & DuPre,
2008). Literature suggests that an important aspect of the child welfare environment is the
relationship between the community and the focal child welfare organization (McCroskey,
2001). The history, culture, and extent of child welfare services within the communities served
influence viewpoints on how to best protect children. For instance, if a disproportionate number
of children of color have been removed from their homes and placed in foster care, the
community may distrust the child welfare agency (Detlaff & Rycraft, 2008), which may lead to
distrusting any efforts the agency makes at change. Change agents must remember that

38

communities include not only residents but also the private agencies that partner with child
welfare systems in service delivery (McCroskey, 2001). The most important environmental
factors to consider are those specific to the unique setting where the practice change is being
implemented. In this study, the implemented practice change was Point of Engagement (POE)
and the communities served by the Los Angeles County Department of Children and Family
Services’ regional offices were the environments. Community relationships and community
needs were factors that could affect operationalization of POE.
Study specific assessment of implementation. Research has identified organizational
actor, and environmental, factors impacting implementation. Organizational factors include
structure, financial and workforce resources, culture, and size. Actor level factors include multi-
level decision-making, collaborative implementation, stakeholders’ perceptions of the practice
change or climate, identified champions, or managerial support of the change being
implemented. Practice change environment factors, such as external partnerships, and child
welfare specific factors such as the community and agency relationship also impact
implementation. Environmental factors unique to the implementation setting, such as community
needs, also require consideration. Though consensus is building about the organizational, actor,
and environmental level variables that influence implementation, causal links between these
variables and implementation outcomes have not been established (Aarons, Hurlburt, & McCue
Horwitz, 2011). Since empirical guidance is still being developed the components of practice
change should be used to prioritize variables. For POE the major components were informed by
philosophy, and operationalized in service delivery. Components included:

39

 Having sufficient front-end staff, including Emergency Response staff and a dedicated staff
member that provides an interoffice connection between the front and back end sections, and
also connects families to services as soon as possible
 Engagement of all pertinent DCFS staff, community based organization representatives, and
the family in a collaborative decision-making and case planning process.
 Intervening as early as possible with appropriate services.

Assessing indicators of implementation success. POE increased the workload for front-
end staff, (ER and bridging staff). One criterion for successful implementation was sufficient
staff resources. Additional staff was needed to handle the increased workload. POE also
required collaboration on multiple levels.
Specifically,
POE requires full disclosure to and collaboration with families. Workers must
communicate openly and respect the family’s ability to make decisions on their own
behalf. This kind of communication is essential in order to assess family strengths and
develop individualized service plans. (McCroskey, 2008, p.9).
The collaboration that was intended to occur under POE can be defined as a relational system
where stakeholders (DCFS staff across units, families, and community-based organization staff)
work together and combine resources for the purposes of meeting agreed upon objectives
(Graham & Barter, 1999; Weinstein, Whittington & Leiba, 2003). The collaboration is critical
because the stakeholders would be unable to achieve the desired objectives on their own
(Graham & Barter, 1999). Through collaboration DCFS staff, particularly front-end workers,
engaged the family. Engagement facilitated the family’s investment in services (DePanfilis &
Salus, 2003). Collaboration facilitates formation of relationships, and these relationships

40

promote change, or results, as illustrated in the Collaboration Loop (see Figure 6). There is also
a feedback process shown in the Loop wherein change results further build relationships.

Figure 6. The Collaboration Loop - Adapted from Duke Corporate Education (2005) p. 2

POE’s philosophical shift supported intervening at the earliest point possible with
appropriate services. Earlier interventions were designed to prevent removal of children from
their home. Focusing on family strengths, instead of on risks, changed interactions. Designers
recognized that the strengths based perspective could help in the formation of positive
partnerships with families (DePanfilis & Salus, 2003). During POE training staff were instructed
to use identified strengths as a starting point for interactions (Simpson, 2006). It was the
worker’s job to discover a family’s capacity and then actively plan services to enhance any
capacities. The philosophy sparked action among workers who believed that people, in the case
of POE parents and/or caretakers, can be agents of change (Kubisch, Auspos, Brown, Chaskin,
Fulbright-Anderson & Hamilton, 2002). POE needed the support of a leader or champion for
successful implementation, particularly since DCFS was and is decentralized at multiple levels.
Results
Relationships
Build
Enable

41

DCFS delivers services through regional offices. Within regional offices there is further
decentralization by work units. Each work unit delivers child welfare services at a specific point
in time on the service continuum (e.g. investigation, case management, permanency planning).
Pre-POE silos within offices were cited as hampering service provision. Meaning staff from the
same office but different units, such as Emergency Response and Family Maintenance units, did
not work together. Further decentralization occurred when units were housed in different
buildings; a few offices were geographically dispersed.
Assessing environmental influences. Implementation research suggests that
stakeholders need to understand the community context of change, meaning the ethnic or
cultural, economic, and environmental factors that preceded and exist in time with the practice
change (Kreger et al, 2007). There were two aspects of the environment with potential to
influence implementation success. The two aspects that could act as barriers or facilitators were
historical relationships between DCFS office and community-based organization staff, and
community needs. Historical relationships with community-based organizations were assessed
because POE was designed to speed up connection to services. DCFS staff had to rely on
community-based organizations, such as mental health agencies, to provide services
(McCroskey, Furman, & Yoo, 2007). If DCFS staff had a strong pre-POE relationships with
staff from community organizations, then collaborating for the purposes of early service
provision was more likely to be successful. Conversely, if DCFS staff had seek out and create
relationships collaboration was less likely to occur.
Community needs were considered important because areas whose residents experienced
more multifaceted and chronic problems, problems beyond maltreatment, may require
specialized implementation. The researcher is not suggesting that practice change is impossible

42

in high need environments. POE was started in perhaps the highest need community of all, and
it is in these types of environments where practice change can make a critical impact. Rather the
researcher was interested in the depth of community needs across the County, and the way that
practice may have been affected by the presence or absence of needs. Data on level of poverty,
unemployment, levels of education, and the extent to which grandparents were financially
responsible for meeting the basic needs of their grandchildren were reviewed because poverty,
unemployment, and low levels of education have been cited in previous research as barriers to
strengths based work (such as POE) (McCroskey et al, 2009a), and poverty, which may be
triggered by unemployment and low levels of education is frequently cited as contributing to
conditions that heighten the risk for child neglect (Shook Slack, Holl, McDaniel, Yoo, & Bolger,
2004). Further, parents with low levels of education, defined as below a high school education,
have less economic stability (National Center for Children in Poverty, 2007)
The number of grandparents responsible for caregiving has steadily increased over the
last few decades (Hayslip & Kaminiski, 2005; Smith & Palmieri, 2007). One study found that
children cared for by grandparents may have more psychological issues than children reared by
other types of caregivers (Smith & Palmieri, 2007). In addition grandparents can be stressed by
the caregiving situation, and their heightened stress can exacerbate children’s issues. Another
study found that grandparents experience as caregivers differ depending on their socioeconomic
resources, the parenting arrangement (co-parent or custodial), and ethnicity (Goodman &
Silverstein, 2006). The final aspect of needs examined was level of DCFS services. The focus
was on three types of services that account for most contact with families, investigation of
alleged maltreatment, known as referrals, removal of a child from his or her home of origin, and
placing that child in out-of-home or foster care, and oversight of a child when he or she is placed

43

in out-of-home care. Higher service needs may overwhelm an office’s resources, creating a
ripple effect that impedes other aspects of implementation.
As discussed implementation science literature does not provide a consistent, empirically
informed definition of implementation success. Most practitioners and change agents view
implementation as having a binary outcome; full implementation equals success, anything less
indicates failure (VanDeusen, Hall & Gerena-Melia, 2010). Those investigating whether or not
intended change is present can assess the degree of implementation success. In this study,
implementation success was measured in terms of degrees of success. Specifically a high,
medium, and low degree of implementation success was operationalized. Offices with five or six
indicators present were characterized as high implementation, offices with three to four
indicators present were characterized as medium implementation offices. Offices with one or
two indicators present were characterized as low implementation offices. If the binary outcome
of implementation success was superimposed onto degrees of success implementation failure
would occur in offices with one to two indicators, partial success would occur in offices with
three to five indicators, and success would occur in offices with all six indicators.
Assessing trends in POE targeted outcomes. The final step in assessing implementation
was to observe trends in POE targeted safety and permanency outcomes for offices with the
highest degree of implementation, i.e. offices with at least five indicators present, and where the
environment influenced implementation success. The researcher chose to study only those
offices with at least five indicators present under the premise that with this level of
implementation an office would be expected to sufficiently reach the desired level of change and
could be examined under the binary definition of success.

44

The safety and permanency outcomes observed in the study were consistent with aims the
DCFS had been targeting for several years. Outcomes included:
 Improved safety: significantly reducing the recurrence rate of abuse or neglect for
children investigated and reduce the rate of abuse in foster care.
 Reduced reliance on detention: reduce reliance on removing children from their
homes through expansion of alternative community-based strategies to help
families.
 Improved permanence: shortening the timelines for permanency for children
removed from their families with a particular emphasis on reunification, kinship
and adoption. This also includes reductions in the transitioning (emancipation)
population (Los Angeles County Department of Children and Family Services,
2009, p.51).
Safety refers to, “the absence of an imminent or immediate threat of moderate-to-serious harm to
the child,” (Goldman, Salus, Wolcott, & Kennedy, 2003, Glossary S). Permanency refers to the
stabilization of the child’s living situation and the maintenance of family ties such that every
child has, “…a legally permanent nurturing family,” (Child Welfare Gateway, n.d.). Through
resource adjustments, collaboration, and intervening as early as possible POE aimed to reduce
the number of removals, or detentions, of children from their home. Removals,
...take away care, custody and control of a dependent child or ward from the child’s
parent or guardian, and places the care, custody, and control of the child with the court,
under the supervision of the agency responsible for the administration of child welfare
(Los Angeles County DCFS, 2009, p.161).

45

If children were removed, POE components were intended facilitate a child’s quick
return home, or reunification, with his or her parents. Through fully engaging the family and
getting their buy-in for services POE designers anticipated that if families could be engaged and
commit to services it might be possible to stop children from being the subject of another
incidence of maltreatment, or recurrence. Recurrence is defined as, “a substantiated report of
child abuse or neglect following a prior substantiation that involved the same child or victim or
family,” (Child Welfare Information Gateway, n.d.b., Glossary R). DCFS was targeting the
listed outcomes largely due to the Child and Family Services Review (CFSR), a federally
mandated process with financial consequences, requiring jurisdictions to report outcomes and
when needed improve (U.S. Department of Health and Human Services, n.d. Child and Family
Services Reviews). Researchers have noted that the outcomes conflict with each other, and
concerns have been raised about the methodology behind the standards used for the outcomes
(Schuerman & Needell, 2009).
In summary, review of child welfare system development shows that modern day systems
have not moved too far from the reactionary position. The multiple levels of regulation,
requiring concurrent decision making, along with the structures and functions of the system
illustrated through the different types of services provided by several types of units, creates a
complex context for practice change. Improvements in safety, permanency, and well-being are
often the only outcomes considered when deciding whether implementation of child welfare
practice change was successful. While these improvements are important, stakeholders need to
know if and how implementation of the practice change is progressing in the period before child
and family outcomes are measurable. Stakeholders also need to know whether outcomes were
affected by level of implementation. Though empirical literature about implementation success

46

is still developing, implementation science literature has found that characteristics of
organizations, actors, and environments influence implementation. Implementation literature and
POE characteristics were used to develop an integrative conceptual model incorporating practice
specific organizational, actor, and environmental level indicators that may influence
implementation (Klein & Sorra, 1996; see Figure 1). An integrative conceptual model has a
premise that provides a rationale for the contents of the model. The integrative premise in this
study was that implementation success or failure was a function of both organizational and actor
level factors, and that the environment could also influence implementation success. The model
also captures the conceptual and methodological steps of the study. First implementation factors
were identified. Then these factors were translated into POE specific indicators. Next
implementation success was explored by noting the presence or absence of the indicators. After
high implementation offices were identified, service data were used to observe trends in child
safety and permanency.

47

Operationalize & Assess for Presence of Indicators of POE
Implementation Success
Organizational - Resource adjustments such that staff were allocated
where needed [ER & ISW/Bridge]
Actor – Climate - changes (collaboration across DCFS units,
collaboration with families, collaboration with community
organizations, intervening as early as possible) perceived as positive
Actor - Leader and/or champion facilitates practice changes
Determine Implementation Factors Relevant to POE Practice
Changes
Organizational – Resource adjustments
Actor – climate, leadership/champions
Environmental – Aspects of the practice context that may impact
implementation (i.e. starting points)
Observe Targeted Safety and Permanency Trends
 Fewer removals from home
 More cases terminated as reunifications
 Reductions in time to reunification
 Lower # of substantiated re- referrals (recurrence)

Operationalize & Assess Environment’s Influence on
Implementation Success
 Historical relationships with community based organizations
 Characteristics of community needs
Step
One
Step
Two
Step
Three
Step
Four

48

Chapter Three: Methods
Research Aim and Questions
This study aims to add more depth to what is known about implementation of child
welfare system practice change by exploring Point of Engagement (POE). POE was a major
shift in practice philosophy and subsequent service delivery redesign implemented in Los
Angeles County’s Department of Children and Family Services (DCFS). DCFS serves a large,
metropolitan, and diverse region. Three questions were answered in the study:
1. What organizational and actor level indicators of implementation success were present in
each office?
Organizational level indicators included resource adjustments such that staff was allocated
where needed. For POE, and for this study, resource adjustments were operationalized as:
1a. adequate staff in the Emergency Response (front-end) units to handle the caseload under
POE, with adequate defined as enough staff to provide services as dictated under POE,
and as reported by the participating staff. A specific number is not used because each
office uses a unique formula to determine its staffing ratios.
1b. available staff to function in the bridging role meant to facilitate more efficient case
transfer between the Emergency Response (front-end) units and the Family Maintenance
and Family Reunification (back-end) units, with available defined as enough staff such
that cases were not being rejected by this type of worker, or ISW units were not closed to
new cases.
Actor level indicators included positive climate, or perception of change. The changes
required for POE, and for this study, were operationalized as:
1c. cross unit collaboration, meaning communication facilitating seamless services,

49

and collegial relationships between all staff (front-end and back-end staff at the worker,
supervisor, and managerial level) involved in providing services to the family
1d. collaboration with families, meaning strengths-based communications between DCFS
staff and families, and involvement of families in decision-making
1e. collaboration with community organizations, meaning communication facilitating
seamless services and collegial relationships with staff from community organizations
(e.g. mental health agencies and substance abuse providers)
1f. intervening as early as possible with appropriate services, meaning recognition that
utilizing family strengths may help avoid the removal of a child from his or her home.
The other actor level indicator included the presence of a leader or champion. For this study the
presence of a leader or champion was operationalized as:
1g. the person(s) that others cited as nurturing the implementation of POE overall, as a
service delivery model or philosophy, or as nurturing the implementation of POE’s main
components.
The next question was:
2. Did the environment influence implementation success? If so was it a barrier or
facilitator for success?

Implementation success was defined by the presence of the two overall types of
indicators in an office (organizational and actor), and the presence of a champion. Although
high implementation offices were identified in question one, in question two the author dug
deeper into the why of the presence or absence of the six indicators defined above, determining
whether defined environmental influences were barriers or facilitators for success. The first
environmental influence was historical relationships with communities. For this study historical
relationships were operationalized as:

50

2a. the quality of pre-POE relationships that DCFS regional office staff had with staff
from community-based organizations.
The second environmental influence was defined as community needs. Community needs were
operationalized as:
2b. families with income below poverty – the percentage of families whose income in
the past 12 months was below the poverty level (U.S. Census 2013b)
13
.
2c. low level of education- of the population that is 25 years and over the percentage of
persons with less than a 9
th
grade education and the percentage of persons with less than a
high school diploma or equivalency (low education), as compared to the percentage of
persons who had a high school diploma or equivalency, and the percentage of those with
a Bachelor’s degree (U.S. Census, 2013c). The two categories under low education (less
than a 9
th
grade education and less than a high school diploma or equivalency) were kept
distinct to emphasize the level of need.
2d. unemployment – of the population 16 years and over in the civilian labor force the
percentage of persons that are unemployed (U.S. Census, 2013d).
2e. grandparents as caregivers – of the number of grandparents that live with their
grandchildren the percentage that is financially responsible for meeting the basic needs of
their grandchildren (Social Explorer, 2013).
2f. referrals-the position or rank of an office based on number of referrals as compared to
other offices.
2g. children removed from home of origin – the position or rank of an office based on

13
The U.S. Census Bureau uses several streams of income to calculate a family’s total income. The final dollar
amount is then compared to one of 48 poverty threshold designations, which vary by size of family and age of the
family members. When the income is less than the dollar amount of a threshold then the family is considered as
having income below poverty. http://www.census.gov/hhes/www/poverty/poverty-cal-in-acs.pdf

51

number of children removed from home of origin as compared to other offices.
2h. children placed in out-of-home care- the position or rank of an office based on number
children placed in out-of-home care as compared to other offices.
The third question was:
3. What were the trends in safety and permanency outcomes for high implementation offices
where the environment facilitated implementation success?
Distinct from the latter part of question two which examined level of need for DCFS services at
one point in time, this question examines federally mandated safety and permanency outcomes
over time. Safety outcomes were defined as removal from the home, and operationalized as:
3a. any time in which the child was removed from his or her home of origin, and placed
in out-of-home care with substitute caregivers.
A second safety outcome was recurrence. Recurrence was operationalized as:
3b. either the substantiation of an allegation of maltreatment or an inconclusive finding
resulting from investigation of an allegation, subsequent to a previously substantiated or
inconclusive allegation.
When the investigatory caseworker finds evidence that proves the existence of the allegation the
allegation is substantiated (Goldman, Salus, Wolcott, & Kennedy, 2003). The most common
example is when a caseworker obtains medical evidence, such as an x-ray, of physical
maltreatment. When the investigatory caseworker cannot find evidence that either proves or
falsifies the allegation the allegation is determined to be inconclusive (Los Angeles County
Department of Children and Family Services, 2009). The substantiated and inconclusive
categories were combined because it is possible that inconclusive allegations are valid instances
of maltreatment. Literature related to allegation outcomes suggests that there may be not be

52

significant differences in maltreatment experienced by children with case evidence and those
without evidence (Fluke, Shusterman, Hollinshead & Yaun, 2008).
In this study the first permanency outcome was defined as reunifications. Reunifications were
operationalized as:
3c. the number of cases terminated as reunifications, meaning the number of children in
out-of-home care whose stay was ended because they were reunited with their parents
(Los Angeles County DCFS, 2009).
The final permanency outcome was timeline to reunification. Timeline to reunification was
operationalized as:
3d. the median number of days a child spent in out-of-home care before being reunified.
Research Design – Case Study
According to Yin (1994), “..a case study is an empirical inquiry that investigates
contemporary phenomenon..,” (p. 13). The case study is one of the most appropriate research
designs for, “..appreciating the complexity of organizational phenomena,” (p. xv) and for
exploratory and descriptive types of research (Aarons, 2009). The case study research design
provides a framework for incorporating multiple sources of data and for obtaining convergence
of data via triangulation. This design is also appropriate to use when the researcher is studying
events over time and placing events, over which he or she has little control, in a real world
context to assess how and why the events occurred. Case studies help in telling the whole story,
which is necessary when the boundaries between the chronicled events and the setting are not
distinct.
Theories or propositions direct case study data collection and data analysis. The theory
helps in development of the research questions, identifies pertinent variables, may predict

53

relationships, and ultimately helps in interpreting results. This case study proposed that
indicators of implementation success could be observed in each unit of analysis, child welfare
regional offices in Los Angeles, and proposed that the environment could influence
implementation success. It was further proposed that when the highest degree of implementation
was observed, and the environment facilitated success then outcomes should be observed. The
phased design of the research was important to telling the whole story. Case studies must also
consider the theory of how a program is supposed to work (Yin, 1994). POE’s theory of change
stipulated that certain outcomes would be achieved if the strengths based philosophy was
embraced and the service delivery changes were implemented.
Unit of Analysis – Regional Offices
Sixteen regional offices of the Los Angeles County Department of Children and Family
Services were the units of analysis.
14
Specifically 16 offices were examined to answer questions
one and two, and two regional offices were examined to answer question three. Seven thousand
DCFS employees (California State Auditor, 2012) work in these offices. From 2009-2012, one
Regional Administrator (RA) managed each office.
15
Assistant Regional Administrators (ARAs)
provide support (SEIU Local 721, 2009). Typically, there are 2-6 ARAs in each office
(depending on the overall caseload managed by the office). Line operations are directed by
Supervising Children’s Social Workers (SCSWs). SCSWs oversee the employees providing
direct services, Children’s Social Workers (CSWs). Depending on their assignment, CSWs may
provide either front-end emergency response services while allegations of child maltreatment are

14
The number of regional offices maintained by DCFS changes often due to: availability and functioning of office
facilities, increases in referrals from specific communities, combination of functions, and/or organizational redesign.
There were 16 offices providing the full range of child protective services at the time when qualitative data were
gathered (2008-2009). After a 2013 organizational re-design and relocation of several offices there were 18 regional
offices.
15
The new organizational plan pairs offices; one RA per type of service unit manages two or more regional offices.

54

being investigated, or back-end services such as family maintenance, family reunification, and
permanency planning services once a case has been opened (either by court order or through
time-limited “voluntary”
16
participation of the family). See Figure 7 for an example of regional
office staffing structure.

Figure 7. Regional Office Staffing Structure 2009-2012
Adapted from Figure 1. Makeup of a typical DCFS Regional Office, in Reforming the Los
Angeles County Department of Children and Family Services Recommendations from Los
Angeles Social Workers (SEIU, 2009, p. 5)

16
There are differing opinions about the level of choice inherent in voluntary service agreements. See Conley
(2007) for a further discussion.
Regional
Administrator
Front - End
Assistant Regional
Administrator
Supervising
Children's Social
Workers
Children's Social
Workers
Back-End Assistant
Regional
Administrator
Supervising
Children's Social
Workers
Children's Social
Workers

55

Data Sources and Procedures
Question one. Two different streams of data were used to answer the questions:
1a. To what extent was the organizational level indicator of adequate staff in Emergency
Response units present
1b. To what extent was the organizational level indicator of staff available to bridge front
and back end functions present in each office?
1c. To what extent was the actor level indicator of a positive climate for communication
facilitating seamless services and collegial relationships between DCFS staff in different
units present in each office?
1d. To what extent was the actor level indicator of a positive climate for strengths-based
communication between staff and families, and family involvement in decision making
present in each office?
1e. To what extent was the actor level indicator of a positive climate for communication
facilitating seamless services and collegial relationships between DCFS staff and
community based organization staff present in each office?
1f. To what extent was the actor level indicator of a positive climate for recognizing that
utilizing family strengths may help avoid removal of a child from his or her home,
present in each office?
1g. To what extent was a leader or champion recognized as nurturing implementation of
POE as a philosophy, service delivery model, and/or as represented by its main
components present in each office?
Data sources included 2008-2009 and 2012 qualitative POE data. The 2008-2009 data
came from a previous study of POE conducted to document the POE implementation process

56

across offices, determine the most important factors supporting the practice, and assess, based on
staff members’ perceptions, how POE impacted the offices and the families served. The current
study adds to the foundation established by previous POE research, but it is set apart through its
conceptual model (see Figure 1), which incorporates additional aspects of the community
context, community needs, and features indicators of implementation success. Two hundred
DCFS employees participated in the 2008-2009 data collection. Participants were involved in 53
scripted interviews and focus groups. Participants included administrative (Deputy Directors),
managerial (Regional Administrators, Assistant Regional Administrators), supervisory
(Supervising Children’s Social Workers), worker level (Children’s Social Workers), and
paraprofessional staff representing the 16 DCFS regional offices. Two satellite offices were
grouped with their parent office for evaluation purposes. Other demographic data on
participants, such as race, age, length of employment, were unevenly recorded and will not be
reported here.
Researchers have offered criteria for data being used for secondary analysis. The
previously collected (2008-2009) data met the criteria. There was a topical match; both studies
focused on POE; original data were sufficient to answer new questions, there was
methodological compatibility, and sample congruence (Medjedović & Witzel, 2005; Notz,
2005). The 2008-2009 data could answer question one and two because data included
information about the organization, and multiple levels of actors’ perceptions of implementation.
There was a methodological fit between the previous study and the current study. The previous
study used grounded theory methodology, allowing themes to emerge from the data. Because it
had no pre-set data structure, the researcher could use appropriate qualitative methodologies to
further analyze the data. Access to data in raw form of transcribed audiotapes also facilitated use

57

of different methods. Finally, the previous study used a representative sample, both in staff level
and across regional offices, which was needed to answer research questions one and two.
An additional data source for question one was qualitative data the researcher collected in
2012. When analysis of staff level data revealed that there were two high implementation
offices, newer (2012) qualitative data were collected to validate the 2008-2009 data. The newer
data used the component of the practice change that was implemented differently in the two
offices as a springboard for further data collection. Specifically, data collection focused on the
function of the bridging worker because it influenced workers’ ability to interact with families.
Also the researcher incorporated terminology in the 2012 protocol compatible with the previous
evaluation protocols (see Appendix D) so that comparisons could be made between the previous
data and the current data.
For the 2012 data collection, key informants were identified through a criterion sampling
approach (Patton, 2001). Key informants were defined as staff responsible for the component of
POE that was differentially implemented in the two offices, the bridging function. The
researcher shared these criteria with the Regional Administrator (RA) for each office and the RA
then selected staff to participate. In the first office, where a single staff member provided
oversight during case transitions, participants included three Children’s Social Workers (CSWs),
two Supervising Children’s Social Workers, and an Assistant Regional Administrator (n=6).
One of the CSWs was responsible for oversight during case transitioning, and the other two
CSWs received cases from the first CSW. In the second office, where oversight was provided by
multiple staff, all participants were CSWs (n=5). CSWs in both offices were from Emergency
Response and case carrying, commonly known as Family Reunification/Maintenance units.

58

Study procedures for both the 2008-2009 and the 2012 studies were reviewed and
approved by the University of Southern California Institutional Review Board. During the 2008-
2009 data collection, after informed consent was obtained, the interviewers used scripts to
conduct the interview sessions or focus groups (see Appendix D for 2012 script). Interviews and
focus groups were recorded and then notes were generated by interviewers and/or the evaluation
team members who were present during interviews for note taking. Several team members
transcribed notes and then the interviewer and his or her support reviewed the transcribed notes
alongside written notes, adding information if needed. After a final set of transcribed notes was
generated for each office, the four senior members of the team coded the notes (McCroskey et
al., 2009b). To insure consistency in the coding, a consensus test was conducted. The senior
members of the team coded data from two offices. All the raters’ codes were compared, a final
set of codes was generated and the raters had 100 % agreement on the final set of codes. The
qualitative lead used the final set of codes to make a template. The team used the template to
code the rest of the data.
In 2012, the researcher conducted three focus groups. Informed consent was obtained
before each began. The focus groups lasted for approximately one hour and were conducted at
the employee’s worksite; all but one was recorded. Participants in the last group did not want to
be recorded; the researcher took notes during that focus group. The researcher transcribed
recordings, and added her notes to the transcriptions as needed.
Question two. Again two streams of data were used to answer the questions:
2a. To what extent did pre-POE relationships between DCFS regional office staff and
community based organizational staff facilitate or create a barrier for POE
implementation.

59

2b. To what extent did community needs (poverty, low level of education,
unemployment, grandparents as caregivers, and DCFS services) facilitate or create a
barrier for POE implementation?
Qualitative data gathered from 2008-2009 were used to answer the first part of question
two (historical relationships). Included details on the environment, such as partnerships with
other service agencies, and selected community strengths and needs were sufficient for
answering question 2a. The methodological compatibility, and sample congruence are detailed
in section above. The second part of question two (community needs) was answered with the
Census Bureau’s American Community Survey data (2013b-d) and Los Angeles County
Department of Children and Family Services data compiled by Healthycity.org. The researcher
selected Census data consistent with previous studies specifying community needs that affect
social services (Hayslip & Kaminiski, 2005; National Center for Children in Poverty, 2007;
McCroskey et al., 2009a; Shook Slack, Holl, McDaniel, Yoo, & Bolger, 2004;Smith & Palmieri,
2007) and data that reflected the community’s need for DCFS services. Because there was a
high amount of variance between all of offices, and between service categories, offices were
ranked (1-15) according to the level of services they provided, and then compared. A rank of
one equals the lowest amount of services amongst the offices for the specified service category
while a rank of 15 equals the highest amount of services amongst the offices for the specified
category. Criteria used to identify appropriate public data included credibility and motivation of
the source, correspondence to community boundaries or closest points of service, frequent
updating, nationally recognized parameters, and inclusiveness of community residents or service
users (Coulton, 2005; Roosa, Jones, Tein, & Cree, 2003). This study used each regional office’s

60

service area, as represented by the aggregation of corresponding zip codes in the Census data,
and as defined by the office’s catchment area on Healthycity.org, as a boundary for the
information. The Census data at the geographic level needed was only available for multiple
years (2007-2011) and the DCFS data for regional offices was only available for one year
(2009). An average for each Census category reported in the study was calculated to obtain data
reflective of a single year. Question two results feature data based on the calculated averages.
Question three. Administrative data were used to answer the questions:
3a. Between 2009-2012, in the high implementation offices where the environment
facilitated implementation success, what were the trends in the number of children
removed from their home?
3b. Between 2009-2012, in the high implementation offices where the environment
facilitated implementation success, what were the trends in the number of those removed
and then reunified?
3c. Between 2009-2012, in the high implementation offices where the environment
facilitated implementation success, what were the trends in the time to a child’s
reunification with his or her parents/guardians?
3d. Between 2009-2012, in the high implementation offices where the environment
facilitated implementation success, what were the trends in recurrences of maltreatment?
Data were obtained nearest the time of the latest POE evaluation (2009-2010) up to the most
recent reporting period available (2012; total years covered 2009-2012; see Table 2 for data
sources and variables by research question) from Offices 14 and 15. The study used data on each
child, rather than an entire family, because he or she represented services that DCFS had to
provide, and the data did not allow for identification of whether or not each child in a family

61

received the same or different services. Each referral was followed to observe whether or not a
removal occurred and if so whether the child was reunified and how long the reunification took.
The initial referral was also tracked to see if it resulted in a substantiated or inconclusive re-
referral or recurrence (see Figure 8). Some children were excluded from analysis if their data for
the variable of interest, such as data for removals, was entered in error or was missing. Referral
data beyond 5/31/2012 was not provided; making the 2012 data censored. Study specific yearly
timelines were established to more accurately reflect caseflow in the data on an annual basis.
The timelines were as follows: Year 1 (5/29/2009 – 5/29/2010), Year 2 (5/30/2010-5/30/2011)
and Year 3 (5/31/2011-5/31/2012). Cases referred before 5/29/2009 (n= 7,751) were excluded
from the yearly referrals count.

62

Figure 8. Samples Referrals to Reunifications and used for Question Three.

County level data was used as a frame for office level data. County data came from the
California Child Welfare Indicators Project. The researcher chose time periods, measurement
intervals and data definitions that were closest to those used for the office level data, however

63

there were no exact matches for any of the categories, on any of the outcomes. Time periods and
definitions for the county data are noted for each outcome in the results section. Finally, DCFS
data added to the framework, detailing how services provided by the two focal offices (14 and
15) compared to those provided by the other regional offices.

64

Over Arching Research Question Crosswalk

Over Arching Research Question Variables Data Source
1. What organizational and actor level
indicators of implementation success
were present in each office?
Indicators of implementation success =
Organizational – resource adjustments in the
form of more ER staff, ISWs or
reconfiguration of staff responsibilities so
that there was a bridge between units

Actor – climate – practice changes (cross unit
staff collaboration, collaboration with
families, and staff from community
organizations, and intervening as early as
possible) perceived positively

Actor – office level leader/champion nurtures
practice changes
2008-2009 interview and focus group data
and 2012 focus group data

2. Did the environment influence
implementation success? If so was it a
barrier or facilitator for success?

Environment – Pre-POE relationships with
community-based organizations, and
community needs (poverty, low education,
unemployment, grandparents as caregivers,
and DCFS service)

2008-2009 interview and focus group data
and 2007-2011 American Community Survey
Data, and 2009 Los Angeles County
Department of Children and Family Services
Data compiled by HealthyCity.org

3. What were the safety and permanency
outcomes for high implementation offices
where the environment facilitated
implementation success?

Safety = removals, recurrence
Permanency = reunification, timeline to
reunification

2009-2012 office level administrative data
Table 2

65

Analysis
Question one. Qualitative data from 16 regional offices were used to determine whether
indicators of implementation success were present. Multiple levels of office staff, including
administrative and front-line personnel participated in data collection. Using a content analysis
method the researcher recorded what staff said to confirm the presence or absence of indicators.
Though content analysis is sometimes classified as a quantitative method, it is not just a method
for counting words (Hsieh & Shannon, 2005). Qualitative content analysis is used to
subjectively interpret textual data, “…through the systematic classification process of coding and
identifying themes or patterns.” (p.1278). There are three approaches to content analysis,
conventional, directed, and summative. The conventional approach is also called inductive
content analysis and the directed approach may be categorized as deductive analysis. This study
used the directed or deductive approach to analyze the qualitative data.
Taking the directed or deductive approach is advised when a study seeks to increase the
description of a phenomenon and the study’s objective is to assess the utility of a theory or
proposition or possibly conceptually extend a theory or proposition in a new and richer context
(Hsieh & Shannon, 2005). Theory may predict relationships, provide a coding scheme, and
ultimately helps in interpreting results. This study proposed that indicators of implementation
success could be observed in the regional offices, and proposed that the environment might
influence implementation success. It was further proposed that when a high degree of
implementation was observed, then it was appropriate to assess outcomes for children served. In
deductive analysis a researcher may use theory to develop predetermined codes and codes are
used to classify data. Data that does not initially fit in the codes is set aside. The researcher used
the indicators of implementation success (see Figure 1) to develop predetermined codes and by

66

classifying the data was able to determine which offices had the indicators and which offices did
not.
In deductive analysis, data set aside are analyzed later to determine if they support an
alternate theory (Hsieh & Shannon, 2005); this alternate theory is conceptually similar to Yin’s
rival hypothesis (1994). The alternate theory or hypothesis may present a different explanation
for the patterns observed in the data. During the coding process, the researcher may need to
subdivide larger categories. For example, resource adjustments operationalized as enough
Emergency Response staff could be further divided into enough Emergency Response front-line
and enough supervisory staff. Results from directed content analysis are typically reported as
either supporting the proposed theory, or as extending or disconfirming the theory.
When conducting content analysis, using any approach, the researcher must first define
the content. For this study, the researcher chose to work primarily with transcribed notes from
focus groups and interviews. These notes were considered raw data in that they were free from
any structure. Since the researcher did not collect data from every respondent involved in the
2008-2009 interviews and focus groups and did not transcribe all data collection notes, when
needed the researcher referred to quotes in the coded data if information from the transcribed
notes was clearly missing. Transcription was not word for word but focused on capturing the
holistic statement and sentiment of the respondent with further details provided in quotes. In the
transcriptions from the 2008-2009 data collection, some transcribers included more of the
respondent’s own words than others.
The next step, when using deductive content analysis is developing a categorization
matrix (Elo & Kyngas, 2008; see Appendix E). Conceptually this matrix functions as an analysis
template. In this matrix, a priori categories help the researcher probe for data correspondence or

67

divergence. The researcher developed a matrix for the data collected from 2008-2009 and
entered transcribed data into each matrix. Each domain on the matrix reflected an indicator of
implementation success. Matrices were also organized by level of worker (RA, ARA, SCSW,
CSW), with a column for each level. Every type of respondent may not have been involved in
every interview and focus group, and if they were involved may not have had a response for each
domain. The templates featured bulleted information within each column. Each bullet
summarized a distinct response to a question. Sub-bullets clarified responses as needed, and
quotes illustrating the responses were featured in a separate section of the matrix.
The integrity of qualitative analysis is important. Several techniques were used to
establish credibility, and to demonstrate completeness of results and trustworthiness of
interpretations, more commonly known in positivist or quantitative research as validity
(Golafshani, 2003; Morazes, Benton, Clark, & Jacquet, 2009). Credibility was enhanced through
use of both a large and exhaustive sample (n=200), that could discuss how POE implementation
affected their work and impacted families. Because staff represented multiple levels,
administrative, supervisory, and front-line, multiple points of view were present. Completeness
of the results speaks to how well the data capture the topic of interest. For the current study, the
sample included ample representation from the universe of respondents, multiple levels of staff
and different implementation contexts. Using data from multiple respondents within the same
office also contributed to the completeness of the results.
Trustworthy qualitative research increases confidence that the results reflect information
provided by respondents rather than the researcher’s opinions. Clearly linking results to the data
helps establish trustworthiness of the researcher’s interpretation (Elo & Kyngas, 2008).
Respondent quotes show the link between the data and results. The researcher also sought to

68

increase the dependability of the study (Shenton, 2004). Dependability is akin to reliability.
Though discussion of assessing reliability in qualitative studies is considered controversial and
even irrelevant by some (Golafshani, 2003), the issue of dependability is important. If a
qualitative study is dependable, another researcher should be able to understand the steps in the
research, agree on their appropriateness, and if desired replicate the research (Shenton, 2004).
However, obtain the same results is not the goal. The goal is assessing fitness of procedures.
The researcher increased dependability through detailed reporting of procedural steps and
decisions in memos, and post analysis reflections on steps taken and the results achieved. Use of
supportive software, Atlas-ti, and hard copy templates also helped in creating an audit trail by
documenting the steps and decisions, and preserving the coding.
Question two: Did the environment influence implementation success? The
researcher used the same steps outlined above during analysis of question two’s data. However,
the researcher developed a different matrix with a priori category corresponding to studied
aspects of the environment, historical relationships with community-based organizations. The
researcher also used the same techniques described above to establish credibility, demonstrate
completeness of results and trustworthiness of interpretations (Golafshani, 2003; Morazes,
Benton, Clark, & Jacquet, 2010). Data on community needs provided a context for the
qualitative data from question one and question two. Community data were quantitatively
prepared for reporting.
Question three. Administrative data from two high implementation offices and the
county were used to describe the safety and permanency outcomes for children served. These
were also the offices where the environment facilitated implementation success. Trend analyses
were conducted to examine patterns of change (Rosenberg, 1997; Daro et al., 2005). The

69

researcher examined how patterns changed and where they changed. For example, did children
serviced by the first office experience an increase or a decrease in being removed from their
homes from 2009 through 2012 and if there was a change how did the children compare to those
served by the second office and countywide. Whenever the possible the researcher determined
what proportion of the sample experienced an outcome, in addition to outcome counts. A closed
cohort was used for calculations with the sample at risk used as the denominator and the number
of valid cases used as the numerator. Percentages were reported. In trend analyses, the unit of
analysis is time periods not people; one time period is compared to another (Rosenberg, 1997).
For this study, data were analyzed across a three-year time span (2009-2012). Outcomes were
plotted by year and over time using the best scale for visual display. The visual display is most
useful for determining whether there are observable differences in the outcomes by office over
time.

70

Chapter Four: Implementation Indicators and Environmental Influences
Office Level Results for Questions One and Two
Results from questions one and two are presented together to provide a full description of
the degree of POE implementation and details on implementation barriers and facilitators. For
community needs, percentages are reported. The first set of questions asked what organizational
and actor level indicators of implementation success were present in each office, specifically:
1a. To what extent was the organizational level indicator of adequate staff in Emergency
Response units present in each office?
1b. To what extent was the organizational level indicator of staff available to bridge front and
back end functions present in each office?
1c. To what extent was the actor level indicator of a positive climate for communication
facilitating seamless services and collegial relationships between DCFS staff in different
units present in each office?
1d. To what extent was the actor level indicator of a positive climate for strengths-based
communication between staff and families, and family involvement in decision making
present in each office?
1e. To what extent was the actor level indicator of a positive climate for communication
facilitating seamless services and collegial relationships between DCFS staff and
community-based organization staff present in each office?
1f. To what extent was the actor level indicator of a positive climate for recognizing that
early intervention, utilizing family strengths, may help avoid removal of a child from his
or her home present in each office?
1g. To what extent was a leader or champion recognized as nurturing implementation of POE

71

as a philosophy, service delivery model, and/or as represented by its main components
present in each office?
During deductive content analysis of the 2008-2009 DCFS staff focus group and interview data,
data were coded to determine the presence or absence of indicators
17
(see Table 3). The
presence of the six indicators served as markers of the extent of implementation success with
more indicators signifying a greater degree of success, and offices being characterized as
follows:
 low implementation - one to two indicators present
 medium implementation – three to four indicators present
 high implementation – five to six indicators present.
Question two asked did the environment influence implementation success. If so, was it a barrier
or facilitator for success, specifically:
2a. To what extent did pre-POE relationships between DCFS regional office staff and
community-based organizational staff facilitate or create a barrier for POE
implementation?
2b. To what extent did community needs (poverty, low education level, unemployment,
grandparents caring for grandchildren, DCFS service needs) or lack of needs facilitate or
create a barrier for POE implementation?
Results for all but one of the offices (n=15) follow (see explanation below), and the reporting of
results for questions one and two concludes with a summary by indicators.

17
Details that suggested the office’s overall level of functioning were sometimes revealed during analysis, but the
results are not intended to be taken as a statement on the office’s level of functioning, rather they are a reporting of
the indicators.

72

Office excluded in question one and two results. As mentioned in the introduction
some components of POE were implemented in several offices ahead of official implementation.
Analysis found that in one office, the indicators of implementation success were present several

73

Table 3

Presence of Indicators by Office

Office Organizational
Resource
Adjustments
Actor
Staff Collaboration Family Collaboration CBO Collaboration Early Intervention
Leader/ champion Total # of
indicators present
1 X 1
2 X 1
3 X 1
4 X X 2
5 X X 2
6 X X 2
7 X X 2
8 X X X 3
9 X X X 3
10 X X X 3
11 X X X 3
12 X X X X 4
13 X X X X 4
14 X X X X X 5
15 X X X X X X 6

Low
implementation
Implementation
Medium High

74

years before the implementation of POE, and the presence of these indicators was attributed to
one component of what would become POE. Over two years, incidents of maltreatment within
this office’s catchment area resulted in child fatalities. The Regional Administrator, responsible
for managing the office, turned to the then DCFS Director for support in changing practices. The
Director suggested implementation of Team Decision Making (TDM) meetings, and the
Regional Administrator decided to try TDMs, adjusting resources to facilitate TDM
implementation. During 2008-2009 data collection, the office’s management and supervisors
reflected back to the earlier implementation of TDMs, and attributed the presence of the actor
and organizational level indicators to TDMs. TDMs were credited for creating relationships that
led to collaboration with community-based organizations, sparking the communication and
collegial relationships of staff across units, which facilitated seamless service delivery, providing
a forum for strengths-based communication with families where families could play a part in
decision making, and allowing DCFS to intervene as early as possible, helping to prevent
removals. This office presented an example of what Yin terms as rival hypotheses (1994).
These alternate theories present a different explanation for the patterns observed in the data. The
alternate explanations, as well as data from this office, will be explored in the discussion section.
Though the initiation of TDM and/or additional components of POE occurred prior to POE
implementation in other offices, none of the respondents in these offices attributed the changes,
or lack of changes, to the preceding component. Because the changes in the excluded office were
attributed to the one component implemented in a time period prior to official POE
implementation, the office was not included in the reporting of the following results.

75

Low Implementation Offices
Office one. Office 1 was a low implementation office; only one indicator was present.
Office 1 had inadequate staffing resources for the Emergency Response (ER) and Intensive
Services Worker (ISW) units. ER staff inadequacies were attributed to the use of ISWs, though
there were only two ISWs. In this small office, staffing the new position (ISW) strained the
already thin resources. Increases in the number of Spanish-speaking families living in the area
also strained resources; there was not enough bilingual staff in any unit to adequately serve the
families. Only five Spanish-speaking staff worked in the office, and whoever was available at
the time translation was needed had to put aside their work to help communicate.
When the ER workload rose, there was no recourse for that staff, but when the ISWs
exceeded their workload, as they often did in this office, ISWs could not take additional cases.
When cases were refused families had to wait longer to be connected to services, which staff
recognized was not the goal of POE. The ISW’s capped caseload also fueled conflict between
different units in the office. Front-end or ER staff felt that there were too many people allocated
to the back-end (or case carrying units
18
), and that the ISW positions were not necessary. Back-
end units only start working with a family after an investigation is concluded and it is determined
that DCFS intervention is necessary. ER staff expressed that they felt responsible for the bulk of
the services, and that they were alone in service provision.
Perhaps these feelings and the underlying conflicts between different types of workers
boiled over. Staff reported that they sometimes argued at Team Decision Making (TDM)
meetings, the meetings held to gain family trust, develop rapport, and determine how the
family’s strengths could be used to mitigate safety issues. Service providing staff (i.e. Children’s

18
Back-end units start working with the family after an investigation concludes that further
DCFS intervention is necessary.

76

Social Workers not administrators or supervisors) noted that it was still typical for workers and
supervisors in different units to perform their job duties in silos, refusing to communicate and
form collegial relationships across units. These silos may have continued to exist; a front-line
worker expressed that POE felt like a directive, and they had to “do” POE with families whether
they wanted it or not. Further, the group decision-making models instituted under POE felt
mandatory to this participant.
Service-providing staff also had issues communicating with families in a strengths-based
manner and involving families in decision making. When “moneyed” or well-resourced families
were the subject of allegations, the families often brought their lawyers to meetings with DCFS
staff. In reaction to issues experienced with “moneyed” families, a staff member in Office 1
expressed a desire to maintain ownership of decision-making because it was hard to collaborate
with families who secured legal representation. Interacting through lawyers did not facilitate
recognition of family strengths nor did it encourage family involvement in decision making.
Rather it was the lawyer trying to assert what the decision should be.
Supervising and service-providing staff did recognize the value in assessing family
strengths and using those strengths to prevent children from being removed from their homes.
Likely this recognition occurred when lawyers were not present. A Children’s Social Worker
stated,
It is helpful in the front end to have the family come in. A lot of the time, they are scared
of the social worker and feel intimated. In here, they feel part of the process and
validated. They like to see the positive points…
A supervisor noted that learning about family strengths during TDMs helped prevent removal of
children from their homes, and subsequent placement into foster care. But work with the family

77

was hampered by insufficient connections to community-based organizations providing social
support services. This office’s lack of pre-POE relationships with community-based
organizations (CBOs) may have created barriers to staff collaboration with their CBO peers.
Though the Regional Administrator in Office 1 had a history of attending community-wide
planning meetings, the Office needed mentorship in communicating and forming collegial
relationships with CBO staff. The administrators turned to the leader of another office for
support. This mentorship provided an opportunity to assess whether the leader or champion
nurturing change needed to be local. The answer was yes, a leader or champion did need to have
a position or integrated role within the office. Office 1 did not have a leader or champion
nurturing POE implementation. There was a mentor providing guidance on one aspect of POE,
and the impact of the guidance did not last after that consultant stopped working with
administrators in the office.
Office 1’s community had several strengths as measured by the characteristics in Table 4.
These strengths did not help implementation. Some characteristics, such as the lack of families
in poverty (conversely the amount of families with wealth) may have presented a barrier to
implementation. This community had a very low percentage of families in poverty as compared
to those sampled across the state (5% vs. 11%), had low unemployment (8% vs. 12% statewide),
high levels of education with 33% (the largest proportion of those sampled) having a Bachelor’s
degree. The office was ranked low for its number of referrals (3 out of 15) with the lower
number equaling less referrals, removals (3
rd
out of 15) and children in out-of-home care (1
st
out
of 15). Though staff reported that the workload was high, their perceptions were likely
attributable to the types of families being served, especially monolingual Spanish-speaking

78

parents, and the staffing allocations relative to the size of the office. Each office used a unique
formula to determine adequate staffing levels, and office size was one part of the formula.
Table 4
Office One Community Needs
Community Need % of Sample

Families with Incomes Below Poverty 5%

Unemployment

8%

Level of Education
19

Less than 9
th
grade – 3%
No diploma – 3%
High school diploma or equivalent – 11%
Bachelor’s degree – 33%

Grandparents as Caregivers

25%

Referrals

Office Ranking

3
rd

Removals

3
rd

Children in Out-of-Home Care

1
st

Office two. Office 2 was a low implementation office; only one indicator was present.
The ER staff was inadequate for the workload that increased in the front-end under POE. Staff
attributed the increased workload to back-end referrals, referrals that were generated by back-end
(Family Maintenance/Family Reunification, FM/R staff) which ER staff had to investigate
20
.
The office also experienced a lot of turnover, particularly in ER when a new administrator was
assigned to the unit, and the turnover meant more ER positions were unfilled. This office used

19
The percentage will not add up to 100% due to the exclusion of non-mutually exclusive categories in the original
dataset (U.S. Census, 2013c).
20
During the implementation timeframe a working group of DCFS administrators decided on this procedure. The
new procedure led to considerable concern throughout the department.

79

ISWs. Pulling staff into that new position created more inadequacies across the office. An
administrator noted that the office was considering using the “non-ISW model” similar to Office
15, due to staff shortages. ISW workers were also stretched thin and at times could not take
additional cases. Responsibility for connecting families to services fell to ER staff and back-end
staff during those times. ISWs were resented by other staff because they did not have full
caseloads but even without full caseloads, the ISWs fell short of their service goals. A
Children’s Social Worker who functioned in the ISW position noted, “… we don’t have enough
time to engage them [families] in the services we are requesting them to attend.”
There was a negative climate for collaboration and lack of collegial relationships across
units largely because of cross unit (ER, ISW, FM/R) disagreements and resentment amongst staff
over workloads and responsibilities. Back-end referrals increased cross-unit animosity. As one
manager noted, “Mikey will eat it, with ER as Mikey,” because ER had to respond to the
referrals. There was not a pervasive positive climate for strengths-based communication
between staff and families and family involvement in decision making. While one Children’s
Social Worker felt encouraged by working with families and involving them in decisions,
another expressed, “The assumption is that you have a meeting and it is just going to take away
everything that has brought the family to the attention of DCFS that to me is unrealistic.” The
worker was referring to Team Decision Making (TDM) meetings, one of the main forums for
collaborating with families. Another worker had difficulty in working with families that had,
“more resources” presumably more socioeconomic resources, because these families were not
interested in collaborative decision making. Conversely, a supervisor and administrator found it
difficult to involve families with multigenerational issues in decision making because there was
“no rehabilitation”, the same issues recurred. It seems these staff members also struggled with

80

identifying and using strengths-based communication if all they could see were the family
problems.
This office had a weak history of relationships with community-based organizations
(CBOs). The history may have created barriers to communication and forming collegial
relationships with community-based organizational staff. Pre-POE relationships were with three
CBOs and administrators stated that they had personal relationships with specific staff. When
those staff either retired or left the organization, the relationships were lost. There had not been
enough networking with other community organizations to make up for the loss. Under POE,
staff in the office started to reach out again to CBOs. The administrator charged with overseeing
these connections noted that communication and relationship building was a challenge. The
CBOs were also experiencing high turnover, and organizations were competitive, due to scarce
funding. The administrator found that the same organizations seemed to receive all of the county
service contracts. This perception of sole source contracting may have limited the office’s
ability to collaborate with new organizations.
Though there was a pervasive and positive climate for collaborating with CBOs DCFS
staff struggled to connect. Lack of personal connections, or relationships, made it harder for
staff to connect families to services. A Children’s Social Workers expressed,
I think that the concept [POE] is great, but the reality is that there are no resources for our
families, especially for the undocumented families. We sit here, and we go around to
many people, but there is no one who can service them.
Another Children’s Social Worker noted how inadequate resources could lead to a removal.

81

The department has these expectations of these families, but we almost set them out to fail
because there are no resources out there. We say if you don’t do A and B, then we are
going to remove your children.
It is possible that the lack of services was a perception issue. Workers that did not have
relationships with or that did not communicate with CBO staff may have been unaware of
existing services.
An administrator was charged with developing relationships with community-based
organizations, but the office had no specific leader or champion nurturing POE implementation
as a whole. The needs of the community served may have contributed to the high workload that
exacerbated staff shortages and contributed to difficulty some staff had in finding family
strengths (see Table 5). The percentage of families with incomes below the poverty level was
more than double the state percentage (31% vs. 11%) and the level of educational achievement
was low (31% less than 9
th
grade and 17% no diploma), though the percentage of unemployed
persons was slightly below the state percentage (11% vs. 12%). The percentage of grandparents
considered to be caregivers was also below the state percentage (23% vs. 28%). The level of
poverty may have led to this office being one of the higher ranked offices for referrals (10
th
out
of 15). The amount of referrals and removals contributed to the high workload, and likely
contributed to the lack of time for cross unit collaboration and collaboration between DCFS and
CBO staff. The office also ranked high in number of removals (10
th
out of 15), and the high
number of removals contributed to the workload of back-end staff. The office had a lower
ranking for number of children in out-of-home care (5
th
out of 15), but this was likely due to the
office’s location. There were fewer out-of-home care providers in this community as compared

82

to other communities in the county; this may have been due to limitations in local placement
resources.
Table 5
Office Two Community Needs
Community Need % of Sample

Families with Incomes Below Poverty 31%

Unemployment

11%

Level of Education

Less than 9
th
grade – 31%
No diploma – 17%
High school diploma or equivalent – 22%
Bachelor’s degree – 6%

Grandparents as Caregivers

23%

Referrals

Office Ranking

10
th

Removals

10
th

Children in Out-of-Home Care

5
th

Office three. Office 3 was a low implementation office; only one indicator was present.
There were several reasons given for inadequate ER and ISW staff. High workload was the most
cited reason for insufficient resource adjustments, the inadequate resources hampered
implementation. ER workers had insufficient time for responding to referrals, scheduling Team
Decision Making (TDM) meetings, and communicating with service providers. Turnover also
exacerbated the insufficient resources along with an office restructuring where some staff were
terminated. At the time of data collection, an administrator reported that 70% of the back-end
staff had started working in the office within the last year. Though the front-end, where ER

83

services were provided, was not in better shape. A supervisor expressed, “we were supposed to
be staffed at a 120%...But the front-end was never staffed at 120%. It really affected how we
went out on all these referrals….Even now, today, the front-end is still in disarray..” A
Children’s Social Worker commented, “…we are not doing POE at the front-end”.
Regional growth in the number of Spanish-speaking families meant that there was
insufficient bilingual staff office-wide. There were also insufficiencies in the allocations for ISW
staff. When the ISW workload was greater than the staff could manage, ISWs could not take
cases. ER workers had to absorb the cases. Eventually, the staff members filling ISW positions
were needed in other units, and the ISW unit was suspended after a year. Because of insufficient
staffing, workers did not have time to communicate across units, even during the case transfer
process. Nor did they have time to form collegial relationships. Back-end workers were not
participating in TDM meetings. A supervisor commented on back-end worker’s absence from
TDMs.
We’ll call our TDMs. We’ll have the family come in. What ends up happening is that
it’s myself [supervisor], the worker, the family and a facilitator. So there’s no other
points of view from a back-end worker or community resources. In reality, we are
making the decisions, I mean I don’t even need the facilitator there….But it would be
nice if we had more eyes, or more points of view, or more opinions…I know that was the
goal…Logistically it isn’t possible with the amount of work the back-end supervisors
have.
Back end workers had to rely on case files to learn about the families being transferred to them.
There was pressure to come up to speed quickly because families were in crisis.

84

Inadequate staffing was cited as preventing full collaboration with families and with the
community-based organizations that provided support services families needed. The lack of
collaboration led a supervisor to comment that POE was, “…not fully implemented at this
time….” A worker noted the lack of partnerships with community-based organizations made it
harder to achieve the goal of intervening as early as possible. Without the services, children
could not remain at home, or be returned home, and another referral could occur resulting in a
removal. Staff reported that removals were prevented when workers could engage families in
strengths-based conversations, and the decision making process, and find services.
Unfortunately, those experiences seemed to be rare in this office.
The office’s lack of pre-POE relationships with community groups may have also created
barriers to collaborating with CBO staff. Staff in Office 3 focused on familiarizing themselves
with potential faith-based partners. Eventually, they developed a relationship with one CBO.
This particular organization served a wider region encompassing the office. It is possible that
Office 3 had to compete with other regional offices for this CBO’s attention possibly limiting
service slots.
There was no leader or champion cited for Office 3. The Regional Administrator’s time
was spent meeting with community representatives and trying to rectify internal operations. This
split attention may have prevented the RA from becoming a leader or champion. Needs in the
community probably did not contribute to low POE implementation (see Table 6). The
proportion of families with incomes below poverty level was less than the proportion observed
statewide (9% vs. 11%), as was the level of unemployment (8% vs. 12%). The community had a
high level of education (high school diplomas 19% & Bachelor’s degrees 9%) and the percentage
of grandparents caring for their grandchildren was the same as the percentage statewide (28%).

85

The office was approximately midway in the ranking of offices on number of referrals (8
th
out of
15), removals (9
th
out of 15), and children in out-of-home care (8
th
out of 15).
Table 6
Office Three Community Needs
Community Need % of Sample

Families with Incomes Below Poverty 9%

Unemployment

8%

Level of Education

Less than 9
th
grade – 9%
No diploma – 7%
High school diploma or equivalent – 19%
Bachelor’s degree – 9%

Grandparents as Caregivers

28%

Referrals

Office Ranking

8
th

Removals

9
th

Children in Out-of-Home Care

8
th

Office four. Office 4 was a low implementation office; two indicators were present. The
office’s issue with turnover, which existed pre-POE, led to insufficient staff office-wide, and the
increased need for bilingual staff worsened the human resource issues. The low staffing levels
also meant that Emergency Response workers reported to different supervisors depending on the
worker’s schedule. These rotations led to inconsistencies in practice.
The high turnover may have prevented cross-unit staff from working well together.
There was not a positive climate for communication, nor were there collegial relationships
between staff in different units. One source of contention was varying opinions about ISWs.

86

Supervisors in different sections were also known to have conflicts. An administrator noted that
staff did communicate and work together when they were forced. Staff members reported
positive perceptions about early intervention noting the benefits of communicating with families
and involving families in strengths-based conversations and decision-making. Two supervisors
noted the change, under POE, in working with a family, “Back in the day the family would not
have an opportunity to voice their concerns or even talk about any of their strengths, we have to
focus on the family’s strengths,” (Supervisor 1). The second supervisor noted,
You see the family as a family as opposed to just a statistic, because on a piece of paper,
the family seems a lot worse, but when you get to meet that family it helps you to make a
decision about what that family needs (Supervisor 2).
The office was not as successful as other offices in its partnerships with community-
based organizations historically or under POE. An administrator from Office 4 began
participating in a community advisory council prior to POE, but the office ended up
“partnering,” likely meaning contracting, with one organization. The exclusivity displeased
other organizations in the community. Office staff also cited territorial issues among local CBOs
as hampering communication and the formation of relationships. A recent strategy in the office
was to try and collaborate with organizations across multiple projects.
There was no cited leader or champion for POE in Office 4. Administrators were focused
on building relationships with community-based organizations and because staff was likely
overwhelmed with service provision, POE implementation did not seem to be a high priority.
Office 4 served a high-need community as defined by most of the characteristics observed (see
Table 7). The proportion of families with incomes below the poverty level in this community
was more than double the proportion statewide (34% vs. 11%), though unemployment was

87

slightly lower (11% vs. 12%). There were also lower levels of education (26% less than 9
th

grade & 15% without a diploma) in the population. A lower proportion of grandparents served
as caregivers (25% vs. 28%). The level of DCFS services required likely contributed to staff’s
high workload. The office was among the highest ranked for number of referrals (14
th
out of 15),
number of removals (14
th
out of 15) and number of children in out-of-home care (11
th
out of 15).
Table 7
Office Four Community Needs
Community Need % of Sample

Families with Incomes Below Poverty 34%

Unemployment

11%

Level of Education

Less than 9
th
grade – 26%
No diploma – 15%
High school diploma or equivalent – 19%
Bachelor’s degree – 3%

Grandparents as Caregivers

25%

Referrals

Office Ranking

14
th

Removals

14
th

Children in Out-of-Home Care

11
th

Office five. Office 5 was a low implementation office; two indicators were present. ER
staffing became inadequate when the number of back-end referrals increased
21
. ISW staffing
was described as inadequate due to regional growth in the population of Spanish-speaking

21
Changes in procedure for these types of referrals occurred at the same time as POE; these procedures were not
part of the POE design.

88

families. There was a negative climate around communication and few relationships between
workers across units.
An administrator noted that ER (front-end) and back-end workers argued over voluntary
cases, specifically circumstances in which voluntary services could be used and circumstances
requiring a detention. Some staff also argued during TDM meetings. Though the cause of the
arguments was unknown, the arguments still had a chilling effect on communication. The ISW
unit seemed to be isolated within the office. A Children’s Social Worker explained that it was
typical to do the work needed without the ISW so there was no need to speak to the ISW.
Staff noted how strengths-based communication with families, and families’ involvement
in decision making, facilitated their intervening at the lowest level possible. A supervisor had
this to say about the communication and subsequent result, “It does give the family a better
understanding. Before back in the day we go in we take your kids, this is what is going to
happen. Now we say let me explain to you what is going on.” An administrator noted that the
collaboration helped to give families ownership over the situation and helped workers realize
that removal was not the first option nor was it always the best solution. Children also became
part of the decision making team, according to a supervisor.
In Office 5, there were some pre-POE relationships between a few DCFS workers and
staff at select CBOs. This office participated in a regional community group, but administrators
felt that another participating regional office got all the attention. Administrators noted that
service resources were scattered, and they were unsure what was available. There was high
turnover within the community organizations administrators did know of; turnover resulted in
inadequate staffing, and subsequently, those organizations had inadequate staffing. The majority

89

of the office staff did not communicate with, nor did they have relationships with, CBO
personnel.
There was no leader or champion cited as nurturing POE implementation within the
office. Perhaps the resource issues, related to staffing and community-based services, pulled
attention away from implementation. It is also likely that the community’s needs contributed to
the workload issues for both the regional DCFS office and community-based organizations. The
community served by the office had some of the highest needs (see Table 8). The percentage of
families with incomes below the poverty level was more than double the statewide percentage
(35% vs. 11%), unemployment was slightly higher (13% vs. 12%) and the population sampled
was almost evenly split between low education levels (19% less than 9
th
grade & 13% no
diploma) and higher levels (high school diploma 23% & Bachelor’s degree 10%), though the
proportion with Bachelor’s degrees was the lowest among all the categories. The office was the
highest ranked (15 out of 15) on all the DCFS service categories (referrals, removals, and
children in out-of-home care), and this surely contributed to staffing shortages. The shortages, in
turn, led to conflict over work assignments.
Table 8
Office Five Community Needs
Community Need % of Sample

Families with Incomes Below Poverty 35%

Unemployment

13%

Level of Education

Less than 9
th
grade – 19%
No diploma – 13%
High school diploma or equivalent – 23%
Bachelor’s degree – 10%

Grandparents as Caregivers

26%

90

Referrals

Office Ranking

15
th

Removals

15
th

Children in Out-of-Home Care

15
th

Office six. Office 6 was a low implementation office; two indicators were present. A
participating Children’s Social Worker noted implementation was a, “…work in progress…”
The lack of adequate staff in the Emergency Response (ER) and Intensive Services Worker
(ISW) units was attributed to high turnover, regional growth, and regional needs. One supervisor
noted that the high turnover meant, “There are not enough experienced workers and so the way
in which public child welfare practices are implemented there are always changing.” The overall
increase in population, especially among monolingual Spanish residents, changed staff workload.
A respondent noted that POE implementation was initially delayed because of staffing, but it
appeared problems still were not resolved when the office did implement.
Staffing insufficiencies became worse when the office decided to use ISWs. There were
not enough ISWs to take all the cases and to allow for time to attend Team Decision Making
meetings. A supervisor noted,
The ISW’s role is very important in this office. But again, I want to reiterate that we are
short staffed here in this office and it clearly negatively impacts our functioning in the
office. I think there is a problem when an ISW is missing TDM’s.
And a Children’s Social Worker stated,
POE is a good philosophy but the main issue of POE in this office is staffing. It’s almost
becoming a disservice to the families when workers are overwhelmed with cases. For

91

example, I think within the last week, I was capped at 38 and we (ISW’s) have not been
able to participate in TDM’s as a result.
If there had been a positive climate for staff communication and collegial relationships,
perhaps non-ISW staff could have helped to connect families to services. Staff in this office
reported that they did not collaborate across units. Respondents expressed that it was common
for workers and supervisors across sections to not work together or support each other. A
Children’s Social Worker felt that POE implementation was a, “..constant struggle,” due to the
lack of collaboration, and the office felt divided according to another participating Children’s
Social Worker. One supervisor gave an example of post-POE implementation activities driving
the division and disagreements between front-end (ER) and back-end (FM/R) staff.
I think that with POE, there is a problem between distinguishing between the
responsibilities of the FM/R and ER workers in some cases. For example, we’ve had
situations where because FM/R workers cannot handle certain referrals, they leave
children in endangering situations and call a referral into the hotline for an ER worker to
respond….
The supervisor was referring to the problematic back-end referral.
There was value attributed to communicating with families in a strengths-based manner
and involving them in decision making. The majority of this communication and decision
making took place at Team Decision Making (TDM) meetings. The office had a pre-POE
history of partnering with CBOs providing Family Preservation services, but these partnerships
were seen as contractual relationships. Staff from the administrative to the front-line level cited
issues in communicating with and forming relationships with community-based organization
staff. An administrator noted that it was hard to work on the management level with community

92

organizations, and when managers did connect, the connection did not “trickle down” to
workers. A Children’s Social Worker framed the problem in terms of service shortages,
I don’t know if it’s funding or a budget issue, but there is always a waiting list for
everything. I don’t see much interaction between our office and other local service
providers. … Our families have to go to ____ to get services. To be honest, there really
aren’t a lot of services out here for them ….
Because of the challenges with local organizations, the office was trying to reach out to faith-
based organizations. Collaborating with faith-based organizations was also strategic because
these organizations could help the office by providing Alternative Response Services (a brief
services response of up to three months), and other types of services that did not require
continued DCFS oversight. A Children’s Social Worker noted the value of Alternative Response
Services as a mechanism for early intervention. The services were seen as a helpful part of POE
because they kept families out of the “system”. However, the lack of communication and
collegial relationships between office staff and community-based organization staff affected
families’ service connections. Additionally there was an administrator that did not think POE
contributed to fewer removals; it just “reinforced” what was already going on in the office.
Perhaps the office was already changing its practices, prior to POE, around intervening at lower
levels.
There was no cited leader or champion in the office. Lack of buy-in at a higher staffing
level may have contributed to low implementation. An administrator did not think POE
contributed to fewer removals but just “reinforced” what was already occurring in the office.
Perhaps this office was changing its practices, prior to POE, and workers were intervening
earlier. Community needs, other than population growth, particularly for Latino/Hispanic

93

residents, did not seem to contribute to the low implementation (see Table 9). Needs were rather
low as measured by poverty level, unemployment, education, caregiving, and the need for DCFS
services were rather low. The percentage of families with below poverty incomes was well
below the state percentage (6% vs. 11%), unemployment was below the state value (9% vs.
11%), education levels were relatively (high school diploma 21% & Bachelor’s degree 20%) and
the proportion of grandparents as caregivers was less than that observed statewide (22% vs.
28%). The office was mid-to-low ranking in the number of referrals (5 out of 15), removals (6
out of 15), and children in out-of-home care (4 out of 15).
Table 9
Office Six Community Needs
Community Need % of Sample

Families with Incomes Below Poverty 6%

Unemployment

9%

Level of Education

Less than 9
th
grade – 8%
No diploma – 9%
High school diploma or equivalent – 21%
Bachelor’s degree – 20%

Grandparents as Caregivers

22%

Referrals

Office Ranking

5
th

Removals

6
th

Children in Out-of-Home Care

4th

Office seven. Office 7 was a low implementation office; two indicators were present.
There was inadequate staff to handle the workload in the ER and ISW units because the office

94

had pre-POE front-line and supervising staff issues. There was high turnover within both staff
levels. An administrator noted that the turnover made implementation of anything difficult.
Administrators and supervisors had to continuously train staff, and this training took time away
from other tasks. When the workload was too high, there was not enough time to train.
Inadequate front-line staffing meant office workers did not have time to communicate
with their counterparts in community organizations. The Regional Administrator and Assistant
Regional Administrator expressed an understanding of POE’s emphasis on collaborating with
community-based organizations. Yet they noted that relationships were a work-in-progress, and
staff at all levels in the office needed to get to know the organizations in the community. The
office did not have history to build on. There were past efforts where administrative staff tried to
reach out to the community, but those efforts were “scattered” because administrators were not
sure how to best use CBO’s resources. The Regional and Assistant Regional Administrator at
that time were making progress in identifying organizations to work with.
Inadequate staffing likely contributed to the lack of collaboration inside the office. An
administrator noted that there was a lot of communication and collegial relationships inside of
the ER unit, but an ER worker felt that it was hard to communicate with back-end workers.
Workers, supervisors, and administrators commented on the new ways in which collaborations
were happening with families, and the effect collaboration had on removals. New ways of
working with families included use of a strengths-based lens to review situations. This,
“philosophy shift,” was noted by a supervisor who also stated that, “… now workers are looking
at collaboration and think of keeping kids in the community, which is really healthy for the
families.”

95

There was no leader or champion nurturing implementation of POE, though a Children’s
Social Worker noted that having a supervisor who supported and embraced POE was helpful.
Staff could have been overwhelmed by some of the community’s needs (see Table 10). The
percentage of families with incomes below the poverty level was well above the state percentage
(16% vs. 11%) though the percentage of unemployed persons was slightly below the state
percentage (11% vs. 12%), and the level of education was high (18% with a high school diploma
& 23% with a Bachelor’s degree). The percentage of grandparents considered to be caregivers
was also below the state percentage (25% vs. 28%). Along with the level of poverty, or perhaps
because of it, this office was one of the higher ranked offices for referrals (12
th
out of 15). The
number of referrals contributed to the high workload, at least for front-end staff, and likely
contributed to the lack of time staff had to collaborate with one another and with workers in
community-based organizations. There was a dramatic change in rankings between the number
of referrals (12
th
) and removals (1
st
) and children in out-of-home care (2
nd
). Perhaps the
strengths-based approach office staff took with families was affecting removals. The likely
contributor to the lower ranking for out-of-home care was the office’s location. Out-of-home
care providers were scarce in the area.
Table 10
Office Seven Community Needs
Community Need % of Sample

Families with Incomes Below Poverty 16%

Unemployment

11%

Level of Education

Less than 9
th
grade – 15%
No diploma – 10%
High school diploma or equivalent – 18%
Bachelor’s degree – 23%

96

Grandparents as Caregivers

25%

Referrals

Office Ranking

12
th

Removals

1
st

Children in Out-of-home care

2
nd

Medium Implementation Offices
Office eight. Office 8 was a medium implementation office; three out of the six
indicators were present. In this office, respondents reflected that use of Intensive Services
Worker (ISW) staff put a strain on the office’s human resources, including Emergency Response.
An administrator noted that the office was “small”, and because the ISWs did not carry full
caseloads, it was difficult to justify staffing the position. ISWs had different opinions from other
staff on their roles, and these different opinions impacted staff collaboration. In addition, non-
ISW staff felt that if they had smaller caseloads, they could cover the responsibilities of the ISW
and the position would not be needed. The tensions around the ISW did not facilitate a positive
climate for communication nor was it helpful for the formation of collegial relationships.
Collaborating with families was seen differently. Children’s Social Workers (CSWs)
commented the most about working with families, and how their collaboration with families
helped to prevent removals of children from their home. They noted that, “Before POE, it was
about detaining. Now we engage the family with the TDM, make the safety plan about whether
the child stays at home or placement….” and that, “I like the engagement of the family, this is
the first time I actually see us really giving the family the power. They have the power. I work
for you, what you want, I do it for you.” However, it was also noted that using strengths-based

97

communication and facilitating family involvement in decision-making was a work-in-progress.
As one CSW expressed, “…striving to work together as a team. It’s hard because sometimes we
believe that the families need to address a particular issue and it is not because we want control
or power but because we really feel that this point of view is what’s best for the child. People
bump heads, and they clash, but we’re trying to be considerate.”
This office had pre-POE relationships with community-based Family Preservation
agencies. The office also implemented the Family to Family initiative prior to POE and that
initiative furthered the communication and collegial relationships with community-based
organizations. These relationships deepened under POE. An administrator noted that the
office’s current work could not be done without the support of community agencies. A
supervisor credited the office’s work with agencies as helping office staff to intervene at lower
levels with families and prevent removals.
There was not a leader or champion cited as nurturing POE implementation, but an
administrator felt that there were, “champion staff” that helped in the implementation of Team
Decision Making meetings. Though championing one aspect of POE falls short of the definition
of champion, as someone who nurtured implementation of POE as a whole (rather than one
component) it may be that staff felt embracing one part of POE, such as TDMs, was sufficient
implementation. Offices were encouraged to customize POE for their site. Further, if one
component was emphasized, then an office-wide champion would not be seen as necessary.
The community served by Office 8 was low need as defined by poverty, unemployment,
education, caregiving and DCFS service level characteristics (see Table 11). It is unlikely that
the community characteristics contributed to the lack of staff collaboration. Poverty and
unemployment levels were lower than those observed state wide (10% vs. 11% and 9% vs. 12%),

98

a higher percentage of the population sampled had higher levels of education (high school
diploma 22% and Bachelor’s degree 18%), and a smaller percentage of grandparents living with
their grandchildren served as caregivers (21%) than those statewide (28%). The office was the
lowest ranked in number of referrals and among the lowest ranked in removals (2
nd
) and children
in out-of-home care (3
rd
).
Table 11
Office Eight Community Needs
Community Need % of Sample

Families with Incomes Below Poverty 10%

Unemployment

9%

Level of Education

Less than 9
th
grade – 11%
No diploma – 10%
High school diploma or equivalent – 22%
Bachelor’s degree – 18%

Grandparents as Caregivers

21%

Referrals

Office Ranking

1
st

Removals

2
nd

Children in Out-of-home care

3
rd

Office nine. Office 9 was a medium implementation office; three out of the six indicators
were present. According to respondents, allocated ISW staff were sometimes unable to take
additional cases, or attend TDM meetings. They also struggled with having enough time to
connect families to services. A supervisor noted that they needed an additional ISW unit. ER
workers felt the burden in their workload from responding to back-end referrals. A back-end

99

supervisor noted that POE had been beneficial for back-end workers because ER workers had to
go out on all the referrals. Tensions between workers, instead of collaboration, about back-end
referrals suggested that perhaps the ER staffing was inadequate.
In Office 9, staff reported that they worked in isolation instead of forming collegial
relationships. It was difficult for the front-end workers to get in touch with back-end workers.
Information was not shared between the units, and there were arguments over the back-end
referrals. Staff from the different units also argued during Team Decision Making meetings,
which of course did not reflect positive communication. An administrator, supervisor, and a
front-line worker noted the “resistance” between cross-unit communication, characterizing
communication as “angry” and cross –unit coordination as “difficult.” ER staff also clashed with
ISW staff. A Children’s Social Worker expressed that roles were not always clear under POE,
and roles changed on a case-by-case basis. The lack of clarity likely contributed to the lack of
positive communication and collegial relationships between staff in the different units.
Though there was a lack of collaboration among the staff, collaborating with families was
seen as a positive aspect of POE. A supervisor stated, “…the Department used to do business a
different way. We never really engaged families; we just told them that you did this and this is
what you have to do. So, it’s [POE] a new practice…” and another felt that POE, “...gives us the
opportunity, right from the very beginning, to develop a partnership with a family, and really
work as a team with that family…” A participating Children’s Services Worker expressed that
talking about family strengths helped during decision making, and a supervisor commented that,
Having the family come to the table with you and having an open feeling of honesty.
They’re safe to say what the problems are or safe to evaluate what’s going on. We can
ask questions. A lot of times that allows us the ability to see what level of energy the

100

parents are willing to put forth to solve their own problems. And because of that, I think
we are able to keep children at home more often than not.
As the supervisor stated above, the strengths-based communication likely facilitated
lower level intervention with families. An administrator discussed how thinking had changed,
and under POE, the question became, “How can we help the family stay intact…” and
particularly how the families could stay together without court mandated services. Connecting
families to services, or as an administrator put it, “…we hook them up with services so we don’t
have to detain” was more feasible in this office because of the relationships office staff had with
community-based organizations.
Staff in Office 9 had pre-POE relationships with community-based organizations. Those
relationships were developed under the Family Preservation initiative largely through regular
meetings with local Family Preservation organizations and staff within the office that maintained
the relationships with these service providers. After POE implementation, communication and
relationships were still strongest with Family Preservation staff, though there was recognition
that this work needed to expand to other types of agencies. An administrator noted that the
office was working on expanding its collaboration with faith-based service providers. Pre-POE
relationships in this office facilitated the positive climate for communication and collegial
relationships, at least with the Family Preservation agencies.
There was no leader or champion cited as nurturing POE implementation. Perhaps this
was due to the type of community served, and feelings that there were resources, at least in terms
of services, available to support this new paradigm. The office served a low-need community, as
defined by poverty, unemployment, education, and caregiver characteristics (see Table 12). The
percentage of the population in poverty and experiencing unemployment levels were both below

101

statewide levels (8% vs. 11% & 10% vs. 12%). For the percentage of the community sampled,
there were higher levels of education (27% with a high school diploma and 14% with a
Bachelor’s degree). Twenty-two percent of grandparents were caregivers in comparison to 28%
statewide. However, this office was in the higher ranks for number of referrals (11
th
out of 15
th
),
and perhaps this contributed to the tensions between workers around ER workload. The office
was slightly higher than midway in its ranking on removals (8
th
) but was high in its ranking on
children in out-of-home care (13
th
out of 15
th
). The high service need around children in out-of-
home care would put more pressure on back-end workers. This may have been why these
workers appreciated POE’s assigning all referrals to ER for investigation.
Table 12
Office Nine Community Needs
Community Need % of Sample

Families with Incomes Below Poverty 8%

Unemployment

10%

Level of Education

Less than 9
th
grade – 12%
No diploma – 10%
High school diploma or equivalent – 27%
Bachelor’s degree – 14%

Grandparents as Caregivers

22%

Referrals

Office Ranking

11
th

Removals

8
th

Children in Out-of-Home Care

13
th

102

Office ten. Office 10 was a medium implementation office; three out of the six indicators
were present. Before, POE Office 10 had insufficient staffing levels throughout its units. When
POE was implemented, Office 10 decided to allocate staff to the Intensive Services Worker
(ISW) position, staffing was stretched even further. The re-allocation of staff increased the
workload on all other staff. A supervisor suggested that the ISW position should be eliminated
to help alleviate the workload. The ISW staff was seen as insufficient because their workload
prevented them from attending some Team Decision Making meetings. In this office, workers
argued, due to their uncertainty over who was responsible for generating the referral. A
supervisor noted that POE was interpreted differently by front and back-end units, and that
affected what actions workers performed. Also the use of ISWs contributed to contention as
back-end staff did not view ISWs as helpful.
There was a positive climate for strengths-based communication between staff and
families and family involvement in decision-making. A supervisor expressed that there was a
shift in working with families, and reaching out first before making decisions. One Children’s
Social Worker had families ask for additional meetings so that the current situation could be
discussed. From a strengths-based perspective, an administrator felt that with the right services,
and a teaming approach, families could preserve themselves, which suggests that DCFS’ role
was to provide services, and not necessarily a lot of oversight.
Office 10 had an inconsistent pre-POE relationship with community-based organizations;
Family to Family had been implemented in the Office, and that initiative generally facilitates
relationships. However, the office staff had relationships with Family Preservation staff because
there was a contract. Relationships were not based on trust nor were they facilitated by regular
communication between DCFS and CBO staff. Though collegial relationships were limited prior

103

to POE, the contracts at least made DCFS staff familiar with the agencies. POE helped develop
these relationships such that there was a positive climate for communication between DCFS staff
and community-based organization staff. An administrator noted, “Community partnering efforts
have proven to be a very important part of POE because of the resources that we did not know
were there,” and the RA further commented that under POE, community-based organizations
were seen as part of the DCFS “team” and “family.” A supervisor commented that DCFS and
community agency staff had each other’s personal telephone numbers and would call each other
to discuss cases.
Office staff expressed that utilizing CBO services, in addition to leveraging family
strengths, enabled them to avoid removal of children from their homes. In one supervisor’s
opinion, pointing safety risks out to families right away helped avoid removals. These issues
could be quickly brought to the families’ attention through a Team Decision Making meeting.
There was no cited champion or leader in the office who was responsible for nurturing POE
implementation. Office 10 served a community that one supervisor characterized as a
microcosm. There were, “…million dollar homes…” on one street and then, “…a block or two
over,” there were dangerous neighborhoods.
The community needs identified for the study reflected that microcosm (see Table 13).
The percentage of families with income below the poverty level, and unemployment were less
than the state average (8% vs. 11% & 8% vs. 12%), the population sampled had higher education
levels (high school diploma 18%, or Bachelor’s degrees 23%), and there was the same
proportion of grandparents caring for grandchildren as there was across the state (28%). Yet the
office was slightly high in its ranking on the number of referrals (9
th
), suggesting there were
some issues that needed to be addressed, and the office’s rank was midway for removals (7
th
) and

104

children in out-of-home care (6
th
). Community needs around some DCFS services may have
contributed to inadequate staffing but did not seem to be a factor in staff collaboration. Internal
or office-level issues hampered staff collaboration. It is possible that the lack of needs made
office stakeholders think no leader or champion was needed for POE implementation. Staff may
have felt that changes would be easy with the type of community they served.
Table 13
Office Ten Community Needs
Community Need % of Sample

Families with Incomes Below Poverty 8%

Unemployment

8%

Level of Education

Less than 9
th
grade – 10%
No diploma – 7%
High school diploma or equivalent – 18%
Bachelor’s degree – 23%

Grandparents as Caregivers

28%

Referrals

Office Ranking
9
th

Removals

7
th

Children in Out-of-Home Care

6
th

Office eleven. Office 11 was a medium implementation office; three out of the six
indicators were present. There was inadequate staff across the office because there was a
problem with high turnover. Still, the ISW was used. Turnover made staffing adjustments
problematic because constant staff training was required. Staff characterized the office as a
training ground for the department. Communication and forming collegial relationships across
units was challenging due to continual changes in staffing. There were also some staff members

105

reporting that joint decision-making and communication was difficult due to other staff feeling
like they were experts and preferring to work alone. One manager commented that lack of
“people skills” led to complaints and prevented interactions. Supervisors went up the chain of
command, to administrators, to resolve conflicts. Meetings were held to encourage
collaboration, but this likely forced collaboration, instead of helping staff value collaboration.
Staff communication and collegial relationships were challenging due to high turnover
and personal animosities that had formed, but strengths-based communication and engaging
families in decision-making was valued. The collaborative and strengths-based decision-making
was mentioned as a mechanism for avoiding removal. Collaborating with families or building
rapport was viewed as getting back to basic social work skills, though a supervisor noted that
working with the family needed to include parents more and not just be child focused. The ISW
and the Team Decision Making meeting were cited as collaboration facilitators. Respondents
also noticed that communication and collegial relationships with staff in community-based
organizations was more successful than office staff collaboration. There was a pre-POE history
of collaboration and an administrator cited the office’s location and size as contributing to the
long history of collaboration, and the personal relationships between staff.
Community needs did not seem to be a factor in staffing shortages or the lack of cross
unit collaboration. The workload was not overwhelming by volume or by the intensity of
intervention needed by families. In this community, the percentage of families whose income
was below the poverty level was below the state’s (10% vs. 11% - see Table 14), though
unemployment was slightly above (13% vs. 12%). There were higher levels of education
amongst the population sampled, and a smaller proportion of grandparents as caregivers (22%
vs. 28%). Office 11 also ranked fairly low on referrals and removals, but much higher on

106

children in out-of-home care. The number of children in out-of-home care could have been
impacted by the office location; this type of community has more out-of-home care providers.
Table 14
Office Eleven Community Needs
Community Need % of Sample

Families with Incomes Below Poverty 10%

Unemployment

13%

Level of Education

Less than 9
th
grade – 10%
No diploma – 11%
High school diploma or equivalent – 26%
Bachelor’s degree – 12%

Grandparents as Caregivers

22%

Referrals

Office Ranking

2
nd

Removals

4
th

Children in Out-of-Home Care

9
th

Office twelve. Office 12 was a medium implementation office; four out of the six
indicators were present. Office 12 was able to adjust staffing resources such that ER was
adequately covered, and the bridging activities were regularly carried out through the creation of
ISW positions. However, ISWs had different opinions from other staff on their roles, and these
different opinions negatively impacted staff communication and formation of collegial
relationships. Communication and relationships within the office were not focused on front and
back-end workers but instead within units, and special workgroups. Staff reported a “pro-
family” approach and discussed how involving the family in decision making, during activities

107

like Team Decision Making meetings, empowered families, and allowed POE staff, “To build
family strength,” (Assistant Regional Administrator).
The Regional (RA) and Assistant Regional Administrator in Office 12 noted that Family
Preservation activities helped in the formation of pre-POE relationships with community-based
organizations. The RA developed a community forum to unify service delivery between the
office and community-based organizations. The staff did want to partner more with the
community-based organizations, particularly in hopes that organizational representatives would
come to Team Decision Making meetings. Staff understood that the services provided by
community organizations were a key component to leveraging family strengths and avoiding
removals. Their desire to have more participation by community staff in TDMs was so they
could “…keep families as much as possible with family,” in other words, to avoid removal of a
child from his or her home.
Though the RA was very active in community agency partnering efforts the RA was not
described by interview participants as a leader or champion for POE. There was no leader or
champion cited for this office. The community served by the office had relatively low needs, the
percentage of families with incomes below poverty level, unemployment, and grandparents as
caregivers were all below state levels (see Table 15), and those sampled had a higher level of
education as reflected by the percent of high school graduates and proportion of the sample
population with a Bachelor’s degree. Yet this office ranked high on referrals, removals, and
children in out-of-home care. The ranking for children in out-of-home care may be in some part
explained by the location; typically there are more out-of-home care providers and placement
possibilities in this type of community. The lack of comments by staff on workload issues, in

108

conjunction with the high DCFS service needs confirms that there were adequate staff
allocations.
Table 15
Office Twelve Community Needs
Community Need % of Sample

Families with Incomes Below Poverty 9%

Unemployment

10%

Level of Education

Less than 9
th
grade – 13%
No diploma – 11%
High school diploma or equivalent – 24%
Bachelor’s degree – 16%

Grandparents as Caregivers

23%

Referrals

Office Ranking

13
th

Removals

13
th

Children in Out-of-Home Care

14
th

Office thirteen. Office 13 was a medium implementation office; four out of the six
indicators were present. Office 13 tried to anticipate and adjust for the demand POE would put
on the Emergency Response (ER) staff. However, the adjustments were not adequate. The high
workload was cited as contributing to inadequate ER staffing. The use of the ISW created
further strain on the resources. Interestingly administrators differed in their opinions on the
staffing levels. One administrator said that the office was, “…very well-staffed now...” while
another stated, “…the extra staff that we have is not enough in my opinion for ER….” Perhaps

109

the workload contributed to the lack of staff communication and collegial relationships across
units. In this office teamwork was only reported within units such as the ER unit.
The lack of staff communication and relationships did not seem to hamper strengths-
based communication with and engagement of families in decision-making. Staff focused on the
power of the Team Decision Making (TDM) meeting in facilitating collaboration. An
administrator stated, “…now it’s the team that includes the family, right there, as part of the
decision,” and another verbalized that the TDM and its inclusion of the family, “…is the most
important thing we have done.” Staff understood that they could leverage family strengths to
prevent removals when the family was involved, and these least restrictive decisions were
characterized as better, and more frequent under POE.
Office 13 had no pre-POE relationships with community-based organizations; the
relationship building started when POE started, and in fact, the lack of historical relationships
spurred the office into action. A Regional Administrator noted, “…we went from virtually no
collaboration to continual outreach to various community agencies, the faith-based community,
the schools. We’ve really focused on community collaboration in the last three or four years.”
The office did not seem to have trouble collaborating once relationships were formed. A
supervisor expressed appreciation for the “insight” of staff from community organizations, and
this new relationship under POE.
The Regional Administrator (RA) was described as the leader and champion. The RA
gave the management team authority to cut through “red tape,” and the managers in turn
“…embraced, encouraged, cajoled, led, massaged, done all the things they need to do to move
this effort [POE] forward…” (Supervising Children’s Social Worker), though perhaps not
enough to ensure staff collaboration across sections and units. The community had relatively

110

low needs; the low level of needs did not ameliorate workload issues (see Table 16). Though a
higher percentage of families in the community, as compared to those across the state, had below
poverty level incomes (15% vs. 11%) the community had the same level of unemployment as
those sampled statewide. Residents had high levels of education (29% high school graduate,
12% with a Bachelor’s degree). There was an almost double proportion of grandparents as
caregivers (40% vs. 28% statewide) yet the office ranked low on referrals (4
th
), removals (5
th
)
and children in out-of-home care (5
th
). It is unknown whether the grandparents provided
resources that contributed to the office’s low rankings in the DCFS service categories, but it is
plausible.
Table 16
Office Thirteen Community Needs
Community Need % of Sample

Families with Incomes Below Poverty 15%

Unemployment

11%

Level of Education

Less than 9
th
grade – 7%
No diploma – 10%
High school diploma or equivalent – 29%
Bachelor’s degree – 12%

Grandparents as Caregivers

40%

Referrals

Office Ranking

4
th

Removals

5
th

Children in Out-of-Home Care

5
th

111

High Implementation Offices
Office fourteen. Office 14 was a high implementation office; five out of the six
indicators were present. The only indicator not present was sufficient resources. Respondents
from Office 14 cited high workload as the reason for insufficient ER staff. A Children’s Social
Worker noted, “POE is a great system, but there are not enough social workers to manage the
case plan at ER.” This office did use the ISW position, and while staff in the office were positive
about the work the ISWs did, it was not clear whether the creation of the position exacerbated
already strained human resources.
Office 14 was one of only two offices where there was an affirmative climate for staff
communication and collegial relationships (see Table 3). Front-line staff shared responsibilities
across the office, and there was cooperation between front and back-end supervisors.
Communication between staff was deemed “incredible” by an Assistant Regional Administrator,
who also noted that staff called each other and visited each other’s cubicles. Supervisors
reported that their workers talked together to make decisions.
Strengths-based communication and involvement of families in decision-making was
clearly valued; a Children’s Social Worker noted, “…the best thing I have is the ability to talk to
the families.” At all levels, staff expressed their positive view of not just working with but
engaging families in decision making and relying on their “voice” (Supervising Children’s Social
Worker). At least one member at every staff level interviewed discussed how DCFS workers
used to be viewed as “baby snatchers” by families but that perception was changing in part
because POE encouraged staff to interact differently with families.
The Regional Administrator (RA) in Office 14 was described as a champion for POE.
The RA communicated the vision of strong community partnerships through the development of

112

a community council that stretched beyond the office’s catchment area, bringing together key
stakeholders in the larger region. Prior to POE, an Assistant Regional Administrator noted that
there was possibly one worker in the office connected to a community-based organization. Then
Family Preservation was implemented, also prior to POE and its implementation along with the
RA’s work and subsequent implementation of Team Decision Making deepened relationships.
The RA engaged in, “…grass roots community organizing, going door to door,” to learn about
the region and what providers had to offer. The RA noted that others thought the region was
lacking in services, in part because needs were high, and it was assumed providers would not be
interested in being in the area due to overwhelming service requests.
Office 14 was one of the higher needs communities with 23% of families sampled
earning a poverty level income, and 14% of those sampled unemployed (see Table 17). Levels
of education were lower (27% less than 9
th
grade education & 18% no high school diploma), and
there were more grandparents caring for their grandchildren as compared to the state percentage
(31% vs. 28%). In terms of services provided, the office was ranked mid-way for referrals (6
th
)
but on the higher end for removals (12
th
) and children in out-of-home care (12
th
).
Community members’ needs may contribute to the high ER workloads experienced by
Office 14, and in turn, the resource adjustment issues in the office. A Children’s Social Worker
stated, “It does get difficult. The higher case loads and high demand from the service area.”
Though Office 14 had a few characteristics in its community that may have contributed to the
high workload and insufficient resources, the environmental characteristics did not seem to
negatively influence any of the other five indicators (see Table 3). The positive climate for staff
collaboration, collaboration with families, and with community-based organizations, likely
facilitated positive views on lower levels of intervention, which created a feedback loop for

113

families who no longer viewed staff as baby snatchers. When discussing lower levels of
intervention, administrative and supervisory staff expressed that removal was the least desirable
action. The RA noted more prevention services were needed to keep families from coming into
contact with DCFS.
Table 17
Office Fourteen Community Needs
Community Need % of Sample

Families with Incomes Below Poverty 23%

Unemployment

14%

Level of Education

Less than 9
th
grade – 27%
No diploma – 18%
High school diploma or equivalent – 26%
Bachelor’s degree – 4%

Grandparents as Caregivers

31%

Referrals

Office Ranking

6
th

Removals

12
th

Children in Out-of-Home Care

12
th

Office fifteen. Office 15 was a high implementation office; all six indicators were
present. Office 15 respondents reported that the ER section was sufficiently staffed, and the
bridging activities were regularly carried out by adding the activities to the responsibilities of
existing staff. Office 15 had high workloads, and that was one of the reasons staff decided to add
the bridging activities to the responsibilities of existing staff instead of creating the Intensive
Services Worker position. During implementation, planning the overall staff structure was

114

reviewed and then adjusted to ensure that no staff units were overwhelmed. In addition, there
was very low turnover in Office 15 so the amount of staff remained relatively stable.
In Office 15, there was an affirmative climate for staff communication and collegial
relationships. Relationships were facilitated by cooperation at both the supervisory and worker
levels across the front and back-ends, as well as communication between staff, not just at
required meetings, such as TDMs, but also during other periods of decision making, such as
during the investigation of back-end referrals, and case transfers. A respondent in Office 15
noted that there were still some disagreements between staff, but they worked together through
an office specific process to resolve the disagreement. For example a Children’s Social Worker
noted,
That is the character of [the office] anyway, among all the units. We all are in different
units, but I can go to anyone for an advice. It’s extremely helpful because everybody is
always available, if they are not available they will tell you to go and talk to so and so, I
don’t know much about this but go talk to so and so.
POE was also described by participants as a unifying practice change, “Since we started POE
and our engagement processes, there is no division, there is no animosity, there is no game-
playing. Everybody works on the same team.” (Assistant Regional Administrator).
There was a Leader/Champion in Office 15 in the form of the Regional Administrator
(RA). The RA communicated the vision of having a strengths-based versus risk and safety
practice, even to resistant supervisors. The communication penetrated as the supervisors then
began to remind workers that they had to be strengths-based with families. An Assistant
Regional Administrator (ARA) stated, “We did a lot of bad TDMs where we couldn’t get
aligned, and the strengths-based philosophy, that is the first thing out of anyone’s mouth around

115

here….” As the ARA continued to reflect on the growth in strengths-based communication with
families, it was noted that though families have challenges, starting from a strengths-based
perspective, “..it healed so much more.” Involving families in decision-making was also critical.
A Children’s Social Worker stated,
…family most of the time understands that they are involved in this decision, and they
are going to be a big part of making that decision. It’s not just the department deciding
for them, we are giving them the opportunity to have some input and make the decision
for themselves.
The Leader/Champion, in addition to a pre-POE initiative, Family to Family, acted as a
catalyst for communication and formation of collegial relationships with community-based
organizations. Prior to Family to Family, staff in the office worked with the same providers over
and over. As the Regional Administrator noted, “...we’re taking it to a different level now where
the agencies see themselves as partners; now they are not just peripheral.” According to staff
Team Decision Making meetings further strengthened collaborations with CBOs. These strong
starting points most likely contributed to the affirmative climate for collaboration.
Having strong service partners may have made it easier for staff to leverage family
strengths and prevent removals. If the child welfare services had the potential to result in
community connections, families may have felt less reluctant about explaining their needs.
Besides understanding family strengths and needs, a shift in perception on what child abuse was
likely facilitated staff intervening with families at the lowest level possible. A Children’s Social
Worker noted that, “…it’s not neglect, it’s just a need. Poverty is not child abuse.” And another
noted, “If the family is homeless, that doesn’t mean we need to intervene.”

116

Some of the characteristics of the community Office 15 served may have facilitated POE
implementation (see Table 18). Though more of the persons sampled had higher levels of
education (a high school diploma or Bachelor’s degree) than lower levels of education (less than
9
th
grade or no high school diploma), unemployment was higher than the statewide recorded high
level during a similar time period,
22
and the percentage of families with incomes below the
poverty level was slightly higher than the statewide percentage (12% vs. 11%). The percentage
of grandparents as caregivers was also higher than the statewide percentage (33% vs. 28%),
When the level of DCFS services was considered, the office was ranked about mid-way on
number of referrals and on the higher end for number of removals and number of children in out-
of-home care. Perhaps having fewer referrals than other offices allowed staff more time to work
with families affected by removals, and those whose children were in out-of-home care.
Table 18
Office Fifteen Community Needs
Community Need % of Sample

Families with Incomes Below Poverty 12%

Unemployment

14%
Level of Education

Less than 9
th
grade – 11%
No diploma – 8%
High school diploma or equivalent – 21%
Bachelor’s degree – 20%

Grandparents as Caregivers

33%

Referrals

Office Ranking

7
th

Removals

11
th

Children in Out-of-Home Care

10
th

22
12.4 % of the labor force in 2010 (United States Department of Labor, 2014).

117

Summary of Offices by Level of Implementation
Following is a summary of offices grouped by the numbers of indicators that were
present (see Table 3).
Low implementation offices. Offices were categorized as low implementation if only
one or two indicators were present. There were seven low implementation offices. In Offices 1,
2, and 3, one indicator was present. There was a positive climate for early intervention, or
recognition that utilizing family strengths may help avoid removal of a child from his or her
home. The positive climate was also present in four other offices (4, 5, 6, & 7) along with
another indicator (n= two indicators). In offices 4-7 there was also a positive climate for family
collaboration, or strengths-based communication between staff and families and family
involvement in decision making.
Medium implementation offices. There were six medium implementation offices (8–
13). Offices were categorized as medium implementation if three or four indicators were
present. Three indicators were present in Offices 8-11 and four indicators were present in
Offices 12 and 13. In all the offices (8-13) there was a positive climate for early intervention or
recognition that utilizing family strengths may help avoid removal of a child from his or her
home. There was also a positive climate for family collaboration or strengths-based
communication between staff and families and family involvement in decision making. The
third indicator present in all medium implementation offices was a positive climate for
collaboration with community organizations, or communication facilitating seamless services
and collegial relationships between DCFS staff and a CBO staff.
This is the first group of offices (8 - 13) with the indicator for community collaboration
(positive climate for communication facilitating seamless services and collegial relationships

118

between DCFS staff and CBO staff). Not surprisingly some offices (8, 9, 11, & 12) in this group
had strong pre-POE relationships with CBOs. Office 8 established relationships through both
Family Preservation and Family to Family. In Offices 9 and 12 Family Preservation also
contributed to pre-POE relationships. The Regional Administrator (RA) in Office 9 specifically
mentioned regular meetings with the area Family Preservation organizations. Office 12’s RA
developed a community forum to unify service delivery between the office and CBOs.
Interestingly, staff in Office 10 had inconsistent pre-POE relationships with CBOs; Family to
Family had been implemented in the office, and that initiative generally facilitates relationships.
The office staff had relationships with Family Preservation staff, but relationships were based on
contracts. The location of Office 11 facilitated pre-POE relationships. The RA there cited the
office’s rather isolated location as contributing to the long history of collaboration with CBOs.
In contrast, Office 13 had no pre-POE relationships with CBOs; the relationship building started
when POE started. The office did not seem to have trouble collaborating once relationships were
formed.
A fourth indicator was present in two offices, along with early involvement, and
collaboration with CBOs and families. In Office 12 there was adequate staff in Emergency
Response units, and staff available to bridge front and back end functions. Office 13 had a
leader or champion recognized as nurturing implementation of POE as a philosophy, service
delivery model, and/or as represented by its main components.
High implementation offices. Two offices (14 & 15) were categorized as high
implementation; five or more indicators were present in these offices. Office 14 had a positive
climate for strengths-based communication between staff and families and family involvement in
decision making; a positive climate for communication which facilitated seamless services and

119

collegial relationships between DCFS staff in different units; a positive climate for
communication which facilitated collegial relationships between DCFS staff and CBO staff; a
positive climate for recognizing and utilizing family strengths to avoid removal of a child from
his or her home whenever possible; and there was a leader or champion recognized as nurturing
implementation of POE as a philosophy, service delivery model, and/or as represented by its
main components. In Office 15 the additional indicator was adequate staff in Emergency
Response units, and staff available to bridge front and back-end.
In Office 14 and 15 pre-POE initiatives acted as catalysts for forming relationships with
their respective community-based organizations. Prior to these initiatives, Office 15 only
worked with the same providers over and over, and an administrator in Office 14 noted there was
possibly one worker in the office connected to a CBO. Family Preservation (Office 14) and the
Family to Family Initiative (Office 15) helped the relationships evolve over time. In Office 14,
the Regional Administrator engaged in grassroots community organizing, going door-to-door to
learn about the region and what providers had to offer. Another key facilitator in pre-POE
relationships was the Team Decision Making (TDM) meetings. Administrative staff in Office 15
said that TDMs helped to strengthen collaborations with CBOs. These strong starting points
most likely contributed to the affirmative climate under POE for collaborating with CBOs.
Having strong service partners may have made it easier to intervene at lower levels, and also
could have facilitated collaboration with families. If the child welfare services had the potential
to result in community connections, families may have felt less reluctant about explaining their
needs.

120

Continued Presence of Indicators in Offices 14 and 15
After identifying the high implementation offices (14 & 15) through the first round of
data collection, the continued presence of the indicators was assessed through a second round of
focus groups. The original interview and focus group data was collected from 2008-2009. New
focus group data was collected in Offices 14 and 15 in 2012 to document the current status of the
indicators, with the exception of leader/champion. Leaders or champions nurture change during
implementation then work to assure that change becomes accepted, regular practice. If a change
becomes regular practice, staff sustains the change, instead of the leader or champion. POE was
accepted, regular practice in Offices 14 and 15. For the purpose of confirming whether staff
sustained change, the researcher read through each focus group transcript (n=3) noting the way
in which resources, collaboration with staff, families, and community-based organizations, and
early intervention were described in the previous round of qualitative data collection.
The researcher found that Office 14 was still experiencing high caseloads, and staff was
stretched thin. In Office 15, high workloads lessened the intensity of staff collaboration.
Though staff reported that they still collaborated during Team Decision Making meetings, they
had a harder time communicating during the rest of the case transfer process. There was still an
affirmative climate in both offices for staff collaboration, collaboration with families and
community-based organizations, and early intervention. Regarding staff collaboration,
respondents from both offices noted, “There’s open communication between all workers; there
are case and staff conferences where everyone is updated.” “Also communication comes
through email and phone calls.” “When you have a lot of people working on finding resources
for families you have better outcomes.” A respondent from Office 14 also discussed
collaborating with families.

121

We hear stories from parents that have had to come back through DCFS, and they
comment on the difference. Now they trust workers because prior they just had contact
with the investigatory worker and were left waiting until the next step or until the
services worker could finally get to them, but all the while, they had no idea who they
could contact. They’ve gained trust.
A Children’s Social Worker in Office 15 noted that collaboration with families came
through quick linkages. The worker was previously assigned to an Emergency Response unit
where workers were required to put services in place before transferring the case, “That was
always my training. Contact had already been made with the family by the service provider, and
that’s when we would transfer.” This level of service was consistent across units.
Respondents in both offices discussed the importance of preventing detentions and one stated,
... ER may have to remove children though due to high safety risks and when they do
once the case comes to ER the ISW can ask court to return the child and also ask for
voluntary services along with the provision of intensive services.
The data confirmed that previously identified indicators were present to a sufficient degree, and
the study could proceed with answering question three.

122

Chapter Five: High Implementation Office Outcomes
Question Three
Question three asked, what were the safety and permanency outcomes for high
implementation offices where the environment influenced implementation success? Offices 14
and 15 had the highest degree of implementation. Five out of the six indicators were present in
Office 14 and all six indicators were present in Office 15. Though Office 14 had a few
characteristics in its community that may have contributed to the high workload and insufficient
resources, the environmental characteristics did not seem to negatively influence any of the other
five indicators (see Table 3). The historical relationships with community-based organizations
for Offices 14 and 15 likely facilitated a positive climate of collaboration with the organizations
and may have helped their collaborations with families and their lower levels of intervention.
The two offices continued to present an interesting contrast as they differed in implementation of
one POE’s key components, staff serving as a bridge between front-end Emergency Response
and back-end units overseeing case management. The offices also served communities at
different socioeconomic levels.
Trends in the safety and permanency outcomes of children served by Offices 14 and 15
were observed over three years (2009-2012). Safety outcomes were operationalized as removals,
any time when the child was removed from his or her home of origin and placed in out-of-home
care with substitute caregivers, and recurrences, defined as the substantiation of an allegation of
maltreatment or an inconclusive finding resulting from an investigation of an allegation
subsequent to a previously substantiated allegation. Permanency outcomes were operationalized
as the number of cases terminated as reunifications and the timeline to permanency
(reunification) for those cases. Outcome tracking started with referrals. Each referral was

123

followed to observe whether or not a removal occurred and if so whether the child was reunified
and how long the reunification took. The initial referral was also tracked to see if it resulted in a
substantiated or inconclusive re-referral or recurrence (see Figure 8).
Level of service in Offices 14 and 15 in comparison to other offices. To get a sense of
how the level of services provided by the two high implementation offices, Offices 14 and 15,
compared to medium and low implementation offices, three major categories of services were
contrasted (referrals, removals, out-of-home care) over one year. Figure 9 details the rankings
for each office, and the actual service category values for each office are featured in Figures 26
through 28 in Appendix F. Overall, Offices 14 and 15 were approximately mid- way in the
ranking (6
th
and 7
th
respectively) for number of referrals, and had higher rankings for number of
removals (12
th
and 11
th
respectively) and number of children in out-of-home care (12
th
and 10
th

respectively). Four other offices, two low implementation offices and two medium
implementation offices, ranked higher in number of removals (Offices 4, 5, & 12) and/or number
of children in out-of-home care (Offices 4, 5, 9 & 12).

124

Office 1 Office 2 Office 3 Office 4 Office 5 Office 6 Office 7 Office 8 Office 9
Office
10
Office
11
Office
12
Office
13
Office
14
Office
15
Referrals 3 10 8 14 15 5 12 1 11 9 2 13 4 6 7
Removals 3 10 9 14 15 6 1 2 8 7 4 13 5 12 11
Children in out of home care 1 5 8 11 15 4 2 3 13 6 9 14 7 12 10
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
*Rank by Service Category
Figure 9. Offices Ranked by Service Category
* Higher ranking (1-15) indicates more families served in a respective category.

125

Office 14 and 15 referral characteristics. All children came to the attention of office
staff through a referral or allegation of maltreatment. From 5/29/2009-5/31/2012 the offices
received 62,883 referrals or allegations of maltreatment that had to be investigated. Staff in
Office 14 responded to a total of 25,056 referrals (see Figure 10) and staff in Office 15
responded to a total of 37,827 (see Figure 11). The number of referrals Office 14 responded to
increased every year, while in Office 15, there was a slight decrease in Year 2. Countywide
referrals increased every year during a similar time period (see Figure 12).

Figure 10. Office 14: 2009-2012 Referrals

7,935
8,544 8,577
0
1,000
2,000
3,000
4,000
5,000
6,000
7,000
8,000
9,000
2009-2010 2010-2011 2011-2012
Number of Referrals
2009-2012
Number of Referrals for Office 14

126

Figure 11. Office 15: 2009-2012 Referrals

Figure 12. 2009-2012 Referrals by Office and County
23

23
California Child Welfare Indicators Project uses the term allegation to designate a referral. Definition of
allegation, “Count each child with a child maltreatment allegation once for each analysis year. If a child has more
12,521
12,256
13,050
0
2,000
4,000
6,000
8,000
10,000
12,000
14,000
2009-2010 2010-2011 2011-2012
Number of Referrals
2009-2012
Number of Referrals for Office 15
Office 14 Office 15 County
Year 1 7,935 12,521 124,716
Year 2 8,544 12,256 132,370
Year 3 8,577 13,050 133,492
0
10,000
20,000
30,000
40,000
50,000
60,000
70,000
80,000
90,000
100,000
110,000
120,000
130,000
140,000
Referrals
2009-2012
Number of Referrals by Office and
County

127

The children referred to Offices 14 and 15 had similar characteristics (see Table 19). The most
prevalent types of allegations were neglect (over 30%), being at risk because the child was a
sibling of another referred child (over 20%), and physical abuse (nearly 20%). More than half
(over 60%) of the referred children were under 12 years of age and were likely to be Hispanic or
African-American.
Table 19

Predominant Characteristics of Referrals for Offices 14 & 15 2009-2012
Office 14
(Out of 25,056 Referrals)
Office 15
(Out of 37,827 Referrals)
Characteristic n % % n
Allegation Type
Neglect
At risk sibling abused
Physical Abuse

Race/Ethnicity
Hispanic
African-American

Ages
0-5
6-11

8,697
6,813
4,079

14,443
8,517

9,474
8,193

34.7
27.2
16.3

57.6
34

37.8
32.7

36.8
24.9
18.4

50.4
26.5

36.6
33.6

13,921
5,001
6,950

19,067
10,036

13,852
12,701

Outcome tracking started with identifying investigations that were substantiated or inconclusive.
For Office 14, 14,097 or 56.3% of referrals were either substantiated (7,488) or inconclusive
(6,609) (See Figure 13). For Office 15, 14,845 or 39.2% of referrals were either substantiated
(7,368) or inconclusive (7,477) (See Figure 14). Across the County, 168,817 of the 390,578

than one allegation in a specific year, they are counted one time in the category of the most severe occurrence.”
http://cssr.berkeley.edu/ucb_childwelfare/Allegations.aspx
Data dates: April 2009-March 2010, April 2010-March 2011, April 2011-March 2012

128

(43%) of referrals were either substantiated or inconclusive during a similar time period (see
Figure 14).

Figure 13. 2009-2012 Referrals and Dispositions by Office

25,056
37,827
14,097
56.3%
14,845
39.2%
0
5,000
10,000
15,000
20,000
25,000
30,000
35,000
40,000
Office 14 Office 15
2009-2012
Number of Referrals and Dispositions by Office
Referrals
Substantiated
/Inconclusive

129

Figure 14. 2009-2012 Number of Referrals and Dispositions by Office and County

Removals. Subsequent to a referral investigation, a case can be opened. If a case is
opened, that means DCFS will be providing further services. Opening of a case may also
indicate that a removal occurred. Out of the 28,942 substantiated or inconclusive referrals in the
two offices, 9,072 cases were opened for further services. Of those 9,072 cases, 3,440 removals
occurred (1,588 in Office 14 and 1,852 in Office 15 - see Table 20).

Office 14 Office 15 County
Referrals 25,056 37,827 390,578
Substantiated &
Inconclusive
14,097 14,845 168,817
0
20,000
40,000
60,000
80,000
100,000
120,000
140,000
160,000
180,000
200,000
220,000
240,000
260,000
280,000
300,000
320,000
340,000
360,000
380,000
400,000
2009-2012
Number of Referrals and Disposition by Office
and County

130

Table 20

2009-2012 Status of Removals by Open Cases & by Offices
Office Year Removals No Removals Totals Cases by
Year
n % % n
14 2009-2010
2010-2011
2011-2012
Total
638
497
453
1,588
43.6
33.5
31.2

56.4
66.5
68.8

824
987
1,000
2,811
1,462
1,484
1,453
4,399

15 2009-2010
2010-2011
2011-2012
Total
566
607
679
1,852
45.6
39.5
35.8
54.4
60.5
64.2
674
930
1,217
2,821
1,240
1,537
1,896
4,673

Removals were intended to decrease under POE, and the yearly percentage of removals, or
prevalence, did decrease (see percentages in Table 20) in both offices (see Figures 15 & 16).

Figure 15. Office 14 Percentage of Removals from 2009-2012

0
10
20
30
40
50
1 2 3
Percentages
Years
2009-2012
Office 14 Percentage of Removals

131

Figure 16. Office 15 Percentage of Removals from 2009-2012

Percentages were not available for county level data due to the proxy variable used
24
. However,
the number of removals decreased countywide during a similar time period (see Figure 17).

24
California Child Welfare Indicators Project entries to foster care category was used as a proxy for removals.
http://cssr.berkeley.edu/ucb_childwelfare/Entries.aspx Data dates: April 2009-March 2010, April 2010-March 2011,
April 2011-March 2012. Data for the entries to care category is not extracted for a defined cohort of children. The
data reflects a point in time measure with time acting as the inclusion criteria. Because the measure is not cohort
based, it does not contain data on the number of cases diverted from foster care.

0
10
20
30
40
50
1 2 3
Percentages
Years
2009-2012
Office 15 Percentage of Removals

132

Figure 17. 2009-2012 Number of Removals by Office and County
25

Reunifications. Of the 3,400 removals tracked across both offices, 2,381 (or 70%) had
identified termination types at the time of the data draw. The termination type describes how the
episode of the child being removed from his or her home ends. Under POE the targeted
termination type was reunification, and 1,983 incidences of reunification occurred (see Table
21). Those that were not reunified were either adopted, placed with a guardian, placed in a non-
child welfare system setting (including a juvenile detention setting), emancipated out of the
system, had their case status change, and in a very small number of cases, the child died.

25
California Child Welfare Indicators Project entries to foster care category was used as a proxy for removals.
http://cssr.berkeley.edu/ucb_childwelfare/Entries.aspx Data dates: April 2009-March 2010, April 2010-March 2011,
April 2011-March 2012.

Office 14 Office 15 County
Year 1 638 566 10,586
Year 2 497 607 10,376
Year 3 453 679 9,731
0
1,000
2,000
3,000
4,000
5,000
6,000
7,000
8,000
9,000
10,000
11,000
2009-2012 Number of Removals by Office
and County

133

Termination types for county-level data included reunification, adoption, placed with a guardian,
exited to non-permanency, or still in care
26
. Still in care data was not included in the reporting
below.
Table 21

2009 – 2012 Reunifications by Office and Countywide
Office Year Reunified Other Exits Totals by Year
n % % n
14 2009-2010
2010-2011
2011-2012
Total*
437
301
214
952
80
84.6
91
83.7
20
15.4
8.9
16.3
109
55
21
185
546
356
235
1,137

15 2009-2010
2010-2011
2011-2012
Total*
345
373
313
1,031
73.7
84.6
93.4
82.9
26.3
15.4
6.6
17.1
123
68
22
213
468
441
335
1,244

County
2009-2010 3,392 47.3% 52.6% 3,772 7,164
2010-2011 3,459 56.1% 43.8% 2,698 6,157
2011-2012 3,485 57.6% 42.3% 2,558 6,043
Total* 10,336 53.3% 46.6% 9,028 19,364
* Total by termination type

POE activities were intended to facilitate reunifications; therefore prevalence of this termination
type should increase. The yearly percentage of reunifications, or prevalence, did increase (see
percentages in Table 21) in both offices (see Figure 18). Reunification percentages were also
increasing countywide during a similar time period.

26
California Child Welfare Indicators Project exits to permanency by the end of the year and before age 18
categories were used http://cssr.berkeley.edu/ucb_childwelfare/C3M1.aspx?r=4 Data dates: April 2009-March
2010, April 2010-March 2011, April 2011-March 2012.

134

Figure 18. Percentage of Reunifications from 2009-2012

Timeline to reunifications. Timelines to reunification were observed for the 1,983
incidences of reunification (see Table 22). Time to reunification was computed in median
number of days. For the entire time period (2009-2012), the median time to reunification for
those children served by Office 14 was 156 days or approximately five months. For children
served by Office 15, median time to reunification (2009-2012) was similar, 151 days or
approximately five months. POE was supposed to facilitate reduced timelines to reunification,
and reductions were observed over time in both offices, though Office 15 held steady for the first
two years observed and then decreased as reflected in the number of days and the percent of
change (see Figures 20 & 21). Both offices experienced these reductions during a similar time
period when the median time to reunification (in months vs. days) was increasing countywide,
0
10
20
30
40
50
60
70
80
90
100
1 2 3
Percentages
Years
2009-2012
Percentage of Reunifications by Office and Countywide
Office 14
Office 15
Countywide

135

although the increase could have been due to the difference in the way the cohort was composed
for the county (see Figure 22).

Table 22

Median time to reunification in days by office

Office

Year

Median Time in Days

% Change
14 2009-2010
2010-2011
2011-2012
167
157
140
N/A
-6
-10.8

15 2009-2010
2010-2011
2011-2012
155
155
131
N/A
0
-15.5

Figure 19. Office 14 Median Number of Days to Reunification from 2009-2012
100
110
120
130
140
150
160
170
180
1 2 3
Median Time in Days
Years Outcome Tracked
2009-2012
Median Time to Reunification Office 14

136

Figure 20. Office 15 Median Number of Days to Reunification from 2009-2012

100
110
120
130
140
150
160
170
1 2 3
Median Time in Days
Years Outcome Tracked
2009-2012
Median Time to Reunification Office 15

137

Figure 21. 2009-2012 Median Time to Reunification in Months by Office & County
27

Recurrence of maltreatment. The final outcome observed was recurrence or
maltreatment occurring in a time period after a previous incidence of maltreatment. There were
28,942 initial incidents of maltreatment across the two offices (14,097 for Office 14, and 14,845
for Office 15) and of those incidents there were 7,512 (3,554 Office 14 and 3,598 Office 15)
incidents of recurrence within the time periods tracked (see Table 23).

27
California Child Welfare Indicators Project median time to reunification definition, “ Of all children discharged
from foster care to reunification during the year who had been in foster care for 8 days or longer, what was the
median length of stay (in months) from the date of latest removal from home until the date of discharge to
reunification?” http://cssr.berkeley.edu/ucb_childwelfare/C1M2.aspx Data dates: April 2009-March 2010, April
2010-March 2011, April 2011-March 2012.

Office 14 Office 15 County
Year 1 13.9 12.9 6.1
Year 2 13 12.9 7
Year 3 11.7 10.9 7.6
0
2
4
6
8
10
12
14
16
2009-2012 Median Time to
Reunification in Months by Offices
and County

138

Table 23
2009 – 2012 Recurrences by Office and Countywide
Office Year Re-referrals Recurrences
n n %*
14 2009-2010
2010-2011
2011-2012
Total (n)
1,564
1,676
1,789
5,029
1,028
1,149
1,377
3,554
65.7
68.5
76.9

15 2009-2010
2010-2011
2011-2012
Total (n)
1,773
1,815
2,102
5,690
1,048
1,141
1,409
3,598
59.1
62.8
67

County
2009-2010 24,405 15,204 62.2
2010-2011 22,075 12,147 55
2011-2012 14,486 7,458 51.4
Total (n) 60,966 34,809
*% of re-referrals found to be recurring maltreatment
Recurrence is considered a safety factor and the decreasing trends are the desired change. The
percent of recurrences increased in both offices with the most increase occurring between Year 2
(2010-2011) to Year 3 (2011-2012) (see Table 23 & Figure 23); though, the offices were similar
in the number of recurrences (see Table 23). During a comparable time period the percent of
recurrences decreased countywide
28
(see Table 23 & Figure 23). However, the researcher had to
use defined follow-up times when selecting county data for use. County level recurrences for

28
California Child Welfare Indicators Project Definition of Recurrence of Allegations, “Of all children who were
subjects of a maltreatment allegation during a 6-month base time period, what percent were subjects of another
maltreatment allegation within 6, 12, 18, and 24 months of that first allegation? The first allegation in the base
period is stratified by disposition type--substantiated, inconclusive, unfounded and assessment only--and recurrence
counts and percents are reported for all disposition types. Children with a recurrence are counted only once,
according to their most severe disposition. Non-Recurrence counts and rates represent children who had no
subsequent allegation during the follow-up period.” http://cssr.berkeley.edu/ucb_childwelfare/RecurAlleg.aspx Data
for similar time periods was available in six-month extracts, October – March and April-September. For example:
October 2009 – March 2010 and April 2010- September 2010.

139

2009-2010 were observed at 24 months post the initial incidence of maltreatment. Recurrences
for 2010-2011 were observed at 18 months post the initial incidence of maltreatment.
Recurrences for 2011-2012 were observed at 6 months post the initial incidence of maltreatment.
The minimum follow-up periods for each year (24, 12, and 6 months) were chosen because in
previous research, risk for recurrence was more likely to be observed soon after initial incidents
of maltreatment and then diminished over time (Fluke, Yuan, & Edwards, 1999).

Figure 22. 2009-2012 Percentage of Recurrences by Office and Countywide

0
10
20
30
40
50
60
70
80
90
Year 1 Year 2 Year 3
Percentages
2009-2012
Percentage of Recurrences by Office and
Countywide
Office 14
Office 15
County

140

Chapter Six: Conclusion
Discussion
This study assessed the implementation of a child welfare system practice change, Point of
Engagement (POE). POE was implemented in the Los Angeles County’s Department of
Children and Family Services (DCFS), and DCFS offices served as the unit of analysis.
Literature suggested that implementation occurs over two to four years (Fixsen et al., 2009).
POE implementation was staggered by office and at the time of the study POE had been
implemented between two to six years. Implementation literature found that organizational and
actor level characteristics and environmental influences impact implementation (Henggeler et al.,
2008; Rye & Kimberly, 2007). There is not yet consensus about which characteristics and
influences are most important, nor is there consensus about operationalizing the characteristics
and influences. Study-specific prioritization and operationalization is necessary. Study-specific
definitions of implementation success are also needed because implementation literature has not
offered a consistent definition.
In the study, six implementation success indicators were assessed, and the influence of the
environment on implementation success was examined. Indicators included early intervention,
collaboration with families and community-based organizations, collaboration amongst office
staff in different units, leader/champion, and resource adjustments. Office-level implementation
was judged as successful or not depending on the degree of implementation. Degree of
implementation, high, medium, or low, was defined by the number of implementation indicators
present. Results from directed content analysis supported the study’s proposed theory
implementation success was observable. A high degree of implementation, operationalized as
the presence of five or more indicators of implementation success, was observed in two offices
(#15 and #14). Six offices (#8-13) had a medium degree of implementation, operationalized as

141

the presence of three to four indicators of implementation success. Seven offices (#1-7) had a
low degree of implementation, operationalized as the presence of one to two indicators of
implementation success. Environmental influences included community needs and an office’s
historical relationships with community-based organizations. Child safety and permanency
outcomes were tracked in high implementation offices, where the environment facilitated
success. This was the final step of implementation assessment. Desired outcomes included
reductions in removals from home of origin, also known as entries to foster care, reductions in
reunification timelines, and reductions in recurring maltreatment. Increase in reunifications was
also a targeted outcome.
Why Did Degree of Implementation Differ?
Analysis of interview and focus group data collected from staff in 15 offices confirmed
meaningful child welfare practice change is difficult but not impossible, and deliberate
differential implementation can be successful. Meaningful practice change is difficult within
child welfare systems because the systems are complex in structure and function. Change is also
complicated because it can assuage or aggravate risk for or experiences of maltreatment. To
accommodate systemic complexity, a singular practice change must have multiple components.
POE had such a design; it was conceptualized as a change to the service delivery system, with
change achieved through multiple components such as Team Decision Making meetings and
bridging staff. The tension between the singular goal of change and plural methods for change
contributed to differences in local implementation. Low to medium implementation offices
focused on components and were not able to successfully implement the change, or POE.
Implementation was also affected by knowing, versus understanding, and then translating POE
philosophy (see Figure 24).

142

Figure 23. Contributors to Degree of Implementation
According to POE philosophy, maltreatment risks and consequences are mitigated by
focusing on family strengths, involving families in decision-making, and engaging community
stakeholders in the creation of a safety net. Staff in low to medium implementation offices were
familiar with POE philosophy, and some staff realized knowing was not enough. A participant
from a low implementation office noted, “I think we are pretty close to being a POE office
because we have all the functional elements included in POE. But where we are working now is
on the philosophy...” (Office 5). The participant’s comment affirms the perception of POE as
components, versus a holistic change, and this perception contributed to low implementation.
Conversely staff in high implementation offices knew and understood the philosophy, and then
purposefully translated the philosophy when making implementation decisions. In these offices,
operationalization of components differed but the function of the components did not differ. For
example, Office 14 used ISWs, and Office 15 did not create the new position; existing workers
were used to create a bridge between front and back-end units.
Knowing, understanding, and then translating environmental influences also contributed to
varying degrees of implementation. In this study, community needs and capacity were
Focus on
Change as
Singular
Goal
Know,
Understand
& Translate
High
implementation
Focus on
Components
of Change
Know
Medium to Low
Implementation
Versus

143

considered. Awareness of needs and capacity sometimes started with pre-POE relationships.
Four offices (#1, 3, 7, & 13) had no pre-POE relationships with the community and organizations
in the community, five offices (#2, 4, 5, 6, & 10) had weak or inconsistent relationships, and six
offices had pre-POE relationships (#8, 9, 11, 12, 14, & 15). Relationships with the community
and community organizations, formed prior to or during POE implementation, helped staff
become aware of what influenced their work. Though study respondents did not cite all the same
needs and capacity issues, they did cite some common issues. Population characteristics such as
language spoken and socioeconomic and documentation status were discussed in the context of
community needs. However, familiarity with the needs and capacities of communities and
community-based organizations was not enough to increase the level of implementation.
Only in the high implementation offices did knowing develop into understanding and then
get translated into implementation planning. In these offices (#14 & 15), staff workgroups and
community partners provided insight on needs and capacity, and also provided a forum for
developing understanding. Needs were considered and discussed during implementation
planning. The ISW still serves as a good example of knowing, understanding, and translating.
Community feedback was one reason Office 15 did not create a new ISW position. Families
expressed a need for consistently assigned workers, instead of changes in assignment.
Conceptualization of change, as a singular goal or multi-component activity, knowledge,
understanding, and translation of change philosophy, and community needs and capacities
contributed to the varying degrees of implementation. There were also aspects of the
implementation success indicators that contributed to findings.

144

What Affected the Presence of Implementation Success Indicators?
The six implementation success indicators were early intervention, collaboration with
families, collaboration between DCFS and CBO staff, collaboration between DCFS staff from
different units, leader/champion, and resource adjustments. Aspects of each indicator affected
implementation and affected the frequency with which indicators were observed across offices
(see Figure 25).

Figure 24. Indicators by Frequency across Offices
Early intervention: Recognizing and utilizing family strengths to avoid removal. Early
intervention meant providing appropriate services, in the least unobtrusive manner possible.
Early intervention was supposed to help prevent removals. Part of the POE philosophy was
recognition that utilizing family strengths could help avoid removals. Because staff only needed
to verbalize their agreement with the philosophy, this indicator was easy to observe; this was the
only indicator present across all offices (n=15). Data for this indicator included language, such
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Early Intervention
Family Collaboration
CBO Collaboration
Champion
Staff Collaboration
Resource Adjustments
Number of Offices
Indicators
Indicators by Frequency across Offices

145

as philosophy shift, strengths-based philosophy, and what was and was not POE. For example, a
Children’s Social Worker noted that opening cases was not “POE.” Nevertheless accepting the
philosophy about family strengths and intervening in families’ lives without removing a child
was a radical change for some respondents and reflected DCFS’ new role in the community.
Across several offices, respondents noted that they used to be referred to as “baby snatchers.”
Moving beyond the philosophy was challenging, and the next indicators reflected a higher degree
of implementation difficulty.
Family collaboration through strengths-based communication and family
involvement in decision making. When philosophy was put into practice, collaboration with
families occurred. That difference between knowing and doing was apparent in 12 out of 14
offices (# 4-15). POE changed the way in which DCFS staff interacted with families by altering
the service delivery structure, most notably in decision-making. Families were engaged in the
decision making about continued safety, risk, and permanency through a strengths-based
approach. The engagement occurred most often during a Team Decision Making (TDM)
meeting, and staff noted that defined activities, such as TDMs facilitated collaboration. In other
words, it was important to provide a forum for collaboration, rather than expecting collaboration
to just occur.
Staff within the offices with an affirmative climate for collaborating with families noted
the positive changes that POE brought (see Table 24),-such as focusing on strengths, as well as
the value of collaboration (see Table 25), which included having a different type of relationship
with the family as well as different outcomes. Most of the respondents who commented on
changes in collaboration were supervisors who had been with DCFS long enough to observe the
changes pre- and post-POE and had experience working directly with families. Children’s Social

146

Workers (CSWs) were also able to comment most frequently on the value of working closely
with families because these workers had the most frequent contact with families.
Table 24

Changes in Collaboration with Families

Supervisor =“Back in the day, the family would not have an opportunity to voice their concerns
or even talk about any of their strengths, we have to focus on the family’s strengths.” (Office 4)

Supervisor=“Prior to POE, the families were angry at the department because they felt the
department made decisions without them and their consideration. Decisions were unilaterally
made and now clients are involved.” (Office 7)

Supervisor=“It’s definitely a change for us in talking with families, talking about their strengths,
it’s already a strength that they are here at the meeting, it’s a strength that they (parents) were
able to talk to their families about this whole situation they may be in.” (Office 15)

Administrator = “Anytime that we’re considering taking the kid into protective custody, we
schedule a TDM, we tell the family, we ask them to bring their support, to bring their friends, to
bring their minister, their pastor, their therapist, the school, anyone close to them. We invite the
community people that we have on our list. So, when we make a decision concerning taking a
child into custody, it is a consensus by everybody present. It used to be that these decisions were
made mostly by the social worker in consultation with his or her supervisor for policy, and now
it’s more than that. It’s no longer the social worker and the supervisor, now it’s the team that
includes the family, right there, as part of the decision. That’s a big change.” (Office 13)

Table 25

Value of Collaboration with Families
SCSW1 =”… when you get to meet that family it helps you to make a decision about what that
family needs…” (Office 4)

SCSW2 =“Our reputation has in some cases been just because we were notorious as being baby
snatchers and all they do is take the kids. I’ve actually heard grandparents say, well you know
when I was coming up with my kids, you guys didn’t have this and if we had had this my kids
wouldn’t have been in the system because I would’ve had my mother at the table... we deal with
the same families forever. But now hopefully we’re breaking that cycle.” (Office 4)

SCSW= “…Having the family come to the table with you and having an open feeling of honesty.
They’re safe to say what the problems are or safe to evaluate what’s going on. We can ask
questions. A lot of times that allows us the ability to see what level of energy the parents are
willing to put forth to solve their own problems. And because of that I think we are able to keep

147

children at home more often than not.” (Office 9)

CSW= “I like the engagement of the family, this is the first time I actually see us really giving
the family the power; they have the power. I work for you, what [do] you want, I do it for you.”
(Office 8)

CSW = “So family most of the time understands that they are involved in this decision and they
are going to be a big part of making that decision, it’s not just the department deciding for them,
we are giving them the opportunity to have some input and make the decision for themselves.”
(Office 15)

The three offices (#1-3) that did not have an affirmative climate for collaborating with families
had different attributions for the absence of the indicator. These attributions were grouped into
family and office or staff-related categories (see Table 26).
Table 26

Attributions for Lack of Collaboration with Families

Family Related Office/ Staff Related
Hard to collaborate with families who secure
legal representation (Office 1)
POE demands and lack of resources make
collaboration difficult (Office 1 & 2)

High socioeconomic families’ sense of
entitlement prevents collaboration (Office 1)

Trying to work with families is hard due to
staff shortages (Office 3)

Multigenerational family issues make
collaboration difficult (Office 2)

Workers don’t need to consult with families;
decisions can be made in house (Office 1)

Collaboration with community-based organizations through communication and
collegial relationships. More challenging to implement was collaboration with community-
based organizations (CBOs). This indicator was observed in eight offices (#8 – 15), and was
absent in seven offices (#1 – 7). Under POE, services were supposed to be provided in a timelier
manner, and to a broader category of families than before. In order to expedite services, and
serve more families, staff needed to have knowledge of and healthy relationships with the

148

community-based organizations in their region. Community-based organizations included social
service providers, such as mental health clinics, substance abuse programs, youth development
entities, and non-profit businesses. Offices with pre-POE relationships had a head start on this
indicator, though some did not get very far (see Table 27). This indicator was also supported by
administration, in most offices, taking an active role in forming partnerships.
Conversely, designated responsibility was problematic in those offices without the indicator.
In some non-indicator offices, collaboration weakened or fell apart when a staff member in either
the regional office or the CBO discontinued their employment. Three different types of resource
issues also hampered collaboration. One office tried to participate in a CBO collaborative that
included organizations across a wide geographical area. Staff from this office felt they were
ignored in this collaborative. A staff member in another office tried to bring together regional
CBOs, but the territorial nature of CBOs, and the subsequent tension, thwarted collaborative
efforts. The financial health of CBOs is sometimes precarious, and organizations often have to
compete for resources. Staff shortages were also cited as a barrier to collaboration, with both
DCFS and CBO staff being insufficient. Inadequate staffing meant there was insufficient time to
collaborate.
Table 27

Facilitators and Barriers to Collaboration with CBOs

Facilitator & Barrier: Designated Responsibility
Regional Administrator developed and maintained collaboration (Offices12 & 14)
Family Preservation Staff developed and maintained collaboration (Office 9)

Facilitator & Barrier: History
Leveraged pre-POE relationships (Offices 8, 9, 11, 12, 15)
Could not leverage pre-POE relationships (Offices 2, 4, 5, 6, & 10)
No history of relationships (Offices 1, 3, 7 & 13)

149

Barrier: Resource Issues
Office not well served by collaborative encompassing a wide geography (Offices 3 & 5)
Limited funding hampers collaboration (Office 2)
CBO’s are territorial (Office 4)
Staff shortages are a barrier to collaboration (Office 7)

Leader/Champion. Interestingly, it was easier to have a POE champion or leader than it
was for staff to collaborate or for there to be adequate staffing levels. There were champions in
three offices (#13-15), and all three were office leaders. As noted in the implementation
literature, champions are important in large and decentralized organizations because the power to
implement a practice change is diffused in these settings (Aarons, Hurlburt & McCue Horwitz,
2011). DCFS had two levels of decentralization; the regional office level, and then within the
regional office level, there was further division by unit (front-end & back-end).
Champions or leaders are largely self-selected. They are people who buy into the change by
putting themselves in the forefront of change events. Front-line (Children’s Social Worker) staff
often cited their supervisor as most helpful in their daily tasks, and supervisors themselves did
express that it was up to them to guide POE implementation. For example, a supervisor stated,
“Any program, the ISW program or whatever, is as effective as the manager that is overseeing
the program, because if you are not working together, every time you turn around you change
things to appease certain people...” Supervisors were not champions for POE, though, because
they did not have the time to nurture implementation of all of POE. They were, rightly so, most
concerned about the part of POE that affected their staff.
Administrators in other offices had a mentor. One mentor supported several
administrators and supervisors across regions. This person came into those regions and helped
the staff establish community relationships. However, when the mentor left, the relationships
were not sustained. A mentor, then, could not function as a part-time champion. Champions

150

need to be continuously present in order to observe and consistently nurture change. The three
POE champions, in Offices 13-15, were office leaders (Regional Administrators). The Regional
Administrators (RAs) helped to nurture POE implementation and accumulate buy-in, utilizing
staff at every level to facilitate change. It was no coincidence that the RAs became champions.
RAs had the most control over their time, and could thereby focus on implementation. RAs were
the only leaders in the office with the broad or strategic view over all areas (see Figure 7), and
they used this view to articulate the POE philosophy in such a way that staff responded. RAs
also had knowledge of the office’s historical successes and failures, staffing, and community
capacities and needs, and translated that knowledge into implementation planning.
Office 15 successfully used, by respondent accounts, an egalitarian method for
accumulating buy-in. A multi-level POE workgroup was created, and members of the group
developed POE protocols for the office. The protocols focused on making practice equitable for
all types of staff, and the result was that staff, “…knew what to expect, we knew what we are
responsible for…” (supervisor) and, “…policy in regards to POE is laid out as to what the ER
responsibilities are, what services workers’ responsibilities are. If you have to forget or miss
something you can always reflect back to this document that has whole POE laid out in regard to
what everybody’s role is,” (Children’s Social Worker). In Office 14, the champion was a
visionary and used vision to nurture change.
The office operated in an area that was historically high needs, but the RA was not
interested in defining the community by what it lacked, rather what DCFS could bring to the
community was envisioned and then implemented. The RA stated, “…the community doesn’t
change service delivery. but service delivery changes the community.” The community’s needs
should not stop change that is needed. Once the change was envisioned, the RA worked with

151

staff at every level, similarly to the RA in Office 15 to further define and then actualize the
vision.
The champion in Office 13 functioned differently. This RA gave the decision making power
over to administrators and supervisors, and then supported what they created. This shared power
facilitated buy-in; as an administrator noted, “…from the very beginning those in charge were all
into it [POE]. The senior managers, they bought into it, and they saw the potential.” This
strategy seemed to facilitate communication and collegial relationships with families and
community-based organizations but the nurturing of staff collaboration did not occur. Perhaps
when the other administrators and the supervisors were designated as change agents, they
concentrated on collaborating with their staff.
Cross unit staff collaboration through communication and collegial relationships.
Cross unit staff collaboration was only observed in two offices (#14 & 15). One of the issues
cited in the DCFS audit that led to POE was the silos within regional offices. Offices were
generally divided into a front-end, or Emergency Response section, that had units which
responded to maltreatment allegations, and a back-end which had units that managed cases
where the allegations merited further DCFS involvement. Back-end units typically included
Family Maintenance and or Family Reunification staff. As depicted in Figure 7, units in both the
front and back-ends were overseen by an Assistant Regional Administrator and staffed with a
Supervising Social Worker and Children’s Social Workers. Pre-POE, it was rare for the front
and back-end sections to collaborate, which led to the silos, and the silos were cited as harming
families in the pre-POE audit (Marts, Lee, McRoy, & McCroskey, 2008) and after POE was
implemented, or implementation was attempted, there were still silos in the majority (13 out of

152

15) of offices (see Table 28). This indicator was affected by the feedback loop in Figure 6 on pg.
40. Because staff did not have relationships before POE, office climate had to enable results.
Respondents noted aspects of both an affirmative and unsupportive collaborative climate.
For example, when teaming was mandatory, the climate was unsupportive. Offices 14 and 15
were the only units of analysis where there was an affirmative climate for staff collaboration (see
Table 3). In both offices collaboration was manifested through cooperation at both the
supervisory and worker level across the front and back-ends, and communication between staff
not just at required meetings, such as TDMs, but also during other periods of decision making,
such as during the investigation of back-end referrals. Though a respondent in Office 15 noted
that there were still some disagreements between staff, they worked together through an office
specific process to resolve the disagreement.

Table 28

Affirmative vs. Unsupportive Climates for Staff Collaboration

Manifestation of
Collaboration
Characteristics of Affirmative
Climate
Characteristics of Unsupportive Climate
Cooperation Staff share responsibilities (Office
14)

Cooperation between front and
back-end supervisors (Office 14
& 15)

Staff still work in isolation (Office 1, 9)
Joint decision making is difficult (Office
11)
Team work only within section, or
section unit (Offices 7, 13)

Managers [across sections] are not on
the same accord (Office 4)
Common for workers and supervisors
[across sections] to not work
together/support each other (Office 1, 6)
Office is divided (Office 6)

Communication

Workers talk as they’re making
decisions together (Office 14)

Argue during TDMs (Offices 1, 5, 9)
Argue when a back-end referral is made
(Office 2)

153

Affirmative vs. Unsupportive Climates for Staff Collaboration

Manifestation of
Collaboration
Characteristics of Affirmative
Climate
Characteristics of Unsupportive Climate

Communication between staff is
incredible (Office 14)

Staff call each other and visit each
other’s cubicles (Office 14)
Lots of communication (Office 15)

Talk to workers that had the case
previously (Office 15)
Argue due to POE collaboration
paradigms (Office 10)

Communication is difficult (Office 11)

Never speak with ISWs (Office 5)
Don’t share information (Office 9)
Isn’t communication during the case
transfer process (Office 3)

Difficult for front-end to get in contact
with the back-end (Office 9)

Extant issues in other indicator areas were mentioned as impacting collaboration.
Insufficient staffing meant that there was no time to collaborate. High turnover certainly did not
create an environment where workers could get to know one another. In offices where staff was
short, ISWs were resented because they did not have full caseloads. However, ISWs had
different opinions from other staff on their roles, and these different opinions impacted
collaboration. Finally, in three offices, staff communicated and worked together, but staff did
not have relationships because their teaming was a “directive” (Office 1), “mandatory” (Office
4), or enforced by “chain of command” (Office 11).
Resource adjustments: Adequate emergency response and bridge worker staff. An
adequate staffing level tied with staff collaboration as the hardest aspect of POE to implement.
This indicator was only present in Offices 12 and 15. Staffing levels were outside of the control
of office staff, and the remedy to insufficient staffing and increased hiring was not a quick

154

solution. Further staffing inadequacies often had a ripple effect across all the other indicators,
and it was the indicator that was most vulnerable to environmental influences.
Offices were supposed to be staffed at 120% before POE was implemented because the
new case flow under POE added responsibilities to Emergency Response (ER) units. The
amount of referrals ER staff had to investigate under POE increased due to the use of the new
Differential or Alternative Response pathways and what were termed as back-end referrals.
POE also called for either the creation of a new worker to serve as a bridge between the front-
end (investigatory/ER) and back-end (case management/Family Maintenance/Family
Reunification) units, or the addition of the bridging responsibilities to an existing staff’s
workload. All offices except Office 12 and 15 struggled with the adequate staffing. These
struggling offices were unable to allocate sufficient staff to their Emergency Response (ER)
units, and/or were unable to sufficiently allocate or utilize bridging staff or add responsibilities to
existing staff tasks. Results showed that in most offices with insufficient staffing, there were
multiple contributors to the absence of the indicator such as high DCFS service needs (Offices 2-
5, 7,9, & 14), mostly in the number of referrals, high community needs (Offices 2-5, 7, & 14),
such as high poverty, turnover (Offices 2-4, 6,7, & 11), increased workload due to back-end
referrals (Offices 2,5, 6, 9 & 10), use of Intensive Services Workers (Offices 1, 2, 6, 8, 10 & 13),
and increased population of Spanish-speaking families (Offices 1, 3 & 4-6) (see Table 29). In
the opinion of one front-line staff member, the high workload meant that staff was unable to
engage with families or operate according to POE proposed timelines. ISW performance was
similarly affected when their unit’s staffing levels did not keep pace with the workload. ISWs
could not take new cases, and they were not “doing POE” at that time, other workers were
linking families to services when possible.

155

Table 29

Contributors to Inadequate Staffing

Office #
High DCFS service needs 2, 4, 5, 7, 9 & 14
(n=6)

High community needs

2, 3, 4, 6, 7 & 11
(n=6)

High turnover

2, 3, 4, 6, 7 & 11
(n=6)

Need for more Spanish speaking staff

1, 3, 4, 5, 6
(n=5)

Staffing shortages exacerbated by use of ISWs

1, 2, 6, 8, 10 & 13
(n=6)

Increased workloads due to back-end referrals

2, 5, 6, 9 & 10
(n=5)

How did the Environment Influence Implementation?
Communities served by each office were assessed for their level of need. Poverty,
unemployment and education levels, the percentage of grandparents as caregivers, and the extent
of DCFS service needs were examined. The environmental assessment also focused on staff’s
pre-POE relationships with community-based organizations. As previously discussed, pre-POE
relationships positively and negatively influenced the collaboration with community-based
organization indicators. In high-need communities (i.e. communities whose residents
experienced high levels of poverty, unemployment, need for DCFS services, and low levels of
education), the needs likely influenced adequate DCFS staff levels. The impact of the
environment on staff was observed even in a high implementation office (#14).

156

Interestingly, in Office 14, other aspects of the environment did not negatively influence
indicators. For example, there was collaboration with community-based organizational staff
despite widespread perception that the community was resource poor. What seemed to make the
difference for Office 14 was a champion/leader who did not buy into the regional stereotypes and
who decided to find out firsthand what was available in the community, while also building up
the indicators within the office. A very strong partnership with a key community-based agency
also seems to have helped Office 14 do more with its minimal staff.
Were Desired Outcomes Observed for Children Served by High Implementation Offices?
Part of the study’s premise was that implementation success should be assessed first, and
then trends in targeted outcomes should be observed. After identifying the two high
implementation offices (Offices 14 and 15), safety and permanency outcomes were tracked for
children served. Trends were observed over three years (2009-2012) to see if removals,
timelines to reunification, and recurrences were reduced and if the number of reunifications
increased. Trends in county data near the times studied were also tracked to provide context for
office level findings. Tracking trends helped to determine if implementation mattered.
Results were mixed (see Table 30). The percentage of removals decreased for both offices
during the periods observed and countywide during similar time periods. Though the
percentages of removals were moving in the desired direction, there may have been a
countywide phenomenon, instead of or in addition to POE that influenced the downward trends.
The percentage of reunifications in both offices moved in the desired directions. Percentages
increased countywide also. Again, though the desired trends were observed at the office level,
there may have been countywide factors, in addition to or instead of POE, influencing the trends.
Time to reunification decreased in both offices, while it increased countywide. It is possible that

157

POE facilitated the increase in reunifications and decrease in time to reunification. It should be
noted that different cohorts were used to examine reunification within the offices and across the
county and perhaps the differences in reunification data is due to cohort composition.
Recurrence was the last outcome examined, and the percentage of recurrences increased in both
offices while it decreased countywide, during the times observed. However, follow-up times,
how long the case was observed in order to determine if maltreatment would reoccur, was not the
same between the offices and the county. There was no limit on the follow-up time for the office
data, beyond the years of the study data. In order to extract the county data, a follow-up time had
to be chosen, and the researcher chose the minimum amount of time. The minimum amount of
time was chosen because in previous research, risk for recurrence was more likely to be observed
soon after initial incidents of maltreatment and then diminished over time (Fluke, Yuan, &
Edwards, 1999).

Table 30

Summary of Safety and Permanency Outcomes

Location
Outcome
Decrease in Increase in Reduced time to Decrease in
Removals reunifications Reunification recurrences
Office 14 Yes Yes Yes No
Office 15 Yes Yes Yes No
County Yes
29
Yes No Yes

29
Percentage of removals was tracked at the office level and number of removals was tracked countywide.

158

Limitations
There were several limitations to the study, and due to the limitations, any results should
be interpreted with caution. There was only one coder of the qualitative data. Typically a coder
will check his or her interpretations of the data with another coder to help increase reliability;
however two or more coders is not the only way to increase reliability. Respondents can be
consulted about the interpretations; in this study, earlier findings were discussed with former
respondents. Also, oversight of the study was provided by one of the Co-Principal Investigators
of the 2008-2010 evaluation, and interpretations were discussed, and as needed, re-explored with
the researcher.
Qualitative findings were based on respondent perceptions, and it is possible that a
different group of participants would have yielded different findings. Using a different definition
of success could have also yielded different findings. Further, the number and type of
respondents differed by office, such that some offices had more data than others. Though raw
data, in the form of verbal transcripts was used, the transcription was not word for word. This
meant that there were differing amounts of quotes in each transcript. Though the study author
was comfortable with the level training the original coders were given, and how this training
translated into quality transcripts, there is more trust required of the consumers of the study since
there was lack of familiarity with the training procedures.
The final limitation is the existence of competing hypotheses, or those explanations that
may be the driving force behind the presence of the indicators, instead of successful POE
implementation. As noted above, there were many changes occurring in the office at once. An
Assistant Regional Administrator stated, “it is difficult to describe how everything interfaces, it
is, … like threads on quilt, though the threads are separate the quilt is still intact.” Though the

159

study did not seek to prove a hypothesis per se, it did follow the premise that indicators of
implementation success could be observed post-POE implementation. Some aspects of POE
were in place ahead of official implementation, and it is possible that the indicators could have
been present ahead of official implementation. Respondents from one office made such a claim
and attributed all the indicators to the implementation of Team Decision Making meetings,
leading to the exclusion of the office in the qualitative analysis. TDMs were an often discussed
component across offices, as was collaborative work done through Family Preservation, and the
Family to Family Initiative, both of which preceded POE implementation in the noted offices.
One Regional Administrator noted that the POE was the “grandchild” of Family Preservation.
However, the pervasive sentiment amongst respondents was though there were practices that
served as catalysts for the indicators; it was POE that served as the backing for the quilt.
Finally, comparison of office and countywide safety and permanency trends is for
descriptive purposes only. This study was not able to control for any differences in the manner
in which the two streams of data were collected, analyzed, and reported, or differences in
variable definitions. Any of the aforementioned differences could be responsible for the
observations noted in the study.
Implications for POE and Child Welfare Practice
Despite the studies’ limitations, the study offers several implications for child welfare
practice in general and for POE practice. First, the systems level must be considered when
designing change. POE was created to be customizable by office, and that is commendable.
Rigid, universal approaches rarely work system-wide. However, the customization needs to be
informed. Local needs assessments should be conducted to determine commonalities and
differences across the system. Information on commonalities and differences should be used to

160

determine what the core of the practice change is, or what is critical to implement across the
system, and what needs to be customized by location. If everything needs to be customized
perhaps, the practice change is not appropriate for systems level implementation. Some offices
considered needs when making POE implementation decisions, but others did not. Of the offices
that did consider needs, few probably cast their net wide enough, and cast it early enough in the
implementation cycle to capture pertinent information on the needs of the community and
community-based organizations.
Local leaders or potential champions are critical to implementation success. In this
study, RAs were natural champions. Perhaps if all RAs had been engaged during the design of
the practice change, each office would have a champion. For example, if they were given
authority to conduct needs assessments, they could identify and later monitor implementation
indicators associated with needs. In order to be a champion, a person must understand the inner-
workings of the practice change. If all RAs had been trained on the philosophy of POE and on
how to nurture change, there could have been an increase in the likelihood of their developing
into champions. All three of the leaders/champions cited in the study would be the best sources
to query about why they accepted this role. What facilitated their leadership in the child welfare
service environment was likely the amount of control they had over their time, and that they
were the strategic planners for the office, using their broad view to assess what has been used
and worked in the office, what was not working, and what was needed to improve performance
in various types of tasks and across different units.
The way in which champions nurtured train should also be noted. The champions
reached out to direct services employees, or street-level bureaucrats to help plan changes, and
continued to keep front-line staff involved in the entire change process. The shared work helped

161

to develop common goals, within the offices, and created relationships built on trust
(McCroskey, 2003). During this work champions constantly emphasized POE philosophy and
implementation of the entire service delivery model.
Turning to POE specific practice, the three indicators that seemed critical to ensuring the
presence of all the others were sufficient staffing, cross unit staff collaboration, and a leader or
champion. Sufficient staffing was impacted by changes in work roles and responsibilities.
These changes appeared to disadvantage some workers and advantage others. As noted in the
literature implementation can compromised when organizational resources are not sufficient
(Durlak & DuPre, 2008; Gotham, 2006; Klein & Knight, 2005; Osilla, Hepner, Munoz, Woo, &
Watkins, 2009; SAMHSA 2007a; SAMHSA, 2007b). Stakeholders should not attempt
implementation if resources are not in place. The changes also hampered staff collaboration.
Offices 14 and 15 offered mechanisms for assessing adequate staffing, distributing workloads,
and thereby, facilitating staff collaboration. They had working groups and asked staff to create
work plans. The administrator in Office 15 listened when staff said they could not absorb a new
position (ISW). Staff engagement helped determine areas for policy flexibility.
Implications for Implementation Assessment
Further study of champions in child welfare contexts might help in understanding how to
boost morale and change practice in the sometimes demoralizing environment within which the
work is done. The three offices with champions (13, 14 & 15) were not without their challenges,
particularly Office 14 which had the highest community needs, but the champions were able to
rally their staff. Core practice models that are differentially implemented across settings need
further study to better understand how the concept of implementation success intersects with
differential implementation. In this study, degrees of implementation success were used to

162

explore what happened when POE, conceptualized as the sum of its parts, was used in parts, and
then how did end users (staff) and families react? There may be other useful conceptualizations
of differential implementation found by future research. Finally, there was a barrier that
impacted multiple indicators and merits further study of constant changes in practice.
Child welfare staff can easily get fatigued by constant change (McCroskey, Furman, &
Yoo, 2007). One respondent, when discussing the changes in the department noted, “It is like
changing a tire on the car, while the car is in motion. That is what we are having to do. We are
having to learn and do things in different ways but we can’t stop, so we have to do it as we go
along.” POE was implemented alongside many other changes, and this caused some confusion.
A Children’s Social Worker expressed uncertainty about policy because it was always changing.
A supervisor cited about 20-30 new policies that were published every month, another stated
things change weekly. One administrator articulated that there had been so many changes over
the last three years that it was getting hard for staff to absorb information, let alone all the new
acronyms. These constant changes meant ongoing training, which took time away from job
responsibilities, but also meant that the effectiveness of changes became questionable, as an
administrator stated, “Things get watered down and diluted. The effectiveness gets diluted.”
Research needs to investigate how to best study change in a constantly evolving environment,
determining how to define and isolate changes. Research on systemic change also needs to note
the differences between programs and systems. This study used program-focused
implementation research for guidance, but then defined, operationalized, and measured
implementation at a systems level.

163

References
Aarons, G. A. (2009). Challenges and opportunities in implementation and change in systems
and organizations. Invited keynote presentation at Administration for Children and
Families Child Welfare Evaluation Summit, Washington, DC.
Aarons, G. A., Hurlburt, M., & McCue Horwitz, S. (2011). Advancing a conceptual model of
evidence-based practice implementation in public service sectors. Administration and
Policy in Mental Health, 38, 4-23.
Aarons, G. A., Sawitzky, A. C. (2006). Organizational climate partially mediates the effect of
culture on work attitudes and staff turnover in mental health services. Administration and
Policy in Mental Health and Mental Health Services Research, 33(3), 289-301.
Aarons, G. A., Sommerfeld, D. H., Walrath Greene, C. M. (2009). Evidence-based practice
implementation: The impact of public versus private sector organization type on
organizational support, provider attitudes, and adoption of evidence-based practice.
Implementation Science, 4 (83), 1-13. Retrieved from
http://www.implementationscience.com/content/pdf/1748-5908-4-83.pdf
American Psychiatric Association (1994). Diagnostic and statistical manual of mental disorders
(4
th
ed.). Washington, DC: American Psychiatric Association.
Bradbury, D. & Eliot, M. (1956). Extended history of the children’s bureau. Retrieved from:
http://www.ssa.gov/history/pdf/child1.pdf
California State Auditor (2012). Los Angeles county department of children and family services
management instability hampered efforts to better protect children. Retrieved from
http://www.bsa.ca.gov
Chakrabarti, A. K. (1974). The role of champion in product innovation. California Management
Review. 58-62.

164

Chan, S. (2001). Complex Adaptive Systems. [PDF document]. Retrieved from Lecture Notes
Online Web site:
http://web.mit.edu/esd.83/www/notebook/Complex%20Adaptive%20Systems.pdf
Chaudoir, S. R., Dugan, A. G., & Barr, C. H. (2013). Measuring factors affecting
implementation of health innovations: A systemic review of structural, organizational,
provider, patient, and innovation level measures. Implementation Science, 8 (22), 8-22.
Child Welfare Information Gateway. (2013a). How the child welfare system works. Washington,
DC: U.S. Department of Health and Human Services, Administration for Children and
Families. Retrieved from https://www.childwelfare.gov/pubs/factsheets/cpswork.pdf
Child Welfare Information Gateway. (2013b). Foster care statistics 2011. Washington, DC: U.S.
Department of Health and Human Services, Administration for Children and Families.
Retrieved from https://www.childwelfare.gov/pubs/factsheets/foster.pdf
Child Welfare Information Gateway. (2012). How Federal legislation impacts child welfare
service delivery. Washington, DC: U.S. Department of Health and Human Services,
Administration for Children and Families. Retrieved from
https://www.childwelfare.gov/pubs/impacts/
Child Welfare Information Gateway. (2011). Definitions of child abuse and neglect. Washington,
DC: U.S. Department of Health and Human Services, Administration for Children and
Families. Retrieved from
https://www.childwelfare.gov/systemwide/laws_policies/statutes/define.pdf
Child Welfare Information Gateway (2008a). What is child abuse and neglect. Washington,
DC: U.S. Department of Health and Human Services, Administration for Children and
Families. Retrieved from www.childwelfare.gov/pubs/factsheets/whatiscan.cfm

165

Child Welfare Information Gateway. (2008b). Differential response to reports of child abuse and
neglect. Washington, DC: U.S. Department of Health and Human Services,
Administration for Children and Families. Retrieved from
https://www.childwelfare.gov/pubs/issue_briefs/differential_response/differential_respons
e.pdf
Child Welfare Information Gateway. (n.d.). Achieving and maintaining permanency overview.
Washington, DC: U.S. Department of Health and Human Services, Administration for
Children and Families. Retrieved from
https://www.childwelfare.gov/permanency/overview/
Child Welfare Information Gateway (n.d.b) Glossary, r, recurrence. Retrieved from
https://www.childwelfare.gov/admin/glossary/glossaryr.cfm
Child Welfare League of America. (n.d.). Child welfare history. Retrieved from
http://www.cwla.org/whowhat/history.htm
Conley, A. (2007). Differential response: A critical examination of a secondary prevention
model. Children and Youth Services Review, 29, 1454-1468.
Corso, P. S., Edwards, V. J., Fang, X., & Mercy, J. A. (2008). Health-related quality of life
among adults who experienced maltreatment during childhood. American Journal of
Public Health, 98(6), 1094-1100.
Coulton, C. J. (2005). The place of community in social work practice research: Conceptual and
methodological developments. Social Work Research, 29(2), 73 – 86.
County of Los Angeles, (n.d). Cities. Retrieved from
http://www.lacounty.gov/wps/portal/lac/residents/cities

166

County of Los Angeles Department of Children and Family Services (2009). Fact sheet child
welfare services calendar year 2009. Retrieved from
http://www.lacdcfs.org/aboutus/fact_sheet/DRS/December2009/CY_2009_Fact_Sheet.ht
m
County of Los Angeles Department of Children and Family Services (2010). Fact sheet child
welfare services calendar year 2010. Retrieved from
http://www.lacdcfs.org/aboutus/fact_sheet/DRS/December2010/CY_2010_Fact_Sheet.ht
m
County of Los Angeles Department of Children and Family Services (2011). Fact sheet child
welfare services calendar year 2011. Retrieved from
http://www.lacdcfs.org/aboutus/fact_sheet/DRS/CY%202011%20Fact%20Sheet/CY_201
1_Fact_Sheet.htm
County of Los Angeles Department of Children and Family Services (2012). Fact sheet child
welfare services calendar year 2012. Retrieved from
http://www.lacdcfs.org/aboutus/fact_sheet/DRS/CY%202012%20Fact%20Sheet/CY_201
2_Fact_Sheet.htm
Courtney, M. E., Dworsky, A., & Napolitano, L. (2013). Providing foster care for young adults:
Early implementation of California’s Fostering Connections Act. Chicago: Chapin Hall at
the University of Chicago.
Curley, A. M. (2010). Relocating the poor: Social capital and neighborhood resources. Journal
of Urban Affairs, 32(1), 79-103.
Currie, J., & Widom, C. S. (2010). Long-term consequences of child abuse and neglect on adult
economic well-being. Child Maltreatment, 15(2), 111-120.
Damanpour, F. (1992). Organizational size and innovation. Organization Studies, 13(3), 375-
402.

167

Daro, D., Budde, S., Baker, S., Nesmith, A., & Harden, A. (2005). Community partnerships for
protecting children: Phase II outcome evaluation. Chapin hall working paper. Chicago,
IL: The Chapin Hall Center for Children at the University of Chicago.
DePanfilis, D. & Salus, M. K. (2003). Child protective services: A guide for caseworkers.
Washington, D. C. : U. S. Department of Health and Human Services.
Detlaff, A. J., & Rycraft, J. R. (2008). Deconstructing disproportionality: Views from multiple
community stakeholders. Child Welfare, 87(2), 37-58.
Drake, R. E., Goldman, H. H., Leff, H. S., Lehman, A. F., Dixon, L., Mueser, K. T., Torrey,
W.C. (2001). Implementing evidence-based practices in routine mental health service
settings. Psychiatric Services, 52, 179–182.
Duke Corporate Education (2005). Leading from the center influencing and collaborating for
results. Chicago, IL: Dearborn Trade Publishing.
Durlak, J. A., & DuPre, E.P. (2008). Implementation matters: A review of research on the
influence of implementation on program outcomes and the factors affecting
implementation. American Journal of Community Psychology, 41, 327–350.
Dusenbury, L., Brannigan, R., Falco, M. & Hansen, W. B. (2003). A review of research
on fidelity of implementation: Implications for drug abuse prevention in school settings.
Health Education Research, 18(2), 237-256.
Elliot, D. S. & Mihalic, S. (2004). Issues in disseminating and replicating effective
prevention programs. Prevention Science, 5(1), 47-53.
Elo, S. & Kyngas, H. (2008). The qualitative content analysis process. Journal of Advanced
Nursing, 62(1), 107-115.

168

Felitti, V. J., Anda, R. F., Nordenberg, D., Williamson, D. F., Spitz, A. M., Edwards, V., Koss,
M. P., & Marks, J.S. (1998). Relationship of childhood abuse and household dysfunction
to many of the leading causes of death in adults. American Journal of Preventive
Medicine, 14(4), 245-258.
Fixsen, D.L., Blase, K.A., Naoom, S.F., & Wallace, F., (2009). Core implementation
components. Research on Social Work Practice, 19 (5), 531-540.
Fixsen, D.L., Naoom, S. F., Blase, K.A., Friedman, R. M. & Wallace, F. (2005).
Implementation research: A synthesis of the literature. Tampa, FL: University of South
Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation
Research Network (FMHI Publication #231).
Flynn, P.M. & Brown, B. S. (2008). Co-occurring disorders in substance abuse treatment:
Issues and prospects. Journal of Substance Abuse Treatment, 34(1), 36-47.
Fluke, J. D., Shusterman, G. R., Hollinshead, D. M., & Yuan, Y. T. (2008). Longitudinal
analysis of repeated child abuse reporting and victimization: Multistate analysis of
associated factors. Child Maltreatment, 13, 76-88.
Fluke, J., Yuan, Y., & Edwards, M. (1999). Recurrence of maltreatment: An application of the
National Child Abuse and Neglect Data System. Child Abuse and Neglect, 23(7), 633-650.
Foster-Fishman, P. G., & Behren, T. R. (2007). Systems change reborn: Rethinking our
theories, methods, and efforts in human services reform and community-based change.
American Journal of Community Psychology, 39,191–196.
Fox, A., Frasch, K., & Berrick, J. D. (2000). Listening to children in foster care: An empirically
based curriculum. Berkeley, CA: University of California at Berkeley, California Social
Work Education Center.

169

Fox, D. P., Gottfredson, D. C., Kumpfer, K. K. & Beatty, P. D. (2004). Challenges in
disseminating model programs: A qualitative analysis of the strengthening
Washington D.C. families program. Clinical Child and Family Psychology Review,7 (3),
165-176.
Fuller, B. E., Rieckmann, T., Nunes, E. V., Miller, M., Arfken, C., Edmundson, E., & McCarty,
D. (2007). Organizational readiness for change and opinions toward treatment
innovations. Journal of Substance Abuse Treatment, 33(2), 183-192.
Generations United (2007). Every child deserves a permanent family: Featuring new data on
latino foster children subsidized guardianship as a common sense solution for children in
long-term relative foster care. Retrieved from
http://www.gu.org/LinkClick.aspx?fileticket=iozTC4LEmOg%3D&tabid=157&mid=606
Glisson, C., & James, L. R. (2002). The cross-level effect of culture and climate in human
service teams. Journal of Organizational Behavior, 23 (6), 767-794.
Golafshani, N. (2003). Understanding reliability and validity in qualitative research. The
Qualitative Report, 8(4), 597-607. Retrieved from http://www.nova.edu/ssss/QR/QR8-
4/golafshani.pdf
Goldman, J., Salus, M. K., Wolcott, D., & Kennedy, K. Y. (2003). A coordinated response to
child abuse and neglect: The foundation for practice. Retrieved from
https://www.childwelfare.gov/pubs/usermanuals/foundation/foundationi.cfm
Goodman, C. C. & Silverstein, M. (2006). Well-being among custodial and coparenting
families grandmothers raising grandchildren: Ethnic and racial differences. Journal of
Family Issues, 27, 1605-1626.
Gotham, H. J. (2004). Diffusion of mental health and substance abuse treatments: Development,

170

dissemination, and implementation. Clinical Psychology: Science and Practice, 11, 160-
176.
Gotham, H. J. (2006). Advancing the implementation of EBPs into clinical practice: How do we
get there from here? Professional Psychology: Research and Practice, 37, 606-613.
Gotham, H.J., Claus, R.E., Selig, K., & Homer, A.L. (2010). Increasing program capability to
provide treatment for co-occurring substance use and mental disorders: Organizational
characteristics. Journal of Substance Abuse Treatment, 38(2), 160-169.
Graham, J. R., & Barter, K. (1999). Collaboration: A social work practice method. Families in
Society: The Journal of Contemporary Social Services, 80, 6-13.
Grol, R. & Grimshaw, J. (2003). From the best evidence to best practice: Effective
implementation of change in patients’ care. The Lancet, 362 (9391), 1225-1230
Hall, G.E. & Loucks, S. F. (1977). A developmental model for determining whether the
treatment is actually implemented. American Educational Research Journal, 14(3), 263-
276.
Harn, B., Parisi, D., & Stoolmiller, M. (2013). Balancing fidelity with flexibility and fit: What
do we really know. Exceptional Children, 79(2), 181-193.
Hayslip, B., Kaminski, P. L. (2005). Grandparents raising their grandchildren: a review of the
literature and suggestions for practice. The Gerontologist, 45(2), 262-269.
Healthy City (2010) Who we are. Retrieved March 1, 2010, from
http://healthycity.org/c/ahc/sc/who_we_are
Heath, H. & Cowley, S. (2004). Developing a grounded theory approach: a comparison of
Glaser and Strauss. International Journal of Nursing Studies, 41, 141-150.

171

Henggeler, S. W., Chapman, J. E., Rowland, M. D., Halliday-Boykins, C. A., Shackelford, J. &
Schoenwald, S. K. (2008). Statewide adoption and initial implementation of contingency
management for substance-abusing adolescents. Journal of Consulting and Clinical
Psychology, 76(4), 556–567.
Hsieh, H. F., & Shannon, S. E. (2005). Three approaches to qualitative content analysis.
Qualitative Health Research, 15, 1277-1288.
Inter-Agency Council on Child Abuse and Neglect. (2012). The state of child abuse in los
angeles county. Los Angeles, CA.
Klein, K. J., & Knight, A. P. (2005). Innovation implementation: Overcoming the challenge.
Current Directions in Psychological Science, 14(5), 243-246.
Klein, K. J. & Sorra, J. S. (1996). The challenge of innovation implementation. The Academy of
Management Review, 21 (4), 1055-1080.
Kolehmainen-Aitken, L. R. (2001) Decentralization and human resources: Implications and
impact. Human Resources Development Journal, 2 (1), 1-28. Retrieved from
http://www.who.int/hrh/hrdj/en/index2.html
Kreger, M. Brindis, C.D., Manuel, D. M., & Sassoubre, L. (2007). Lessons learned in systems
change initiatives: Benchmarks and indicators. American Journal of Community
Psychology, 39, 301–320.
Kubisch, A. C., Auspos, P., Brown, P., Chaskin, R., Fulbright-Anderson, K., & Hamilton, R.
(2002). Voices from the field II reflections on comprehensive community change,
Washington , D. C. : The Aspen Institute.
Lipsky, M. (1971). Street-level bureaucracy and the analysis of urban reform. Urban Affairs
Review, 6, 391-409.

172

Longoria, R. A. (2005). Is inter-organizational collaboration always a good thing? Journal of
Sociology and Social Welfare, 32 (3), 123-138.
Los Angeles County Department of Children and Family Services. (2009). CWSH glossary.
Los Angeles, CA.
Los Angeles County Department of Public Health. (2009). Key indicators of health by service
planning area. Los Angeles, CA.
Mallon, G. P., & McCartt Hess, P. M. (2005). Child welfare for the twenty-first century: A
handbook of practices, policies, and programs. Chichester, NY: Columbia University
Press.
Marts, E. J., Lee, E. K., McRoy, R., & McCroskey, J. (2008). Point of engagement: Reducing
disproportionality and improving child and family outcomes. Child Welfare, 87(2), 335-
358.
McCroskey, J. (2008). Los angeles point of engagement evaluation, Unpublished document.
McCroskey, J. (2003). Walking the collaboration talk ten lessons learned from the Los
Angeles County Children’s Planning Council. Los Angeles, CA: Los Angeles
County Children’s Planning Council.
McCroskey, J. (2001). What is family preservation and why does it matter? Journal of Family
Strengths, 5(2), 1-24. Retrieved from
http://digitalcommons.library.tmc.edu/jfs/vol5/iss2/4
McCroskey, J. Christie, T. A., Lorthridge, J., Chambers, R., Pecora, P. J., Azzam, T…Wold, C.
(2009a). Prevention initiative demonstration project (PIDP) year one evaluation
summary report. Retrieved from
http://www.casey.org/Resources/Publications/pdf/pidp_fr.pdf

173

McCroskey, J., Furman, W., & Yoo, J. (2007). Finding common ground in community based
child welfare: Qualitative data on los angeles county’s point of engagement initiative.
Unpublished manuscript.
McCroskey, J., Yoo, J., Lorthridge, J., Chambers, R., Carter, S., Cienfuegos-Mercado, Y.
(2009b).
A report describing implementation of practice changes in Department of Children and
Family Services regional offices. Unpublished manuscript.
McGowan, B. (2005). Historical evolution of child welfare services. In G. P. Mallon, & P. M.
Hess (Eds.), Child welfare for the twenty-first century: A handbook of practices, policies,
and programs. (pp. 10-46). Chichester, NY: Columbia University Press.
Medjedović, I. & Witzel, A. (2007). Secondary analysis of interviews: Using codes
and theoretical concepts from the primary study [78 paragraphs]. Forum Qualitative
Sozialforschung / Forum: Qualitative Social Research, 6(1), Art. 46, Retrieved from
http://www.qualitative-research.net/index.php/fqs/article/view/507/1094
Metz, A. J. R., Blase, K., & Bowie M.A., L. (October 2007). Implementing evidence-based
practices: Six “drivers” of success. Part 3 in a series on fostering the adoption of
evidence-based practices in out-of-school time programs. Retrieved from
http://www.childtrends.org/wp-content/uploads/2013/07/2007-29EVPSuccess.pdf
Morazes, J. L., Benton, A. D., Clark, S. J., & Jacquet, S. E. (2010). Views of specially-trained
child welfare social workers. Qualitative Social Work, 9 (2), 227-247.
Morgan, P. (2005). The idea and practice of systems thinking and their relevance for
capacity development. Retrieved from
http://portals.wi.wur.nl/files/docs/SPICAD/13.%20Idea%20and

174

%20practice%20of%20systems%20thinking%20(ECDPM).pdf
Myers, J.E.B. (2011). The APSAC handbook on child maltreatment,(3
rd
ed.). Thousand Oaks,
CA: SAGE Publications, Inc.
National Center for Children in Poverty (2007). Parents’ low education leads to low income,
despite full-time employment. Retrieved from
http://www.nccp.org/publications/pdf/text_786.pdf
National Scientific Council on the Developing Child (2007). The timing and quality of early
experiences combine to shape brain architecture: Working paper #5. Retrieved from
http://www.developingchild.net
Needell, B., Webster, D., Armijo, M., Lee, S., Dawson, W., Magruder, J., Exel, M., …Lawson, J.
(2014). CCWIP reports. Retrieved from http://cssr.berkeley.edu/ucb_childwelfare
Needell, B., Webster, D., Armijo, M., Lee, S., Dawson, W., Magruder, J., Exel, M., …Lawson, J.
(2013). CCWIP reports. Retrieved from http://cssr.berkeley.edu/ucb_childwelfare
New York Society for the Prevention of Cruelty to Children (2000). The New York society for
the prevention of cruelty to children 125th anniversary 1875-2000. Retrieved from
http://www.nyspcc.org/nyspcc/history/attachment:en-us.pdf
Notz, P. (2005). Secondary qualitative analysis of interviews: A method used for gaining insight
into the work/life balance of middle managers in Germany. Forum: Qualitative Social
Research, 6(1), Art. 34, 1-10.
O’Neill Murray, K., & Gesiriech, S. (2004). A brief legislative history of the child welfare
system. Retrieved from
http://www.pewtrusts.org/uploadedFiles/wwwpewtrustsorg/Reports/Foster_care_reform/L
egislativeHistory2004.pdf

175

Osilla, K. C., Hepner, K. A., Munoz, R. F., Woo, S., & Watkins, K. (2009). Developing an
integrated treatment for substance use and depression using cognitive- behavioral therapy.
Journal of Substance Abuse Treatment, 37, 412-420.
Osland, J. (2004) Managing global change by building community. In H. Lane, M. Mendenhall,
M. Maznevski, & J. McNett (Eds.), Handbook of Global Management: A Guide to
Managing Complexity. (pp. 134-151). Oxford: Blackwell.
Patton, M. Q. (2001). Qualitative evaluation and research methods (3rd ed.). Thousand Oaks,
CA: Sage Publications, Inc.
Pinnock, H. Huby, G. Tiernay, A., Hamilton, S., Powell, A., Kielmann, T., Sheikh, A. (2009).
Is multidisciplinary teamwork the key? A qualitative study of development of respiratory
services in the UK. Journal of the Royal Society of Medicine, 102, 378-390.
Putnam-Hornstein, E. (2011). Report of maltreatment as a risk factor for injury death: A
prospective birth cohort study. Child Maltreatment, 16(3) 163-174.
Roosa, M.W., Jones, S., Tein, J-Y, & Cree, W. (2003). Prevention science and neighborhood
influences on low-income children’s development: Theoretical and methodological
issues. American Journal of Community Psychology, 31(1/2), 55-72.
Rosenberg, D. (1997). Trend analysis and interpretation: Key concepts and methods for
maternal and child health professionals. Rockville, MD: Division of Science, Education,
and Analysis, Maternal and Child Health Bureau, Health Resources and Service
Administration, U.S. Department of Health and Human Services.
Rye, C. B., & Kimberly, J. R. (2007). The adoption of innovations by provider organizations in
health care. Medical Care Research and Review, 64(3), 235-278.

176

Samuels, B. (2012). Information memorandum ACYF-CB-IM-12-04 promoting social and
emotional well-being for children and youth receiving child welfare services. Retrieved from
http://www.acf.hhs.gov/sites/default/files/cb/im1204.pdf
Schuerman, J. & Needell, B. (2009). The child and family services review composite scores:
Accountability off the track. Retrieved from
http://www.chapinhall.org/sites/default/files/Schuerman_Needell_12_15.pdf
SEIU Local 721 (2009). Reforming the Los Angeles County department of children and family
services recommendations from Los Angeles social workers. Los Angeles, CA.
Shenton, A. K. (2004). Strategies for ensuring trustworthiness in qualitative research projects.
Education for Information, 22, 63-75.
Shook Slack, K., Holl, J. L., McDaniel, M., Yoo, J., & Bolger, K. (2004). Understanding the
risks of child neglect: An exploration of poverty and parenting characteristics. Child
Maltreatment, 9, 395-408.
Simpson, D. (2006). POE stories volume 1. Los Angeles, CA: Los Angeles County Department
of Children and Family Services.
Simpson, D. D., & Flynn, P. M. (2007). Moving innovations into treatment: A stage-based
approach to program change. Journal of Substance Abuse Treatment, 33(2), 111–120.
Smith, G. C., Palmieri, P.A. (2007). Risk of psychological difficulties among children raised by
custodial grandparents. Psychiatric Services, 58(10), 1303-1310.
Social Explorer. (2013). Data dictionary. Retrieved from
http://www.socialexplorer.com/data/ACS2011_5yr/metadata/?ds=ACS11_5yr&var=B10051F00
2

177

Substance Abuse and Mental Health Services Administration (SAMHSA). (2007a). Services
integration: COCE overview paper number 6. Rockville, MD: Substance Abuse and
Mental Health Services.
Substance Abuse and Mental Health Services Administration (SAMHSA). (2007b). Systems
integration: COCE overview paper number 7. Rockville, MD.: Substance Abuse and
Mental Health Services Administration.
Title IV Social Security Act Part B Child and Family Services of 1967§ 421, 42 U.S.C.§ 621
(2013).
Title IV Social Security Act Part E Federal Payments for Foster Care and Adoption Assistance of
1980§ 470, 42 U.S.C.§ 670 (2013).
Toth, S. L., & Cicchetti, D. (2011). Frontiers in translational research on trauma. Development
and Psychopathology, 23, 353-355.
Trickett, P., Noll, J., & Putnam, F. (2011). The impact of sexual abuse of female development:
Lessons from a multigenerational, longitudinal research study. Development and
Psychopathology, 23, 453-476.
United States Department of Labor, Bureau of Labor Statistics (2014). Local area
unemployment statistics. Retrieved from http://www.bls.gov/web/laus/lauhsthl.htm
U.S. Census Bureau. (2013a). State and county quick facts. Retrieved from
http://quickfacts.census.gov/qfd/states/06/06037.html
U.S. Census Bureau. (2013b). 2007-2011 American community survey 5-year estimates, total
population, zip code tabulation areas (zcta) in California as extracted by California state
census data center, demographic research unit, department of finance. Retrieved from
http://www.dof.ca.gov/research/demographic/

178

U.S. Census Bureau (2013c). 2007-2011 American community survey 5-year estimates, selected
economic characteristics, employment status table, zip code tabulation areas in
California extracted by California state census data center, demographic research unit,
department of finance
http://www.dof.ca.gov/research/demographic/
U.S. Census Bureau. (2013d). 2007-2011 American community survey 5-year estimates,
educational attainment, educational attainment for population 25 years and over table,
zip code tabulation areas in California extracted by California state census data center,
demographic research unit, department of finance
http://www.dof.ca.gov/research/demographic/
U.S. Department of Health and Human Services, Administration for Children and Families,
Administration on Children, Youth and Families, Children’s Bureau. (2009). Child
Maltreatment 2009. Retrieved from http://www.acf.hhs.gov/programs/cb/research-data-
technology/statistics-research/child-maltreatment.
U.S. Department of Health and Human Services, Administration for Children and Families,
Administration on Children, Youth and Families, Children’s Bureau. (2012b). About
AFCARS. Retrieved from http://www.acf.hhs.gov/programs/cb/resource/about-afcars
U.S. Department of Health and Human Services, Administration for Children and Families,
Administration on Children, Youth and Families, Children’s Bureau. (2012c). The
AFCARS report Preliminary FY1 2011 Estimates as of July 2012 No. 19. Retrieved from
http://www.acf.hhs.gov/sites/default/files/cb/afcarsreport19.pdf
U.S. Department of Health and Human Services, Administration for Children and Families,
Administration on Children, Youth and Families, Children’s Bureau. (n.d.). Child and

179

family service reviews fact sheet. Retrieved from
http://www.acf.hhs.gov/sites/default/files/cb/cfsr_factsheet.pdf
VanDeusen, L., Hall, C., Gerena-Melia, M. (2010, March). Challenges in measuring
implementation success, Paper presented at the 3
rd
Annual NIH Conference on Science of
Dissemination and Implementation Methods and Measurement, Bethesda, MD.
Walsh, J. (1998). The eye of the storm: Ten years on the frontlines of new futures. Baltimore,
MD: Annie E. Casey Foundation.
Watkins, S. A. (1990). The mary ellen myth: Correcting child welfare history. Social Work,
35(6), 500-503.
Weinstein, J., Whittington, C., & Leiba, T. (Eds.). (2003). Collaboration in social work
practice. New York: Jessica Kingsley Publishers.
Wulczyn, F., Daro, D., Fluke, J., Feldman, S., Glodek, C.,& Lifanda, K. (2010). Adapting a
systems approach to child protection: Key concepts and considerations. New York:
United Nations Children’s Fund (UNICEF).
Yin, R. K. (1994). Case study research design & methods, (2
nd
ed.). Thousand Oaks, CA: Sage
Publications, Inc.

180

Appendix A
Federal Child Welfare Structure

United States Department of
Health & Human Services
Administration for Children and
Families
Administration on Children,
Youth, and Families
The Children’s Bureau Family Youth and Services Bureau
Figure A1. Federal Child Welfare Structure

181

Appendix B1
Point of Engagement Process Flowchart

REFERRAL RECEIVED BY THE HOTLINE
(SDM Response Priority Tool is used)

OPEN REFERRAL SENT TO THE
REGION FOR INVESTIGATION
(SDM Safety and Risk Assessments
are completed)
REFERRAL CONCLUSION AND DISPOSITION
Informed by Case Review or Team Decision Making Meeting

Community
Response
Low Risk -
Unfounded &
Inconclusive
(with no priors)

Families are referred
and linked to
community based
services.

DCFS referral is
closed.
Alternative Response
Low (with priors) &
Moderate/High Risk –
Inconclusive

Families are referred
and linked to a
Family Preservation
Agency that provides
Alternative Response
Services

DCFS referral is
closed.

VFM/VFR Services

Moderate/High Risk –
Substantiated

CSW talks to the family
about Voluntary Family
Maintenance Services.
Family agrees to
services and the VFM
contract is signed.
Family connected to
services through the
CASC.
Or
Voluntary placement
services considered.
 Kinship Placement
and services
provided
 Foster Placement
and resources
provided.
DCFS case is opened and
VFM/VFR services
continue for up to 6
months.
VFM with Family
Preservation Services
High Risk & Very High
Risk (with mitigated
safety factors) –
Substantiated

CSW talks to the family
about Family
Preservation (FP)
Services.

Family agrees to FP
services and is linked to
a Family Preservation
Agency.

MCPC is completed.

DCFS case is opened.
FP services continue for
6 months and may be
extended up to 12
months.

Detention
Unmitigated safety
factors -
Substantiated

Medical Exam
done.
TDM meeting held
within 72 hours.
If safety is
mitigated, child is
returned home.

If safety is not
mitigated, Kinship
placement and
resources
considered and
child is placed in
the most familiar
and least
restrictive
environment.

Medical Evidentiary Exam
Front-End Assessment on High-Risk Referrals
Domestic Violence
Mental Health
Substance Abuse
Post Detention
DCFS case opened.
An ISW is assigned.
CP is initiated. A
MAT Assessment is
initiated.
Parent/child linked to
needed services.
SDM Reunification
Assessment done A
PPC is held.
case will continue
until family is
reunified or legal
guardianship/
adoption is
established.
Figure B1. Point of Engagement Process Flowchart

182

Appendix C
POE Components
30

Overview
POE is a multidisciplinary, family-centered approach that enlists the support of community from
both providers and citizens to prevent and address child abuse issues. POE is characterized by a
collaborative approach to partnerships that supports seamless and timely transfer of
responsibility from initial investigations to actual service delivery.
This partnership philosophy guides organization within the Compton office so that everyone in
the building, from receptionists to social workers, see themselves as members of a team. This
approach promoted more effective cooperation for all involved to engage families, provides
comprehensive assessments and individualized treatment planning, and assures that families
receive the services they need.
Perhaps most importantly, the team reaches outside the child welfare office to include CBOs,
faith-based groups, local businesses and community leaders who care about children.
Although a few CBOs receive contracts for their intensive involvement in assessment and
treatment, literally hundreds of groups in the community should count themselves as part of the
essential community safety net that works with DCFS to support these children and their
families. Part of the value of POE is the way that it harnesses the power of social networks.

Core Elements
POE utilizes a multi-disciplinary team decision-making approach that includes the family in the
process of selecting and planning for the delivery of needed services. POE actively engages
resources within DCFS and other County services such as the Departments of Mental Health, the
Department of Health, the Department of Probation, and the Department of Public Social
Services along with the Sheriff’s Office.
The team has identified key local resources such as churches, food banks, Women Infant and
Children (WIC) food programs, child care and other programs that provide essential resources
for families – all of which are involved in the extended POE network. These community
resources also have special relationships with Compton City offices and the local schools that
serve this population.
POE provides a faster response for the provision of services and, through the use of a team
approach, actively placing an emphasis on shared decision-making and comprehensive case

30
Originally Appendix A, Practice Principles Associated with Point of Engagement POE Components, Los Angeles
Point of Engagement Evaluation (2008); information also featured in:
Marts, E.J., Lee, R. McRoy, R. & McCroskey, J. (2008). Point of engagement: reducing disproportionality and
improving child and family outcomes. Child Welfare.
McCroskey, J. , Furman, W. & Yoo, J. (2008). Finding common ground in community based child welfare:
Qualitative data on Los Angeles County’s point of engagement initiative. Children and Youth Services Review

183

evaluation and investigation. The following components of the POE Model establish a seamless
service delivery continuum that integrates departmental programs and initiatives so that children
are safe and families receive the necessary services in a timely manner.
 Informal resources are provided for families who are “evaluated out” at the hotline, along
with follow up with families living in Compton in order to offer referrals and facilitate
access to informal resources from CBOs and faith-based organizations. Compton is
unique among the DCFS offices in LA County in offering information on informal
resources to all families identified by calls to the child abuse hotline.
 Differential response provides a community-based network of formal and informal
support services for children and families assessed with an inconclusive child abuse and
neglect referral in order to divert families from potentially entering the child protective
system.
 Alternative response provides a community-based network of formal and informal
support and services for children and families with multiple inconclusive child abuse and
neglect referrals in order to divert families from further disruptions and entering the child
protective system.
 Voluntary services provide voluntary family maintenance/reunification and family
preservation services to families that have been assessed to be at moderate to high risk
and for whom a child protective case is opened.
 Intensive services workers conduct child safety conferences shortly after detention to
assess for possible return of children and/to connect children and families to services
immediately following detention.
 Team-decision-making/child safety conferences provide a forum for the family, relatives,
friends, social workers and community service providers to share information,
observations, and concerns. The team identifies strengths and resources to assist in the
development of an appropriate service plan for the family.
 Emergency response investigations of referrals on open cases provide consistency of
emergency response investigations to reduce the likelihood of future abuse.

When child abuse/neglect allegations have been substantiated, the service delivery model
involves the following specific strategies and steps:
1) Initiate concurrent planning: While disclosing and recognizing the possibility of
adoption, family reunification is emphasized.
2) Review of any mental health/substance abuse or domestic violence issues
3) Assign Intensive Service Workers (ISW) who link families to services, work on
reunification within 30 days, stabilize the immediate risks to the children and family
while beginning the process of obtaining basic identifying data such as birth records, the
father of the child, and accessing eligible benefits such as social security, along with
other case management protocol.
4) Identify relative caregivers if needed.

184

5) Provide Kinship support which assists in completing the federal eligibility application,
eligibility determination for TANF as a Non-Needy caregiver, facilitating community
resources, referral to kinship training and discussion of adoption and guardianship.
6) Identify non-offending parents, particularly fathers.
7) Refer children and families to Multi-disciplinary Assessment teams (MATS) to assess
children for mental health, developmental and educational issues within 30-45 days after
placement. These efforts accomplish the following: determine treatment needs; stabilize
the relevant relationships; address the issues for placement assess biological parents for
mental health issues, evaluate current caregivers for suitability and for permanency if
children need a permanent family and develop back-up family members who could step
up if needed.
8) Complete the Multi-disciplinary Assessment Team process with a Team Decision
Meeting to update the service plan, link children and family to appropriate services and
conduct an adoption disclosure.
Collaboration between DCFS and CBO partners was mentioned most often as having changed
dramatically since the advent of POE. CBOs participate in assessment processes, case planning,
ongoing service delivery and monitoring of all types of cases (referrals with no open DCFS case,
voluntary, and court ordered).

Key Practice Principles
POE requires full disclosure to and collaboration with families. Workers must communicate
openly and respect the family’s ability to make decisions on their own behalf. This kind of
communication is essential in order to assess family strengths and develop individualized service
plans.
Families are seen as full partners in the process, rather than as “cases” who need others to solve
problems for them. Linking families to services more quickly because the internal steps have
been streamlined – agreement on an overall vision and goals that everyone in the office buys into
means that people can work together rather than focusing only on their own contribution,
protecting their own turf and holding up the progress for families.
There is an authentic emphasis upon changing the relationship between DCFS and community
partners; and as a result this new atmosphere has affected relationships among local CBOs. They
reported that community-based organizations are working together to increase cross-referrals
between agencies, looking at one another differently, and relying more on other agencies for
support. There should be fewer adversarial relationships between CBOs and more focus on
advocacy for clients. Community-based agencies have increased knowledge of DCFS processes
and procedures, and have increased trust and respect for DCFS staff.

185

Appendix D

Informed Consent and Script for 2012 Focus Groups

POINT OF ENGAGEMENT: CRITICAL SUCCESS FACTORS VERBAL CONSENT FOR
DIRECT SERVICE PROVIDERS

Hi, my name is Jaymie Lorthridge and I am conducting research on Point of Engagement (POE)
and the factors that are critical to the success of POE. I am a PhD Candidate at the University of
Southern California School of Social Work and I am being advised by Dr. Jacquelyn
McCroskey. You are eligible to participate in this research because you are an employee of
DCFS whose job duties include coordination of intensive services or you are an employee of
DCFS that receives cases from the front-end workers in your office. Your participation is
voluntary, and you can stop participating at any time without any consequences to you, nor will
your supervisor know if you have participated or not.

You are being asked to take part in the research because I and those persons responsible for
program and policy planning at DCFS are trying to learn more about your work, specifically
about provision of intensive services and the impact that intensive services make. This research
builds on prior interviews and focus groups that were conducted with the Executive Director,
Deputy Directors, RAs, ARAs, SCSWs, CSWs and other classified personnel in 2009. The prior
interviews and focus groups sought to capture the work surrounding child abuse and
maltreatment prevention in Los Angeles County related to POE and the Prevention Initiative
Demonstration Project (PIDP). Today you will be asked to respond to questions centering on
topics such as your experiences within your office around servicing families and coordination of
intensive services.

De-identified information, or information without names or work locations attached, from
today’s session may provide valuable information for academic research and for DCFS policy
and program managers. This means that in addition to a final report that will be prepared for the
DCFS Executive Team, the de-identified information will be used for a dissertation and may be
used in the future for academic journal articles or presentations at academic conferences. Your
privacy will be maintained in all reports, academic research, journal articles or presentations.

Participant(s) agreed to allow session information to be used for future journal articles or
Presentations

Participant(s) agreed to allow session information to be used for academic research and
DCFS reporting only.

186

Date of Preparation: 3/18/12 – Verbal Consent Script
To ensure accuracy I will be using an audio recorder. The recordings of this session will be
turned into notes and then stored. In the notes, your name will not be used; rather you will be
identified with a code, replacing your name. You have the right to review the tapes containing
your data. Notes and audiotapes will be stored in a locked file cabinet in Dr. McCroskey’s
office. The researcher and Dr. McCroskey will be the only persons who will have access to the
data. At the end of six years, the audio tapes will be destroyed. All participants in the session
are asked to respect the confidentiality of information shared by other participants. This means
that you should not discuss this session with any other persons not in this room today, nor should
you repeat any comments made during today’s session. While it is hoped that confidentiality is
maintained by all other participants, I cannot guarantee that other participants will not discuss
session proceedings. If you do not wish to participate in a group session an individual interview
can be arranged. If you do not wish to be audiotaped you may indicate your wishes.

Participant(s) agreed to be audiotaped.
Participant(s) did not agree to be audiotaped, audiotape will not be used.

For sessions involving more than one staff member
Participants agreed to focus group
Participant did not wish to be in a focus group, individual interview arranged

Does anyone have any questions?

Does everyone agree to participate in this research study?

Does everyone have an hour and a half for participation?

I will now start the session, when you answer the following questions you are consenting to
participate in the research.

187

Date of Preparation: 3/18/12 – Verbal Consent Script
DCFS Direct Services Worker Session

Read Verbal Consent Statement

Date:___________________________________

I want to talk with you about the ways in which you provide services to families and how you
coordinate intensive services within your office. I will use a set of questions to guide our
conversation about what you are doing, how your activities are going and what you view as
important. Please feel free to give me your opinions about these topics or also add any other
information that you feel is important to helping me understand your work. First I am going to
ask some questions about your work background to help me understand if your work is in any
related to your background. When the notes from this session are summarized your identifying
information will not be used.

May we begin? Is it all right if I turn on the tape recorder?

1. To start with, I am Jaymie Lorthridge and I am a PhD Candidate at the University of
Southern California’s School of Social Work. Can you tell me your initials, your job
title, how long you have been in your current position and how long you have been at
DCFS? (probe: if they have been in the current position for a short amount of time and
have significant time as a DCFS employee ask what their previous job titles were).

2. Can you give me your definition of what intensive services are?

3. Are you able to provide or coordinate intensive services as you have described them
(probe: why or why not)

4. What do you feel is the purpose of intensive services (probe: is this a change in how
services were previously provided and if so what has changed)?

5. In your experience what ensures or stops the intensive services you provide from
fulfilling their purposes?

6. What, if anything, are the differences in processing a case where intensive services have
been provided vs. cases where intensive services have not been provided?

7. How, if at all, have your job duties changed as a result of the focus on providing intensive
services?

188

8. How, if at all, has the flow of cases between the front-end and the back-end changed as a
result of the focus on providing intensive services (probe: do you feel that the changes
benefit or do not benefit families and why, can you give case examples)?

9. How, if at all, has communication between the front and back-end workers changed as
result of the focus on providing intensive services (probe: do you feel that the changes
benefit or do not benefit families and why, can you give case examples)?

10. At what point in the case processing timeline are intensive services provided?

11. At what point in the case processing timeline do you receive the case (probe: as
investigation is ongoing, post-investigation)?

12. According to policies is this when you are supposed to receive the case (probe: if no,
when you are supposed to receive the case, why do you think you don’t receive the case
within the designated period, how do delays in receiving the case affect your work)?

13. What forms or documents do you use to help in the provision of intensive services?

14. The Intensive Services Procedural Guide states that, “…timely provision of intensive
services,” helps reduce the risk of detention. Can you give examples of how your
services have helped to reduce the risk of detention or if you feel your services do not
reduce the risk what would need to change so your services could reduce the risk of
detention?

15. The Procedural Guide also states that when detention is unavoidable intensive services
helps to ensure collaborative planning. Who do you collaborate with? How do you
collaborate (probes: do you collaborate with other workers, CBOs, foster care providers,
do you collaborate via TDMs, MAT meetings?)

16. What helps or hinders your work around collaborative planning?

17. What impact, if any, has provision of intensive services made on dependency
investigations (probe: can you give case examples)?

18. Is there anything you would like to add? Thank you for your time.

189

Appendix E

Template for Questions 1 & 2 - 2008-2009 Data

1.) What organizational and actor level indicators of implementation success were present in each
office

Indicators of POE Implementation Success
 Organizational - Resource adjustments such that staff were allocated where needed [ER &
ISW/Bridge]
Organizational level indicators included resource adjustments such that staff was allocated where needed.
For POE, and for this study, resource adjustments were operationalized as adequate staff in the
Emergency Response units, and available staff to function in the bridging role meant to facilitate more
efficient case transfer between the Emergency Response units and the Family Maintenance/Family
Preservation and Family Reunification units
 Actor – Climate - changes (collaboration with other staff, collaboration with families, collaboration
with community organizations, intervening at the lowest level possible) perceived as positive
Climate was defined as collective, i.e. pervasive within an office, positive perceptions of changes. The
positive perceptions of actors increases the probability of the actor’s utilizing the practice change
(Henggeler et al., 2008). Changes were operationalized as collaboration with other staff, collaboration
with families, collaboration with community organizations, and intervening at the lowest level possible.
 Actor - Leader and/or champion facilitates practice changes
A leader and/or champion is an actor who facilitates implementation of practice change by accumulating
buy-in (Chakrabarti, 1974) and dedicating attention to the process of change (Demiris, Oliver,
Wittenberg-Lyles, 2009; Durlak & DuPre, 2008; Pinnock et al., 2009).

190

Date: Office: Coder: Jaymie Lorthridge
Domain A: Resource Adjustments (Staff allocation)
Sub-domain A-1: Investigatory staff [ER, Front-line, Up-front]
Sub-domain A-2: Bridging Staff [ISW or ISW like coverage]
Sub-domain A-3 Other notes
RA ARA SCSW CSW
Sub-domain A-1 (Investigatory
Staff):

Sub-domain A-1 (Investigatory
Staff):

Sub-domain A-1
(Investigatory Staff):

Sub-domain A-1 (Investigatory
Staff):


Sub-domain A-2 (Bridging
Staff):

Sub-domain A-2 (Bridging Staff):

Sub-domain A-2 (Bridging
Staff):

Sub-domain A-2 (Bridging
Staff):

Sub-domain A-3 ():

Sub-domain A-3 ():

Sub-domain A-3 ():

Sub-domain A-3 ():

Quotes:

191

Office: Coder: Jaymie Lorthridge
Domain B-1: Climate (positive perceptions of change)
Sub-domain B-1a: Collaboration with other staff
Sub-domain B-1b: Collaboration with families
Sub-domain B-1c: Collaboration with community organizations
Sub-domain B-1d: Intervening at the lowest level possible
Sub-domain B-1e: Other notes
RA ARA SCSW CSW
Sub-domain B-1a (Collaboration
with other staff):

Sub-domain B-1a
(Collaboration with other
staff):


Sub-domain B-1a
(Collaboration with other
staff):

Sub-domain B-1a (Collaboration
with other staff):

Sub-domain B-1b (Collaboration
with Families):

Sub-domain B-1b
(Collaboration with
Families):

Sub-domain B-1b
(Collaboration with
Families):

Sub-domain B-1b (Collaboration
with Families):

Sub-domain B-1c (Collaboration
with CBOs):

Sub-domain B-1c
(Collaboration with CBOs):

Sub-domain B-1c
(Collaboration with CBOs):

Sub-domain B-1c (Collaboration
with CBOs):

Sub-domain B-1d (Low level
intervention):

Sub-domain B-1d (Low
level intervention):

Sub-domain B-1d (Low level
intervention):

Sub-domain B-1d (Low level
intervention):

Sub-domain B-1e ():

Sub-domain B-1e ():

Sub-domain B-1e ():

Sub-domain B-1e ():

Quotes:

192

Office: Coder: Jaymie Lorthridge
Domain B-2: Leader/champion (accumulates buy-in, attends to/nurtures change)
Sub-domain B-2a: Leader
Sub-domain B-2b: Champion
Sub-domain B-2c: Other Notes
RA ARA SCSW CSW
Sub-domain B-2a (Leader):

Sub-domain B-2a (Leader):

Sub-domain B-2a (Leader):

Sub-domain B-2a (Leader):

Sub-domain B-2b (Champion):

Sub-domain B-2b
(Champion):

Sub-domain B-2b
(Champion):

Sub-domain B-2b (Champion):

Sub-domain B-2c ():

Sub-domain B-2c ():

Sub-domain B-2c ():

Sub-domain B-2c ():

Quotes:

Office: Coder: Jaymie Lorthridge
Domain C: Historical Relationships
Sub-domain C-1: Community-based organizations
Sub-domain C-2: Other Notes
RA ARA SCSW CSW
Sub-domain C-1 (CBOs):

Sub-domain C-1 (CBOs):

Sub-domain C-1 (CBOs):

Sub-domain C-1
(CBOs):

Sub-domain C-2 ():

Sub-domain C-2 ():

Sub-domain C-2 ():

Sub-domain C-2 ():

Quotes:

193

Appendix F
2009 Referrals by Office

Figure F1. 2009 Referrals by Office
Office
8
Office
11
Office
1
Office
13
Office
6
Office
14
Office
15
Office
3
Office
10
Office
2
Office
9
Office
7
Office
12
Office
4
Office
5
Referrals 3,717 4,213 4,332 4,694 6,657 7,349 7,493 7,799 8,864 9,093 9,095 9,131 9,670 11,033 11,679
0
2,000
4,000
6,000
8,000
10,000
12,000
14,000
2009 Referrals by Office

194

Appendix G
2009 Removals by Office

Figure G1. 2009 Removals by Office
Office 7 Office 8 Office 1
Office
11
Office
13
Office 6
Office
10
Office 9 Office 3 Office 2
Office
15
Office
14
Office
12
Office 4 Office 5
Removals 54 188 194 347 379 415 473 503 519 647 656 736 825 978 1007
0
200
400
600
800
1000
1200
2009 Removals by Office

195

Appendix H
2009 Children in Out-of-Home Care by Office

Figure H1. 2009 Children in Out-of-Home care by Office
Office
1
Office
7
Office
8
Office
6
Office
2
Office
10
Office
13
Office
3
Office
11
Office
15
Office
4
Office
14
Office
9
Office
12
Office
5
Children in out of home care 334 530 693 709 879 893 905 1,153 1,224 1,236 1,242 1,361 1,372 1,631 2,073
0
500
1000
1500
2000
2500
2009 Children in Out of Home Care by Office

Abstract (if available)

Abstract Meaningful child welfare system practice change is challenging. The complexity of the child welfare system makes change challenging. When stakeholders are able to wade through the complexity and change practice, they are under pressure to quickly establish the significance of the change. The dire consequences of maltreatment and the urgency of family needs demand immediate results. Stakeholders turn to child outcomes when assessing change impact. Yet the assessment of change needs to start with an examination of implementation. After successful implementation is documented then trends in outcomes can be tracked. ❧ This study examined implementation of a practice change at the systems level. Point of Engagement (POE) was implemented within the Los Angeles County Department of Children and Family Services (DCFS). POE was a philosophical shift that informed a redesigned case flow process (Marts, Lee, McRoy & McCroskey, 2008). The change was intended to facilitate early engagement of families whose children were likely to be placed in foster care, and to preserve the family by intervening with appropriate services as early possible. DCFS offices served as the unit of analysis and POE implementation was staggered by office. At the time of the study, POE had been implemented between two to six years. Literature suggests that implementation occurs over two to four years (Fixsen et al., 2009), thus regional offices were informative units of analysis. ❧ Implementation literature has found that organizational and actor level characteristics and environmental influences affect implementation (Henggeler et al., 2008

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

Exploring the implementation of the Indian Child of Welfare Act of 1978

PDF

A system framework for evidence based implementations in a health care organization

PDF

Racial/ethnic diversity and disparities in the child welfare system: a multilevel perspective

PDF

Multilevel influences on organizational adoption of innovative practices in children's mental health services

PDF

Immigrants at a loss: the need for services that promote child well-being among Latino families with child welfare contact

PDF

How organizations adapt during EBP implementation

PDF

An examination of child protective service involvement among offspring born to young mothers in foster care

PDF

Fairness in machine learning applied to child welfare

PDF

Strength-Based Reporting: a trauma-informed practice for mandated reporters, to address behavioral health concerns in children at risk of child welfare involvement

PDF

Father engagement in the child welfare system: a promising practice study

PDF

Child welfare turnover: an evaluation study of work-life balance practices

PDF

Effective leadership practices used by middle school principals in the implementation of instructional change

PDF

How urban high school principals implement social and emotional learning (SEL)

PDF

A case study of the instructional leader's role in leading change: preparing for the implementation of Common Core State Standards in elementary schools

PDF

A relational approach to discipline: a comparative case study of restorative justice implementation in US secondary schools

PDF

Interagency collaboration: cultivating resources and strengthening substance use service delivery in child welfare

PDF

The link between maternal depression and adolescent daughters' risk behavior: the mediating and moderating role of family

PDF

Outcomes-based contracting through impact bonds: ties to social innovation, systems change, and international development

PDF

Community development agreements: addressing inequality through urban development projects

PDF

School accountability and reform in Oregon: effecting system change with Achievement Compacts

Asset Metadata

Creator Lorthridge, Jaymie (author)

Core Title Assessing implementation of a child welfare system practice change

School School of Social Work

Degree Doctor of Philosophy

Degree Program Social Work

Publication Date 11/05/2014

Defense Date 03/07/2014

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag Child welfare,implementation,implementation assessment,OAI-PMH Harvest,practice change,systemic change,systems change

Format application/pdf (imt)

Language English

Contributor Electronically uploaded by the author (provenance)

Advisor McCroskey, Jacquelyn (committee chair), Hurlburt, Michael S. (committee member), Sloane, David C. (committee member)

Creator Email lorthrid@gmail.com

Permanent Link (DOI) https://doi.org/10.25549/usctheses-c3-411437

Unique identifier UC11296371

Identifier etd-Lorthridge-2455.pdf (filename),usctheses-c3-411437 (legacy record id)

Legacy Identifier etd-Lorthridge-2455.pdf

Dmrecord 411437

Document Type Dissertation

Format application/pdf (imt)

Rights Lorthridge, Jaymie

Type texts

Source University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Access Conditions The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the a...

Repository Name University of Southern California Digital Library

Repository Location USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA