Close
About
FAQ
Home
Collections
Login
USC Login
Register
0
Selected
Invert selection
Deselect all
Deselect all
Click here to refresh results
Click here to refresh results
USC
/
Digital Library
/
University of Southern California Dissertations and Theses
/
How organizations adapt during EBP implementation
(USC Thesis Other)
How organizations adapt during EBP implementation
PDF
Download
Share
Open document
Flip pages
Contact Us
Contact Us
Copy asset link
Request this asset
Transcript (if available)
Content
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 1
How organizations adapt during EBP implementation
Dissertation by
Rebecca Lengnick-Hall, MSSW, MPAff
Doctor of Philosophy (Social Work)
Degree Conferral Date August 2019
Faculty of the USC Graduate School
University of Southern California
Dissertation Guidance Committee
Michael Hurlburt, Ph.D. (Chair)
Benjamin Henwood, Ph.D.
Peer Fiss, Ph.D.
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 2
Acknowledgements
Getting a Ph.D. is truly a team effort and I am so grateful for the mentor, peer, and
personal support that I have had these past six years. To Drs. Mike Hurlburt and Greg Aarons:
Thank you for pushing, challenging, trusting, encouraging, and understanding me. You changed
the trajectory of my career and I will continue to bug you for many years to come. To Dr. Ben
Henwood: Thank you for showing me how fun qualitative research is and how fun qualitative
people are. To Drs. Michàlle Mor Barak, Alicia Bunger, and Mόnica Pérez Jolles: Thank you for
being strong, trailblazing, and caring female mentors. To Dr. Peer Fiss: Thank you for serving on
my committees and for getting me genuinely excited about organizational theory. To Dr. Catie
Willging, Dr. Amy Green, and other members of the CASRC research team: Thank you for all of
the support that you gave me as I completed my own small piece of your incredible SafeCare
program of research. I also deeply appreciate all of the SafeCare study participants’ time and
energy. To My Lovely (Official and Unofficial) Cohort: Thank you for cheering me on and
keeping me laughing through all of the program’s (and life’s!) ups and downs these past years
(especially you, KMF!). Mom and Dad: Thank you for your unwavering support, words of
encouragement, and real understanding of both silly and serious academic stresses. To My Other
Family Members and Friends: Thank you for being proud of me and showing interest…but not
too much interest thus making me remember that there is so much life outside of all of this. To
NIMH (F31MH113292): Thank you for your generous support and for giving me the extra time
and training that I needed to answer the questions that I wanted to ask.
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 3
Table of Contents
Acknowledgements ......................................................................................................................... 2
List of Figures and Tables............................................................................................................... 6
Abstract ........................................................................................................................................... 8
Chapter One: Rethinking Organizational Implementation Research ........................................... 10
Think “Context” and You Will See Context ............................................................................. 10
Organizations Are Implementation Actors ............................................................................... 12
How Do Organizations Engage In The Adaptation Process? ................................................... 15
SafeCare Program of Research ................................................................................................. 16
Dissertation Data Sources and Sample...................................................................................... 17
Dissertation Roadmap ............................................................................................................... 19
Rethinking Organizational Implementation Research: First Steps ........................................... 21
References ................................................................................................................................. 22
Chapter Two (Study 1): The Importance of an Open Systems Approach to EBP Implementation:
A SafeCare Contracting Arrangements Example ......................................................................... 31
Abstract ..................................................................................................................................... 31
Introduction ............................................................................................................................... 32
Methods ..................................................................................................................................... 37
Results ....................................................................................................................................... 40
Discussion ................................................................................................................................. 53
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 4
References ................................................................................................................................. 60
Chapter Three (Study 2): Approaches to Organizational Adaptation during EBP
Implementation: Incorporators, Early Investors, and Learners ................................................... 72
Abstract ..................................................................................................................................... 72
Introduction ............................................................................................................................... 73
Methods ..................................................................................................................................... 77
Results ....................................................................................................................................... 83
Discussion ................................................................................................................................. 88
References ................................................................................................................................. 96
Appendix ................................................................................................................................. 102
Chapter Four (Study 3): Understanding Outer and Inner Context Connections during EBP
Implementation: A Qualitative Comparative Analysis Demonstration ..................................... 108
Abstract ................................................................................................................................... 108
Introduction ............................................................................................................................. 109
Methods ................................................................................................................................... 116
Results ..................................................................................................................................... 120
Discussion ............................................................................................................................... 122
References ............................................................................................................................... 128
Appendix ................................................................................................................................. 133
Chapter Five: Organizations as Implementation Actors: A Research Agenda ......................... 137
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 5
Why We Need to Rethink Organizational Implementation Research..................................... 137
Organizations Are Implementation Actors ............................................................................. 138
Key Contributions of this Dissertation .................................................................................... 139
Looking Across the Three Studies .......................................................................................... 143
Organizations are Implementation Actors: A Research Agenda ............................................ 145
References ............................................................................................................................... 150
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 6
List of Figures and Tables
Figure 1.1. SafeCare Program of Research
Figure 1.2. Key Elements of and Connections between Dissertation Studies
Table 2.1. Service System Descriptive Information
Table 2.2. Data Sources for Each Service System
Table 2.3. Codes for Content Analysis
Table 2.4. Codes for Focused Coding of Transcripts
Table 2.5. Service System Characteristics
Table 2.6. Content Analysis of SafeCare Contracting Documents by Service System
Figure 2.1. Sources of Influence across the Outer and Inner Contexts
Figure 2.2. Dependences across the Outer and Inner Contexts
Table 3.1. Data Types and Years Collected by Service System
Table 3.2. Document Review of SafeCare Manuals
Table 3.3. Participant Demographics for Qualitative Data
Table 3.4. Distribution of Adaptation Profiles by Service Systems
Figure 3.1. “Adaptation Snapshot” Visual Elicitation Tool
Figure 3.2. Overview of the Adaptation Profiles
Table 4.1. Assumptions about Causal Mechanisms
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 7
Table 4.2. QCA Demonstration Causal Conditions
Table 4.3. Consistency and Coverage Definitions
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 8
Abstract
This dissertation proposes a new metaphor for implementation science: organizations as
implementation actors. What does it mean for an organization to be an actor and not just the
context for EBP implementation? First, this metaphor highlights and acknowledges the relational
nature of organizations. Second, thinking about organizations as implementation actors reminds
us that organizations are a combination of interacting elements, agents, and processes. This
interaction among factors may be lost when we parcel out and focus on isolated organizational
characteristics. Third, this metaphor requires identifying and classifying organizational behavior
in a way that allows for the examination of different behavioral pathways that lead to a particular
outcome. This study illustrates these three elements of the organizations as implementation
actors metaphor through an exploration of how organizations that sustained SafeCare engaged in
the process of adaptation.
Study 1 (Chapter 2) is a conceptual paper that argues for a more open systems approach
to implementation research using organization-service system interactions around contracting
arrangements as an illustrative example. The research question for dissertation Study1 is: How
do child welfare systems and community-based organizations interact around SafeCare
contracting arrangements? This study uses content analysis and multiple case study analysis
approaches with a dataset of 88 transcripts and over 100 SafeCare contracting documents. Study
1 findings illustrate and explain bi-directional influence, dependencies and resources exchanged
between child welfare systems and organizations during SafeCare implementation. This study
provides important contextual information for understanding adaptation in Study 2.
Study 2 (Chapter 3) qualitatively explores the question: How do organizations approach
the adaptation process during SafeCare implementation? This study uses a typological approach
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 9
and demonstrates a way to holistically examine organizational level behavior around a specific
implementation process (adaptation). Study 2 uses a rigorous set of qualitative methods
including coding of transcripts, document review, multiple case study analysis, and member
checking using a visual elicitation tool. Data sources include 177 transcripts, contracting
documents from Study 1, and home visitor, coach, and trainer SafeCare manuals. Iterative peer
debriefing and member checking meetings with the research team and practitioners were an
important methodological element in Study 2. Key findings are three organizational level
approaches to adaptation during SafeCare implementation: Incorporators, Early Investors and
Learners. These organizational level approaches to adaptation are the outcome variable in Study
3.
The broad goal of Study 3 (Chapter 4) is to demonstrate the usefulness of Qualitative
Comparative Analysis (QCA) for addressing a particular literature gap: how an organization’s
collection of relationships with actors in the external environment combine and influence
behavior around a specific implementation process. This study examines three environmental
features that describe the organization’s relationship with other actors: lead agency status (an
organization’s position within the network of other organizations that are providing SafeCare),
funding level in SafeCare contracting arrangements (resources exchanged between organization
and system) and the organization’s ability to influence the SafeCare contract (openness of
relationship between organization and system). The implementation process of interest is how
organizations adapt during EBP implementation and this study focuses on the Incorporator and
Early Investor profiles developed in Study 2.
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 10
Chapter One:
Rethinking Organizational Implementation Research
…all theories of organization and management are based on implicit images or metaphors
that lead us to see, understand, and manage organizations in distinctive yet partial ways
(Morgan, 1997, p.4).
Think “structure” and you’ll see structure. Think “culture,” and you’ll see all kinds of
cultural dimensions. Think “politics” and you’ll find politics. Think in terms of system
patterns and loops, and you’ll find a whole range of them (Morgan, 1997, p.349).
Think “Context ” and You Will See Context
The current goal of organizationally focused implementation research is to guide and
shape the context around EBP implementation (Aarons, Moullin, & Ehrhart, 2018). Context can
be defined as “…stimuli and phenomena that surround and thus exist in the environment external
to the individual, most often at a different level of analysis” (Mowday & Sutton, 1993).
Implementation researchers typically take one of two approaches to studying the organizational
context. The first approach is to report organizational-level barriers and facilitators in different
clinical or service settings (e.g. Jabbour et al., 2018; Geerligs et al., 2018; Templeton et al.,
2016). Under this approach, organizations are usually included in descriptive multilevel
assessments that attempt to characterize challenges and supports so that they can be proactively
addressed when planning for implementation.
While the first approach is more descriptive in nature, the second approach aims to
identify relationships between isolated organizational constructs and specific clinical or
implementation outcomes (Andrew et al., 2019; Smith et al., 2019, Williams et al., 2018).
Williams and colleagues (2018), for example, examine relationships between two types of
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 11
climates (molar and strategic implementation) and clinicians’ use of evidence-based
psychotherapy techniques. Great attention is paid to organizational constructs like climate,
culture, leadership, readiness for change, and provider attitudes towards evidence-based practice
in both of the approaches described above (e.g. Williams et al., 2019; Aarons, Ehrhart, &
Farahnak, 2014; Shea et al., 2014; Rye et al., 2017).
These approaches to organizational research are a direct product of widely used
conceptual frameworks and measures. Frameworks like the Exploration, Preparation,
Implementation, Sustainment (EPIS), Consolidated Framework for Implementation Research
(CFIR), and Practical, Robust Implementation and Sustainability Model (PRISM) describe and
separate factors in the inner (inside the organization) and outer (outside of the organization)
contexts (Moullin et al., 2019; Fernandez et al., 2018; Damschroder et al., 2009; Feldstein &
Glasgow, 2008). These conceptual frameworks inform open-ended qualitative interviews that
assess implementation barriers and facilitators (e.g. Bramoweth et al., 2018) and studies that
quantitatively test relationships between organizational constructs and specific clinical or
implementation outcomes. Popular measures include the Organizational Readiness for
Implementing Change (ORIC), Organizational Readiness for Change Assessment (ORCA),
Organizational Social Context (OSC) Scale, Implementation Climate Scale (ICS) and
Implementation Leadership Scale (ILS) (Ramsey et al., 2016).
Implementation researchers use these frameworks and measures to study organizations as
context that can be deconstructed, examined, and manipulated in discrete layers (e.g. culture and
climate). Viewing organizations only as context is limiting, passive, and can place a heavy
burden on practitioners to understand and synthesize a multitude of potentially relevant
contextual pieces. In contrast, organizational theorizing in the management field shows rich and
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 12
diverse ways of thinking about organizations. Morgan (1997) shows how we can think of
organizations as machines (bureaucratic organizations), as organisms (open systems approach
and resource dependence theory), as brains (organizational learning), as cultures (organizational
cultural context and sense making theory), and as political systems (organizational interests,
conflict, and power). Each organizational metaphor is useful yet incomplete, reminding us that to
truly understand organizations, we need to be able to take on and off different theoretical lenses.
In addition to having multiple conceptualizations of organizations, theorists in the
management field also refer to organizations as entities that have agency. Organizations make
transactions with the external environment (Pfeffer & Salancik, 2003). Organizations enact
engrained and rationalized societal myths (Meyer & Rowan, 1977). Organizations have cognitive
systems and memories (Daft & Weick, 1984). Organizations can even experience birth, survival,
and death (Hannan & Freeman, 1977). Birken and colleagues (2017) aptly note, “Organizational
theories offer implementation researchers a host of existing, highly relevant, and heretofore
largely untapped explanations of the complex interaction between organizations and their
environment.”
Organizations Are Implementation Actors
This dissertation advances and elaborates upon the EPIS framework in several important
ways. To start, this dissertation echoes Birken et al.’s (2017) call to use and integrate existing
organizational theoretical lenses in implementation research. Study 1 shows how resource
dependence theory can be layered upon the EPIS framework to better understand organizational
and system level behavior around contracting arrangements. This dissertation also tackles and
elaborates upon the EPIS framework’s dynamic outer- inner context boundary. Study 1
articulates bidirectional influence, dependencies, and resources exchanged across the outer-inner
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 13
boundary in the context of contracting arrangements. Study 2 explores how organizations adapt
over the course of implementation and acknowledges ways in which an organization’s approach
to adaptation is a response to other organizations in the external environment. Study 3 provides a
demonstration of QCA as a methodological tool for connecting and disentangling ways in which
the outer and inner context combine and influence EBP implementation.
Finally, this dissertation proposes a new metaphor for the field: organizations as
implementation actors. The goal of this metaphor is to challenge implementation researchers to
create and consider new theoretical lenses that go beyond the idea that organizations are context.
This metaphor is not meant to replace conceptual frameworks that treat organizations as context.
Instead, the goal is to encourage and inspire implementation researchers to take on and off
different theoretical lenses that represent organizations in more diverse ways. The organizations
as implementation actors metaphor can enhance our use of the EPIS and other implementation
frameworks.
What does it mean for an organization to be an actor and not just the context for EBP
implementation? First, this metaphor highlights and acknowledges the relational nature of
organizations. Organizations implementing an evidence-based practice, like the ones described in
this dissertation, exist within a complex relational web that can include state and local agencies,
other organizations providing the EBP, academic partners, intervention developers, and private
foundations. These entities interact with each other in a variety of ways, including contractual
and funding arrangements, subcontracting relationships, research studies and learning
collaboratives, and system-wide EBP training. These relationships— and the resources that flow
between entities—create dependencies, constraints, and opportunities over the course of
implementation.
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 14
Understanding organizations only as context strips away the critical dynamics and
interactions that occur within the organization’s social web. Furthermore, relationships across
organizations and across the layers of a service system may influence internal characteristics
within organizations (e.g., a county department’s buy-in to an EBP can affect staff attitudes and
EBP use in organizations adopting the practice). Implementation researchers use social network
approaches to understand communication and information sharing within and between
organizations. While this work begins to hint at the relational nature of organizations, existing
work tends to primarily focus on interpersonal relationships and individual level behavior, not
relationships between organizations (e.g. Cranley et al., 2019; Bunger et al., 2018; Gainforth et
al., 2014). A relationship with a human service organization is different from relationships with
individual executive directors or clinicians.
Additionally, thinking about organizations as implementation actors reminds us that
organizations are a combination of interacting elements. Establishing the importance of key
organizational constructs was a critical first step in the early development of implementation
science. However, thinking about organizations as implementation actors can help us understand
why some organizations achieve EBP sustainment and others fail to do so. The Stages of
Implementation (SIC) is a novel tool that tracks if and when sites achieve specific milestones
across the pre-implementation, implementation, and sustainment stages (Chamberlain, Hendricks
Brown, & Saldana, 2016; Saldana, Chamberlain, Wang, & Hendricks Brown, 2012). The SIC
looks holistically at longitudinal site level behavioral patterns, namely the proportion of activities
completed and time to completion at different time points (Chamberlain, Hendricks Brown, &
Saldana, 2016).
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 15
The organizations as context metaphor can help us isolate and assess the influence of
specific organizational characteristics in successful and unsuccessful sites. In contrast, an
organizations as implementation actors metaphor can help us think about organizations as
holistic entities comprised of interacting elements. This new metaphor encourages us to consider
ways in which implementation success is influenced by the organization’s relationships,
dynamics, resource dependencies, and interactions with other entities (e.g. service system actors,
funders, and peer organizations).
How Do Organizations Engage In The Adaptation Process?
The overarching goal of this dissertation is to understand how organizations approach the
process of adaptation during EBP implementation. Baumann, Cabassa, and Stirman (2018)
provide a framework for understanding EBP adaptation. This framework includes three types of
adaptation: changes to content, context, and training and evaluation (Baumann, Cabassa, &
Stirman, 2018). In a recent systematic review, Stirman and colleagues identify a variety of
content changes including tailoring, removing, adding, shortening, lengthening, integrating,
repeating, reordering and loosening intervention elements (Stirman, Gamarra, Bartlett, Calloway,
& Gutner, 2017). This type of adaptation receives the most attention from researchers and
directly speaks to the popular “fidelity-adaptation balance” debate (Allen, Shelton, Emmons, &
Linnan, 2018).
Context level changes involve changing the intervention format (e.g. group vs.
individual), setting (outpatient vs. community-based setting), personnel (mental health
professional vs. peer specialist), and population (tailoring intervention to a new client group, e.g.
gay men or veterans) (Baumann, Cabassa, and Stirman, 2018). Adaptation may also involve
changing training and evaluation materials and processes to facilitate implementation (Baumann,
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 16
Cabassa, and Stirman, 2018). In this framework, the organization is treated as a “structural factor
that can influence adaptation” (Baumann, Cabassa, and Stirman, 2018). Escoffery and colleages’
(2018) systematic review of adaptations focuses on similar categories: changes to content,
context, cultural modifications and delivery (Escoffery et al., 2018). Neither of these
comprehensive conceptualizations of adaptation explicitly address how organizations must adapt
to accommodate the implementation of a new EBP, and this category of adaptation remains
unclear and unexamined in the implementation literature (Stirman et al., 2017).
This dissertation takes a longitudinal approach to understanding organizational
adaptation. Rather than describing and categorizing types of changes that organizations make
during implementation, this work treats organizations as implementation actors that engage in
the process of adaptation over time. The EBP of interest is SafeCare, a well-established and
behaviorally prescriptive EBP designed to prevent child neglect and improve outcomes for
children at risk of abuse and neglect (Gardner, Hodson, Churchill, & Cotmore, 2014; Chaffin,
Hecht, Bard, Silovsky, & Beasley, 2012).
SafeCare Program of Research
This dissertation is situated within a series of federally funded projects that examine
SafeCare implementation. SafeCare home visitors work in families’ homes to improve parent
skills in parent-infant and child interactions, home safety, and health (National SafeCare
Training and Research Center, 2016). Home visitors, coaches, and trainers achieve and maintain
certification, and coaches work closely with home visitors to ensure ongoing adherence to the
SafeCare model (National SafeCare Training and Research Center, 2016).
Project 1, from 2005 to 2010, examined SafeCare implementation in Oklahoma. This
project included the exploration of key organizational and provider characteristics that
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 17
influenced implementation at these sites (R01MH072961). From 2009 to 2012, investigators
continued to examine SafeCare and focused on a “dynamic adaptation process” (DAP) in six
regions of California in Project 2 (R01CE001556). The DAP approach was designed to
systematically address intervention-level adaptation during SafeCare implementation so that
changes are planned and discussed with a team of experts. This project also continued to track
and examine organizational and provider characteristics that influence SafeCare implementation.
Project 3, conducted 2011 to 2016 (R01MH09295) examined the Interagency
Collaborative Teams (ICT) approach in California. The ICT approach developed an interagency
seed team that was then responsible for training subsequent waves of providers in SafeCare
throughout a larger service system. Project 4, from 2012 to 2017, explored sustainment processes
and outcomes (R01MH072961). This project followed experiences in sites implementing
SafeCare in Oklahoma and California from the previous studies, plus new sites, and examined
organizational and provider characteristics that influence SafeCare sustainment. Across all
projects, rich qualitative data were routinely collected from a variety of stakeholders. All four
projects are listed here to emphasize the fact that in addition to raw data, this dissertation’s PI
worked closely with a research team that had deep institutional memory and long-term
experience with these organizations and service systems. Figure 1.1 summarizes the SafeCare
studies used in this dissertation.
Dissertation Data Sources and Sample
Data from Project 2 (Dynamic Adaptation Process “DAP” in California) acted as a
preliminary study for this dissertation. While the “DAP” focused on SafeCare adaptations, this
study allowed the PI to identify other adaptation examples (e.g. to the organization and service
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 18
system), learn how practitioners talk about adaptation in real world settings, and become familiar
with some of the organizations and systems included in the dissertation sample.
The primary data source for the dissertation is Project 4, which focuses on the process of
EBP sustainment. An EBP is sustained “if core elements are maintained or delivered at a
sufficient level of fidelity after initial implementation support has been withdrawn, and adequate
capacity exists to continue maintaining these core elements” (Green et al., 2016; Stirman et al.,
2012). Aarons et al. (2016) examined SafeCare sustainment in eleven service systems. In this
study, three of the systems were classified as “no sustainment” meaning they no longer provided
SafeCare. One of the systems was classified as “partial sustainment” because it met some of the
SafeCare model fidelity requirements. Seven of the systems met key SafeCare fidelity
requirements and were classified as “full sustainment”. By the time of this dissertation work,
eight systems were classified as “full sustainment”. These eight systems and the 17 community-
based organizations embedded within them are this dissertation’s sample.
To be included in this dissertation sample, an organization has to: (1) have achieved
SafeCare sustainment status and (2) have at least two time points for qualitative data collection.
Of the 17 organizations in the dissertation sample, two organizations only participated in the
2016 round of qualitative data collection in Project 4. The dissertation PI was able to conduct
follow up interviews for these two organizations in August and September 2018, thus adding a
second time point. An 18
th
organization participated in the 2016 wave of qualitative data
collection in Project 4, but the dissertation PI was unable to reach a participant for a follow up
interview. This organization was dropped from the final dissertation sample. Data from Project 3
(Interagency Collaborative Teams approach in California) provided supplementary information
for four of the organizations (one service system) in the dissertation sample. In addition to
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 19
qualitative transcripts, this dissertation also uses SafeCare contracting documents from all eight
child welfare systems and SafeCare program manuals from the National SafeCare Training and
Research Center.
Published work from Project 4 (SafeCare sustainment in Oklahoma and California)
explores topics such as: policymaker perspectives on EBP implementation and sustainment
(Willging et al., 2015), the role of leadership in EBP sustainment (Aarons et al. 2016), the role of
collaboration in EBP sustainment (Green et al., 2016), how frontline workers navigate EBP
implementation and sustainment (Willging et al., 2017), community-based organization manager
perspectives on EBP implementation and sustainment (Willging et al., 2018), implementation
and sustainment factors that affect coaching (Gunderson et al., 2018), and the ways that funding
arrangements affect EBP sustainment (Trott Jaramillo et al., 2018). This dissertation
meaningfully adds to and complements this published work by focusing on how organizations
engage in adaptation over the course of SafeCare implementation. Focusing only on sustaining
sites is important for two reasons. One, the focus is on paths to success rather than contrasting
success and failure. Two, because it does not predetermine that a specific set of actions or
relationships is required for sustainment, it allows for the unique examination of equifinality, or
different adaptation approaches that led to the same desired outcome of sustainment.
Dissertation Roadmap
Study 1 is a conceptual paper that argues for a more open systems approach to
implementation research using organization-service system interactions around contracting
arrangements as an illustrative example. The research question for dissertation Study1 is: How
do child welfare systems and community-based organizations interact around SafeCare
contracting arrangements? Contracting arrangements in human services give public entities (like
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 20
a county child welfare department) the authority to structure the way that services are delivered
by community-based organizations (McBeath et al. 2019). Contracting arrangements were
selected for this study because they are a dynamic medium through which organizations and
child welfare systems interact with each other and exchange resources. This study uses content
analysis and multiple case study analysis approaches with a dataset of 88 transcripts and over
100 SafeCare contracting documents. Study 1 findings illustrate and explain bi-directional
influence, dependencies and resources exchanged between child welfare systems and
organizations during SafeCare implementation. This study provides important contextual
information for understanding adaptation in Study 2.
Study 2 qualitatively explores the question: How do organizations approach the
adaptation process during SafeCare implementation? This study uses a typological approach and
demonstrates a way to holistically examine organizational level behavior around a specific
implementation process (adaptation). Study 2 uses a rigorous set of qualitative methods
including coding of transcripts, document review, multiple case study analysis, and member
checking using a visual elicitation tool. Data sources include 177 transcripts, contracting
documents from Study 1, and home visitor, coach, and trainer SafeCare manuals. Iterative peer
debriefing and member checking meetings with the research team and practitioners were an
important methodological element in Study 2. Key findings are three organizational level
approaches to adaptation during SafeCare implementation: Incorporators, Early Investors and
Learners. These organizational level approaches to adaptation are the outcome variable in Study
3.
The broad goal of Study 3 is to demonstrate the usefulness of Qualitative Comparative
Analysis (QCA) for addressing a particular literature gap: how an organization’s collection of
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 21
relationships with actors in the external environment combine and influence behavior around a
specific implementation process. This study examines three environmental features that describe
the organization’s relationship with other actors: lead agency status (an organization’s position
within the network of other organizations that are providing SafeCare), funding level in SafeCare
contracting arrangements (resources exchanged between organization and system) and the
organization’s ability to influence the SafeCare contract (openness of relationship between
organization and system). The implementation process of interest is how organizations adapt
during EBP implementation and this study focuses on the Incorporator and Early Investor
profiles developed in Study 2.
Rethinking Organizational Implementation Research: First Steps
This dissertation treats organizations as implementation actors and illustrates a new way
of approaching organizational implementation research. Several methodological features of this
dissertation illustrate what an organizations as implementation actors approach could look like.
To start, this dissertation includes perspectives about the organization’s behavior from
individuals both inside and outside of the organization. This work also explores organizational
level approaches to a key implementation process, rather than focusing on specific, isolated
organizational characteristics. This dissertation acknowledges the organization’s relationship
with other actors (e.g. state or county funders) and the impact of these relationships on the
organization’s response to EBP implementation. Finally, this work uses data from multiple time
points to assess behavior over time, and compares behavior patterns across multiple
organizations implementing the same EBP. Figure 1.2 illustrates each study’s research question,
methodological approaches, data sources and general findings. Chapter 5 will discuss overall
findings and implications in greater detail.
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 22
References
Aarons, G.A., Ehrhart, M.G., & Farahnak, L.R. (2014). The implementation leadership scale
(ILS): Development of a brief measure of unit level implementation leadership.
Implementation Science, 9, 45-55.
Aarons, G. A., Hurlburt, M., & Horwitz, S. M. (2011). Advancing a conceptual model of
evidence-based practice implementation in public service sectors. Administration and
Policy in Mental Health and Mental Health Services Research, 38, 4–23.
Aarons, G.A., Green, A.E., Trott, E., Willging, C.E., Torres, E.M., … Roesch, S.C. (2016). The
roles of system and organizational leadership in system wide evidence-based intervention
sustainment: A mixed method study. Administration and Policy in Mental Health and
Mental Health Services Research, 43(6), 991-1008.
Aarons, G.A., Moullin, J.C., & Ehrhart, M.G. (2018). The role of organizational processes in
dissemination and implementation research. In R.C. Brownson, Colditz, G.A., & E.K.
Proctor (Eds.), Dissemination and Implementation Research in Health (p.121-142). New
York, NY: Oxford University Press.
Allen, J.D., Shelton, R.C., Emmons, K.M., & Linnan, L.A. (2018). Fidelity and its relationship to
implementation effectiveness, adaptation, and dissemination. In R.C. Brownson, Colditz,
G.A., & E.K. Proctor (Eds.), Dissemination and Implementation Research in Health
(p.267-284). New York, NY: Oxford University Press.
Andrew, N.E., Middleton, S., Grimley, R., Anderson, C.S., Donnan, G.A., Lannin,
N.A…Cadilhac, D.A. (2019). Hospital organizational context and delivery of evidence-
based stroke care: A cross-sectional study. Implementation Science, 14, 6-18.
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 23
Baumann, A.A., Cabassa, L.J., & Stirman, S.W. (2018). Adaptation in dissemination and
implementation science. In R.C. Brownson, Colditz, G.A., & E.K. Proctor (Eds.),
Dissemination and Implementation Research in Health (p.285-300). New York, NY:
Oxford University Press.
Birken, S.A., Bunger, A.C., Powell, B.J., Turner, K., Clary, A.S., Klaman, S.L…Weiner, B.J.
(2017). Organizational theory for dissemination and implementation research.
Implementation Science, 12, 62-77.
Bramoweth, A. D., Germain, A., Youk, A. O., Rodriguez, K. L., Chinman, M. J. (2018). A
hybrid type I trial to increase veterans” access to insomnia care: Study protocol for a
randomized controlled trial. Trials, 19(1), 73-83.
Bunger, A.C., Doogan, N., Hanson, R.F., & Birken, S.A. (2018). Advice-seeking during
implementation: A network study of clinicians participating in a learning collaborative.
Implementation Science, 13, 101-113.
Chaffin, M., Hecht, D., Bard, D., Silovsky, J. F., & Beasley, W. H. (2012). A statewide trial of
the SafeCare home-based services model with parents in child protective services.
Pediatrics, 129, 509–515.
Chamberlain, P., Brown, C.H., & Saldana, L. (2016). Observational measure of implementation
progress in community based settings: The Stages of Implementation Completion (SIC).
Implementation Science, 6, 116-124.
Cranley, L.A., Keefe, J.M., Taylor, D., Thompson, G., Beacom, A.M. Squires, J.E…Berta, W.B.
(2019). Understanding professional advice networks in long-term care: An outside-inside
view of best practice pathways for diffusion. Implementation Science, 14, 10-23.
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 24
Daft, R.L. & Weick, K.E. (1984). Toward a model of organizations as interpretation systems.
The Academy of Management Review, 9(2), 284-295.
Damschroder, L.J., Aron, D.C., Keith, R.E., Kirsh, S.R., Alexander, J.A., & Lowery, J.C. (2009).
Fostering implementation of health services research findings into practice: A consolidated
framework for advancing implementation science. Implementation Science, 4, 50-65.
Escoffery, C., Lebow-Skelley, E., Haardoerfer, R., Boing, E., Udelson, H., Wood, R…& Mullen,
P.D. (2018). A systematic review of adaptations of evidence-based public health
interventions globally. Implementation Science, 13, 125-146.
Feldstein, A.C. & Glasgow, R.E. (2008). A Practical, Robust and Sustainability Model (PRISM)
for integrating research findings into practice. The Joint Commission Journal on Quality and
Patient Safety, 34(4), 228- 243.
Fernandez, M.E., Walker, T.J., Weiner, B.J., Calo, W.A., Liang, S., Risendal, B…Kegler, M.C.
(2018). Developing measures to assess constructs from the inner setting domain of the
Consolidated Framework for Implementation Research. Implementation Science,13, 52-
65.
Gainforth, H.L., Latimer-Cheung, A.E., Athanasopoulos, P., Moore, S., & Martin Ginis, K.A.
(2014). The role of interpersonal communication in the process of knowledge
mobilization within a community-based organization: A network analysis.
Implementation Science, 9, 59-67.
Gardner, R., Hodson, D., Churchill, G., & Cotmore, R. (2014). Transporting and implementing
the SafeCare® home-based programme for parents, designed to reduce and mitigate the
effects of child neglect: An initial progress report. Child Abuse Review, 23, 297–303.
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 25
Geerligs, L., Rankin, N.M., Shepherd, H.L., & Butow, P. (2018). Hospital-based interventions: A
systematic review of staff-reported barriers and facilitators to implementation processes.
Implementation Science, 13, 36-53.
Green, A.E., Trott, E., Willging, C.E., Finn, N.K., Ehrhart, M.G., & Aarons, G.A. (2016). The
role of collaborations in sustaining an evidence-based intervention to reduce child neglect.
Child Abuse & Neglect, 53, 4-16.
Gunderson, L.M., Willging, C.E., Trott Jaramillo, E., Green, A.E., Fettes, D., Aarons, G.A., &
Hecht, D.B. (2018). The good coach: implementation and sustainment factors that affect
coaching as evidence-based intervention fidelity support. Journal of Children’s Services,
13(1), 1-17.
Hannan, M.T. & Freeman, J. (1977). The population ecology of organizations. The American
Journal of Sociology, 82(5), 929-964.
Jabbour, M., Newton, A.S., Johnson, D., & Curran, J.A. (2018). Defining barriers and enablers
for clinical pathway implementation in complex clinical settings. Implementation
Science, 13, 139- 152.
McBeath, B., Chuang, E., Carnochan, S., Austin, M.J., & Stuart, M. (2019). Service coordination
by public sector managers in a human service contracting environment. Administration and
Policy in Mental Health and Mental Health Services Research, 46, 115-127.
Meyer, J.W. & Rowan, B. (1977). Institutionalized organizations: Formal structure as myth and
ceremony. The American Journal of Sociology, 83(2), 340-363.
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 26
Mowday, R.T. & Sutton, R.I. (1993). Organizational behavior: Linking individuals and groups to
organizational contexts. Annual Review of Psychology, 44, 195–229.
Morgan, G. (1997). Images of organization. Thousand Oaks, CA: SAGE Publications, Inc.
Moullin, J.C., Dickson, K.S., Stadnick, N., Rabin, B., Aarons, G.A. (2019). Systematic review of
the exploration, preparation, implementation, sustainment (EPIS) framework.
Implementation Science, 14, 1-17.
National SafeCare Training and Research Center. (2016). Provider manual: SafeCare overview.
Atlanta, GA: Georgia State University.
Pfeffer, J. & Salancik, G.R. (2003). The external control of organizations: A resource
dependence perspective. Stanford, CA: Stanford University Press.
Ramsey, A., Lawrence, K., Prusaczyk, B., Baumann, A., Kryzer, E., & Proctor, E.
Organizational Measures. [Internet]. St. Louis, MO: Washington University; 2016
October. Eight toolkits related to Dissemination and Implementation. Available from
https://sites.wustl.edu/wudandi .
Rye, M., Torres, E.M., Friborg, O., Skre, I., & Aarons, G.A. (2017). The Evidence-based
Practice Attitude Scale-36 (EBPAS-36): A brief and pragmatic measure of attitudes to
evidence-based practice validated in US and Norwegian samples. Implementation
Science, 12, 44-55.
Saldana, L., Chamberlain, P., Wang, W., & Henricks Brown, C. (2012). Predicting program
start-up using the stages of implementation measure. Administration and Policy in Mental
Health and Mental Health Services Research, 39, 419-425.
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 27
Shea, C.M., Jacobs, S.R., Esserman, D.A., Bruce, K., & Weiner, B.J. (2014). Organizational
readiness for implementing change: a psychometric assessment of a new measure.
Implementation Science, 9, 7-22.
Smith, S.N., Almirall, D., Prenovost, K., Goodrich, D.E., Abraham, K.M., Liebrecht, C., &
Kilbourne, A.M. (2018). Organizational culture and climate as moderators of enhanced
outreach for persons with serious mental illness: Results from a cluster-randomized trial
of adaptive implementation strategies. Implementation Science, 13, 93-108.
Stirman, S.W., Gamarra, J.M., Bartlett, B.A., Calloway, A., & Gutner, C.A. (2017). Empirical
examinations of modifications and adaptations to evidence-based psychotherapies:
Methodologies, impact, and future directions. Clinical Psychology Science and Practice,
24, 396-420.
Stirman, S.W., Kimberly, J., Cook, N., Calloway, A., Castro, F., & Charns, M. (2012). The
sustainability of new programs and innovations: A review of the empirical literature and
recommendations for future research. Implementation Science, 7, 17-36.
Templeton, A.R., Young, L., Bish, A., Gnich, W., Cassie, H., Treweek, S…Ramsay, C. (2016).
Patient-, organization-, and system-level barriers and facilitators to preventive oral health
care: A convergent mixed-methods study in primary dental care. Implementation Science,
11, 5-19.
Trott Jaramillo, E., Willging, C.E., Green, A.E., Gunderson, L.M., Fettes, D., & Aarons, G.A.
(2018). “Creative Financing”: Funding evidence-based interventions in human service
systems. Journal of Behavioral Health Services & Research [Epub ahead of print].
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 28
Willging, C.E., Green, A.E., Gunderson, L., Chaffin, M., & Aarons, G.A. (2015). From a
““perfect storm”“ to ““smooth sailing”“: Policymaker perspectives on implementation and
sustainment of an evidence-based practice in two states. Child Maltreatment, 20(1), 24-36.
Willging, C.E., Trott, E.M., Fettes, D., Gunderson, L., Green, A.E., Hurlburt, M.S., & Aarons,
G.A. (2017). Research-supported intervention and discretion among frontline workers
implementing home visitation services. Research on Social Work Practice, 27(6), 664-675.
Willging, C.E., Gunderson, L., Green, A.E., Trott Jaramillo, E., Garrison, L., Ehrhart, M.G., &
Aarons, G.A. (2018). Perspectives from community-based organizational managers on
implementing and sustaining evidence-based interventions in child welfare. Human Service
Organizations: Management, Leadership & Governance, 42(4), 359-379.
Williams, N.J., Ehrhart, M.G., Aarons, G.A., Marcus, S.C., & Beidas, R.S. (2018) Linking molar
organizational climate and strategic implementation climate to clinicians” use of
evidence-based psychotherapy techniques: Cross-sectional and lagged analyses from a 2-
year observational study. Implementation Science, 13, 85-98.
Williams, N.J., Frank, H.E., Frederick, L., Beidas, R.S., Mandell, D.S., Aarons, G.A…Locke, J.
(2019). Organizational culture and climate profiles: Relationships with fidelity to three
evidence-based practices for autism in elementary schools. Implementation Science, 14,
15-29.
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 29
Figure 1.1. SafeCare Program of Research
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 30
Figure 1.2. Key Elements of and Connections between Dissertation Studies
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 31
Chapter Two (Study 1):
The Importance of an Open Systems Approach to EBP Implementation:
A SafeCare Contracting Arrangements Example
Abstract
The boundary between the outer and inner contexts is largely unexplored territory.
Implementation researchers do not often delve into the complex ways that (1) the external
environment influences and constrains organizations when a new evidence-based practice (EBP)
is introduced into the system, or (2) organizations influence the external environment during
EBP implementation. An open systems approach emphasizes the interdependence and
interactions between internal and external settings. Applying an open systems approach to EBP
implementation requires a close examination of the relationships between the outer and inner
contexts for implementation. Contracting arrangements are a good example of how an open
systems approach can be applied to implementation science because they are a medium through
which human service organizations and child welfare systems interact and exchange resources.
The present study links multiple years of contracting documents and qualitative data across eight
diverse child welfare systems and 17 community-based organizations. Findings reveal multiple
sources of bidirectional influence around SafeCare contracting arrangements. While these
findings may be unsurprising to practitioners, existing and widely used conceptual frameworks
may miss these important dynamics because they are not often used in a way that treats
organizations as open systems. This chapter also applies a theoretical lens based on resource
dependence theory and illustrates specific ways that the organization-service system relationship
creates dependencies and constraints over the course of SafeCare implementation.
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 32
Introduction
A key feature of commonly used implementation science conceptual frameworks (e.g.
EPIS, CIFR, and PRISM) is a distinction between the outer and inner contexts of EBP
implementation (Aarons, Hurlburt, & Horwitz, 2011; Damschroder et al., 2009; Feldstein &
Glasgow, 2008). In the Exploration- Preparation- Implementation- Sustainment (EPIS)
framework, the outer context encompasses factors outside of the organization. Examples are
federal and state legislative priorities, the local service system, funding arrangements and inter-
organizational networks (Aarons, Hurlburt, & Horwitz, 2011). The inner context of the EPIS
framework captures factors inside of the organization including organizational characteristics,
leadership, individual adopter characteristics, fidelity monitoring and supervision (Aarons,
Hurlburt, & Horwitz, 2011). This chapter aims to elaborate upon this feature of the EPIS
framework and argues for a deeper examination of the boundary between the outer and inner
contexts. Interactions around contracting arrangements illustrate the importance of reexamining
this contextual division.
In Novins and colleagues’ (2013) systematic review of evidence-based practices (EBPs)
for child and adolescent mental health, 67 articles explicitly examined inner contextual factors,
27 explicitly examined outer contextual factors, and 23 articles separately examined both types
(e.g. training in the outer context and fidelity monitoring and coaching in the inner context). This
review is representative of current implementation research, which often relies upon the
prevailing outer-inner contextual distinction. A shortcoming of this approach is that we set aside
the relationships between outer and inner contextual factors and implicitly treat organizations as
closed systems. Implementation researchers do not often delve into the complex ways that (1) the
external environment influences and constrains organizations when a new evidence-based
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 33
practice (EBP) is introduced into the system, or (2) organizations influence the external
environment during EBP implementation.
This chapter takes an open systems approach and highlights the interdependence between
the external environment and organizations (Katz & Kahn, 1966; Scott & Davis, 2007). Open
systems are defined as “systems capable of self-maintenance based on a throughput of resources
from their environment” (Scott & Davis, 2007). To illustrate the importance of an open systems
approach in implementation science, this chapter uses interactions around contracting
arrangements. Contracting arrangements in human services give public entities (like a county
child welfare department) the authority to structure the way that services are delivered by
community-based organizations (McBeath et al. 2019). Ways of structuring service delivery
through contracts include specifying services, setting eligibility requirements, determining
processes for billing and outcome reporting, and deciding the number of types of clients that can
be referred to a particular program (McBeath et al. 2019; Trott Jaramillo et al., 2018). Public
entities have fiduciary authority and the contracting out of services requires interorganizational
collaboration between the public agency and the organizations that deliver services (McBeath et
al. 2019; Bunger et al. 2016).
Contracting arrangements are a good example of how an open systems approach can be
applied to implementation science because they are a medium through which organizations and
child welfare systems interact and exchange resources. Resource dependence theory is a useful
theoretical lens for understanding these interactions and exchanges. Resource dependence theory
asserts that organizations are inextricably tied to the environment and organizational survival
depends on acquiring and maintaining essential resources (Pfeffer & Salancik, 1978).
Additionally, this theory asserts that organizational behavior can be understood by looking at
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 34
how organizations react to environmental contingencies and constraints (Pfeffer & Salancik,
1978). Resource dependence theory can help us understand how organizational leaders acquire
resources, form relationships, maintain autonomy, and manage dependence on other actors over
the course of EBP implementation (Willging et al., 2018; Birken et al., 2017).
Study Context
In 2015, nearly half a million children in the United States experienced neglect,
representing 75% of the total child maltreatment cases (U.S. Department of Health & Human
Services, 2017). SafeCare is a well-established evidence-based practice aimed directly at
reducing and preventing child neglect (Gardner, Hodson, Churchill, & Cotmore, 2014; Chaffin,
Hecht, Bard, Silovsky, & Beasley, 2012). SafeCare is highly structured and focuses on
improving parent skills in parent-infant and child interactions, home safety, and health (National
SafeCare Training and Research Center, 2016). Services are provided in the family’s home.
Home visitors, coaches, and trainers achieve and maintain certification, and coaches work
closely with home visitors to ensure ongoing adherence to the SafeCare model (National
SafeCare Training and Research Center, 2016).
Child welfare services, including SafeCare, are funded through a combination of federal,
state, and local sources (National Conference of State Legislatures, 2017). Titles IV-B and IV-E
of the Social Security Act are the largest federal child welfare funding sources and increasingly,
child welfare systems are tapping into state and local sources to compensate for federal cuts
(National Conference of State Legislatures, 2017; Roskinsky & Connelly, 2016). In the United
States, 38 states have a state-level centralized child welfare administrative system, 9 states are
county administered, and 3 have a hybrid state-county administration structure (U.S. Department
of Health & Human Services, 2018).
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 35
Each child welfare system must design and implement contracting arrangements that
transform public dollars into client services. In 2006, the U.S. Department of Health and Human
Services funded the Child Welfare Privatization Initiatives Project (U.S. Department of Health
and Human Services, 2007). Definitions of privatization in the child welfare context vary, but the
term typically refers to a public child welfare entity contracting out specific functions and
responsibilities (e.g. case management) to private contractors (U.S. Department of Health and
Human Services, 2007). This study examines interactions between public child welfare entities
and agencies that are privately contracted to provide SafeCare.
Two payment structures are often used to structure public-private child welfare
relationships: cost reimbursement and performance-based (U.S. Department of Health and
Human Services, 2007). Under traditional cost reimbursement or fee-for-service contracts, the
public child welfare entity assumes much of the risk of and responsibility for delivering services
(U.S. Department of Health and Human Services, 2007). In contrast, performance-based
contracts distribute the risk and responsibility and offer opportunities for innovation and
increased collaboration between the public entity and local contractors (U.S. Department of
Health and Human Services, 2007; Willging et al., 2016). This study includes one state and
seven county-based child welfare systems that utilize both cost reimbursement and performance-
based contracting arrangements to fund SafeCare.
The present study uses data from two large-scale parent studies and directly builds upon
published work from this program of SafeCare implementation research. Willging, Green,
Gunderson, Chaffin and Aarons (2015) qualitatively assessed policymaker perspectives about
implementation challenges and system-level factors that affect SafeCare sustainment.
Policymakers assert that well-specified requests for proposals and contracts facilitate SafeCare
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 36
implementation and sustainment (Willging et al. 2015). Green et al. (2016) qualitatively assessed
the role of collaboration during SafeCare implementation. In systems that sustained SafeCare,
community-based organizations often had longstanding relationships with each other and shared
child welfare contracts and resources (Green et al., 2016). Findings also pointed to the role of
academic partners in communicating the logistics and benefits of SafeCare, which can help
facilitate contract-related conversations between service system and organizational leaders
(Green et al., 2016). While these studies peripherally acknowledge the importance of contracts in
SafeCare implementation and sustainment, the question remains: How do organizations and
systems interact around these contracting arrangements?
In this study, multiple waves of qualitative data were linked drawing from eight diverse
child welfare systems and 17 community-based organizations within these systems. All systems
reached SafeCare sustainment and had at least two data points for the qualitative transcripts. This
study also drew upon multiple years of SafeCare contracting documents from each child welfare
system. In addition to enhancing our understanding of service system and organizational
behavior during EBP implementation, this study is relevant to practice because it begins to
untangle and describe the dynamic outer-inner boundary that service system and organizational
leaders must continually confront when implementing and sustaining an EBP. Acknowledging
and articulating the relationship between the outer and inner contexts can help practitioners
proactively: (1) identify key ways that the service system shapes behavior in their organization;
(2) identify key ways that their organization can influence the service system; and (3) plan,
select, and tailor multilevel strategies that consider these bi-directional influences.
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 37
Methods
Study Sample
The present study examined contracting arrangements utilizing secondary qualitative data
collected during two iterative mixed methods parent studies. These two parent studies were
conducted 2008-2013 and 2012-2017 and built upon a program of SafeCare implementation
research that began in 2005. Inclusion criteria for the systems in this dissertation sample were
that the system: (1) achieved SafeCare sustainment status and (2) had at least two time points for
the secondary qualitative data. Service systems that did not fully sustain SafeCare were excluded
from this dissertation study’s sample. Sustainment status was assessed by Aarons and colleagues
(Aarons et. al., 2016). The system achieved SafeCare sustainment “if core elements are
maintained or delivered at a sufficient level of fidelity after initial implementation support has
been withdrawn, and adequate capacity exists to continue maintaining these core elements”
(Green et al., 2016; Stirman et al., 2012). The sample for this dissertation study is one state-
operated and seven county-operated child welfare systems. Embedded within these eight service
systems were 17 community-based organizations. Table 2.1 describes population, income,
geographical information, and year that SafeCare started for each of the eight service systems in
this sample.
Data Sources
A total of 88 transcripts including 73 individual interviews, 14 small group interviews,
and one focus group were analyzed. Because this study focused on service system and
organizational interactions around contracting arrangements, data reflected perspectives of state
or county level child welfare system personnel, and agency leaders or other key members of the
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 38
organization’s upper management who might have knowledge of this issue. If available,
SafeCare coordinators and academic partners were also included because they participated in
contracting processes or were aware of contracting arrangements. Transcripts from SafeCare
home visitors, SafeCare coaches, and lower level management were excluded because these staff
members were not involved in contracting processes. There were 111 participants including 44
state and county (“service system”) personnel, 51 agency leaders and members of upper
management, 13 SafeCare coordinators, and 3 academic partners. In the parent studies, all
transcripts were digitally recorded, professionally transcribed, de-identified, and checked for
accuracy by at least one of the interviewers.
To supplement the transcripts, this study’s PI reviewed 113 SafeCare-related contract
documents. The research team collected 16 of the contracting documents during the parent
studies, but this did not include all systems or years. The dissertation study’s PI collected an
additional 97 contracting documents through email communication (September – December
2018) with the appropriate child welfare department and/or contracting department personnel in
each system. Data sources for each service system are described in Table 2.2.
Data Analysis
Data analysis occurred in three stages: (1) a content analysis of contracting documents;
(2) open coding of transcripts to identify examples of bidirectional service system and
organizational influence around these contracting arrangements; (3) multiple case study analysis
across the eight service systems. Strategies of rigor included co-coding, peer debriefing, and
triangulation of contracting documents and transcripts (Padgett, 2016). A fourth strategy of rigor
was the creation of a decision-making trail for auditing that documented unclear passages,
illustrative quotes, and coding decisions (Padgett, 2016).
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 39
In the first analytic stage, a content analysis was completed following the steps outlined
by Bernard and colleagues (2017). Step 1 is formulating a research question. The research
question for the content analysis is: What are specific ways that the child welfare system directed
SafeCare implementation in the community-based organizations through contracting
arrangements? Step 2 is selecting texts to test the research question. The dissertation study’s PI
requested any contracting documents that were related to SafeCare implementation and these
documents included contract agreements, scopes of work, performance work statements, requests
for proposals and pricing schedules. Step 3 is creating a set of codes for the content analysis (see
Table 2.3). Step 4 is pretesting the codes on a few selected texts and addressing any problems
with the codebook. Documents from System B were reviewed first because it was the service
system that had the first complete set of documents. The codebook was slightly revised after this
initial review (e.g. adding home visitor qualifications code). Step 5 is applying codes to the rest
of text, in this case, the remaining service systems. Step 6 is creating a “case by variable” matrix
from the texts and codes. A matrix was created whereby contract documents comprised the
matrix rows and content analysis codes comprised the matrix columns. If the code was not
present in the text, the matrix cell was left blank. Step 7 is analyze matrix using the appropriate
level of analysis. In this study, the level of analysis is the service system and a separate matrix
was created for each system.
The content analysis of contracting documents provided critical background knowledge
for understanding the terminology, references, and examples that emerged in the second analytic
phase. In this second stage, transcripts for each service system were analyzed using open coding
around two broad sensitizing concepts: (1) service system influence on organizations through
contracts and (2) organizations’ influence on service system through contracts (Charmaz, 2006).
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 40
Coding collectively considered all organizations within a specific service system (rather than
between organization analysis across systems). A collaborator who worked with the PI on a
previous SafeCare study co-coded 20% of the transcripts in Dedoose. Discrepancies and themes
in coded material were discussed and resolved between the two coders.
The third analytic stage was a multiple case study analysis. Contracting document
information and relevant coded transcript material were combined and organized for each of the
eight service systems. This created eight cases. Next, patterns and themes across the eight cases
were assessed (Stake, 2006). Through this cross-system comparison process, more specific codes
emerged. Transcripts and contracting documents were re-reviewed using focused coding around
these specific cross-case themes (see Table 2.4). Illustrative examples across the cases were
extracted for each theme. Final results were reviewed and discussed with members of the parent
study research team.
Results
Several sets of results are described below. The first set of results is a description of key
service features that emerged in the contracting documents and transcripts. The second set of
results pertains to the content analysis. The third set of results describes bi-directional influence
between systems (outer context) and organizations (inner context) around SafeCare contracting
arrangements. The fourth set of results describes dependencies across systems (outer context)
and organizations (inner context) around SafeCare contracting arrangements.
Service System Characteristics
Table 2.5 describes key features of the eight service systems drawn from the contracting
documents and transcripts. Information pertains to the 17 community-based organizations
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 41
(CBOs) included in the study sample. The service system may have other organizations that are
implementing SafeCare but are not represented in these findings.
Content Analysis
The goal of the content analysis was to identify specific ways that the child welfare
system directed SafeCare implementation in the community-based organizations through
contracting arrangements. Content analysis revealed that contracts outlined and directed
SafeCare implementation in the following ways (out of 8 service systems): caseload size (n=7),
length of service delivery period (n=7), length of sessions (n=6), use of other curricula (n=5),
client age (n=8), specific number of SafeCare positions (n=6), home visitor qualifications (n=6),
details about SafeCare training (n=8), details about SafeCare coaching (n=6), details about
referral processes for SafeCare cases (n=8), details about how to track and report SafeCare
program data (n=7), formally designating a community-based organization as a lead agency
(n=3), SafeCare sustainment explicitly mentioned (n=4) and delineation of specific SafeCare
outcomes (n=7). Table 2.6 summarizes these content analysis findings.
The next section describes specific ways that service systems and organizations
interacted around contracting arrangements. As this section illustrates, SafeCare contracts are a
conduit for bi-directional influence across the outer and inner contexts. Figure 2.1 summarizes
types of influence for each level of analysis.
Outer to Inner Context Influence around Contracting Arrangements
As Figure 2.1 illustrates, the data pointed to several ways that service systems can
influence organizations through contracting arrangements during SafeCare implementation.
Contracting requirements dictated by the service system can result in changes to organizational
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 42
level processes, procedures and/or staffing decisions. Insufficient contract funding can require an
organizational response to keep SafeCare going. Additionally, contracting arrangements decided
at the service system level can alter inter-organizational relationships and dynamics. Each type of
outer to inner context influence is explained below.
Contract requirements result in changes to organizational processes, procedures,
and/or staffing decisions.
First, service systems influence organizations through contracts when contract
requirements prompt organizations to change their processes, procedures, and staffing decisions
in order to comply. The content analysis illustrated multiple ways that service system decision
makers direct SafeCare implementation at the organizational level through the way that contracts
are structured. For example, the contract may dictate the number of SafeCare home visitors an
organization needs to have, their qualifications, and how they are trained and coached.
Another example of how service system level contracts influence organizational level
behavior occurred in a service system in which the contract for SafeCare became performance-
based. As a result of this contract change, organizations in that system had to create internal
processes to collect and report the information required by the service system. As one agency
leader explained,
…contracts are now much more outcome oriented so we’re developing monthly reports
where we submit outcomes based on what we projected we would do in the bidding
process. We’re just now working up the process for making those reports more
standardized so we can generate them without a lot of extra effort.
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 43
Another agency leader in this system talked about using the reports generated for contract
compliance for internal quality assurance purposes,
It’s allowed [participant name] to be able to give the team immediate feedback on a
monthly basis just like she’s giving to the [service system] saying okay this is where we
say we are, this was what our target was and this is what we’re hitting.
In a different service system, the organization designated as a lead agency developed a
new position to help manage the administrative requirements of the contract:
…that was something new we did a couple years ago was creating that position so she
still carries a smaller caseload but she’s also sort of the backup for the program
manager in terms of we really wanted to build a program capacity and… someone who
knew how to do the data management… [agency leader].
In other service systems, organizations changed hiring processes to interview and
promote individuals based on the SafeCare roles outlined in the contract, changed hiring
processes and employee engagement strategies to comply with a requirement that SafeCare home
visitors have a Bachelor’s degree, and updated organizational procedures to match the contract’s
scope of work. These examples demonstrate ways in which the service system can influence
organizational processes, procedures, and staffing decisions through SafeCare contract
requirements.
Insufficient contract funding requires an organizational response to keep SafeCare
going.
A second way that contracting arrangements were a conduit for influence across the outer
and inner contexts had to do with insufficient funds. Funding decisions at the service system
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 44
level created barriers or supports for organizations implementing SafeCare. In some systems,
SafeCare funding was comprehensive and included money for things like training, program
evaluation, and supplies for families. In other systems, organizations had to find ways to make
up the difference between what the contract covered and what was actually required to keep
SafeCare going in their communities. As one agency leader stated, “We do literally lose money
operating the contracts.” A SafeCare coordinator in a different system explained, “We work on
strictly the budget we were given from Child Welfare. In their defense I don’t think they were
prepared enough to know what it costs to run the SafeCare program.”
One way that organizations compensated for insufficient funding was cross training staff.
As one agency leader explained,
With how SafeCare is structured and to maintain fidelity and to maintain X amount that
they want you to carry, there’s no way you’re going to meet that number with one person.
So what we had to do was cross train additional staff members and each of them carry a
case, sometimes some of them carry a little more if they’re not super busy but that’s the
only way that we’re able to meet our program goals.
Other ways that organizations responded to insufficient SafeCare contract funding
included securing private donations, fundraising, pulling from other internal funding sources
(e.g. discretionary funds that are not earmarked for a specific purpose), laying off staff, reducing
hours devoted to SafeCare, creating a client waitlist, and providing SafeCare training to other
organizations in the system at no cost. As these examples illustrate, insufficient contract funding
at the service system level can influence organizational behavior during SafeCare
implementation.
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 45
Contracting arrangements alter inter-organizational relationships and dynamics.
A third way that service systems influence organizations through SafeCare contracts is
when contract-related decisions alter an organization’s relationship with other organizations in
the system. Three of the systems had contracts that formally designated one or more of the
community-based organizations as a lead agency. Three additional systems had lead agencies but
they were not formally designated as such in the contracts. Lead agencies may have
subcontracting relationships with other agencies and are a hub of SafeCare training, coaching,
and expertise. Lead agencies may get more funding to support this larger role. Coordinators at
lead agencies travel to other sites and work directly with employees at other agencies. One
SafeCare coordinator at a lead agency explained this dynamic:
…since automatically they are a subcontractor throughout main contract then definitely I
have more interaction with them because I know exactly what it is that they’re getting. I
know exactly what referrals are going to them. I know exactly when their stats are due
because they’re due to us.
Having a contractually dedicated lead agency can foster a sense of community across
organizations that are providing SafeCare. For example, a SafeCare coordinator at another lead
agency described:
We all see ourselves as SafeCare providers, not I’m from [agency name X], I’m from
[agency name Y], I’m from [agency name Z]. We’re all SafeCare providers. Even though
we’re stationed in [city name], they know our contract has us going all over the
county…There’s more consultation that’s more open with serving the families than there
used to be.
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 46
However, not being a lead agency can result in feelings of dependence, especially around
training new staff and having to work within the constraints of the lead agency. Additionally, in
one system, an organization went from lead agency to subcontractor. This altered inter-
organizational dynamics in that, “people are less open in those meetings now than they used to
be because one of the agencies is now subcontracting with another” [academic collaborator].
When asked how the contracting process affected relationships with other organizations, the
agency leader at this newly subcontracted agency noted that, “It probably strengthened it with
[lead agency X]. I think it’s really weakened it with [lead agency Y].” Contract decisions made at
the service system level can influence an organization’s position within the network of
organizations that provide SafeCare and this influences inter-organizational relationships.
Inner to Outer Context Influence around Contracting Arrangements
This section reverses the lens and looks at influence from the organizational to service
system level. Referring back to Figure 2.1, data revealed several ways that organizations can
influence service systems through SafeCare contracting arrangements. Organizations can alter
contract details based upon on-the-ground implementation realities. Organizations can
collaborate during contract bidding processes. Additionally, organizations can advocate for
changes in service system level SafeCare referral processes that are in the contracts. Each type of
inner to outer context influence is described in detail below.
Organizations can alter contracts based upon on-the-ground implementation
realities.
First, although service system leaders ultimately decide the terms and structure of
SafeCare contracts, the data revealed multiple examples of organizations pushing back on
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 47
contract details based on the realities of implementing SafeCare. Organizational leaders
negotiated with service system leaders around contract details including number of client visits,
caseload size, getting paid for “drop-bys”, funding for administrative support to comply with
SafeCare certification standards, funding for an additional home visitor, training dollars, and
overall funding increases (e.g. daily rates or unit costs). A larger topic of negotiation was
changing the type of contract:
[Service system] was very open to hearing our feedback about wanting to structure it
differently to where it was based on actual amounts instead of just one block amount split
evenly over three years so they were willing to work with us to have it so it was actually
reflecting the true cost of what we planned.
Another significant topic of negotiation in one service system was figuring out how to
keep caseloads up to service system standards, while also paying for the time that it takes home
visitors to get SafeCare certified. This issue required a series of conversations between
organizational and service system leaders. One agency leader summarized this situation as:
And what we finally had to say to them point blank was SafeCare will reduce capacity in
the system by X amount of money and X amount of families…Therefore…you have
they’ve got to pay us additional money to hire additional staff to get the caseload
numbers back up to where they belong…And that was a barrier we had to overcome. We
did overcome that.
In sum, organizations confront daily realities and challenges of SafeCare implementation and
used these experiences to influence the service system by negotiating for contract changes.
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 48
Organizations can collaborate during contract bidding processes.
Second, in two service systems in particular, there were clear examples of organizations
collaborating with each other when applying for SafeCare contracts. This is important because it
restricts the make-up of eligible contract awardees that service system leaders can choose from
and influences the type and quality of bids that service system leaders receive. While individuals
in both systems were conscientious about avoiding collusion, they were also frank about the
informal conversations that they had with each other about subcontracting relationships,
proposed rates, and service areas to bid on. One service system administrator explained these
interactions in this way:
…they just sort of have a conversation across the room and you know, [agency name X]
and [agency name Y] kind of went oh yeah we’ll do it, and the other two knowing they
were weaker said oh yeah, alright well just let them do it, and don’t even bother with the
process. The contractors are a group who by and large with the exception of one agency,
who have been having lunch together once a month for years. So they make their own
sort of internal decisions.
An agency leader in that service system reiterated this collaboration:
I think [agency name X] and [agency name Y] wanted to take the lead in it and decided
to go together, because I told you we felt that given the strength of our two organizations
that we would be best to be the ones who first get it...
A similar situation happened in a different service system. One agency leader said the
following: “We compared notes a lot with financial notes. We wanted to make sure that our bids
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 49
were similar; we wanted to make sure that our price points were similar… we didn’t want to
underbid or overbid each other.”
Another agency leader in that system further explained, “We worked somewhat together when we
were preparing these proposals…When [agency name X] got called back we had almost
identical proposals.” These examples show that while service systems have significant influence
over contract funding and awardees, organizations can influence the service system by
coordinating with each other around the initial bidding process for a SafeCare contract.
Organizations can advocate for changes in service system level referral processes.
A third type of inner to outer context influence is that organizations can advocate for
changes in service system level referral processes. As alluded to above, service systems can
greatly affect an organization’s ability to successfully implement SafeCare by setting service
delivery standards. One specific way that they can affect implementation is through the type of
clients that are referred to the contracted organizations. In two service systems in particular,
organizations actively tried to change SafeCare referral processes at the service system level.
As one organizational level upper manager recounted: “…the biggest challenge we had
was the timeliness of receiving the referrals…to the point where we had to really advocate for a
change…” This participant also said the organization had to “push back” and “remind [service
system entity]” around the issue of SafeCare referrals.
In another system, organizations worked with the service system to create a new type of
referral that combined a SafeCare home visitor and a liaison from another program:
…and so managing that relationship and that connection between another program and
SafeCare home visitors and what’s going on with that, how are things being coordinated,
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 50
you know we had to go through this whole thing about at one point the discussion was
about who has the lead. There’s been that whole piece that was not in existence before
[agency leader].
To generate referrals, this service system also created more stringent screening processes and
more data tracking around referrals (e.g. contract requires quarterly reports that include the
number of referrals, broken out by source and the reasons for closing both a referral and a case).
An organization’s ability to implement SafeCare is completely dependent on having enough
eligible clients. Advocating for changes in the contract’s referral processes is a way that
organizations can influence service systems during SafeCare implementation.
Data also revealed multiple dependencies around contracting arrangements. Figure 2.2
summarizes dependencies for each level of analysis and each item is described in greater detail
in the next sections.
Service Systems ” Dependencies on Contracted Organizations
While much of the influence around contracting arrangements lies at the service system
level, several sources of service system dependence on organizations emerged in the data. First,
because service systems are contracting out SafeCare services, organizations are often the EBP
experts and can be keepers of institutional memory when service system level leadership
changes. This institutional memory represents knowledge about the history of SafeCare
implementation in the system, including initial contract development, contract decision-making,
and contract changes over time. As one service system administrator described,
It’s nice because they can give the history. Sometimes if I’m not sure about something I
can call the contractors and say why is the process this way and they’ll give the history of
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 51
that and I can trust that they’re giving me—because I’ve gone back and looked at past
contracts and I’m like oh they gave me exactly that.”
An agency leader in a different system also discussed the need to “educate” the service
system when there is turnover:
…you have to retrain and reeducate the new [service system] people again on what
SafeCare is and how it functions and this is the past history…and then we have to come
back and do a little bit more re-education and building again.
Service system administrators may be dependent on organizational level players to understand
SafeCare and communicate how it has been implemented in the system over time.
A SafeCare coordinator in an organization in another service system further explained the
dynamic of the organization being the hub of SafeCare expertise: “The [service system] would
call me. The resource managers would call me. In a meeting where we were going over quarterly
numbers… they started shooting these questions and it just kind of evolved. I’m the go-to
person.” Again, service system administrators may be dependent upon contracted organizations
when understanding what SafeCare is and how it works in their system.
Second, service systems may be dependent on organizations when they are the only ones
interacting with program developers at the National SafeCare Training and Research Center
(NSTRC). One service system administrator describes the lead organization as “bypassing us”:
“…NSTRC and the coach are in direct contact. We are not in that interaction... Unless she sends
that and copies us in the process we know nothing about what’s going on between them” [service
system administrator]. An administrator in another service system also discussed how the
organization’s SafeCare coordinator works directly with NSTRC, manages SafeCare program
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 52
data for the system, and “keeps the model fidelity going” throughout the system. This type of
dynamic can make service system administrators dependent upon agency leaders to reliably relay
information and feedback from NSTRC.
Third, service systems may be dependent on contracted organizations when they are the
sole source of SafeCare training and coaching for the entire system. One service system
administrator discussed two options for addressing this issue:
One way—what we need to do is diminish our sole reliance on one entity so one way is
bringing internal capacity to do the coaching or training. The other is contract it out with
multiple providers so if you lose one, you’re okay you’ve got some redundancy,
somebody else can do it... so there isn’t just one entity that if we lose them we’re in real
trouble.”
This issue occurred in another service system as well: “The CBOs actually are the ones who have
the expertise. I think they even train our [staff]...” [service system administrator].
To summarize, data revealed the following potential sources of service system
dependencies on organizations around contracting arrangements: organizations are often the EBP
experts and can be keepers of institutional memory, organizations may be the only link to
program developers at NSTRC, and organizations may be the sole source of SafeCare training
and coaching for the entire system.
Contracted Organizations ” Dependencies on Service Systems
Several sources of organizations” dependencies on the service system around contracting
arrangements also emerged in the data. Most obviously, service system leaders control funding
and decide which organizations are awarded contracts. Additionally, individuals in the service
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 53
system often dictate eligibility requirements and control number and type of client referrals that
organizations receive. Eligible client flow can greatly impact the organization’s ability to
implement SafeCare. For example, one service system increased the minimum age for SafeCare
clients and this led to organizations receiving an influx of clients who were not appropriate for
(and therefore unlikely to successfully complete) the SafeCare model. A leader in this system
explained,“… they were getting a lot of inappropriate referrals they said, a lot of teens, and
really cases that weren’t appropriate for SafeCare”.
Service system leaders can also decide if other services can be used concurrently with
SafeCare. Similar to client flow, this is a source of organizational dependence because using
other services with SafeCare can affect an organization’s ability to engage and retain SafeCare
clients (and meet program goals). As one agency leader described:
… there were just a number of cases where folks were just not ready to receive just
SafeCare. There are things that need to happen, there are more pressing issues and
families that we really were not contracted to provide and so it looks like we’re going to
be doing some expanding work in addition to SafeCare. We’re excited about that, yeah.
Organizations are dependent on service systems because contracting arrangements
structure service delivery in ways that affect the likelihood of implementation success: funding,
client flow, and ability to use other programs or services concurrently with SafeCare.
Discussion
In privatized child welfare systems, contracting arrangements link public child welfare
entities with contracted organizations. If a public child welfare system decides to adopt a new
evidence-based practice, contracting arrangements structure service delivery and are a critical
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 54
medium through which service systems and organizations communicate, interact, and negotiate
over the course of implementation. Rather than examining contracting arrangements as a passive
feature of the outer context, this study actively examines the ways in which service systems and
organizations influence each other through contracts over the course of SafeCare
implementation. Taking an open systems approach to EBP implementation requires this kind of
examination of directions of influence, types of influence, and resources that are needed and
exchanged across the outer and inner contexts.
The longitudinal analysis of eight diverse service systems and 17 organizations revealed
multiple sources of bi-directional influence around SafeCare contracting arrangements. The data
revealed several ways that service systems can influence organizations through contracting
arrangements during SafeCare implementation. Contracting arrangements decided at the service
system level can result in: (1) changes to organizational level processes, procedures and/or
staffing decisions, (2) organizational responses to compensate for insufficient funding, and (3)
changes in inter-organizational relationships and dynamics. These findings are consistent with
previous work regarding the influence of contracting arrangements on human service
organizations (McBeath et al. 2019; Trott Jaramillo et al., 2018). The data also revealed several
ways that organizations can influence service systems through contracting arrangements.
Organizations can influence the service system by: (1) altering contracts based upon on-the-
ground implementation realities, (2) collaborating during contract bidding processes, and (3)
advocating for changes in the contract’s referral processes. Content analysis of contract
documents triangulated transcript data and illustrated detailed ways that contracts structure the
way that SafeCare is delivered by community-based organizations.
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 55
While these findings may be unsurprising to practitioners, existing and widely used
conceptual frameworks miss these important dynamics because they do not treat organizations as
open systems. When the outer and inner contexts are separated, organizational ties to the external
environment are severed, and these sources of bi-directional influence are often unaccounted for.
Separating the outer and inner implementation contexts also ignores dependencies between
systems and organizations. Resource dependence theory is a useful theoretical lens for
considering this aspect of the findings and enhancing our use of frameworks, like the EPIS.
Resource dependence theory asserts that organizational survival depends on acquiring and
maintaining essential resources (Pfeffer & Salancik, 1978).
Not surprisingly, privately contracted organizations are dependent on public child welfare
entities for sufficient and sustained funding. Findings revealed another resource that service
systems have and organizations need: appropriate number and quality of clients that can
successfully complete SafeCare and allow the organization to reach program goals. Public child
welfare entities can control critical aspects of the referral process including actual number of
clients that organizations receive, eligibility requirements, and an organization’s ability to
engage and retain clients by supplementing SafeCare with other curricula. Resource dependence
theory also asserts that organizational behavior can be understood by looking at reactions to
environmental contingencies and constraints (Pfeffer & Salancik, 1978). The organizational
responses identified in this study, for example changing organizational processes to align with
contract requirements or finding additional funding to supplement contract inadequacies, shows
an attempt to adjust to environmental constraints imposed by the service system.
More interestingly perhaps, is that data also revealed critical resources that can make
public child welfare entities dependent upon contracted organizations. Service system leaders
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 56
who make contracting decisions are accountable to a range of stakeholders including state and
federal legislatures, and the general public. Successful system-wide implementation of an EBP
depends on the abilities of the organizations that are contracted to deliver the practice to clients.
Resources that contracted organizations have are EBP expertise (including access to program
developers), and training and coaching capacity. These are assets that organizations build over
time. Service system leaders drew attention to these dependencies, for example, by discussing
the need for in-house SafeCare trainers and coaches to spread risk and minimize dependence on
the system’s lead agency.
This chapter illustrates the organizations as implementation actors approach described in
the Introduction. This new theoretical metaphor for implementation science highlights and
acknowledges the relational nature of organizations. Organizations implementing an evidence-
based practice exist within an intricate relational web, and this chapter shows how organizations
and child welfare systems interact and exchange resources through contracting arrangements.
Additionally, this chapter applies a resource dependence theoretical lens and illustrates specific
ways that the organization-service system relationship creates dependencies and constraints over
the course of SafeCare implementation. An open systems approach to EBP implementation
acknowledges the interdependence between the external environment and organizations.
This study also has implications for practice. Addressing the dynamic outer-inner
boundary can enhance our use of frameworks like the EPIS, and make these types of frameworks
more reflective of real-world complexity. Additionally, analyzing influence, dependencies, and
resources across the outer-inner context boundary, has direct implications for selecting and
tailoring implementation strategies (Birken et al., 2017). Implementation strategies are defined as
“a systematic intervention process to adopt and integrate evidence-based health innovations into
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 57
usual care” (Powell, et al. 2012). Implementation strategies are categorized based on key
implementation processes: planning (e.g. build buy-in), educating (e.g. conduct educational
outreach visits), financing (e.g. modify incentives), restructuring (e.g. create new clinical teams),
quality management (e.g. develop tools for quality monitoring), and attending to the policy
context (e.g. change accreditation requirements) (Powell, et al. 2012; Hurlburt et al. 2014).
Current research has shifted from a focus on describing and defining, to selecting and
tailoring strategies (Powell, et al. 2017). Potential methods for selecting and tailoring
implementation strategies include concept mapping (stakeholders brainstorm, sort, and rate
tasks), group model building (stakeholders create and discuss a simulation model), and conjoint
analysis (stakeholders state and compare preferences) (Powell, et al. 2017). An explicit inclusion
of outer-inner influences and resource dependencies could enhance all of these methods and
bring to light critical environmental constraints and opportunities that affect the potential success
of different implementation strategies in a particular service system (Birken et al., 2017). A
realistic and concurrent consideration of available and needed resources (human capital, social
capital, informational) in system and organizational levels can better represent the environmental
constraints that practitioners face when selecting implementation strategies. Findings from this
study, for example, could help leaders who are considering system-wide SafeCare
implementation better assess which strategies are most likely to be feasible and sustainable given
the structure of and resources included in the contracting arrangements.
Strengths, Limitations and Future Research Directions
This study used a comprehensive longitudinal dataset, included outer and inner context
stakeholder perspectives, and triangulated transcripts and contracting documents. Key limitations
are that the analysis only focused on a single evidence-based practice and sustaining sites. Outer-
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 58
inner interactions around contracting arrangements could be quite different in the context of
another evidence-based practice. These interactions could also vary in systems that failed to
successfully implement and sustain SafeCare. Moreover, systems are not static. As service
systems and human service organizations evolve, existing resource dependencies may change
and new dependencies may emerge. Even though this analysis is longitudinal, it still represents a
specific period of SafeCare implementation in each of these systems.
There are countless directions for future research on this understudied topic. One broad
avenue of future research is examining outer-inner interactions around contracting arrangements
in the context of EBPs other than SafeCare. Future research can also explore influence and
dependencies in other organizational relationships (e.g. relationships with intervention
developers, EBP intermediaries, and academic partners). Additionally, the new implementation
capital framework could enhance future examinations of the resources exchanged within an
organization’s social web (Neal & Neal, 2019). The implementation capital framework combines
implementation outcomes with the social capital construct and focuses on ways that
implementers’ social networks can support or hinder the implementation of EBPs (Neal & Neal,
2019). The authors of this new framework suggest that rather than focusing on individual level
forms of social capital, it is more helpful to focus on “small worlds” and examine bridging and
bonding capital at the setting level (Neal & Neal, 2019).
Another question that remains after this study is: How do organizations respond to
environmental constraints (like contracting arrangements) and adapt over the course of EBP
implementation? Do organizations approach the process of adaptation differently during EBP
implementation? The next chapter is situated within the organizational level of analysis and more
closely examines the 17 organizations embedded in the eight child welfare systems described
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 59
here. The goal of Chapter 3 is to identify and compare different organizational approaches to
adaptation that occurred during SafeCare implementation and sustainment.
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 60
References
Aarons, G.A., Green, A.E., Trott, E., Willging, C.E., Torres, E.M, Ehrhart, M.G., & Roesch, S.C.
(2016). The roles of system and organizational leadership in system wide evidence-based
intervention sustainment: A mixed method study. Administration and Policy in Mental
Health and Mental Health Services Research, 43(6), 991–1008.
Aarons, G. A., Hurlburt, M., & Horwitz, S. M. (2011). Advancing a conceptual model of
evidence-based practice implementation in public service sectors. Administration and Policy
in Mental Health and Mental Health Services Research, 38, 4–23.
Bernard, H.R., Wutich, A., & Ryan, G.W. (2017). Analyzing Qualitative Data: Systematic
Approaches (2
nd
ed.). Thousand Oaks, CA: Sage Publications, Inc.
Birken, S.A., Bunger, A.C., Powell, B.J., Turner, K., Clary, A.S., Klaman, S.L…Weiner, B.J.
(2017). Organizational theory for dissemination and implementation research.
Implementation Science, 12, 62-77.
Bunger, A.C., Cao, Y., Girth, A.M., Hoffman, J., & Robertson, H.A. (2016). Constraints and
benefits of child welfare contracts with behavioral health providers: Conditions that shape
service access. Administration and Policy in Mental Health and Mental Health Services
Research, 43, 728-739.
Chaffin, M., Hecht, D., Bard, D., Silovsky, J. F., & Beasley, W. H. (2012). A statewide trial of
the SafeCare home-based services model with parents in child protective services. Pediatrics,
129, 509–515.
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 61
Damschroder, L.J., Aron, D.C., Keith, R.E., Kirsh, S.R., Alexander, J.A., & Lowery, J.C. (2009).
Fostering implementation of health services research findings into practice: A consolidated
framework for advancing implementation science. Implementation Science, 4, 50-65.
Feldstein, A.C. & Glasgow, R.E. (2008). A Practical, Robust and Sustainability Model (PRISM)
for integrating research findings into practice. The Joint Commission Journal on Quality and
Patient Safety, 34(4), 228- 243.
Gardner, R., Hodson, D., Churchill, G., & Cotmore, R. (2014). Transporting and implementing
the SafeCare® home-based programme for parents, designed to reduce and mitigate the
effects of child neglect: An initial progress report. Child Abuse Review, 23, 297–303.
Green, A.E., Trott, E., Willging, C.E., Finn, N.K., Ehrhart, M.G., & Aarons, G.A. (2016). The
role of collaborations in sustaining an evidence-based intervention to reduce child neglect.
Child Abuse & Neglect, 53, 4-16.
Hurlburt, M., Aarons, G., Fettes, D., Willging, C., Gunderson, L., & Chaffin, M. (2014).
Interagency Collaborative Team model for capacity building to scale-up evidence-based
practice. Children and Youth Services Review, 39, 160–168.
Katz, D. & Kahn, R.L. (1966). The social psychology of organizations. New York, NY: John
Wiley & Sons, Inc.
McBeath, B., Chuang, E., Carnochan, S., Austin, M.J., & Stuart, M. (2019). Service coordination
by public sector managers in a human service contracting environment. Administration and
Policy in Mental Health and Mental Health Services Research, 46, 115-127.
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 62
National Conference of State Legislatures. (2017). Child welfare financing 101. Retrieved
January 7, 2019 from http://www.ncsl.org/research/human-services/child-welfare-financing-
101.aspx .
National SafeCare Training and Research Center. (2016). Provider manual: SafeCare overview.
Atlanta, GA: Georgia State University.
Neal, J.W. & Neal, Z.P. (2019). Implementation capital: Merging frameworks of implementation
outcomes and social capital to support the use of evidence-based practices. Implementation
Science, 14, 16-25
Novins, D.K., Green, A.G., Legha, R.K., and Aarons, G.A. (2013). Dissemination and
implementation of evidence-based practices for child and adolescent mental health: A
systematic review. Journal of the American Academy of Child & Adolescent Psychiatry,
52(10), 1009-1025.
Pfeffer, J. & Salancik, G.R. (1978). The external control of organizations. New York, NY:
Harper & Row.
Powell, B.J., McMillen, C.J., Proctor, E.K…& York, J.L. (2012). A compilation of strategies for
implementing clinical innovations in health and mental health. Medical Care Research and
Review, 69(2), 123-157.
Powell, B.J., Beidas, R.S., Lewis, C.C…& Mandell, D.S. (2017). Methods to improve the
selection and tailoring of implementation strategies. Journal of Behavioral Health Services &
Research, 44(2), 177–194.
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 63
Roskinsky, K. & Connelly, D. (2016). Child welfare financing SFY 2014: A survey of federal,
state, and local expenditures (Publication # 2016-53). Retrieved from Child Trends website:
https://www.childtrends.org/wp-content/uploads/2016/10/2016-
53ChildWelfareFinancingSFY2014-1.pdf .
Stake, R.E. (2006). Multiple case study analysis. New York, NY: The Guilford Press.
Stirman, S.W., Kimberly, J., Cook, N., Calloway, A., Castro, F., & Charns, M. (2012). The
sustainability of new programs and innovations: A review of the empirical literature and
recommendations for future research. Implementation Science, 7, 17-36.
Trott Jaramillo, E., Willging, C.E., Green, A.E., Gunderson, L.M., Fettes, D.L. & Aarons, G.A.
(2018). “Creative financing”: Funding evidence-based interventions in human service
systems. Journal of Behavioral Health Services & Research [Epub ahead of print].
U.S. Department of Health & Human Services, Administration for Children and Families,
Administration on Children, Youth and Families, Children’s Bureau. (2017). Child
maltreatment 2015. Available from http://www.acf.hhs.gov/programs/cb/research-data-
technology/statistics-research/child-maltreatment .
National Institute of Health. (2017). National Institute of Mental Health Strategic Plan for
Research. Available from https://www.nimh.nih.gov/about/strategic-planning-
reports/nimh_strategicplanforresearch_508compliant_corrected_final_149979.pdf .
U.S. Department of Health & Human Services, Administration for Children and Families,
Administration on Children, Youth and Families, Children’s Bureau. (2018). State vs. county
administration of child welfare services. Available from
https://www.childwelfare.gov/pubs/factsheets/services/ .
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 64
U.S. Department of Health and Human Services Office of the Assistant Secretary for Planning
and Evaluation. (2007). Topical paper #2: Program and fiscal design elements of child
welfare privatization initiatives. Retrieved from https://aspe.hhs.gov/basic-report/program-
and-fiscal-design-elements-child-welfare-privatization-initiatives .
Willging, C., E., Aarons, G.A., Trott, E.M…Hecht, D.B. (2016). Contracting and procurement
for evidence-based interventions in public-sector human services: A case study.
Administration and Policy in Mental Health and Mental Health Services Research, 43, 675-
692.
Willging, C.E., Green, A.E., Gunderson, L., Chaffin, M., & Aarons, G.A. (2015). From a
““perfect storm”“ to ““smooth sailing”“: Policymaker perspectives on implementation and
sustainment of an evidence-based practice in two states. Child Maltreatment, 20(1), 24-36.
Willging, C.E., Gunderson, L., Green, A.E., Trott Jaramillo, E., Garrison, L., Ehrhart, M.G., &
Aarons, G.A. (2018). Perspectives from community-based organizational managers on
implementing and sustaining evidence-based interventions in child welfare. Human Service
Organizations: Management, Leadership and Governance, 42(4), 359-379.
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 65
Table 2.1.
Service System Descriptive Information
Service
System
Population
Estimate
a,b
Median Household
Income
a
Persons in
poverty
a
Population per
square mile
c
Land in
square miles
c
Year system
started SafeCare
State 3,943,079 $49,767 15.8% 54.7 68,595 2003
County 1 1,419,516 $71,535 14.5% 4,020.4 325 2008
County 2 464,493 $44,871 24.0% 91.1 4,824 2009
County 3 2,423,266 $60,807 12.9% 303.8 7,206 2012
County 4 884,363 $96,265 10.1% 17,179.1 47 2011
County 5 448,150 $68,023 14.2% 155.0 2,735 2010
County 6 179,921 $47,258 17.0% 46.9 3,775 2009
County 7 854,223 $81,972 9.5% 446.7 1,843 2012
a
2018 U.S. Census Bureau data.
b
2017 U.S. Census Bureau data.
c
2010 U.S. Census Bureau data.
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 66
Table 2.2.
Data Sources for Each Service System
Service
System
Date range of
collected data
# of
organizations
Contracting
documents
Qualitative data
# of
documents
# of
pages
Individual
interviews
Small group
interviews
Focus
groups
State 2005- 2016 3 21 606 19 6 0
County 1 2008- 2018 4 17 1759 20 1 0
County 2 2009- 2018 3 13 242 7 1 1
County 3 2011- 2018 2 18 274 4 3 0
County 4 2011- 2018 2 12 161 9 0 0
County 5 2010- 2018 2 15 419 5 1 0
County 6 2009- 2018 1 12 403 6 0 0
County 7 2012- 2018 1 5 275 3 2 0
Total 18 113 4,139 73 14 1
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 67
Table 2.3.
Codes for Content Analysis
Service delivery codes
Caseload size
Length of service delivery period
Length of sessions
Use of other services
Client age
Staffing codes
Specific # of SafeCare home visitors, coaches or trainers
Home visitor qualifications
SafeCare training
SafeCare coaching
Process codes
Referral processes
Data reporting processes
SafeCare outcomes
Other
Lead agency designated
Sustainment explicitly mentioned
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 68
Table 2.4.
Codes for Focused Coding of Transcripts
Org dependence on service system
ex: referrals
Insufficient funds force org response
ex: cross-training
Building in implementation supports
ex: funding for training
Contract requirements altering org behavior
ex: staffing decisions
Contracts alter org relationships
ex: new subcontracting relationships
Service system dependence on orgs
ex: orgs as SafeCare experts
Orgs negotiating SafeCare contract details
ex: caseload size
Orgs influencing service system processes
ex: referral process
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 69
Table 2.5.
Service System Characteristics
Service
System
Contract type as
of 2016
Full costs
covered by
SC contract
Stability of SC
contract
Rigor of SC
contract
oversight
CBOs able
to influence
SC
contract
Contract
stipulate work
with academic
partners
SC-related
academic
partnerships
State Performance-based Yes
Unstable during
major contract
change
High No Yes Long term
County 1
Cost
reimbursement &
performance-based
Yes Stable High Yes In early contracts Long term
County 2
Cost
reimbursement
Mixed views Stable Low
Mixed
views
No Short term
County 3
Cost
reimbursement
Mixed views Stable High Yes No Short term
County 4
Cost
reimbursement
Yes Stable Low Yes In early contracts Short term
County 5
Cost
reimbursement
No Unstable Low
Mixed
views
In early contracts Short term
County 6
Cost
reimbursement
No Stable Low Yes No Short term
County 7
Cost
reimbursement
Yes Stable Low Yes No Short term
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 70
Table 2.6.
Content Analysis of SafeCare Contracting Documents by Service System
S* C1* C2 C3 C4 C5 C6* C7*
Service delivery codes
Caseload size x x x x x x x
Length of service delivery period x x x x x x x
Length of sessions x x x x x x
Use of other services x x x x x
Client age x x x x x x x x
Staffing codes
Specific # of SafeCare home visitors, coaches, or trainers x x x x x x
Home visitor qualifications x x x x x x
SafeCare training x x x x x x x x
SafeCare coaching x x x x x x
Process codes
Referral processes x x x x x x x x
Data reporting processes x x x x x x x
SafeCare outcomes x x x x x x x
Other
Lead agency designated x x x
Sustainment explicitly mentioned x x x x
Notes. S= State; C1= County 1, C2= County 2, etc. *SafeCare is embedded within a broader child welfare
program, not stand-alone SafeCare contract
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 71
Figure 2.1. Sources of Influence across the Outer and Inner Contexts
Figure 2.2. Dependences across the Outer and Inner Contexts
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 72
Chapter Three (Study 2):
Approaches to Organizational Adaptation during EBP Implementation:
Incorporators, Early Investors, and Learners
Abstract
While current literature acknowledges that organizations and service systems must adapt
during EBP implementation, this category of adaptation is still unclear and unexamined. The
present analysis aims to address the following research question: How do organizations adapt
over the course of SafeCare implementation? The goal of this study is to identify and compare
different organizational-level approaches to adaptation that emerged during SafeCare
implementation and sustainment. Longitudinal qualitative data from 17 community-based
organizations in one state and seven county-based child welfare systems are used. Three
organizational-level profiles of adaptation emerged: Incorporators (n=7), Early Investors (n=6),
and Learners (n=4). A key feature of the Incorporators is that they are primarily integrating
SafeCare into what they already do as an organization, rather than making substantial changes or
creating new ways of operating. Unlike Incorporators, Early Investors changed their adaptation
approach over time. Early Investors made substantial adaptations during early implementation,
then overall engagement in adaptation tapered off as SafeCare became more embedded in the
organization and service system. What characterized the Learners is steady and continuous
attention to new ways that the organization can support SafeCare. Learners seem to have a
culture and/or leaders that support a commitment to adapting, improving, and keeping SafeCare
fresh in the organization throughout the implementation lifecycle. Findings have both practical
and theoretical implications.
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 73
Introduction
Organizational Adaptation: An Unexamined Topic in Implementation Research
Sustainment of an evidence-based practice (EBP) occurs when an organization
demonstrates the capacity to deliver an intervention’s core components long-term and without
implementation support (Hunter, Han, Slaughter, Godley, & Garner, 2017; Stirman et al., 2012).
Sustainment depends on individual practitioners’ ability to acquire new knowledge and training,
unlearn behaviors, change attitudes and beliefs, and adapt (Gupta, Boland, & Aron, 2017;
Kennedy et al., 2014; Stirman et al., 2015). EBP sustainment also depends on an organization’s
ability to learn, unlearn, change norms and shared perceptions, and adapt (Aarons, Moullin, &
Ehrhart, 2018; Baumann, Cabassa, & Stirman, 2018). This study treats organizations as
implementation actors that are just as important to EBP sustainment as individual clinicians and
providers.
Adaptation is defined as the systematic modification of an EBP to fit provider
characteristics, organizational contexts, and service settings (Baumann, Cabassa, & Stirman,
2018). It is widely accepted that adaptation is an unavoidable and even beneficial aspect of EBP
implementation (Baumann, Cabassa, & Stirman, 2018). Baumann, Cabassa, and Stirman (2018)
provide a framework for understanding EBP adaptation that includes three types of adaptation:
changes to content, context, and training and evaluation (Baumann, Cabassa, & Stirman, 2018).
In a recent systematic review, Stirman and colleagues identify a variety of content changes
including tailoring, removing, adding, shortening, lengthening, integrating, repeating, reordering
and loosening intervention elements (Stirman, Gamarra, Bartlett, Calloway, & Gutner, 2017).
Context level changes involve changing the intervention format (e.g. group vs.
individual), setting (outpatient vs. community-based setting), personnel (mental health
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 74
professional vs. peer specialist), and population (tailoring intervention to a new client group, e.g.
gay men or veterans) (Baumann, Cabassa, and Stirman, 2018). Adaptation may also involve
changing training and evaluation materials and processes to facilitate implementation (Baumann,
Cabassa, and Stirman, 2018). In this framework, the organization is treated as a “structural factor
that can influence adaptation” (Baumann, Cabassa, and Stirman, 2018). Escoffery and
colleagues’ (2018) systematic review of adaptations focuses on similar categories: changes to
content, context, cultural modifications and delivery (Escoffery et al., 2018). Neither of these
comprehensive conceptualizations of adaptation explicitly address how organizations themselves
must adapt to accommodate the implementation of a new EBP, and this category of adaptation
remains unclear and unexamined in the implementation literature (Stirman et al., 2017).
SafeCare Adaptation Research
The current study explores how organizations adapt over the course of SafeCare
implementation. SafeCare is a well-established and behaviorally prescriptive EBP designed to
prevent child neglect and improve outcomes for children at risk of abuse and neglect (Gardner,
Hodson, Churchill, & Cotmore, 2014; Chaffin, Hecht, Bard, Silovsky, & Beasley, 2012).
SafeCare home visitors work in families” homes to improve parent skills in parent-infant and
child interactions, home safety, and health (National SafeCare Training and Research Center,
2016). Home visitors, coaches, and trainers achieve and maintain certification, and coaches work
closely with home visitors to ensure ongoing adherence to the SafeCare model (National
SafeCare Training and Research Center, 2016).
Current SafeCare adaptation research focuses on three topics: (1) changes to the content
of the intervention (e.g. adding motivational interviewing and adapting material for male
caregivers and mothers with intellectual disabilities), (2) cultural adaptations (e.g. adaptations for
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 75
Latino families), and (3) changes to delivery (e.g. using iPhones and tablets) (Silovsky et al.,
2011; Gaskin, Lutzker, Crimmins, & Robinson, 2012; Self-Brown et al., 2017; Beasley et al.,
2014; Lefever et al., 2017, Self-Brown et al., 2017b; Finno-Velasquez, 2014 ). The present study
extends this work by exploring different ways that organizations adapt to accommodate SafeCare
implementation.
Profiles of Organizational Adaptation during EBP Implementation
A typological approach provides the foundation for describing organizational adaptation
in this study. Rather than listing and categorizing specific types of changes that organizations
make during implementation, organizations that have sustained SafeCare are treated as
implementation actors that engage in the process of adaptation in different but equally effective
ways. Typologies are a widely used theory-building tool in the organizational and strategic
management literatures and can be a useful way for organizing “complex webs” of cause-effect
relationships across contextual, structural and strategic dimensions (Fiss, 2011; Doty & Glick,
1994). One popular example is from Miles and Snow (1978) who characterized organizations
based on four types of strategies: prospector, analyzer, defender and reactor. Prospector
organizations, for example, embrace change and opportunity in the environment and actively
create new products and seek out new initiatives (Miles & Snow, 1978). Analyzer organizations,
on the other hand, prioritize stability of the organization’s activity and typically innovate in
limited and constrained ways that are related to the core business (Miles & Snow, 1978). Within
each strategy type a variety of activities and actions can occur.
Typologies are a useful way of thinking about organizational adaptation during EBP
implementation because they “identify multiple ideal types, each of which represents a unique
combination of the organizational attributes that are believed to determine the relevant outcome”
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 76
(Doty & Glick, 1994). This aligns with the organizations as implementation actors metaphor
because it acknowledges that organizations are a collection of attributes. Typologies allow for
the examination of combinations of interacting characteristics that drive organizational behavior
(rather than de-contextualizing specific organizational features, e.g. culture). The term
“adaptation profile” is used in this dissertation study to describe organizational adaptation
approaches. Each profile captures that “unique combination of the organizational attributes” that
led to that particular way of adapting and ultimately, to achieving SafeCare sustainment.
Use of the Exploration-Preparation-Implementation-Sustainment (EPIS) Framework
The EPIS framework outlines inner (client, staff, organization) and outer (system,
community) contextual features that are most relevant to four phases of EBP implementation:
Exploration, Preparation, Implementation, and Sustainment (Aarons, Hurlburt, & Horwitz,
2011). The EPIS framework also describes two types of practice-organization fit (Aarons,
Hurlburt, & Horwitz, 2011). “Structural fit” describes the degree to which an EBP (or the
process of implementing the EBP) aligns with the organization’s existing roles, responsibilities,
resources, and capabilities. “Ideological fit” describes the ways that an EBP aligns with the
organization’s mission and the values, goals, and priorities of the individuals within it (Aarons,
Hurlburt, & Horwitz, 2011). Both structural fit and ideological fit are considered in this study’s
adaptation profiles. The EPIS framework is a conceptual lens that helps to: (1) distinguish
adaptation that is occurring at the system versus the organizational level; (2) map changes in
adaptation behavior across the implementation stages; and (3) identify examples of adaptation
that are occurring due to both structural and ideological misfit.
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 77
Study Goal
The present analysis aims to address the following research question: How do
organizations adapt over the course of SafeCare implementation? The goal of this study is to
identify and compare different organizational-level approaches to adaptation that emerged during
SafeCare implementation and sustainment. Longitudinal qualitative data from 17 community-
based organizations in one state and seven county-based child welfare systems are used. Rather
than focusing on narrow examples or specific instances of organizational changes, this study
aims to create composite organizational “adaptation profiles”. To the author’s knowledge, there
are no previous implementation studies that assess adaptation in this way.
Methods
Study Context
The present study utilizes secondary qualitative data collected during two iterative mixed
methods parent studies. These two parent studies were conducted 2008-2013 and 2012-2017 and
were part of a program of SafeCare implementation research that began in 2005. Inclusion
criteria for the organizations in this dissertation sample are that they: (1) were in a system that
achieved SafeCare sustainment status and (2) had at least two time points for the secondary
qualitative data. Sustainment status was assessed by Aarons and colleagues (Aarons et. al.,
2016). The sample for this study is 17 community-based organizations in one state (State) and
seven county-based systems (County A, County B, etc.). Features of these systems are described
in Chapter 2. While the parent studies broadly assessed SafeCare implementation and
sustainment processes, this dissertation study narrows the focus to organizational adaptation. The
working definition of organizational adaptation developed for this study is “any changes or extra
effort that organizations make to support SafeCare implementation over time.”
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 78
Data Collection
The unit of analysis is organizations and each organization represents a case. To assess
each of the 17 cases, qualitative data from diverse stakeholders including frontline staff, team
leaders and supervisors, agency leaders, and county administrators were used. Interview and
focus group script questions assessed various aspects of SafeCare implementation and
sustainment. The transcripts used in the 2016 data collection cycle included adaptation questions
and prompts developed by this dissertation’s PI. These adaptation questions are included in the
Appendix. In the parent study, all transcripts were digitally recorded, professionally transcribed,
de-identified, and checked for accuracy by at least one of the interviewers.
A total of 177 transcripts were analyzed including 119 individual interviews, 20 small
group interviews and 38 focus groups. Some of the focus groups brought together participants
from different organizations within the same child welfare system. Table 3.1 describes the data
types and years collected for each service system. The analysis of 113 SafeCare contracting
documents from Chapter 2 was used to contextualize, triangulate, and expand upon the
transcripts (Palinkas et al., 2011).
The number of qualitative sources and data collection time periods depended on when
each organization participated in the parent studies. Given the specific aims of the parent studies,
some organizations have more transcripts than others. Several steps were taken to mitigate these
discrepancies and ensure that thematic saturation was reached for all cases: (1) Of the 17
organizations in the dissertation sample, two organizations only had one round of qualitative
data. The dissertation PI was able to conduct follow-up interviews for these two organizations in
August and September 2018. (2) Peer debriefing and member-checking activities (discussed
below) were another opportunity to learn about each organization and supplemented what was
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 79
learned in the transcripts. (3) Participants frequently discussed other organizational and service
system actors in their interviews and focus groups. As a result, information about each
organization not only included the perspective of the individuals in that organization, but also the
perspectives of other participants who knew and talked about that organization.
To understand how adaptation is formally discussed by program developers, this study’s
PI also collected and reviewed the most recent home visitor (272 pages), coach (68 pages) and
trainer (68 pages) SafeCare manuals from the National SafeCare Training and Research Center.
The home visitor manual includes an overview of SafeCare and how to engage clients, as well as
detailed steps for implementing each of the SafeCare modules. The coach manual provides an
overview of SafeCare coaching, descriptions of SafeCare core concepts and the coach training
process, and instructions for fidelity assessment, coaching sessions, and SafeCare meetings.
Adaptation, cultural considerations, and common coaching challenges are also included. The
trainer manual provides an overview of the SafeCare trainer role, instructions for conducting
home visitor and coach workshops, and information about how to support coaches and complete
the trainer training process.
Data Analysis Overview
Adaptation profile development occurred in five stages. The first stage was initial coding
of the transcripts and peer debriefing meetings (Charmaz, 2006; Padgett, 2017). The second
stage was focused coding around topics that emerged as particularly important to understanding
adaptation in this study context (Bernard, Wutich, & Ryan; 2017). The third stage was a
document review of SafeCare manuals and triangulation with the contracting documents
assessed in Chapter 2. The fourth stage was within and between case analyses using a multiple
case study approach (Stake, 2006). The fifth stage was the creation of a visual elicitation tool and
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 80
validation of the adaptation profiles through member checking (Bagnoli, 2009; Padgett, 2017).
Peer debriefing was used in the coding process, whereas member checking was used to finalize
the adaptation profiles. The next sections describe each stage.
Five Analytic Stages
The first stage of analysis consisted of line-by-line initial coding of the 177 qualitative
transcripts for anything potentially adaptation related. Initial coding is “provisional, comparative,
and grounded in the data” (Charmaz, 2006). Coded material was arranged by organization so that
the dissertation PI could review each organization’s codes in peer debriefing meetings. Padgett
(2017) describes peer debriefing as “a type of group reflexivity that gives the researcher fresh
perspectives and guards against bias”. The purpose of the peer debriefing meetings in this
analysis was to review initial coding decisions, questions, and gaps. The dissertation PI engaged
in five peer debriefing meetings from November 2017 to June 2018. The dissertation PI met with
lead investigators and a qualitative data expert from the parent studies, all of whom have
extensive experience with the participating organizations and SafeCare implementation issues.
Feedback from these peer debriefing meetings highlighted factors that required further
investigation and attention. The lead investigators and a qualitative data expert from the parent
studies highlighted certain topics that were particularly relevant to understanding and assessing
organizational adaptation in this study context. In response to peer debriefing, the second stage
of analysis consisted of focused coding around specific categories (Bernard, Wutich, & Ryan,
2017. The categories were academic partnerships, referral processes, job roles, and client age.
Transcripts were searched for terms related to academic partnerships (academic, research,
national, project, Dr., the names of the previous SafeCare projects, and names of the research
team members and their affiliations), referral processes (refer-), job roles (aid, educator, liaison,
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 81
intermed-, consult-, advocate, nurse, partner), and client age (age, 8, eight, old, zero, 0, year, ten,
10, twelve, 12). All transcripts were reviewed a second time using these search terms.
The third stage of analysis consisted of a document review. The dissertation PI contacted
the National SafeCare Training and Research Center (NSTRC) and obtained the most recent
home visitor, coach and trainer manuals. These documents were reviewed to help the dissertation
PI distinguish between unique adaptations that organizations were making versus flexibility that
is built into the SafeCare model. The document review was organized around several questions.
These questions and examples are provided in Table 3.2.
In addition to the document review of SafeCare manuals, the extensive assessment of
contracting arrangements completed in Chapter 2 was used to contextualize and triangulate the
transcripts. For this study, the contracting arrangements were used to contextualize several areas
relevant to organizational adaptation: how contracts supported SafeCare implementation, how
contracts constrained SafeCare implementation, if organizations were adapting in response to
contracting arrangements, and if changes to the contracting arrangements were a type of
adaptation. The contracting arrangements also served a triangulation function (e.g. if an agency
leader said that they added SafeCare staff, the contracting document could reflect an increase in
funding for the new positions).
In the fourth stage of analysis, the dissertation PI completed within and between case
analyses using a multiple case study approach (Stake, 2006). Case summaries were created for
each organization, which involved assembling and summarizing types of adaptation, the context
for adaptation, and when changes were made so that each organization could be viewed
holistically across the implementation stages (Padgett, 2017). Within case analysis was used to
describe each individual organization’s approach to adaptation. Between case analysis was used
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 82
to identify patterns across the organizations. More specifically, the between case analysis
allowed the dissertation PI to identify similarities and differences across the organizational
adaptation approaches. At the end of this stage, the three organizational adaptation profiles were
formulated and each case was assigned to one of these profile categories.
The fifth stage of analysis consisted of the creation of a visual elicitation tool and
validation of the adaptation profiles through member checking. Visual elicitation tools can
include photographs, relational maps, and timelines (Padgett, 2017; Bagnoli, 2009). In this study,
the dissertation PI created “adaptation snapshots” for each organization (see Figure 3.1). The
“adaptation snapshots” were not meant to identify and count every instance of adaptation.
Instead, they served as a visual tool for illustrating the multidimensionality of organizational
adaptation, eliciting creative interpretations, and making analytic decisions during member
checking (Padgett, 2017).
The dissertation PI led six member-checking meetings from July 2018 to September
2018. Member checking helped to guard against researcher bias and allowed the PI to verify the
adaptation profiles with individuals who participated in the parent studies (Padgett, 2017).
Member checking participants were seven researchers who were highly involved in the parent
studies (three of whom also participated in the peer debriefing meetings) and three practitioners.
One of the practitioners assisted with SafeCare implementation in five of the eight service
systems. The other two practitioners are from organizations that only participated in the most
recent iteration of parent study, so had less data than other organizations in the study sample.
Meetings occurred in-person, via phone call and by video conference. The goal of member
checking in the final analytic phase was to review, adjust, and validate the final profile labels
assigned to each case. During these meetings, specific examples, and the organization’s overall
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 83
approach to adaptation were discussed using the “adaptation snapshot” visual elicitation tool.
Case summaries and profile labels were revised after each meeting.
Results
Table 3.3 describes demographics for the individuals who participated in the secondary
interviews and focus groups used in this dissertation study. The next section describes the three
adaptation profiles that emerged in the data: Incorporators, Early Investors, and Learners.
Overview of Adaptation Profiles
Figure 3.2 provides an overview of the three profiles. In this study, there were seven
Incorporators, six Early Investors and four Learners. Table 3.4 illustrates the distribution of
adaptation profiles across the eight service systems.
Incorporators
Incorporators (n=7) were present in five of the service systems. Incorporators were
organizations that folded SafeCare into existing structures, processes, and capabilities. A key
feature of Incorporators was that they were primarily integrating SafeCare into what they already
as an organization, rather than making substantial changes or creating new ways of operating.
Over time, Incorporators made relatively small-scale adaptations in response to contract and
intervention requirements. Small-scale adaptation included changes such as adding screening
assessments or supplemental material required by the service system to the SafeCare home visit,
adjusting caseloads, adding SafeCare to job descriptions and titles, modifying supervision format
to accommodate SafeCare coaching requirements, and adding the SafeCare program to existing
agency forms and documents. As one Incorporator agency leader commented about SafeCare
implementation, “…you have to do some things but it’s not overwhelming or hard to do.”
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 84
Another Incorporator agency leader said in a question specifically about adaptation efforts at
their organization: “I would say minimal. Again just the scale of it really wouldn”t drive a lot of
heavy lifting changes.”
Five of the Incorporators were in a service system that had a dominant lead agency (three
within the same service system). One Incorporator agency leader explained the dynamics with
the lead agency as, “Over time especially I have really learned how to allow the coaches and to
allow [lead agency name] to be the people who they go to for implementation issues.” The major
responsibilities associated with initiating, building and maintaining SafeCare in the system fell
on these lead agencies, not the Incorporators. The role of lead agencies is further discussed in the
Early Investors section.
Six of the Incorporators had contracting arrangements that covered the full costs of
implementing SafeCare. An Incorporator agency supervisor asserted,
…I cannot think of anything that needs to be changed for us to better do SafeCare…We
have our supplies, we have our transportation, we have our supervision, we have
everything so the agency is really supportive and no, there is no policy that needs to
change…It’s ready to go.
Another Incorporator agency leader said, “I don’t think there was a need to make extraordinary
changes with the resources that were put in place.” Changes related to insufficient funding or
lack of resources were not an issue for Incorporators.
Finally, for two of the Incorporators, SafeCare was specifically described as a small part
of the agency’s service array: “In our menu of services it’s a very small portion…It’s a small
piece of kind of the tapestry of who we are…. Again we have one [SafeCare] position, our
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 85
program has literally one person” (Incorporator agency leader). When asked about the
percentage of total revenue that is SafeCare, the agency leader at the other Incorporator
organization responded, “A small percentage, very small percentage.”
Early Investors
Early Investors (n=6) were present in six of the service systems. Early Investors were
organizations that made substantial adaptations during the initial phases of implementation.
Agency leaders at two different Early Investors described this initial effort as “blood, sweat, and
tears getting this up and running,” and “I think because our agency had a bigger capacity than
some of the smaller like [agency names]…We…set the stage for how that model would look…”
The substantial adaptations that Early Investors made in early implementation were
related to the critical system-wide implementation role that these organizations played. Five of
the six Early Investors acted as lead SafeCare agencies in their service system. The sixth Early
Investor was initially a lead agency then became a subcontractor after contracts were
renegotiated. Lead agencies bore significant responsibilities in initiating and maintaining
SafeCare in their system and may have been formally designated as leads in the contracts. Early
Investors made adaptations to fulfill a variety of lead agency responsibilities. These
responsibilities were: (1) coordinating and conducting SafeCare training, coaching, and fidelity
monitoring across organizations, (2) acting as a liaison with the service system, other
community-based organizations, and the National SafeCare Training and Research Center, (3)
managing system-wide SafeCare data, and (4) housing, supporting, and finding a way to pay for
at least one SafeCare coordinator. Terms used across the transcripts to describe SafeCare
coordinators included: “main contact person,” “a resource,” “SafeCare subject matter expert,”
“a buffer,” “de facto champion,” and “honorary administrator.” All of these lead agency
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 86
responsibilities required substantial adaptation related to developing SafeCare expertise, creating
new processes and procedures, hiring staff or modifying staff roles (especially in the context of
SafeCare training and coaching), and building leadership and data management capacity. These
types of changes required more organizational effort and resources than the types of adaptations
that Incorporators made (e.g. changing forms, adding assessments, and changing the names of
job titles).
Unlike Incorporators, Early Investors changed their adaptation approach over time. While
Early Investors maintained the organizational changes that they made during early
implementation, overall engagement in adaptation tapered off as SafeCare became more
embedded in the organization and service system. When reflecting on adaptation over the course
of SafeCare implementation, one Early Investor agency leader stated, “I’d say we made extensive
changes in the beginning and then minimal changes to sustain it over time.” Once initial
implementation passed, another Early Investor agency leader described SafeCare as “…second
nature to us…It’s just part of what we do.” An Early Investor team leader at a different agency
noted, “Since we’ve been doing this for a few years now, it’s the expected practice…We have a
well-oiled machine.” Another Early Investor supervisor characterized the later implementation
stages as, “Creativity had been performed already.” For Early Investors, SafeCare
implementation felt like “blood, sweat, and tears” during initial implementation and “a well-
oiled machine” during later stages.
Among four of the Early Investors, SafeCare implementation represented a major
cultural shift in the organization. Overcoming ideological misfit was part of the “early
investment” that these organizations had to make to support SafeCare implementation. In these
organizations, frontline staff were resistant to using evidence-based practices broadly and/or the
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 87
SafeCare program specifically. In 2013, an Early Investor agency leader described SafeCare as
“very regimented,” “monotonous,” and “frustrating for staff.” In 2016, an Early Investor
agency leader at this same organization explained overcoming this initial resistance:
I would say that the biggest thing was at the beginning getting used to a new model,
finding some of the things that were looked at a little differently …it was us really saying
okay we’re doing this, let’s find out what’s good about it and what’s different about it
and let’s move forward.
Learners
Learners (n=4) were present in three service systems. What characterized Learners was
steady and continuous attention to new ways that the organization could support SafeCare.
Learners did not necessarily have more instances of organizational level changes. Instead, these
organizations appeared to have an intangible quality related to a culture and/or leader attitudes
that reflected and encouraged curiosity, openness, and a focus on continuous improvement. One
Learner agency leader described adaptation in response to SafeCare implementation as, “I want
to say we’ve made a lot of changes just to maintain or support the program…I think that says
that we are constantly trying to adapt....” Another Learner agency leader noted, “I’m committed
to knowing or finding out if we’re being effective and wanting to evaluate. I think we have to. We
can”t just stumble along in the dark…”
When talking about SafeCare implementation experiences, individuals in Learner
organizations used phrases such as: “continually update,” “took the initiative,” “able to quickly
adapt or make changes as needed,” “constantly trying to assess and reassess,” “keeps us from
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 88
being stale,” “constantly tweaking”, “ongoing conversation,” “is there another way we can
deliver the service that we haven”t thought about”, and “we’re just ever evolving.”
When reflecting on SafeCare implementation, a Learner agency leader explained, “I think
there’s just always something to look at through the eyes of how do we sustain and make it part
of the overall program versus it’s just something that we do.” An agency leader at a different
Learner organization noted, “we’ve fostered or trained ourselves to really value that ongoing
learning” and talked about how “we invite these folks to help us, change the course of what
we’re doing. We invite them to come to the table with their ideas and so it’s a fun place to be.”
For Learners, managing and adapting to SafeCare implementation was viewed as a “great
learning experience.”
Two of the Learners were in stable, well-funded service systems in terms of SafeCare
implementation, but two Learners did not have SafeCare contracts that covered full costs. As one
of these Learner agency leaders explained, “we have to be quite vigilant about keeping up the
caseloads and keeping up the presence and the visibility so that we don’t get forgotten about”
and “It’s going to be struggling to just keep our little program going.” The other Learner agency
leader with an underfunded SafeCare contract discussed multiple financially-related adaptations
that they had made including cross-training staff (beyond the full-time employee covered by the
SafeCare contract) and pulling from discretionary funds to cover the costs of SafeCare
implementation. Similar to Incorporators, Learners maintained their overall approach to
adaptation across the implementation stages.
Discussion
This study tackled the unexplored topic of how organizations adapt over the course of
EBP implementation. The goal of this study was to identify different adaptation profiles among
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 89
organizations that sustained SafeCare, taking into account the unique combination of
organizational attributes that led to these different approaches. Treating these organizations as
implementation actors required a thorough examination of behavior over time. This was done
through initial coding, focused coding, a document review, a multiple case study analysis, and
member checking using a visual elicitation tool. This study used a comprehensive longitudinal
dataset and the sample was 17 community-based organizations in eight diverse child welfare
systems. Three organizational-level profiles of adaptation emerged: Incorporators (n=7), Early
Investors (n=6), and Learners (n=4).
A key feature of the seven Incorporators was that they primarily integrated SafeCare into
what they already did as an organization, rather than making substantial changes or creating new
ways of operating. Five of the Incorporators were in a service system that had a dominant lead
agency. Six of the Incorporators had contracting arrangements that covered the full costs of
implementing SafeCare. For two of the Incorporators, SafeCare was described as a very small
part of the agency’s service array. All of these factors perhaps point to the concept of beneficial
inertia, which occurs when an organization faces no (or minimal) need to change in order to meet
the demands of the external environment (Zajac, Kraatz, & Bresser, 2000). Zajac and colleagues
(2000) further note that, “factors internal to the organization may also play a significant role in
defining when inertia is beneficial, such as the situation in which a firm possesses resources that
offset external pressures for change.” Having a lead agency to depend on, adequate resources,
and SafeCare simply not being a large part of overall services may help to explain why an
Incorporator approach was the most beneficial for these organizations.
Early Investors were organizations that made substantial adaptations during the initial
phases of implementation. Unlike Incorporators, a key feature of the six Early Investors is that
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 90
they changed their adaptation approach over time in order to align with the changing needs of the
external environment. Overall engagement in adaptation tapered off as SafeCare became more
embedded in the organization and service system. Five of the six Early Investors acted as lead
SafeCare agencies in their service system. The sixth Early Investor was initially a lead agency
and then became a subcontractor after contracts were renegotiated. Early Investors played an
important system-wide implementation role, and this finding points to the importance of
examining organizational behavior within the context of interorganizational networks. The
burden of implementing a new evidence-based practice is particularly high for lead agencies that
are responsible for training, coaching, managing data, and coordinating across other agencies and
with system level leaders.
Furthermore, the Incorporators may have been able to choose their adaptation approach
because of the presence of the Early Investors. Organizations occupy different spaces in the
network of entities involved in EBP implementation, are interdependent, and may fill different
functional niches within the system as a whole. This type of information could be used when
planning for system-wide EBP implementation. Knowing one’s implementation role relative to
other organizations within the system can help with a variety of planning tasks including: the
assessment and allocation of resources across the system, identification of service redundancies
and gaps, identification of interorganizational synergy (e.g. coaching and training across
agencies to mitigate turnover and maintain a trained workforce within the system as a whole),
and the development of contracting arrangements that accurately reflect and financially support
the different roles that organizations fill.
What characterized the four Learners was steady and continuous attention to new ways
that the organization could support SafeCare. Learners appeared to have a culture and/or leaders
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 91
who were committed to adapting, improving, and keeping SafeCare implementation fresh in the
organization throughout the implementation lifecycle. The decision to use the “Learner” label
was inspired by the concept of learning organizations. Garvin (1993) defines learning
organizations as, “an organization skilled at creating, acquiring and transferring knowledge, and
at modifying its behavior to reflect new knowledge and insights”. O’Sullivan (1999) further
explains, “…a learning organization must have the capacity to both think and act in new ways
based on new knowledge”. Learning organizations must attend to three types of management
tasks: making sense of the environment, learning from doing, and continuous organizational
design (McDaniel, 1998). Identifying and better understanding Learners may be useful when
organizations are in implementation environments that are particularly dynamic and complex
(and thus require a nimble, proactive approach).
More broadly, the emergence of the three profiles demonstrates equifinality. Equifinality
is the idea that there can be multiple effective paths to the same outcome (Van de Ven & Drazin,
1985; Jennings & Seaman, 1994). In this study, three different organizational-level approaches
all led to the desired outcome: SafeCare sustainment. This observation illustrates that there is no
optimal level of adaptation or ideal adaptation approach that fits every organization or is
necessary for SafeCare sustainment. Another theme in the strategic management literature that
can help to explain the observation of multiple profiles is that organizations calibrated adaptation
levels to fit their existing scale of operations, structures, and the immediate environmental
demands (Scott & Davis, 2007; Jennings & Seaman, 1994). Successfully implementing a new
EBP is one of many tasks that an organization has to attend to and this study normalized the
existence of different (but equally effective) approaches to making a new EBP work and last.
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 92
Identifying organizational adaptation profiles can enhance the uptake and use of
implementation strategies or “methods to enhance the adoption, implementation, sustainment,
and scale up of an innovation” (Proctor, Powell & McMillan, 2013). Recent research has focused
on identifying, classifying, and testing specific strategies. The ERIC project, for example,
yielded 73 implementation strategies (Powell et al., 2015). Knowing an organization’s holistic
approach to adaptation can help researchers identify, explain, and communicate the advantages
of specific implementation strategies to organizational leaders.
For example, Incorporator leaders may be most interested in a small set of strategies that
closely align with their current organizational functioning (e.g. updating record systems, revising
professional roles, or developing new educational materials). Early Investors leaders, on the
other hand, may be particularly interested in strategies that address organizational needs related
to initiating and establishing the EBP during the early implementation stages (e.g. developing a
formal implementation blueprint, identifying and preparing champions, and developing tools for
quality monitoring). In contrast, Learner leaders may want to continually seek out and
experiment with a variety of strategies that more substantially alter the organization’s core
functions (e.g. forming new academic partnerships, accessing new funding streams, or creating
new clinical teams). In fact, identifying the organization’s adaptation approach to a particular
EBP implementation effort (and seeing how this approach fits with the organization’s other
services) may in and of itself be a helpful implementation strategy.
Limitations
The first set of limitations has to do with the study context. This study focuses on one
EBP. Intervention complexity varies widely (Carroll et al. 2007) and different organizational
approaches to adaptation may emerge with other EBPs. This study’s profiles are only a first step
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 93
in categorizing and articulating organizational behavior during EBP implementation. A second
limitation is that the sample is comprised of organizations that participated in formal research
studies. Research participation provides access to expertise and technical assistance that may not
be available in “real world” contexts. However, the participating organizations varied in the
degree to which they had formal and active academic partnerships over the course of
implementation. Additionally, this study examined organizational adaptation in the context of
systems that utilized contractual arrangements to promote and implement SafeCare.
Organizations in systems that do not have contractual obligations to implement a particular
evidence-based practice may engage in other adaptation approaches.
The second set of limitations has to do with the data sources. This study uses secondary
qualitative data to understand adaptation. The quality of responses may be influenced by recall
and social desirability biases and the participant’s tenure within the organization. Additionally,
participants were recalling and reflecting upon past events. This study used a document review
and member checking activities to help triangulate the results. Limitations related to adaptation-
related data collection echo the broader issue that adaptation can be difficult for practitioners to
explicitly recognize and report, and accurately assessing adaptation is a persistent issue in the
implementation science field (Baumann, Cabassa, & Stirman, 2018).
Organizational level adaptation is an inherent part of EBP implementation, and this study
is an essential first step in developing theory that allows for the prediction of organizational
adaptation behavior during EBP implementation. That being said, a third set of limitations relates
to the creation of typologies. The adaptation profiles in this study represent “ideal types” and
there are various nuances that require further exploration. One nuance is the potential for
adaptation profiles to change over time and in response to environmental threats (e.g.
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 94
implementing a new EBP that competes with organizational resources). Another nuance is the
potential blurry boundary between profiles and the fact that they might not always be mutually
exclusive. For example, it is possible that Early Investors may also be Learners, but it is more
difficult to detect this approach within the context of talking about the lead agency’s role.
Another issue related to the few number of Learners. While the importance of cultural and
leadership factors emerged in the qualitative analysis, understanding the specific leadership and
cultural elements that distinguish this profile from the others requires further mixed methods
investigation and more specific data collection around these topics.
Next Steps
In addition to the questions raised above, this study lays the groundwork for multiple
avenues of future research. First, is the exploration of organizational behavior and the creation of
adaptation profiles for other contexts: (1) organizations that do not reach sustainment, (2)
organizations that are de-implementing a non-evidence based practice, and (3) organizations
implementing a practice other than SafeCare. As alluded to above, future research can also
explore which implementation activities and strategies are most effective for Incorporators, Early
Investors, and Learners. Third, quantitative methods can be used to explore characteristics that
define the profiles as well as the relationship between the profiles and specific implementation
(e.g. fidelity) and client outcomes (e.g. engagement and satisfaction). Fourth, future research can
explore adaptation in the context of the collection of EBPs that an organization implements.
Organizations may use one adaptation approach for one EBP and a different approach for a
different practice, or an overarching adaptation philosophy may influence adaptation approaches
across EBPs. Decisions about adaptation approaches may be influenced by the total set of
services that the organization delivers and this topic warrants further exploration. Finally, the
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 95
idea of adaptation approaches could be applied to the system level: How do service systems
adapt to accommodate a new evidence-based practice?
The next chapter demonstrates the usefulness of Qualitative Comparative Analysis
(QCA) for addressing how an organization’s collection of relationships with actors in the
external environment combine and influence behavior around a specific implementation process.
Chapter 4 focuses on three environmental features that emerged in this chapter’s analysis. These
features describe the organization’s relationship with other actors and are: lead agency status (an
organization’s position within the network of other organizations that are providing SafeCare),
funding level in SafeCare contracting arrangements (resources exchanged between organization
and system) and the organization’s ability to influence the SafeCare contract (openness of
relationship between organization and system). Chapter 4 focuses on the process of adaptation
and uses Incorporators and Early Investors in a QCA demonstration.
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 96
References
Aarons, G.A., Moullin, J.C., & Ehrhart, M.G. (2018). The role of organizational processes in
dissemination and implementation research. In R.C. Brownson, Colditz, G.A., & E.K.
Proctor (Eds.), Dissemination and Implementation Research in Health (p.121-142). New
York, NY: Oxford University Press.
Aarons, G.A., Green, A.E., Trott, E., Willging, C.E., Torres, E.M., … Roesch, S.C. (2016). The
roles of system and organizational leadership in system wide evidence-based intervention
sustainment: A mixed method study. Administration and Policy in Mental Health and
Mental Health Services Research, 43(6), 991-1008.
Aarons, G. A., Hurlburt, M., & Horwitz, S. M. (2011). Advancing a conceptual model of
evidence-based practice implementation in public service sectors. Administration and
Policy in Mental Health and Mental Health Services Research, 38, 4–23.
Bagnoli, A. (2009). Beyond the standard interview: The use of graphic elicitation and arts-based
methods. Qualitative Research, 9(5), 547-570.
Baumann, A.A., Cabassa, L.J., & Stirman, S.W. (2018). Adaptation in dissemination and
implementation science. In R.C. Brownson, Colditz, G.A., & E.K. Proctor (Eds.),
Dissemination and Implementation Research in Health (p.285-300). New York, NY:
Oxford University Press.
Baumann, A.A., Powell, B.J., Kohl, P.L., Tabak, R.G., Penalba, V., Proctor, E.K., … Cabassa,
L.J. (2015). Cultural adaptation and implementation of evidence-based parent-training: A
systematic review and critique of guiding evidence. Children and Youth Services Review,
53, 113-120.
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 97
Beasley, L.O.; Silovsky, J.F.; Owora, A.; Burris, L.; Hecht, D.; Demoraes-Huffine, P . . . Tolma,
E. (2014). Mixed-methods feasibility study on the cultural adaptation of a child abuse
prevention model. Child Abuse & Neglect, 38(9),1496-1507.
Bernard, H.R., Wutich, A., & Ryan, G.W. (2017). Analyzing qualitative data: Systematic
approaches (2
nd
ed.). Thousand Oaks, CA: Sage Publications, Inc.
Carroll, C., Patterson, M., Wood, S., Booth, A., Rick., J., & Balain, S. (2007). A conceptual
framework for implementation fidelity. Implementation Science, 2, 40-49.
Chaffin, M., Hecht, D., Bard, D., Silovsky, J. F., & Beasley, W. H. (2012). A statewide trial of
the SafeCare home-based services model with parents in child protective services.
Pediatrics, 129, 509–515.
Charmaz, K. (2010). Constructing grounded theory: A practical guide through qualitative
analysis. Thousand Oaks, CA: Sage Publications, Ltd.
Doty, D. H., & Glick, W. H. 1994. Typologies as a unique form of theory building: Toward
improved understanding and modeling. Academy of Management Review, 19, 230–251.
Finno-Velasquez, M., L. Fettes, D., A. Aarons, G., & S. Hurlburt, M. (2014). Cultural adaptation
of an evidence-based home visitation programme: Latino clients’ experiences of service
delivery during implementation. Journal of Children’s Services, 9(4), 280–294.
Fiss, P.C. (2011). Building better causal theories: A fuzzy set approach to typologies in
organization research. Academy of Management Journal, 54(2), 393-420.
Gaskin, E. H., Lutzker, J.R., Crimmins, D.B., & Robinson, L. (2012). Using a digital frame and
pictorial information to enhance the SafeCare parent-infant interactions module with a
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 98
mother with intellectual disabilities: Results of a pilot study. Journal of Mental Health
Research in Intellectual Disabilities, 5(2), 187-202.
Gardner, R., Hodson, D., Churchill, G., & Cotmore, R. (2014). Transporting and implementing
the SafeCare® home-based programme for parents, designed to reduce and mitigate the
effects of child neglect: An initial progress report. Child Abuse Review, 23, 297–303.
Garvin, D.A. (1993). Building a learning organization. Harvard Business Review, 71(4), 78-91.
Gupta, D.M., Boland, Jr., R.J., & Aron, D.C. (2017). The physician’s experience of changing
clinical practice: A struggle to unlearn. Implementation Science, 12, 28-39.
Hunter, SB., Han, B., Slaughter, M.E., Godley, S.H., and Garner, B.R. (2017). Predicting
evidence-based treatment sustainment: Results from a longitudinal study of Adolescent-
Community Reinforcement Approach. Implementation Science, 12, 75-89.
Jennings, D.F. & Seaman, S.L. (1994). High and low levels of organizational adaptation: An
empirical analysis of strategy, structure, and performance. Strategic Management
Journal, 15, 459-475.
Katz, D. & Kahn, R.L. (1966). The social psychology of organizations. New York, NY: John
Wiley & Sons, Inc.
Kennedy, A., Rogers, A. Chew-Graham, C., Blakeman, T., Bowen, R. … Protheroe, J. (2014).
Implementation of a self-management support approach (WISE) across a health system:
A process evaluation explaining what did and did not work for organisations, clinicians
and patients. Implementation Science, 9, 129-145.
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 99
Lefever, J. E. B., Bigelow, K.M., Carta, J.J., Borkowski, J.G, . . . Warren, S. F. (2017). Long-
term impact of a cell phone–enhanced parenting intervention. Child Maltreatment, 22(4),
305-314.
McDaniel, R. (1998). Strategic leadership: A view from quantum and chaos theories. In W.
Duncan, L. Swayne, & P. Ginter (Eds.), Handbook of healthcare management (p. 339-
367). Maiden, MA: Blackwell.
Miles, R.E., & Snow, C.C. (1978). Organizational strategy, structure and process. New York:
McGraw-Hill.
National SafeCare Training and Research Center. (2016). Provider manual: SafeCare overview.
Atlanta, GA: Georgia State University.
O’Sullivan, M. (1999). Adapting to managed care by becoming a learning organization.
Administration and Policy in Mental Health and Mental Health Services Research, 26(4),
239-252.
Padgett, D.K. (2017). Qualitative methods in social work research. (3
rd
ed.) Los Angeles, CA:
Sage Publications, Inc.
Palinkas, L.A., Aarons, G.A., Horwitz, S., Chamberlain, P., Hurlburt, M., & Landsverk, J.
(2011). Mixed methods designs in implementation research. Administration and Policy in
Mental Health and Mental Health Services Research, 38, 44-53.
Pfeffer, J. & Salancik, G.R. (1978). The external control of organizations. New York, NY:
Harper & Row.
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 100
Powell, B.J., McMillen, C.J., Proctor, E.K…& York, J.L. (2012). A compilation of strategies for
implementing clinical innovations in health and mental health. Medical Care Research and
Review, 69(2), 123-157.
Powell, B.J., Waltz, T.J., Chinman, M.J., Damschroder, L.J., Smith, J.L., Matthieu,
M.M…Kirchner, J.E. (2015). A refined compilation of implementation strategies: Results
from the Expert Recommendations for Implementing Change (ERIC) project.
Implementation Science, 10, 21-35.
Proctor, E., Powell, B., & McMillen, J. (2013). Implementation strategies: Recommendations for
specifying and reporting. Implementation Science, 8, 139-150.
Self-Brown, S., Osborne, M., Lai, C., Veause Brown, B., Glasheen, S., & Adams, N. (2017).
Initial findings from a feasibility trial examining the SafeCare Dad to Kids Program with
marginalized fathers. Journal of Family Violence, 32(8), 751-766.
Self-Brown, S., Osborne, M.C., Rostad, W., & Feil, E. (2017b). A technology-mediated
approach to the implementation of an evidence-based child maltreatment prevention
program. Child Maltreatment, 22(4), 344-353.
Scott, W. R. & Davis, G.F. (2007). Organizations and organizing: Rational, natural, and open
systems perspectives. Upper Saddle River, NJ: Pearson Education, Inc.
Silovsky, J.F., Bard, D., Chaffin, M., Hecht, D., Burrish, L., Owora, A., Beasley, L., Doughty,
D., & Lutzker, J.R. (2011). Prevention of child maltreatment in high-risk rural families:
A randomized clinical trial with child welfare outcomes. Children and Youth Services
Review, 33, 1435-1444.
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 101
Stake, R.E. (2006). Multiple case study analysis. New York, NY: The Guilford Press.
Stirman, S.W., Gamarra, J.M., Bartlett, B.A., Calloway, A., & Gutner, C.A. (2017). Empirical
examinations of modifications and adaptations to evidence-based psychotherapies:
Methodologies, impact, and future directions. Clinical Psychology Science and Practice,
24, 396-420.
Stirman, S.W., Gutner, C.A., Crits-Cristoph, P., Edmunds, J., Evans, A.C., & Beidas, R.S.
(2015). Relationships between clinician-level attributes and fidelity-consistent and
fidelity-inconsistent modifications to an evidence-based psychotherapy. Implementation
Science, 10, 115-125.
Stirman, S.W., Kimberly, J., Cook, N., Calloway, A., Castro, F., & Charns, M. (2012). The
sustainability of new programs and innovations: A review of the empirical literature and
recommendations for future research. Implementation Science, 7, 17-36.
Van de Ven, A. H. and R. Drazin (1985). The concept of fit in contingency theory. In B. M. Staw
and L. L. Cummings (Eds.), Research in organizational behavior (p. 333-375).
Greenwich, CT: JAI Press.
Zajac, E.J., Kraatz, M.S., & Bresser, R.K.F. (2000). Modeling the dynamics of strategic fit: A
normative approach to strategic change. Strategic Management Journal, 21(4), 429-453.
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 102
Appendix
Adaptation Questions Included in 2016 Interview Scripts
1. How do you know if your agency is providing Safe Care as intended by its developers,
the National SafeCare Training and Research Center?
2. Are there times the SafeCare model does not work so well?
3. Thinking back since this organization started delivering SafeCare to the present, what
internal changes were initiated to facilitate its use? Probes:
a. Hiring staff
b. Supervision
c. Billing
d. Distribution of workload
e. Assignment and definition of staff responsibilities
f. Agency policies and procedures
g. Referral processes
h. Staff recognition or rewards
4. Considering some of the changes we just discussed would you say that overall this
agency has made minimal, moderate, or extensive organizational adaptations to support
implementation and sustainment of SafeCare?
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 103
Table 3.1.
Data Types and Years Collected by Service System
System Organization(s)
Transcript
Years
#
Individual
Interviews
# Small
Group
Interviews
#
Focus
Groups
Contract
Doc Years
# Contract
Docs
County A 1, 2, 3, 4 2008-2016 49 2 17 2008-2018 17 (1759 pages)
County B 5, 6, 7 2013-2016 9 1 3 2009-2018 13 (242 pages)
County C 8 2013-2016 4 3 2 2011-2018 18 (274 pages)
County D 9, 10 2013-2016 10 0 1 2011-2018 12 (161 pages)
County E 11, 12 2013-2016 6 2 1 2010-2018 15 (419 pages)
County F 13 2012-2016 8 0 3 2009-2018 12 (403 pages)
County G 14 2013-2016 4 2 1 2012-2018 5 (275 pages)
State 15, 15, 17 2013-2016 29 10 10 2005-2016 21 (606 pages)
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 104
Table 3.2.
Document Review of SafeCare Manuals
Document Review
Questions
Examples
How do the SafeCare
manuals talk about
flexibility?
“SafeCare is structured but flexible in its delivery.” (SafeCare Overview)
“…it is important to have flexibility to allow the Provider to discuss other related topics. This balance of
structure and flexibility will ensure that the Provider knows what to expect out of coaching while also
feeling free to discuss other relevant issues.” (Coach manual)
How do the SafeCare
manuals talk about the
process of adaptation?
“It is critical that any adaptations that are made do not compromise the model structure or lead to a
Provider not maintaining model fidelity.” (Coach manual)
“Experience and support from your Trainer will help you understand the parameters by which you can
facilitate the Provider’s delivery of the program and what adaptations are appropriate and those that
deviate from the core of the model and its research base.” (Coach manual)
What examples of
adaptation are provided
in the SafeCare
manuals?
“Providers may need to adapt a session by changing step order, condensing or expanding the number of
sessions, and/or adding additional information or practice time to match the family’s circumstances.”
(Coach manual)
“SafeCare is still appropriate for families in transitional housing and families experiencing
homelessness…If necessary, a mock room can be set up to help the parent learn the process of identifying
and removing hazards.” (Provider manual)
What are the SafeCare
model standards? (used
to identify deviation
from these standards)
Client age: 5 years old or younger. Parent-Child Interaction Module (PCI) is for parents of children ages
18 months through 5 years old. (Provider manual)
Combining with other services: SafeCare can be conducted by itself or with other services. (Provider
manual)
What kinds of potential
organizational
adaptation are discussed
“…your agency’s implementation may require more sessions or more frequent coaching.” (Coach
manual)
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 105
in the SafeCare
manuals?
“your site’s implementation may require more sessions or more frequent Coach support.” (Trainer
manual)
“…agencies may choose to present the parent with a certificate of completion, either for each module
and/or for completing the overall program. You may modify these certificates for your agency and
families.” (Provider manual)
Table 3.3.
Participant Demographics for Qualitative Data
County A
(n=133) ***
County B
(n=28)
County C
(n=22)
County D
(n=17)
County
E (n=11)
County F
(n=25)
County G
(n=13)
State
(n=92)
Job Role
Home Visitor ** 10 8 9 5 10 5 46
Coach ** 7 3 0 1 8 1 6
Agency leader ** 7 3 2 2 1 2 12
County/state role ** 3 6 4 1 3 4 10
Other* ** 1 2 2 2 3 1 18
Gender
Female ** 24 16 16 8 23 13 74
Male ** 4 5 0 3 2 0 17
Missing ** 0 1 1 1 0 0 1
Notes. * SafeCare coordinator, supervisor, team leader, program manager, grant writer, academic partner, private funder. **
Demographics for qualitative participants not collected in parent studies. *** Total (n=133) does not account for individuals
who participated in multiple waves of data collection. Percentage of individuals who participated in multiple waves in other
systems was County B (10%), County C (4%), County D (10%), County E (15%), County F (7%), County G (7%), State
(19%). These individuals were not double counted.
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 106
Table 3.4.
Distribution of Adaptation Profiles by Service Systems
Service System (n=8) Organizations (n=17)
County A 3 Incorporators, 1 Early Investor
County B 1 Incorporator, 1 Early Investor, 1 Learner
County C 1 Early Investor
County D 1 Incorporator, 1 Early Investor
County E 1 Incorporator, 1 Learner
County F 1 Early Investor
County G 1 Incorporator
State 1 Early Investor, 2 Learners
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 107
Figure 3.1. “Adaptation Snapshot” Visual Elicitation Tool
Figure 3.2. Overview of the Adaptation Profiles
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 108
Chapter Four (Study 3):
Understanding Outer and Inner Context Connections during EBP Implementation:
A Qualitative Comparative Analysis Demonstration
Abstract
The broad goal of this chapter is to demonstrate the usefulness of Qualitative
Comparative Analysis (QCA) for addressing a particular literature gap. The literature gap is the
underrepresentation of research studies that examine how an organization’s collection of
relationships with actors in the external environment combine and influence behavior around a
specific implementation process. QCA is a method that can harness the complexity of outer
context features in implementation research. QCA allows the investigator to collectively assess
combinations of environmental features, and to link these features to organizational behavior
around a specific implementation process. This study focusses on three environmental (or
“service system”) features that describe the organization’s relationship with other actors. The
implementation process of interest is how organizations adapt during EBP implementation.
While this study’s sample does not have the preferred minimum of 10 cases in the outcome
variables (Schlos-Berg & De Meur, 2009), QCA’s analytical steps are demonstrated to show
how this method is useful for complex research questions that connect the outer and inner
contexts.
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 109
Introduction
Literature Gap
Viewing organizations as implementation actors acknowledges the importance of
organizational relationships during evidence-based practice (EBP) implementation.
Organizations exist within a complex social web that can include state and local agencies, other
organizations providing the same EBP, academic partners, intervention developers, and private
foundations. These entities can interact with each other in a variety of ways during
implementation, including contractual and funding arrangements, subcontracting relationships,
research studies, learning collaboratives, and system-wide EBP training. Relationships— and the
resources that flow between entities—create dependencies, constraints, and opportunities that
influence how an organization responds to a new EBP. Paying closer attention to these
relationships can improve our understanding of the ways in which the dynamic interplay between
the organization and external environment influences key implementation processes.
The original Exploration-Preparation-Implementation-Sustainment framework (EPIS)
acknowledges interconnections between the outer and inner contexts and the recently revised
EPIS proposes the concept of “bridging factors” that link the outer and inner contexts (Aarons,
Hurlburt, & Horwitz, 2011; Moullin et al., 2019). Examples of bridging factors include academic
partnerships, purveyors/intermediaries, and interagency collaboration, and as Moullin and
colleagues (2019) note, this is an important area for future research. These system-organizational
relationships and the dynamics, interactions, and exchanges across the outer-inner boundary are
aspects of the EPIS (and other frequently cited implementation frameworks) that typically
receive less attention and use.
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 110
As previously discussed in this dissertation, splitting the outer and inner contexts and
removing the organization from the external environment is a feature of commonly used
implementation conceptual frameworks (e.g. EPIS, CIFR, and PRISM) (Aarons, Hurlburt, &
Horwitz, 2011; Damschroder et al., 2009; Feldstein & Glasgow, 2008). This division informs the
design of many implementation studies. For example, Brookman-Frazee and colleagues (2019)
qualitatively explored factors that influenced implementation of evidence-based autism
interventions in two service systems. The authors describe outer context features (leadership,
service system structure) and inner context features (program and district leadership, provider
and client factors). Then, they describe the different ways that these outer and inner features
support or hinder EBP implementation in the two service systems (Brookman-Frazee et al.,
2019). Scholars separately describe outer and inner contexts in a variety of other implementation
settings including school behavioral health, EBP use in a publicly funded mental health system,
person-centered recovery planning, and tobacco use (Connors et al., 2019; Beidas et al., 2016;
Lodge, Kaufman, & Manser, 2017; Van Devanter, et al., 2017). Again, separate examinations of
the outer and inner contexts treat organizations as if they are in a relational vacuum and fail to
capture the critical ways that the environment shapes organizational behavior during EBP
implementation.
Although less common in the literature, some scholars have begun to tackle relationships
that organizations have with the external environment during EBP implementation. Willging,
Lamphere, and Rylko-Bauer (2015) explored how actions in the outer context created inner
context consequences during a statewide initiative to establish a recovery-oriented behavioral
healthcare system. One finding, for example, linked insufficient attention to training in the outer
context to under-skilled staff who were not able to implement the model uniformly and routinely
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 111
in the inner context (Willging, Lamphere, & Rylko-Bauer, 2015). Aarons et al. (2016)
acknowledge that some implementation factors, such as strong leadership and the use of
management strategies, span the outer and inner contexts and this idea is reflected in the addition
of “bridging factors” to the revised EPIS framework (Moullin et al., 2019). More recently,
Willging and colleagues (2018) explored how outer context factors influenced managers in
community-based organizations during SafeCare implementation. Important outer context
features that affected these managers were legislation and funding priorities, contracting
processes, and funding instability (Willging, et al. 2018). Gunderson et al. (2019) explored how
coaches span the outer and inner contexts, in terms of their activities and responsibilities, as well
as their accountability to and interactions with individuals in both contextual levels.
Collaboration and social networks are other streams of implementation literature that
begin to tap into the importance of relationships that connect organizations to the external
environment. However, collaboration-focused studies often describe different types of
collaborative relationships that are formed during EBP implementation, (e.g. Harris et al., 2012;
Craig & Petzel, 2009; Carter et al., 2014). For example, Carter et al. (2014) described participant
experiences in a quality improvement collaborative for stroke care. Findings focused on how the
collaborative fostered a sense of community among participants and increased attention to stroke
care in their home organizations. These kinds of studies do not speak to ways that relationships
between organizations change as a result of formal collaborations, or how collaboration
influences organizational level processes during EBP implementation.
Additionally, even if different organizational units are included in the sample, social
network-focused implementation studies usually assess interpersonal relationships between
individuals or relationships between interdisciplinary teams (e.g. Cranley et al., 2019; Bunger et
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 112
al., 2018; Bunger et al., 2016; Wensing et al., 2011; Norman & Huerta; 2006). One notable
exception is Palinkas et al. (2011) who examined interorganizational relationships and stage of
implementation among county based child welfare agencies. Treating organizations as
implementation actors requires attention to the dependencies, constraints, and opportunities that
influence how an organization responds to a new EBP. This is not captured in analyses that focus
on the individual or team levels. What is missing in the existing literature is research studies that
examine how an organization’s collection of relationships with actors in the external
environment combine and influence behavior around a specific implementation process in
complex and iterative ways.
Study Goal
The broad goal of Chapter 4 is to demonstrate the usefulness of Qualitative Comparative
Analysis (QCA) for addressing this particular literature gap. QCA is a method that can harness
the complexity of outer context features in implementation research. QCA allows the
investigator to collectively assess combinations of environmental features, and to link these
features to organizational behavior around a specific implementation process. QCA is also
appropriate for smaller sample sizes (Rihoux, 2006), which is often an unavoidable aspect of
organizational and system level implementation studies.
This study focusses on three environmental (or “service system”) features that describe
the organization’s relationship with other actors. The first feature is lead agency status, which
describes an organization’s position within the network of other organizations that are providing
SafeCare. The remaining two features describe aspects of the organization’s relationship with the
child welfare system: funding level in SafeCare contracting arrangements (resources exchanged
between organization and system) and the organization’s ability to influence the SafeCare
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 113
contract (openness of relationship between organization and system). The implementation
process of interest and outcome for this demonstration is how organizations adapt during EBP
implementation. While this study’s sample does not have the preferred minimum of 10 cases in
the outcome variables (Schlos-Berg & De Meur, 2009), QCA’s analytical steps are demonstrated
to show how this method can address this particular literature gap. The next sections provide
more detail about the organizational adaptation profiles and service system features included in
this QCA demonstration.
Study Context
Chapter 3 of this dissertation explored different ways that organizations adapt over the
course of SafeCare implementation. Three organizational-level profiles of adaptation emerged:
Incorporators (n=7), Early Investors (n=6), and Learners (n=4). A key feature of Incorporators is
that they primarily integrate SafeCare into what they already do as an organization, rather than
making substantial changes or creating new ways of operating. Unlike Incorporators, Early
Investors change their adaptation approach over time. Early Investors make substantial
adaptations during early implementation, and then overall engagement in adaptation tapers off as
SafeCare becomes more embedded in the organization and service system. What characterizes
Learners is steady and continuous attention to new ways that the organization can support
SafeCare. Learners appear to have a culture and/or leaders that support adapting, improving, and
keeping SafeCare fresh in the organization throughout the implementation lifecycle. The
qualitative analysis in Chapter 3 suggested that Incorporator and Early Investor adaptation
profiles seem to be related to service system features (e.g. the organization’s situation and
position within the broader system of organizations providing SafeCare). In contrast, Learners
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 114
seemed to be characterized by internal organizational characteristics (e.g. culture and leadership).
This chapter focuses on Incorporators and Early Investors.
Selection of Causal Conditions
As described in Chapter 3, the development of adaptation profiles involved a rigorous set
of qualitative methods including coding of qualitative transcripts, document review, multiple
case study analysis, and member checking with a visual elicitation tool. Throughout this iterative
analytic process, the dissertation PI discussed and revised each case, went back to the data, and
consulted with members of the parent studies research team. The dissertation PI discussed
individual cases as well as relationships between cases within each service system. Using the
deep case knowledge developed in Chapter 3’s study, this demonstration explores three service
system features that emerged as relevant to Incorporators and Early Investors. These conditions
are (1) status as lead agency, (2) funding level in SafeCare contracting arrangements, and (3) the
organization’s ability to influence the SafeCare contract. The research question used in this QCA
demonstration is: Which combinations of service system features explain the Incorporator and
Early Investor adaptation profiles?
Assumptions about Causal Mechanisms
In depth case knowledge developed in the previous dissertation analyses informs several
assumptions about the causal mechanisms behind these three service system features. First, lead
agency status may increase the need for organizational-level adaptation because lead agencies
are expected to make substantial investment in SafeCare and play a leadership role within the
broader service system. Lead agency responsibilities include system-wide training, coaching,
fidelity monitoring and SafeCare data management. Lead agencies often act as a liaison with the
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 115
service system, other community-based organizations, and the National SafeCare Training and
Research Center. Being a lead agency requires substantial organizational investment and
resources to support SafeCare implementation. In contrast, SafeCare contracts that cover full
costs of implementation may decrease the need for organizational adaptation. Organizations that
have adequate resources to implement SafeCare may not need to make changes to compensate
for insufficient funding and resources. Finally, positive beliefs that the organization can
influence the child welfare system around SafeCare contract arrangements may support
organizational-level adaptation. Agency leaders may feel that organizational changes related to
SafeCare implementation will be accepted and financially supported by service system leaders
(e.g. creating, recruiting, and paying for a SafeCare coordinator position in the agency). Table
4.1 summarizes the three service system features and assumptions about causal mechanisms.
Incorporators make relatively small-scale adaptations to fulfill basic intervention or
contracting requirements over the course of SafeCare implementation. For Incorporators, not
being a lead agency, having SafeCare arrangements that cover full costs of implementation, and
not being able to influence SafeCare contracting arrangements may help to explain this approach
to organizational adaptation. In contrast, Early Investors make substantial adaptations at the
beginning of SafeCare implementation. Their engagement in adaptation tapers off as SafeCare
becomes routine and embedded. The conditions that may be most important for explaining Early
Investors are being a lead agency and being able to influence contracting arrangements (to
support the tasks and responsibilities associated with being a lead agency).
In QCA, these arguments are often structured using necessary and sufficient conditions.
A necessary condition is always present when the outcome occurs (Rihoux & Ragin, 2009). For
sufficient conditions, the outcome always occurs when the sufficient condition is present, but the
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 116
outcome can also occur for other reasons (Rihoux & Ragin, 2009). Unlike necessary conditions,
sufficient conditions are not essential and irreplaceable causal ingredients for producing the
outcome. This QCA demonstration is simple and has a small number of causal conditions with a
limited number of potential configurations. Two example hypotheses are provided below (with
the understanding that hypotheses generation will be more meaningful in a more complex
model).
An example hypothesis with a necessary condition is:
H1: Non-lead agency status is a necessary condition for Incorporators.
An example hypothesis with a sufficient condition is:
H2: The ability to influence SafeCare contracting arrangements is a sufficient condition
for Early Investors.
Methods
Qualitative Comparative Analysis (QCA) Description
QCA is a technique that uses Boolean algebra to systematically identify cross-case
patterns (Ragin, 1987). It is also a well-established method for small-to-intermediate N research
designs, e.g. 10-50 cases (Rihoux, 2006). Unlike traditional correlation-based statistical
techniques that assess the isolated effect of each independent variable on an outcome, QCA
examines causal relationships based on set relations (Ragin, 1987; Ragin, 2008).
QCA assesses
which combinations (or “recipes”) of causally relevant conditions lead to a particular outcome
(Ragin, 1987; Ragin, 2008). QCA relies upon truth table analysis and this study uses the fsQCA
software (Ragin & Davey, 2016). One of the hallmarks and strengths of QCA is continual
reflection upon and immersion back into the cases to refine and interpret the solutions produced
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 117
by the fsQCA software (Ragin, 1987; Ragin, 2008). The QCA demonstration in this dissertation
uses crisp sets, which means that outcomes and casual conditions are transformed into sets that
are coded with a 1 or 0. A score of 1 means that the case is in the set and a score of 0 means that
the case is out of the set (Ragin & Fiss, 2017).
Sample
The sample for this QCA demonstration is 17 organizations in eight child welfare
systems. Each organization is a case. Inclusion criteria are that organizations: (1) achieved
SafeCare sustainment status and (2) have at least two time points for qualitative data collection
in the parent studies. SafeCare sustainment status was determined in previous work (Aarons et
al., 2016), and sustainment is defined as “…core elements are maintained or delivered at a
sufficient level of fidelity after initial implementation support has been withdrawn, and adequate
capacity exists to continue maintaining these core elements” (Green et al., 2016; Stirman et al.,
2012).
Measures
Outcome variable. The outcomes examined in this QCA demonstration are set
membership in the Incorporator and Early Investor adaptation profiles. Developing the
adaptation profiles and assigning membership for each case was an iterative process that
included multiple longitudinal data sources and ongoing interactions with the research team.
Chapter 2 provided a detailed description of the development of the adaptation profiles and
assignment of set membership. This QCA demonstration examines two outcomes: set
membership in the Incorporator adaptation profile and set membership in the Early Investor
adaptation profile. In the Incorporator model, organizations were coded as “1” if they were
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 118
assigned the Incorporator adaptation label in Chapter 2 (n=7 cases). In the Early Investor model,
organizations were coded as “1” if they were assigned the Early Investor adaptation label in
Chapter 2 (n=6 cases).
Causal conditions. Three causal conditions are explored in this demonstration. The first
condition is lead agency status. Several data sources were used to code this condition.
Contracting documents were assessed to see if the organization: (1) is officially designated as a
lead in the SafeCare contract, (2) has contract responsibilities and funding dedicated to training,
coaching, or administrative responsibilities that indicates a leadership role in the system.
Additionally, transcripts were manually searched for the term “lead agency” to see how agency
and system leaders talked about each organization’s status within the network of agencies
providing SafeCare. Lead agency status was also discussed with research team members in the
peer debriefing and member checking meetings described in Chapter 2. Organizations may or
may not be formally designated as lead agencies in the SafeCare contracts. In the Incorporator
model, the condition was coded as “notLead” and organizations that do not have lead agency
status were coded with a “1”. In the Early Investor model, the condition was coded as “Lead”
and organizations that have lead agency status were coded with a “1”.
The second condition is whether the SafeCare contracting arrangements cover the full
costs of implementation from the organization’s perspective. The primary data source for this
condition was a question in the 2016 interview script used for agency leaders: “Does current
funding cover the full cost of delivering and administering SafeCare?” Responses to this
question were triangulated with the contracting documents and previous waves of interviews.
Special attention was paid to passages in older transcripts that included discussions of adequacy
of funding in the SafeCare contracts and the aspects of SafeCare implementation that contracts
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 119
did or did not cover from the agency leaders” perspectives. If contracts covered full costs of
SafeCare implementation, then the condition was coded as “1”.
The third condition is positive agency leader perceptions about the organization’s ability
to influence SafeCare contracts. The primary data source for this condition was two questions in
the 2016 interview script used for agency leaders: “How much power has the agency had in
negotiating terms of contracts to deliver SafeCare?” and “How much power does the agency
have in renegotiating once the contract has been awarded?” Responses to these questions were
triangulated with previous waves of interviews. Special attention was paid to coded passages
used in the Chapter 1 analysis that included descriptions of interactions between agency and
system leaders, contract changes the agency would like to see, and contract changes that agencies
were (or were not) successful in making in the SafeCare contracts. If there were positive
perceptions that the organization can influence SafeCare contracting arrangements (from the
agency leaders” perspectives), then the condition is coded as “1”. Table 4.2 summarizes the
conditions, coding, and data sources for each condition.
Analytical Steps
Fiss (2011) summarizes the three analytical steps in a standard QCA model. The first step
is creating a truth table in which each row represents specific combinations of causal conditions.
In a full truth table, all possible combinations of casual conditions are listed (even if the study
sample does not contain instances of that combination). The second analytical step is reducing
rows in the truth table based on (1) the minimum number of cases required for a solution to be
considered and (2) the minimum consistency level of a solution (Fiss, 2011). This is completed
using the “code and delete” function in fsQCA which sets the consistency threshold at 0.8 and
the minimum number of cases to 1 case. Please see the Appendix for Incorporator and Early
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 120
Investor model truth tables (after fsQCA “code and delete” function). In the third step, fsQCA
uses Boolean algebra to reduce the truth table rows into simplified combinations or “recipes”
(Fiss, 2011). FsQCA provides complex, intermediate, and parsimonious solutions as well as
coverage and consistency statistics. As per Fiss’s (2011) recommendation, this demonstration
examined both parsimonious and intermediate solutions. Table 4.3 includes consistency and
coverage definitions provided in the fsQCA manual (Ragin & Davey, 2016). Consistency
describes how consistently organizations with a particular causal recipe(s) are in the Incorporator
(or Early Investor) adaptation profile. Coverage describes how many organizations in the
Incorporator (or Early Investor) adaptation profiles are covered by a particular causal recipe(s).
Results
This section describes the solutions (“causal recipes”), overall coverage, and overall
consistency scores for the Incorporator and Early Investor models. In this simple QCA
demonstration, each model produced one solution.
Incorporator Model
The analysis of necessary conditions in the Incorporator model showed that “Notlead” is
a necessary condition (consistency score of 1.0). The Incorporator solution is Notlead*Fullcost.
Not being a lead agency and having SafeCare contracting arrangements that cover full costs of
implementation are present in the Incorporator solution. The ability to influence SafeCare
contracting arrangements is absent from the Incorporator solution. The overall consistency score
of 1 indicates that all of the cases with the solution Notlead*Fullcost display the Incorporator
outcome. The coverage scores indicate that 86% of the Incorporators display this
Notlead*Fullcost solution. Referring back to the study sample, this means that 6 of the 7
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 121
Incorporators were not lead agencies and had SafeCare arrangements that covered full
implementation costs (Notlead*Fullcost).
An exemplar case for this Incorporator solution is Case 3. Case 3 is in a service system
with a dominant lead agency. The lead agency coordinates training, coaching, data reporting and
interactions with the service system for Case 3. Additionally, Case 3’s contract with the service
system has been stable for multiple years and fully covers SafeCare implementation costs.
Adaptations made by Case 3 that represent an Incorporator approach include adjusting caseloads,
updating an internal agency policy to align with one of the SafeCare modules, and adding an
interview question related to the applicant’s comfort with using manualized interventions.
Early Investor Model
The analysis of necessary conditions in the Early Investor model showed that “Lead” is a
necessary condition (consistency score of 1.0). The Early Investor solution is Lead*Influence.
Being a lead agency and the having the ability to influence SafeCare contracting arrangements
are present in the Early Investor solution. SafeCare contracting arrangements that cover full costs
of implementation is absent from the Early Investor solution. The overall consistency score of 1
indicates that all of the cases with the solution Lead*Influence display the Early Investor
outcome. The coverage scores indicate that 83% of the Early Investors display this
Lead*Influence solution. Referring back to the study sample, this means that 5 of the 6 Early
Investors were lead agencies and were able to influence SafeCare contracting arrangements
(Lead*Influence).
An exemplar case for this Early Investor solution is Case 9. Case 9 is a lead agency that
trains, coaches, and coordinates between agencies that are implementing SafeCare in the system.
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 122
Case 9 also and manages data that are reported back to the service system. Leaders from Case 9
reported that they are able to ‘control’ significant areas of their SafeCare contract (e.g. scope of
work, roles, and responsibilities). Adaptations made by Case 9 that represent an Early Investor
approach include adding new staff, having a staff member act as ‘point person’ for SafeCare and
liaison between the system and organization, teaching other organizations how to market
SafeCare to the broader community, and major internal policy updates (‘a whole set of SafeCare
guidelines’). These adaptations required a lot of organizational effort and were initiated during
the early stages of SafeCare implementation (then maintained over time), thus representing the
Early Investor adaptation approach.
Discussion
Much of the existing implementation research separates the outer and inner contexts (e.g.
Williams, et al., 2019; Lyon et al., 2018; Smith et al., 2019). This dominant approach removes
organizations from the external environment and ignores the critical and specific ways that the
external environment shapes organizational behavior during EBP implementation. For example,
organizations have relationships with a variety of actors in the outer context and these actors can
shape an organization’s ability to successfully implement a new practice. Existing
implementation research touches upon the relational nature of organizations by connecting
discrete pieces of outer and inner contexts (e.g. Willging et al., 2018), describing types of
collaboration (e.g. Carter et al., 2014), and examining interpersonal and team network
connections across organizations (e.g. Bunger et al. 2018). One way to meaningfully build upon
this literature and frameworks like the EPIS is to treat organizations as implementation actors.
Treating organizations in this way requires a careful examination of the ways in which an
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 123
organization’s collection of relationships with actors in the external environment combine and
influence behavior around a specific implementation process.
Taking into account an organization’s social web and the effect that these relationships
have on EBP implementation is not an easy task. The goal of this chapter is to demonstrate how
Qualitative Comparative Analysis (QCA) is a methodological option for addressing this
important literature gap. QCA is an ideal method for these kinds of research questions because it
allows the investigator to collectively assess combinations of environmental features, thus
reflecting the complex and interacting nature of the outer context. Additionally, QCA allows the
investigator to link combinations of environmental features to organizational behavior around a
specific implementation process. This helps researchers identify clear linkages between the outer
context and organizational responses to EBP implementation. Finally, QCA is appropriate for
smaller sample sizes, which is often an unavoidable aspect of organizational and system level
implementation studies.
This chapter illustrates key QCA steps including selection of causal conditions, coding
based on set membership, and interpretation of output. To illustrate how the QCA method could
be applied to this implementation literature gap, this chapter explores combinations of service
system features and organizational behavior around the process of adaptation. In depth case
knowledge was used to select causal conditions that capture several aspects of the organization’s
social web. The first condition describes the organization’s position within the network of other
organizations that are providing SafeCare (status as lead agency). The second condition
describes resources exchanged between the organization and child welfare system (funding level
in SafeCare contracting arrangements). The third condition describes the openness of
relationship between organization and child welfare system (the organization’s ability to
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 124
influence the SafeCare contract). The outcome variables (Incorporator or Early Investor set
membership) describe two different organizational approaches to adaptation during EBP
implementation.
The sample size in the outcome variables in this demonstration is seven cases in the
Incorporator model and six cases in the Early Investor model. This is below the recommended
ten cases per outcome (Rihoux, 2006) and represents a serious limitation of the analyses.
However, the two final solutions are a good illustrative example of the usefulness of QCA for
complex research questions that connect the outer and inner contexts. In this demonstration, the
Incorporator and Early Investor solutions are different. This shows how different combinations
of outer context features can be associated with different organizational responses around a key
implementation process. For Incorporators, not being a lead agency and having SafeCare
contracts that cover full implementation costs is important (Notlead*Fullcost). The Incorporator
adaptation profile is characterized by an integration of SafeCare into core functions, rather than
making substantial changes or creating new ways of operating. Having a lead agency that bears
much of the implementation burden for the whole service system and having adequate funding
and resources may be well aligned with an adaptation approach that requires minimal
engagement and changes.
In contrast, for Early Investors, being a lead agency and being able to influence SafeCare
contracting arrangements is important (Lead*Influence). The Early Investor adaptation profile is
characterized by substantial adaptations during early implementation, and then a reduction in
adaptation as SafeCare became more embedded in the organization and service system. Being a
lead agency requires extensive investment and effort to get the EBP going. Lead agencies are
also a hub for communication across entities, including service system leaders that control and
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 125
dictate contracting arrangements. Being a lead agency and having the ability to influence
contracting arrangements to support the tasks and responsibilities of this role may be well
aligned with an Early Investor adaptation approach. Both model solutions highlight how QCA
can add value to the qualitative approaches that are more commonly used in implementation
research (e.g. stakeholder interviews and focus groups) and articulate and discriminate among
specific outer and inner context relationships.
Challenges
As stated earlier, the low sample size (under 10 cases) for the outcome variables limits
the usability of this study’s findings. However, this QCA demonstration brings to light some of
the broader challenges of using QCA to connect outer and inner contexts in implementation
research. The first challenge is case selection. De Meur, Rihoux and Yamasaki (2009) describe
case-sensitivity as “both an asset and a problem in QCA.” This chapter’s demonstration drew the
line at organizations that achieved SafeCare sustainment and had two waves of data collection in
the parent studies. Determining which cases are relevant to a study that tackles both outer and
inner contexts could be difficult for a variety of reasons including the fact that organizations start
implementation at different times and pass through implementation stages at various paces.
Another challenge related to case selection is that implementation success and determining an
organization’s place in the web of relevant implementation actors would most certainly require
longitudinal data collection and the inclusion of diverse system and organizational level
participants. Furthermore, holistically categorizing organizational behavior around key processes
(using profiles or typologies, for example) is still a novel approach to organizationally focused
implementation research. The lack of organizational typologies may also present a challenge for
identifying and selecting appropriate cases in this research space.
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 126
A second challenge is selecting conditions (De Meur, Rihoux & Yamasaki, 2009). What
is specific to QCA is the necessity of in depth case knowledge. As mentioned in the Introduction,
the outer context is comprised of complex macro level system features such as legislative and
policy priorities, funding arrangements, contracts, relationships with program developers, and
inter-organizational networks. The outer context has received less attention in the literature
(compared to the inner context), and because of this, implementation researchers do not have
many tools for systematically measuring, assessing, and prioritizing different outer context
features. The demonstration in this chapter benefited from in depth analysis across multiple
waves of data collection and two large-scale parent studies.
Third, QCA does not identify actual casual mechanisms. As De Meur and colleagues
(2009) note, this is the job of the investigator who must call upon her deep understanding of the
cases. QCA can begin to address the intersectional nature of relevant outer context features and
link combinations of conditions to a particular organizational process or outcome (e.g. approach
to adaptation). However, determining time order will require a deep understanding of dynamics,
interactions and relationships within a service system. This type of information will require
longitudinal data from diverse stakeholders, and the ability to see implementation processes and
dynamics unfold between actors across multiple years. This chapter’s QCA demonstration drew
upon up to 10 years of implementation experience for the participating organizations.
Implications for Future Research
QCA can be a helpful methodological tool for creating testable multilevel theory that
captures both the outer and inner implementation contexts. There are countless potential
applications of QCA that could enhance our understanding of implementation processes and
relationships. While this QCA demonstration focuses on adaptation, future studies could use this
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 127
method to understand other ways that organizational behavior is influenced by the service system
during EBP implementation (e.g. outer context factors that influence an organization’s selection
of a particular EBP or set of EBPs). QCA can be used to examine other collections of both outer
and inner context features around key organizational processes (like adaptation). In future work,
for example, inner context factors (e.g. leadership and culture) could be combined with outer
context features (e.g. network position, funding levels) so that more complex causal recipes can
be compared across Incorporators, Early Investors, Learners, and other adaptation profiles that
emerge. These types of analyses can help us build theoretical frameworks that capture the
complex sets of relationships, contributing factors, and interactions that shape important
organizational behavior during implementation.
Finally, QCA can be used to help us better understand implementation failure, including
the relationship between outer context features and (1) why failure occurs at different points in
the implementation process, and (2) why some organizations in the same network fail and others
are successful. We may find, for example, that when conditions seen as necessary (for example,
resource adequacy for Incorporators) are absent, the desired outcome (sustainment) may be less
likely. QCA is a structured analytic process that can be used to organize and make sense of
diverse, complex, and interactive systemic conditions and consequences. It is a method that
meets the pressing needs of multilevel implementation science research.
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 128
References
Aarons, G.A., Green, A.E., Trott, E., Willging, C.E., Torres, E.M., … Roesch, S.C. (2016). The
roles of system and organizational leadership in system wide evidence-based intervention
sustainment: A mixed method study. Administration and Policy in Mental Health and
Mental Health Services Research, 43(6), 991-1008.
Beidas, R.S., Stewart, R.E., Adams, D.R., Fernandez, T., Lustbader, S., Powell, B.J…Barg, F.K.
(2016). A multi-level examination of stakeholder perspectives of implementation of
evidence-based practices in a large urban publicly-funded mental health system.
Administration and Policy in Mental Health and Mental Health Services Research, 43,
893-908.
Brookman-Frazee, L., Chlebowski, C., Suhrheinrich, J., Finn, N., Dickson, K.S., Aarons, G.A. &
Stahmer, A. (2019). Characterizing shared and unique implementation influences in two
community services systems for autism: Applying the EPIS framework to two large-scale
autism intervention community effectiveness trials. Administration and Policy in Mental
Health and Mental Health Services Research [Epub ahead of print].
Bunger, A.C., Doogan, N., Hanson, R.F., Birken, S.A. (2018). Advice-seeking during
implementation: A network study of clinicians participating in a learning collaborative.
Implementation Science, 13, 101-113.
Bunger, A., Hanson, R., Doogan, N., Powell, B., Cao, Y., & Dunn, J. (2016). Can learning
collaboratives support implementation by rewiring professional networks?
Administration and Policy in Mental Health and Mental Health Services Research, 43(1),
79–92. https://doi.org/10.1007/s10488-014-0621-x.
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 129
Carter, P., Ozieranski, P., McNicol, S., Power, M., & Dixon-Woods, M. (2014). How
collaborative are quality improvement collaboratives: A qualitative study in stroke care.
Implementation Science, 9:32, https://doi.org/10.1186/1748-5908-9-32.
Connors, E.H., Schiffman, J., Stein, K., LeDoux, S., Landsverk, J., & Hoover, S. (2019). Factors
associated with community-partnered school behavioral health clinicians” adoption and
implementation of evidence-based practices. Administration and Policy in Mental Health
and Mental Health Services Research, 46, 91-104.
Craig, T.J. & Petzel, R. Management perspectives on research contributions to practice through
collaboration in the U.S. Veterans Health Administration: QUERI Series. Implementation
Science, 4:8, https://doi.org/10.1186/1748-5908-4-8.
Cranley, L.A., Keefe, J.M., Taylor, D., Thompson, G., Beacom, A.M. Squires, J.E…Berta, W.B.
(2019). Understanding professional advice networks in long-term care: an outside-inside
view of best practice pathways for diffusion. Implementation Science, 14:10,
https://doi.org/10.1186/s13012-019-0858-6.
Fiss, P.C. (2011). Building better causal theories: A fuzzy set approach to typologies in
organization research. Academy of Management Journal, 54(2), 393-420.
Gunderson, L., Willging, C., Trott Jaramillo, E., Green, A., Fettes, D., Hecht, D., & Aarons, G.
(2018). The good coach: implementation and sustainment factors that affect coaching as
evidence-based intervention fidelity support. Journal of Children’s Services, 13(1), 1–17.
https://doi.org/10.1108/JCS-09-2017-0043.
Harris, J.K., Provan, K., Johnson, K.J., & Leischow, S.J. (2012). Drawbacks and benefits
associated with inter-organizational collaboration along the discovery-development-
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 130
delivery continuum: A cancer research network case study. Implementation Science,
7:69, https://doi.org/10.1186/1748-5908-7-69.
Lodge, A.C., Kaufman, L., Manser, S.S. (2017). Barriers to implementing person-centered
recovery planning in public mental health organizations in Texas: Results from nine
focus groups. Administration and Policy in Mental Health and Mental Health Services
Research, 44, 413-429.
Lyon, A.R., Cook, C.R., Brown, E.C., Locke, J., Davis, C., Ehrhart, M., Aarons, G.A. (2018).
Assessing organizational implementation context in the education sector: Confirmatory
factor analysis of measures of implementation leadership, climate, and citizenship.
Implementation Science, 13:5, https://doi.org/10.1186/s13012-017-0705-6.
Moullin, J.C., Dickson, K.S., Stadnick, N., Rabin, B., Aarons, G.A. (2019). Systematic review of
the exploration, preparation, implementation, sustainment (EPIS) framework.
Implementation Science, 14, 1-17.
Norman, C.D. & Huerta, T. (2006). Knowledge transfer & exchange through social networks:
Building foundations for a community of practice within tobacco control. Implementation
Science, 1, 20-31.
Palinkas, L.A. Holloway, I.W., Rice, E., Fuentes, D., Wu, Q., & Chamberlain, P. (2011). Social
networks and implementation of evidence-based practices in public youth-serving systems: A
mixed-methods study. Implementation Science, 6, 113-124.
Ragin, C.C. & Fiss, P.C. (2017). Intersectional inequality. Chicago, IL: The University of
Chicago Press, Ltd.
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 131
Ragin, C. C. (1987). The comparative method: Moving beyond qualitative and quantitative
strategies. Berkeley, CA: University of California Press.
Ragin, C.C. (2008). Redesigning social inquiry: Fuzzy sets and beyond. Chicago, IL: The
University of Chicago Press.
Ragin, C. C. & Davey, S. (2016). Fuzzy-set/qualitative comparative analysis 3.0. Irvine,
California: Department of Sociology, University of California.
Rihoux, B. & Ragin, C.C. (2009). Configurational comparative methods: Qualitative
comparative analysis (QCA) and related techniques. Thousand Oaks, CA: Sage
Publications, Inc.
Rihoux, B. (2006). Qualitative comparative analysis (QCA) and related systematic comparative
methods: Recent advances and remaining challenges for social science research.
International Sociology, 21(5), 679-706.
Schlos-Berg, D. & De Meur, G. (2009). Comparative research design: Case and variable
selection. In B. Rihoux & C. Ragin (Eds.), Configurational Comparative Methods:
Qualitative Comparative Analysis (QCA) and Related Techniques (p.19-32). Thousand
Oaks, CA: Sage Publications, Inc.
Smith, S.N., Almirall, D., Prenovost, K., Goodrich, D.E., Abraham, K.M., Liebrecht, C., &
Kilbourne, A.M. (2018). Organizational culture and climate as moderators of enhanced
outreach for persons with serious mental illness: Results from a cluster-randomized trial
of adaptive implementation strategies. Implementation Science, 13, 93-108.
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 132
Van Devanter, N., Kumar, P., Nguyen, N., Nguyen, L., Nguyen, T., Stillman, F…Shelley, D.
(2017). Application of the Consolidated Framework for Implementation Research to
assess factors that may influence implementation of tobacco use treatment guidelines in
the Viet Nam public health care delivery system. Implementation Science, 12, 27-35.
Wensing, M., van der Eijk, M., Koetsenruijter, J., Bloem, B.R., Munneke, M., & Faber, M.
(2011). Connectedness of healthcare professionals involved in the treatment of patients
with Parkinson”s disease: A social networks study. Implementation Science, 6, 67-75.
Willging, C.E., Lamphere, L., & Rylko-Bauer (2015). The transformation of behavioral
healthcare in New Mexico. Administration and Policy in Mental Health and Mental
Health Services Research, 42, 343-355.
Williams, N.J., Frank, H.E., Frederick, L., Beidas, R.S., Mandell, D.S., Aarons, G.A…Locke, J.
(2019). Organizational culture and climate profiles: Relationships with fidelity to three
evidence-based practices for autism in elementary schools. Implementation Science, 14,
15-29.
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 133
Appendix
Truth Table for Incorporator Model
Notlead Fullcost Influence # of cases Incorporator Raw
consistency
1 1 1 5 1 1
1 1 0 1 1 1
1 0 0 2 0 .5
0 1 0 3 0 0
0 1 1 3 0 0
0 0 1 2 0 0
1 0 1 1 0 0
Truth Table for Early Investor Model
Lead Fullcost Influence # of cases Early
Investor
Raw
consistency
1 1 1 3 1 1
1 0 1 2 1 1
1 1 0 3 0 .333
0 1 1 5 0 0
0 0 0 2 0 0
0 1 0 1 0 0
0 0 1 1 0 0
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 134
Table 4.1.
Assumptions about Causal Mechanisms
Service system feature Assumptions about causal mechanism
Lead agency status
Increase need for organizational-level adaptation: Because of their unique role in the
service system, lead agencies have to make substantial investment in SafeCare,
especially during early implementation stages.
SafeCare contracting arrangements
that cover full costs of implementation
Decrease need for organizational-level adaptation: Well-resourced SafeCare contracts
may reduce the need for organizational-level adaptation because organizations have the
tools and support that they need to implement.
Organizations believe that they can
influence SafeCare contracting
arrangements
Support organizational-level adaptation: Organizations” belief that they can influence
the system around SafeCare contracts arrangements may encourage organizational-level
adaptation. Agency leaders may believe that if they make organizational changes, they
will be compensated and supported through updated contracting arrangements.
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 135
Table 4.2.
QCA Demonstration Causal Conditions
Variable
Name
Description of Condition Coding Data Sources
Notlead/
Lead*
The organization’s status
within the broader network
of organizations
implementing SafeCare
(lead or not lead agency)
"1" if organization is not a
lead agency (Incorporator
model)
Contracting documents, “lead agency” language used in
any of the transcripts
"1" if organization is a
lead agency (Early
Investor model)
Triangulation with: Discussions with research team about
the organization’s status in the service system
Fullcost
Whether or not SafeCare
contracting arrangements
cover full costs of
implementation from
organization’s perspective
(yes or no)
"1" if SafeCare contracts
cover full costs
2016 interview question: Does current funding cover the
full cost of delivering and administering SafeCare?
Triangulation with: Contracting documents and previous
waves of interviews
Influence
Organizations believe that
they can influence SafeCare
contracting arrangements
(yes or no)
"1" if organizations
believe that they can
influence SafeCare
contracts
2016 interview questions: How much power has the
agency had in negotiating terms of contracts to deliver
SafeCare? How much power does the agency have in
renegotiating once the contract has been awarded?
Triangulation with: Previous waves of interviews
Note. *Notlead used in Incorporator model. Lead used in Early Investor model.
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 136
Table 4.3.
Consistency and Coverage Definitions
Consistency
the degree to which membership in each solution term is a subset of the
outcome
Solution
consistency
the degree to which membership in the solution (the set of solution terms) is
a subset of membership in the outcome
Raw coverage
the proportion of memberships in the outcome explained by each term of
the solution
Unique
coverage
the proportion of memberships in the outcome explained solely by each
individual solution term (memberships that are not covered by other
solution terms)
Solution
coverage
the proportion of memberships in the outcome that is explained by the
complete solution
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 137
Chapter Five:
Organizations as Implementation Actors:
A Research Agenda
For if managers are not engaged in an active reading that embraces different points of
view, much of the richness and complexity of organizational life is passing them by.
They are simply not seeing what is really going on. Also, they are surrendering their
ability to shape what is going on (Morgan, 1997, p.350).
Limit your thinking and you will limit your range of action (Morgan, 1997, p.351).
Why We Need to Rethink Organizational Implementation Research
This dissertation proposes the idea that implementation science is currently limited by a
single metaphor for organizations: organizations are context. Implementation researchers
typically take one of two approaches to studying this organizational context. The first approach
is to report organizational-level barriers and facilitators in different clinical or service settings
(e.g. Jabbour et al., 2018; Geerligs et al., 2018; Templeton et al., 2016). While this first approach
is more descriptive in nature, the second approach aims to identify relationships between isolated
organizational constructs and specific clinical or implementation outcomes (e.g. Andrew et al.,
2019; Smith et al., 2019; Williams et al., 2018). Organizational constructs like climate, culture,
leadership, readiness for change, and provider attitudes towards evidence-based practice receive
great attention in both of the approaches described above (e.g. Williams et al., 2019; Aarons,
Ehrhart, & Farahnak, 2014; Shea et al., 2014; Rye et al., 2017).
These approaches to organizational research are a direct product of the ways in which we
use conceptual frameworks and measures to separate the organization (inner context) from the
external environment (outer context) (e.g. Moullin et al., 2019; Fernandez et al., 2018; Ramsey et
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 138
al., 2016; Damschroder et al., 2009; Feldstein & Glasgow, 2008). The field’s dominant
frameworks and measures treat organizations as context that can be decontextualized, examined,
and manipulated in discrete layers. When we understand organizations only as context, we strip
away the critical dynamics and interactions that occur within an organization’s relational web.
We set aside the relationships between outer and inner contextual factors and implicitly treat
organizations as closed systems. This prevents us from identifying ways in which the
environment shapes internal organizational life, and ways that organizations influence the
external environment during EBP implementation.
Even more importantly, viewing organizations only as context places a heavy burden on
practitioners who use our research. Right now, practitioners must understand and synthesize a
multitude of potentially relevant contextual pieces in order to create a holistic organizational
picture. What’s more, this picture still does not often include the critical dynamics and
interdependence between their organization and their service environment. As Morgan (1997)
asserts: “Limit your thinking and you will limit your range of action.” Our reliance upon a single
metaphor for organizations in implementation research is not just a literature gap. It seriously
limits the usability of our research and hinders our ability to meaningfully support managers and
leaders who must implement evidence-based practices (EBPs) in complex real-world
environments. It encourages a static, rather than dynamic, approach to implementation.
Organizations Are Implementation Actors
This dissertation echoes Birken et al.’s (2017) call to use and integrate existing
organizational theories in implementation research, and proposes a new metaphor for the
implementation science field. This metaphor is organizations as implementation actors. What
does it mean for an organization to be an actor and not just the context for EBP implementation?
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 139
First, this metaphor highlights and acknowledges the relational nature of organizations. Second,
thinking about organizations as implementation actors reminds us that organizations are a
combination of interacting elements, agents, and processes. This interaction among factors may
be lost when we continually parcel out and focus on isolated organizational characteristics (e.g.
types of climate). Third, this metaphor requires identifying and classifying organizational
behavior in a way that allows for the examination of different behavioral pathways that lead to a
particular outcome. This study illustrates these three elements of the organizations as
implementation actors metaphor through an exploration of how organizations that sustained
SafeCare engaged in the process of adaptation. Taken together, this new metaphor integrates
concepts of equifinality (there is more than one viable path to a desirable outcome), resource
dependence (necessary resource exchanges among diverse organizations across the outer-inner
boundary) and open systems thinking (permeable boundaries between an organization and its
environment that allow resources and information to flow in both directions) for implementation
research.
Key Contributions of this Dissertation
Each of this dissertation’s three studies uses and models the organizations as
implementation actors metaphor. Main contributions from each study are described below.
Study 1. Study 1 is a conceptual paper that argues for a more open systems approach to
implementation research using organization-service system interactions around SafeCare
contracting arrangements as an illustrative example. Contracting arrangements were selected for
this study because they are a dynamic medium through which organizations and child welfare
systems interact with each other and exchange resources. Resource dependence theory is used to
understand these interactions and exchanges. One main contribution of Study 1 is the bi-
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 140
directional application of resource dependence theory. This theoretical application highlights the
fact that organizations are not passively controlled and constrained by the environment. Instead,
they are actors that can shape the broader service system during EBP implementation. These
findings also further elaborate and advance the concept of ‘bridging factors’ in the EPIS
framework (Moullin et al., 2019).
A second key contribution of Study 1 is the articulation of specific resources that are
exchanged across the outer and inner contexts. This is important because it highlights the
interdependence between organizations and service systems in a way that is specific to EBP
implementation. Service systems may control funding and structure service delivery; but
organizations have resources that the system also needs including EBP expertise, institutional
knowledge, and training and coaching capacity. A key takeaway is the observation that
organizations and systems need each other to achieve implementation success. This
interdependence evolves and persists as long as the EBP is being used, and our conceptual
frameworks and methodological approaches can and should do more to reflect this.
Study 2. Study 2 demonstrates an innovative way to examine organizational level
behavior around a specific implementation process (adaptation). Rather than listing and
categorizing specific types of changes that organizations make during implementation, this study
considers adaptation from a typological perspective. Typologies are a useful way of thinking
about organizational adaptation during EBP implementation because they “identify multiple
ideal types, each of which represents a unique combination of the organizational attributes that
are believed to determine the relevant outcome” (Doty & Glick, 1994). This aligns with the
organizations as implementation actors metaphor because it acknowledges that organizations are
a collection of attributes. A typology perspective also acknowledges that there is generally more
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 141
than one bundle of choices and actions that lead to the desired collection of attributes. A
significant contribution of Study 2 is the use of rigorous qualitative methods and longitudinal
data to develop three profiles of organizational adaptation: Incorporators, Early Investors, and
Learners. To the dissertation PI’s knowledge, no implementation study has examined adaptation
in this way or described organizational level approaches.
Another major contribution of Study 2 is the identification of different but equally
effective ways that organizations that sustained SafeCare approached the adaptation process.
Focusing only on sustaining sites is important for two reasons. One, the focus is on paths to
success rather than contrasting success and failure. Two, because it does not predetermine that a
specific set of actions or relationships is required for sustainment, it allows for the unique
examination of equifinality, or different adaptation approaches that led to implementation
success. Incorporators primarily integrated SafeCare into what they already did as an
organization, rather than making substantial changes or creating new ways of operating. Early
Investors made substantial adaptations during the initial phases of implementation and
engagement in adaptation tapered off as SafeCare became more embedded in the organization
and service system. Learners appeared to have a culture and/or leaders who were committed to
adapting, improving, and keeping SafeCare implementation fresh in the organization throughout
the implementation lifecycle. All of these approaches led to the desired goal of EBP sustainment
and represent distinct ways that organizations calibrated adaptation approaches to fit their
existing scale of operations, structures, and the immediate environmental demands (Scott &
Davis, 2007; Jennings & Seaman, 1994).
Study 3. Study 3 begins to address the methodological demands of the organizations as
implementation actors metaphor by offering Qualitative Comparative Analysis (QCA) as a
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 142
useful tool. The first overarching contribution of Study 3 is the articulation of a literature gap
that is particularly well suited to QCA. What is missing in the existing literature is research
studies that examine how an organization’s collection of relationships with actors in the external
environment combine and influence behavior around a specific implementation process in
complex and iterative ways. Collaboration research often does not speak to ways that
relationships between organizations change as a result of formal collaborations, or how
collaboration influences organizational level processes during EBP implementation (Harris et al.,
2012; Craig & Petzel, 2009; Carter et al., 2014). Additionally, even if different organizational
units are included in the sample, social network-focused implementation studies usually assess
interpersonal relationships between individuals or relationships between interdisciplinary teams
(e.g. Cranley et al., 2019; Bunger et al., 2018; Bunger et al., 2016). Treating organizations as
implementation actors requires attention to the dependencies, constraints, and opportunities that
influence how an organization responds to a new EBP, and this is not captured in analyses that
focus on the individual or team levels.
The second contribution of Study 3 is a demonstration of QCA as a method for
addressing this specific literature gap. This chapter illustrates key QCA steps including selection
of causal conditions, coding based on set membership, and interpretation of output. The QCA
demonstration illustrates several useful aspects of QCA for this topic. First, QCA can harness the
complexity of outer context features in implementation research. Second, QCA allows the
investigator to collectively assess combinations of environmental features, and to link these
features to organizational behavior around a specific implementation process (e.g. adaptation).
Third, QCA can be used with smaller sample sizes (Rihoux, 2006). Fourth, the model solutions
in this demonstration highlight how QCA can add value to the qualitative approaches that are
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 143
more commonly used in implementation research (e.g. stakeholder interviews and focus groups)
because it articulates and discriminates among specific outer and inner context relationships. In
this demonstration, for example, the causal recipe that explains Incorporators is different than the
causal recipe that explains Early Investors.
Looking Across the Three Studies
Implementation research largely considers organizations to be separate and equal actors
within a service system or market space. EBP implementation is frequently presented as distinct
organizations making choices about individual EBP implementation, adaptation, and sustainment
processes. This dissertation provokes larger conversations about how implementation and
sustainment occurs within the context of a structured and interconnected service system. Service
system interconnections can take a variety of forms. One way that relationships between service
system and organizational actors can be structured is through contracting arrangements.
This dissertation encourages the reader to think about implementation and sustainment in
an environment in which organizations have contractual obligations, role responsibilities relative
to one another, and dynamic interactions with respect to the payor for services (in this study, the
child welfare entity). One broad question that this dissertation’s adaptation profiles raise is how
do individual organizations approach key implementation processes (e.g. adoption,
implementation, adaptation, and sustainment) when organizations have explicitly different and
complementary roles within a broader system? The Early Investors profile, for example, seems
to be tied to a unique role that an organization plays within the broader systemic structure (in this
case, SafeCare lead agency).
This dissertation also raises questions about ways in which EBP implementation and
sustainment are tightly linked to strategic decision making within this larger systemic context.
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 144
For example, the Incorporators profile highlights that organizations may choose to adapt in ways
that align with their existing core functions. The Incorporator approach may also be a strategic
reaction to the presence of Early Investors in the service system. The relationships between these
adaptation profiles again, reiterate the broader idea that each organization occupies an
interconnected position within a given network of entities that are implementing a new EBP.
The Learner approach appears to be more “trait-like” in its quality. This profile leaves
many questions for future research including whether these traits are specific to a particular
evidence-based practice, or whether it is something that is likely to be infused within the
organization’s culture and therefore to pervade the implementation of other practices. The idea
that organizational traits could permeate the implementation of other practices brings to light the
importance of examining an organization’s total service array. Organizations are often
implementing a collection of EBPs with staggered start dates, and different contractual needs,
organizational demands, and client populations. One issue that arose in the development of these
adaptation profiles was the potential for blurry boundaries between profiles. For example, it is
possible that Early Investors may also be Learners, but it is more difficult to detect this approach
within the context of talking about the lead agency’s role. It is also possible that given an
organization’s position in the service system and its total package of EBPs, the organization
could be an Incorporator in one EBP implementation effort and an Early Investor in another. The
Learner profile brings to light the need to distinguish between different organizational
approaches to implementation processes vs. organizational traits that cut across the concurrent
implementation of multiple practices. The next sections discuss implications for building a
research agenda around this idea of organizations as implementation actors.
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 145
Organizations are Implementation Actors: A Research Agenda
Methodological implications. As the previous section alludes to, one overall implication
of this dissertation is the idea that EBPs are implemented within a system that contains
interconnected and interdependent actors. If this is the case, then we need to rethink our unit of
analysis for implementation studies. Rather than looking at behavior from the perspective of an
individual organization, we need to have more studies that examine EBP implementation at the
system level. Examining relationships within a system will allow us to identify bi-directional
dependencies, complementary roles, system level strategic decision-making, and the critical
ways that organizations are behaving in response to these system level interconnections and to
other actors within the larger system. Another methodological implication of treating
organizations as implementation actors is the need for longitudinal data collection and analyses.
This dissertation would not have been possible without multiple years of data and the rich
institutional knowledge of the parent study research team. Observing, interpreting, and
characterizing how an organization engages in an implementation process (like adaptation)
cannot be done in a cross-sectional design. Treating organizations as implementation actors
requires the tracking of behavior over time. A third methodological implication is the need for
methods that can handle the complex, interactional, and configurational nature of
implementation environments. Study 3 demonstrates how QCA will be an important
methodological tool for this organizations as implementation actors research agenda.
Theoretical implications. A reoccurring theme throughout this dissertation is the lack of
diverse theoretical lenses for thinking about organizational behavior in implementation research.
The original Exploration-Preparation-Implementation-Sustainment framework (EPIS) visually
depicts “interconnections” between the outer and inner contexts and the revised EPIS lists
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 146
community academic partnerships and EBP purveyors/intermediaries as “bridging factors”
between the outer and inner contexts (Aarons, Hurlburt, & Horwitz, 2011; Moullin et al., 2019).
Elaborating upon these system-organizational relationships and assessing how dynamics,
interactions, and exchanges across the outer-inner boundary actually affect implementation
processes is an urgent area of theoretical development.
One way to develop theory that describes and reflects outer-inner contextual dynamics is
to use and adapt existing organizational theory in the EBP implementation context (as also
mentioned by Birken and colleagues, 2017). These theories should not be used to superficially
explain findings but should instead inspire new ways of designing organizationally focused
implementation studies and asking new questions that treat organizations as actors and not
context. This dissertation uses resource dependence theory to understand how organizations and
systems influence and are dependent on each other during SafeCare implementation. Other
organizational theories that may be particularly useful for implementation science research
designs are institutional, organization learning, and sensemaking theories.
In the context of organizational adaptation, for example, institutional theory could help us
assess when procedural and practice adaptation occurs because the organization feels the needs
to increase legitimacy and compatibility with the service system—not because these procedural
and practices changes necessarily make the organization more efficient (Meyer & Rowan, 1977;
DiMaggio & Powell, 1983). Organizational learning theory can help us better understand why
organizations act or fail to act on new insights about the systemic relationships or the EBP
(Garvin, 2000). Sensemaking theory could be layered onto resource dependence theory to assess
not just the objective dependencies and resources that are exchanged across the outer and inner
contexts, but also how the organization’s interpretation of these relationships influences
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 147
adaptation behavior over the course of EBP implementation. Organizations interpret the
environment in different ways and these differences in interpretation depend on whether or not
managers think the environment can be analyzed and how actively managers want to try to
understand their environment (Daft & Weick; 1984). Differences in environmental interpretation
could help further delineate the adaptation profiles that emerged in this dissertation.
Another way to supplement and advance prevalent conceptual frameworks is to use
typologies as a way to build new theory for implementation science. To this author’s knowledge,
this dissertation’s creation of adaptation profiles is the first of its kind and organizational
typologies of other key implementation processes have yet to be identified. Doty & Glick (1994)
describe typologies as “complex theories that should be subjected to quantitative modeling and
rigorous empirical testing.” One could see, for example, potential quantitative testing of
relationships between the adaptation profiles identified in this study and client outcomes (e.g.
fidelity), staff outcomes (e.g. turnover and EBP attitudes), and organizational level outcomes
(e.g. the proportion of activities completed and time to completion across the pre-
implementation, implementation, and sustainment stages) (Chamberlain, Hendricks Brown, &
Saldana, 2016). One could also imagine using quantitative analyses to further define, describe,
and measure the ideal Incorporator, Early Investor, and Learner types.
Practical implications. As stated earlier, using a single theoretical metaphor for
organizations (as context) is not just an interesting limitation for researchers to ponder and
discuss. The dominance of this one-dimensional lens has serious implications for the usability
and relevance of our research. Our limited thinking as researchers limits the range of possible
actions that we present to practitioners. The interconnectedness of the outer and inner contexts is
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 148
the reality for system leaders, organizational leaders, and frontline staff who deliver evidence-
base based services to clients.
Study 1 of this dissertation is relevant to practitioners because it first, acknowledges the
fact that not only will the system influence their organization, but there are ways that the
organization might expect to shape the system. Practitioners benefit when they understand that
they are not limited to being passive reactors to systemic directives. In addition, interactions with
the external environment do not stop once service system leaders adopt a new practice and
allocate funding. Instead, organizational leaders should expect to collaborate and negotiate with
service system leaders throughout the lifecycle of the EBP. Furthermore, the burden of EBP
success does not only fall on the organizations responsible for service delivery. Systems and
organizations must work together to keep an EBP alive and flourishing. Study 1 is also relevant
to practitioners because it identifies organizational and service system resources that are needed
and exchanged over the course of implementation. Non-monetary resources such as client flow,
EBP expertise, institutional knowledge, and training and coaching capacity are applicable to a
variety of EBP contexts.
Broadly speaking, Study 2 is useful to practitioners because it normalizes and endorses
different adaptation process (all of which lead to sustainment). There is no one right way to adapt
or amount of adaptation that is required for sustainment. As described earlier in this dissertation,
Study 2 can also be useful for practitioners who are considering using implementation strategies.
Implementation strategies are “methods to enhance the adoption, implementation, sustainment,
and scale up of an innovation” (Proctor, Powell & McMillan, 2013). Organizational leaders can
determine their adaptation profile, then use this information to help with the selection of
implementation strategies. Incorporator leaders may be most interested in a small set of strategies
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 149
that closely align with their current organizational functioning (e.g. updating record systems,
revising professional roles, or developing educational materials). Learner leaders, on the other
hand, may want to continually experiment with a variety of strategies that more substantially
alter the organization’s core functions (e.g. forming new academic partnerships, accessing new
funding streams, or creating a system-wide learning collaborative).
My research trajectory. This dissertation is only a preliminary step in my
implementation research trajectory. In addition to validating, elaborating upon, and uncovering
new organizational adaptation profiles, I hope to focus on decision-making. How do practitioners
make adaptation decisions in an environment characterized by a dynamic, permeable, and
transactional outer and inner context boundary? I hope to use practitioner insight and experience
to flesh out these adaptation profiles and get a better understanding of the combinations of
environmental and internal factors that drive different adaptation choices over the course of EBP
implementation. Using the knowledge I gained from this dissertation, I hope to longitudinally
track decision-making across the implementation stages. I want to learn how practitioners
respond to the external environment in both planned and unplanned ways. Finally, I hope to map
decision-making within a system of actors so that I can better understand the degree to which
adaptation decision-making is a product of system level dynamics, contingencies, and
dependencies.
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 150
References
Aarons, G.A., Ehrhart, M.G., & Farahnak, L.R. (2014). The implementation leadership scale
(ILS): Development of a brief measure of unit level implementation leadership.
Implementation Science, Implementation Science, 9, 45-55.
Aarons, G. A., Hurlburt, M., & Horwitz, S. M. (2011). Advancing a conceptual model of
evidence-based practice implementation in public service sectors. Administration and
Policy in Mental Health and Mental Health Services Research, 38, 4–23.
Andrew, N.E., Middleton, S., Grimley, R., Anderson, C.S., Donnan, G.A., Lannin,
N.A…Cadilhac, D.A. (2019). Hospital organizational context and delivery of evidence-
based stroke care: A cross-sectional study. Implementation Science, 14, 6-18.
Birken, S.A., Bunger, A.C., Powell, B.J., Turner, K., Clary, A.S., Klaman, S.L…Weiner, B.J.
(2017). Organizational theory for dissemination and implementation research.
Implementation Science, 12, 62-77.
Bunger, A.C., Doogan, N., Hanson, R.F., Birken, S.A. (2018). Advice-seeking during
implementation: A network study of clinicians participating in a learning collaborative.
Implementation Science, 13, 101-113.
Bunger, A., Hanson, R., Doogan, N., Powell, B., Cao, Y., & Dunn, J. (2016). Can learning
collaboratives support implementation by rewiring professional networks?
Administration and Policy in Mental Health and Mental Health Services Research, 43(1),
79–92.
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 151
Carter, P., Ozieranski, P., McNicol, S., Power, M., & Dixon-Woods, M. (2014). How
collaborative are quality improvement collaboratives: A qualitative study in stroke care.
Implementation Science, 9, 32-43.
Chamberlain, P., Hendricks Brown, C., & Saldana, L. (2016). Observational measure of
implementation progress in community based settings: The Stages of Implementation
Completion (SIC). Implementation Science, 6, 116-124.
Craig, T.J. & Petzel, R. (2009). Management perspectives on research contributions to practice
through collaboration in the U.S. Veterans Health Administration: QUERI series.
Implementation Science, 4, 8-13.
Cranley, L.A., Keefe, J.M., Taylor, D., Thompson, G., Beacom, A.M. Squires, J.E…Berta, W.B.
(2019). Understanding professional advice networks in long-term care: An outside-inside
view of best practice pathways for diffusion. Implementation Science, 14, 10-23.
Daft, R.L., & Weick, K.E. (1984). Toward a model of organizations as interpretation systems.
Academy of Management Review, 9, 284-295.
Damschroder, L.J., Aron, D.C., Keith, R.E., Kirsh, S.R., Alexander, J.A., & Lowery, J.C. (2009).
Fostering implementation of health services research findings into practice: A consolidated
framework for advancing implementation science. Implementation Science, 4, 50-65.
DiMaggio, P. & W.W. Powell. (1983) The Iron Cage revisited: Institutional isomorphism and
collective rationality in organizational fields. American Sociological Review, 48(2), 147-160.
Doty, D. H., & Glick, W. H. 1994. Typologies as a unique form of theory building: Toward
improved understanding and modeling. Academy of Management Review, 19, 230–251.
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 152
Feldstein, A.C. & Glasgow, R.E. (2008). A Practical, Robust and Sustainability Model (PRISM)
for integrating research findings into practice. The Joint Commission Journal on Quality and
Patient Safety, 34(4), 228- 243.
Fernandez, M.E., Walker, T.J., Weiner, B.J., Calo, W.A., Liang, S., Risendal, B…Kegler, M.C.
(2018). Developing measures to assess constructs from the inner setting domain of the
Consolidated Framework for Implementation Research. Implementation Science, 13, 52-
65.
Garvin, D.A. (2000). Learning in action: A guide to putting the learning organization to work.
Boston, MA: Harvard Business School Press.
Geerligs, L., Rankin, N.M., Shepherd, H.L., & Butow, P. (2018). Hospital-based interventions: A
systematic review of staff-reported barriers and facilitators to implementation processes.
Implementation Science, 13, 36-53.
Harris, J.K., Provan, K., Johnson, K.J., & Leischow, S.J. (2012). Drawbacks and benefits
associated with inter-organizational collaboration along the discovery-development-
delivery continuum: A cancer research network case study. Implementation Science, 7,
69-82.
Jabbour, M., Newton, A.S., Johnson, D., & Curran, J.A. (2018). Defining barriers and enablers
for clinical pathway implementation in complex clinical settings. Implementation
Science, 13, 139- 152.
Jennings, D.F. & Seaman, S.L. (1994). High and low levels of organizational adaptation: An
empirical analysis of strategy, structure, and performance. Strategic Management
Journal, 15, 459-475.
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 153
Meyer, J.W. & Rowan, B. (1977). Institutionalized organizations: Formal structure as myth and
ceremony. The American Journal of Sociology, 83(2), 340-363.
Morgan, G. (1997). Images of organization. Thousand Oaks, CA: Sage Publications, Inc.
Moullin, J.C., Dickson, K.S., Stadnick, N., Rabin, B., Aarons, G.A. (2019). Systematic review of
the exploration, preparation, implementation, sustainment (EPIS) framework.
Implementation Science, 14, 1-17.
Proctor, E., Powell, B., & McMillen, J. (2013). Implementation strategies: Recommendations for
specifying and reporting. Implementation Science, 8, 139-150.
Ramsey, A., Lawrence, K., Prusaczyk, B., Baumann, A., Kryzer, E., & Proctor, E.
Organizational Measures. [Internet]. St. Louis, MO: Washington University; 2016
October. Eight toolkits related to Dissemination and Implementation. Available from
https://sites.wustl.edu/wudandi.
Rihoux, B. (2006). Qualitative comparative analysis (QCA) and related systematic comparative
methods: Recent advances and remaining challenges for social science research.
International Sociology, 21(5), 679-706.
Rye, M., Torres, E.M., Friborg, O., Skre, I., & Aarons, G.A. (2017). The Evidence-based
Practice Attitude Scale-36 (EBPAS-36): A brief and pragmatic measure of attitudes to
evidence-based practice validated in US and Norwegian samples. Implementation
Science, 12, 44-55.
Scott, W. R. & Davis, G.F. (2007). Organizations and organizing: Rational, natural, and open
systems perspectives. Upper Saddle River, NJ: Pearson Education, Inc.
HOW ORGANIZATIONS ADAPT DURING EBP IMPLEMENTATION 154
Shea, C.M., Jacobs, S.R., Esserman, D.A., Bruce, K., & Weiner, B.J. (2014). Organizational
readiness for implementing change: a psychometric assessment of a new measure.
Implementation Science, 9, 7-22.
Smith, S.N., Almirall, D., Prenovost, K., Goodrich, D.E., Abraham, K.M., Liebrecht, C., &
Kilbourne, A.M. (2018). Organizational culture and climate as moderators of enhanced
outreach for persons with serious mental illness: Results from a cluster-randomized trial
of adaptive implementation strategies. Implementation Science, 13, 93-108.
Templeton, A.R., Young, L., Bish, A., Gnich, W., Cassie, H., Treweek, S…Ramsay, C. (2016).
Patient-, organization-, and system-level barriers and facilitators to preventive oral health
care: A convergent mixed-methods study in primary dental care. Implementation Science,
11, 5-19.
Williams, N.J., Ehrhart, M.G., Aarons, G.A., Marcus, S.C., & Beidas, R.S. (2018) Linking molar
organizational climate and strategic implementation climate to clinicians” use of
evidence-based psychotherapy techniques: cross-sectional and lagged analyses from a 2-
year observational study. Implementation Science, 13, 85-98.
Williams, N.J., Frank, H.E., Frederick, L., Beidas, R.S., Mandell, D.S., Aarons, G.A…Locke, J.
(2019). Organizational culture and climate profiles: Relationships with fidelity to three
evidence-based practices for autism in elementary schools. Implementation Science, 14,
15-29.
Abstract (if available)
Abstract
This dissertation proposes a new metaphor for implementation science: organizations as implementation actors. What does it mean for an organization to be an actor and not just the context for EBP implementation? First, this metaphor highlights and acknowledges the relational nature of organizations. Second, thinking about organizations as implementation actors reminds us that organizations are a combination of interacting elements, agents, and processes. This interaction among factors may be lost when we parcel out and focus on isolated organizational characteristics. Third, this metaphor requires identifying and classifying organizational behavior in a way that allows for the examination of different behavioral pathways that lead to a particular outcome. This study illustrates these three elements of the organizations as implementation actors metaphor through an exploration of how organizations that sustained SafeCare engaged in the process of adaptation. ❧ Study 1 (Chapter 2) is a conceptual paper that argues for a more open systems approach to implementation research using organization-service system interactions around contracting arrangements as an illustrative example. The research question for dissertation Study 1 is: How do child welfare systems and community-based organizations interact around SafeCare contracting arrangements? This study uses content analysis and multiple case study analysis approaches with a dataset of 88 transcripts and over 100 SafeCare contracting documents. Study 1 findings illustrate and explain bi-directional influence, dependencies and resources exchanged between child welfare systems and organizations during SafeCare implementation. This study provides important contextual information for understanding adaptation in Study 2. ❧ Study 2 (Chapter 3) qualitatively explores the question: How do organizations approach the adaptation process during SafeCare implementation? This study uses a typological approach and demonstrates a way to holistically examine organizational level behavior around a specific implementation process (adaptation). Study 2 uses a rigorous set of qualitative methods including coding of transcripts, document review, multiple case study analysis, and member checking using a visual elicitation tool. Data sources include 177 transcripts, contracting documents from Study 1, and home visitor, coach, and trainer SafeCare manuals. Iterative peer debriefing and member checking meetings with the research team and practitioners were an important methodological element in Study 2. Key findings are three organizational level approaches to adaptation during SafeCare implementation: Incorporators, Early Investors and Learners. These organizational level approaches to adaptation are the outcome variable in Study 3. ❧ The broad goal of Study 3 (Chapter 4) is to demonstrate the usefulness of Qualitative Comparative Analysis (QCA) for addressing a particular literature gap: how an organization’s collection of relationships with actors in the external environment combine and influence behavior around a specific implementation process. This study examines three environmental features that describe the organization’s relationship with other actors: lead agency status (an organization’s position within the network of other organizations that are providing SafeCare), funding level in SafeCare contracting arrangements (resources exchanged between organization and system) and the organization’s ability to influence the SafeCare contract (openness of relationship between organization and system). The implementation process of interest is how organizations adapt during EBP implementation and this study focuses on the Incorporator and Early Investor profiles developed in Study 2.
Linked assets
University of Southern California Dissertations and Theses
Conceptually similar
PDF
Multilevel influences on organizational adoption of innovative practices in children's mental health services
PDF
Delineating the processes and mechanisms of sustainment: understanding how evidence-based prevention initiatives are maintained after initial funding ends
PDF
A system framework for evidence based implementations in a health care organization
PDF
Exploring the social determinants of health in a population with similar access to healthcare: experiences from United States active-duty army wives
PDF
Implementation of peer providers in integrated health care settings
PDF
Integration of behavioral health outcomes into electric health records to improve patient care
PDF
Promoting smart decarceration: mitigating racial bias as a strategy for addressing disproportionate minority contact and confinement (DMC)
PDF
Tobacco use change among formerly homeless supportive housing residents: socioecological barriers and facilitators to cessation
PDF
Strength-Based Reporting: a trauma-informed practice for mandated reporters, to address behavioral health concerns in children at risk of child welfare involvement
PDF
How can metrics matter: performance management reforms in the City of Los Angeles
PDF
Engaging homeless men and shelter providers to adapt an existing evidence-based HIV prevention intervention
PDF
Full spectrum transition assistance: preventing loneliness and social isolation in military members during moves
PDF
Conjoint homeless prevention services for older adults
PDF
Inter professional education and practice in the health care setting: an innovative model using human simulation learning
PDF
Assessing implementation of a child welfare system practice change
PDF
LGBT+Aging Immersion Experience: an innovative LGBT cultural competency course for healthcare professionals and students
PDF
Exploring the implementation of the Indian Child of Welfare Act of 1978
PDF
Substance use disorder treatment clinician and director challenges working with collaborative justice court and child welfare systems: A multisite qualitative study
PDF
Improving continuity of care in the juvenile justice system: A pilot proposal
PDF
Acculturation team-based clinical program: pilot program to address acculturative stress and mental health in the Latino community
Asset Metadata
Creator
Lengnick-Hall, Rebecca
(author)
Core Title
How organizations adapt during EBP implementation
School
Suzanne Dworak-Peck School of Social Work
Degree
Doctor of Philosophy
Degree Program
Social Work
Publication Date
07/24/2019
Defense Date
05/08/2019
Publisher
University of Southern California
(original),
University of Southern California. Libraries
(digital)
Tag
adaptation,community-based organizations,evidence-based practice,implementation,OAI-PMH Harvest
Format
application/pdf
(imt)
Language
English
Contributor
Electronically uploaded by the author
(provenance)
Advisor
Hurlburt, Michael (
committee chair
), Fiss, Peer (
committee member
), Henwood, Benjamin (
committee member
)
Creator Email
rebecca.lengnick.hall@gmail.com,rlengnick-hall@email.wustl.edu
Permanent Link (DOI)
https://doi.org/10.25549/usctheses-c89-189001
Unique identifier
UC11663178
Identifier
etd-LengnickHa-7591.pdf (filename),usctheses-c89-189001 (legacy record id)
Legacy Identifier
etd-LengnickHa-7591.pdf
Dmrecord
189001
Document Type
Dissertation
Format
application/pdf (imt)
Rights
Lengnick-Hall, Rebecca
Type
texts
Source
University of Southern California
(contributing entity),
University of Southern California Dissertations and Theses
(collection)
Access Conditions
The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the a...
Repository Name
University of Southern California Digital Library
Repository Location
USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA
Tags
adaptation
community-based organizations
evidence-based practice
implementation