University of Southern California Dissertations and Theses

The influence of contextual factors on the processes of adoption and implementation of evidence-based substance use prevention and tobacco cessation programs in schools

(USC Thesis Other)

The influence of contextual factors on the processes of adoption and implementation of evidence-based substance use prevention and tobacco cessation programs in schools

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content

THE INFLUENCE OF CONTEXTUAL FACTORS ON THE PROCESSES OF ADOPTION AND
IMPLEMENTATION OF EVIDENCE-BASED SUBSTANCE USE PREVENTION AND TOBACCO
CESSATION PROGRAMS IN SCHOOLS

by

Melissa A. Little

A Dissertation Presented to the
FACULTY OF THE USC GRADUATE SCHOOL
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulfillment of the
Requirements for the Degree
DOCTOR OF PHILOSOPHY
(PREVENTIVE MEDICINE)

May 2012

Copyright 2012 Melissa A. Little

ii

ACKNOWLEDGEMENTS
I would like thank my committee members for their encouragement, guidance and support.
Dr. Steven Sussman, thank you for the wisdom and encouragement to be a productive
researcher. Dr. Nathaniel Riggs, thank you for fostering my independence as a researcher. Dr.
Ping Sun, thank you for always answering my complicated statistical questions. Dr. Larry
Palinkas, thank you for introducing me to a different perspective. I would especially like to thank
Dr. Luanne Rohrbach for encouraging me to develop my own research ideas and pursue
independent funding. I am forever grateful to her for sharing her knowledge and passion for
translational research, without her none of this would have been possible.
I would like to extend a heartfelt thanks to the IPR faculty. Thank you for allowing me to
collect your data and develop a passion for prevention research at an early age. I would not be
the researcher I am today without those experiences. To all of the countless students and staff
who have supported me both directly and indirectly throughout the years, I sincerely thank you.
I would also like to thank Marny Barovich for her guidance and support. I would like to thank Dr.
Christopher Ringwalt for graciously allowing me to use his data for one of my studies. In
addition, these studies would not have been possible without the thousands of students,
teachers and administrators who were willing to share their thoughts, opinions and behaviors.
I would also like to thank my family for always believing in me, even when I didn’t. Without
your generosity, love and support I would not be the woman I am today. To my Mom, thank you
for teaching me everything you know about managing a project, and supporting me on my own
project. To my husband, Tim, thank you for your sacrifice, support and most of all love.
This research was supported by Tobacco-Related Disease Research Program Dissertation
Award (19DT-0002).
iii

TABLE OF CONTENTS
Acknowledgements ii
List of Tables vi
List of Figures vii
Abstract viii
Chapter 1. Introduction
Specific Aims 1
Study 1 5
Study 2 5
Study 3 6
Background 6
Theoretical Rationale 10
Empirical Research Results Guiding Model Development 14
Community Context 16
Organizational Context 19
Implementation System 22
Provider Characteristics 24
Program Characteristics and the Audiences Targeted 25
Chapter 2. Study 1: Socio-ecological factors related to use of evidence-based
substance use prevention programs in middle schools by school districts nationwide
Abstract 27
Introduction 29
The Current Study 34
Methods 35
Study Sample and Data Collection 35
Measurement 37
Dependent Variable 38
Independent Variables 39
Data Analysis 40
Results 41
District Characteristics 41
Predictors of Overall Use 43
Comparison of Early Versus Late Adopters 46
Discussion 48
Limitations 51
Conclusion 52

iv

Chapter 3. Study 2: Community and Organizational Factors Related to the Fidelity
of Implementation of Project TND in High Schools across the Nation
Abstract 54
Introduction 55
The Current Study 58
Methods 59
School Selection and Experimental Design 59
Subjects 60
Data Collection and Measurement 61
Teacher Self-Report Measures 62
Assessment of Implementation Fidelity 63
School Administrator Assessment 64
Student Surveys 64
Additional School Context Measures 66
Data Analysis 66
Results 67
Correlates of Fidelity of Implementation 67
Relationships between Fidelity of Implementation and Student
Outcomes 68
Discussion 69
Limitations 72
Conclusion 73
Chapter 4. Study 3: Study of Adoption and Use of Research-Validated Tobacco Use
Prevention and Cessation Programs by Districts throughout California
Abstract 74
Introduction 76
The Current Study 82
Design and Methods 82
Target Sample 82
Data Collection 83
Final Study Sample 84
Measures 84
Dependent Variables 85
Independent Variables 86
Data Analysis 88
Results 89
Sample Characteristics 89
Funding and Program Use 90
Correlates of Adoption and Use of Evidence-based Programs 93
Discussion 101
Limitations 105
Conclusion 106

v

Chapter 5. Summary, Limitations, Strengths and Implications
Summary of Findings 107
Strengths and Limitations 111
Implications for Prevention 113
References 118
Appendix A. Research-validated tobacco and substance use prevention programs
in 2010 131
Appendix B. Approved Tobacco Cessation Programs in 2010 133

vi

LIST OF TABLES
Table 2.1. Characteristics of school districts 42
Table 2.2. Summary of hierarchical regression analysis for variables predicting
the use of evidence-based substance use prevention programs at Wave 2 45

Table 2.3. Comparison of early and late adopters of evidence-based programs 47
Table 3.1. Characteristics of teachers and school administrators 61
Table 3.2. Correlates of fidelity of TND program implementation 68
Table 3.3. Relationships between implementation fidelity and student outcomes 69
Table 4.1. Characteristics of organization (N=235) 90
Table 4.2. Summary of hierarchical regression analysis for variables correlated with
program adoption (applying for TUPE grant) 94

Table 4.3. Summary of hierarchical regression analysis for variables correlated with
using a research-validated tobacco use prevention program 96

Table 4.4. Summary of hierarchical regression analysis for variables correlated with
using an approved tobacco cessation program 100

vii

LIST OF FIGURES
Figure 1.1. Heuristic model of diffusion of substance use prevention programs in
schools 4
Figure 1.2. Conceptual model of diffusion of substance use prevention programs in
schools 14

Figure 4.1. TUPE funding status and evidence-based program use among
organizations 91

viii

ABSTRACT
Although considerable resources have been spent developing and disseminating effective
school-based substance use prevention programs, many school districts in the United States fail
to use prevention programs with proven effectiveness. Moreover, there is a dearth of research
understanding the factors that promote adoption, implementation fidelity, and sustained use of
these programs in schools. Consequently, a significant gap remains in what we know about how
to effectively “translate” evidence-based programs from research to practice (L. A. Rohrbach,
Grana, Sussman, & Valente, 2006).
Research grounded in the diffusion of innovations theory (Rogers, 1983), theory driven
evaluations (Chen, 1998), systems models (Estabrooks & Glasgow, 2006) and recent reviews of
the literature (Durlak & DuPre, 2008) have identified some community-, organizational-, and
individual-level factors that are correlated with the translation of evidence-based programs in
schools. The studies presented here explore the relationships between factors at several levels
of the ecological framework and program adoption, implementation fidelity and sustained use
of evidence-based substance use prevention and tobacco cessation programs in schools.
The findings presented in these studies suggest that school-based substance use prevention
programming is influenced by a variety of contextual factors occurring at multiple ecological
levels. Characteristics of the community and organization, beliefs of key decision makers, and
funding were important factors in promoting adoption, implementation fidelity and sustained
use of evidence-based substance use prevention programs in schools. In order to move the field
ahead, researchers need to account for the multiple systems at play in schools when designing
research trials and school-based programs. Ultimately, translational research can be used to
ix

increase the use of evidence-based prevention programs in schools, which should lead to
reductions in substance use and other problem behaviors among adolescents.

1

CHAPTER 1. INTRODUCTION
SPECIFIC AIMS
Rates of substance use among adolescents have leveled off in recent years; however, the
prevalence of substance use among American youth remains sufficiently widespread to merit
concern (Johnston, O’Malley, Bachman, & Schulenberg, 2009). Nearly three quarters (72%) of
adolescents report drinking alcohol by the time they reach 12
th
grade and almost half have tried
cigarettes (45%) and illicit drugs (47%) (Johnston, et al., 2009). In order to reduce these rates of
use, a comprehensive approach to substance use among adolescents is needed. School-based
substance use prevention and cessation programming is an integral part of this approach (Lantz,
et al., 2000).
There is now considerable empirical evidence indicating that a number of school-based
programs are effective in preventing and reducing substance use among adolescents (Skara &
Sussman, 2003); yet a recent study examining the utilization of these prevention programs by
school districts across the nation found that less than half reported using evidence-based
programs (Ringwalt, et al., 2011). This is in spite of the fact that both state and federal policies
of the last decade for substance use prevention programs have stipulated that schools receiving
public funding implement ‘research-based’ programs (California Department of Education, 2011;
United States Department of Education, 1998). These findings suggest that a significant gap
remains in what we know about how to effectively “translate” evidence-based programs from
research to practice (Rohrbach, Grana, Sussman, & Valente, 2006). Although we know that the
translation of innovations generally occurs through a four-stage process, including:
dissemination (gaining knowledge about the program and forming beliefs), adoption (deciding
to use the program), implementation (taking action, i.e., putting the program into use) and
2

sustainability (continuing to use the program) (Rohrbach, et al., 2006), we know very little about
the factors that influence organizations’ actions and intentions at each stage of this process.
Currently, there is a dearth of empirically supported research about the contextual
conditions that promote adoption, implementation and sustainability of evidence-based
substance use prevention programs in schools (Domitrovich & Greenberg, 2000; Gregory, Henry,
& Schoeny, 2007; Scheirer & Dearing, 2011; Shinn, 2003). Additionally, there are several key
limitations to existing research on the issue. One limitation is that studies have focused on the
actions of individuals who make program adoption decisions, without taking into account the
organization- and system-level contexts in which these decisions are made (Davies & Nutley,
2008). Although the attitudes, beliefs and motivations of individuals influence a school district’s
decision to adopt an evidence-based prevention program, more research needs to be devoted
to understanding the context surrounding the individual’s decision, such as the organizational
climate, districts support and decision-making power within the district. Additionally, few
studies have examined the use of evidence-based research findings in program adoption
decisions made by mid-level stakeholders in service delivery organizations, such as school
district administrators. Additionally, the majority of translational research has focused on the
earlier phases of diffusion (Brownson, et al., 2007; Durlak & DuPre, 2008; Mihalic, Fagan, &
Argamaso, 2008; Ringwalt, et al., 2003; Rohrbach, Ringwalt, Ennett, & Vincus, 2005), while very
little attention has been given to the long-term viability of programs (Scheirer & Dearing, 2011),
although there are clear arguments why sustainability should be a priority for public health
researchers and practitioners. Changing health behaviors is often a slow process and
terminating a program after it has been successfully implemented is not only wasteful of the
significant start-up costs in human, fiscal and technical resources that go into developing and
3

adopting a program, but it also has the potential to increase recidivism in negative health
outcomes (Shediac-Rizkallah & Bone, 1998). Another limitation of extant research is that a large
majority of studies are anecdotal case studies, informal reviews and commentaries, with very
few empirical studies. There is a growing consensus in the field that the translation of evidence-
based prevention programs from research to practice is a complex endeavor, even more so than
the programs that are the subject of the translation efforts (Fixsen, Naoom, Blase, Friedman, &
Wallace, 2005; Rohrbach, et al., 2006). However despite the growing body of literature
supporting the importance of implementation fidelity in producing desired program effects
(Durlak & DuPre, 2008), little is known about how contextual factors of the school and
community influence the relationship between the implementation fidelity and program
outcomes.
In the current dissertation studies, we attempt to overcome these limitations by examining
the processes of adoption, implementation and sustainability in three large datasets, two of
which have longitudinal data. Moreover, the research examines the processes of adoption,
implementation and sustainability from multiple perspectives (i.e., district administrators,
school administrators, and teachers). Additionally, the studies apply a theory-based analytic
strategy, hierarchical regression (Victora, Huttly, Fuchs, & Olinto, 1997), to explore the complex
relationships between factors from a variety of contextual levels (i.e., the community,
organization, provider, program and audience targeted), and use of evidence-based prevention
programs, to help us understand what factors are most important to guide translation efforts in
the future. Previous reviews of the literature support a multi-level ecological approach to the
study of translation (Durlak & DuPre, 2008; Fixsen, et al., 2005). The following contextual factors
have been associated with the use of evidence-based prevention programs in schools: (1)
4

community context, (2) organizational context, (3) implementation system (i.e., training,
technical assistance, etc.), (4) provider characteristics, and (5) program characteristics and the
audience targeted. The present studies use a multi-level ecological framework, guided by the
diffusion of innovations model (Rogers, 2002), theory driven evaluations (Chen, 1998), systems
models (Estabrooks & Glasgow, 2006) and recent reviews of the literature (Durlak & DuPre,
2008; Fixsen, et al., 2005) to explore factors that affect adoption, implementation and
sustainability of evidence-based prevention programs in schools (as presented in Figure 1.1).
Figure 1.1. Heuristic model of diffusion of substance use prevention programs in schools

5

Study 1
Data for the first study comes from the School-based Substance Use Prevention Programs
Study (SSUPPS), a longitudinal nationwide survey of substance use prevention practices in
middle schools. The specific aims of study 1 were as follows:
 Aim 1: Examine the relative impact of community factors, organizational factors and
individual beliefs about the program on districts’ decisions to adopt and sustain
evidence-based substance use prevention programs in schools.
 Aim 2: Compare characteristics of early and late adopters of evidence-based substance
use prevention programs in a sample of school districts.
Study 2
Data for the second study comes from the Dissemination Trial of Project TND (Rohrbach,
Gunning, Sun, & Sussman, 2010), a national study of the relative effectiveness of two teacher
training approaches, a standard training workshop compared to a comprehensive training and
implementation support model (Rohrbach, Gunning, Sun, et al., 2010). The specific aims of study
2 were as follows:
 Aim 1: Explore the extent to which community, organizational and provider
characteristics influence the fidelity of implementation of Project TND in a sample of
high school teachers from around the country.
 Aim 2: Examine whether implementation fidelity predicts program outcomes,
controlling for contextual and individual factors.

6

Study 3
In the third study, school district administrators and county office of education TUPE
coordinators throughout California were surveyed regarding their use of tobacco prevention and
cessation programs in schools. The specific aims of study 3 were as follows:
 Aim 1: Identify the community, organizational and individual-level factors that influence
the adoption of research-validated tobacco prevention programs in schools.
 Aim 2: Examine the community, organizational and individual-level factors that
influence the use (implementation) of research-validated tobacco prevention programs
in schools.
 Aim 3: Examine the community, organizational and individual-level factors that
influence the use of approved tobacco cessation programs in schools.
Background
Although rates of substance use among adolescents have leveled off in recent years, the
problem of substance use among American youth remains sufficiently widespread to warrant
concern (Johnston, et al., 2009). Almost half (45%) of adolescents in the United States (U.S.)
have tried cigarettes by their senior year of high school, and one in five 12
th
graders are current
smokers (Johnston, et al., 2009). As early as 8
th
grade, one in five adolescents have tried
cigarettes and 7% are current smokers. In California, currently the prevalence of tobacco use
among adolescents is increasing in contrast to previous years of decline (McCarthy, et al., 2008).
With regards to alcohol, nearly three quarters (72%) of seniors have reported alcohol
consumption and 39% have had at least one alcoholic drink by 8
th
grade. Roughly half (47%) of
adolescents in the U.S. have tried an illicit drug by the time they finish high school, with one in
four (28%) having done so by 8
th
grade (Johnston, et al., 2009). Additionally, one quarter of
7

adolescents have tried an illicit drug other than marijuana by the time they finish high school.
These findings highlight the need for continued use of substance use prevention programming
with adolescents. In order to prevent the prevalence of substance use among adolescents, a
comprehensive approach is needed and school-based substance use prevention and cessation
programming is an integral part of this approach (Lantz, et al., 2000).
Over the past several decades, there has been considerable research devoted to the testing
of school-based programs designed to prevent a variety of health problems among youth. As a
result, there are now a number of research validated prevention programs that have been
shown to reduce the onset of substance abuse, mental health and violent behaviors (Rohrbach,
et al., 2006). In addition, there is now substantial empirical evidence that school-based
prevention programs, particularly those targeting substance abuse, can have long-term
effectiveness (Flay, 2009; National Cancer Institute & Substance Abuse and Mental Health
Services Administration, 2010; Skara & Sussman, 2003). As a result, various federal agencies
have developed lists of ‘best practice,’ ‘research-tested,’ and ‘evidence-based’ prevention
programs that have proven effectiveness in reducing youth risk behaviors (Fagan, Hanson,
Hawkins, & Arthur, 2008a; National Cancer Institute & Substance Abuse and Mental Health
Services Administration, 2010; United States Department of Health and Human Services &
Substance Abuse and Mental Health Services Administration, 2010).
Given the obligatory nature of school attendance, schools have been the primary context for
delivering health interventions targeting adolescents (Ellickson, 1995; Gregory, et al., 2007).
Although there are now a number of effective school-based evidence-based prevention
programs available for dissemination, many school districts fail to adopt and implement these
programs. This is in spite of the fact that many federal and state policies of the last decade (e.g.,
8

Safe and Drug Free Schools Act of 1999 and the No Child Left Behind Act of 2001) mandated that
schools receiving certain state and federal funding use programs with proven effectiveness
(Hallfors & Godette, 2002). However school districts’ decisions regarding the use of evidence-
based substance use prevention programs are complicated by increasing demands on schools to
focus their efforts on core subjects (e.g., English, math and science) (Kaftarian, Robertson,
Compton, Davis, & Volkow, 2004), rising budget crises in states nationwide (Dusenbury &
Hansen, 2004) and recent changes to federal funding for substance use prevention education. In
2010, the Safe and Drug Free Schools and Communities (SDFSC) state grants program, the main
source of funding for substance use prevention education in schools, was eliminated from the
federal budget (Hallfors & Godette, 2002). Moreover, in California, prior to 2009, school
districts were given an annual entitlement to spend on tobacco education based on enrollment
of students in grades four through eight. However, recent changes to the California Health and
Safety Code provisions eliminated the grades four through eight annual entitlements and
established a competitive grant process for school-based tobacco education funds. Given these
recent policy changes, it is essential for researchers to understand what factors lead school
districts to use research-validated programs in this new climate.
In a national study assessing the use of evidence-based substance use prevention curricula
in middle schools, Ringwalt et al., (2011) found that only 46.9% were using an evidence-based
program. Furthermore, only 25.9% of respondents reported using an evidence-based program
more than other prevention programs. Instead, many school districts adopt and implement
heavily marketed curricula that have not been evaluated, have been evaluated insufficiently, or
have been show to be unsuccessful in reducing problem behaviors (Hallfors & Godette, 2002;
Rohrbach, D'Onofrio, Backer, & Montgomery, 1996; Swisher, 2000; Tobler & Stratton, 1997).
9

Still others implement ‘homegrown’ prevention curricula that they developed themselves
(Hansen & McNeal, 1997). Unfortunately, we do not know whether these programs are effective
in reducing substance use among adolescents.
Similarly, many schools fail to implement evidence-based tobacco cessation programs in the
U.S. In one of the only studies assessing the prevalence of use of smoking cessation curricula,
Curry et al. (2007) found that although two-thirds of counties across the nation reported
utilizing at least one adolescent smoking cessation program, less than half of those surveyed
implemented a program developed by an outside agency. Among those with externally-
developed programs, reasons for adoption included characteristics of the curricula, such as its
evidence-base, its ease of adoption, and the fact that it was recommended by tobacco cessation
experts and other colleagues (Curry, et al., 2007). Forty percent of counties cited an
organizational initiative as the main impetus for offering a youth cessation program. At present,
there is a need for more studies about how and why schools adopt and implement evidence-
based tobacco cessation programs.
The gap between what we know is effective and what is actually done in the “real world”
remains a critical issue in the health prevention field (Hallfors & Godette, 2002). Researchers
who have studied the process of moving public health programs into practice have labeled this
process as translation, dissemination, implementation, and diffusion, with little agreement on
these terms (Bowen, et al., 2009; Rohrbach, et al., 2006). The National Institutes of Health (NIH)
has defined two phases of translation research (Office of Behavioral and Social Sciences,
National Institutes of Health, & Office of the Director, 2005). The first phase, “Type I
translation,” is aimed at moving basic sciences into a realm of social or personal significance
(Sussman, Valente, Pentz, Rohrbach, & Skara, 2006). The second phase, “Type II translation,”
10

focuses on institutionalizing effective programs, products and services in the community
(Rohrbach, et al., 2006). It is commonly agreed upon that the translation of evidence-based
prevention programs from research to practice in schools occurs in four phases: (1)
dissemination, (2) adoption, (3) implementation, and (4) sustainability (Rohrbach, et al., 2006).
The proposed dissertation research focuses on type 2 translation.
Theoretical Rationale
Rogers’ Diffusion of Innovations Theory has been the predominant model for
conceptualizing the complicated, long-term process of diffusing or translating effective public
health programs to many types of settings, including schools (Rogers, 1983). The key
assumptions of this model are that knowledge is a product, generalizable across contexts, and
effective research will naturally be adopted by consumers (Estabrooks & Glasgow, 2006). Based
on these assumptions, Rogers’ innovation-decision process posits that an individual or decision-
making unit passes through five successive phases in regard to a specific innovation, including:
(1) knowledge, or awareness of the innovation; (2) persuasion, or forming an opinion about the
innovation; (3) decision about whether to adopt or reject the innovation; (4) implementation of
the innovation; and (5) confirmation, or reinforcement of the decision made (Rogers, 1983).
Perceived characteristics of the innovation (e.g., relative advantage, compatibility, complexity,
trialability, and observability) are thought to influence an individual’s behavior at the persuasion
stage. However, linear diffusion models such as this have focused primarily on individual
adopters of innovations, neglecting the organizational- and system-level contexts in which
adoption decisions are made. Thus, as a field we still do not fully understand how to translate
health interventions from research to practice.
11

To address these limitations, researchers have proposed new models of decision-making
called “systems thinking” or “systems models” (Estabrooks & Glasgow, 2006). Consistent with
social ecological models, systems models posit that knowledge integration is contextual and tied
to organizational priorities and culture. They assume that relationships are critical, but must be
understood from a multilevel systems perspective. In doing so, systems approaches consider
important relationships between components of a system and changes in the system over time
(Ginexi, et al., 2010). Applying this to school-based research we must take into account the
multiple systems (e.g., state agencies, school districts, school administrators, teachers, unions,
school boards, etc.) affecting decision-making within schools. Chen (1998) proposed a
conceptual model to understand factors that influence program implementation, including the
organizational context in which the program is implemented (e.g., administrator support, school
climate), characteristics of the implementer, and the implementation system (e.g., staff training
and infrastructure to coordinate implementation). In Chen’s model (1998), interventions take
place within an implementation system, which affords the means and context for delivering the
intervention (Greenberg, Domitrovich, Graczyk, & Zins, 2005). Chen argues that the
implementation system, which is embedded in the broader organizational and community
context, is vital to program success and therefore must be monitored as part of program
evaluation (Greenberg, et al., 2005).
Other frameworks, such as Glasgow and colleagues’ RE-AIM model, build on these models
by emphasizing the importance of both the individual and the organization in maximizing the
public health impact of effective interventions (Glasgow, Vogt, & Boles, 1999). The RE-AIM
framework specifies five dimensions that are relevant to moving research into practice,
including: (1) Reach into the target population, (2) Effectiveness, or the intended results of the
12

intervention, (3) Adoption, or the uptake of the intervention among targeted settings and
providers, (4) Implementation, as measured by the quantity, quality, and consistency of program
delivery, and (5) Maintenance, which is the long-term results of the program and/or
institutionalization of it in targeted settings (Glasgow, et al., 1999).
In the present dissertation research, we examine both behaviors (i.e., implementation and
sustained use) and cognitions (i.e., awareness, beliefs, and adoption or intentions to use) of
individuals within school organizations. Both Rogers and RE-AIM posit, implicitly, that adoption
(intentions to use) leads to actual use (implementation) of an innovation. The Theory of
Reasoned Action (TRA), asserts that the most important determinant of behavior is one’s
behavioral intention (Montano & Kasprzyk, 2002). TRA focuses on the cognitive factors (beliefs
and attitudes) that determine a behavioral intention (adoption). TRA defines a behavioral
intention as the perceived likelihood of performing the behavior and posits that behavioral
intentions are influenced by subjective norms (beliefs about whether most people approve or
disapprove of a behavior) and attitudes toward the behavior (Montano & Kasprzyk, 2002).
Subjective norms are influenced by normative beliefs and motivation to comply with the
referents for the beliefs. Applying this theory to the processes of translation would suggest that
normative influences such as the perceived prevalence of use of an evidence-based prevention
program by other school districts will influence the intention to use an evidence-based
prevention programs.
In both Rogers’ theory and the RE-AIM framework, a decision is made during the adoption
phase to either use or reject the innovation which leads directly to the implementation phase
(actual use). Therefore, it is in line with TRA, Rogers’ theory and the RE-AIM framework to
consider the adoption phase as school districts’ intentions to use the program. TRA will help us
13

to understand the cognitive processes or beliefs that lead to the action of program delivery. In
the application of theory to the diffusion of evidence-based programs and practices, researchers
often fail to distinguish between the stages of adoption and implementation, referring to them
simply as studies of the implementation or use of evidence-based prevention programs.
However, given that they are theoretically distinct stages, throughout this proposal we will refer
to them separately. Districts’ and schools’ intentions to use evidence-based programs will be
considered adoption, their actual use implementation, and their use continued use over time
will be considered sustained use.
The conceptual model for organizing our findings draws from Rogers’ Diffusion theory
(Rogers, 1983), Chen’s conceptual model (Chen, 1998) and systems frameworks (Estabrooks &
Glasgow, 2006) (Figure 1.2). We posit that the translation of evidence-based prevention
programs from research to practice in school districts occurs in five phases: (1) awareness of the
program, (2) beliefs about the program, (3) adoption, (4) implementation, and (5) sustainability.
We hypothesize that community and organizational factors provide the context for the
translation of evidence-based prevention programs. Other factors hypothesized to influence this
process include the implementation system (e.g., provision of training and technical assistance),
provider characteristics, program characteristics and the audience targeted. In the current
studies, we examine factors related to the adoption, implementation and sustainability of
evidence-based prevention programs in schools.

14

Figure 1.2. Conceptual model of diffusion of substance use prevention programs in schools

Empirical Research Results Guiding Model Development
Empirical work from the educational policy field is useful in conceptualizing the specific case
of how school districts adopt, implement and sustain prevention programs. To begin with,
school districts in the United States are both vertically and horizontally segmented (Meyer &
Scott, 1983). Because of the nature of school settings, with multiple levels of decision-making
dispersed among a central administration and multiple schools, decisions about implementation
are rather complex (Shinn, 2003; Spillane, 1998). A district-level decision to adopt an evidence-
based prevention program may bear little resemblance to what the classroom teacher actually
implements. Additionally, the wider institutional environment and the amount of influence
different actors have in shaping educational policies and agendas vary by district. For instance,
districts with “strong unions, pressing clients, and aggressive state departments of education are
clearly different from those in districts with more placid environments” (Hannaway, 1993).
15

Although the influence of the broader environment is often overlooked, it is in fact very
important in influencing the decisions and ultimate actions of school districts.
Huberman (1983) drew from knowledge transfer theory (Havelock, 1969) to explain how
different types of knowledge are translated from researchers to school districts (Huberman,
1983). He emphasized the importance of antecedent coupling (e.g., collaboration, formal
linkages), school district commitment (e.g., priority, administrator support), intermediary unit-
staff characteristics (e.g., staff stability, energy of coordinator), program characteristics (e.g.,
perceived ease of use) and external resources (e.g., funding) (Huberman, 1983). More recently,
in a review on the use of research evidence by school districts, Honig & Coburn (2008) found
that district administrators’ decisions about program adoption were informed by a broad range
of evidence sources, from published scientific articles to practitioner knowledge (Honig &
Coburn, 2008). In their decision-making process, districts begin by searching for evidence from
this broad range of sources and ultimately incorporate it through a highly interactive process
that often involves many people. Consistent with the model put forth by Huberman (1983),
Honig & Coburn (2008) proposed seven factors thought to influence districts’ intentions to
adopt evidence-based programs, including: features of the evidence itself, working knowledge
(i.e., individuals’ beliefs, expectations, and preferences), social capital (i.e., formal and informal
ties with others), central office organization, normative influences, political dynamics, and
federal and state accountability (Honig & Coburn, 2008; Huberman, 1983). Normative influences
refer to district support and interest in the innovation, while working knowledge refers to the
interpretative process of how individuals interpret evidence as part of decision-making.
Researchers have found that district administrators pay greater attention to evidence that is in
16

line with their preexisting beliefs, which in turn affects their decision-making (Coburn, Toure, &
Yamashita, 2009).
School districts are responsible for reviewing the available evidence and deciding what
programs to adopt in their schools; yet, to date, there are relatively few empirical studies
assessing the role of evidence in districts’ decision-making (Coburn, et al., 2009). Based on this
complexity, there is a critical need for research to guide the translation of evidence-based
prevention programs into schools (Rohrbach, et al., 2006).
In response to this need, there have been several comprehensive reviews identifying factors
related to implementation of various types of health and social programs (Domitrovich &
Greenberg, 2000; Durlak & DuPre, 2008; Fixsen, et al., 2005; Greenhalgh, Robert, Macfarlane,
Bate, & Kyriakidou, 2004; Stith, et al., 2006). Although these reviews vary widely in scope and
purpose, several general themes emerge regarding the multi-level ecological nature of program
implementation. Consistent with systems theories, these reviews confirm the importance of
characteristics related to the community, organization, implementation system (e.g., provision
of training and technical assistance), provider, program and the audience targeted in the
process of translating evidence-based programs. In the following sections we will review the
research on factors affecting adoption and implementation of evidence-based programs in
schools. Overall, these findings lend support to an ecological approach to studying the
translation process in schools.
Community Context
With regard to the community context, district size has consistently been found to
influence adoption and implementation, with larger districts more likely to adopt and
implement evidence-based prevention programs (Cho, Hallfors, Iritani, & Hartman, 2009;
17

Ennett, et al., 2003; Ringwalt, et al., 2002; Rohrbach, et al., 2005). Because SDFS funding, the
principal source of federal substance use prevention funding for schools prior to 2010, was
distributed based on enrollment, larger school districts had more funds available to offset the
high costs associated with implementing tobacco-specific and other substance abuse prevention
programs compared to smaller school districts (Cho, et al., 2009; Rohrbach, et al., 2005). In the
past schools and school districts with sources of funding in addition to SDFS funds were more
likely to adopt and implement evidence-based prevention programs than those that relied on
SDFS alone (Cho, et al., 2009; Fagan, et al., 2008a; St. Pierre & Kaltreider, 2004; Thaker, et al.,
2008). However, with the current economic strain facing school districts, especially those in
California, it is unlikely that these institutions will be able to adopt evidence-based substance
use prevention programs without multiple sources of funding. Given the high costs associated
with adopting evidence-based tobacco use prevention curricula (Hallfors & Godette, 2002) it can
be assumed that school districts without competitive state funds will no longer be able to
provide substance use prevention and tobacco cessation programs.
Findings from the California TUPE evaluation found that high schools with competitive TUPE
grants were more likely than non-grantee high schools to provide tobacco use prevention and
cessation programs (McCarthy, et al., 2008). Yet, there were no significant differences between
competitive grantee and non-grantee high schools in regards to student enrollment, ethnicity,
poverty (% subsidized meals), academic performance and parent education (McCarthy, et al.,
2008). These findings imply that there are other organizational- and systems-level factors
influencing districts’ intentions to adopt evidence-based tobacco use prevention and cessation
programs.
18

Additionally, communities with a higher level of collaboration between schools and outside
constituency groups (e.g., parents, state level agencies, and holding public discussions) have
been found to be more likely to adopt and sustain evidence-based prevention programs (Blake,
et al., 2005; Gomez, Greenberg, & Feinberg, 2005; Rohrbach, et al., 2005). Researchers have
begun fostering community collaboration through the formation of community prevention
coalitions and community-university partnerships (Brown, Feinberg, & Greenberg, 2010; Riggs,
Nakawatase, & Pentz, 2008; Spoth, Greenberg, Bierman, & Redmond, 2004). Several
characteristics of community coalitions have been associated with increased implementation
and sustained use of evidence-based programs, including: higher levels of funding and shared
funding; articulated organizational structure; professional representation; leadership strength;
board efficiency; and strong internal and external relationships (Brown, et al., 2010; Feinberg,
Bontempo, & Greenberg, 2008; Gomez, et al., 2005; Jasuja, et al., 2005).
Other factors within the community context found to be associated with adoption and
implementation included the location, population density, poverty, and policies of the school.
Schools from the Midwest (Ringwalt, et al., 2002), public schools (Ringwalt, et al., 2003;
Ringwalt, et al., 2002), schools from urban and suburban areas (Rohrbach, et al., 2005), and
schools with more poverty (Ringwalt, et al., 2003) have been associated with greater adoption
and implementation of evidence-based substance use prevention programs. In addition,
adoption, implementation and sustainability have been greater among schools with a positive
external environment (e.g., stability outside of school, support, less opposition, mandates and
policies supporting prevention programs and less bureaucracy) (Gingiss, Roberts-Gray, & Boerm,
2006; Roberts-Gray, Gingiss, & Boerm, 2007; Scheirer, 2005).

19

Organizational Context
Several organizational level factors have been found to influence program adoption,
implementation and sustainability. Program coordinators are able to provide direction,
leadership and motivation to keep evidence-based prevention programs on track (Mihalic, et al.,
2008); Program coordinators are found at both the school and school district level, and are
responsible for organizing trainings, ensuring teachers have the resources and materials needed
for implementation, and identifying potential problems before they become major obstacles.
The amount of program coordinators’ effort (i.e., percentage of time devoted to prevention
program coordination) and having a program champion have repeatedly been found to
influence program adoption, implementation and sustainability across studies (Blake, et al.,
2005; Fagan, et al., 2008a; Fagan & Mihalic, 2003; Gingiss, et al., 2006; Mihalic, et al., 2008;
Roberts-Gray, et al., 2007; Rohrbach, et al., 2005; Scheirer, 2005).
The ‘climate’ of the workplace and its effect on employees and their quality of work has long
been the subject of examination by researchers. Organizational psychologists have
operationalized climate as the shared perception of the work environment (Glisson & James,
2002). Schools are seen as having positive climates when both vertical (principal—teacher,
teacher—student) and horizontal (teacher—teacher) relationships are open and supportive
(Gregory, et al., 2007). Positive district and school climate has been associated with program
adoption and implementation in a number of studies (Beets, et al., 2008; Ennett, et al., 2003;
Gittelsohn, et al., 2003; Kallestad & Olweus, 2003; Klimes-Dougan, et al., Oct 2009; Rohrbach, et
al., 2005). The school climate has the potential to influence staff behaviors, their willingness to
try programs and their ability to effectively deal with problems that arise during implementation
(Gittelsohn, et al., 2003). Therefore, teachers who work in a positive school climate with open
20

communication and collaboration among staff can use this resource to enhance their ability to
implement evidence-based prevention programs (Kallestad & Olweus, 2003). However, looking
only at the horizontal level, Kallestad and Ollweus found that greater teacher-to-teacher
collaboration was negatively related to program implementation (Kallestad & Olweus, 2003).
The authors speculate that a positive working relationship between teachers does not
necessarily promote proactive attitudes towards problem solving, and in fact can influence
teachers to avoid change.
Another aspect of school climate relates to the level of administrator support for the
prevention intervention. Administrator support has consistently been associated with more
successful program implementation and sustainability (Gingiss, et al., 2006; Gittelsohn, et al.,
2003; Mihalic, et al., 2008; Ringwalt, et al., 2003; Roberts-Gray, et al., 2007; Scheirer, 2005; St.
Pierre & Kaltreider, 2004; Thaker, et al., 2008). Principal support can increase teachers’ interest
in the prevention program through the provision of positive incentives for program
implementation, and thus intensify the care to which teachers implement the program
(Gregory, et al., 2007; Kam, Greenberg, & Walls, 2003; Ringwalt, et al., 2003). Additionally,
principals can be central in promoting positive attitudes towards the program among parents
and the community through parent orientation meetings and contacts with local officials. Lastly,
principals can foster a sense of collaboration among teachers and other staff to assist in the
implementation of the prevention program (Kam, et al., 2003).
Another organizational level factor that is consistently associated with successful program
implementation relates to teachers’ perceptions that they have enough time to plan and
implement the prevention program in addition to delivering the ‘core’ curriculum (Fagan,
Hanson, Hawkins, & Arthur, 2008b; Gingiss, et al., 2006; Hahn, Noland, Rayens, & Christie, 2002;
21

Mihalic, et al., 2008; Roberts-Gray, et al., 2007; Thaker, et al., 2008; Wiecha, et al., 2004). Many
teachers are facing considerable pressures to prepare students for standardized tests, and they
often view prevention curricula as ancillary to these tasks (Ringwalt, et al., 2003).
Teacher autonomy (Ringwalt, et al., 2003) and school staff turnover (Thaker, et al., 2008;
Wiecha, et al., 2004) have been found to be negatively related to program implementation.
Specifically, the more discretion teachers perceived themselves having, the less likely they are to
implement an evidence-based program with fidelity, or the more likely they are to make
adaptations. Current research suggests that when programs get implemented in real-world
situations, they are often adapted to meet the needs of the recipients, with implementers
deviating from the program as written (Pentz, et al., 1990; Ringwalt, et al., 2003; Rohrbach,
Graham, & Hansen, 1993). This is a major concern given the fact that many evidence-based
programs are ineffective when poorly implemented (Durlak & DuPre, 2008). Consequently,
assessing the quality of implementation fidelity has become the key to determining whether
non significant effects are due to program failure or inadequate program delivery (Dane &
Schneider, 1998; Gresham, 1989; Harachi, Abbott, Catalano, Haggerty, & Fleming, 1999).
Interestingly, despite the growing body of evidence suggesting the importance of contextual and
individual factors on the fidelity of implementation, there has been little examination of the role
of these factors influencing the relationship between the implementation fidelity and program
outcomes.
Currently, there is no shared definition of fidelity of implementation. Most researchers
conceptualize fidelity as the degree to which program providers implement programs as
intended by program developers (Dusenbury, Brannigan, Falco, & Hansen, 2003); however often
they measure different dimension of this process and refer to these dimensions using different
22

terminology. The most agreed-upon dimensions, proposed by Dane and Schneider (1998)
include adherence to the program, dose (amount of the program delivered), quality of program
delivery and participant acceptance.
Additionally, there are no standard measures for assessing program fidelity. Previous studies
comparing the efficacy of observer-reported implementation ratings compared with provider-
reported self-ratings found the former to significantly predict program related outcomes
whereas there was no association with the latter (Lillehoj, Griffin, & Spoth, 2004). These findings
highlight the potential of a social desirability bias in provider self-report ratings of program
implementation data, which suggests observer-reported implementation ratings may be more a
valid and reliable measure of program fidelity.
Implementation System
The main components of the implementation system include training and technical
assistance that program providers receive before and during program implementation (Durlak &
DuPre, 2008). In school-based prevention, training tends to be provided before the initial
implementation via face-to-face workshops. Typically, these workshops consist of motivating
teachers, modeling intervention components, communicating teaching responsibilities (e.g.,
importance of teaching with fidelity), and providing implementers with the necessary skills,
information and self-efficacy to effectively deliver the curriculum (Rohrbach, et al., 2006).
Several studies have found that the provision of training increases program implementation by
classroom teachers (Blake, et al., 2005; Ennett, et al., 2003; Fagan, et al., 2008a; Gingiss, et al.,
2006; Hallfors & Godette, 2002; Ringwalt, et al., 2003; Roberts-Gray, et al., 2007; Rohrbach,
Gunning, Sun, et al., 2010). Given the challenges associated with implementing interactive
teaching methods typical of prevention curricula, training may be essential in demonstrating
23

and reinforcing the use of these methods in the context of prevention programs (Ennett, et al.,
2003).
Technical assistance varies by study, but usually refers to resources offered to implementers
once program implementation has begun (Durlak & DuPre, 2008). Technical assistance can take
the form of ongoing formal or informal training, which may be delivered via face-to-face,
telephone and/or electronic contacts by program developers, program coordinators or trained
program staff ( Rohrbach, et al., 2006). Technical assistance has been associated with improved
program implementation (Fagan, et al., 2008a; Gingiss, et al., 2006; Roberts-Gray, et al., 2007;
Rohrbach, Gunning, Sun, et al., 2010).
Two recent studies have evaluated the incremental effects of ongoing training for teachers.
Rohrbach and colleagues (2010) tested whether a comprehensive training condition that
included a pre-implementation training workshop, technical assistance, coaching during
implementation, and web-based resources for teachers would produce higher levels of
implementation of an evidence-based high school drug prevention curriculum compared to the
pre-implementation training workshop alone. They found that the comprehensive training
condition produced significantly higher levels of implementation fidelity, but it did not result in
more positive short-term student outcomes (Rohrbach, Gunning, Sun, et al., 2010). Ringwalt et
al. (2009) investigated whether the addition of coaching to standard workshop training
increased the effectiveness of an evidence-based middle school drug prevention curriculum.
They found that a flexible approach, in which coaching was tailored to each teacher’s particular
needs and proficiency, had a positive effect on students’ cigarette use, but this finding could
have been a result of marked differences in cigarette use at baseline among the treatment and
control groups (Ringwalt, et al., 2009).There were no other effects of the coaching approach on
24

students’ alcohol use, marijuana use, or any of the several variables thought to mediate
curriculum effects, and only minimal effects on teacher fidelity.. Teachers who received
coaching were more likely than those in the noncoached group to teach each component of
each lesson and spend at least 30 minutes preparing for each lesson, however there were no
differences in the total number of lessons taught (Ringwalt, et al., 2009). The authors conclude
that the effects of coaching on teachers may not become evident in the initial implementation.
Teachers need to gain adequate familiarity with the curriculum before they can be expected to
produce significant changes in their students’ behavior.
Provider Characteristics
As a result of training workshops, program implementers often form beliefs regarding the
prevention program (Durlak & DuPre, 2008). When these beliefs—such as the perceived relative
effectiveness or importance of the program with their target population—are favorable, they
have been shown to increase program implementation and sustainability (Beets, et al., 2008;
Kallestad & Olweus, 2003; Klimes-Dougan, et al., Oct 2009; Ringwalt, et al., 2003; Scheirer,
2005). These beliefs have also been found to shape attitudes towards the program, and positive
attitudes towards the program have been associated with increased program implementation
(Beets, et al., 2008; Ringwalt, et al., 2003). Training can also influence other teacher
characteristics such as support for, and self-efficacy to implement the program. Several studies
have shown that higher self-efficacy is associated with increased program implementation
(Ennett, et al., 2003; Klimes-Dougan, et al., Oct 2009; Ringwalt, et al., 2003). One study reported
that teacher support for the evidence-based program was inversely associated with use of the
program’s interactive teaching techniques (Mihalic, et al., 2008). The authors hypothesized that
teachers may have been motivated to teach the program, but lacked the skills necessary to
25

implement the interactive teaching techniques required by the curriculum. This hypothesis is
consistent with a study by Ennett and colleagues, who found that comfort using interactive
teaching techniques was positively associated with program implementation (Ennett, et al.,
2003).
One study showed that female teachers and younger teachers were more likely to
implement evidence-based prevention programs (Ennett, et al., 2003); however, other studies
failed to find an association between teachers’ background characteristics and program
implementation (Kallestad & Olweus, 2003; McGraw, et al., 1996).
Program Characteristics and the Audiences Targeted
Rogers’ diffusion theory states that programs have perceived attributes that affect the
translation process (Rogers, 1983). Consistent with this theory, numerous studies have found
that characteristics of the program (e.g., relative advantage, compatibility, complexity,
observability and trialability) are associated with adoption and implementation of evidence-
based prevention programs in schools (Fagan & Mihalic, 2003; Mihalic, et al., 2008; Pankratz,
Hallfors, & Cho, 2002; Rohrbach, Gunning, Grana, Gunning, & Sussman, 2010; Rohrbach, et al.,
2005; Thaker, et al., 2008; Wiecha, et al., 2004). Another important characteristic affecting
adoption is program cost. More expensive evidence-based programs are less likely to get
implemented in schools (Gingiss, et al., 2006; Hahn, et al., 2002; Roberts-Gray, et al., 2007;
Rohrbach, et al., 2005).
Characteristics of the audiences targeted in the evidence-based prevention programs have
also been linked with program implementation. For instance, students’ behaviors and positive
reactions to the prevention programs have been associated with increased program
implementation (Fagan, et al., 2008b; Mihalic, et al., 2008; Ringwalt, et al., 2003). When
26

teachers see their students eliciting a positive response to the program they are less likely to
make adaptations.
In summary, our review of the literature supports the use of a multi-level ecological
approach to the study of adoption, implementation and sustainability of evidence-based
prevention programs in school settings. These findings lend support to the notion that if we are
to successfully translate prevention programs into ‘real world’ settings, each of these levels
needs to be addressed. In the present studies we explore the relationships between factors at
several levels of the ecological framework and program adoption, implementation fidelity and
sustained use of evidence-based substance use prevention and tobacco cessation programs in
schools.

27

CHAPTER 2. STUDY 1: SOCIO-ECOLOGICAL FACTORS RELATED TO USE OF EVIDENCE-BASED
SUBSTANCE USE PREVENTION PROGRAMS IN MIDDLE SCHOOLS BY SCHOOL DISTRICTS
NATIONWIDE

ABSTRACT
There is now considerable empirical evidence indicating that a number of school-based
programs are effective in preventing and reducing substance use among adolescents (Skara and
Sussman 2003); yet a recent study examining the utilization of these prevention programs by
school districts across the nation found that less than half reported using evidence-based
programs (Ringwalt, Vincus et al. 2009). The purpose of this research is to explore socio-
ecological factors that influence the use of evidence-based programs in schools.
Data used for this study comes from the School-based Substance Use Prevention Programs
Study (SSUPPS), which included a survey of school district administrators associated with a
national random sample of middle schools. The final analytic sample included N = 1,169 school
districts with data collected in both 1999 and 2005. Independent variables included community
characteristics, funding, organizational capacity, and beliefs about substance use prevention
programs. School districts were divided into four groups: 1) lack of uptake (no use of an
evidence-based program in 1999 or 2005); 2) adoption (late adopters) (no use of an evidence-
based program in 1999, but use of evidence-based programs in 2005); 3) sustained use (late
adopters) (use of an evidence-based program in 1999 and 2005); 4) terminated use (use of an
evidence-based program in 1999, but not in 2005). Next we combined the four groups into two
groups: those that did not use an evidence-based program in 2005 (lack of uptake and
terminated use), vs. those that did use an evidence-based programs in 2005 (adoption and
sustained use). Hierarchical logistic regression analyses were performed using the PROC SURVEY
28

feature in SAS to account for sampling weights. Next we compared early and late adopters of
evidence-based substance use prevention programs.
Results of the hierarchical regression revealed that funding, community-, organizational-,
and individual-level factors were related to adoption and sustained use of evidence-based
substance use prevention programs in schools. We found that early adopters were larger, were
more likely to be from the South, had more district coordinator devoted to substance use
prevention, had carried out more prevention planning actions, had more district interest in
substance use prevention programming, and had stronger beliefs that classroom curricula were
effective in reducing substance use (p’s < .10).
Findings support a multi-level ecological approach to the study of the use of evidence-based
programs in schools. Given the dearth of research on this topic, more research is needed to
improve our understanding of translation of evidence-based programs in schools. In particular,
future studies should further explore the relationship between organizational climate and use of
evidence-based programs.

29

INTRODUCTION
Over the past several decades, the school system has been the primary venue for delivering
substance use prevention programming to adolescents. Schools provide an easy avenue for
reaching a large number of adolescents in a controlled environment. As a result, there are now a
number of school-based prevention programs that have proven effectiveness in reducing youth
risk behaviors available for wide-scale dissemination, and various federal agencies have
developed lists of these ‘best practice,’ ‘research-tested,’ and ‘evidence-based’ prevention
programs (Fagan, Hanson, Hawkins, & Arthur, 2008; National Cancer Institute & Substance
Abuse and Mental Health Services Administration, 2010; United States Department of Health
and Human Services Substance Abuse, 2010). However, many school districts in the United
States fail to adopt, implement and sustain these programs, and the research exploring these
behaviors remains sparse. Consequently, a significant gap remains in what we know about how
to effectively “translate” evidence-based programs from research to practice (Rohrbach, Grana,
Sussman, & Valente, 2006).
Diffusion of innovations theory has been the predominant model for conceptualizing the
process of translating evidence-based substance use prevention programs in schools
(Schoenwald & Hoagwood, 2001). The stages of diffusion of evidence-based programs are
commonly modeled through a series of distinct linear stages, beginning with dissemination
(gaining knowledge about the program and forming beliefs), leading into adoption (deciding to
use the program) and implementation (taking action, i.e., putting the program into use), and
culminating with sustained use (continuing to use the program, also referred to as
“sustainability”) (Goodman & Steckler, 1989; O'Loughlin, Renaud, Richard, Gomez, & Paradis,
1998; Pluye, Potvin, & Denis, 2004; Shediac-Rizkallah & Bone, 1998). However, the majority of
30

translational research has focused on the earlier phases of diffusion (Brownson, et al., 2007;
Durlak & DuPre, 2008; Mihalic, Fagan, & Argamaso, 2008; Ringwalt, et al., 2003; Rohrbach,
Ringwalt, Ennett, & Vincus, 2005), while very little attention has been given to the long-term
viability of programs (Scheirer & Dearing, 2011). Yet there are clear arguments as to why
sustainability should be a priority for public health researchers and practitioners. Changing
health behaviors is often a slow process and terminating a program after it has been successfully
implemented is not only wasteful of the significant start-up costs in human, fiscal and technical
resources that go into developing and adopting a program, but it also has the potential to
increase recidivism in negative health outcomes (Shediac-Rizkallah & Bone, 1998).
Additionally, very little is known about the factors that influence early compared to late
adopters (Rogers, 2002) of evidence-based substance use prevention programs in schools. The
key assumptions of diffusion of innovations theory are that knowledge is a product,
generalizable across contexts, and effective research, and innovations generated by it, will
naturally be adopted by consumers (Estabrooks & Glasgow, 2006). However, a recent national
study found that only 46.9% of school districts were using an evidence-based substance use
prevention program and only 25.9% were using an evidence-based program more than other
(non-evidence-based) prevention programs (Ringwalt, et al., 2011). Instead of choosing
programs with proven effectiveness, often school districts use heavily marketed curricula that
have not been evaluated, have been evaluated insufficiently, or have been shown to be
unsuccessful in reducing problem behaviors (Hallfors & Godette, 2002; Rohrbach, D'Onofrio,
Backer, & Montgomery, 1996; Swisher, 2000; Tobler & Stratton, 1997). Likewise, many school
districts use ‘homegrown’ prevention curricula that they developed themselves (Hansen &
McNeal, 1997). This is despite the fact that several federal and state policies of the last decade
31

(e.g., Safe and Drug Free Schools Act of 1999 and the No Child Left Behind Act of 2001) have
mandated that schools receiving certain state and federal funding use programs with proven
effectiveness (Hallfors & Godette, 2002). However, school districts’ decisions regarding the use
of evidence-based substance use prevention programs are complicated by increasing demands
on schools to focus their efforts on core subjects (e.g., English, math and science) (Kaftarian,
Robertson, Compton, Davis, & Volkow, 2004), rising budget crises in states nationwide
(Dusenbury & Hansen, 2004) and recent changes to federal funding for substance use
prevention education. In 2010, the Safe and Drug Free Schools and Communities (SDFSC) grants
program, the main source of funding for substance use prevention education in schools, was
eliminated from the federal budget (Hallfors & Godette, 2002). Given these challenges, the use
of evidence-based substance use prevention programs is unlikely to increase unless
considerable efforts are made to disseminate information about programs with proven
effectiveness (Ringwalt, et al., 2002).
To date, the diffusion of innovations model (Rogers, 1983) has focused primarily on
individual adopters of innovations, failing to consider the organizational- and system-level
contexts in which adoption decisions are made. As a result, we still do not fully understand the
process by which evidence-based interventions are translated from research to practice. In an
attempt to better understand the process of diffusion, some researchers have begun to
challenge the existing paradigm of program diffusion. It has been suggested that program
diffusion is not a set of distinct linear phases, but rather a dynamic process that involves the
interplay of actions made by the individual decision-makers, embedded within organizations
that operate within a community context (Scheirer & Dearing, 2011). This is in line with
“systems thinking” or “systems models” which posit that knowledge integration is contextual
32

and tied to organizational priorities and culture, and relationships must be understood from a
multilevel systems perspective (Estabrooks & Glasgow, 2006). Applying a systems model
perspective to school-based research we must take into account the multiple systems (e.g.,
state agencies, school districts, school administrators, teachers, unions, school boards, etc.)
affecting decision-making within schools. In line with systems thinking, Chen’s conceptual model
(1998) purports that program implementation is influenced by the organizational context (e.g.,
administrator support, school climate), characteristics of the implementer, and the
implementation system (e.g., staff training and infrastructure to coordinate implementation).
Recently there has been a push to study factors related to the overall process of diffusion
(Brown & Flynn, 2002; Pluye, et al., 2004; Swisher, 2000). Most of the research on translation
has examined these stages of diffusion separately and as a result, researchers have paid little
attention to the contextual factors that might be influencing change in every stage of the
process. In a study of five community health centers in Canada, Pluye et al. (2005) found that
factors associated with sustained use were concomitant to those associated with
implementation (Pluye, Potvin, Denis, Pelletier, & Mannoni, 2005). These findings support the
idea that factors facilitating earlier stages of the diffusion process may also be predictive of
longer-term sustainability. By understanding the factors that are most influential in multiple
stages of the diffusion process, researchers can improve program utilization in schools and
ultimately reduce substance use among adolescents.
Theory driven evaluations (Chen, 1998), the diffusion of innovations model (Rogers, 1983),
and systems models (Estabrooks & Glasgow, 2006), have guided recent efforts to explore
characteristics of the community and organization and beliefs about the program at the
individual provider level that are associated with the use of research-validated prevention
33

programs in schools (Durlak & DuPre, 2008; Little & Rohrbach, Manuscript submitted for
publication; Scheirer, 2005; Scheirer & Dearing, 2011; Shediac-Rizkallah & Bone, 1998).
One of the most consistent community contextual factors leading to program use is district
size. Several studies have shown that larger districts are more likely to adopt and implement
evidence-based prevention programs (Cho, Hallfors, Iritani, & Hartman, 2009; Ennett, et al.,
2003; Ringwalt, et al., 2002; Rohrbach, et al., 2005). In addition, schools that have support from
community groups are more likely to adopt evidence-based prevention programs (Blake, et al.,
2005; Rohrbach, et al., 2005). The availability of external funding has also been associated with
adoption, implementation and sustained use of evidence-based programs in schools (Cho, et al.,
2009; Fagan, et al., 2008; Shediac-Rizkallah & Bone, 1998; St. Pierre & Kaltreider, 2004; Thaker,
et al., 2008).
Various characteristics of the organization have been found to influence use of evidence-
based substance use prevention programs in schools. Positive climate has been associated with
program use because of its ability to influence staff behaviors and motivation to try new
programs (Beets, et al., 2008; Ennett, et al., 2003; Gittelsohn, et al., 2003; Kallestad & Olweus,
2003; Klimes-Dougan, et al., Oct 2009; Rohrbach, et al., 2005). Other factors within the
organization, such as overall district support for, and interest in substance use prevention can
influence districts’ use of a program (Rohrbach, et al., 2005). Additionally, the proportion of a
school or district coordinator’s time that is devoted to substance use prevention has been
associated with prevention program adoption and implementation (Blake, et al., 2005; Fagan,
et al., 2008; Fagan & Mihalic, 2003; Gingiss, Roberts-Gray, & Boerm, 2006; Mihalic, et al., 2008;
Roberts-Gray, Gingiss, & Boerm, 2007; Rohrbach, et al., 2005).
34

District decisions to use evidence-based prevention programs are also affected by features
of the program itself. Districts are more likely to use programs that are compatible with their
needs and have a perceived relative advantage over existing programs and practices (Pankratz,
Hallfors, & Cho, 2002; Rohrbach, Gunning, Grana, Gunning, & Sussman, 2010; Rohrbach, et al.,
2005). Additionally, when evaluating new programs, district administrators tend to pay greater
attention to evidence that is in line with their preexisting beliefs, which in turn affects their
decision-making (Coburn, Toure, & Yamashita, 2009).
The Current Study
Although there is now a growing body of literature on factors related to the translation of
evidence-based prevention programs in schools, there are number of limitations to the existing
research. There is a dearth of research exploring the relative importance of characteristics of the
community, organization, and individual decision-makers predicting program adoption and
sustained use (Beets, et al., 2008; Payne & Eckert, 2010). Additionally, we are not aware of any
studies that have compared factors associated with being an early versus late adopter of
evidence-based substance use prevention programs in schools, nor are we aware of any studies
that have compared predictors of program utilization in schools across multiple stages of the
diffusion process. The current study addresses these limitations by utilizing data from the
School-based Substance Use Prevention Programs Study (SSUPPS), a longitudinal national survey
of substance use prevention practices in middle schools. Previous SSUPPS studies have
examined the prevalence of districts’ use of evidence-based prevention programs (Ringwalt, et
al., 2008; Ringwalt, et al., 2011; Ringwalt, et al., 2009; Ringwalt, et al., 2002), as well as
correlates of adoption and fidelity of implementation of evidence-based prevention programs in
cross-sectional samples (Ringwalt, et al., 2003; Rohrbach, et al., 2005). In the present study, we
35

build on this work by comparing characteristics of early and late adopters of evidence-based
substance use prevention programs in schools in a longitudinal sample of school districts.
Furthermore, we utilize hierarchical regression analyses to examine the relative impact of
community characteristics, funding, organizational capacity, and beliefs about substance use
prevention programs on districts’ decisions to adopt and sustain evidence-based substance use
prevention programs in schools. By applying a theory-based analytic strategy, hierarchical
regression (Victora, Huttly, Fuchs, & Olinto, 1997), that is guided by the diffusion of innovations
model (Rogers, 2002), theory driven evaluations (Chen, 1998), systems models (Estabrooks &
Glasgow, 2006) and recent reviews of the literature (Durlak & DuPre, 2008) we aim to better
understand decisions to adopt and sustain the use of evidence-based substance use prevention
programs in schools.
METHODS
Study Sample and Data Collection
Data used for this study comes from the School-based Substance Use Prevention Programs
Study (SSUPPS), which included a survey of a national random sample of schools with middle
school grades, as well as a survey of school district administrators associated with those schools
(Ringwalt, et al., 2002). The sample of schools was drawn in two phases, the first of which
employed a 1997-1998 sampling frame from Quality Education Data, Inc. (Quality Education
Data Inc., 1998) of all regular schools in the 50 states and the District of Columbia that included
middle school grades. Schools eligible for the sample included Grades 7 or 8, or were limited to
Grade 6 or to Grades 5 and 6. Excluded from the sample were schools that enrolled fewer than
20 students, schools that were non-regular, and schools that were special education, vocational,
36

or other/alternative schools. The sampling frame yielded 2,273 eligible public schools (Ringwalt,
et al., 2002).
Applying the same criteria to a 2002-2003 sampling frame drawn from the Common Core of Data
(National Center for Education Statistics, 2004) as we did in 1997-1998, we added 210 schools to our
sampling frame. Because we conducted the first round of data collection in the 1998-1999 school year,
we added the 210 schools to our second round of data collection so that we could account for new
schools that had opened between 1999 and 2003. Given that schools vary widely across the country, we
stratified our sample, with equal probability within each stratum, to ensure adequate representation of
schools along three key characteristics: population density, school size, and poverty level. We chose to
stratify our sample to reduce sampling error and therefore increase the precision of our estimates.
Because of the possibility of error on the sampling frame, we contacted sampled schools (2,483)
between October 2004 and January 2005 to confirm their eligibility status, a process that yielded 2,204
verified eligible schools with middle schools grades. We found that 279 schools were ineligible either
because of their grade span, school type, school size, or because the school had closed. In 2005, we
screened all respondents to ensure that someone in the school was teaching drug use prevention. Less
than 2% of our sampled schools reported not teaching any drug prevention lessons, and they were not
asked any further questions. Once the sample of schools was selected, the associated sample of
districts was identified. Given that the probability of selection of each of these districts is known, the
sample of districts can be used to generate valid estimates for districts across the nation that include
schools with middle school grades.
The focus of the present study is on predicting use of evidence-based prevention programs
in middle schools between 1999 and 2005; thus, the analytic sample was restricted to include
only those districts that had data at both waves (N = 1,209 of 2,204). Respondents that
37

reported receiving a waiver from the requirements of the Office of Safe and Drug-Free Schools
(OSDFS) to implement evidence-based prevention programs were also excluded (N = 40 of
1,209). The final analytic sample included N = 1,169 school districts with data from both 1999
and 2005 (53% of original sample (2,204); 97% of 1,209 sample).
Data used in these analyses were collected in 1999 and 2005. In 1999, we collected data
from February through September exclusively by mailing a survey to the person considered to
be the most knowledgeable about drug use prevention in the selected school. We identified that
person in advance of data collection via phone calls to the school or its district. We included a
prepaid incentive of $10 in our first mailing and provided a letter of support from the Director of
the Safe and Drug-Free Schools Program in our third mailing of the survey. Our data collection
efforts yielded an overall response rate of 80.3%. More details about our approach in 1999 can
be found elsewhere (Ringwalt, et al., 2002).
We made three changes to our data collection approach for 2005: we limited our sample to
public schools only; expanded the modes of data collection to include Web, paper, and phone;
and curtailed data collection in July. In October 2004, we initiated a calling effort in advance of
our second round of data collection which occurred during spring 2005. The purpose of our calls
was to establish schools’ eligibility and to identify the appropriate respondent for each school
district. We directed our invitations to participate in the study to the person who had been
identified. We made no effort to return to our initial district-level respondent.
Measurement
Some of the items included in the district coordinator survey were adapted from previous
studies (e.g., (Battistich, Solomon, Kim, Watson, & Schaps, 1995; Rohrbach, Dent, Johnson,
Unger, & Gunning, 1998; Steckler, Goodman, McLeroy, Davis, & Koch, 1992) and others were
38

developed specifically for the study. Variables used in the present study came from the SSUPPS
1999 and 2005 waves of district data and the Common Core of Data file, a national dataset
containing district demographic characteristics (Thomas, Sable, Dalton, & Sietsema, 2007).
Dependent Variable
On both the 1999 and 2005 surveys, respondents were asked to check which substance use
prevention curriculum they were using in middle schools in their district from a list of both
evidence-based and non-evidence-based curricula. Programs were defined as evidence-based if
they were identified as such in at least one of four reviews of school-based substance use
prevention curricula (Center for Substance Abuse and Prevention, 2001; Centers for Disease
Control and Prevention, 1994; Drug Strategies, 1999; National Institute on Drug Abuse &
National Institutes of Health, 1997; Safe and Drug Free Schools Program, 2001). In each of these
reviews, criteria for program review had been established, requiring that programs have
evidence of program effects based on outcome evaluations and the evidence be reviewed by a
panel of experts in the field of substance use prevention (Ringwalt, et al., 2002).
For data analyses, responses were coded such that districts were given a ‘1’ if they reported
using at least one of the evidence-based programs cited in these reviews; otherwise they
received a ‘0’. Thus, we created an estimate of the use of evidence-based substance use
prevention programs for both the 1999 and 2005 waves of data. School districts were then
divided into four groups: 1) lack of uptake (no use of an EBP in 1999 or 2005); 2) uptake of
use/late adopters (no use of an EBP in 1999, but use of EBP in 2005); 3) sustained use/early
adopters (use of an EBP in 1999 and 2005); 4) terminated use (use of an EBP in 1999, but not in
2005).

39

Independent Variables
Measures of community characteristics included geographic region of the U.S. in which the
school is located, district size (number of schools in the district; 1 = 1 school, 2 = 2-3 schools, 3 =
4-6 schools, 4 = 7-11 schools, 5 = 11-18 schools, 6 = >18 schools), district poverty (% eligible for
free or reduced-price lunch) and ethnic composition of students. These variables were taken
from the Common Core of Data file (Thomas, et al., 2007). Region was classified according to US
Census regions (Northeast, Midwest, South and West) (United States Census Bureau, 2000).
Funding for substance use prevention was assessed by summing six items that asked about
sources of substance use prevention funding other than SDFS funds (e.g., federal funds, state
funds, school district funds, local government funds, community funds, etc.) that the district
received during the 2004-2005 school year. Community support for substance use prevention
education was measured through 5 items that focused on support for substance use prevention
education from different groups within the community (e.g., teachers, parents, students,
community groups, and local police), which were averaged to create an index (4-point scales; 1
= not at all to 4 = very supportive; α = 0.79). Organizational capacity to implement the program
was assessed through several items, including the district coordinator’s time spent on substance
abuse prevention (1 = 0-4 hours to 6 = 40 hours or more), the presence of a program champion
(0=no to 1=yes), and district interest in substance use prevention education programming which
was assessed by averaging 2 items (4-point scales; 1=not at all true to 4=very true; r=0.68).
Organizational climate was assessed through one index, perception of a positive district climate,
with 10 items (e.g., “In this school district, there is a feeling that everyone is working together
toward common goals”) that were averaged (5-point scales; 1=strongly disagree to 5=strongly
agree; α=0.90). District support for substance use prevention education was measured through
40

10 items (e.g., “The key decision makers in our district believe that preventing substance use
among youth is very important”) that were averaged to create an index (4-point scales; 1=not at
all to 4=very supportive; 5-point scales; 1=strongly disagree to 5=strongly agree; α=0.89).
Prevention planning actions consisted of six items related to OSDFS requirements (e.g.,
“Administered a student survey about substance use”) (0=no to 1=yes; α = 0.69). Problems
implementing the curriculum was assessed by summing 14 items (e.g., “Our district lacks the
budget to pay for this curriculum,” 0=no to 1=yes).
Features of the program were assessed through two measures, perceived effectiveness of
classroom strategies in reducing substance use among middle school students in their district (4-
point scale; 1=not effective at all to 4=very effective), and seventeen items that measured
perceived positive attributes of the curriculum (e.g., “More effective”) (4-point scales; 1=not at
all to 4=a great deal; α=0.93).
DATA ANALYSIS
Since data for this study came from a national probability sample of schools, weights were
applied to the analyses based on original selection probabilities. Weights adjusted estimates for
district poverty, number of schools, and population density (National Center for Education
Statistics 2006a). We computed weighted means or point estimates for continuous or
categorical variables, respectively, and the associated 95% confidence interval for the entire
sample.
First, we combined the school districts into two groups: those that did not use an evidence-
based program in 2005, vs. those that did use an evidence-based program in 2005 (early and
late adopters). We employed hierarchical logistic regression analyses (Victora, et al., 1997) using
the PROC SURVEY feature in SAS to account for sampling weights, to examine the complex
41

hierarchical inter-relationships between community characteristics, funding, organizational
capacity, and beliefs about substance use prevention programs, with the dependent variable
use of an evidence-based substance use prevention program in 2005. To avoid excessive
parameters, variables not reaching p < .10 were dropped from subsequent analyses. All analyses
controlled for use of an evidence-based program in 1999. Variables were standardized mean = 0
and standard deviation = 1. Odds ratios and 95% confidence intervals were reported using two
tailed significance tests.
Next we weighted means or point estimates and the associated 95% confidence interval
among early versus late adopters of evidence-based substance use prevention programs. Early
adopters were defined as those districts that reported using an evidence-based program in 1999
and 2005 (N=431). Late adopters were defined as districts that were not using an evidence-
based program in 1999, but were using on in 2005 (N=198). To compute differences between
dichotomous variables among the two groups of adopters, we used Rao-Scott Chi-Square Test.
To test for group differences between continuous variables, we ran a series of weighted linear
regression models with each of our variables of interest as the dependent variable and the
group (early vs. later adopters) as the independent variable. Significance was computed using
two tailed tests. Analyses were conducted using SAS (v.9.1.3) statistical package (SAS Institute
Inc. SAS/C Online Doc TM, 2000).
RESULTS
District Characteristics
Table 2.1 presents weighted means or point estimates and the associated 95% confidence
interval on the variables of interest for the entire sample. In 2005, 17% of districts were late
adopters (no use of an EBP in 1999, but use of EBP in 2005; N=198), 37% districts were early
42

adopters (use of an EBP in 1999 and 2005; N=431). Of the remaining sample, 23% of districts
reported never using an evidence-based program (no use of an EBP in 1999 or 2005; N=270),
and 23% districts reported terminating the use of an evidence-base program (use of an EBP in
1999, but not in 2005; N=270).
Table 2.1. Characteristics of school districts
1
(% or μ (95% CI))
1

Characteristics % or μ (95% CI)
2
Community Factors
District size
3
3.65 (3.56, 3.75)
District poverty (%) 38.48 (37.14, 39.82)
Proportion of White students (%) 70.74 (69.03, 72.46)
Region (%)
Northeast 13.54 (11.58, 15.50)
Midwest 33.24 (30.51, 35.98)
South 33.00 (30.25, 35.75)
West 20.21 (17.87, 22.56)
Community support for SUP
4
3.35 (3.32, 3.38)

Funding
Number of SUP funding sources in addition to SDFS funds 1.81 (1.71, 1.90)

Organizational Capacity
Positive district climate 3.79 (3.75, 3.83)
District support for SUP 3.57 (3.54, 3.60)
District coordinator’s effort devoted to SUP 2.12 (2.02, 2.22)
Program champion 49.72 (46.81, 52.63)
Prevention planning actions 0.79 (0.78, 0.81)
District interest in SUP programming 3.36 (3.32, 3.39)
Barriers to implementation 1.40 (1.28, 1.52)

Features of the Program
Perceived effectiveness of classroom curricula in reducing
substance use
2.96 (2.92, 2.99)
Perceived positive attributes of the program 3.42 (3.39, 3.46)
Notes:
1
N=1,169;
2
All percentages or means reported, and associated confidence intervals, are weighted.
3
Scale is 1 = 1 school, 2 = 2-3 schools, 3 = 4-6 schools, 4 = 7-11 schools, 5 = 11-18 schools, 6 = >18 schools;
4
SUP=substance abuse prevention

43

Predictors of Overall Use
As indicated in the Data Analysis section, funding, community characteristics, organizational
factors, and beliefs about substance use prevention programs were entered into a series of
hierarchical logistic regression models as predictors of use of an evidence-based substance use
prevention program (see Table 2.2). Model 1 included the four community characteristics:
district size, district poverty, proportion of White students in the district and community support
for substance use prevention. Among these variables, being a larger district (p< .05) and having
less community support for substance use prevention were predictive of use of a prevention
program (p < .10). Model 2 included district size and community support for substance use
prevention, and added receiving sources of substance use prevention funding in addition to
SDFS funds. Receiving additional sources of substance use prevention funding was significantly
associated with use (p < .05). Model 3 included district size, community support for substance
use prevention, and receiving sources of substance use prevention funding in addition to SDFS
funds, and added positive district climate, district support for substance use prevention, district
coordinator effort devoted to tobacco use prevention, having a program champion, carrying out
prevention planning actions, district interest in substance use prevention programming and
barriers to implementation. Among the organizational characteristics, having a less positive
district climate, more district coordinator effort devoted to tobacco use prevention, having a
program champion, and carrying out prevention planning actions were predictive of using a
substance use prevention program (p’s < .05). The final model, Model 4, included district size,
community support for substance use prevention, receiving sources of substance use prevention
funding in addition to SDFS funds, having a positive district climate, district coordinator effort
devoted to tobacco use prevention, having a program champion, and carrying out prevention
44

planning actions and added beliefs about the effectiveness of classroom curricula in reducing
substance use among adolescents and the perceived positive attributes of the program. Among
the beliefs items, perceiving positive attributes of the program was predictive of use of an
evidence-based program (p’s < .05).

45

Table 2.2. Summary of hierarchical regression analysis for variables predicting the use of
evidence-based substance use prevention programs at Wave 2

Notes: All analyses controlled for use of an evidence-based prevention program in 1999 (Wave 1).
Significant predictors at p < .10 were kept in subsequent models. * p < .05; + p < .10

Characteristics OR, 95% CI
Model 1
District size 1.77 (1.53, 2.05)*
District poverty 1.00 (0.87, 1.15)
Proportion of White students 1.02 (0.87, 1.19)
Community support for SUP 0.88 (0.78, 1.00)
+

Model 2
District size 1.72 (1.47, 2.00)*
Community support for SUP 0.86 (0.74, 0.99)*
Number of SUP funding sources in addition to SDFS funds 1.40 (1.20, 1.62)*
Model 3
District size 1.24 (1.00, 1.53)
+

Community support for SUP 0.92 (0.71, 1.19)
Number of SUP funding sources in addition to SDFS funds 1.17 (0.95, 1.44)
Positive district climate 0.71 (0.57, 0.90)*
District support for SUP 0.92 (0.71, 1.19)
District coordinator’s effort devoted to SUP 1.18 (1.01, 1.37)*
Program champion 1.36 (1.11, 1.68)*
Prevention planning actions 1.46 (1.20, 1.79)*
District interest in SUP programming 1.09 (0.88, 1.35)
Barriers to implementation 0.97 (0.80, 1.18)
Model 4
District size 1.22 (0.98, 1.51)
+

Community support for SUP 0.83 (0.66, 1.04)
Number of SUP funding sources in addition to SDFS funds 1.18 (0.96, 1.46)
Positive district climate 0.72 (0.58, 0.91)*
District coordinator’s effort devoted to SUP 1.17 (1.00, 1.36)*
Program champion 1.31 (1.05, 1.64)*
Prevention planning actions 1.48 (1.19, 1.83)*
Perceived effectiveness of classroom curricula in reducing
substance use
1.08 (0.88, 1.34)
Perceived positive attributes of the program 1.30 (1.06, 1.60)*
46

Comparison of Early Versus Late Adopters
Table 2.3 shows the weighted means or point estimates and the associated 95% confidence
interval for early versus late adopters of evidence-based substance use prevention programs.
Districts that were early adopters were larger, were more likely to be from the South, had more
district coordinator devoted to substance use prevention, had carried out more prevention
planning actions, had more district interest in substance use prevention (SUP) programming,
and had stronger beliefs that classroom curricula were effective in reducing substance use (p’s <
.10).

47

Table 2.3. Comparison of early and late adopters of evidence-based programs

Characteristics
% or μ (95% CI)
1
Early Adopters
(N = 431)
Late Adopters
(N = 198)
Community Factors
District size 4.27 (4.11, 4.43) 3.64 (3.42, 3.87)*
District poverty 38.55 (36.31, 40.78) 40.26 (37.04, 43.48)
Proportion of White students 66.24 (36.28, 69.20) 70.38 (66.20, 74.56)
Region
Northeast 12.18 (9.03, 15.33) 13.61 (8.86, 18.36)
Midwest 25.54 (21.35, 29.72) 29.37(22.95, 35.78)
South 41.52(36.76, 46.28) 33.05 (26.36, 39.75)+
West 20.77(16.88, 24.65) 23.97(17.86, 30.08)
Community support for SUP
2
3.34 (3.31, 3.39) 3.30 (3.23, 3.37)
Funding
Number of SUP funding sources in addition to
SDFS funds
1.97 (1.81, 2.13) 2.14 (1.92, 2.36)
Organizational Capacity
Positive district climate 3.70 (3.64, 3.76) 3.71 (3.62, 3.81)
District support for SUP 3.51 (3.46, 3.56) 3.53 (3.46, 3.61)
District coordinator’s effort devoted to SUP 2.60 (3.46, 3.56) 2.22 (1.97, 2.46)*
Program champion 71.16 (66.79, 75.52) 70.50 (64.02, 76.97)
Prevention planning actions 0.89 (0.87, 0.91) 0.83 (0.79, 0.87)*
District interest in SUP programming 3.43 (3.37, 3.49) 3.34 (3.26, 3.42)+
Barriers to implementation 3.53 (3.19, 3.88) 3.24 (2.75, 3.72)
Features of the Program
Perceived effectiveness of classroom
curricula in reducing substance use
3.04 (3.00, 3.09) 2.92 (2.84, 3.00)*
Perceived positive attributes of the program 3.49 (3.45, 3.54) 3.45 (3.39, 3.51)
Notes:
1
All percentages or means reported, and associated confidence intervals, are weighted.
2
SUP =
substance use prevention; To test for differences between dichotomous items, we used Rao-Scott Chi-
Square Test. To test for group differences between continuous variables, we ran a series of weighted
linear regression models with each of our variables of interest as the dependent variable and the group
(adopters vs. sustainers) as the predictor variable. * p < .05; + p < .10

48

DISCUSSION

Due to the universal nature of schooling, schools have come to play a vital role in promoting
well-being among adolescence and reducing disease risk (Greenberg, 2010). As a result, there
are now a number of effective school-based prevention programs for reducing risky behaviors.
Unfortunately, the science related to the translation of these programs into the ‘real world’ lags
far behind the science related to their development. If prevention researchers are going to make
a substantial impact on the prevalence of substance use among youth, we need to understand
the factors that lead school districts to adopt, implement and sustain programs with proven
effectiveness. A deeper understanding of these processes will guide interventions and inform
policies, which will ultimately lead to increased use of evidence-based prevention programs in
schools and reductions in substance use among adolescents.
In the present study, we compared characteristics of early and late adopters of evidence-
based substance use prevention programs in schools in a longitudinal national random sample
of school districts. Although there has been a push over the last decade to disseminate
information about evidence-based substance use prevention programs, these programs have
not been adopted by a large percentage of districts. We found that early adopter districts were
larger, were more likely to be from the South, had more district coordinator devoted to
substance use prevention, had carried out more prevention planning actions, had more district
interest in substance use prevention programming, and had stronger beliefs that classroom
curricula were effective in reducing substance use. An understanding of the modifiable factors
related to program adoption can inform interventions aimed at increasing adoption of evidence-
based prevention programs in schools.
49

We utilized hierarchical regression analyses informed by the diffusion of innovations model
(Rogers, 2002), theory driven evaluations (Chen, 1998), systems models (Estabrooks & Glasgow,
2006) and recent reviews of the literature (Durlak & DuPre, 2008) to examine complex
hierarchical inter-relationships between community factors, organizational factors and
individual beliefs about the program and districts’ use of evidence-based substance use
prevention programs in schools. Interestingly, roughly half of the districts surveyed reported not
using an evidence-based substance use prevention program in 2005, with 23% reporting
terminating the use of such a program. These findings are surprising in light of the fact that
considerable resources were spent during these years by both state and federal agencies to
increase the use of evidence-based programs in the nation’s schools. Additionally, there data
were collected before the recent changes to SDFS funding, which eliminated this funding
mechanism for school districts. Given these changes, it is likely that even fewer school districts
in the coming years will be able to afford to carry out substance use prevention programming.
With this in mind, it is critical for the field to understand what factors in addition to funding lead
school districts to adopt and sustain evidence-based substance use prevention programs.
As expected, we found that larger districts were more likely to use evidence-based
substance use prevention programs. Prior to the changes in SDFS funding, there was a direct
association between district size and substance use prevention funding in schools. SDFS funds
were distributed based on enrollment, which provided larger districts more funding to cover the
high costs associated with using evidence-based prevention curricula (Hallfors & Godette, 2002).
Additionally, the No Child Left Behind Act of 2001 allowed districts to transfer a percentage of
their substance use prevention funds to meet locally determined educational needs (Ringwalt,
et al., 2009). Small and rural districts were allowed to transfer all of their SDFS funds as they saw
50

fit, which could have led them to transfer away all of the necessary funding for carrying out
substance use prevention programming. Not surprising, we found that having additional sources
of substance use prevention funding available was significantly associated with use of evidence-
based programs.
Consistent with previous research (Blake, et al., 2005; Fagan, et al., 2008; Fagan & Mihalic,
2003; Gingiss, et al., 2006; Mihalic, et al., 2008; Roberts-Gray, et al., 2007; Rohrbach, et al.,
2005) and Chen’s conceptual model (1998), we found that factors related to the district’s
capacity to deliver prevention programs were directly related to their use of evidence-based
prevention programs. Previous studies have found that the ability of district coordinators to
monitor the program adoption and implementation stages (e.g., provide training and necessary
resources) is critical to the successful implementation of prevention programs (Fagan & Mihalic,
2003). Given our findings, it appears that district coordinators are also critical to the sustained
use of evidence-based programs and should be targeted in interventions aimed at promoting
sustainability of prevention programs in schools.
Surprisingly, we found that perceptions of a positive district climate negatively predicted use
of an evidence-based prevention program. In addition, community support for substance use
prevention was negatively associated with the use of evidence-based prevention program.
These findings are inconsistent with previous research (Beets, et al., 2008; Ennett, et al., 2003;
Gittelsohn, et al., 2003; Kallestad & Olweus, 2003; Klimes-Dougan, et al., Oct 2009; Rohrbach, et
al., 2005). To further understand these associations, we ran logistic regression models predicting
use of a homegrown program and found that districts reporting positive perceptions of district
climate were 1.29 times as likely (C.I. 1.11, 1.51) to use a homegrown program (p < .05). Based
on our findings, one could speculate that districts with open and supportive climates, embedded
51

within communities that support substance use prevention education, are more willing to
develop a program to combat substance use rather than adopt a program with proven
effectiveness. Unfortunately, these programs have never been tested, and therefore run the risk
of being ineffective and wasteful of the time and money spent developing and implementing
them. These findings lend support for community-based partnerships between school
personnel, representatives of community agencies, and providers of technical assistance to
build capacity for the use evidence-based programs in schools (Spoth, Greenberg, Bierman, &
Redmond, 2004). Recent evaluations have established support for the effectiveness of these
partnerships in producing high levels of adoption and implementation of evidence-based
prevention programs (Hawkins, et al., 2008; Spoth, et al., 2007). These strategies may be an
important tool for increasing the utilization of evidence-based interventions in schools where
there is a high level of community and district support for substance use prevention and an
open and support climate.
Consistent with the diffusion of innovations theory (Rogers, 2002), we found that perceiving
positive attributes of the program was predictive of program use (Mihalic, et al., 2008; Thaker,
et al., 2008). Factors such as the curricula layout and the number of sessions are important
determinants of whether a school district will adopt and sustain the use of an evidence-based
prevention program. Therefore, researchers should be mindful of program characteristics and
their perceived compatibility with district needs when developing prevention programs.
Limitations
A limitation to the current study was our assessment of district climate. First, we assessed
district climate from a single individual within the district. Because climate refers to the shared
perception of the work environment (Charles Glisson & James, 2002), we were not truly
52

assessing district climate. Rather, we were measuring a proxy of climate, psychological climate
or the perceptions of the district climate, from a single respondent. Because of the large number
of districts involved in the current study, it was not feasible to survey all employees of each
district. However, future studies should explore the association between district climate as
assessed from all employees of a district and program use. Secondly, our measure of climate
may not have captured the entire range of climate. For instance, Glisson’s Organizational Social
Context (OSC) scale assesses climate through eight first-order scales and three second-order
scales (Charles Glisson, et al., 2008). Due to the large scope of the study, assessing climate
through more than 10 items was not feasible. However, future studies should explore the
associations of district climate, assessed through a more comprehensive scale, with use of
evidence-based prevention programs.
Another limitation of the current study pertains to the individual respondent within each
district. Attempts were not made to survey the same respondent from each district in both
waves of data due to feasibility constraints. However, the respondents did hold the same
position in the school district (i.e., the Safe and Drug Free Schools Coordinator) in both waves of
data collection.
Conclusion
The current study is critical to improving our understanding of translation, because despite
the empirical evidence demonstrating the effectiveness of a number of school-based prevention
programs in promoting adolescent health (L. A. Rohrbach, et al., 2006), the literature on factors
influencing the use of these programs remains sparse. Our findings suggest that characteristics
influencing districts’ decisions to adopt evidence-based prevention programs also influence their
sustained use of these programs, thus suggesting that one examine multiple processes of
53

diffusion together. This is the first study we are aware of to compare early and late adopters of
evidence-based substance use prevention programs in schools. Additionally, this is the first
study we are know of that utilizes a theory-driven analytic approach, hierarchical regression, in a
longitudinal design to explore the role of funding and community, organizational, and individual-
level factors in predicting the use of evidence-based prevention programs in a nationally
representative sample of school districts. Our findings attest to the use of an ecological
approach for the study of use of evidence-based prevention programs in schools. Future studies
should build on our findings by intervening on these contextual factors in order to increase the
probability that programs with proven effectiveness be utilized by schools, which will ultimately
lead to reductions in negative health outcomes among adolescents.

54

CHAPTER 3. STUDY 2: COMMUNITY AND ORGANIZATIONAL FACTORS RELATED TO THE
FIDELITY OF IMPLEMENTATION OF PROJECT TND IN HIGH SCHOOLS ACROSS THE NATION

ABSTRACT
In the present study, we examine relationships between program provider and contextual
factors and implementation fidelity of an evidence-based substance use prevention curriculum,
Project Towards No Drug Abuse (TND), in a sample of high school teachers from 65 high schools
in 8 states. We also examine associations between implementation fidelity and program
outcomes, and the effects of provider, school and community factors on these relationships. We
found that implementation fidelity in schools is influenced by a variety of factors related to the
provider, school and community, and implementation fidelity is positively associated with
program outcomes after controlling for these contextual factors. The present study is one of the
few to examine the relative contribution of contextual and individual factors on implementation
fidelity and student outcomes using multilevel modeling techniques. We discuss the implications
of the study for future work in the field of translation of prevention programs with proven
effectiveness from research settings to schools.

55

INTRODUCTION
Considerable advances have been made in the field of school-based substance abuse
prevention in the last quarter century. There are now a number of successful evidence-based
programs that have been shown to reduce or prevent substance abuse among adolescents
which are ready for wide-scale dissemination (Denise C. Gottfredson & Wilson, 2003; Skara &
Sussman, 2003; Tobler, et al., 2000; Tobler & Stratton, 1997). Adoption of these programs by
schools has been aided by federal and state policies of the last decade (e.g., Safe and Drug Free
Schools Act of 1999 and the No Child Left Behind Act of 2001) which have mandated that
schools receiving certain government funding use programs with proven effectiveness (Hallfors
& Godette, 2002). As a result, nearly half of school districts nationwide surveyed in a recent
study of the use of evidence-based substance use prevention curricula in middle schools
reported implementing an evidence-based prevention program with their students (Ringwalt, et
al., 2011). However, as a field we know surprisingly little about what happens when evidence-
based programs are implemented in schools under real-world conditions. Most of what we do
know about the implementation fidelity, or the extent to which implementation of the program
corresponds to the originally intended program, takes place in the context of small scale efficacy
and effectiveness trials (Lillehoj, Griffin, & Spoth, 2004; Pentz, et al., 1990; Rohrbach, Graham, &
Hansen, 1993; Rohrbach, Gunning, Grana, & Sussman, 2007). Furthermore, there is a dearth of
research on the relationship between implementation fidelity and program outcomes in large
scale dissemination trials, where implementation fidelity would be assumed to be closer to what
occurs in the real-world (Rohrbach, et al., 2007). Such research is essential to improving our
understanding of the ideal conditions necessary for effective program utilization (Kam,
Greenberg, & Walls, 2003).
56

Current research suggests that when programs get implemented in real-world situations,
they are often adapted to meet the needs of the recipients, with implementers deviating from
the program as written (Pentz, et al., 1990; Ringwalt, et al., 2003; Rohrbach, et al., 1993). This is
a major concern given that many evidence-based programs are ineffective when poorly
implemented (Durlak & DuPre, 2008). Consequently, assessing implementation fidelity has
become key to determining whether non-significant effects are due to program failure or
inadequate program delivery (Dane & Schneider, 1998; Gresham, 1989; Harachi, Abbott,
Catalano, Haggerty, & Fleming, 1999).
Schools offer enormous opportunity to reach large numbers of adolescents in well
controlled environments; however unique contextual factors embedded within the school
structure are likely to influence the fidelity of implementation of prevention programs (Chen,
1998; Domitrovich, et al., 2008). Schools are often overburdened meeting academic and policy-
related priorities. Furthermore, implementing prevention programs in schools often requires
approval and buy-in from multiple levels of decision-makers including superintendents,
principals and teachers, as well as school boards and community partners (Greenberg, 2010). An
integral yet often overlooked part of implementation research is the influence of these types of
contextual factors on the fidelity of prevention program implementation in schools. In Chen’s
conceptual model (1998) program implementation is influenced by three levels of factors, the
organizational context (e.g., administrator support, school climate), characteristics of the
implementer, and the implementation system (e.g., staff training and infrastructure to
coordinate implementation). In the diffusion of innovations theory, beliefs about the program
(e.g., whether it is consistent with existing values and beliefs) are thought to influence whether
it is adopted (Rogers, 1983). Therefore, one can assume that holding favorable beliefs about a
57

program will influence the attitudes formed by program providers, and ultimately, the quality of
program delivery (Ajzen, 1991).
Recently, there has been a push to understand the ideal context necessary to produce
successful program implementation in schools. As a result, a growing body of literature has
emerged that is guided by Chen’s conceptual model (1998), the diffusion of innovations theory
(1983), and an ecological perspective on the translation process (Domitrovich, et al., 2008;
Durlak & DuPre, 2008; Fixsen, Naoom, Blase, Friedman, & Wallace, 2005; Little & Rohrbach,
Manuscript submitted for publication). At the core of program implementation in schools are
the teachers who administer the programs. The quality of implementation has been found to
increase when teachers are comfortable with the program and delivery method, have self-
efficacy to implement the program, greater teaching skills and fewer years of teaching
experience (Ringwalt, et al., 2002; Rohrbach, et al., 1993). Organizational characteristics are also
central to understanding implementation fidelity, because administrators, teachers and
students are embedded within this shared environment (Domitrovich, et al., 2008).
Organizational factors that have been associated with high-quality implementation include
aspects of the school climate, such as capacity for change, openness to change, and positive
communication between teachers and administrators (Beets, et al., 2008; Ennett, et al., 2003;
Gittelsohn, et al., 2003; Gottfredson & Gottfredson, 2002; Kallestad & Olweus, 2003; Klimes-
Dougan, et al., Oct 2009; Rohrbach, Ringwalt, Ennett, & Vincus, 2005). Additionally, when a
program aligns with a school’s policy, it is more likely to be implemented with quality (Kallestad
& Olweus, 2003; Payne, Gottfredson, & Gottfredson, 2006). Factors related to the structure of
the school and surrounding community, such as school size and urbanicity, have also been
58

associated with prevention program implementation, such that larger schools in more urban
areas demonstrate higher levels of program use (Payne, 2009; Payne, et al., 2006).
Interestingly, despite the growing body of evidence suggesting the importance of contextual
and provider factors in program implementation, there has been little examination of the role of
these factors in understanding the relationship between implementation fidelity and program
outcomes. These factors could be mediators or moderators of the relationship between
implementation fidelity and program outcomes. In their effectiveness trial of a school-based
curriculum, Kam et al. (2003) found that their program was only effective in schools with both
high principal support and high fidelity, suggesting that principal support is a moderator of the
relationship between implementation fidelity and program outcomes. Unfortunately, the
current study was not powered to detect mediated and moderated effects. However, in order to
assess ‘true’ relationships between implementation fidelity and program outcomes, it is
important to at least control for contextual factors.
The Current Study
The present study examines the influence of contextual and provider-level factors on the
implementation fidelity and outcomes of Project Towards No Drug Abuse (TND), an evidence-
based substance abuse prevention program for high school students (Sussman, McCuller, &
Dent, 2003). The program has been evaluated in seven randomized trials, which have
demonstrated the impact of the program on 30-day substance use at a 1-year follow-up or
longer (Dent, Sussman, & Stacy, 2001; Rohrbach, Sun, & Sussman, 2010; Sun, Sussman, Dent, &
Rohrbach, 2008; Sussman, Dent, & Stacy, 2002; Sussman, Dent, Stacy, & Craig, 1998; S.
Sussman, et al., 2003; Sussman, Sun, Rohrbach, & Spruijt-Metz, 2011; Valente, Okamoto,
Pumpuang, Okamoto, & Sussman, 2007), (Sun, Skara, Sun, Dent, & Sussman, 2006). The present
59

study utilizes data from the TND Dissemination Trial (Rohrbach, Gunning, Sun, & Sussman,
2010), in which the program was implemented by high school teachers in 10 school districts
from around the country. We hypothesized that a number of contextual and provider-level
factors would influence implementation fidelity of Project TND. We also hypothesized that
fidelity of implementation would be positively associated with program outcomes after
accounting for significant contextual and provider-level factors.
METHODS
School Selection and Experimental Design
Data for this study come from the Project Towards No Drug Abuse (TND) Dissemination
Trial, which compared the relative effectiveness of two approaches to training regular high
school teachers to implement Project TND, a standard training workshop and a comprehensive
training and implementation support model (Rohrbach, et al., 2010). A total of 65 high schools
from 10 school districts across the United States were recruited for the study (Rohrbach, et al.,
2010). The sample was derived from a pool of school districts that had requested information
about purchasing Project TND, but had not yet adopted the curriculum. Districts were
approached if they had at least 3 schools that could be randomized to the experimental
conditions. Within each school district, participating schools were randomly assigned to one of
three experimental conditions: comprehensive implementation support for Project TND
teachers (IMP-SUPPORT), regular workshop training only for TND teachers (REGULAR), or
standard care control (CTRL). The current study focuses on the 43 schools in the IMP-SUPPORT
or REGULAR (program) conditions.
Within high schools in the two program conditions, at least one teacher was recruited to
participate in the training intervention and deliver Project TND to his/her students. The school
60

designated the subject area for program implementation (health or physical education). Delivery
of the program took place in existing class groupings of students at all schools in the two
program conditions. Teachers completed two surveys: prior to implementation (baseline) and
immediately following implementation (immediate posttest). School administrators completed
one survey immediately following implementation. Students completed surveys at baseline, one
to two-weeks following implementation (immediate posttest), and one year following
implementation. A more detailed account of the school selection and experimental design can
be found in Rohrbach et al. (2010, 2010a).
Subjects
Students were eligible to participate in the study if they were enrolled in a randomly
selected class taught by one of the health or physical education teachers participating in the
study. All students provided informed assent and parents provided written or verbal informed
consent to participate in the study. Of the 4,351 students enrolled in the 43 schools assigned to
the two program conditions, 3,751 were consented for participation in the study (86.2% of
students in the selected classes). Of the 3,751 consented students, 3,346 took the pretest survey
(89.2%). Of the 3,346 students who took the pretest survey, n=2,983 completed the immediate
posttest (89.2% retention rate) and n= 2,563 completed the one-year follow-up (76.6%
retention rate). The student sample was 46.7% male; 40.0% white, 28.2% Hispanic, 15.6%
African American, 3.2% Asian, 7.2% mixed ethnicity, and 5.8% other.
In addition to students, study subjects included 46 teachers who completed a survey at
baseline and immediate follow-up, whose program delivery was observed at least once
(described below), and whose school administrator completed a survey. Of these teachers 33
were in the IMP-SUPPORT training condition and 27 were in the REGULAR training condition.
61

Characteristics of the teachers and school administrators are presented in Table 3.1. There were
no significant differences between the two program conditions in the demographic
characteristics of teachers, administrators, and students.
Table 3.1. Characteristics of teachers and school administrators

µ (std) or %
Teacher Characteristics (N = 60)
Teaching experience (in years) (µ) 15.2(11.3)
Age (µ) 36.9(16.9)
Female (%) 55.9
Ethnicity (%)
White, non-Hispanic 79.0
Latino 7.0
African American 12.3
Asian 1.8
Health teacher (%) 45.2
Certified to teach health (%) 79.0
Masters degree (%) 52.6
School Administrator Characteristics (N = 41)
Experience as principal (in years) (µ) 9.2(8.6)
Age (µ) 49.2(8.7)
Female (%) 20.0
Ethnicity (%)
White, non-Hispanic 65.9
Latino 7.3
African American 17.1
Asian or Pacific Islander 2.4
American Indian or Alaskan Native 2.4
Other 2.4
Data Collection and Measurement
Many of the teacher and administrator items included in the TND Dissemination trial were
adapted from previous studies (e.g., (Battistich, Solomon, Kim, Watson, & Schaps, 1995; Ennett,
et al., 2003; Rohrbach, Dent, Johnson, Unger, & Gunning, 1998; Rohrbach, et al., 1993; Steckler,
Goodman, McLeroy, Davis, & Koch, 1992) and others were developed specifically for the study.
62

Items included on the student surveys were from previous studies of Project TND (Sussman, et
al., 2002).
Teacher Self-Report Measures
Teachers completed self-report, closed-ended response surveys prior to and immediately
following program implementation. Following the training workshop, teachers were given a
baseline survey that assessed their background characteristics, teaching style and perceptions of
the school climate. Background characteristics included gender, age, ethnicity, overall teaching
experience and experience with drug education (in years), degrees obtained (masters degree or
less), certification to teach health (yes or no), and subject taught (physical education (PE) or
health). Other measures included the extent to which their teaching style was disciplinary (2
items that were averaged; each used a 4-point scale, 1=definitely not me to 4=definitely me; α=
.77), perceptions of positive aspects of the work climate at their school (e.g., “Teachers in this
district feel free to communicate with district administrators.") (6 items that were averaged;
each on 5-point scales, 1=strongly disagree to 5=strongly agree; α= .83), and perceptions of the
extent to which their school administration expected them to implement Project TND (1 item; 7-
point scale; 1=not at all to 7=a great deal).
Immediately following program implementation, teachers were asked to complete an
immediate posttest survey that assessed their overall impressions of the curriculum and the
likelihood of them teaching it in the future. Comfort with the TND approach was measured at
immediate posttest through 7 items that were averaged (e.g., “Overall, how comfortable are
you with the overall approach of the TND curriculum?") (each on 7-point scales; 1=not at all to
7=a great deal; α=.73).

63

Assessment of Implementation Fidelity
Fidelity of program implementation was assessed with a classroom observation procedure
that was used in previous TND trials (Rohrbach, et al., 2007). Initially, the goal was to observe
each teacher in both program conditions twice, while he/she delivered the same TND lesson
(#5) to two separate class groupings. However, for 7 of the 46 teachers, observation was
possible during only one classroom period; thus, analyses of implementation fidelity data are
based on a total of 99 classroom observations. The lesson that was observed (#5) utilized
psychodrama techniques to simulate a talk show in which various negative consequences of
drug abuse were presented. This lesson was selected because it is highly interactive and the
classroom observation procedure emphasizes assessment of program process. Observations
were conducted by trained members of the Project TND staff, and observers were not blind to
the experimental condition of the school. The observation instrument assessed three domains
of implementation fidelity: classroom process, perceived student acceptance, and quality of
delivery. All of the items utilized seven-point rating scales that specified behaviorally anchored
criteria for the end- and mid-points of the scales (e.g., Mowbray, Holter, Teague, & Bybee,
2003). Classroom process (3 items) assessed how well the lesson went overall, the extent to
which the teacher elicited student participation and responses, and whether the objectives of
the lesson were met (7-point scales; 1=not at all to 7=a great deal). These items were averaged
to create an index (α=0.88). The quality of delivery index is the mean of 4 items that assessed
teacher enthusiasm and confidence and the extent to which the teacher treated students
respectfully (7-point scales; 1=not at all to 7=a great deal; α=0.92). The perceived student
acceptance index is comprised of 3 items (averaged) that measured how interested the students
appeared to be, how much they seemed to like the teacher, and class control (7-point scales;
64

1=not at all to 7=a great deal; α=0.93). Because these three indexes were highly inter-correlated
(r=0.95), for data analyses they were averaged and standardized (mean=0; std=1) to create a
composite implementation fidelity score (α=0.96). For data analyses, implementation fidelity
scores were dichotomized into high fidelity and low fidelity groups on the basis of a median split
(Rohrbach, et al., 1993).
School Administrator Assessment
School Administrators (principals) completed a self-report, closed-ended response survey
following initial program implementation. Measures included administrators’ perceptions of a
clear mandate from the district to implement substance use prevention education (1 item; 5-
point scale; 1=strongly disagree to 5=strongly agree), the districts’ encouragement to use an
evidence-based substance abuse program (1 item; 5-point scale; 1=strongly disagree to
5=strongly agree), the school’s openness to change (1 item; 5-point scale; 1=strongly disagree to
5=strongly agree) and a collaborative environment at their school (3 items, averaged; 5-point
scales, 1=strongly disagree to 5=strongly agree; α=.81). The organizational capacity index was
the mean of four items, staff turnover (1= high to 3=low), experience as a principal (in years),
the presence of specific teachers within the school who were responsible for substance use
prevention education (yes/no) and whether teachers received the resources they need to
implement substance use prevention curricula (yes/no) (α=.56).
Student Surveys
Baseline and immediate posttest measures were collected from students using paper-and-
pencil, closed-ended response surveys. The surveys were administered solely by project staff at
single classroom sessions during regular school hours. Those absent from the classroom on
testing days were left absentee packets, and local school staff were asked to distribute the
65

questionnaires to the appropriate students and mail the completed questionnaires to the
research staff.
The immediate posttest assessed seven potential program mediators of Project TND that
have been associated with substance use in previous studies (Rohrbach, Gunning, Sun, &
Sussman, 2010; Rohrbach, et al., 2010; Rohrbach, Sussman, & Dent, 2005; Sussman & Dent,
1996; Sussman, et al., 2003). Program-specific knowledge was measured with 22 items designed
to assess the extent to which students learned the content of the curriculum. Items were scored
as correct or incorrect, summed, and converted to a percentage correct score for analysis.
Seven items comprised the pro-drug-use myths index which assessed cognitive misperceptions
about physical, emotional, and social aspects of drug use (α =0.70; see (Sussman, Dent, & Stacy,
1996)). For data analyses, a score was calculated representing the percentage of pro-drug myths
with which the subject agreed. Four items that assessed beliefs about the immorality of drug use
(e.g., “How wrong is it to use drugs?”) were averaged to create an index (1 = not wrong at all to
4 = very wrong; α =0.92). Health-as-a-value is an index that averages three items about the
importance of health as a life value (1=not at all important to 4 = very important; α = 0.73; see
(Lau, Hartman, & Ware, 1986)). Three items adapted from Wills (1986) comprised the negative
coping strategies that assessed how often subjects used each of three disengagement coping
strategies when they had a problem at school or home during the previous year (e.g., “I party
with other youth;” 1= never to 5 = almost always; α =0.71). Substance use intentions were
assessed with three items, including how likely the student was to use cigarettes, alcohol, and
marijuana, respectively, in the next 12 months (1 = definitely not to 5 = very likely).

66

Additional School Context Measures
School demographic characteristics, including percentage of white students, population
density (recoded into three categories: urban, suburban and rural) and student poverty
(percentage of students falling below the federal government poverty) were taken from the
national Common Core Data file (Thomas, Sable, Dalton, & Sietsema, 2007). The urbanicity index
was created by averaging these four school demographic characteristics (α = .85).
DATA ANALYSIS
All analyses were conducted at the classroom level (N = 99) using generalized mixed-linear
modeling (Murray & Hannan, 1990). Prior to conducting regression models, variables were
standardized (mean = 0, std = 1) and a class-level mean was created for each item. Random
effects adjusted for in the models included: school district, school and teacher. All analyses
controlled for the experimental condition of the school. Betas and standard errors are reported.
All analyses were conducted using SAS statistical package (SAS Institute Inc. SAS/C Online Doc
TM, 2000). We used two-tailed tests with significance set at p < .05.
To explore our first question of what contextual and individual provider-level factors were
related to fidelity of implementation, the composite fidelity score was regressed on each of the
correlates independently as a fixed effect variable at the class level. Variables that showed a
statistically significant effect on fidelity of implementation in the bivariate analyses at p < .10
were included in the final multivariate model.
To examine our second question of whether implementation fidelity was related to program
outcomes controlling for contextual and individual-level factors, each of the student outcome
variables was regressed against implementation fidelity as a fixed effects variable. Included in
the models were those contextual and individual-level factors that were significant correlates of
67

implementation fidelity (at p <.10) in the previous multivariate analysis. In the second set of
models, fixed effects variables included: the pretest score for the specific dependent variable;
student ethnicity, age and gender; experimental condition; urbanicity; teachers’ comfort with
TND approach; teacher’s perceived positive school climate; and teacher’s perceived
expectations to implement Project TND.
RESULTS
Correlates of Fidelity of Implementation
The results of the mixed linear regression models exploring relationships between
implementation fidelity and contextual and individual-level factors are shown in Table 3.2. In the
bivariate models, urbanicity (p < .10), teachers’ perceptions of a positive school climate and
perceived expectations to implement Project TND (p’s < .05) were negatively associated with
fidelity of implementation. Teachers’ comfort with the Project TND approach and the
organizational capacity of the school were positively associated with fidelity (p’s < .10). In the
multivariate model, comfort with the Project TND approach remained a positive correlate of
implementation fidelity (p < .05), and teachers’ positive perceptions of the school climate and
perceived expectations to implement Project (p’s < .05) were retained as negative correlates of
fidelity.

68

Table 3.2. Correlates of fidelity of TND program implementation

Correlates
Bivariate Multivariate
Beta
(std error)
p-value Beta
(std error)
p-value

School District Context
Clear mandate to implement TND
1
0.13 (0.14) 0.36 -- --
Encouraged use of evidence-based program
1
0.12 (0.12) 0.32 -- --
Urbanicity -0.25 (0.13) 0.07 -0.16 (0.13) 0.19
Student baseline drug use
2
-0.07 (0.12) 0.58

School Context
School’s openness to change
1
0.20 (0.13) 0.13 -- --
Collaborative school environment
1
0.08 (0.13) 0.55 -- --
Organizational capacity
1
0.21 (0.12) 0.09 -0.03 (0.13) 0.80

Teacher Characteristics and Beliefs
Female
3
0.14 (0.11) 0.22 -- --
Prior experience with drug education
3
-0.11 (0.12) 0.36 -- --
Health teacher
3
0.09 (0.12) 0.49 -- --
Disciplinary teaching style
3
-0.09 (0.12) 0.46 -- --
Comfort with TND approach
3
0.20 (0.11) 0.08 0.24 (0.11) 0.03
Perceived positive school climate
3
-0.34 (0.11) 0.003 -0.26 (0.12) 0.03
Perceived expectations to implement TND
3
-0.31 (0.11) 0.01 -0.27 (0.12) 0.03
Notes: Variables are standardized (mean = 0 std =1). All models were at the classroom level, N=99.
Random effects included school district, school and teacher; fixed effects included program condition. P-
values reported using 2-tailed tests;
1
Assessed on school administrator survey;
2
Assessed on student
baseline survey;
3
Assessed on teacher baseline or posttest survey

Relationships between Fidelity of Implementation and Student Outcomes
Table 3.3 shows the results of the regression models that examined whether
implementation fidelity predicted changes in program outcomes, controlling for contextual and
individual-level factors. Implementation fidelity predicted a significant increase in students’
beliefs about the immorality of drug use and health-as-a-value (p’s < .05), and it was marginally
associated with an increase in program-specific knowledge (p = .05). Additionally,
implementation fidelity was associated with a reduction in students’ beliefs about pro-drug use
69

myths and students’ marijuana use intentions (p’s < .05). There was a marginally significant
effect of implementation fidelity on reductions in cigarette use intentions (p < .10).
Table 3.3. Relationships between implementation fidelity and student outcomes

Student Outcomes
Beta
(std error)
P-value
Program-Specific Knowledge
b
0.13 (0.07) 0.05

Beliefs
b

Pro-drug-use myths -0.23 (0.09) 0.01
Immorality of drug use 0.19 (0.06) 0.01
Health-as-a-value 0.36 (0.10) 0.001

Skills
b

Negative coping strategies -0.08 (0.09) 0.40

Drug Use Intentions
b

Alcohol -0.09 (0.06) 0.11
Cigarettes -0.10 (0.05) 0.08
Marijuana -0.20 (0.08) 0.01
Notes: Variables are standardized (mean = 0, std = 1); All models were at the classroom level, N=99.
Random effects included school district, school and teacher; fixed effects included program condition;
pretest value on dependent variable; student ethnicity, age and gender; experimental condition;
urbanicity; teacher comfort with TND approach; teacher’s perceived positive school climate; and teacher’s
perceived expectations to implement TND. P-values reported using 2-tailed tests;
b
: Assessed on the
immediate follow-up survey;

DISCUSSION
Although there are now a number of successful evidence-based substance use prevention
programs available for wide-scale dissemination (Gottfredson & Wilson, 2003; Skara & Sussman,
2003; Tobler, et al., 2000; Tobler & Stratton, 1997), there is a dearth of research understanding
the factors that promote successful implementation in schools. Researchers have only recently
begun to examine contextual and individual-level factors that promote the fidelity of
implementation of school-based prevention programs (Chen, 1998; Domitrovich, et al., 2008). In
order to reduce adolescent substance use, we need a better understanding of the ideal
70

conditions necessary for quality implementation of effective school-based substance use
prevention programs (Kam, et al., 2003).
Guided by several conceptual models (e.g., Chen, 1998; Domitrovich, et al., 2008; Durlak &
DuPre, 2008; Fixen, et al., 2005; Rogers, 1983), the current study examined whether contextual
and individual-level factors influenced the implementation fidelity of an evidence-based
substance abuse prevention program in a sample of high schools from around the country. We
also explored relationships between implementation fidelity and program outcomes while
controlling for contextual and individual-level factors. We found that several contextual and
individual-level factors were associated with implementation fidelity and fidelity predicted
positive changes in student outcomes after controlling for the contextual and individual-level
factors.
Despite previous research supporting a relationship between teacher characteristics such as
teaching experience and style, and implementation fidelity (Ringwalt, et al., 2002; Rohrbach, et
al., 1993), we did not find support for these relationships. We found that teachers’ attitudes and
beliefs were associated with fidelity, but teachers’ demographic characteristics, experience, and
teaching style were not related. Consistent with the diffusion of innovations theory, we found
that teachers who were more comfortable with the evidence-based (Project TND) approach
were more likely to implement the program with fidelity. This finding is consistent with previous
research (Beets, et al., 2008; Ringwalt, et al., 2003), suggesting that teachers that have more
confidence with the overall prevention approach will put more effort into implementing the
program with fidelity. Surprisingly, positive beliefs regarding the school climate and perceived
expectations to implement Project TND were negatively related to fidelity of implementation.
These findings are contrary to what we expected based on previous research (Beets, et al., 2008;
71

Ennett, et al., 2003; Gittelsohn, et al., 2003; Kallestad & Olweus, 2003; Klimes-Dougan, et al., Oct
2009; Rohrbach, et al., 2005). One would expect to observe stronger fidelity stemming from a
school environment that promotes open communication, shared decision-making, and openness
to change. Additionally, strong support for prevention programs from school administrators has
been consistently associated with successful implementation (Gingiss, Roberts-Gray, & Boerm,
2006; Gittelsohn, et al., 2003; Mihalic, Fagan, & Argamaso, 2008; Ringwalt, et al., 2003; Roberts-
Gray, Gingiss, & Boerm, 2007; St. Pierre & Kaltreider, 2004; Thaker, et al., 2008). Interestingly,
we did not find significant associations between administrators’ perceptions of school climate
(e.g., collaborative school environment and openness to change) and fidelity of implementation.
It is possible that these findings were spurious or reflect the influence of something else at play
in the school environment that we did not assess.
Consistent with previous research (Fagan, Hanson, Hawkins, & Arthur, 2008; Mihalic, et al.,
2008; Thaker, et al., 2008; Wiecha, et al., 2004) and Chen’s conceptual model (1998), we found
that organizational capacity was related to higher levels of implementation fidelity in the
bivariate models. These findings suggest that implementation quality will be higher in schools
where specific teachers are given the necessary resources and responsibility to teach substance
abuse prevention programs. Another contextual variable that we assessed, urbanicity, was
negatively associated with implementation fidelity. Future studies should investigate possible
reasons why implementation fidelity of evidence-based programs might be lower in more urban
environments. For example, teachers in urban schools may be more likely to adapt programs to
better fit the needs of their students. Some researchers have argued that adaptations are
essential for addressing the individual needs of program recipients (Dusenbury, Brannigan,
72

Falco, & Hansen, 2003); however, more work is needed to understand the types of adaptations
being made and the effects of these adaptations on program outcomes.
We found that after controlling for significant contextual and individual-level correlates of
fidelity, implementation fidelity predicted positive changes in one-half of the student outcomes
we assessed. Our findings confirm the importance of assessing implementation fidelity when
determining the effectiveness of prevention interventions in schools. There is a growing
consensus in the field that monitoring program implementation is critical for determining the
internal and external validity of interventions (Durlak & DuPre, 2008). We support this
suggestion and further suggest that researchers assess and consider the context surrounding
program implementation in schools in order to truly understand program effectiveness. This is
the first study we are aware of that controlled for contextual and individual-level factors when
examining the relationship between implementation fidelity of an evidence-based program and
program outcomes in schools. These factors could be mediators or moderators of the
relationship between implementation fidelity and program outcomes; however, the current
study was not powered to detect mediated and moderated effects. Future studies should
examine interrelationships among contextual factors, implementation fidelity and program
outcomes
Limitations
One of the strengths of the current study is the use of classroom observations to assess
implementation fidelity. Observations of implementation fidelity by trained observers are
considered the most objective way to measure program implementation (Dane & Schneider,
1998; Dusenbury, et al., 2003). A limitation to the current study is that we have not conducted
studies that identify specific mediators of Project TND. However, the immediate outcome
73

variables included in the current study are hypothesized program mediators, and have been
shown to predict substance use in previous studies (Rohrbach, et al., 2005; Sussman & Dent,
1996; Sussman, et al., 2003). An additional limitation of the study relates to our assessment of
organizational climate. Because we did not assess climate from all members of the school, we
did not truly assess climate. Rather, we obtained a proxy measure of climate, psychological
climate, or an individual’s perceptions of the school climate. Because of the large number of
schools involved in the current study, it was not feasible to survey all employees of each school
to obtain an estimate of school climate. However, future studies should explore the association
between school climate as assessed from all employees, and fidelity of implementation.
Conclusion
School-based prevention is influenced by a variety of contextual factors occurring at
multiple ecological levels. In order to move the field ahead, future effectiveness and
dissemination studies need to account for the complex nature of schools in analyses of program
effects. This is one of the few studies to examine contextual and individual-level correlates of
implementation fidelity, as well as control for these factors while using multi-level modeling
techniques to examine the effect of implementation fidelity on program outcomes. In order to
reduce substance use in adolescents, we must continue to conduct research that increases
understanding of how to get prevention programs with proven effectiveness successfully
implemented in schools.

74

CHAPTER 4. STUDY 3: STUDY OF ADOPTION AND USE OF RESEARCH-VALIDATED TOBACCO USE
PREVENTION AND CESSATION PROGRAMS BY DISTRICTS THROUGHOUT CALIFORNIA

ABSTRACT
Over the past several decades, the school system has been the primary venue for delivering
tobacco use prevention programming to adolescents. As a result, there are now a number of
research-validated programs available for wide-scale dissemination in schools. Unfortunately, in
2010 the main source of funding for school-based substance use prevention programming, the
Safe and Drug Free Schools and Communities (SDFSC) state grants program, was eliminated
from the federal budget. In addition, recent changes to the California Health and Safety Code
provisions eliminated the annual entitlements for school-based tobacco education funds (TUPE)
in California and established a competitive grant process. Given these recent changes, it is
essential for researchers to understand what factors lead school districts to use research-
validated tobacco use prevention and cessation programs in this new climate.
Data used for this study comes from a cross-sectional survey of 235 California school district
administrators and county office of education TUPE coordinators surveyed during 2011 via a
web-based survey. We used a theory-driven approach to examine the relative influence of
community-, organizational- and individual-level factors on the adoption and implementation of
evidence-based tobacco prevention and cessation programs in California schools. We found that
funding, community-, organizational-, and individual-level factors were related to adoption and
use of research-validated tobacco prevention and approved tobacco cessation programs.
Findings support a multi-level ecological approach to the study of the use of research-validated
tobacco prevention and cessation programs in schools. These results should be used to inform
policies that affect school districts’ use of research-validated tobacco prevention and cessation
75

programming, which will ultimately lead to reductions in negative health outcomes among
adolescents.

76

INTRODUCTION
Although rates of lifetime smoking have continued to decline among eighth, tenth and
twelfth graders in California since 1995, the prevalence of current (30-day) tobacco use has
increased in recent years (McCarthy, et al., 2008). The prevalence of current smoking is also
increasing with subsequent age groups, such that between sixth and twelfth grade the
prevalence of smoking increases from 2.6% to 19.7% (McCarthy, et al., 2008). These findings are
concerning given that one out of five high school students who experiments with tobacco
becomes a regular user, despite their general acknowledgement that long-term tobacco use has
detrimental health and social consequences (Johnston, O’Malley, Bachman, & Schulenberg,
2009). In addition, quit rates among youth are low (Stanton, Lowe, & Gillespie, 1996). In a recent
survey of California youth, 46.6% of lifetime smokers and 46.7% of current smokers had made at
least one unsuccessful quit attempt (McCarthy, et al., 2008). These findings highlight the need
for more tobacco control measures targeting adolescents in California.
An important tobacco control approach with adolescents is the use of school-based
prevention and cessation programs. There is now substantial empirical evidence demonstrating
long term positive effects of school-based prevention programs in reducing adolescent smoking
(Flay, 2009), and a number of school-based prevention programs have been made available for
wide-scale dissemination. Unfortunately, in 2010 the main source of funding for school-based
tobacco and other substance use prevention programming, the Safe and Drug Free Schools and
Communities (SDFSC) grants program, was eliminated from the federal budget. In addition,
recent changes to the California Health and Safety Code provisions have removed the annual
entitlements for tobacco use education in schools and replaced them with competitive funding.
77

The California Tobacco Use Prevention Education (TUPE) fund was established in 1989, after
the passage of a statewide referendum (Proposition 99) that increased the tax on tobacco
products (Rohrbach, et al., 2002). The TUPE funds, administered by the California Department of
Education, are designated to provide school-based tobacco-specific student instruction,
reinforcement activities, special events, and intervention and cessation programs for students.
Prior to 2009, school districts in California were given an annual entitlement to spend on
tobacco education based on student enrollment in grades four through eight. However, recent
changes to California law eliminated theses entitlements and established a competitive grant
process for TUPE funds for programming in grades 4-12. Given these policy changes, it is
essential for researchers to identify the factors that lead school districts to use research-
validated programs in this new climate.
Rogers’ Diffusion of Innovations Theory has been the predominant model for
conceptualizing the complicated, long-term process of diffusing or translating effective public
health programs to many types of settings, including schools (Rogers, 1983). The key
assumptions of this model are that knowledge is a product, generalizable across contexts, and
effective research will naturally be adopted by consumers (Estabrooks & Glasgow, 2006).
However, due to the nature of school settings, with multiple levels of decision-making dispersed
among a central administration and multiple schools, decisions about implementation of
evidence-based programs are rather complex (Shinn, 2003; Spillane, 1998). In a recent national
study, Ringwalt et al. (2011) found that only 46.9% of school districts reported using an
evidence-based substance use prevention program with their students, and only 25.9% of
respondents reported using an evidence-based program more than other (non-evidence-based)
prevention programs. This is despite years of federal and state policies (e.g., Safe and Drug Free
78

Schools Act of 1999 and the No Child Left Behind Act of 2001) mandating the use of research-
validated programs (Hallfors & Godette, 2002), and publication of several lists of ‘best practice,’
‘research-validated,’ and ‘evidence-based’ programs that have proven effectiveness in reducing
youth risk behaviors (Fagan, Hanson, Hawkins, & Arthur, 2008; National Cancer Institute &
Substance Abuse and Mental Health Services Administration, 2010; United States Department of
Health and Human Services & Substance Abuse and Mental Health Services Administration,
2010). Given these findings, the use of research-validated programs in schools is unlikely to
increase in the absence of substantial efforts to improve dissemination of information about
preferred curriculum (Ringwalt, et al., 2002).
Program diffusion or translation is thought to occur through a series of distinct linear stages,
beginning with dissemination (gaining knowledge about the program and forming beliefs),
leading to adoption (deciding to use the program) and implementation (taking action, i.e.,
putting the program into use), and culminating with sustained use (continuing to use the
program) (Goodman & Steckler, 1989; O'Loughlin, Renaud, Richard, Gomez, & Paradis, 1998;
Pluye, Potvin, & Denis, 2004; Shediac-Rizkallah & Bone, 1998). To date, in linear models of
translation, such as the diffusion of innovations model (Rogers, 1983), the primary emphasis has
been placed on explaining individual adopters of innovations rather than the organizational- and
system-level contexts in which adoption decisions are made. Thus, there is a need for a greater
understanding of the context for evidence-based program adoption and implementation, in
order to guide the translation of interventions from research to practice.
To address these limitations, researchers have proposed new models of decision-making
called “systems thinking” or “systems models” (Estabrooks & Glasgow, 2006). Consistent with
social ecological models, systems approaches posit that knowledge integration is contextual and
79

tied to organizational priorities and culture. They assume that relationships are critical, but must
be understood from a multilevel systems perspective. Applying this to school-based research,
we must take into account the multiple systems (e.g., state agencies, school districts, school
administrators, teachers, unions, school boards, etc.) affecting decision-making within schools.
Chen (1998) proposed three categories of factors that influence program implementation,
including the organizational context in which the program is implemented (e.g., administrator
support, school climate), characteristics of the implementer, and the implementation system
(e.g., staff training and infrastructure to coordinate implementation). In Chen’s model (1998),
interventions take place within an implementation system, which affords the means and context
for delivering the intervention (Greenberg, Domitrovich, Graczyk, & Zins, 2005). Guided by
theory driven evaluations (Chen, 1998), the diffusion of innovations model (Rogers, 1983), and
systems models (Estabrooks & Glasgow, 2006), researchers have begun to identify
characteristics of the community and organizational context, as well as beliefs of the individual
decision-maker, that are associated with the adoption and implementation of research-
validated programs in schools (Durlak & DuPre, 2008; Little & Rohrbach, Manuscript submitted
for publication).
Available funding is often cited as a key factor influencing school districts’ intentions to
adopt evidence-based tobacco use prevention programs, given the high costs associated with
purchasing training and program materials (Cho, Hallfors, Iritani, & Hartman, 2009; Hallfors &
Godette, 2002; Rohrbach, Grana, Sussman, & Valente, 2006). Findings from the California
Tobacco Use Prevention Education (TUPE) program evaluation found that high schools with
competitive TUPE grants were more likely than non-grantee high schools to provide tobacco use
prevention and cessation programs (McCarthy, et al., 2008). Yet, there were no significant
80

differences between grantee and non-grantee high schools in regards to student enrollment,
ethnicity, poverty (% subsidized meals), academic performance and parent education
(McCarthy, et al., 2008). This implies that there are other organizational- and systems-level
factors besides district demographic characteristics influencing districts’ intentions to adopt and
use evidence-based tobacco use prevention and cessation programs.
District size and population density have been associated with adoption and use of tobacco
use prevention programs, with larger districts from more urban areas more likely to adopt and
implement evidence-based prevention programs (Cho, et al., 2009; Ennett, et al., 2003;
Ringwalt, et al., 2002; Rohrbach, Ringwalt, Ennett, & Vincus, 2005). Additionally, communities
with a higher level of collaboration between schools and outside constituency groups (e.g.,
parents and state level agencies) have been found to be more likely to adopt evidence-based
prevention programs (Blake, et al., 2005; Rohrbach, et al., 2005). Adoption has also been greater
among schools with a positive external environment (e.g., greater stability outside of school,
less opposition to prevention, more mandates and policies supporting prevention programs and
less bureaucracy) (Gingiss, Roberts-Gray, & Boerm, 2006; Roberts-Gray, Gingiss, & Boerm,
2007).
Two characteristics of the organization that have been found to influence the use of
evidence-based substance use prevention programs in schools include the amount of effort that
a district-level coordinator devotes to substance use prevention and the presence of a program
champion (Fagan, et al., 2008; Fagan & Mihalic, 2003; Gingiss, et al., 2006; Mihalic, Fagan, &
Argamaso, 2008; Roberts-Gray, et al., 2007; Rohrbach, et al., 2006; Rohrbach, et al., 2005).
Aspects of the organizational climate (e.g., district innovativeness, collaboration and open
communication) have also been associated with evidenced-based program adoption and use
81

(Beets, et al., 2008; Ennett, et al., 2003; Gittelsohn, et al., 2003; Kallestad & Olweus, 2003;
Klimes-Dougan, et al., Oct 2009; Rohrbach, et al., 2005).
Beliefs of the individual decision-maker are thought to influence the adoption and use of
evidence-based programs. Studies have shown that district administrators pay greater attention
to evidence that is in line with their preexisting beliefs, which in turn affects their decision-
making (Coburn, Toure, & Yamashita, 2009). In one national study assessing predictors of
adoption decisions, mid-level district administrators were the most important decision-makers
in regard to substance use prevention programs (Rohrbach, et al., 2005). These administrators
employed a variety of sources to gather information to guide their decisions, and decision-
making was also shaped by organizational priorities and available resources. Overall, few
studies have examined the relationship between beliefs of the individual decision-maker and
adoption of evidence-based programs; thus, there remains a substantial gap in our
understanding of how to translate these programs on a wide scale (Rohrbach, et al., 2006;
Woolf, 2008).
Similarly, few studies have looked at factors related to the use of effective tobacco cessation
programs that target youth. In one of the only studies assessing the prevalence of use of
adolescent tobacco cessation curricula, Curry et al. (2007) found that two-thirds of counties
across the nation reported utilizing at least one tobacco cessation program. However, less than
half of those surveyed implemented a program developed by an outside agency. Among those
with externally-developed programs, reasons for adoption included perceived positive
characteristics of the curricula, such as its evidence-base, the ease of adoption, and a
recommendation for its use from tobacco cessation experts and other colleagues (Curry, et al.,
82

2007). Forty percent of counties cited an organizational initiative as the main impetus for
offering a youth cessation program.
The Current Study
While there is now a growing body of literature examining factors related to the adoption
and implementation of effective substance use prevention programs, it suffers from some
limitations. Notably, few studies have examined factors that affect program adoption in schools
(Blake, et al., 2005; Cho, et al., 2009; Fagan, et al., 2008; Hallfors & Godette, 2002; Rohrbach,
Gunning, Grana, Gunning, & Sussman, 2010; Rohrbach, et al., 2005) and even less have looked
at adoption and use of tobacco cessation programs (Curry, et al., 2007). Additionally, based on
the existing literature we know very little about the relative importance of correlates of
program adoption and implementation at the provider, organizational, and community levels
(Beets, et al., 2008; Payne & Eckert, 2010). In the present study, we use a theory-driven
approach, drawing on diffusion of innovations theory (Rogers, 2002), Chen’s model (Chen,
1998), systems models (e.g., Estabrooks & Glasgow, 2006), and recent reviews of the literature
(e.g., Durlak & DuPre, 2008) to examine the relative influence of community-, organizational-
and individual-level factors on the adoption and implementation of evidence-based tobacco
prevention and cessation programs in California schools.
DESIGN AND METHODS
Target Sample
Data for this study come from a cross-sectional survey of administrators and TUPE
coordinators in school districts and county offices across the state of California. There are 58
county offices of education in California that play a vital role in providing technical assistance,
staff development, curriculum and instructional support and oversight to the school districts
83

located within their counties. When conditions (e.g., economic or technical) make county
services more appropriate for students, county offices of education may provide a wide range of
services including tobacco use prevention and cessation programs for youth (California
Department of Education, 2011a). In California, school districts, direct-funded charter schools,
and county offices of education that serve students in grades six through twelve and have met
the tobacco-free school district requirements are eligible to apply for competitive TUPE funds.
In the present study, the sample was drawn from two matched groups of school districts
and county offices of education: (a) those that submitted an application to the California
Department of Education (CDE) in 2009 for TUPE grades 6-12 competitive funds (N=134) and (b)
a matched group that did not apply (n=134). The two groups of districts and county offices of
education were matched on three key demographic variables obtained from the Quality
Education Direct’s California District 2008 file (Quality Education Direct Inc., 2009), including
population density, district size, and student ethnicity, in order to reduce bias and increase
precision in our estimates. The matching score was derived from beta estimates of the key
demographic variables taken from a nationally representative sample of school districts. We
created an equation using these estimates and applied it to the sample of California school
districts and county offices of education. The remaining 30 California county offices of education
were also included in the target sample. The final target sample included 298 school districts
and county offices of education.
Data Collection
Data collection occurred between January and October 2011. The target respondent in each
county office of education was the TUPE coordinator, and in each school district was the
substance use prevention or TUPE coordinator. Previous research suggests that this
84

administrator is the key decision-maker in regards to tobacco use prevention education within
the organization (Rohrbach, et al., 2005). However, because of the recent elimination of Safe
and Drug Free Schools (SDFS) funds, many school districts no longer have a SDFS coordinator. In
those cases, the respondent was the individual within the district that knew the most about the
tobacco prevention and cessation programs currently being offered to students. To establish the
appropriate respondent for our survey, organizations were contacted by telephone in advance
of data collection. Next, this individual was invited by phone and/or email to participate in the
study and complete a 25-minute survey via a secure Web site, for which they were provided a
prepaid $20 cash incentive. We made a maximum of five attempts to reach the target
respondent and obtain his/her informed consent to participate in the survey. Respondents who
did not complete the Web-based survey after repeated contacts were contacted for a brief
telephone interview (N = 7).
Final Study Sample
A total of 276 (92.6%) of the target school district administrators and county office of
education TUPE coordinators were reached, and of those, 258 (93.4%) agreed to participate in
the study. Of those agreeing to participate, we surveyed 235 administrators (91.1% response
rate).
Measures
The survey included n=89 items, the majority of which were adapted from previous studies
(Battistich, Solomon, Kim, Watson, & Schaps, 1995; Cho, et al., 2009; Ringwalt, et al., 2003;
Rohrbach, Gunning, Sussman, & Sun, 2008; Rohrbach, et al., 2005).

85

Dependent Variables
To assess the use of a research-validated prevention program, we first asked respondents to
list which prevention program they were using the most with their students in middle school.
Districts that serve grades 9-12 only were asked to list which prevention program they were
using the most with their students in high school. These items were then coded as research
validated (1) or not (0). Programs were determined to be research-validated if they were
identified in one of the following state or federal registries as a school-based program that is
effective in preventing tobacco use behaviors. The registries included: California Healthy Kids
Resource Center (California HealthyKids Resource Center, 2010), National Registry of Evidence-
based Programs and Practices (NREPP) (Substance Abuse and Mental Health Services
Administration & United States Department of Health and Human Services, 2010), Exemplary
and Promising: Safe, Disciplined, and Drug-Free Schools Programs (Office of Safe and Drug Free
Schools & United States Department of Education, 2010), Research-Tested Intervention
Programs (National Cancer Institute & Substance Abuse and Mental Health Services
Administration, 2010), and Preventing Drug Use Among Children and Adolescents: A Research-
Based Guide (National Institute on Drug Abuse, National Institute of Health, & United States
Department of Health and Human Services, 2010). For a list of research-validated tobacco
prevention programs, see Appendix A.
To assess the use of an approved tobacco cessation program, respondents were asked to list
which tobacco cessation program they were using the most with their students. The program
was then coded as “approved” if it was included in one of the state or federal registries listed
above (0 = no or 1 = yes). Additionally, a program was coded as approved if it was listed on the
86

TUPE program 2009 competitive grant application as an approved program for tobacco
cessation for grades 7-12. For a list of approved tobacco cessation programs, see Appendix B.
Program adoption (intention to use) was operationalized as the submission of an application
to the California Department of Education (CDE) for competitive TUPE grant funds between
2009 and 2011 (0= did not apply, 1 = applied). The names of school districts and county offices
of education that had applied for a competitive TUPE grant were provided by the CDE (California
Department of Education, 2011b).
Independent Variables
Independent variables assessed community characteristics, organizational factors,
normative beliefs, beliefs about tobacco use prevention strategies, and funding. Variables
measuring community characteristics, including population density (urban, suburban, or rural),
organization size (number of schools in the district or districts, 1 = 1 school, 2 = 2-3 schools, 3 =
4-6 schools , 4 = 7-11 schools, 5 = 11-18 schools, 6 = >18 schools), and student ethnicity were
obtained from Quality Education Direct’s California District 2008 file (Quality Education Direct
Inc., 2009). For data analyses, population density was dichotomized (0=suburban or rural,
1=urban) and the percentage of white students in the district or districts was used to
operationalize ethnicity. Additional community factors were measured through two items that
included the degree to which tobacco use prevention was a priority in the community (1 = not a
priority at all to 4 = high priority), and how big of a problem tobacco use was in the community
(1 = smallest problem to 6 = biggest problem).
The names of school districts and county offices of education that had been awarded a
competitive TUPE grant were provided by CDE (California Department of Education, 2011b).
Districts were considered currently funded if they were awarded a competitive TUPE grant
87

between 2008 and 2011, and previously funded if they were awarded a competitive TUPE grant
between 2006 and 2007 (0 = not funded, 1= funded).
Organizational factors were assessed through several items, including the coordinator’s
effort devoted to tobacco use prevention (1=0-4 hours to 6=40 hours or more per week), the
presence of a program champion (0 = no to 1= yes), and an organizational mandate to use
specific prevention curriculum (0 = no to 1 = yes). Organizational support for tobacco use
prevention was assessed through four items (1 = strongly disagree to 5 = strongly agree; α
=0.79). Fifteen items adapted from Patterson et al. (2005) and Glisson et al. (2008) comprised
the five organizational climate indices. Positive perceived organizational climate assessed
growth and achievement (3 items; e.g., “This school district provides numerous opportunities to
advance if you work for it.” 1 = never to 5 = always; α =0.74), innovation and flexibility (3 items;
“New ideas are readily accepted here.” 1 = never to 5 = always; α =0.64), cooperation (3 items;
“To what extent do your coworkers trust each other?” 1 = never to 5 = always; α =0.79), and
effort (3 items; “People here always want to perform to the best of their ability.” 1 = never to 5
= always; α =0.71). Because these four indexes were highly inter-correlated (α=0.83), for data
analyses the individual items were combined to create a composite positive climate scale (12
items; α=0.88). Negative perceived organizational climate was assessed with the role overload
subscale (3 items; “There are not enough people in my school district to get the work done.” 1 =
never to 5 = always; α =0.79).
Four separate belief items assessed the administrator’s perceived effectiveness of classroom
strategies, nonclassroom-based tobacco use prevention activities (e.g., assemblies, Red Ribbon
week, drug free clubs), student assistance programs, and tobacco use policies in reducing
substance use among students in their district (4-point scales; 1=not effective at all to 4=very
88

effective). Normative beliefs were assessed through one item regarding the perceived
prevalence of use of research-validated tobacco use prevention programs in California school
districts (1 = 0-25% to 4 = 75-100%).
Additional items assessed the primary reason for deciding to offer a youth cessation
program, the primary reason for adopting the specific tobacco prevention and cessation
programs currently being delivered in schools, the sources of information used when selecting
the tobacco prevention program, the use of locally developed tobacco prevention and cessation
programs, the type of implementer of the cessation program, the percentage of students
mandated to attend the cessation program, additional tobacco use prevention materials and
activities offered to students, and supplemental youth development activities.
DATA ANALYSIS
First, reliability of multi-item scales was assessed. Cronbach’s alpha for all scales was
computed. If the alpha was unacceptably low, exploratory factor analyses were conducted to
determine whether the scale represented two or more underlying constructs, and items were
deleted or grouped into subscales as necessary to produce scales with adequate internal
consistency. Next, we tested for multi-colinearity between items and scales. If two items were
highly intercorrelated, they were combined or eliminated from the analyses.
We employed hierarchical logistic regression analyses (Victora, Huttly, Fuchs, & Olinto,
1997) to examine the complex hierarchical relationships between community characteristics,
funding, organizational factors, and beliefs about tobacco use prevention strategies, with each
of the dichotomous dependent variables (program adoption, use of research-validated tobacco
prevention program and use of approved tobacco cessation program). To avoid excessive
parameters, variables not reaching p < .10 were dropped from subsequent analyses. All analyses
89

controlled for the type of organization (school district or county office of education). Variables
were standardized (mean = 0 and standard deviation = 1). Odds ratios and 95% confidence
intervals were reported using two-tailed significance tests. Analyses were conducted using the
SAS (v.9.1.3) statistical package (SAS Institute Inc. SAS/C Online Doc TM, 2000).
RESULTS
Sample Characteristics
Table 4.1 presents data on characteristics of the organizations that participated in the study.
More than three quarters of participants (77.9%) were regular school districts. Organizations
had on average 7-11 schools, with nearly three-fourths of students being either of White
(38.7%) or Hispanic (36.5%) ethnicity. Nearly half of the organizations were located in suburban
areas (46.8%).

90

Table 4.1. Characteristics of organizations (N=235)

Notes:
a
Organization size was coded as: 1 = 1 school, 2 = 2-3 schools, 3 = 4-6 schools, 4 = 7-11 schools, 5 =
11-18 schools, 6 = >18 schools;

Funding and Program Use
Figure 4.1 shows the TUPE funding status and evidence-based program use among the
organizations in the study. Out of 235 participating organizations, more than half (57.9%)
applied for competitive TUPE funds and of those, two-thirds received funding (66.9%). Among
Characteristics
μ (SD) or
Frequency(%)
Community Characteristics
Tobacco use prevention is a community priority 2.59(0.84)
Tobacco use is a problem facing community 3.77(1.44)
Organization Type
County Office of Education 52(22.13%)
Regular Districts 183(77.87%)
Organization Size
a
4.27(1.45)
Population Density
Urban 89(37.87%)
Suburban 110(46.81%)
Rural 36(15.32%)
Percent of White Students 38.65(23.70)
Funding
Previously had TUPE Competitive Grant 48(20.43%)
Currently has TUPE Competitive Grant Funding 92(39.15%)
Organizational Factors
Organizational mandate to use specific prevention curriculum 107(45.53%)
Program Champion 88(37.45%)
Coordinator effort devoted to tobacco use prevention 2.06(1.42)
Organizational support for tobacco use prevention 3.73(0.70)
Organizational Climate
Positive Climate 3.57(0.48)
Role Overload 3.53(0.62)
Beliefs
Beliefs regarding prevalence of evidence-based program use in California 2.56(0.92)
Classroom Curricula effectiveness 3.05(0.51)
Nonclassroom-based tobacco use prevention activities effectiveness 2.75(0.66)
Student Assistance Programs effectiveness 3.17(0.67)
Tobacco Use Policies effectiveness 2.96(0.76)
91

these 91 organizations that were awarded TUPE grants, over three-fourths (76.9%) used
research-validated tobacco use prevention programs and more than half (56.0%) used approved
tobacco cessation programs. Among the 45 organizations that applied but did not receive
competitive TUPE funding, roughly half (51.1%) used research-validated tobacco use prevention
programs and less than a quarter (17.8%) used approved tobacco cessation programs. Among
the 99 organizations that did not apply for competitive TUPE funding, less than half (43.4%) used
research-validated tobacco use prevention programs and less than a quarter (17.2%) used
approved tobacco cessation programs.
Figure 4.1 TUPE funding status and evidence-based program use among organizations

Among the organizations that reported using a tobacco prevention program, the most
prevalent programs were Project Alert (45.1%), Project Toward No Drug Use (28.1%), Too Good
for Drugs (22.6%), Life Skills Training (16.6%) and Project Toward No Tobacco Use (12.8%). Over
one quarter of the organizations that reported using a tobacco prevention program (28.9%)
Sample
(N = 235)
Applied for TUPE
funding
(N = 136, 57.9%)
Received
funding
(N = 91, 66.9%)
Used EVB
Prevention
Program
(N = 70, 76.9%)
Used Approved
Cessation
Program
(N = 51, 56.0%)
Did not receive
funding
(N = 45, 33.1%)
Used EVB
Prevention
Program
(N = 23, 51.1%)
Used Approved
Cessation
Program
(N = 8, 17.8%)
Did not apply
(N = 99, 42.1%)
Used EVB
Prevention
Program
(N = 43, 43.4%)
Used Approved
Cessation
Program
(N = 17, 17.2%)
92

reported using a locally developed program. In addition, nearly all of these districts reported
holding Red Ribbon Week and distributing pamphlets, posters or other tobacco use prevention
materials (85.2% and 80.4%, respectively). The two most strongly endorsed reasons for adopting
the tobacco use prevention program they were using the most were its evidence base (μ = 3.45,
SD = 0.56) and specific qualities of the program (e.g., cost, layout, appeared easy to use) (μ =
3.28, SD = 0.60). The most important sources of information influencing the selection of the
tobacco use prevention program they were using the most were data from the districts'
California Healthy Kids Survey administration or other needs assessment data (μ = 3.35, SD =
0.74) and information from the California Healthy Kids Resource Center (μ = 3.33, SD = 0.71).
Almost half (49.4%) of the organizations surveyed reported using a tobacco cessation
program, and among those that reported using a tobacco cessation program, more than half
(53.4%) reporting using an approved tobacco cessation curriculum. One quarter of those that
reported using a tobacco cessation program reported using a locally developed program. The
most commonly reported supplemental youth development activities included having youth
involved in school- or district-wide TUPE program planning (37.6%), peer educators for TUPE
(34.9%) and media literacy/production to counter the influence of the tobacco industry (33.3%).
The most strongly endorsed reason for deciding to offer a tobacco cessation program was the
availability of competitive grant funds from the CDE (μ = 3.49, SD = 1.00), and the primary
reason for adopting the specific youth cessation program they were using the most was because
it was research validated (μ = 3.62, SD = 0.61). Classroom teachers and community agencies
were the most commonly reported types of cessation program implementers (42.2% and 41.4%,
respectively).

93

Correlates of Adoption and Use of Evidence-based Programs
Four sets of variables, community characteristics, funding, organizational factors, and beliefs
about tobacco use prevention strategies, were entered into a series of hierarchical logistic
regression models as correlates of program adoption, use of a research-validated tobacco
prevention program and use of an approved tobacco cessation program.

94

Table 4.2. Summary of hierarchical regression analysis for variables correlated with program
adoption (applying for TUPE grant)

Notes: All analyses controlled for type of organization (1=COE, 0=Other); Significant predictors at p < .10
were kept in subsequent models. * p < .05; + p < .10
Table 4.2 shows the results from the models that examined correlates of organizations’
adoption of (intentions to use) an evidence-based tobacco use prevention or cessation program.
Model 1 included the five community characteristics variables: tobacco use prevention is a
OR (95%CI)
Model 1: Community Characteristics
Tobacco use prevention is a community priority 1.01(0.75, 1.34)
Tobacco use is a problem facing community 0.90(0.67, 1.22)
Organization size 1.90(1.34, 2.70)*
Population Density 0.82(0.60, 1.11)
Percent of White Students 1.04(0.76, 1.43)
Model 2: Funding
Organization size 1.48(1.08, 2.03)*
Previously had TUPE Competitive Grant 3.50(1.95, 6.29)*
Model 3: Organizational Factors
Organization size 1.23(0.86, 1.76)
Previously had TUPE Competitive Grant 3.09(1.70, 5.64)*
Organizational mandate to use specific prevention curriculum 1.21(0.86, 1.72)
Program Champion 1.34(0.95, 1.90)+
Coordinator effort devoted to tobacco use prevention 1.76(1.14, 2.70)*
Organizational support for tobacco use prevention 1.12(0.78, 1.60)
Organizational Climate
Positive Climate 0.99(0.69, 1.43)
Role Overload 1.20(0.86, 1.66)
Model 4: Beliefs
Organization size 1.39(0.95, 2.03)
+

Previously had TUPE Competitive Grant 3.13(1.72, 5.72)*
Program Champion 1.44(1.01, 2.07)
+

Coordinator effort devoted to tobacco use prevention 1.56(1.01, 2.43)
+

Beliefs regarding prevalence of evidence-based program use in
California
1.09(0.77, 1.55)
Classroom Curricula effectiveness 1.03(0.71, 1.50)
Nonclassroom-based tobacco use prevention activities effectiveness 0.88(0.62, 1.24)
Student Assistance Programs effectiveness 0.96(0.68, 1.37)
Tobacco Use Policies effectiveness 1.11(0.79, 1.57)
95

community priority, tobacco use is a problem facing the community, size of the organization,
population density, and percent of White students. Among these variables, size of the
organization was the only significant correlate of adoption, with larger districts more likely to
adopt tobacco prevention and cessation programs (p < .05). Model 2 included size of the
organization and added funding (whether the organization had previously received competitive
TUPE funds). Previously receiving competitive TUPE funds was significantly associated with
adoption (p < .05). Model 3 included the two significant correlates from the previous models,
size of the organization and previously received competitive TUPE funds, and added
organizational factors: having an organizational mandate to use a research-validated program,
having a program champion, coordinator effort devoted to tobacco use prevention,
organizational support for tobacco use prevention, positive organizational climate and role
overload. Among the organizational factors, having a program champion (p < .10) and more
coordinator effort devoted to tobacco use prevention (p <.05) were significantly associated with
program adoption. Last, Model 4 included size of the organization, previously received
competitive TUPE funds, having a program champion, and coordinator effort devoted to tobacco
use prevention, and added beliefs items, including perceived prevalence of use of evidence-
based tobacco use programs, and beliefs about the effectiveness of classroom curricula,
nonclassroom-based tobacco use prevention activities, Student Assistance Programs, and
tobacco use policies in reducing tobacco use among adolescents. None of the added belief items
was significantly associated with program adoption.

96

Table 4.3. Summary of hierarchical regression analysis for variables correlated with using a
research-validated tobacco use prevention program

Notes: All analyses controlled for type of organization (1=COE, 0=Other); Significant predictors at p < .10
were kept in subsequent models. * p < .05; + p < .10

OR (95%CI)
Model 1: Community Characteristics
Tobacco use prevention is a community priority 1.60(1.16, 2.20)*
Tobacco use is a problem facing community 0.88(0.64, 1.21)
Organization size 2.17(1.49, 3.17)*
Population Density 0.67(0.48, 0.94)*
Percent of White Students 0.68(0.49, 0.96)*
Model 2: Funding
Tobacco use prevention is a community priority 1.74(1.25, 2.42)*
Organization size 2.29(1.55, 3.39)*
Population Density 0.65(0.46, 0.92)
+

Percent of White Students 0.74(0.53, 1.04)
Previously had TUPE Competitive Grant 1.02(0.74, 1.42)
Currently has TUPE Competitive Grant Funding 1.87(1.34, 2.60)*
Model 3: Organizational Factors
Tobacco use prevention is a community priority 1.59(1.07, 2.36)*
Organization size 2.28(1.41, 3.69)*
Population Density 0.52(0.34, 0.79)*
Percent of White Students 0.75(0.50, 1.11)
Currently has TUPE Competitive Grant Funding 1.60(1.07, 2.40)*
Organizational mandate to use specific prevention curriculum 2.59(1.73, 3.88)*
Program Champion 1.64(1.10, 2.44)*
Coordinator effort devoted to tobacco use prevention 1.33(0.86, 2.05)
Organizational support for tobacco use prevention 0.65(0.41, 1.02)+
Organizational Climate
Positive Climate 1.22(0.80, 1.86)
Role Overload 0.67(0.45, 0.98)*
97

Table 4.3. Continued

Notes: All analyses controlled for type of organization (1=COE, 0=Other); Significant predictors at p < .10
were kept in subsequent models. * p < .05; + p < .10
Results of the hierarchical logistic regression models that examined correlates of the use of
research-validated tobacco use prevention programs are presented in Table 4.3. Model 1
included the same five community characteristics variables. Among these variables, identifying
tobacco use as a community priority, larger organization size, lower population density, and a
lower proportion of white students were positive correlates of use of research-validated
programs (p’s < .05). Model 2 included tobacco use prevention as a community priority, size of
the organization, population density, and percent of White students, and added two funding
variables: previously received competitive TUPE funds and currently receiving competitive TUPE
funds. Currently receiving competitive TUPE funds was significantly associated with research-
validated program use (p < .05). Model 3 included tobacco use prevention as a community
priority, size of the organization, population density, percent of White students, and currently
OR (95%CI)
Model 4: Beliefs
Tobacco use prevention is a community priority 1.57(1.01, 2.43)*
Organization size 2.64(1.55, 4.49)*
Population Density 0.52(0.32, 0.83)*
Percent of White Students 0.75(0.48, 1.17)
Currently has TUPE Competitive Grant Funding 1.44(0.95, 2.20)
+

Organizational mandate to use specific prevention curriculum 2.05(1.32, 3.16)*
Program Champion 1.78(1.14, 2.78)*
Organizational support for tobacco use prevention 0.68(0.43, 1.07)
+

Organizational Climate
Role Overload 0.60(0.39, 0.93)*
Beliefs regarding prevalence of evidence-based program use in California 1.44(0.92, 2.24)
Classroom Curricula effectiveness 1.14(0.69, 1.88)
Nonclassroom-based tobacco use prevention activities effectiveness 0.60(0.39, 0.93)*
Student Assistance Programs effectiveness 1.19(0.77, 1.84)
Tobacco Use Policies effectiveness 1.53(1.01, 2.31)*
98

receiving competitive TUPE funds, and added having an organizational mandate to use a
research-validated program, having a program champion, coordinator effort devoted to tobacco
use prevention, organizational support for tobacco use prevention, positive organizational
climate and role overload. Among the organizational items, having an organizational mandate to
use a research-validated program, a program champion, less job overload (p’s < .05), and
organizational support for tobacco use prevention (p < .10) were positively associated with
program use. Last, Model 4 included tobacco use prevention as a community priority, size of the
organization, population density, and percent of White students, currently receiving
competitive TUPE funds, having an organizational mandate to use a research-validated program,
having a program champion, organizational support for tobacco use prevention, and role
overload, and added perceived prevalence of evidence-based tobacco program use, and beliefs
about the effectiveness of classroom curricula, nonclassroom-based tobacco use prevention
activities, Student Assistance Programs, and tobacco use policies in reducing tobacco use among
adolescents. Among the beliefs items, believing that nonclassroom-based tobacco use
prevention activities were less effective in reducing tobacco use among adolescents and tobacco
use policies were effective in reducing use were significantly associated with use of a research-
validated program (p’s < .05).
Table 4.4 presents the results of the hierarchical logistic regression models that examined
correlates of the use of an approved tobacco cessation program. Model 1 included the five
community characteristics variables. Among these variables, organization size and identifying
tobacco use as a problem facing the community were positive correlates of using an approved
cessation program (p’s < .05). Model 2 included tobacco use as a problem facing the community
and size of the organization, and added the two funding variables, previously received
99

competitive TUPE funds and currently receiving competitive TUPE funds. Both previously
receiving and currently receiving competitive TUPE funds were associated with use (p’s < .05).
Model 3 included tobacco use as a problem facing the community, size of the organization,
previously received competitive TUPE funds, and currently receiving competitive TUPE funds,
and added having an organizational mandate to use a research-validated program, having a
program champion, coordinator effort devoted to tobacco use prevention, organizational
support for tobacco use prevention, positive organizational climate and role overload. Among
the organizational items, having an organizational mandate to use a research-validated
program, a program champion, and greater coordinator effort devoted to tobacco use
prevention (p’s <.05) were significantly associated with use. Last, Model 4 included tobacco use
as a problem facing the community, size of the organization, previously received competitive
TUPE funds, currently receiving competitive TUPE funds, having an organizational mandate to
use a research-validated program, having a program champion, and coordinator effort devoted
to tobacco use prevention, and added perceived prevalence of evidence-based program use,
and beliefs about the effectiveness of classroom curricula, nonclassroom-based tobacco use
prevention activities, Student Assistance Programs, and tobacco use policies in reducing tobacco
use among adolescents. None of the added belief items was significantly associated with use of
an approved tobacco cessation program.

100

Table 4.4 Summary of hierarchical regression analysis for variables correlated with using an
approved tobacco cessation program

Notes: All analyses controlled for type of organization (1=COE, 0=Other); Significant predictors at p < .10
were kept in subsequent models. * p < .05; + p < .10

OR (95%CI)
Model 1: Community Characteristics
Tobacco use prevention is a community priority 0.98(0.73, 1.32)
Tobacco use is a problem facing community 1.42(1.02, 1.97)*
Organization size 2.14(1.42, 3.24)*
Population Density 0.84(0.61, 1.15)
Percent of White Students 1.32(0.94, 1.87)
Model 2: Funding
Tobacco use is a problem facing community 1.34(0.96, 1.87)
+

Organization size 1.55(1.07, 2.25)*
Previously had TUPE Competitive Grant 1.36(1.01, 1.85)*
Currently has TUPE Competitive Grant Funding 2.06(1.50, 2.83)*
Model 3: Organizational Factors
Tobacco use is a problem facing community 1.34(0.90, 1.99)
Organization size 1.32(0.84, 2.08)
Previously had TUPE Competitive Grant 1.21(0.86, 1.71)
Currently has TUPE Competitive Grant Funding 1.57(1.07, 2.32)*
Organizational mandate to use specific prevention curriculum 2.07(1.35, 3.17)*
Program Champion 1.53(1.06, 2.22)*
Coordinator effort devoted to tobacco use prevention 2.57(1.63, 4.03)*
Organizational support for tobacco use prevention 0.85(0.55, 1.32)
Organizational Climate
Positive Climate 1.04(0.68, 1.58)
Role Overload 1.08(0.72, 1.63)
Model 4: Beliefs
Tobacco use is a problem facing community 1.39(0.90, 2.13)
Organization size 1.28(0.79, 2.07)
Previously had TUPE Competitive Grant 1.32(0.92, 1.91)
Currently has TUPE Competitive Grant Funding 1.34(0.88, 2.05)
Organizational mandate to use specific prevention curriculum 1.97(1.26, 3.06)*
Program Champion 1.59(1.05, 2.40)*
Coordinator effort devoted to tobacco use prevention 2.14(1.32, 3.48)*
Beliefs regarding prevalence of evidence-based program use in California 0.94(0.61, 1.43)
Classroom Curricula effectiveness 1.09(0.68, 1.74)
Nonclassroom-based tobacco use prevention activities effectiveness 0.99(0.65, 1.50)
Student Assistance Programs effectiveness 1.37(0.87, 2.17)
Tobacco Use Policies effectiveness 1.39(0.92, 2.11)
101

DISCUSSION
School-based tobacco education is an integral part of tobacco control among adolescence.
However, due to the complex nature of decision-making in schools, decisions to offer tobacco
education programs are influenced by a variety of factors at the community, organization, and
individual decision-maker levels. By understanding the factors that lead districts to adopt and
use research-validated tobacco prevention and cessation programs in schools, researchers and
policymakers can work together to shape policies that will increase schools’ use of tobacco
education programs, which will ultimately lead to reductions in tobacco use among adolescents.
The present study utilized data from a cross-sectional sample of school district
administrators and county office of education TUPE coordinators throughout California. Using a
theory-based analytic strategy, hierarchical regression (Victora, et al., 1997), that was guided by
the diffusion of innovations model (Rogers, 2002), theory driven evaluations (Chen, 1998),
systems models (Estabrooks & Glasgow, 2006) and recent reviews of the literature (Durlak &
DuPre, 2008), we explored the complex inter-relationships between community, organizational
and individual-level factors that lead organizations to adopt and implement research-validated
tobacco use prevention and cessation programs in schools. Nearly all of the organizations
surveyed (80.4%) reported using a tobacco use prevention program during the current school
year, and roughly half of the organizations (53.4%) reporting using a tobacco cessation program.
However, only 57.9% reported using a research-validated tobacco use prevention program and
only 32.3% reporting using an approved tobacco cessation program. Furthermore, roughly one-
quarter of the sample reported using locally developed tobacco use prevention and cessation
programs, despite years of state and federal agencies calling for schools to implement
prevention programs with proven effectiveness. However, these findings are not surprising
102

given the high costs associated with using research-validated prevention programs (Hallfors &
Godette, 2002) and the recent reductions in state and federal funding for tobacco use
prevention education.
Among the organizations that were awarded TUPE grants, over three-fourths used research-
validated tobacco use prevention programs and more than half used approved tobacco
cessation programs. Whereas among the organizations that did not have competitive TUPE
funding, only half used research-validated tobacco use prevention programs and less than a
quarter used approved tobacco cessation programs. Nearly half of the organizations that
reported using a substance use prevention program reported using Project Alert (45.1%), and
the most prevalent type of tobacco cessation program provided to students was curriculum
based programs (e.g., Project Ex) (53.4%). Other types of cessation programs, such as quit lines,
smokeless school days and counseling, were less prevalent. Interestingly, 10.3% of organizations
that reported using a tobacco cessation program were using a core subject text book (e.g.,
science, health) or tobacco prevention curriculum (e.g., Project Alert), neither of which has been
endorsed as an approved tobacco cessation program for adolescents. It could be that
organizations failing to use approved tobacco cessation programs may be unaware of the
research findings or unsure how to identify approved cessation curricula. In light of the
substantial demands placed on schools to meet reading, math and science benchmarks
(Kaftarian, Robertson, Compton, Davis, & Volkow, 2004), the use of approved tobacco cessation
programs is unlikely to increase unless substantial efforts are made to disseminate information
about programs with proven effectiveness.
As expected, several community factors were related to adoption and implementation of
tobacco prevention and cessation programs in schools. Consistent with previous research (Cho,
103

et al., 2009; Ringwalt, et al., 2002; Rohrbach, et al., 2005), organizational size was the strongest
community-level correlate of adoption and implementation, which suggests that larger
organizations may have more infrastructure in place to support the implementation of tobacco-
specific education. Other community factors, including population density, student ethnicity,
identifying tobacco use as a problem facing the community and as a community priority, were
all related to the use of research-validated tobacco prevention and cessation programs. Recent
evaluations have established support for the effectiveness of community prevention coalitions
and community-university partnerships in producing high levels of adoption and
implementation of evidence-based prevention programs (Hawkins, et al., 2008; Spoth, et al.,
2007). Our findings support the use of these strategies for increasing the utilization of research-
validated tobacco use program in schools.
Funding has long been cited as an important indicator of whether schools are able to
provide research-validated tobacco use prevention education. Accounting for the community
context, we found that organizations that received TUPE competitive funds were 1.87 times
more likely to use a research-validated tobacco prevention program (CI 1.34, 2.60) and 2.06
times as likely to use an approved tobacco cessation program (CI 1.50, 2.83), than were
organizations without competitive TUPE funds. In fact, the most strongly endorsed reason for
deciding to offer a tobacco cessation program was the availability of competitive grant funds
from the CDE (μ = 3.49, SD = 1.00). Even adoption decisions were influenced by funding, with
organizations that had previously received competitive funding being 3.50 times as likely to
adopt a tobacco use program (CI 1.95, 6.29) as organizations that had not received funding.
Based on these findings, it appears that the availability of dedicated tobacco education funds is
critical to the decision to adopt and implement programs with proven effectiveness in schools.
104

In order to increase the use of research-validated tobacco prevention and cessation programs in
schools throughout California, additional tobacco use prevention education funds need to be
made available and new districts need to be encouraged to apply.
Consistent with Chen’s (1998) conceptual model, we found that decisions to adopt and use
research-validated tobacco prevention and cessation programs were influenced by a number of
organizational factors that relate to the implementation system, including having an
organizational mandate to use a research-validated program, a program champion and more
coordinator effort devoted to tobacco use prevention. These findings are consistent with
previous research (Fagan, et al., 2008; Fagan & Mihalic, 2003; Gingiss, et al., 2006; Mihalic, et al.,
2008; Roberts-Gray, et al., 2007; Rohrbach, et al., 2006; Rohrbach, et al., 2005). Not surprising,
we found that having an organizational climate characterized by being overburdened with work
led to less use of research-validated tobacco prevention programs. Our findings suggest that
implementation systems, which are embedded within the broader organizational and
community context, are important to not only implementation but also the decision to offer
tobacco education programming (Greenberg, et al., 2005).
Interestingly, individual beliefs were related to the use of research-validated tobacco
prevention programs, but not to adoption decisions or the use of approved tobacco cessation
programs. Believing that nonclassroom-based tobacco use prevention activities were less
effective and tobacco use policies were effective in reducing tobacco use among adolescents
were significantly associated with use of a research-validated program. These findings suggest
that the beliefs of administrators play an important role in the use of research-validated tobacco
prevention programs, and should be part of a targeted approach to increase the use of these
programs in schools.
105

Limitations
A limitation to the current study is that respondents included both district administrators
and county office of education coordinators. Although we originally intended to survey only
districts, we found that a number of county offices of education had applied and received
competitive TUPE awards. Additionally, in some areas of the state, county offices of education
are responsible for tobacco use prevention programming within schools. Unfortunately, the
county TUPE coordinator’s role varies by county, and therefore their knowledge on the use of
tobacco prevention and cessation programs in schools in their jurisdiction may vary
considerably. To account for these differences, we included type of organization as a control in
each of our models.
Another limitation to the current study was our assessment of organizational climate from a
single individual within the organization. Because climate refers to the shared perception of the
work environment (Charles Glisson & James, 2002), we were not truly assessing organizational
climate. Rather, we were measuring a proxy of climate, psychological climate, or the
perceptions of the organizational climate from a single respondent. Because of the large
number of organizations involved in the current study, it was not feasible to survey all
employees of each organization involved in the study. However, future studies should explore
the association between organizational climate as assessed from all employees of an
organization, and program use.
A final limitation of the current study pertains to the individual respondent within each
organization. We intended to survey the SDFS or TUPE coordinator in each district, because
previous research suggests that this administrator is the key decision-maker with regard to
tobacco use prevention education (Rohrbach, et al., 2005). However, due to the changes in SDFS
106

and TUPE funding, many districts no longer have a SDFS or TUPE coordinator, which made it
difficult to identify the most appropriate person to complete the survey. Although we
attempted to find the administrator that was the most knowledgeable about tobacco use
prevention education, it is likely that administrators were less familiar with tobacco education
overall than were designated prevention coordinators.
Conclusion
In the current study, we sought to fill a gap in the school-based tobacco prevention
literature by investigating the impact of community, organizational and individual-level factors
as well as funding on district decisions to adopt and use tobacco prevention and cessation
programs. This is the first study we are aware of to use a theory-based analytic strategy,
hierarchical regression (Victora, et al., 1997), to explore these outcomes. We found that several
characteristics of the community and organization, beliefs of key decision makers, and funding
were important factors in school districts’ decisions to adopt and use tobacco prevention and
cessation programs with proven effectiveness, which supports a multilevel systems perspective.
These results should be used to inform policies that affect the use of research-validated tobacco
prevention and cessation programs in schools. Ultimately, the increased use of research-
validated prevention programs should lead to reductions in negative health outcomes among
adolescents.

107

CHAPTER 5. DISCUSSION
Despite the fact that considerable resources that have been spent developing and
disseminating effective school-based substance use prevention programs, many school districts
in the United States fail to use prevention programs with proven effectiveness. Furthermore,
there is a dearth of research understanding the factors that promote adoption, implementation
fidelity, and sustained use of these programs in schools. Consequently, a significant gap remains
in what we know about how to effectively “translate” evidence-based programs from research
to practice (Rohrbach, Grana, Sussman, & Valente, 2006).
Due to the universal nature of schooling, schools have come to play a vital role in promoting
well-being among adolescence and reducing disease risk (Greenberg, 2010). However, because
schools are complex organizations with multiple-levels of decision-making, decisions to offer
prevention programs in schools are influenced by a variety of factors. Research grounded in the
diffusion of innovations theory (Rogers, 1983), theory driven evaluations (Chen, 1998), systems
models (Estabrooks & Glasgow, 2006) and recent reviews of the literature (Durlak & DuPre,
2008) have identified some community-, organizational-, and individual-level factors that are
correlated with the translation of evidence-based programs in schools. However, much of the
research on program translation in schools is nascent. The studies presented here address gaps
in the literature by exploring the relationships between factors at several levels of the ecological
framework and program adoption, implementation fidelity and sustained use of evidence-based
substance use prevention and tobacco cessation programs in schools.
Summary of Findings
Chapter 2 presents results from study 1, which utilized data from the School-based
Substance Use Prevention Programs Study (SSUPPS), a longitudinal sample of substance use
108

prevention practices in middle schools nationwide. In this study, we used a theory-driven
approach, drawing on diffusion of innovations theory (Rogers, 2002), Chen’s model (Chen,
1998), systems models (e.g., Estabrooks & Glasgow, 2006), and recent reviews of the literature
(e.g., Durlak & DuPre, 2008) to examine the relative influence of community-, organizational-
and individual-level factors on districts’ decisions to adopt and sustain evidence-based
substance use prevention programs in schools. By applying a theory-based analytic strategy, we
were able to handle the complex relationships between our independent variables and
dependent variables, adoption and sustained use of evidence-based prevention programs in
schools. Results of the hierarchical regression analyses revealed that larger districts location in
the Midwestern U.S. and having multiple sources of substance use prevention funding were
significantly associated with use of evidence-based programs. We also found that factors related
to the district’s capacity to deliver prevention programs were directly related to their use of
evidence-based prevention programs. Surprisingly, we found that perceptions of a positive
district work climate negatively predicted use of an evidence-based prevention program. In
addition, community support for substance use prevention was negatively associated with the
use of evidence-based prevention program. Lastly, we found that factors such as the curricula
layout and the number of sessions are important determinants of whether a school district will
adopt and sustain the use of an evidence-based prevention program.
Additionally, we explored characteristics of early and late adopters of evidence-based
substance use prevention programs in schools. Districts that were early adopters were larger,
were more likely to be from the South, had more district coordinator time devoted to substance
use prevention, had carried out more prevention planning actions, had more perceived district
109

interest in substance use prevention programming, and had stronger beliefs that classroom
curricula were effective in reducing substance use.
Study 2, presented in chapter 3, utilized data from the Dissemination Trial of Project TND
(Rohrbach, Gunning, Sun, & Sussman, 2010) to explore the extent to which contextual and
individual factors influenced the fidelity of implementation of Project TND in a sample of high
school teachers from around the country. Despite previous research supporting a relationship
between teacher characteristics such as teaching experience and style, and implementation
fidelity (Ringwalt, et al., 2002; Rohrbach, Graham, & Hansen, 1993), we did not find support for
these relationships. We did, however, find that teachers who were more comfortable with the
evidence-based (Project TND) approach were more likely to implement the program with
fidelity. Surprisingly, positive beliefs regarding the school climate and perceived expectations to
implement Project TND were negatively related to fidelity of implementation. However,
organizational capacity was positively related to implementation fidelity. Another contextual
variable that we assessed, urbanicity, was negatively associated with implementation fidelity.
In addition, the study explored whether implementation fidelity influenced program
outcomes in students. We found that after controlling for significant contextual and individual-
level correlates of fidelity, implementation fidelity predicted positive changes in three-fourths of
the student outcomes we assessed. These findings confirm the importance of assessing
implementation fidelity when determining the effectiveness of prevention interventions in
schools.
Study 3, shown in chapter 4, utilized data from a cross-sectional sample of school district
and county office of education Tobacco Use Prevention Education (TUPE) coordinators
throughout California. In this study, we used a theory-driven approach, drawing on diffusion of
110

innovations theory (Rogers, 2002), Chen’s model (Chen, 1998), systems models (e.g., Estabrooks
& Glasgow, 2006), and recent reviews of the literature (e.g., Durlak & DuPre, 2008) to examine
the relative influence of community-, organizational- and individual-level factors on the
adoption and implementation of evidence-based tobacco prevention and cessation programs in
California schools. We found that community factors, including urbanicity, student ethnicity,
believing that tobacco use was a problem facing the community and that tobacco use
prevention was a community priority, were all related to the use of research-validated tobacco
prevention and cessation programs. Organizations that were larger, received TUPE competitive
funds, had an organizational mandate to use a research-validated program, had a program
champion, and had more coordinator effort devoted to tobacco use prevention were more likely
to adopt and implement tobacco prevention and cessation programs. Not surprising, having less
job overload led to greater use of research-validated tobacco prevention programs.
Interestingly, individual beliefs were related to the use of research-validated tobacco prevention
programs, but not to adoption decisions or the use of approved tobacco cessation programs.
Taken together, the three studies presented here suggest that school-based substance use
prevention programming is influenced by a variety of contextual factors occurring at multiple
ecological levels. Characteristics of the community and organization, beliefs of key decision
makers, and funding were important factors in promoting adoption, implementation fidelity and
sustained use of evidence-based substance use prevention programs in schools. In order to
move the field ahead, researchers need to account for the multiple systems at play in schools
when designing research trials and school-based programs.

111

Strengths and Limitations
Despite the substantial body of literature supporting the importance of school-based
prevention programs in reducing adolescent substance use, little is known about how school
and community contextual factors influence the adoption, implementation, and sustained use of
evidence-based prevention programs in schools. Overall, the current studies addressed several
gaps and limitations in the existing research on translation of effective prevention programs to
school settings. Several methodological features of the studies represent improvements over
previous research. First in study 2 we used mixed modeling techniques to account for the
nested structure of the data. Failing to account for the nested structure of the data is a violation
of the assumption of independent errors, and can cause serious errors of inference (Shinn,
2003). Researchers have termed this tendency to ignore the individual’s surroundings as
‘context minimization error’ (Shinn & Toohey, 2003). In studies 1 and 3, we used a theory-based
analytic strategy, hierarchical regression (Victora, Huttly, Fuchs, & Olinto, 1997), to explore the
complex relationships between factors from a variety of contexts (i.e., the community,
organization, provider, program and audience targeted) and adoption and implementation of
evidence-based programs. Study 1 is the first study of which we are aware to apply a
longitudinal design to examine contextual factors related to prevention program adoption and
sustained use. Study 3 is the first study of which we are aware that statistically models the
relationship between contextual factors and the use of school-based tobacco cessation
programs. Given recent changes in state and federal funding for school-based substance use
prevention programming, study 3 makes a unique contribution to the literature by examining
the impact of these policies on tobacco prevention and cessation service delivery in schools.
112

The study findings should be considered in light of several limitations. A limitation to all
three studies was our assessment of organizational climate. Because we did not assess climate
from all individuals within the work environment, we were not truly assessing climate. Rather,
we were measuring a proxy of climate, psychological climate or the perceptions of the climate.
However, due to the large number of organizations involved in the current studies, it was not
feasible to survey all employees of each organization. Future studies should explore the
association between organizational climate as assessed from all employees of a district and
evidence-based program use. Secondly, our measures of climate may not have captured the
entire range of climate. For instance, Glisson’s Organizational Social Context (OSC) scale assesses
climate through eight first-order scales and three second-order scales (Glisson, et al., 2008). Due
to the large scope of the present studies, assessing climate through more items was not
feasible. In study 3, we did measure four indices of positive climate and found that they were
highly inter-correlated (α=0.83); thus, we combined them into one scale. However, future
studies should explore the associations between climate, assessed through a more
comprehensive scale, and the use of evidence-based prevention programs.
Another limitation of the current studies pertains to the individual respondent within each
district. In study 1, attempts were not made to survey the same respondent from each district in
both waves of data collection due to feasibility constraints. However, the respondents did hold
the same position in the school district (i.e., the Safe and Drug Free Schools Coordinator) in both
waves. In study 3, we intended to survey the SDFS or TUPE coordinator in each district, because
previous research suggests that this administrator is the key decision-maker with regard to
tobacco use prevention education (Rohrbach, Ringwalt, Ennett, & Vincus, 2005). However, due
to recent changes in SDFS and TUPE funding, many districts no longer have a SDFS or TUPE
113

coordinator, which made it challenging to identify the most appropriate person to complete the
survey. Although we attempted to find the administrator that was the most knowledgeable
about tobacco use prevention education, it is likely that administrators were less familiar with
tobacco education overall than were designated prevention coordinators.
Additionally, in study 3 respondents included both district administrators and county office
of education coordinators. Although we originally intended to survey only districts, we found
that a number of county offices of education had applied for and received competitive TUPE
awards. In some areas of the state, county offices of education are responsible for tobacco use
prevention programming within schools. Unfortunately, the county TUPE coordinator’s role
varies by county, and therefore their knowledge on the use of tobacco prevention and cessation
programs in schools in their jurisdiction may vary considerably. To account for these differences,
we included type of organization as a control in each of our models.
A limitation to study 2 is that we have not conducted studies that identify specific mediators
of Project TND. However, the immediate outcome variables included in the current study are
hypothesized program mediators, and have been shown to predict substance use in previous
studies (Rohrbach, Sussman, & Dent, 2005; Sussman & Dent, 1996; Sussman, McCuller, & Dent,
2003).
Implications for Prevention
Consistent with previous research (Cho, Hallfors, Iritani, & Hartman, 2009; Ringwalt, et al.,
2002; Rohrbach, et al., 2005), several community factors were related to the use of substance
use prevention and tobacco cessation programs in schools. In studies 1 and 3, organizational
size was associated with adoption, implementation and sustained use, which suggests that
larger school organizations may have more infrastructure in place to support the use of
114

substance use prevention programming. In contrast, in studies 2 and 3 we found that greater
population density and a larger percentage of non-White students were associated with less
program adoption and implementation fidelity. These findings suggest that prevention
scientists need to develop and test strategies for increasing evidence-based program utilization
in high risk urban settings. One strategy that has been successful in building capacity for the use
evidence-based programs in schools is the use of community-based partnerships between
school personnel, representatives of community agencies, and providers of technical assistance
(Spoth, Greenberg, Bierman, & Redmond, 2004). Recent evaluations have established support
for the effectiveness of these partnerships in producing high levels of adoption and
implementation of evidence-based prevention programs in schools, especially where there is
already community support for substance use prevention (Hawkins, et al., 2008; Spoth, et al.,
2007). In studies 1 and 3, we found that community support was related to substance use
prevention programming in schools, but surprisingly, we found contradicting associations across
the two studies. In study 1, community support was negatively associated with adoption and
sustained use, while in study 3 community support was positively related to use. A possible
explanation for this different pattern of results relate to the policy changes that occurred
between the time that the data was collected for studies 1 and 3. The elimination of the Safe
and Drug Free Schools and Communities (SDFSC) grants program from the federal budget in
2010 and changes to the California Health and Safety Code provisions which removed the annual
entitlements for tobacco use education in schools and replaced them with competitive funding,
appears to have had a substantial impact on substance use prevention education in schools.
Funding has long been cited as an important indicator of whether schools are able to
provide research-validated substance use prevention education, and this was no different in the
115

current studies. In studies 1 and 3, we found that the having dedicated substance use
prevention education funds was critical to the decision to adopt, implement and sustain
programs with proven effectiveness in schools. Particularly in study 3, we found that among
organizations that were awarded competitive TUPE grants, over three-fourths used research-
validated tobacco use prevention programs and more than half used approved tobacco
cessation programs. In comparison, among the organizations that did not have competitive
TUPE funding, only half used research-validated tobacco use prevention programs and less than
one quarter used approved tobacco cessation programs. These findings highlight the
importance of external funding to support school-based substance use prevention education. In
order to increase the use of research-validated prevention programs in schools, additional
public funds need to be made available for programming and districts need to be encouraged to
apply for them.
Consistent with previous research (Blake, et al., 2005; Fagan, Hanson, Hawkins, & Arthur,
2008; Fagan & Mihalic, 2003; Gingiss, Roberts-Gray, & Boerm, 2006; Mihalic, Fagan, &
Argamaso, 2008; Roberts-Gray, Gingiss, & Boerm, 2007; Rohrbach, et al., 2005) and Chen’s
conceptual model (1998), we found that various aspects of the district’s capacity to deliver
prevention programs were directly related to adoption, implementation fidelity and sustained
use of evidence-based prevention programs across the three studies. These findings suggest
that implementation systems, which are embedded within the broader organizational and
community context, are important to not only implementation, but also to the adoption and
sustained use of evidence-based substance use prevention programming (Greenberg,
Domitrovich, Graczyk, & Zins, 2005).
116

Surprisingly, in both studies 1 and 2 we found that perceptions of a positive school climate
were negatively related to adoption, implementation fidelity and sustained use of prevention
programs. These findings are contrary to what we expected based on previous research (Beets,
et al., 2008; Ennett, et al., 2003; Gittelsohn, et al., 2003; Kallestad & Olweus, 2003; Klimes-
Dougan, et al., 2009; Rohrbach, et al., 2005). However in study 3, which was conducted after the
policy changes to substance use prevention education, positive school climate was not related
to adoption or implementation, although a subscale of negative climate, role overload, was
associated with less use of a research-validated tobacco program. Given the increasing demands
on schools to focus their efforts on core subjects (e.g., English, math and science) (Kaftarian,
Robertson, Compton, Davis, & Volkow, 2004), rising budget crises in states nationwide
(Dusenbury & Hansen, 2004) and recent changes to federal and state funding for substance use
prevention education, it is critical for researchers to explore innovative ways of increasing
adoption, implementation fidelity and sustained use of evidence-based substance use
prevention programs in schools. Findings from study 3 suggest that schools that are
overburdened with meeting core subject benchmarks will not put forth the resources to try to
secure substance use prevention education funds, nor will they use valuable class time to
implement tobacco and substance use prevention programs in the absence of external funding.
In order to make substance abuse prevention programs more appealing, we need to focus on
making programs more compatible with schools’ needs. There is now a growing body of
research demonstrating that many negative outcomes (e.g. psychopathology, substance abuse,
delinquency and school failure) have related risk factors and co-morbidity (Greenberg, 2010).
Efforts should be made to disseminate this research to key decision-makers in school districts to
117

promote the use of programming to prevent a broad range of social and emotional problems
among youth (National Research Council and Institute of Medicine, 2009).
Across the three studies we found that individual administrators’ beliefs about the program
were related to districts’ adoption, implementation fidelity and sustained use of evidence-based
prevention programs. These findings are consistent with the diffusion of innovations theory
(Rogers, 2002), and suggest that researchers should be mindful of program characteristics and
their compatibility with district needs and teachers’ instructional styles when developing
prevention programs. If districts administrators and teachers are aware of the benefits of
prevention programs in promoting improved behavior and well-being among students
(Greenberg, 2010), they may be more likely to put in the effort adopt, implement and sustain
these programs.
The findings presented in these studies elucidate potential community, organizational and
individual targets for increasing adoption, implementation fidelity and sustained use of
evidence-based prevention programs in schools, and should be used to guide interventions and
inform policies. Ultimately, translational research can be used to increase the use of evidence-
based prevention programs in schools, which should lead to reductions in substance use and
other problem behaviors among adolescents.

118

REFERENCES
Ajzen, I. (1991). The theory of planned behavior. Organizational Behavior and Human Decision
Process, 50, 179 - 211.
Battistich, V., Solomon, D., Kim, D., Watson, M., & Schaps, E. (1995). Schools as communities,
poverty levels of student populations and students’ attitudes, motives and performance:
a multilevel analysis. American Educational Research Journal, 32, 627-658.
Beets, M., Flay, B., Vuchinich, S., Acock, A., Li, K.-K., & Allred, C. (2008). School Climate and
Teachers’ Beliefs and Attitudes Associated with Implementation of the Positive Action
Program: A Diffusion of Innovations Model. Prevention Science.
Blake, S. M., Ledsky, R. A., Sawyer, R. J., Goodenow, C., Banspach, S., Lohrmann, D. K., et al.
(2005). Local school district adoption of state-recommended policies on HIV prevention
education. Preventive Medicine: An International Journal Devoted to Practice and
Theory, 40(2), 239-248.
Bowen, D. J., Sorensen, G., Weiner, B. J., Campbell, M., Emmons, K., & Melvin, C. (2009).
Dissemination research in cancer control: where are we and where should we go?
Cancer Causes Control, 20(4), 473-485.
Brown, B. S., & Flynn, P. M. (2002). The federal role in drug abuse technology transfer: a history
and perspective. Journal of Substance Abuse Treatment, 22(4), 245-257.
Brownson, R. C., Ballew, P., Dieffenderfer, B., Haire-Joshu, D., Heath, G. W., Kreuter, M. W., et
al. (2007). Evidence-Based Interventions to Promote Physical Activity: What Contributes
to Dissemination by State Health Departments. American Journal of Preventive
Medicine, 33(1, Supplement 1), S66-S78.
California Department of Education (2011a). County Offices of Education Retrieved November 9,
2011, from http://www.cde.ca.gov/re/sd/co/coes.asp
California Department of Education (2011b). TUPE Competitive Grant Applicants: Safe and
Healthy Kids Program Office, California Department of Education.
California HealthyKids Resource Center (2010). Research Validated Titles. Retrieved July 27,
2010, from California HealthyKids Resource Center:
http://www.californiahealthykids.org/c/@MxTzHUbt3Nbj2/Pages/searchres.html
Center for Substance Abuse and Prevention (2001). CSAP’s prevention portal: Model programs.
Retrieved Dec. 12, 2001:
http://www.samhsa.gov/centers/csap/modelprograms/programs.cfm
Centers for Disease Control and Prevention (1994). Guidelines for school health programs to
prevent tobacco use and addiction. Retrieved May 19, 2001:
http://www.cdc.gov/nccdphp/dash/ptuaaag.htm
119

Chen, H-T. (1998). Theory-driven evaluations. Adv Educ Productivity 7, 15-34.
Cho, H., Hallfors, D. D., Iritani, B. J., & Hartman, S. (2009). The Influence of "No Child Left
Behind" Legislation on Drug Prevention in U.S. Schools. Eval Rev, 0193841X09335050.
Coburn, C., Toure, J., & Yamashita, M. (2009). Evidence, Interpretation, and Persuasion:
Instructional Decision Making at the District Central Office. Teachers College Record
111(4), 1115-1161.
Curry, S. J., Emery, S., Sporer, A. K., Mermelstein, R., Flay, B. R., Berbaum, M., et al. (2007). A
National Survey of Tobacco Cessation Programs for Youths. Am J Public Health, 97(1),
171-177.
Dane, A. V., & Schneider, B. H. (1998). Program integrity in primary and early secondary
prevention: are implementation effects out of control? Clinical Psychology Review,
18(1), 23-45.
Davies, H. T. O., & Nutley, S. M. (2008). Learning More about How Research-Based Knowledge
Gets Used: Guidance in the Development of New Empirical Research New York: William
T. Grant Foundation.
Dent, C. W., Sussman, S., & Stacy, A. W. (2001). Project Towards No Drug Abuse: Generalizability
to a General High School Sample. Preventive Medicine, 32(6), 514-520.
Domitrovich, C. E., Bradshaw, C. P., Poduska, J. M., Hoagwood, K., Buckley, J. A., Olin, S., et al.
(2008). Maximizing the Implementation Quality of Evidence-Based Preventive
Interventions in Schools: A Conceptual Framework. Advances in School Mental Health
Promotion, 1(3), 6-28.
Domitrovich, C. E., & Greenberg, M. T. (2000). The Study of Implementation: Current Findings
From Effective Programs that Prevent Mental Disorders in School-Aged Children. Journal
of Educational and Psychological Consultation, 11(2), 193 - 221.
Drug Strategies, I. (1999). Making the grade: A guide to school drug prevention programs.
Washington, DC: Drug Strategies, Inc.
Durlak, J., & DuPre, E. (2008). Implementation Matters: A Review of Research on the Influence
of Implementation on Program Outcomes and the Factors Affecting Implementation.
American Journal of Community Psychology, 41(3), 327-350.
Dusenbury, L., & Hansen, W. B. (2004). Pursuing the course from research to practice. Prev Sci,
5(1), 55-59.
Dusenbury, L., Brannigan, R., Falco, M., & Hansen, W. B. (2003). A review of research on fidelity
of implementation: implications for drug abuse prevention in school settings. Health
Educ. Res., 18(2), 237-256.
120

Ellickson, P. L. (1995). Schools. In R. H. Coombs & D. Ziedonis (Eds.), Handbook on Drug Abuse
Prevention: A Comprehensive Strategy to Prevent the Abuse of Alcohol and Other Drugs
(pp. 93-120). Needham Heights, MA: Ally & Bacon.
Ennett, S. T., Ringwalt, C. L., Thorne, J., Rohrbach, L. A., Vincus, A., Simons-Rudolph, A., et al.
(2003). A Comparison of Current Practice in School-Based Substance Use Prevention
Programs with Meta-Analysis Findings. Prevention Science, 4(1), 1-14.
Estabrooks, P. A., & Glasgow, R. E. (2006). Translating Effective Clinic-Based Physical Activity
Interventions into Practice. American Journal of Preventive Medicine, 31(4, Supplement
1), 45-56.
Fagan, A. A., Hanson, K., Hawkins, J. D., & Arthur, M. W. (2008). Bridging science to practice:
achieving prevention program implementation fidelity in the community youth
development study. American Journal of Community Psychology, 41(3-4), 235-249.
Fagan, A. A., Hanson, K., Hawkins, J. D., & Arthur, M. W. (2008b). Implementing Effective
Community-Based Prevention Programs in the Community Youth Development Study.
Youth Violence and Juvenile Justice, 6(3), 256-278.
Fagan, A. A., & Mihalic, S. (2003). Strategies for enhancing the adoption of school-based
prevention programs: Lessons learned from the blueprints for violence prevention
replications of the Life Skills Training program. Journal of Community Psychology, 31(3),
235-253.
Feinberg, M. E., Bontempo, D. E., & Greenberg, M. T. (2008). Predictors and Level of
Sustainability of Community Prevention Coalitions. American Journal of Preventive
Medicine, 34(6), 495-501.
Fixsen, D., Naoom, S. F., Blase, D. A., Friedman, R. M., & Wallace, F. (2005). Implementation
research: A synthesis of the literature. Tampa, Fl.: University of South Florida, Louis de la
Parte Florida Mental Health Institute, The National Implementation Research Network
(FMHI Publication #231).
Flay, B. (2009). School-based smoking prevention programs with the promise of long-term
effects. Tobacco Induced Diseases, 5(1), 6.
Ginexi, E., Mabry, P., Collins, L., Rivera, D., Ridenour, T., Valente, T., et al. (2010). Systems
Science Methodology for Prevention Science. Paper presented at the Society for
Prevention Research 18th Annual Meeting.
Gingiss, P. M., Roberts-Gray, C., & Boerm, M. (2006). Bridge-It: A System for Predicting
Implementation Fidelity for School-Based Tobacco Prevention Programs. Prevention
Science, 7(2), 197-207.
Gittelsohn, J., Merkle, S., Story, M., Stone, E. J., Steckler, A., Noel, J., et al. (2003). School climate
and implementation of the Pathways study. Preventive Medicine, 37(Supplement 1),
S97-S106.
121

Glasgow, R. E., Vogt, T. M., & Boles, S. M. (1999). Evaluating the public health impact of health
promotion interventions: the RE-AIM framework. Am J Public Health, 89(9), 1322-1327.
Glisson, C., & James, L. R. (2002). The cross-level effects of culture and climate in human service
teams. Journal of Organizational Behavior, 23(6), 767-794.
Glisson, C., Landsverk, J., Schoenwald, S., Kelleher, K., Hoagwood, K., Mayberg, S., et al. (2008).
Assessing the Organizational Social Context (OSC) of Mental Health Services:
Implications for Research and Practice. Administration and Policy in Mental Health and
Mental Health Services Research, 35(1), 98-113.
Gomez, B. J., Greenberg, M. T., & Feinberg, M. E. (2005). Sustainability of community coalitions:
An evaluation of communities that care. Prevention Science, 6(3), 199-202.
Goodman, R. M., & Steckler, A. B. (1989). A model for the institutionalization of health
promotion programs. Family Community Health, 11(4), 63-78.
Gottfredson, D. C., & Gottfredson, G. D. (2002). Quality of school-based prevention programs:
Results from a national survey. Journal of Research in Crime and Delinquency, 39(1), 3-
35.
Gottfredson, D. C., & Wilson, D. B. (2003). Characteristics of Effective School-Based Substance
Abuse Prevention. Prevention Science, 4(1), 27-38.
Greenberg, M. T. (2010). School-based prevention: current status and future challenges.
Effective Education, 2(1), 27-52.
Greenberg, M. T., Domitrovich, C. E., Graczyk, P. A., & Zins, J. E. (2005). The Study of
Implementation in School-Based Preventive Interventions: Theory, Research, and
Practice. Rockville, MD: Center for Mental Health Services, Substance Abuse and Mental
Health Services Administration.
Greenhalgh, T., Robert, G., Macfarlane, F., Bate, P., & Kyriakidou, O. (2004). Diffusion of
Innovations in Service Organizations: Systematic Review and Recommendations. The
Milbank Quarterly, 82(4), 581-629.
Gregory, A., Henry, D. B., & Schoeny, M. E. (2007). School climate and implementation of a
preventive intervention. American Journal of Community Psychology, 40(3-4), 250-260.
Gresham, F. M. (1989). Assessment of Treatment Integrity in School Consultation and
Prereferral Intervention. School Psychology Review, 18(1), 37-50.
Hahn, E. J., Noland, M. P., Rayens, M. K., & Christie, D. M. (2002). Efficacy of training and fidelity
of implementation of the life skills training program. The Journal of School Health, 72(7),
282.
Hallfors, D., & Godette, D. (2002). Will the `Principles of Effectiveness' improve prevention
practice? Early findings from a diffusion study. Health Educ. Res., 17(4), 461-470.
122

Hannaway, J. (1993). Political Pressure and Decentralization in Institutional Organizations: The
Case of School Districts. Sociology of Education, 66(3), 147-163.
Hansen, W. B., & McNeal, R. B., Jr. (1997). How D.A.R.E. works: an examination of program
effects on mediating variables. Health Educ Behav, 24(2), 165-176.
Harachi, T. W., Abbott, R. D., Catalano, R. F., Haggerty, K. P., & Fleming, C. B. (1999). Opening the
black box: Using process evaluation measures to assess implementation and theory
building. American Journal of Community Psychology. Special Issue: Prevention Science,
Part II, 27(5), 711-731.
Havelock, R. (1969). Planning for Innovation. Ann Arbor: Institute for Social Research, CRUSK,
University of Michigan.
Hawkins, J. D., Brown, E. C., Oesterle, S., Arthur, M. W., Abbott, R. D., & Catalano, R. F. (2008).
Early Effects of Communities That Care on Targeted Risks and Initiation of Delinquent
Behavior and Substance Use. Journal of Adolescent Health, 43(1), 15-22.
Honig, M. I., & Coburn, C. (2008). Evidence-Based Decision Making in School District Central
Offices: Toward a Policy and Research Agenda. Educational Policy, 22(4), 578-608.
Huberman, A. M. (1983). Improving Social Practice through the Utilization of University-Based
Knowledge. Higher Education, 12(3), 257-272.
Jasuja, G. K., Chou, C.-P., Bernstein, K., Wang, E., McClure, M., & Pentz, M. A. (2005). Using
structural characteristics of community coalitions to predict progress in adopting
evidence-based prevention programs. Evaluation and Program Planning, 28(2), 173-184.
Johnston, L. D., O’Malley, P. M., Bachman, J. G., & Schulenberg, J. E. (2009). Monitoring the
Future national results on adolescent drug use: Overview of key findings, 2008 (No. NIH
Publication No. 09-7401). Bethesda, MD: National Institute on Drug Abuse.
Kaftarian, S., Robertson, E., Compton, W., Davis, B. W., & Volkow, N. (2004). Blending
Prevention Research and Practice in Schools: Critical Issues and Suggestions Prev Sci,
5(1), 1-3.
Kallestad, J. H., & Olweus, D. (2003). Predicting Teachers' and Schools' Implementation of the
Olweus Bullying Prevention Program: A Multilevel Study. Prevention & Treatment, 6(1).
Kam, C.-M., Greenberg, M. T., & Walls, C. T. (2003). Examining the role of implementation
quality in school-based prevention using the PATHS curriculum. Prevention Science, 4(1),
55-63.
Klimes-Dougan, B., August, G. J., Lee, C.-Y. S., Realmuto, G. M., Bloomquist, M. L., Horowitz, J. L.,
et al. (Oct 2009). Practitioner and site characteristics that relate to fidelity of
implementation: The Early Risers prevention program in a going-to-scale intervention
trial. Professional Psychology: Research and Practice, 40(5), 467-475.
123

Lau, R. R., Hartman, K. A., & Ware, J. E. (1986). Health as a value: Methodological and theoretical
considerations. Health Psychology, 5, 25-43.
Lantz, P. M., Jacobson, P. D., Warner, K. E., Wasserman, J., Pollack, H. A., Berson, J., et al. (2000).
Investing in youth tobacco control: a review of smoking prevention and control
strategies. Tob Control, 9(1), 47-63.
Lillehoj, C. J., Griffin, K. W., & Spoth, R. (2004). Program Provider and Observer Ratings of
School-Based Preventive Intervention Implementation: Agreement and Relation to
Youth Outcomes. Health Educ Behav, 31(2), 242-257.
Little, M. A., & Rohrbach, L. ( Manuscript submitted for publication). Contextual Factors that
Influence the Adoption and Implementation of Evidence-based Prevention Programs in
Schools: A Review of the Literature.
McCarthy, W. J., Dietsch, B. J., Dent, C., Zheng, H., Bono, G., Bailey, J. D., et al. (2008). Evaluation
of the In-School Tobacco Use Prevention Education Program, 2005-2006: Technical
Findings and Documentation. Sacramento, Ca.
McGraw, S. A., Sellers, D. E., Johnson, C. C., Stone, E. J., Bachman, K. J., Bebchuk, J., et al. (1996).
Using Process Data To Explain Outcomes: An Illustration From the Child and Adolescent
Trial for Cardiovascular Health (CATCH). Eval Rev, 20(3), 291-312.
Meyer, J. W., & Scott, W. R. (1983). Organizational environments: Ritual and rationality. Beverly
Hills: Sage.
Mihalic, S., Fagan, A., & Argamaso, S. (2008). Implementing the LifeSkills Training drug
prevention program: factors related to implementation fidelity. Implementation Science,
3(1), 5.
Mowbray, C. T., Holter, M. C., Teague, G. B., & Bybee, D. (2003). Fidelity Criteria: Development,
Measurement, and Validation. American Journal of Evaluation, 24(3), 315-340.
Montano, D. E., & Kasprzyk, D. (2002). The Theory of Reasoned Action and the Theory of
Planned Behavior. In K. Glanz, B. K. Rimer & F. M. Lewis (Eds.), Health Behavior and
Health Education, Theory, Research and Practice (3rd ed., pp. 67 - 98). San Francisco:
Jossey-Bass.
Murray, D. M., & Hannan, P. J. (1990). Planning for the appropriate analysis in school-based
drug-use prevention studies. Journal of Consulting & Clinical Psychology, 58(4), 458-468.
National Cancer Institute, & SAMHSA (2010). Research-Tested Intervention Programs. Retrieved
July 27, 2010, from U.S. Department of Health and Human Services:
http://rtips.cancer.gov/rtips/index.do
National Center for Education Statistics (2004). Common Core of Data: Local education agency
(school district) universe survey data, 2002–03 [Data file]. Retrieved February 21, 2007
from http://nces.ed.gov/ccd/pubagency.asp.:
124

National Institute on Drug Abuse, & National Institutes of Health (1997). Preventing drug use
among children and adolescents: A research-based guide. Rockville, MD: National
Institute on Drug Abuse.
National Institute on Drug Abuse, National Institute of Health, & U.S. Department of Health and
Human Services (2010). Preventing Drug Use Among Children and Adolescents: A
Research-Based Guide. Retrieved July 27, 2010, from U.S. Department of Health and
Human Services: http://www.nida.nih.gov/Prevention/Prevopen.html
National Research Council and Institute of Medicine. (2009). Preventing Mental, Emotional, and
Behavioral Disorders Among Young People: Progress and Possibilities. Committee on the
Prevention of Mental Disorders and Substance Abuse Among Children, Youth, and
Young Adults: Research Advances and Promising Interventions. Mary Ellen O’Connell,
Thomas Boat, and Kenneth E. Warner, Editors. Board on Children, Youth, and Families,
Division of Behavioral and Social Sciences and Education. Washington, DC: The National
Academies Press.
O'Loughlin, J., Renaud, L., Richard, L., Gomez, L. S., & Paradis, G. (1998). Correlates of the
Sustainability of Community-Based Heart Health Promotion Interventions. Preventive
Medicine, 27(5), 702-712.
Office of Behavioral and Social Sciences, National Institutes of Health, & Office of the Director
(2005). Strategic Plan. Available at http://obssr.od.nih.gov/about/plan.html.
Office of Safe and Drug Free Schools, & United States Department of Education (2010).
Exemplary and Promising: Safe, Disciplined, and Drug-Free Schools Programs. Retrieved
July 27, 2010, from United States Department of Education:
http://www2.ed.gov/admins/lead/safety/exemplary01/report_pg9.html
Pankratz, M., Hallfors, D., & Cho, H. (2002). Measuring perceptions of innovation adoption: the
diffusion of a federal drug prevention policy. Health Educ. Res., 17(3), 315-326.
Patterson, M. G., West, M. A., Shackleton, V. J., Dawson, J. F., Lawthom, R., Maitlis, S., et al.
(2005). Validating the organizational climate measure: links to managerial practices,
productivity and innovation. Journal of Organizational Behavior, 26(4), 379-408.
Payne, A., & Eckert, R. (2010). The Relative Importance of Provider, Program, School, and
Community Predictors of the Implementation Quality of School-Based Prevention
Programs. Prevention Science, 11(2), 126-141.
Payne, A. (2009). Do Predictors of the Implementation Quality of School-Based Prevention
Programs Differ by Program Type? Prevention Science, 10(2), 151-167.
Payne, A. A., Gottfredson, D. C., & Gottfredson, G. D. (2006). School Predictors of the Intensity of
Implementation of School-Based Prevention Programs: Results from a National Study.
Prevention Science, 7(2), 225-237.
125

Pentz, M. A., Trebow, E. A., Hansen, W. B., MacKinnon, D. P., Dwyer, J. H., Johnson, C. A., et al.
(1990). Effects of Program Implementation on Adolescent Drug Use Behavior: The
Midwestern Prevention Project (MPP). Eval Rev, 14(3), 264-289.
Pluye, P., Potvin, L., Denis, J.-L., Pelletier, J., & Mannoni, C. (2005). Program sustainability begins
with the first events. Evaluation and Program Planning, 28(2), 123-137.
Pluye, P., Potvin, L., & Denis, J.-L. (2004). Making public health programs last: conceptualizing
sustainability. Evaluation and Program Planning, 27(2), 121-133.
Quality Education Direct Inc. (2009). QED California District 2008 Database: Data users guide.
Denver, CO: Quality Education Direct Inc.
Quality Education Data Inc. (1998). QED national education database: Data users guide (Version
4.6 ed.). Denver, CO: Quality Education Data, Inc.
Riggs, N., Nakawatase, M., & Pentz, M. (2008). Promoting Community Coalition Functioning:
Effects of Project STEP. Prevention Science, 9(2), 63-72.
Ringwalt, C., Vincus, A., Hanley, S., Ennett, S., Bowling, J., & Haws, S. (2011). The Prevalence of
Evidence-based Drug Use Prevention Curricula in U.S. Middle Schools in 2008.
Prevention Science, 12(1), 63-69.
Ringwalt, C., Vincus, A., Hanley, S., Ennett, S., Bowling, J., & Rohrbach, L. (2009). The Prevalence
of Evidence-Based Drug Use Prevention Curricula in U.S. Middle Schools in 2005.
Prevention Science, 10(1), 33-40.
Ringwalt, C., Hanley, S., Vincus, A., Ennett, S., Rohrbach, L., & Bowling, J. (2008). The Prevalence
of Effective Substance Use Prevention Curricula in the Nation’s High Schools. The Journal
of Primary Prevention, 29(6), 479-488.
Ringwalt, C. L., Ennett, S., Johnson, R., Rohrbach, L. A., Simons-Rudolph, A., Vincus, A., et al.
(2003). Factors Associated with Fidelity to Substance Use Prevention Curriculum Guides
in the Nation's Middle Schools. Health Educ Behav, 30(3), 375-391.
Ringwalt, C. L., Ennett, S., Vincus, A., Thorne, J., Rohrbach, L. A., & Simons-Rudolph, A. (2002).
The Prevalence of Effective Substance Use Prevention Curricula in U.S. Middle Schools.
Prevention Science, 3(4), 257-265.
Ringwalt, C. L., Pankratz, M. M., Hansen, W. B., Dusenbury, L., Jackson-Newsom, J., Giles, S. M.,
et al. (2009). The Potential of Coaching as a Strategy to Improve the Effectiveness of
School-Based Substance Use Prevention Curricula. Health Educ Behav, 36(4), 696-710.
Roberts-Gray, C., Gingiss, P. M., & Boerm, M. (2007). Evaluating school capacity to implement
new programs. Evaluation and Program Planning, 30(3), 247-257.
Rogers, E. M. (1983). Diffusion of Innovations (2nd ed.). New York: The Free Press.
Rogers, E. M. (2002). Diffusion of preventive innovations. Addictive Behaviors, 27(6), 989-993.
126

Rohrbach, L. A., D'Onofrio, C. N., Backer, T. E., & Montgomery, S. B. (1996). Diffusion of School-
Based Substance Abuse Prevention Programs. American Behavioral Scientist, 39(7), 919-
934.
Rohrbach, L. A., Dent, C. W., Johnson, C. A., Unger, J., & Gunning, G. (1998). Evaluation of the
School Tobacco Use Prevention Education program. In: Independent Evaluation
Consortium, Final Report: Independent Evaluation of the California Tobacco Control
Program. Wave 1 Data, 1996-1997. Sacramento, CA: California Department of Health
Services, Tobacco Control Section.
Rohrbach, L. A., Graham, J. W., & Hansen, W. B. (1993). Diffusion of a School-Based Substance
Abuse Prevention Program: Predictors of Program Implementation. Preventive
Medicine, 22(2), 237-260.
Rohrbach, L. A., Grana, R., Sussman, S., & Valente, T. W. (2006). Type II Translation: Transporting
Prevention Interventions From Research to Real-World Settings. Eval Health Prof, 29(3),
302-333.
Rohrbach, L. A., Gunning, M., Grana, R., Gunning, G., & Sussman, S. (2010). Dissemination of
Project Towards No Drug Abuse (TND): Findings From a Survey of Program Adopters.
Subst Use Misuse, 45(14), 2551-2566.
Rohrbach, L., Gunning, M., Sun, P., & Sussman, S. (2010). Erratum to: The Project Towards No
Drug Abuse (TND) Dissemination Trial: Implementation Fidelity and Immediate
Outcomes. Prevention Science, 11(1), 113-113.
Rohrbach, L. A., Sun, P., & Sussman, S. (2010). One-year follow-up evaluation of the Project
Towards No Drug Abuse (TND) dissemination trial. Preventive Medicine, 51(3-4), 313-
319.
Rohrbach, L. A., Gunning, M. A., Sussman, S., & Sun, P. (2008). Predictors of Implementation
Fidelity in the Project Towards No Drug Abuse (TND) Dissemination Trial. Paper
presented at the Presented at the 16th Annual Society for Prevention Research
Conference, San Francisco, CA.
Rohrbach, L. A., Gunning, M. A., Grana, R., & Sussman, S. (2007). Taking Project Towards No
Drug Abuse (TND) to Scale: Program Adoption and Implementation. Paper presented at
the Poster presented at the 15th Annual Society for Prevention Research Conference,
Washington, DC.
Rohrbach, L. A., Ringwalt, C. L., Ennett, S. T., & Vincus, A. A. (2005). Factors associated with
adoption of evidence-based substance use prevention curricula in US school districts.
Health Educ. Res., 20(5), 514-526.
Rohrbach, L. A., Sussman, S., & Dent, C. W. (2005). Tobacco, Alcohol, and Other Drug Use among
High-Risk Young People: A Five-Year Longitudinal Study from Adolescence to Emerging
Adulthood. Journal of Drug Issues, 35(2), 333-355.
127

Rohrbach, L. A., Howard-Pitney, B., Unger, J. B., Dent, C. W., Howard, K. A., Cruz, T. B., et al.
(2002). Independent Evaluation of the California Tobacco Control Program:
Relationships Between Program Exposure and Outcomes, 1996-1998. Am J Public
Health, 92(6), 975-984.
Safe and Drug Free Schools Program, U. S. D. o. E. (2001). Retrieved June 26, 2001:
http://www.ed.gov/offices/OESE/SDFS/programs.html
SAS Institute Inc. SAS/C Online Doc TM (2000). Release 9.0. Cary, NC: SAS Institute.
Scheirer, M. A. (2005). Is Sustainability Possible? A Review and Commentary on Empirical
Studies of Program Sustainability. American Journal of Evaluation, 26(3), 320-347.
Scheirer, M. A., & Dearing, J. W. (2011). An Agenda for Research on the Sustainability of Public
Health Programs. Am J Public Health.
Schoenwald, S. K., & Hoagwood, K. (2001). Effectiveness, Transportability, and Dissemination of
Interventions: What Matters When? Psychiatr Serv, 52(9), 1190-1197.
Shediac-Rizkallah, M. C., & Bone, L. R. (1998). Planning for the sustainability of community-
based health programs: conceptual frameworks and future directions for research,
practice and policy. Health Educ. Res., 13(1), 87-108.
Skara, S., & Sussman, S. (2003). A review of 25 long-term adolescent tobacco and other drug use
prevention program evaluations. Preventive Medicine, 37(5), 451-474.
Shinn, M. (2003). Understanding Implementation of Programs in Multilevel Systems. Prevention
& Treatment, 6(1).
Shinn, M., & Toohey, S. M. (2003). Community Contexts of Human Welfare. Annual Review of
Psychology, 54(1), 427-459.
Spillane, J. P. (1998). State Policy and the Non-Monolithic Nature of the Local School District:
Organizational and Professional Considerations. American Educational Research Journal,
35(1), 33-63.
Spoth, R., Greenberg, M., Bierman, K., & Redmond, C. (2004). PROSPER Community–University
Partnership Model for Public Education Systems: Capacity-Building for Evidence-Based,
Competence-Building Prevention. Prevention Science, 5(1), 31-39.
Spoth, R., Redmond, C., Shin, C., Greenberg, M., Clair, S., & Feinberg, M. (2007). Substance-Use
Outcomes at 18 Months Past Baseline: The PROSPER Community-University Partnership
Trial. American Journal of Preventive Medicine, 32(5), 395-402.
St. Pierre, T. L., & Kaltreider, D. L. (2004). Tales of Refusal, Adoption, and Maintenance:
Evidence-Based Substance Abuse Prevention Via School-Extension Collaborations.
American Journal of Evaluation, 25(4), 479-491. Shediac-Rizkallah, M. C., & Bone, L. R.
(1998). Planning for the sustainability of community-based health programs: conceptual
128

frameworks and future directions for research, practice and policy. Health Educ. Res.,
13(1), 87-108.
Stanton, W. R., Lowe, J. B., & Gillespie, A. M. (1996). Adolescents’ experiences of smoking
cessation. Drug and Alcohol Dependence, 43, 63-70.
Steckler, A., Goodman, R. M., McLeroy, K. R., Davis, S., & Koch, G. (1992). Measuring the
diffusion of innovative health promotion programs. American Journal of Health
Promotion, 6, 214-224.
Stith, S., Pruitt, I., Dees, J., Fronce, M., Green, N., Som, A., et al. (2006). Implementing
Community-Based Prevention Programming: A Review of the Literature. The Journal of
Primary Prevention, 27(6), 599-617.
Substance Abuse and Mental Health Services Administration, & United States Department of
Health and Human Services (2010). SAMHSA Model Programs. Retrieved July 27, 2010,
from United States Department of Health and Human Services:
http://nrepp.samhsa.gov/
Sun, P., Sussman, S., Dent, C. W., & Rohrbach, L. A. (2008). One-year follow-up evaluation of
Project Towards No Drug Abuse (TND-4). Preventive Medicine, 47(4), 438-442.
Sun, W., Skara, S., Sun, P., Dent, C. W., & Sussman, S. (2006). Project Towards No Drug Abuse:
Long-term substance use outcomes evaluation. Preventive Medicine, 42(3), 188-192.
Sussman, S., & Dent, C. W. (1996). Correlates of addiction concern among adolescents at high
risk for substance abuse. Journal of Substance Abuse, 8, 361-370.
Sussman, S., Dent, C. W., & Stacy, A. W. (1996). The relation of pro-drug use myths with self-
reported drug use among youth at continuation high schools. Journal of Applied Social
Psychology, 26, 214-267.
Sussman, S., Dent, C. W., & Stacy, A. W. (2002). Project Towards No Drug Abuse: A review of the
findings and future directions. American Journal of Health Behavior, 26(5), 354-365.
Sussman, S., Dent, C. W., Stacy, A. W., & Craig, S. (1998). One-Year Outcomes of Project Towards
No Drug Abuse. Preventive Medicine, 27(4), 632-642.
Sussman, S., McCuller, W. J., & Dent, C. W. (2003). The associations of social self-control,
personality disorders, and demographics with drug use among high-risk youth. Addictive
Behaviors, 28, 1159-1166.
Sussman, S., Sun, P., Rohrbach, L. A., & Spruijt-Metz, D. (2011). One-year outcomes of a drug
abuse prevention program for older teens and emerging adults: Evaluating a
motivational interviewing booster component. Health Psychol.
129

Sussman, S., Valente, T. W., Pentz, M. A., Rohrbach, L. A., & Skara, S. (2006). Translation in the
health professions: Converting science into action. Evaluation and the Health
Professions, 29(1), 7-32.
Swisher, J. D. (2000). Sustainability of prevention. Addictive Behaviors, 25(6), 965-973.
Thaker, S., Steckler, A., Sanchez, V., Khatapoush, S., Rose, J., & Hallfors, D. D. (2008). Program
characteristics and organizational factors affecting the implementation of a school-
based indicated prevention program. Health Education Research, 23(2), 238-248.
Thomas, J. M., Sable, J., Dalton, B., & Sietsema, J. (2007). Documentation to the NCES Common
Core of Data Local Education Agency Universe Survey: School Year 2004–05 (NCES 2006-
440rev). Washington, DC.: National Center for Education Statistics, Institute of
Education Sciences, U.S. Department of Education.
Tobler, N. S., Roona, M. R., Ochshorn, P., Marshall, D. G., Streke, A. V., & Stackpole, K. M. (2000).
School-Based Adolescent Drug Prevention Programs: 1998 Meta-Analysis. The Journal of
Primary Prevention, 20(4), 275-336.
Tobler, N. S., & Stratton, H. H. (1997). Effectiveness of School-Based Drug Prevention Programs:
A Meta-Analysis of the Research. The Journal of Primary Prevention, 18(1), 71-128.
United States Department of Education (1998). Safe and drug-free schools program: Notice of
final principles of effectiveness.
United States Department of Health and Human Services, & SAMHSA (2010, April 4, 2010).
National Registry of Evidence-based Programs and Practices (NREPP) Retrieved April 22,
2010, from http://nrepp.samhsa.gov/index.asp
United States Department of Health and Human Services Substance Abuse, M. H. S. A. (2010,
April 4, 2010). National Registry of Evidence-based Programs and Practices (NREPP)
Retrieved April 22, 2010, from http://nrepp.samhsa.gov/index.asp
United States Census Bureau (2000). Census 2000: census regions cartographic boundary files
Retrieved 2007, December 17 from http://www.census.gov/geo/www/cob/rg2000.html
Valente, T. W., Okamoto, J., Pumpuang, P., Okamoto, P., & Sussman, S. (2007). Differences in
perceived implementation of a standard versus peer-led interactive substance abuse
prevention program. Am J Health Behav, 31(3), 297-311.
Victora, C. G., Huttly, S. R., Fuchs, S. C., & Olinto, M. T. (1997). The role of conceptual
frameworks in epidemiological analysis: a hierarchical approach. Int J Epidemiol, 26(1),
224-227.
Wiecha, J. L., El Ayadi, A. M., Fuemmeler, B. F., Carter, J. E., Handler, S., Johnson, S., et al. (2004).
Diffusion of an Integrated Health Education Program in an Urban School System: Planet
Health. J. Pediatr. Psychol., 29(6), 467-474.
130

Wills, T. A. (1986). Stress and coping in early adolescence: Relationships to substance use in
urban school samples. Health Psychology, 5, 503-529.
Woolf, S. H. (2008). The Meaning of Translational Research and Why It Matters. JAMA, 299(2),
211-213.

131

APPENDIX A. LIST OF RESEARCH-VALIDATED TOBACCO AND SUBSTANCE USE PREVENTION
PROGRAMS IN 2010

1.
Adolescent Transitions Program (ATP)
2.
Adolescents Training and Learning to Avoid Steroids (ATLAS)
3.
All Stars
4.
Athletes Training and Learning to Avoid Steroids (ATLAS)
5.
Botvin's Life Skills Training
6.
Caring School Community Program (Formerly, Child Development Project)
7.
Class Action
8.
Classroom-Centered (CC) and Family-School Partnership (FSP) Intervention
9.
Coping Power
10.
Creating Lasting Family Connections
11.
Early Risers “Skills for Success” Risk Prevention Program
12.
Family Matters
13.
Fast Track Prevention Trial for Conduct Problems
14.
Focus on Families (FOF)
15.
Guiding Good Choices (GGC)
16.
Head On
17.
I'm Special
18.
Keeping it Real
19.
Kentuky Adolescent Tobacco Prevention Project
20.
Lions-Quest Skills for Adolescence (SFA)
21.
Minnesota Smoking Prevention Program
22.
Pathways to Health
23.
Personal/Social Skills Lessons: Missing Link
24.
Positive Action
25.
Project Alert
26.
Project Ex
27.
Project Northland
28.
Project SHOUT (Students Helping Others Understand Tobacco)
29.
Project STAR
30.
Project SUCCESS
31.
Project Towards No Drug Abuse (Project TND)
32.
Project Towards No Tobacco Use (Project TNT)
33.
Project Venture
34.
Promoting Alternative Thinking Strategies (PATHS)
35.
Reconnecting Youth Program (RY)
36.
Residential Student Assistance Program
37.
Sembrando Salud
38.
Skills, Opportunity, And Recognition (SOAR)
39.
Spit Tobacco Intervention for High School Athletes

40.
Teens Tackle Tobacco: Triple T
41.
The Strengthening Families Program: For Parents and Youth 10–14 (SFP 10–14)
42.
Too Good for Drugs
43.
Too Good for Violence
NOTES: Registries included: California Healthy Kids Resource Center (California HealthyKids Resource Center, 2010),
National Registry of Evidence-based Programs and Practices (NREPP) (Substance Abuse and Mental Health Services
Administration & United States Department of Health and Human Services, 2010), Exemplary and Promising: Safe,
Disciplined, and Drug-Free Schools Programs (Office of Safe and Drug Free Schools & United States Department of
132

Education, 2010), Research-Tested Intervention Programs (National Cancer Institute & SAMHSA, 2010), and
Preventing Drug Use Among Children and Adolescents: A Research-Based Guide (National Institute on Drug Abuse,
National Institute of Health, & U.S. Department of Health and Human Services, 2010)

133

APPENDIX B. LIST OF APPROVED TOBACCO CESSATION PROGRAMS IN 2010
1. California Smokers’ Helpline
2. Helping Teens Stop Using Tobacco (TAP)
3. I Decide: Teen Tobacco Cessation program
4. I Quit
5. Intervening with Teen Tobacco Users (TEG)
6. Not-On-Tobacco (N-O-T)
7. Project Ex
8. Smokeless School Days: Smokeless Saturday School
9. Spit Tobacco Intervention for High School Athletes

NOTES: Program was approved if listed in one of the following registries: California Healthy Kids Resource Center
(California HealthyKids Resource Center, 2010), National Registry of Evidence-based Programs and Practices (NREPP)
(Substance Abuse and Mental Health Services Administration & United States Department of Health and Human
Services, 2010), Exemplary and Promising: Safe, Disciplined, and Drug-Free Schools Programs (Office of Safe and Drug
Free Schools & United States Department of Education, 2010), Research-Tested Intervention Programs (National
Cancer Institute & SAMHSA, 2010), and Preventing Drug Use Among Children and Adolescents: A Research-Based
Guide (National Institute on Drug Abuse, National Institute of Health, & U.S. Department of Health and Human
Services, 2010). Additionally, a program was considered approved if it was listed on the TUPE program 2009
competitive grant application as an approved program for tobacco cessation for grades 7-12.

Abstract (if available)

Abstract Although considerable resources have been spent developing and disseminating effective school-based substance use prevention programs, many school districts in the United States fail to use prevention programs with proven effectiveness. Moreover, there is a dearth of research understanding the factors that promote adoption, implementation fidelity, and sustained use of these programs in schools. Consequently, a significant gap remains in what we know about how to effectively “translate” evidence-based programs from research to practice (L. A. Rohrbach, Grana, Sussman, & Valente, 2006). ❧ Research grounded in the diffusion of innovations theory (Rogers, 1983), theory driven evaluations (Chen, 1998), systems models (Estabrooks & Glasgow, 2006) and recent reviews of the literature (Durlak & DuPre, 2008) have identified some community-, organizational-, and individual-level factors that are correlated with the translation of evidence-based programs in schools. The studies presented here explore the relationships between factors at several levels of the ecological framework and program adoption, implementation fidelity and sustained use of evidence-based substance use prevention and tobacco cessation programs in schools. ❧ The findings presented in these studies suggest that school-based substance use prevention programming is influenced by a variety of contextual factors occurring at multiple ecological levels. Characteristics of the community and organization, beliefs of key decision makers, and funding were important factors in promoting adoption, implementation fidelity and sustained use of evidence-based substance use prevention programs in schools. In order to move the field ahead, researchers need to account for the multiple systems at play in schools when designing research trials and school-based programs. Ultimately, translational research can be used to increase the use of evidence-based prevention programs in schools, which should lead to reductions in substance use and other problem behaviors among adolescents.

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

Social self-control and adolescent substance use

PDF

Motivational interviewing with adolescent substance users: a closer look

PDF

Exploring the role of peer influence, linguistic acculturation, and social networks in substance use

PDF

Contextualizing social network influences on substance use among high risk adolescents

PDF

Sociocultural stress, coping and substance use among Hispanic/Latino adolescents

PDF

The dynamic relationship of emerging adulthood and substance use

PDF

The role of depression symptoms on social information processing and tobacco use among adolescents

PDF

Examining tobacco regulation opinions and policy acceptance among key opinion leaders and tobacco retailers in low socioeconomic status African American, Hispanic, and non-Hispanic White communities

PDF

The Internet activities, gratifications, and health consequences related to compulsive Internet use

PDF

The effects of mindfulness on adolescent cigarette smoking: Measurement, mechanisms, and theory

PDF

Energy drink consumption, substance use and attention-deficit/hyperactivity disorder among adolescents

PDF

Post-traumatic growth among high-risk youth: predictors, impact of stressful life events, and relationship with changes in substance use behaviors

PDF

Smoke-free housing policies and secondhand smoke exposure in low income multiunit housing in Los Angeles County

PDF

Tobacco and marijuana surveillance using Twitter data

PDF

Intrapersonal and environmental factors associated with Chinese youth alcohol use experimentation and binge drinking behaviors

PDF

Delineating the processes and mechanisms of sustainment: understanding how evidence-based prevention initiatives are maintained after initial funding ends

PDF

Role transitions, past life events, and their associations with multiple categories of substance use among emerging adults

PDF

A network analysis of online and offline social influence processes in relation to adolescent smoking and alcohol use

PDF

Psychosocial and cultural factors in the primary prevention of melanoma targeted to multiethnic children

PDF

The Filipino family initiative: preliminary effects of an evidence-based parenting intervention offered in churches on parent and child outcomes

Asset Metadata

Creator Little, Melissa A. (author)

Core Title The influence of contextual factors on the processes of adoption and implementation of evidence-based substance use prevention and tobacco cessation programs in schools

School Keck School of Medicine

Degree Doctor of Philosophy

Degree Program Preventive Medicine (Health Behavior)

Publication Date 04/24/2012

Defense Date 12/13/2011

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag Adoption,implementation,OAI-PMH Harvest,Prevention,school-based,substance use,tobacco cessation,translation

Language English

Contributor Electronically uploaded by the author (provenance)

Advisor Rohrbach, Louise Ann (committee chair), Palinkas, Lawrence A. (committee member), Riggs, Nathaniel (committee member), Sun, Ping (committee member), Sussman, Steven (committee member)

Creator Email mgunning@usc.edu,mlittle@cc.hawaii.edu

Permanent Link (DOI) https://doi.org/10.25549/usctheses-c3-9215

Unique identifier UC11288341

Identifier usctheses-c3-9215 (legacy record id)

Legacy Identifier etd-LittleMeli-632-0.pdf

Dmrecord 9215

Document Type Dissertation

Rights Little, Melissa A.

Type texts

Source University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Access Conditions The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the a...

Repository Name University of Southern California Digital Library

Repository Location USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA