Close
About
FAQ
Home
Collections
Login
USC Login
Register
0
Selected
Invert selection
Deselect all
Deselect all
Click here to refresh results
Click here to refresh results
USC
/
Digital Library
/
University of Southern California Dissertations and Theses
/
Collective Impact: a framework to advance health promotion in higher education
(USC Thesis Other)
Collective Impact: a framework to advance health promotion in higher education
PDF
Download
Share
Open document
Flip pages
Contact Us
Contact Us
Copy asset link
Request this asset
Transcript (if available)
Content
Copyright 2020 Amanda Katherine Vanni
COLLECTIVE IMPACT: A FRAMEWORK TO ADVANCE HEALTH PROMOTION IN
HIGHER EDUCATION
by
Amanda Katherine Vanni
A Dissertation Presented to the
FACULTY OF THE ROSSIER SCHOOL OF EDUCATION
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulfillment of the
Requirements for the Degree
DOCTOR OF EDUCATION
May 2020
ii
Dedication
To realizing communal wellbeing as the foundation and education as the accelerator to equity.
iii
Acknowledgements
To the many individuals who have encouraged, supported, and remained by my side
through this arduous journey. To my biggest ally and partner through this journey and many
more, Matthew. We simultaneously made it through three years of residency and a doctorate
program. While these years felt like the most challenging yet, I have no doubt that they have
made us stronger, more resilient, and more ready for our next phase of life. Thank you for being
my rock, thank you for spending late nights up together studying and working, thank you for
reminding me to take breaks, and for helping me to always keep things in perspective. To my
family for cheering me on each step of the way and for answering my late night phone calls. To
my friends, who are still my friends after years of “not this weekend” or “hopefully soon.” To
my supervisor and HPS colleagues. You have weathered this process with me as much as
anyone, and I owe you all an immense amount of gratitude for your support and patience.
To my Rosier colleagues and faculty. Four strong, empowered women inspired me,
cheered me on, gave me career and professional advice, and provided me the assurance that we
would get through this together. Thank you to my SCquad: Christine Ruth, Chelsea Heyward,
Leslie Hughes, and Kelly Esperias. Another three women have guided my journey, supported my
understanding of academic rigor and excellence, and paved the way for many more women to
accomplish great things, as they have. Sincere appreciation and gratitude is extended to my
dissertation committee: Dr. Stowe (chair), Dr. Malloy, and Dr. Kumar.
iv
Table of Contents
Dedication ........................................................................................................................... ii
Acknowledgements ............................................................................................................ iii
List of Tables .................................................................................................................... vii
List of Figures .................................................................................................................. viii
List of Abbreviations ......................................................................................................... ix
Abstract ................................................................................................................................x
Chapter One: Overview of the Study ...................................................................................1
Organizational Context and Mission .............................................................................1
Organizational Performance Goal ..................................................................................2
Related Literature...........................................................................................................3
Importance of the Evaluation .........................................................................................5
Description of Stakeholder Groups ................................................................................7
Stakeholder for the Study and Stakeholder Performance Goal .....................................8
Purpose of the Project and Questions ............................................................................9
Methodological Framework .........................................................................................10
Definitions....................................................................................................................10
Organization of the Study ............................................................................................12
Chapter Two: Review of Literature ...................................................................................13
Health and Wellbeing ..................................................................................................13
Defining Health and Wellbeing .............................................................................14
Health Promotion .........................................................................................................16
Health Promotion and the Settings Approach ........................................................17
Health Promotion in Higher Education..................................................................19
Collective Impact .........................................................................................................24
Conditions, Pre-conditions and Principles for Collective Impact ..........................24
Collective Impact: A Promising Approach ............................................................26
Clark and Estes Gap Analytic Conceptual Framework ...............................................28
Stakeholder Knowledge, Motivation, and Organizational Factors ..............................30
Knowledge and Skills ............................................................................................30
Motivation ..............................................................................................................36
Organization Influences .........................................................................................39
Conceptual Framework: Interaction of Knowledge, Motivation and Organizational
Context .............................................................................................................44
Application to the University Setting ....................................................................46
Chapter Three: Methodology .............................................................................................49
Participating Stakeholders ...........................................................................................51
v
Data Collection ............................................................................................................54
Interview Protocol ..................................................................................................54
Interview Procedures .............................................................................................56
Credibility and Trustworthiness ...................................................................................57
Ethics ..........................................................................................................................58
Chapter Four: Findings ......................................................................................................61
Participating Stakeholders ...........................................................................................61
Findings........................................................................................................................62
Finding 1: Collective Impact Creates an Infrastructure for Collaboration and has
Motivational Effects when Aligned to Departmental Goals ............................64
Finding 2: To Advance the Wellbeing of the Student Population via Collective
Impact, the Common Agenda Needs to be a University-wide Initiative ..........66
SC members Need Commitment and Expertise from Campus Partners ................68
University-wide initiatives need senior leadership buy-in and support .................70
University-wide priorities conveyed by accountability .........................................72
Finding 3: Progress Made and Opportunities Exist to Fully Establish the Five
Conditions of Collective Impact to Advance Student Wellbeing ....................74
Common Agenda: Calls Attention to a Focal Point and Creates Sustainability ....74
Mutually Reinforcing Strategies: Unit-Specific Alignment without a
Coordinated Plan ........................................................................................75
Continuous Communication: Important and Challenging .....................................78
Shared Measurement System: Data Used for Equity and Engagement .................80
Backbone Organization Supports the Four Conditions .........................................86
Chapter Five: Discussion ...................................................................................................92
Recommendations for Practice to Address Knowledge Influences .............................93
Provide a Job Aid to Identify Campus Partners and their Roles............................94
Provide Training on Applying Mapping Techniques ............................................95
Facilitate Peer Modeling and Discussion of Data as a Tool ..................................95
Recommendations for Practice to Address Motivation Influences..............................96
Increase Utility Value of the Common Agenda for SC Members .........................97
Increase Collective Efficacy of the Steering Committee via a Coordinated
Action Plan and Powerful Sponsor ............................................................99
Recommendations for Practice to Address Organizational Influences ......................100
Create Feedback Loops to Foster a Culture of Collective Learning and
Adaptability..............................................................................................102
Embed Wellbeing throughout University Structures and Communication
Mechanisms .............................................................................................102
Summary of KMO Recommendations.......................................................................104
Integrated Implementation and Evaluation Plan ........................................................105
Level 4: Results and Leading Indicators ..............................................................106
Level 3: Critical Behaviors ..................................................................................108
Level 2: Learning .................................................................................................114
Level 1: Reaction .................................................................................................119
Evaluation Tools ..................................................................................................119
vi
Data Analysis and Reporting ...............................................................................121
Summary ..............................................................................................................122
Limitations and Delimitations ....................................................................................123
Future Research .........................................................................................................125
Conclusion .................................................................................................................127
References ........................................................................................................................130
Appendices .......................................................................................................................143
Appendix A: Interview Protocol ................................................................................143
Appendix B: Survey Protocol - Immediate ................................................................147
Appendix C: Survey Protocol - Delayed ...................................................................149
Appendix D: Data Analysis Chart .............................................................................151
vii
List of Tables
Table 1: Organizational Mission, Global Goal and Stakeholder Performance Goals 8
Table 2: Knowledge Influence, Knowledge Type, and Knowledge Assessment 35
Table 3: Motivation Influence and Motivation Influence Assessment 39
Table 4: Organization Influences and Organization Influence Assessment 43
Table 5: Summary of Findings with embedded Knowledge, Motivation, and
Organizational Influences 63
Table 6: Summary of Knowledge, Motivation, and Organizational Influences 89
Table 7: Summary of Knowledge Influences and Recommendations 94
Table 8: Summary of Motivation Influences and Recommendations 97
Table 9: Summary of Organization Influences and Recommendations 101
Table 10: Summary of KMO Influences and Recommendations 104
Table 11: Outcomes, Metrics, and Methods for External and Internal Outcomes 105
Table 12: Critical Behaviors, Metrics, Methods, and Timing for Evaluation 109
Table 13: Required Drivers to Support Critical Behaviors 112
Table 14: Evaluation of the Components of Learning for the Program 118
Table 15: Components to Measure Reactions to the Program 119
viii
List of Figures
Figure 1: Example of a University System as a Setting. 18
Figure 2: Illustration of Health Promotion Actions to Health Promotion Outcomes to
Health Outcomes. 22
Figure 3: Five Conditions, Three Preconditions, and Three Phases of Collective Impact. 25
Figure 4: Conceptual framework of cross-sector collaboration for social change to
promote population health. 42
Figure 5: Knowledge, Motivation, and Organization Factors related to Collective Impact
for student wellbeing in Higher Education 46
ix
List of Abbreviations
USSS University of Siloed Student Services
DSA Division of Student Affairs
SC Steering Committee
KMO Knowledge, Motivation, and Organization
WHO World Health Organization
x
Abstract
In the United States, institutions of higher education are increasingly faced with complex public
health issues among their student populations, yet have failed to devise solutions that account for
the dynamic interplay of individual, community, and environmental factors. This dissertation
examined one university’s implementation of a framework for collaboration – Collective Impact
– that has shown promise in addressing complex social issues in other settings. The organization
of study is a large, urban university in the Western United States that seeks to address four
student-focused public health issues utilizing the Collective Impact framework. A qualitative
case study research design was used. Data collection included in-depth interviews with key
stakeholders who functioned together as the Steering Committee. Three research questions
guided this study and addressed 1) the extent to which the Steering Committee had met its goal
of establishing all five conditions of Collective Impact, 2) the knowledge and motivation
influences related to achieving this goal, and 3) the interactive effects of organizational culture
and context. Three main findings emerged and suggest that 1) progress was made, though further
opportunities existed to establish the five conditions of Collective Impact, 2) Collective Impact
creates an infrastructure for collaboration and has motivational effects when aligned to unit
goals, and 3) to leverage its effectiveness in the higher education setting, the common agenda
needs to be a university-wide initiative. Furthermore, organization context could impede the
Steering Committee from accomplishing its goal. Subsequent recommendations are based on the
New World Kirkpatrick model for program implementation and evaluation.
Keywords: Collective Impact, public health, student wellbeing, student well-being, health
promotion, higher education.
1
Chapter One: Overview of the Study
Despite the evidence-based effectiveness of a whole-system, settings-based approach to
promoting campus health and wellbeing, institutions of higher education have been slow to adopt
such a model (Cawood et al., 2010; Eifert et al., 2017). The settings approach to health
promotion is one in which the focus moves beyond individual risk factors or lifestyle orientation
to that of the influence of the system or environment on health outcomes (Antonovsky, 1987;
Baric, 1993; Dooris, 2006). A number of studies establish the link between the health status of
college students and academic success and retention, yet the majority of research and practice
focuses on a single health issue and health education as the solution (Dooris et al., 2014), as
opposed to systems or infrastructure that address the dynamic interplay of individual,
community, and environmental factors related to health and wellbeing. Behavioral approaches to
health promotion – those that encourage individual behavior change – and health policies alone
have proven inadequate in addressing complex student health issues such as those faced by
institutions of higher education (e.g., increasing rates of mental illness, substance abuse, sexual
assault, and violence on campus) (Baum & Fisher, 2014). As indicated by the World Health
Organization (1997), colleges and universities offer an important setting for comprehensively
addressing the complex interaction of individual health, the physical environment and social and
economic conditions, and have an obligation to equip students for success during their time on
campus and in preparation for entering the workforce.
Organizational Context and Mission
The Division of Student Affairs (DSA) at the University of Siloed Student Services
(USSS) is a non-academic unit that provides developmentally appropriate resources and
engagement opportunities for all students to support their academic and professional goals.
2
Located in the western United States, the university has a large, diverse student body, including
both graduate and undergraduate programs. The mission of the USSS DSA is to enhance student
wellbeing by addressing major public health concerns on campus, such as mental health issues,
at-risk substance use, sexual assault and misconduct, and inequitable conditions. The Division
employs over 200 individuals, and provides student engagement programs and activities as well
as student conduct, student health, career, and disability services.
Organizational Performance Goal
The USSS DSA’s goal is to establish and implement all five conditions of Collective
Impact by September 2020 in order to support the advancement of the four public health goals of
the DSA 2017-2020 Strategic Plan, and ultimately increase the wellbeing of the student
population. Collective Impact is an evidence-based framework designed to address complex
social issues, which involves the following five conditions: a common agenda, shared
measurement, continuous communication, mutually reinforcing activities, and a staffing
infrastructure referred to as the “backbone support organization” (Kania & Kramer, 2011). The
USSS DSA established this goal in fall 2017 after an external consultant recommended that the
Division needed a collaborative and integrated model to support the achievement of the four
public health goals of the 2017-2020 Strategic Plan. The four public health goals, which were
identified early in the strategic planning process in collaboration with staff and student
stakeholders, include the following: 1) create a more equitable and inclusive campus
environment 2) improve student mental health 3) decrease at-risk consumption of alcohol and
other drugs and 4) decrease the incidence of sexual misconduct. The desired outcome of
addressing the four aforementioned public health goals is to enhance overall student wellbeing.
At the end of the strategic plan cycle (2020), the DSA plans to conduct an internal audit of its
3
progress on the five conditions of Collective Impact, including 1) the establishment of a common
agenda across the Division 2) the creation of a shared measurement system for assessing
progress on the common agenda 3) the alignment of each unit’s primary function to the common
agenda, i.e., mutually reinforcing activities 4) the establishment of a Division-wide
communication system and 5) the identification of a group of individuals to serve as the
backbone support organization. These five conditions are italicized throughout the document to
highlight their connection to the Collective Impact framework.
Related Literature
According to the 2015 Okanagan Charter, an international charter for health promoting
universities and colleges created in conjunction with the World Health Organization, institutions
of higher education are called to “embed health into all aspects of campus culture, across the
administration, operations and academic mandates” (International Conference on Health
Promoting Universities & Colleges, 2015). Yet, while a small number of European universities
have pursued a system-wide, settings approach to health promotion, there is a paucity of
institutions, particularly in the United States, realizing this effort (Dooris, 2001; Dooris &
Doherty, 2010; Knight & La Placa, 2013; Tsouros et al., 1998; World Health Organization,
2006).
At a national level, the United States has been on a different trajectory compared to other
developed nations as it pertains to health promotion. The reasons for this difference in the United
States are varied and complex. Early in the 21
st
century, researchers identified a gap between
health promotion research and practice that led to disjointed approaches to achieving wide-
spread health (Best, et al, 2003). Compared to our European counterparts, the United States
emphasized strategies targeting individual behavior change as opposed to a settings-based
4
approach to health that acknowledges the intersections of individual lifestyles and the social and
environmental determinants of health (Dooris, 2004; Kickbusch, 2003). In effect from this
national leadership agenda and due to the lack of government and institutional financial support,
university health promotion efforts in the United States remain siloed and research remains
focused on strategies for individual behavior modification as opposed to collaborative, multi-
disciplinary efforts that span a system (Eifert et al., 2017; NACCHO, 2013). Furthermore, the
process of health promotion is inherently cross-sectoral, requiring collaboration and
infrastructure across diverse entities to achieve positive social change (McQueen et al., 2012).
Systematic, cross-sectoral work is not only challenging to accomplish, it is even more difficult to
evaluate, which then feeds back into the gap between theory, research, and practice, and thus,
funding (Dooris, 2006).
It is important to address the siloed approach to student health promotion for a variety of
reasons. While a wealth of studies link specific student health issues (e.g., substance use, mental
health, physical activity, etc.) with student success and retention (Brock et al., 2015; Grizzel &
McNeil, 2007; Gruttadaro & Crudo, 2012; Lederer & Oswalt, 2017; Martinez et al., 2015;
Piazza-Gardner et al., 2016), these studies only address one dimension of individual health and
wellbeing. This research fails to recognize the complex interactions between and within people
and their environments as described in the socioecological model (IOM, 2000). More
importantly, these topic-focused studies, and the lack of data on multidisciplinary, systems-wide
approaches to health promotion indicate that institutions of higher education, health promotion
professionals, and researchers have yet to realize that “comprehensive approaches to health
development are the most effective” (WHO, 1997, pg. 3).
5
From an institutional business standpoint, there are also financial implications. As
described by Melville-Ross (2010), institutional leadership faces monumental challenges:
decreasing budgets, increasing accountability from students and society, and increasing
competition in the midst of demographic and political change will necessitate more efficient
operations and collaborative shared service models. Faced with increasingly complex social
problems and decreasing budgets, institutions of higher education will need to employ
innovative, collaborative approaches that account for the multi-layered and multi-component
aspects that affect and contribute to a population’s health and wellbeing (Dooris, 2013). As an
example of one health-related issue, a number of research studies show a high prevalence of
mental health disorders in college students (Hunt & Eisenberg, 2010; Ibrahim et al., 2013;
Pedrelli et al., 2015). These disorders, such as mood, anxiety and substance use disorders, not
only cause stress during a major life transition, they are also correlated with significant
disruption in academic performance (Auerbach et al., 2016; Bruffaerts et al., 2018). With regard
to mental illness alone, the majority of college counseling centers are unable to meet the needs of
this growing issue (Auerbach et al., 2016; Beiter et al., 2015; Xiao et al., 2017), leaving many
colleges and universities at a loss. Beyond mental illness is a myriad of other health-related
issues such as food insecurity, homelessness, interpersonal violence, and other major health-
related problems faced by institutions of higher education today. Thus, colleges and universities
will continue to experience financial and resource strain unless they determine more efficient and
comprehensive approaches to promoting student health and wellbeing.
Importance of the Evaluation
This evaluation study of the DSA’s goal of implementing the Collective Impact
framework was important for a number of reasons. While Collective Impact has been utilized in
6
a variety of other settings, USSS was one of the first universities to implement and evaluate the
framework in the higher education setting. Thus, this study offers insight for institutions and
practitioners who are considering using this framework to accomplish their divisional or
institutional goals. Furthermore, this study served as a formative assessment of the Division’s
progress in achieving its goal, and offered feedback that could be used to course-correct or
advance current efforts.
Although not explicit in the DSA goal of employing Collective Impact, the overarching
purpose of this organizational goal was to improve the health and wellbeing of students through
coordinated and collaborative efforts. More concretely, this goal asked that the entire DSA,
including all its component units, orient their work towards health promotion. Health Promotion
is the process of enabling people to increase control over and therefore improve their health
(WHO, 1986). It is based on the fact that health is complex and multilayered, and determined
primarily by factors outside of health care such as where people live, work, learn and play
(Antonovsky, 1987; Stokols, 1996). In the context of higher education, this suggests that
organizations or entities across the university such as Housing, Transportation, faculty, and
senior administrators among others, consider how their efforts intersect and create the conditions,
activities and culture that promote positive health and wellbeing.
Health promotion necessitates buy-in and commitment from leaders of various units and
organizations across campus, which includes setting agendas and establishing a culture that
supports healthy outcomes. Dooris (2013) argues the importance of avoiding siloed efforts in
multiple settings, and that those engaged in health promotion should aim to find “synergies” and
“commonalities” in order to “forge links across professions, disciplines, and sectors” (pg. 47).
Similarly, the goal of Collective Impact is to integrate the work of multiple entities towards a
7
common goal in an attempt to magnify their respective efforts (Kania & Kramer, 2011). Thus,
this evaluation served to identify the mechanisms by which the DSA leadership was pursuing the
work of health promotion via the Collective Impact framework. It was beyond the scope of this
evaluation to determine if the aforementioned efforts will have any correlation with student
health outcomes. However, this evaluation offered insight into the complex, multi-factorial
process of enhancing student health and wellbeing.
Description of Stakeholder Groups
Since the implementation of a Collective Impact framework is a multi-sectoral effort, a
number of involved stakeholders are involved. The USSS DSA Executive Team was comprised
of Directors, Senior Directors and Associate Vice Provosts, and this stakeholder group directly
contributed to the achievement of the organizational goal through providing strategic direction,
championing the effort, and aligning their unit’s work with the common agenda of the four
public health goals. Another important stakeholder was the backbone support entity. This entity,
which was formed in year one of the Strategic Plan, consisted of a small group of professional
staff that functioned as the central support organization: they provided ongoing coordination,
collected and reported data, communicated across units, and handled all administrative actions.
Finally, a third stakeholder was DSA staff, comprised of 250 professionals who provide the
traditional range of student affairs’ services including student life, conduct, career services,
disability services, student health, and operations. DSA staff contributed to the organizational
goal through participation in working groups which aided in the creation and implementation of
strategies; alignment of their units’ activities to the common agenda; and engagement with the
student population. Table 1 provides an overview of the three stakeholder goals relative to the
organizational mission and performance goal.
8
Table 1.
Organizational Mission, Global Goal and Stakeholder Performance Goals
Organizational Mission
The USSS Division of Student Affairs’ mission is to advance student wellbeing by addressing campus-wide public
health concerns.
Organizational Performance Goal
By September 2020, the USSS DSA will fully implement a Collective Impact framework to enhance the wellbeing of
the student population (per the 2017-2020 Strategic Plan).
Stakeholder 1: DSA
Executive Team
Stakeholder 2: Backbone Support
Organization
Stakeholder 3: DSA Staff/Working
Groups
By September 2019, the DSA
Steering Committee will establish
100% of the conditions of Collective
Impact as measured by an internal
audit.
By December 2018, the Backbone
Support Organization will identify
key performance indicators for all
four goals of the 2017-2020 Strategic
Plan.
By September 2018, 100% of DSA
working groups will identify
objectives for their specific goal
group.
Stakeholder for the Study and Stakeholder Performance Goal
Inherent in the Collective Impact process is buy-in and alignment of multiple
stakeholders from multiple sectors. While all stakeholders mentioned above contributed to the
full implementation of a Collective Impact framework, for the purposes of this analysis the
primary focus was the DSA Executive Team. This group was chosen because they played the
pivotal role of Steering Committee (SC), thereby providing strategic direction for the Collective
Impact effort, which supported the advancement of the DSA’s mission of enhancing student
wellbeing.
The stakeholder goal, which was closely linked to the organizational goal of fully
implementing the Collective Impact framework, was that the DSA SC would establish 100% of
the conditions for Collective Impact by September 2019. The five conditions of Collective
Impact include 1) a common agenda, 2) a shared measurement system, 3) mutually reinforcing
activities, 4) continuous communication, and 5) a backbone support organization. The nascent
9
Collective Impact literature suggests more specific and concrete skills, values and organizational
constructs for an SC to achieve those conditions, such as the use of mapping techniques, the use
of data for reflection and adaption, and collective efficacy of the SC, among others. Failure to
implement the five conditions of Collective Impact could impact the effectiveness of the
Division: without a common goal and coordinated activities and infrastructure to accomplish that
goal, duplication of services and disjointed efforts across Division silos could have negative
financial implications. The continuation of misaligned goals and unit activities, disconnected
programming, and lack of communication could also prevent the Division from accomplishing
its four public health goals of the DSA 2017-2020 Strategic Plan, which could ultimately lead to
continued negative health outcomes for students.
Purpose of the Project and Questions
The purpose of this project was to evaluate the degree to which the USSS DSA was
meeting its goal of fully implementing a Collective Impact framework to enhance student
wellbeing. The analysis focused on knowledge, motivation and organizational influences related
to achieving the organizational goal. While a complete performance evaluation would focus on
all stakeholders, for practical purposes the stakeholder of focus in this analysis was the DSA
Executive Team, also referred to as the Steering Committee.
As such, the questions that guided this study were the following:
1. To what extent is the USSS DSA meeting its goal of implementing a Collective Impact
framework?
2. What is the Steering Committee members’ knowledge and motivation related to
achieving this organizational goal?
10
3. What is the interaction between organizational culture and context and Steering
Committee members’ knowledge and motivation?
Methodological Framework
Congruent theoretical and methodological frameworks guided this study and supported
the evaluation process. Clark and Estes’ (2008) gap analysis provided a structured and evidence-
based process to assess stakeholder knowledge, motivation, and organization (KMO) influences,
and then identify performance gaps related to the achievement of a goal. Related literature was
used to identify the KMO influences for the SC, followed by qualitative methods to evaluate the
SC’s performance relative to the goal. The qualitative research method employed included in-
depth interviews, with an emphasis on gathering rich, holistic data on the implementation
process of the Collective Impact framework (Merriam & Tisdell, 2016). To conclude the study,
research-based recommendations are offered and analyzed to build upon existing literature and
offer insight to current and future practitioners.
Definitions
Collective Impact. A type of collaboration in which a group of important actors from different
sectors commit to a common agenda to solve a complex social problem. The collaboration is
guided by a framework which involves the five conditions below (Kania & Kramer, 2011).
Common Agenda. A shared understanding of the problem and an agreed-upon approach
to solving it (Kania & Kramer, 2011).
Shared Measurement. Agreed upon measures of success and a reporting system to share
data across partners (Kania & Kramer, 2011).
Mutually reinforcing activities. Coordination of differentiated partner activities through a
comprehensive plan of action (Kania & Kramer, 2011).
11
Continuous Communication. Consistent and purposeful communication across key
partners via a variety of methods (i.e., meetings, web-based tools, etc.) (Kania & Kramer,
2011).
Backbone Support Organization. An identified and separate group of individuals who
drive the coordination of the process, through logistical, administrative and facilitation
support (Kania & Kramer, 2011).
Health. A state of complete physical, mental, and social wellbeing and not merely the absence of
disease and infirmity (World Health Organization, 1948).
Health. Promotion. The process of enabling people to increase control over, and to improve,
their health (World Health Organization, 1986).
Setting. The place or social context in which people engage in daily activities in which
environmental, organizational, and personal factors interact to affect health and wellbeing
(World Health Organization, 2013).
Settings Approach. An approach to health promotion in which the focus moves beyond an
individual risk factor or lifestyle orientation to that of the influence of the system or environment
on health outcomes (Antonovsky, 1987; Baric, 1993; Dooris, 2006).
Steering Committee (SC). A group of cross-sector community partners representative of the
relevant ecosystem that provides strategic direction for the Collective Impact initiative and
champions its work (Collective Impact Forum, 2013).
Wellbeing. A multi-dimensional construct in which there is no consensus on a single definition.
However, there is general agreement that it includes both subjective (i.e., personal) and objective
(i.e., living and working conditions) aspects that pertain to positive emotion, mood, functioning
12
and life satisfaction (Cooke et al., 2016; Lent, 2004; Middlemark & Bull, 2013; Ryan & Deci,
2001).
Organization of the Study
This study is organized into five chapters to provide a comprehensive overview of the
problem, discussion of the identified KMO influences, performance of the gap analysis, and
recommended solutions. Chapter one offers background on the problem, describes the
organization and stakeholder of interest, and details the framework used and key terminology.
Chapter two provides a review of the related literature and historical context of the problem.
Topics include discussions on the definitions of health and wellbeing, the history and challenges
of actualizing the health promotion process, and the efficacy of Collective Impact and its
potential for health promotion. Chapter two also identifies from the literature the knowledge,
motivation, and organizational factors that could influence the SC’s performance. Chapter three
covers the methodology, data collection, and analysis. Chapter four describes the findings.
Chapter five concludes with organizational recommendations based on the findings and related
literature.
13
Chapter Two: Review of Literature
This literature review examines the history and theoretical foundations of health,
wellbeing, and health promotion, and the potential efficacy of a whole-system, settings-based
approach to college health promotion. The review begins broadly with an overview of the
constructs of health, wellbeing, and health promotion, including challenges identified in the
literature with each. Next, research on the importance of a settings-based approach to health
promotion is introduced. More specifically, the review presents an in-depth discussion on the
challenges to implementing and conducting a settings-based approach to health promotion,
juxtaposed with the unidimensional focus of most college student health research. This is
followed by a review of the literature of a potentially effective and complementary framework
known as Collective Impact, including emerging literature on its use in health promotion efforts.
Finally, Clark and Estes’ (2008) gap analysis framework aids in identifying the knowledge,
motivation and organizational influences for the stakeholder group of interest, followed by the
presentation of a comprehensive conceptual framework.
Health and Wellbeing
The first chapter of this dissertation made multiple references to the concept of enhanced
student health and wellbeing as a second order effect of the implementation of Collective Impact
tied with health promotion. In essence, the Division of Student Affairs (DSA) was attempting to
use a Collective Impact framework to guide strategy on addressing public health issues across
the student population, in order to improve student health and wellbeing. In order to introduce
nuance and clarity to this topic, as well as advance research on this topic, it is necessary to define
health and wellbeing, as a variety of conceptualizations and measurements exist in both the
literature and mainstream vernacular.
14
Defining Health and Wellbeing
According to Lent (2004), the debate on optimal life and wellbeing dates back to the 8
th
century BC, where scholars including Aristotle, debated the concept of an optimal life and what
comprises wellbeing. While the construct of wellbeing is expansive and multi-dimensional, two
general domains exist: objective and subjective wellbeing. Primarily explored in the field of
psychology, Lent (2004) describes the concept of subjective wellbeing as stemming from
hedonic and eudemonic representations. Cooke et al. (2016) add quality of life and wellness as
constructs that aid in the understanding of subjective wellbeing. The hedonic view represents
wellbeing as life satisfaction, happiness, and the presence of positive affect and/or absence of
negative affect (Ryan & Deci, 2001). The eudemonic tradition goes beyond personal happiness
to actualization of self, personal development, and finding meaning and purpose in one’s life
(Lent, 2004). Used primarily in medical settings, quality of life is defined by the World Health
Organization as a “broad range concept affected in a complex way by the persons’ physical
health, psychological state, level of independence, social relationships and their relationship to
salient features of their environment” (WHOQOL Group, 1998, p. 1570). Although less clearly
defined in the literature, wellness is typically conceptualized as a multidimensional construct that
integrates mental, emotional, spiritual, and physical aspects, among others (Roscoe, 2009).
Whereas subjective wellbeing provides more of an internal or personalized perspective, objective
wellbeing captures the societal influences that allow humans to realize their potential such as
opportunities for education, safe environments, income stability, and healthy social and natural
environments (Alatartseva & Barysheva, 2015). Cooke et al. (2016) offer a broad review of these
various conceptualizations of wellbeing and the number of instruments used to measure it, and
conclude that “there is little or no consensus as to what constitutes wellbeing and how it should
15
be measured… [or] operationalized” (pg. 748). Although a true and widely accepted definition of
health and wellbeing might not ever be agreed upon, Middlemark and Bull (2013) assert that
researchers and practitioners should invoke a range of conceptualizations and measurement tools
for a more inclusive construct.
Health offers another lens in the prism-like conceptualization of wellbeing. Defined by
the World Health Organization (1948) as “a state of complete physical, mental, and social
wellbeing and not merely the absence of disease and infirmity,” this definition highlights a
salutogenic feature, similar to the “wellbeing” component mentioned above, which was a novel
addition; previous conceptualizations of health focused only on the pathologic component, or the
absence of disease (Cooke et al., 2016). According to Middlemark and Bull (2013), the
salutogenic model of health put forth by Antonovsky (1979, 1996) called for a focus on the
positive aspects of health (i.e., ease), in addition to the negative aspects (i.e., dis-ease), in terms
of a continuum of health that spans from ease to dis-ease. The theory of salutogenesis posits that
health, in terms of the ease/ dis-ease continuum, is “nested in the higher order construct [of]
wellbeing;” thus researchers and practitioners are called to shift their priorities from disease
treatment and prevention alone, to a more “balanced approach” which includes “positive aspects
of wellbeing” (Middlemark & Bull, 2013, pg. 37). Contrary to the WHO’s holistic definition and
Antonovsky’s call to a focus on upstream aspects of health, the medical model of disease
diagnosis and treatment has primarily prevailed, and failed to recognize that prevention and
treatment alone will not lead to positive health and wellbeing (Cooke at al., 2016; Keyes, 2002).
While the medical model has been effective in advancing biomedical innovations and curative
technology, the focus on bio-behavioral factors such as lifestyles and individual risk factors has
16
turned the health care sector into a sick-care system (Hancock, 1985), and thus left a gap in the
advancement of salutogenic health and wellbeing on a population level (Kickbusch, 2003).
Health Promotion
Health promotion calls for an approach to population health that extends beyond the
biomedical/ bio-behavioral model espoused in traditional health care systems, to one that focuses
more on social determinants of health and a salutogenic model. Formally established in 1986 by
the World Health Organization (WHO), health promotion is the process [emphasis added] of
enabling people to increase control over, and to improve, their health (WHO, 1986). Based on
the 1946 preamble to the WHO Constitution, the 1974 Lalonde Report, and the 1978 Alma Ata
Declaration, the Ottawa Charter (1986) for health promotion laid the foundation for and
legitimized the strategies and actions for population health. The charter established health as a
positive construct rather than merely the absence of disease; proffered that health care alone is
not the most important determinant of health, therefore strategies must extend beyond the health
care system; and declared health as a human right, calling all nations to take responsibility for
health equity, or health for all persons (Potvin & Jones, 2011). Referring back to the definition
provided above, process is emphasized to highlight that health promotion is not a product or an
outcome, but rather an ongoing series of strategies and actions in pursuit of population level
health and wellbeing. The Ottawa Charter outlines the following strategies and action items for
health promotion: strategies include advocate, mediate, and enable; and action items include 1)
build healthy public policy 2) create supportive environments 3) strengthen community actions
4) develop personal skills and 5) reorient health services (WHO, 1986). As described in Potvin
and Jones’ (2011) reflection on the 25
th
anniversary of the Ottawa Charter for Health Promotion,
from 1986-2011, health promotion achieved global status, established its positionality and
17
relevance in both developed and developing nations, and identified the importance of social
determinants of health and health-promoting policies outside the health sector.
Health Promotion and the Settings Approach
The settings approach to health promotion is based on the premise that health is created at
multiple levels through systems interactions, and therefore, the health-promoting qualities of the
system should be evaluated in addition to the health of the populations within that system.
Referring back to Antonovsky’s seminal work in this field, the author asserts that health is
created in the places where people live, love, learn, work and play, essentially at the intersection
between people and their environments or settings (WHO, 1986). A more recent voice in the
field, Kickbusch (1997, 2003) argues that the settings approach shifts the focus away from
individual behavior and the traditional disease prevention model to a population level strategy
where the setting builds capacity for health. Kickbusch goes on to describe that the shift in
orientation from health creation in the health sector to health creation in settings involves the
following: 1) common health goals developed across partners outside the health sector; 2)
commitment to collaboration; 3) cross-sector communication which facilitates knowledge
transfer and collective learning on how health is created at a systems level; 4) dissemination of
initiatives across networks; 5) constituency participation and empowerment; 6) and new
opportunities for public health professionals and departments to play strategic roles in shifting
the production of health to outside the health sector (2003). Of note, this reorientation described
by Kickbusch closely resembles the five conditions of Collective Impact, which suggests that
Collective Impact could be an appropriate framework for addressing public health goals vis-à-vis
the settings approach.
18
As described by Dooris (2006), who has published extensively on the settings approach,
the conceptual framework of the approach entails 1) an ecological model of health promotion, 2)
a systems perspective, and 3) a whole system organization development and change focus. The
author describes the ecological model as the contextual and holistic nature of health – the
complex intersection of individual, organizational, and environmental factors outside of
healthcare that create health – and the required shift from pathogenesis to salutogenesis, and risk
reduction to health enhancement as described above. Dooris continues on to explain the systems
perspective, where settings are viewed as “dynamic complex systems with inputs, throughputs,
outputs and impacts… [which] [function] as an ‘open system’ in synergistic exchange with the
wider environment… and other settings” (pg. 56). Figure 1 provides an example of the systems
perspective in the university setting (Dooris, 2006).
Figure 1. Example of a university system as a setting. Reprinted from “Healthy settings: Challenges to generating
evidence of effectiveness” by M. Dooris, 2006, Health Promotion International, 21(1), p. 56.
Finally, Dooris (2004) proposes that a whole system approach to a healthy setting includes: 1)
combining organizational change management with high level projects, 2) incorporating top-
19
down and bottom-up strategic partnerships, and 3) considering both public health and ‘core
business’ agendas when designing initiatives.
Although it is beyond the scope of this research to present the literature on evaluation of
settings approaches to health promotion, it should be noted that a number of challenges exist.
These include 1) the challenge of generating evidence for complex, multidimensional processes
like health promotion, which is cyclically reinforced by the tendency for researchers to focus on
interventions for a specific disease or risk factor as opposed to whole systems, 2) the variance in
conceptualization and praxis, and 3) the complexity of ecosystems with their interrelationships
and synergies, where health promotion work becomes less visible as it is effectively engrained in
the cultures and structures of a setting (Dooris, 2006). Despite these challenges, the settings
approach to health promotion has been enacted in cities, schools, workplaces, hospitals, and
prisons, and demonstrated moderate effectiveness. Given that universities are in many ways a
microcosm of the larger society, the university setting could be suitable for the application of the
settings approach to health promotion.
Health Promotion in Higher Education
Spanning the three decades after the Ottawa Charter for health promotion (1986), a
number of documents, international conferences, and theoretical frameworks have verified the
importance of health promotion as a systems-wide process in the higher education setting. The
first conference on health-promoting universities, held in the UK in 1996, established the
foundation for the first WHO document on healthy universities and introduced the idea that
health should be integrated “into the culture, processes, and policies of the university” (Tsouros,
1998, pg. 11). Emerging from praxis-based theory from the European Healthy Cities project,
Tsouros (1998) argued that university settings parallel the city setting and require similar
20
political buy-in, systems-level reform, collective ownership, infrastructure support, and
leadership support for health promotion. Around that same time in the United States, a seminal
article was published by Jackson and Weinstein (1997), a Vice President for Student Affairs and
Psychiatry Professor, respectively, purporting that a healthy university community is critical to
the mission of the academy in the preparation of students through knowledge that allows them to
improve their lives, be productive citizens and take on leadership roles that improve the lives of
others. Over the next decade (2006-2015), three major documents supported the application of
health promotion to the high education sector: the WHO Edmonton Charter for Health
Promoting Universities and Institutions of Higher Education (WHO, 2006), the American
College Health Association Healthy Campus 2020, and finally the Okanagan Charter (2015),
which reaffirmed the role of higher education in developing “individuals, communities, societies,
and cultures” and influencing the health of individuals and communities. Given this international
attention on health promotion, researchers began evaluating the role of the university as a health
promoting setting and identifying challenges in implementation and the development of an
evidence base of effectiveness.
Challenges: Implementation and Evaluation
While evidence and theoretical research on the benefits of health promotion is
developing, particularly in the higher education setting, the field is stilted by an overall lack of
evidence and challenges in evaluation. In a scoping review on the implementation and evaluation
of universities as health-promoting settings, Dooris et al. (2016) assert that theory-based praxis
and evaluation is necessary given the complexity of health and how it is created. In their review
process, the authors found that only 26 of the initial 156 identified references focus on the whole
university community, demonstrating the challenges of evaluating healthy university settings; the
21
remaining 130 referenced specific health issues or targeted interventions. As evidence of the
youth of this field, it should be noted that 22 of the 26 references included or cited Dooris
(Dooris et al., 2016), thus helping to explain the prominence of his work in this section and
throughout this dissertation. Another review by Sarmiento (2017) further supports the challenges
of health promotion. The author details an exhaustive list of studies examining individual health
outcomes (e.g., nutrition, physical fitness, mental health, substance use, and sexual education,
among others), but concludes that these studies rarely evaluate a comprehensive system of
initiatives striving towards a healthy setting. A key challenge lies in the fact that health
promotion is a process. As described above, the process of health promotion includes the five
actions of building healthy public policy, creating supportive environments, strengthening
community actions, developing personal skills, and reorienting health services (WHO, 1986).
Thus, evaluation should examine health promotion outcomes as the end goals in and of
themselves, as opposed to health outcomes. Nutbeam (2000) offers a framework that outlines the
progression from health promotion actions to health promotion outcomes, then to intermediate
health outcomes, which are the determinants of health and social outcomes. Figure 2
demonstrates Nutbeam’s (2006) outcome model for health promotion.
22
Figure 2. Illustration of health promotion actions to health promotion outcomes to health outcomes. Adapted from
“Health literacy as a public health goal: a challenge for contemporary health education and communication
strategies into the 21st century” by N. Nutbeam, 2006, Health Promotion International, 15(3), p. 262.
In examining Nutbeam’s (2006) framework, an important question that remains is how
practitioners proceed from actions to outcomes, in essence opening the ‘black box’ of
implementation.
Given that health promotion is a process, appropriate evaluation methods are necessary.
Nutbeam (2000) argues that the gold standard randomized control trial is irrelevant to the
complex, systems-nature of health promotion. A proposed solution to the challenges of
evaluation and evidence generation is theory-based evaluation (Dooris, 2006). Theory based
approaches to evaluation start with a logic model, outlining a sequence of events and resultant
outcomes, and make explicit the underlying assumptions and theories of change in order to
assess how theory worked when implemented (Connell et al, 1995; Weiss, 1995). In order to
23
capture the what, who, and how of the change process, outcomes must be assessed in terms of
mechanisms and context (Wimbush & Watson, 2000), which stems from the Realist Evaluation
approach to evaluating social programs (Pawson & Tilley, 1997). Thus, in order to generate
evidence of effectiveness for health promotion efforts, researchers should utilize a combination
of quantitative and qualitative methods to understand the change process itself rather than the
downstream outcomes, articulate the underpinning theories, and consider how evidence will be
used in decision-making (Dooris, 2006).
In addition to the evaluative challenges mentioned previously, the literature has also
identified implementation challenges. In a national level qualitative research study, Dooris and
Doherty (2010) identify key issues with the implementation of a comprehensive whole system
approach to improving health in higher education. These include limited data and evaluation on
effectiveness, challenges in garnering buy-in from university departments/ divisions that do not
see health as core to their mission, and the difficulty of organizational change. A systematic
review by Suarez-Reyes and Van den Brouke (2016) demonstrated that the development and
implementation of healthy policy is the most important aspect of the health promoting university
model, yet universities often do not fully accomplish this goal. The authors’ review further
identified the challenges of 1) garnering buy-in from senior leadership to see health promotion as
a priority and 2) active engagement of community members in the planning, implementation, and
evaluation of the initiative. Based on the aforementioned challenges in evaluation and
implementation, the following section presents literature on a promising framework that could
guide future efforts for health promotion in higher education.
24
Collective Impact
Based on the emerging literature, Collective Impact provides a potentially effective
framework to address some of the aforementioned challenges in advancing the process of health
promotion. Collective Impact as a formal term, as opposed to the more ordinary idiosyncratic use
of the term, was introduced to the literature in 2011 after a landmark article in the Stanford
Social Innovation Review (SSIR) by Kania and Kramer (2011). Although the formal use of the
term is relatively new, the underlying concept of place-based, cross-sector collaboration to solve
complex social problems existed well before the article brought a renewed attention to the work
(Henig et al., 2015). Cabaj and Weaver (2016) describe it as a “distillation” of existing
community change knowledge and practice that provided a “new operating system” for
practitioners (pg. 1). What the authors brought to the literature stage was a structured and semi-
prescriptive method of coordinating multiple entities or sectors in a systemic way that leads to
collective as opposed to isolated impact (Kania & Kramer, 2011).
Conditions, Pre-conditions and Principles for Collective Impact
Three articles, published between 2011-2013, laid out the emerging theory of Collective
Impact, including the conditions, pre-conditions, and timeline that guide successful efforts
(Hanleybrown et al., 2012; Kania & Kramer, 2011, 2013; LeChasseur, 2016). As described
earlier, the conditions for Collective Impact include 1) a common agenda, 2) a shared
measurement system, 3) mutually reinforcing activities, 4) continuous communication, and 5) a
backbone support organization. The subsequent SSIR article by Hanleybrown et al. (2012)
offered a refined framework, which proposed necessary pre-conditions and a timeline for
implementation. The pre-conditions include 1) an influential champion to garner buy-in from
senior leaders across sectors, 2) commitment from a funder(s) for two-three years, and 3) a
25
perception of crisis where a new approach is seen as critical to solving the problem. The three
phases in terms of timeline include 1) initiate action, 2) organize for impact, and 3) sustain action
and impact. Figure 3 provides a more detailed description of the five conditions (Kania &
Kramer, 2011), three preconditions, and three phases (Hanleybrown et al., 2012).
Figure 3. Five Conditions, three preconditions, and three phases of Collective Impact. Adapted from Hanleybrown
et al., (2012) and Kania and Kramer (2011).
In the third article, Kania and Kramer (2013) argue that true social change occurs when the
conditions and pre-conditions for Collective Impact are implemented with the lens of emergent
26
solutions, collective vigilance for change, continuous learning through established feedback
loops, and unified action across participatory organizations. The authors describe emergent
solutions as solutions that adapt to unpredictable environments, and collective vigilance as the
creative use of resources and the ability to see a range of solutions. Further, they contend that
Collective Impact does not ask participants to merely utilize current solutions in concert; rather
Collective Impact truly magnifies progress when concerted efforts lead to social innovation and
resource efficiency. Based on the underlying structure of Collective Impact, including the
unification of multiple partners towards a common goal, the focus on social change through
systems-level and policy change efforts, and the use of feedback loops from various stakeholders
(including community members), a number of health promotion efforts have started utilizing the
Collective Impact framework.
Collective Impact: A Promising Approach
Since the first publication on Collective Impact by Kania and Kramer (2011), a handful
of peer-reviewed articles have documented the process of utilizing Collective Impact, including
its strengths and challenges. Given the recent establishment of Collective Impact as a formal
framework/term, the evidence base is emergent, and similar to other collaborative efforts,
evaluating Collective Impact poses significant methodological challenges (Cortis, 2008; Moore
et al., 2014). Further, the majority of research provides insight into the process of implementing
the Collective Impact framework, but has not yet demonstrated empirical evidence of
effectiveness and/or population level outcomes (Cabaj & Weaver, 2016; Christens & Inzeo,
2015; Gillam et al., 2016; Moore et al., 2014). Three recently published literature reviews
provide important context on the use of the Collective Impact framework (Christens & Inzeo,
2015; Henig et al., 2015; Danielle Wood, 2016). These reviews suggest that Collective Impact is
27
not as new and innovative as some of the hype might suggest; rather, they conclude that
Collective Impact is an approach that should be situated within the larger literature base on
collaborative frameworks. The reviews also argue that more research is needed on the evaluation
of collaborative efforts towards the end goal of systems-level change. Finally, each review
asserts that empirical evidence exists on when and how collaboration works, and that Collective
Impact may offer a new flavor of collaboration. Although Collective Impact as a “new”
framework has been characterized as a promising approach that requires additional research and
time to develop (Flood et al., 2015), similar place-based collaborative initiatives have
demonstrated that population-level change is possible (Chilenski et al., 2014; Feinberg et al.,
2010).
Despite the aforementioned challenges, the use of Collective Impact for health promotion
initiatives has shown early signs of success. For example, Flood et al. (2015) conducted a case
study analysis of a Collective Impact initiative in which the San Francisco Tenderloin Healthy
Corner Store Coalition aimed to decrease tobacco outlet density and increase access to healthy
foods. After three years of utilizing the framework, the authors found the following evidence of
positive change: the successful passage and implementation of a healthy retailer ordinance, an
increase in the number of corner stores which improved their “healthy options” ratings according
to store assessments, increased funding, and formal establishment of the program within the
city’s government. Key strengths among the initiative included agreement across partners to the
common agenda and strong leadership by the backbone organization. Challenges included the
process of establishing shared measurement and the need to secure funding from multiple
sources since a single support entity was not in place. Another study by Amed et al. (2015)
examined the use of Collective Impact in conjunction with community-based participatory
28
research (CBPR) to achieve cross-sectoral engagement and collaborative action towards
childhood obesity prevention. The authors used a process evaluation plan to answer key
questions about the engagement and implementation process, and framed lessons learned within
the Collective Impact framework. Among other findings, the authors concluded that Collective
Impact “provide[s] a remarkably constructive framework” in engaging multiple partners across
sectors, including community members (p. 432). Moreover, they found that health promotion
efforts became visible across organizations and garnered attention among leaders to consider
future collaborations. Similar to the previous study, establishing funding across partnerships as
opposed to individual sector funding was a challenge, and the authors also acknowledged that
future research will need to address population level health changes. A key takeaway from these
two sources is that Collective Impact has the potential to address key gaps identified in the
implementation of health promotion efforts.
Clark and Estes Gap Analytic Conceptual Framework
This study utilized a problem-solving process based on the Clark and Estes (2008)
framework known as the knowledge, motivation, and organization (KMO) analysis. The
framework details a systematic, analytic process to diagnose performance problems, identify
gaps, and suggest evidence-based solutions to organizational needs. To begin, a global
organizational goal is identified, followed by more refined stakeholder goals that support the
global goal. Next, a literature review provides a synthesis of the assumed performance influences
in the categories of knowledge, motivation, and organization factors. Finally, the gap analysis
measures actual performance relative to the established goal and the framework examines
stakeholder performance and its impact relative to each of the identified KMO influences.
29
Once organizations clearly establish their goals, individuals and teams need to know what
behaviors or performance indicators are necessary to achieve that goal; subsequently, a gap-
analysis determines if a gap exists between the performance state and the goal state, and whether
knowledge, motivation, and/or organization factors contribute to that gap (Clark & Estes, 2008;
Rueda, 2011). Clark and Estes (2008) explain it is necessary to classify knowledge influences
into the appropriate sub-type – declarative, procedural, or metacognitive – to guide the design
and implementation of the appropriate performance improvement program. Declarative
knowledge includes both factual and conceptual knowledge. Factual knowledge refers to the
basic elements, facts, or terminology individuals need to know to do their jobs; conceptual
knowledge involves the ability to identify relationships between entities or pieces of information.
Procedural knowledge answers the question “how?” and guides implementation of skills or
information, and metacognitive knowledge evolves through reflection on one’s own thinking and
learning processes (Krathwohl, 2002; Rueda, 2011). The next component, motivation, is the
internal drive that moves people into action and directs that action towards a goal (Pintrich,
2003; Rueda, 2011). Motivation researchers tend to agree that there are three ways to determine
if motivation is at play: if someone is actively choosing to do one thing over another, referred to
as active choice; if someone persists in the face of challenge or distraction, known as persistence;
and if someone puts in the mental work needed to learn something new and has the confidence to
achieve a challenging goal, or mental effort (Mayer, 2011; Clark & Estes, 2008; Rueda, 2011).
Organization factors, the third and final component according to Clark and Estes (2008), include
the resources, processes and culture that influence performance.
30
Stakeholder Knowledge, Motivation, and Organizational Factors
Based on the literature, knowledge, motivation, and organization influences related to the
implementation of Collective Impact are detailed below. Given the relative newness of the
literature on Collective Impact, and the multiple entities involved in Collective Impact initiatives,
general influences found to be effective were extended to the role of a SC member. Descriptions
of the different types of knowledge, motivation, and organizational constructs are also offered to
add nuance and clarity to the discussion, along with respective tables that describe the
assessment approach for those factors.
Knowledge and Skills
In order to determine the knowledge and skills necessary to establish the conditions for
Collective Impact, literature from the private, public, and non-profit sectors was reviewed, and
important knowledge-related influences were identified. According to Clark and Estes (2008),
knowledge and skill deficits are one of three influences, the others being motivation factors and
organizational barriers, that cause performance gaps. Further, the authors explain it is necessary
to classify knowledge influences into the appropriate sub-type – declarative, procedural, or
metacognitive – to guide the design and implementation of the appropriate performance
improvement program. Declarative knowledge includes both factual and conceptual knowledge.
Factual knowledge refers to the basic elements, facts, or terminology individuals need to know to
do their jobs; conceptual knowledge involves the ability to identify relationships between entities
or pieces of information. The next sub-type, procedural knowledge, answers the question “how?”
and guides implementation of skills or information, whereas metacognitive knowledge evolves
through reflection on one’s own thinking and learning processes (Krathwohl, 2002; Rueda,
2011). Once organizations clearly establish their goals, individuals and teams need to know what
31
behaviors or performance indicators are necessary to achieve that goal; subsequently, a gap-
analysis is necessary to determine if a gap exists between the performance state and the goal
state, and whether knowledge, motivation, and/or organization factors contribute to that gap
(Clark & Estes, 2008; Rueda, 2011). The following three sections identify and discuss
knowledge influences as they pertain to the stakeholder of interest, the SC, and the goal of
fulling implementing the Collective Impact framework. Table 2 then outlines the declarative,
procedural, and metacognitive knowledge influences that contribute to the goal of implementing
a Collective Impact framework in higher education towards promoting student health and
wellbeing.
Differentiating Campus Partner Roles
In order for an SC to effectively pursue a Collective Impact framework, and bring
multiple entities into a collaborative effort, its members should be able to identify important
partners and specify the unique ways in which that entity or individual can contribute to the
larger goal (Grumbach et al., 2017). This type of knowledge, categorized as declarative, or more
specifically conceptual, is the ability to classify and differentiate between entities (Rueda, 2011).
According to Krathwohl (2002), it also involves “the interrelationships among the basic elements
within a larger structure that enable them to function together” (p. 214). Thus, SC members
should be able to demonstrate systems-level thinking by articulating the role and relationships of
campus partners in the initiative (Best et al., 2003), while also recognizing the limitations of
existing, siloed approaches (Flood et al., 2015). Inherent in that process, committee members
need to recall prior knowledge of various campus offices/individuals, categorize the type of work
performed by that office/individual, and organize that information into a schema to determine
interrelationships and create new knowledge. According to Cowan (2000), there is a limit on this
32
type of information processing in which only two or three elements of new information can be
organized or worked on at once. Furthermore, new information processing occurs in the working
memory, which is limited to remembering approximately seven new items at any one time
(Miller, 1956; Baddeley, 1992). Thus, this type of information retrieval and processing could
entail a high cognitive load due to working memory constraints (Schraw & McCrudden, 2006;
Kirschner et al., 2006), suggesting that a visual representation could be a useful tool.
Utilizing Mapping Techniques
Once SC members conceptually organize the partnerships necessary to accomplish their
goals, they need to be able to visually display that information to focus subsequent action steps.
While this information could be visually represented in a number of different ways, the literature
describes two types of mapping, referred to as asset and density mapping. An asset map is a
visual aid that focuses on the strengths of a community and highlights important relationships
among assets and resources (Kansas University Work Group on Health Promotion and
Community Development, 2000). Density mapping is a tool used to visually graph the
concentration of a particular phenomenon or item of interest on a geographical map, typically
involving the use of GIS software (Altaweel, 2017). Categorized as procedural knowledge
(Rueda, 2011), SC members need to know how to perform these techniques in order to prioritize
and direct their efforts.
Due to the complexity and number of systems which can promote or prevent health in
school settings (Dooris, 2013), SC members need to identify their assets and demonstrate
important interrelationships, but also break down silos across campus and “deterritorialize
health” (Kickbusch, 2007). In a case study on childhood obesity prevention conducted by Amed
et al. (2015), community asset mapping was used to share important resources with the
33
community and connect interested partners with project stakeholders. The central SC for the
project created a childhood obesity prevention resource map that identified involved stakeholders
and offered community specific resources, tools, and best-practices for engagement and action.
As a result, the website (where the asset map is hosted) experienced more than 16,000 page
views and 1000 resource downloads in the first year of implementation, indicating that
previously disengaged community members were linking in to the resources and initiative
stakeholders. Stemming from this model, SC members need to know how to create a context-
specific map for its goal(s), and include assets such as specific actors from across campus,
resources and best-practices, and policies that affect the goal. In doing so, they can cultivate
important relationships and identify gaps in resources and policies to advance their work, while
simultaneously laying a foundation to link disparate entities.
In another version of this technique, two case studies detailing attempts to decrease
unhealthy behaviors through decreasing availability of unhealthy substances demonstrated the
value of density mapping. Flood et al. (2015) mapped the density of tobacco and junk-food
outlets in San Francisco’s Tenderloin District. The authors found the density mapping technique
was an effective tool in bringing more attention to the health problems of the district, garnering
buy-in from business owners where these items were sold, and ultimately increasing the number
of corner stores with healthy options. In another study by Grumbach et al. (2017), partnership
working groups, which included SC members, mapped alcohol outlet density and compared it to
demographics and crime rates in San Francisco neighborhoods. This resulted in a convening
between the SC and regulatory partners who had a direct influence on state law as it pertained to
a specific alcohol policy, notable since San Francisco has the highest density of alcohol outlets of
any California city. As demonstrated by these two studies, SC members need to be able to
34
identify high volume and/or high density areas where a problem exists, examine the policies that
influence that problem, and share that information with the appropriate stakeholders. This
technique can have a profound impact on a SC’s ability to provide evidence for the problem,
engage important others in the process, and influence policy changes. In the conditions for
Collective Impact, these two techniques can aid in fostering mutually reinforcing activities as
well as the establishment of a common agenda. Thus, it is vital that SC members have or acquire
the procedural knowledge to carry out these mapping methods.
Data as a Tool for Reflection and Adaptation
Collective Impact is a process of continual refinement and learning. The second condition
of Collective Impact – shared measurement – entails data collection, meaning making, and
dissemination over the course of the initiative (Kania & Kramer, 2011). Although many health-
related initiatives establish goals and strategic plans early on, oftentimes, data is collected only at
baseline and again at post-intervention to determine outcomes, and strategic plans remain
dormant and rigid. Rather than seeing data and strategic plans as a static entity, Collective
Impact calls for data collection and usage to be an active process that perpetually informs
decisions and actions, and strategic plans to be flexible and learning-oriented (Hanleybrown et
al., 2012; Pennington, 2008). This requires then that SC members actively engage with data,
reflect on their work throughout the process, and identify opportunities to grow, learn, and adapt
their course.
In addition to learning from data, SC members need to view the strategic planning
process as a reflective, responsive, and active learning process. By paying attention to new
opportunities and viewing the process as iterative, timely adjustments lead to improved team
dynamics and alignment of resources (Kania & Kramer, 2013; Surman & Surman 2008). Further
35
described in the conceptual framework referenced above (De Montigny et al., 2017), this type of
collective learning is important to solving cross-disciplinary, complex social problems because
of the nature of collaboration; specifically, members must transfer learning individually and
across the group and employ collective learning strategies in order for true collaboration to occur
(Pennington, 2008). To be effective in this regard, the literature indicates that metacognitive
strategies are of value. The case study by Flood et al. (2015) captured the importance of
reflection for process improvement as SC members attempt to advance a Collective Impact
effort, while Amed et al (2015) described the need for continuous knowledge translation and
exchange throughout all phases and across entities. Categorized as metacognitive knowledge,
this type of knowledge involves “awareness of one’s own cognition… [and] is a key aspect of
strategic behavior in problem solving” (Rueda, 2011, p. 29). Employing metacognitive strategies
is particularly useful in promoting the transfer of learning, in that individuals can more readily
apply knowledge from one context to another and learn through peer discussion in social
contexts (Mayer, 2011). SC members who view the strategic planning process as a flexible,
learning-oriented process can thus enhance their own learning and the collective learning of the
group through actively utilizing data, reflecting on their use of strategies, and continuously
sharing information.
Table 2
Knowledge Influence, Knowledge Type, and Knowledge Assessment
Organizational Mission
The USSS Division of Student Affairs’ (DSA) mission is to advance student wellbeing by addressing
campus-wide public health concerns.
Organizational Global Goal
By September 2020, the USSS DSA will fully implement a Collective Impact framework to increase the
wellbeing of the student population (per the 2017-2020 Strategic Plan).
Stakeholder Goal
By September 2019, the DSA Steering Committee will establish 100% of the conditions of Collective
Impact as measured by an internal audit.
36
Knowledge Influence Knowledge Type Knowledge Assessment
1. SC members demonstrate
systems thinking by
differentiating the role of campus
partners in accomplishing their
goals
Conceptual Interview:
SC members were asked to identify key
campus partners and describe their role in
the initiative.
2. SC members need to know how
to apply Mapping Techniques to
their specific initiative
Procedural Interview:
SC members were asked to describe their
level of familiarity with performing asset
and/or density mapping and to provide an
example of when they used this
technique.
3. SC members utilize data as a tool
for reflection and adaptation
throughout the strategic planning
and implementation process
Metacognitive Interview:
SC members were asked how they define
and measure success; asked to provide an
example of when they used data to adapt
and modify strategies during the strategic
planning and implementation process.
Motivation
Returning to the Clark and Estes’ model (2008), once an organization establishes its
goal(s), and determines the employee knowledge needed to accomplish said goal(s), a second
component is the motivation influences. Motivation is the internal drive that moves people into
action and directs that action towards a goal (Pintrich, 2003; Rueda, 2011), and gaps in
motivation can lead to gaps in performance. While there is no single or primary theory of
motivation, component sub theories have evolved from individual cognition to socio-cognitive
and sociocultural influences, highlighting the importance of context and culture on motivation
(Rueda, 2011). Motivation researchers also tend to agree that there are three ways to determine if
motivation is at play: if someone is actively choosing to do one thing over another, referred to as
active choice; if someone persists in the face of challenge or distraction, known as persistence;
and if someone puts in the mental work needed to learn something new and has the confidence to
achieve a challenging goal, referred to as mental effort (Mayer, 2011; Clark & Estes, 2008;
Rueda, 2011). A full review of the motivation theories and principles is beyond the scope of this
paper, however two dimensions identified in the literature as pertinent to SC members are utility
37
value and collective efficacy. Table 3 outlines two motivational influences that contribute to the
goal of implementing a Collective Impact framework in higher education towards promoting
student health and wellbeing.
Utility Value of Common Agenda (i.e., Student Wellbeing)
The pursuit of a Collective Impact initiative is inherently collective. While that may
sound redundant or simple, it is quite contrary to the way many organizations function, and in
particular, pursue health-related goals (Dooris, 2013). As specified in the landmark article by
Kania and Kramer (2011), one of the five conditions for Collective Impact, a common agenda,
describes this feature: “Collective Impact requires all participants to have a shared vision for
change, one that includes a common understanding of the problem and a joint approach to
solving it through agreed upon actions” (p.39). Due to the multi-sectoral nature of a Collective
Impact initiative, alignment of individual organizational activities to the larger collective plan is
paramount. Since SC members typically hold the decision-making authority to cultivate such
alignment, it follows that a motivational factor would be involved to drive the active choice to
dedicate time, resources, and efforts towards the common agenda.
When diverse organizations or entities join to work towards a common goal, shared
values of those within the organization lay the foundation for effective collaboration. A review
of the literature on cross-sector collaboration for social change by De Montigny et al. (2017)
resulted in a conceptual framework that described the interplay of motivation, engagement, and
capacity for action, and highlighted the role of values. The framework describes collaborative
engagement as a process of individuals “discovering shared interests, concerns, and values,”
which is necessary “to foster collaborative behavior” (p. 4). In order for individuals to be
motivated to collaborate, utility value for each participant is necessary. Defined by the fit of a
38
task with the goals or needs of an individual, utility value is conceptualized as one’s desire to
complete a task because it is of use to them, so long as they know they can accomplish the task
(Eccles, 2006). In line with this principle, SC members will be more likely to engage in a
Collective Impact initiative if the common agenda or goal is beneficial to them in fulfilling their
individual and/or business goals.
Collective Efficacy
Once a shared agenda is established, SC members must also hold a shared belief that they
will be more effective when working in concert as opposed to on their own. Pajares (2006)
describes this collective efficacy as an extension of self-efficacy and highlights its effects on
“empowering and vitalizing” group members (p. 2). Bandura (2000), known as one of the
foremost experts on human agency through collective efficacy, expands on the motivation effects
to include the resilience and performance effects that take place when individuals exercise
agency at the group-level and, more importantly, perceive a “collective power to produce desired
results ” (pg. 75). The author goes on to argue that synergistic dynamics of a group are more than
the sum of individual efforts or self-efficacies; rather an emergent level of group agency
materializes through the interaction and coordination of shared knowledge, skills, and
motivation. Literature from the settings approach to health promotion, which aims to address
systemic health issues such as those under consideration by the DSA SC members, highlights the
importance of integrative multi-sectoral partnerships, and the need for each entity’s contribution
towards a collective effort (Dooris, 2009). As SC members face large-scale, complex issues with
multiple stakeholders and competing interests, it is likely that they will persist in their endeavors
if they 1) perceive a concerted group effort and 2) believe that the interdependence of its
participating organizations will be more successful than individual successes of each (De
39
Montigny et al., 2017). As an additive effect to their individual self-efficacy, SC members must
have a shared belief in the collective action of the group to solve complex problems
collaboratively. In essence, the literature demonstrates that SC members will feel more motivated
to accomplish their goal(s) if they perceive the emergent group level dynamic of collective
efficacy.
Table 3
Motivation Influences and Motivation Influence Assessment
Organizational Mission
The USSS Division of Student Affairs’ (DSA) mission is to advance student wellbeing by addressing campus-wide
public health concerns.
Organizational Global Goal
By September 2020, the USSS DSA will fully implement a Collective Impact framework to increase the wellbeing
of the student population (per the 2017-2020 Strategic Plan).
Stakeholder Goal
By September 2019, the DSA Steering Committee will establish 100% of the conditions of Collective Impact as
measured by an internal audit.
Assumed Motivation Influences
1. Utility Value – Individual SC members see the
usefulness of the common agenda of student
wellbeing to their unit or departmental goals.
Interview: SC members were asked to describe if/
how the four goals of the Division fit with the goals
of their department; asked to describe the shared
values of the DSA and how the Division arrived at
those values
2. Collective Efficacy – SC members perceive that
the collective efforts of the group will be sufficient
to accomplish the strategic goals.
Interview: SC members were asked to describe the
benefits and challenges of Collective Impact; how
well they think they are doing as a group and what
they should be doing differently; asked about
successes and factors contributing to that success
Organization Influences
The third and final component of the Clark and Estes (2008) Gap Analytic Framework
addresses organizational factors that contribute to performance gaps. According to the
framework, organizational barriers to performance can include “inadequate, inefficient, missing
or conflicting policies and processes and/or unavailability of needed tools or materials” (p. 107).
However, the authors contend that this level of gap analysis is not as straightforward as the
40
aforementioned factors indicate due to the complex and systemic nature of organizations and the
interaction of both culture and change that are constant in today’s organizations.
As a bounded group of individuals, an organization both influences and is influenced by
its internal and external environment. The internal environment of an organization, often referred
to as its culture, is the shared learning of a given group that leads to espoused values, beliefs, and
assumptions; the external environment includes the larger systems and cultures that the
organization exists within, such as national cultures and sociopolitical systems (Schein, 2004).
While a full review of systems theory and socio-ecology is beyond the scope of this paper, it
should be noted that organizations (i.e., groups of individuals) and their cultures are comprised
of multiple systems and nested within larger systems, thus leading to bidirectional influences
(Bronfenbrenner, 1989; Cabrara & Cabrara, 2015). Table 4 describes the organization influences
in terms of cultural models and settings that contribute to the goal of implementing a Collective
Impact framework in higher education towards promoting student health.
One method for analyzing an organization’s culture is through the examination of cultural
models and cultural settings (Gallimore & Goldenberg, 2001). Models and settings span from
abstract and invisible assumptions to concrete and contextual factors, respectively, and are
“constrained and enabled by the ecological niche in which they reside” (p. 47). More
specifically, the authors describe cultural models as mental models, both individually, and
collectively as in “the way things are and should be,” and cultural settings as the places where
people come together to accomplish something of value (p. 47). The following sections detail the
cultural settings and models identified in the literature as important to implementing a Collective
Impact framework in a higher education setting.
41
Collective Learning and Adaptability
Organizations that foster collective learning and adaptability are more agile and more
likely to create social change. In the general organizational change literature and in the context of
higher education, change theorists argue that in order to remain viable and help people embrace
change, organizations must concurrently sustain a shared vision and goal(s) with the ability to
adapt, fail, and learn (Gill, 2002; Kezar, 2001; Senge, 1990). These theories align with the
common agenda and shared measurement aspects of the Collective Impact framework in that
effective organizations embrace collaborative, cross-sectoral work towards a common goal, and
utilize data as a tool for collective reflection, learning, and adaptation (Kania & Kramer, 2011).
As a cultural model, collective learning is comprised of three components as illustrated below in
Figure 4: the first is a collaborative engagement process including the identification of shared
values and purpose (i.e., the motivation factor identified above), invitation of diverse
perspectives, and flexibility in the strategic planning process; the second is motivation for
collaborative engagement which is built on the interaction of communication, trust, perception of
benefits, and shared commitment to a goal; and third is the capacity for collaborative action and
adaptability, described as the leadership, resources, skills, and institutional structure that support
the aforementioned components (De Montigny et al., 2017).
42
Figure 4. Conceptual framework of cross-sector collaboration for social change to promote population health.
Reprinted from “The fundamentals of cross-sector collaboration for social change to promote population health,” by
J. De Montigny, S. Desjardins, and L. Bouchard, 2017, Global Health Promotion, 0(0), p. 1-10. Copyright 2017 by
The Authors. Reprinted with permission.
The conceptual framework proposed by De Montigny et al., (2017) describes a
continuous and iterative cycle of learning, planning, and acting which allows collaborative
partners to be agile and at the same time synergistic in their strategies and activities. Due to the
complexity of social problems, the authors argue that this collective learning-focused, trial and
error approach provides a framework for cross-sector collaborations to effectively promote
positive population health. From the lens of an organizational factor, the ability to fail, learn and
adapt – as an organization – communicates a culture that encourages experimentation in which
individuals can take risks without fear of penalty. This culture, or organizational mental model,
has the potential to foster the emergent solutions needed to address the major social challenges of
the current higher education environment.
Conveying Health as a Priority
The cultural setting in which the Collective Impact initiative takes place should convey
that health is a priority. Clear, consistent, and continuous communication of organizational
values and agenda is one of the most commonly identified, impactful organizational change tools
(Burke, 2018; Northouse, 2016; Schein, 2004). In organizations that have effectively employed a
whole system, healthy setting/university approach, senior administrators conveyed their
commitment to a healthy university through dedicated institutional structures, roles, and
resources (Newton et al., 2016). This indicates that communication of health as a priority goes
beyond the more common methods of memos, health fairs, and promotion of healthcare services;
organizations, and, in this case, universities must have systems and infrastructure in place that
encourage cross-sectoral collaboration and the promotion of health in all sectors, i.e., in the
43
curriculum, the built environment, transportation and housing, and the core business functions of
the university (Eifert et al., 2017; Mittlemark, 2007; Newton et al., 2016). Further, two-way
communication and inclusion of the beneficiaries’ (i.e., students) voice to inform action plans
also communicates what the organization values, and to what degree it desires collaborative and
participatory solutions (Christens and Inzeo, 2015; Flood et al., 2015; Sarmiento, 2017). A key
component of successful collaborative efforts and thus collective impact is community
engagement and the building of public will (Christens & Inzeo, 2015) suggesting the importance
of student engagement and empowerment in the university’s process of implementing health and
wellbeing initiatives. In this way, creating a setting that establishes health as a priority includes
both top-down and bottom-up efforts, and bi-directional communication across the organization.
Table 4
Organization Influences and Organization Influence Assessment
Organizational Mission
The USSS Division of Student Affairs’ (DSA) mission is to advance student wellbeing by addressing campus-wide
public health concerns.
Organizational Global Goal
By September 2020, the USSS DSA will fully implement a Collective Impact framework to increase the wellbeing
of the student population (per the 2017-2020 Strategic Plan).
Stakeholder Goal
By September 2019, the DSA Steering Committee will establish 100% of the conditions of Collective Impact as
measured by an internal audit.
Assumed Organization Influences Organization Influence Assessment
1. Cultural Model – The organization fosters a culture
of inquiry, one that supports failing, learning, and
adapting as part of the collective learning process.
Interview questions: SC members were asked to
describe how they define and measure success, and
what the challenges were; asked to describe an
instance when their unit/department failed and how
that instance was viewed by the DSA; asked to
identify how the university as a whole views failure
2. Cultural Setting – The organization conveys that
student wellbeing is a priority through institutional
structures, roles, and resources.
Interview questions: SC members were asked to
identify their perception of the university’s priorities
and if/how student wellbeing fits into those priorities;
asked to identify how those priorities are
communicated to various stakeholders
44
Conceptual Framework: Interaction of Knowledge, Motivation and Organizational
Context
A myriad of concepts, theories, and research have thus far been presented on the
implementation of Collective Impact in a higher education setting for the advancement of student
wellbeing. To systematize and illustrate the underlying structure and connections of the various
components, a guiding framework is offered. Rocco and Plakhotnik (2004) describe such a
conceptual framework as a map that categorizes and demonstrates the relationships between the
relevant research, theory, and concepts. Maxwell (2013) adds that the researcher’s “assumptions,
expectations, and beliefs,” or experiential knowledge, is also a key component that contributes to
the model and “tentative theory of the phenomena” to be studied (p. 39). The purpose of the
conceptual framework is to provide an explanation of the researcher’s logic or conception, which
builds upon prior knowledge, represents the related concepts either visually or narratively, and
guides the research design and interpretation of results (Rocco & Plakhotnik, 2009; Merriam &
Tisdell, 2016).
Experiential knowledge is an important aspect of the conceptual framework. Whereas
some view a researcher’s background and experience as bias that should be controlled for or
eliminated from the research design, even in qualitative studies, Reason (1988, 1994) argues that
a “critical subjectivity” should be called upon to surface the researcher’s assumptions and values
and include it in the inquiry process (as cited in Maxwell, 2013). Given that this study focuses on
the use of collective efforts towards the improvement of population-level student health and
wellbeing, it should be noted that the researcher claims public health as their discipline, and has
extensive experience in the field of higher education and student development. Further, the
researcher’s constructivist worldview and basic assumptions support the belief that health is a
45
positive construct, not merely the absence of disease; created in community, in that one cannot
be healthy in isolation; and that health is a social systems and settings issue as opposed to an
individual issue to be solved.
Previous researchers have identified a number of concepts, theories, and values that
support the shift from individual behavior and/or program-oriented health promotion efforts to
settings-based, high-impact collective and systemic efforts. In a scoping literature review by
Dooris et al. (2014), the authors found seven theoretical or conceptual frameworks relied upon in
creating a healthy university setting: the Ottawa Charter, salutogenesis, socio-ecology,
organizational development, whole system change, a settings approach, and systems thinking,
which were previously described in this chapter. Collective Impact, the framework of interest to
the USSS DSA to enhance student wellbeing, provides a methodology for large-scale social
change within complex systems by calling for cross-sector coordination and systemic change.
The aforementioned frameworks identified by Dooris et al. (2014), combined with the
knowledge, motivation and organizational factors identified in the Collective Impact framework,
provide the interaction necessary to enhance student health and wellbeing on a population level.
Figure 5 demonstrates the interaction of the aforementioned factors. The following section
describes and graphically represents the interrelationships and overarching model of
conceptual/theoretical frameworks and knowledge, motivation, and organizational factors
necessary to implement a Collective Impact framework towards the advancement of a healthy
university setting.
46
Figure 5. Knowledge, Motivation, and Organization Factors Related to Collective Impact for Student Wellbeing in
Higher Education
Application to the University Setting
The University of Siloed Student Services is a large, urban university with a
decentralized academic infrastructure which often leads to siloed efforts to address systems level
47
problems. In implementing a Collective Impact approach to solve some of the health-related
systemic problems in the student population and advance overall wellbeing, organizational
culture, including knowledge and motivation factors, and the aforementioned conceptual and
theoretical frameworks intersect. The light blue and green concentric circles in the illustration
above demonstrate the bidirectional relationship between knowledge, motivation and
organizational factors: organizational culture both influences and is influenced by the knowledge
and motivation factors. The related conceptual and theoretical frameworks in the dark blue circle
are indicated in the following section in parentheses and italics (e.g., Ottawa Charter).
In this embedded conceptual framework, the university is viewed as an ecological system
of dynamic interrelations between individual and environmental factors (socio-ecology). A
university that fosters a cultural model of collective learning and adaptability will provide formal
and informal opportunities, training, and resources for its employees to utilize data for reflection,
experiment with new solutions, and course correct in an iterative way (De Montigny et al., 2017;
Gill, 2002; Kezar, 2001; Senge, 1990). In order for that reflection, learning, and adaptability to
be effective in the context of student wellbeing, SC members must see the connection between
student wellbeing and academic success (socio-ecology), and be motivated by a shared goal of
advancing student wellbeing. Further, in a hierarchical institution of higher education similar to
USSS, the organization, in particular the leadership, needs to convey that student wellbeing is a
priority. However, this message of “wellbeing as a priority” must go beyond the more common
methods of memos, health fairs, and promotion of healthcare services; organizations, and in this
case universities must have systems, processes, policies and infrastructure in place (whole
systems change, Ottawa Charter, organizational development) that encourage cross-sectoral
collaboration and the promotion of health in all sectors (settings approach), i.e., in the
48
curriculum, the built environment, transportation and housing, and the core business functions of
the university (Eifert et al., 2017; Mittlemark, 2007; Newton et al., 2016). With a cultural setting
that conveys this priority, health must be “deterritorializ[ed]” (Kickbush, 2007) and SC members
must believe that they will only be as effective as the collective efforts of the group and their
respective units. Once they are motivated to work collaboratively and across silos within an
organization that values student wellbeing, systems level thinking is necessary to differentiate
the role of campus partners in accomplishing the wellbeing related goal(s). Of important note,
these goals cannot merely focus on the elimination of health problems and disease, but must also
aim to work upstream to enhance the positive aspects of health towards the wellbeing end of
health continuum (salutogenesis) (Mittlemark & Bull, 2013).
49
Chapter Three: Methodology
This evaluation study aimed to examine the process of implementing the Collective Impact
framework to advance student wellbeing at the University of Siloed Student Services (USSS).
The Division of Student Affairs (DSA) desired to achieve this goal in order to support their
mission of advancing student wellbeing by addressing campus-wide public health concerns.
Utilizing Clark and Estes’ (2008) analytic framework for identifying and addressing performance
gaps, this study employed a qualitative case study design to examine the knowledge, motivation,
and organizational influences that impacted the DSA Steering Committee’s (SC) ability to
achieve its goal, and assess the extent to which the DSA was meeting the goal (Creswell, 2014).
Based on needs identified in the literature, this analysis evaluated stakeholder performance,
identified gaps, and recommends solutions to support goal attainment efforts. Chapter three
details the research design and methodology utilized to conduct the evaluation, which is guided
by the following research questions:
1. To what extent is the USSS DSA meeting its goal of implementing a Collective Impact
framework?
2. What is the Steering Committee members’ knowledge and motivation related to
achieving this organizational goal?
3. What is the interaction between organizational culture and context and Steering
Committee members’ knowledge and motivation?
This chapter begins with a description of the research design and methodology, specifically the
inclusion criteria and rationale for participants and artifacts in each data collection method. Next,
a detailed account of the data collection methods and data analysis is provided, highlighting the
50
connection to the research questions. Finally, a discussion on credibility and trustworthiness and
ethics concludes the chapter.
The concept of Collective Impact emerged when Kania and Kramer (2011) published a
landmark article in the Stanford Social Innovation Review. Since then, a number of theoretical
and research-based articles on Collective Impact in the public and non-profit sectors have
contributed to the knowledge base, including in the K-12 setting (Amed at al., 2015; Flood et al.,
2015; Graham & O’Neil, 2014; Pearson, 2014). However, little to no research exists on
Collective Impact in higher education. Given that this study and the research questions listed
above aimed to describe the process of implementing Collective Impact in a new setting, a
qualitative research design was indicated.
Employing qualitative methods in this study, in particular a case study, allowed the
researcher to explore the phenomenon of Collective Impact in higher education in a generative,
inductive, and context-specific way (Creswell, 2014). The knowledge, motivation, and
organizational influence analysis describes the implementation process that took place in the
DSA and attempted to answer the question, “How does it work in a particular setting?”
(Maxwell, 2013; McEwan & McEwan, 2003). Collective Impact itself is a process – comprised
of multiple entities (e.g., a steering committee, a backbone infrastructure, participating
organizations, etc.), five conditions for impact, eight principles of practice (Kania & Kramer,
2013) – and was largely theoretical at the time of this study. Thus, the case study approach
provided a deeper dive into the complexity of Collective Impact, and a rich example of what it
looks like in practice in a given context and setting.
Finally, the role of the researcher was accounted for relative to the design of this study
and the Collective Impact initiative. Within the Collective Impact initiative, the researcher
51
worked closely with the executive team members as one of the backbone representatives. While
this established a level of trust and communication with the executive team members, it was also
possible that study participants could have been hesitant to share negative sentiments given the
nature of their work with the researcher. Furthermore, the researcher holds a constructivist
worldview (Creswell, 2014), and comes from a public health background with a bias towards
systems thinking. Since the researcher functioned as an instrument in the qualitative research
process, this required continuous reflection in addition to member checking to address bias and
verify results (Creswell, 2014; Merriam & Tisdell, 2016). Analytic memos were used throughout
data collection and analysis and in synthesizing findings in order to surface the researcher’s
reflections on the data and findings, and to identify connections between emergent patterns and
themes (Miles et al., 2014).
Participating Stakeholders
In order to capture the population of interest in this study, two tier sampling was used
(Merriam & Tisdell, 2016). The criteria for the program of interest, or case, included the
following: the Collective Impact program had been implemented by a formal division or
department within an institution of higher education; the program was in Phase I or Phase II of
implementation as specified by Kania and Kramer (2011); and the program had designated a SC,
backbone organization, and participating organizations. Meeting these criteria indicated the
program had been formalized into the university structure and made enough progress to evaluate
performance in a formative way so that course corrections were still feasible and could improve
the likelihood of success. For the purposes of this study, the “first-tier” case was the USSS DSA.
The “second-tier” case captured a sub-population within the DSA, a group of senior
administrators who functioned together as the SC. The criteria for the senior administrators
52
included the following: held a Director/Senior Director or Associate Vice Provost title or was a
member of the Vice President’s cabinet; regularly attended SC meetings; had been involved with
the Collective Impact initiative for a minimum of a year. These criteria narrowed in on the
leadership of the Division and captured a high level of involvement and experience in
implementing the Collective Impact framework to advance student wellbeing. Although the DSA
organization as a whole spearheads the Collective Impact initiative in the larger context of the
USSS, the researcher purposely selected this small group of stakeholders (n=9) due to their
position, level of authority and involvement in the initiative.
Interview Sampling Criteria and Rationale
The following sampling criteria were used to select participants for the interviews. The
rationale is provided below.
Criterion 1
Eligible interview participants were senior administrators from USSS who held a
Director/Senior Director or Associate Vice Provost title or sat on the Vice President’s cabinet
within the DSA. This indicated that the individual played a leadership role within the DSA and
had a high level of decision making authority within the organization to advance strategic goals.
Criterion 2
Eligible interview participants were senior administrators from USSS who meet the first
criterion and also participated regularly in the SC meetings, as measured by meeting attendance.
Knowledge, motivation, and organizational influences directly influenced the individuals on the
SC as their role is to advance the Collective Impact effort.
53
Criterion 3
Eligible interview participants were senior administrators from USSS who meet criteria
one and two and had been involved with the Collective Impact initiative for a minimum of a
year. It was necessary to engage those individuals who had a beginning to moderate level of
involvement in implementing the Collective Impact initiative in order to evaluate their
performance with enough time to adapt and address performance gaps.
Interview Sampling (Recruitment) Strategy and Rationale
The sampling strategy was purposeful and nonrandom in order to recruit information rich
cases where stakeholder knowledge and motivation could be examined, in addition to the
influence of the organizational culture and group dynamics (Creswell, 2014; Clark & Estes,
2008). More specifically, a census method was appropriate due to the small number of
individuals who met the above criteria (n=9) (Creswell, 2014), and the insight these
administrators could contribute to the understanding of what drives success or failure in
implementing a Collective Impact initiative in higher education. Although this sampling
technique does not provide generalizability to a broader population (Johnson & Christensen,
2014), it offers insight into the stakeholder group of focus and supports the performance gap
analysis aims of this study.
Although the research took place within the researcher’s place of employment, all
necessary precautions were taken to maintain credibility and formality of the study. To begin the
recruitment process, the researcher met with the Vice President of Student Affairs (VPSA) in
early spring 2019 to provide the background and aims of the study and to discuss parameters
around data collection and results distribution (Creswell, 2014). The researcher committed to
both the VPSA and the study participants that anonymity of the study site and confidentiality of
54
the participants’ information and responses would be guaranteed. Once approval was received
from the VPSA, a formal letter describing the aims and purpose of the study, the guarantee of
confidentiality, and the informed consent process was presented to each senior administrator on
the SC (Merriam & Tisdell, 2016). No incentives were offered as the senior administrators were
familiar with the academic setting and the research required in a doctoral program. At the
conclusion of the study, the results were offered to both the VPSA and the SC members as an
opportunity for reflection and process improvement.
Data Collection
In-depth interviews were utilized in this study in order to gather rich data and answer the
research questions. Interviews, as the singular data collection technique, provided information
rich cases (Merriam & Tisdell, 2016) to answer the second research question concerning the
knowledge and skills and motivation influences for the SC who are primarily responsible for
advancing the Collective Impact framework to enhance student wellbeing. Interviews also
provided information on SC progress on implementing the five conditions of Collective Impact,
which addresses the first research question on extent of implementation. This section discusses
the methods used to collect data in the order in which they will take place in the study.
Interview Protocol
According to Patton (2015), qualitative interviewing allows the researcher the
opportunity to hear respondents’ own perceptions and understandings of the world in their own
language. For this study, a standardized open-ended interview combined with informal
conversation was utilized as the primary data collection method to gain a deep, detailed
understanding and description of the process of implementing the Collective Impact framework
(Weiss, 1994). This methodology was chosen because it allowed the researcher to assess the
55
individual knowledge, motivation and organizational influences for the SC, and at the same time,
gain emerging insight into the interrelationships of those influences (Weiss, 1994). The
standardized open-ended interview structure focused and established priorities for the limited
time with the interviewees and provided comparability across answers and analysis while
limiting interviewer effects; the subsequent informal conversation component of the interview
allowed the researcher to revisit sensitizing concepts and gave the respondent an opportunity to
share any additional insights not covered in the structured section (Patton, 2015).
Each interview question addressed some aspect of each of the knowledge, motivation,
and organizational influences. As described by Patton (2015), knowledge and experience
questions address the conceptual, procedural, and metacognitive knowledge influences. These
questions linked specifically to the stakeholders’ ability to use systems thinking with regard to
describing campus partners’ roles in the work, familiarity with mapping techniques, and
reflection on their use of data as a tool for adaption. Motivation influences were examined
through the use of opinion and value, and feeling questions (Patton, 2015), aligned with the
conceptual framework, which examines SC collective efficacy (Bandura, 2000). The interview
questions on motivation helped to determine how efficacious SC members felt the group was in
accomplishing the goal, how that interrelated with group level agency, and how the alignment of
group member values was linked with motivation. Organizational influences were explored
through opinion and value, and sensory type questions (Patton, 2015). These questions
illuminated how SC members think, perceive, and experience the cultural setting(s) and model(s)
of the organization, and provided insight on the related theoretical frameworks of whole systems
change, socio-ecology, and health promotion as detailed in the conceptual framework. While
each knowledge, motivation and organizational influence was addressed through interview
56
questions, reflexivity and member checks provided additional measures for establishing
credibility.
Interview Procedures
Interviews were conducted with the stakeholders’ needs, timing and availability in mind
given their high work load and busy schedules (Merriam & Tisdell, 2016) and after the study
received IRB approval. The start of the fall semester in the academic school year is typically a
very busy time of year, and staff and faculty often take time off during the summer, thus data
collection via interviews spanned from April 2019-November 2019. Given the number of
questions in the interview guide plus the opportunity for informal conversation, the researcher
conducted 70-90 minute interviews with each SC member. The researcher highlighted the
importance of privacy and confidentiality and provided each participant the option to choose a
location for the interview that was most convenient and comfortable for them. All participants
consented to audio recording of the interview, and the researcher also captured data through
computer note-taking. Given that the researcher had a collegial relationship with the respondents,
the researcher structured the interview in both a formal and informal manner. The formal
component occurred at the beginning of the interview to distinguish the role of the researcher
from their normal work with the respondents and to emphasize confidentiality of the content
shared. The researcher then shifted to a more informal and conversational tone to create a space
for the respondent to authentically discuss the topic areas (Bogden & Biklen, 2007). Although
the researcher originally planned to conduct a focus group to explore collective efficacy of the
stakeholder group, this proved to be exceedingly difficult given the time of year and busy
schedules of the stakeholders of interest. Furthermore, after initial data analysis, the researcher
had sufficient data to explore the construct of collective efficacy. The literature also supports that
57
a group’s assessment of its efficacy through open discussion can be affected by social dynamics
and power differentials (Bandura, 2000), which could have played out with the Steering
Committee given the various roles and titles of those in the group.
Credibility and Trustworthiness
In order to provide an in-depth and transparent representation of the stakeholders’
constructions of their worlds, the researcher employed member checking, familiarity with the
culture of the organization of study, and reflexivity via analytic memos to establish credibility
and trustworthiness (Merriam & Tisdell, 2016; Shenton, 2004). Via member checking, the
researcher utilized interviews in the latter half of data collection to test emergent interpretations
and themes that surfaced from earlier interviews. Maxwell (2013) points out that member
checking provides a reflexive tool for the researcher to identify their own biases and
misinterpretations of the respondents’ words and actions. Another tool, familiarity with the study
site/ organization and participants allowed the researcher to establish trust with the study
participants (Shenton, 2004). Too much familiarity with the study site can affect researcher
objectivity, and thus additional efforts were made to surface and decrease the influence of any
such bias. Researcher integrity, as demonstrated by reflexivity and positionality (Probst &
Berenson, 2014) assisted in establishing trustworthiness. Throughout the study design phase, and
more specifically in the reporting of the findings, the researcher identifies and presents their
worldview, biases, and assumptions in an attempt to disclose “how [their] values and
expectations influenced the conduct and conclusions of the study” (Maxwell, 2013). The
combination of the aforementioned strategies plus the peer review component inherent in the
doctoral committee review process (Merriam & Tisdell, 2016) aided in establishing the
credibility of the findings and trustworthiness of the researcher and research process.
58
Ethics
Guided by the principles of the Belmont Report (United States, 1978) and the guidelines
of the USC Institutional Review Board (IRB), this study incorporated the three principles of
respect for persons, beneficence and justice. First and foremost, IRB approval for the study in
terms of design and data collection was obtained prior to the commencement of any study related
activities. In pursuit of justice and respect for persons, informed consent, communication with
the participants, and transparency of intent were of utmost importance. Due to the nature of the
researcher’s relationship to the Collective Impact initiative and the study participants, the
researcher had access to the “private spaces” of the Collective Impact work (Stake, 2005), and in
some ways, the researcher’s “regular work” could have blended with that of the research. To
address this situation, informed consent was obtained from all willing participants prior to the
start of data collection in order to explicitly delineate the researcher’s academic work from their
:regular work”; further, study participants were continuously reminded and informed when the
researcher was collecting data for the purposes of this study. During the interview process with
the DSA SC members, each participant was reminded that their participation was voluntary, that
all data collected during the research would be kept confidential, and that withdrawal from the
study was at their discretion; they were also given the opportunity to ask questions regarding the
nature and purpose of the study (Patton, 2015). Furthermore, participants were informed of
potential benefits and risks of participation in the study. According to Glesne (2011), researchers
should examine who benefits from the study and what the potential negative outcomes are. In
terms of beneficence, the researcher’s aim for the study was to provide useful information to help
the DSA succeed in its pursuit of advancing student wellbeing. Thus, the benefits include
potentially valuable data to help the SC continue pursuing the Collective Impact initiative
59
towards the goal of student wellbeing. Risks, though minimal, included results that could
indicate that the SC is “failing” in terms of knowledge, skills and/or motivation. Some
individuals could have seen this as a threat to their viability or role within the organization. To
address this potential risk, the researcher de-identified all information gathered, and reported
only aggregate data. Finally, all data and responses were coded and stored on a secure, password-
protected computer drive accessible only to the researcher.
Given the researcher’s position in relation to the implementation of the Collective Impact
framework (i.e., they are part of the backbone support organization), it was necessary to examine
assumptions and biases that could be present. In the backbone position, the researcher was not in
a position of authority to any of the SC members, though they worked closely with the SC in a
role similar to an internal consultant. Thus, the researcher had pre-existing knowledge of and
experience with each of the members, which could bias her data collection and interpretation
efforts. Rather than controlling for this bias, the researcher attempted to remain neutral to the
degree possible, and utilized reflective practices throughout the data collection process to make
explicit any biases present (Creswell, 2014; Merriam & Tisdell, 2016). It should also be noted
that in this backbone position, the researcher desired for the Collective Impact effort to be
successful and to maintain positive relationships with all involved parties. From an
organizational standpoint during her regular work, the researcher was providing a service to the
SC members. This dynamic could have resulted in pressure to produce a desirable result.
However, an ethical commitment from the Vice President of Student Affairs to support the
research helped to mitigate such pressure and ensure authenticity of the results. Overall, the
researcher remained cognizant of their position relative to the study participants and the ethical
60
considerations of their decisions throughout the design, data collection and analysis phases
(Creswell, 2014; Maxwell, 2012; Merriam & Tisdell, 2016).
61
Chapter Four: Findings
The purpose of this project was to evaluate the degree to which the University of Siloed
Student Services (USSS) Division of Student Affairs (DSA) was meeting its goal of fully
implementing a Collective Impact framework to advance student wellbeing. Nested in the
aforementioned organizational goal, the analysis focused on Steering Committee (SC) members’
knowledge and motivation, and organizational influences related to achieving the organization’s
global goal (Clark & Estes, 2008). Since Collective Impact was a relatively new framework, in
particular in higher education at the time of this study, the researcher conducted an extensive
literature review and conducted in-depth interviews with the SC members to understand the
implementation of Collective Impact in the higher education setting. This chapter presents the
findings from analysis of SC member interviews in relation to the literature in order to illuminate
knowledge and motivation related to implementing Collective Impact, and organizational factors
that may have affected the realization of that goal. In order to conduct this evaluation and
develop subsequent recommendations, the following questions guided this study:
1. To what extent is the USSS DSA meeting its goal of implementing a Collective Impact
framework?
2. What is the Steering Committee members’ knowledge and motivation related to
achieving this organizational goal?
3. What is the interaction between organizational culture and context and Steering
Committee members’ knowledge and motivation?
Participating Stakeholders
The stakeholders for this study were a group of DSA senior administrators who
functioned together as the Steering Committee for the Collective Impact effort. The researcher
62
invited each individual to participate through an email request followed by a phone call or in
person description of the study. The individuals who participated were members of the Vice
President of Student Affairs’ leadership team, meaning they were in a position that managed an
entire unit or a group of units; regularly attended SC meetings; and had been involved with the
Collective Impact effort for a minimum of a year. All nine participants agreed to participate
(N=9) and completed a 60-90 minute interview. Seven SC members were female-identified,
while two were male-identified. The average duration of time served in a Student Affairs
position was 13 years. Due to the small size of the stakeholder group, additional information is
withheld to maintain anonymity. SC members are coded as SC member 1 – SC member 9 to
identify responses across findings.
Findings
Three main findings are described in the subsequent sections, with knowledge,
motivation, and organization influences identified throughout. These findings address all three
research questions, including the evaluation of the extent to which the DSA was accomplishing
its goal of fully implementing the Collective Impact framework, described in Finding 3. Of note,
the third finding has five sub-sections which provide an assessment of the five conditions of
Collective Impact. Table 5 offers a summary of the three findings, including the knowledge,
motivation, and organization influences connected to those findings.
63
Table 5
Summary of Findings with embedded Knowledge, Motivation, and Organizational Influences
Main Findings KMO Influence Findings
Finding 1: Collective Impact creates an infrastructure for
collaboration and has motivational effects when aligned to
unit goals.
Finding 2: To advance the wellbeing of the student
population via Collective Impact, the common agenda needs
to be a university-wide initiative.
• Commitment and unit-specific expertise are needed
from campus partners external to the DSA
• University-wide initiatives need senior leadership
buy-in and support.
• University-wide priorities are conveyed by
accountability.
Finding 3: Progress made and opportunities exist in fully
establishing the five conditions of Collective Impact to
advance student wellbeing.
• Common agenda: calls attention to a focal point;
lacking in agreed-upon solutions
• Mutually reinforcing strategies: unit-specific
alignment without a coordinated plan
• Continuous communication: important and
challenging in a decentralized organization
• Shared measurement system: data used for equity
and engagement
• Backbone organization: supports the four conditions
and requires additional analysis
Motivation – Utility Value
SC members have sufficient utility value and see
the value of the common agenda to their unit
goals; may interact with USSS communication of
wellbeing as a priority
Motivation – Collective Efficacy
SC members do not perceive that the collective
efforts of the Steering Committee alone are
sufficient to accomplish the four goals.
Knowledge – Conceptual
SC members describe the role of campus partners
as bringing “commitment” and “unit-specific
expertise,” but do not differentiate campus partner
roles in accomplishing their goals.
Organization – Cultural Setting
SC members do not perceive that USSS conveys
to its stakeholders that student wellbeing is a
priority.
Knowledge – Metacognitive
SC members sufficiently utilize data as a tool for
reflection and adaptation.
Knowledge – Procedural
Most SC members do not know how to apply
mapping techniques
Organization – Cultural Model
SC members perceive that the DSA embraces a
culture of inquiry (i.e., one that supports failing,
learning, and adapting as part of the collective
learning process), but do not perceive that same
culture within the larger context of USSS due to
resource constraints.
The chapter concludes with a synthesis of the findings. As a precursor to the recommendations in
Chapter 5, Table 6 at the end of this chapter explicitly identifies the knowledge, motivation, and
organizational influences and if a gap was validated, partially validated, or not validated
according to the analysis.
64
Finding 1: Collective Impact Creates an Infrastructure for Collaboration and has
Motivational Effects when Aligned to Departmental Goals
As an overall assessment of the Collective Impact framework, SC members described
Collective Impact as an effective tool or model for collaboration, particularly when dealing with
societal, systemic or cultural issues. When addressing complex societal issues such as systems of
inequality, cultures that normalize substance abuse, or the mental illness epidemic, a mental
model for collaboration as well as tools to engage others in the work can help SC members
manage the scope and complexity of such issues. In this study, SC members did not perceive
Collective Impact necessarily as a new or innovative concept, but rather that it created an
intuitive and helpful way to think about collaborative processes and engage others in addressing
complex problems. When asked to describe how they feel about the Collective Impact
framework, eight of nine SC members indicated that it “made sense” or was “clear.” When
probed further to describe the framework, SC member 2 said,
Collective impact is really a way to think about complex problems…. It gives us the set
us tools to…. Acknowledge that we’re different people, to explicitly recognize that we
have to do the work of gathering the data…. developing aligned strategies, understanding
how they work together, communicating with each other, of creating a structure to
support that.
This SC member highlighted the usefulness of Collective Impact as a mental model to process
complex problems, but also to provide a structure to harness the strengths of multiple units
working collaboratively. SC member 9 mentioned the language and identified structure of
Collective Impact, and highlighted the benefit that,
There is a framework that identifies, gives language to, helps articulate how the units
should be working together across campus with each other. I think it helped us elevate the
concept of collaborative to give folks a more concrete, tangible plan of really how to do
that as a collective.
65
The value of the Collective Impact model to drive collaboration through identified language and
a structured framework is evident in this response and a number of others. This finding reflects
the importance of tools, language, and structure when engaging in collaborative efforts to
address wide-reaching, seemingly insurmountable “wicked problems” (Churchman, 1967).
Given that collaborative efforts involving a large number of individuals working on varied
interconnected strategies to address complex problems could impose a high cognitive load on SC
members, a schema or mental framework like Collective Impact can help with processing and
categorizing such information (Kirschner et al, 2006). Beyond the conceptual benefits, the
Collective Impact framework also impacted SC member motivation.
SC Members are Motivated When the Common Agenda Fits with Professional Values
In this study, DSA SC members acknowledged the value of the common agenda of
student wellbeing to their units’ work because of the alignment between Student Affairs values
and principles (as a professional field) and the promotion of student health and wellbeing. They
also believed the Collective Impact framework was necessary to stimulate collaborative efforts
towards addressing complex public health goals, such as those of the common agenda. However,
SC members identified a limit to merely using the Collective Impact framework within the DSA,
which will be described in a later section (Finding 2).
The common agenda of advancing student wellbeing via the four public health goals of
the 2017-2020 DSA Strategic Plan mostly aligned with the values of the SC members’ units or
cluster of units. Seven of nine SC members described the four goals of the Strategic Plan (i.e.,
common agenda) as directly connected to the work of their unit, while two indicated that it was
indirectly or tangentially connected. Respectively, SC members 2 and 4 stated, “They are what
we do,” and, “The goals of the Division are the goals of [my unit].” SC members attributed this
66
fit between unit-level goals and student wellbeing to the foundation of the Student Affairs
profession, which they described as “supporting students” in their development outside of the
classroom. The common agenda of advancing student wellbeing is valuable to Student Affairs
professionals, and in this case, SC members who oversee units within Student Affairs because it
was relevant to the work of their units and fit with the shared values of those in the profession.
When asked to describe the shared values of the DSA, the theme of “supporting students”
emerged across all nine interviews. More specifically, SC members expanded on “support
students” to include efforts around fostering equity and inclusion, sense of belonging, mental
health and thriving, safety, and student development. The aforementioned topics closely aligned
with the four goals of the common agenda (i.e., Strategic Plan), which addressed student mental
health and thriving, at-risk substance use, gender-based violence, and equity and inclusion. The
fit between unit goals, the principles of the student affairs profession, and student wellbeing
suggests there was sufficient utility value in the common agenda to motivate SC members to
engage in and support relevant activities. At face value, this would not validate a gap in the
motivational influence of utility value: the finding supports that SC members see the usefulness
of the common agenda of student wellbeing to their individual or unit goals (i.e., utility value is
present). However, a finding at the organizational level may impact SC member motivation in
another way.
Finding 2: To Advance the Wellbeing of the Student Population via Collective Impact, the
Common Agenda needs to be a University-wide Initiative
In this study, SC members believed that the Collective Impact initiative addressing four
complex public health goals needed to be a university-wide initiative with support from senior
leadership. All nine SC members advocated that the Collective Impact effort needed to expand
67
outside the DSA. When asked to describe barriers preventing the Division from achieving its
goal, SC member 2 stated,
The external challenges are really just the complexity of the university and the number of
stakeholders that need to be engaged. It isn’t through lack of trying, and I think that’s our
next step. This is not [emphasis added] a division project, and there’s no way we can
have any success if this is about the division.
SC member 2 called attention to the size and structure of the university, and hinted at the
struggle to engage campus partners outside the DSA when they declared in a matter-of-fact way,
“It isn’t through lack of trying.” Later in the interview, SC member 2 expanded on the number
of stakeholders needed, saying, “so many complex public health problems are not solved by one
sector.” Based on the researcher’s interactions with this SC member as part of the backbone and
professional experience in public health, the researcher also deduced that this SC member was
suggesting that success around student health and wellbeing necessitates the contributions of
sectors outside the DSA, such as public safety, housing, and financial aid, among others. In a
similar sentiment, SC member 5 conveyed the importance of the role of campus partners outside
the DSA, sharing in a passionate proclamation, “It’s crucial. In order to create cultural change in
general, I don’t foresee Student Affairs doing it on its own. It has to be more of a true collective
with other units on campus, with other people buying in.” This SC member perceived a need for
change at the cultural level to include collective efforts across the university to advance student
wellbeing, and voiced concern of a lack of awareness outside the DSA. They gave the example
of a campus-wide council they serve on, in which individuals on that council “had no idea we
were doing Collective Impact…. I think it was very surprising for them.” As another example of
the perceived need of a campus-wide initiative, SC member 3 put the DSA as well as the public
health goals in perspective in the following way:
68
It’s not enough that we’re using Collective Impact in Student Health and Student Affairs
because we are one drop in this huge place. If we really want to make student health and
wellness the priority, then you need to use this framework university wide…. Because
these four issues, they touch on everybody’s work at the university. They show up in the
classroom, they show up in football games. It’s in all facets of university life.
SC member 3 described student health and wellbeing as connected to “all facets” of students’
experience at a university, and therefore contends that the prioritization of such requires
Collective Impact as a “university wide” effort. SC members clearly articulated that the success
of the Collective Impact effort was contingent upon the buy-in and commitment of other units
outside the Division. This suggests that a gap in the motivational influence of collective efficacy
– that SC members believe that the collective efforts of the group are necessary to accomplish
the four goals – is validated. Based on SC member responses, the data suggest that the SC as a
group does not feel efficacious to meet the long-term goal of advancing student wellbeing within
the boundary of the DSA. Rather, they believe that to truly be successful the efforts must expand
to a larger group that includes campus partners outside the DSA.
SC members Need Commitment and Expertise from Campus Partners
Connected to belief that the Collective Impact initiative needed to expand outside the
Division, SC members primarily discussed the role of campus partners in terms of a need for
commitment to advancing the Strategic Plan. When asking the question, “How would you
describe the role of campus partners in achieving the goal of enhancing student wellbeing?” the
researcher anticipated that interviewees would provide specific examples like “Campus Safety
needs to do X” or “Housing needs to do Y.” This assumption was based on the literature review
which suggested that SC members needed to identify the specific role(s) of partners in
accomplishing the organizational goal(s). In this study however, and in response to the
aforementioned question and other interview questions, bringing content or unit-specific
69
expertise was cited by seven of the nine SC members, and commitment to advancing the student
health and wellbeing goals in eight of nine interviews. SC member 3 discussed content or unit-
specific expertise as campus partners being an “informant,” stating, “It would have been great to
have them [in committee meetings]… it’s useful to have their insight because they deal with
[student health problems] in the moment, they deal with the repercussions… They need to
inform us.” Responding to “What resources or assets do campus partners need to contribute?”
SC member 6 stated, “Their expertise in their own areas of work.” SC member 2 indicated that
the resources or assets needed from campus partners vary, depending on the organization. “In
some cases, it’s the content expertise, and is some cases it’s just being an advocate or being a
spokesperson.” SC member 3 made the connection between informing the work of the
Collective, and motivation to engage, stating, “Then hopefully by informing it and having some
ownership in it, there would be willing participants in helping it move forward.” A willingness to
engage in the work was a primary way that SC members described commitment. SC member 9
hinted at campus partners’ motivation to align efforts, stating the need for “their interest and
willingness to join efforts to contribute their energy in the directions that we’re going.” Said SC
member 8, “Willingness to engage and change. Really, I think that’s the biggest one, because I
think the university probably has the resources to make measureable change on all of our goals.”
A change in attitude or mindset also came up in the interviews with SC members 7, 1 and 4.
Respectively, they shared that campus partners need to bring “a willingness to think about things
differently”; “the right attitude and… motivation… and willingness to work with other people”;
and,
Campus partners need to be willing to look at their practices and functions of their
units… and they have to be willing to question why they’ve been doing things the same
way all these years. And they need to be willing to think differently… It’s a mindset.
70
Based on this response and others, commitment from campus partners espoused a variety of
forms, including attitude and mindset, resource allocation, taking ownership, and advocacy for
the work. Commitment in addition to content or unit specific expertise formed the basis of
campus partners’ “role” that SC members described as necessary in order to accomplish their
goal. Accordingly, this finding partially validates a gap in the knowledge influence of SC
members’ ability to describe the role of campus partners. Given that the Collective Impact
initiative was not yet a university-wide initiative, this finding could help to explain why the SC
members perceived a lack of commitment from partners external to the DSA. Furthermore, to
truly become a university-wide initiative with buy-in from campus partners, the SC members
argued that student wellbeing needs to be communicated from senior leadership.
University-wide initiatives need senior leadership buy-in and support
Eight of nine SC members expressed the importance of buy-in and support from senior
leadership at USSS. Each SC member was asked to describe barriers to utilizing the Collective
Impact framework and barriers to accomplishing the four public health goals of the Division.
The answers to these two questions often elicited similar responses, with eight of nine SC
members directly naming the President, Provost and/or senior leadership buy-in as necessary or
important to accomplish the student health and wellbeing goals. Senior leadership was primarily
identified as the President and Provost of USSS, though the academic deans were mentioned as
well. Discussing what needed to happen moving forward with the Collective Impact initiative,
SC member 4 fervently argued,
We need commitment and buy in from all levels…. We need a guarantee from [the
President] that these four overarching goals that are affecting all colleges and
universities, that [the President] will make it a priority…. I think until we hear the
President say ‘This is our common agenda, these are the priorities that I expect every unit
to work towards, there is [sic] going to be those barriers that exist between the priorities
of the…. academic colleges and [the Collective Impact initiative].
71
The prioritization of the common agenda of student health and wellbeing from the President of
the university was suggested here as a critical need in order to stimulate coordinated and
concentrated effort across all facets of the university. Said SC member 7, describing an initiative
related to one of the four public health goals, “At least in this university, if the upper
administration is on board, it’s something that we can really move forward with.” SC member 1
echoed this sentiment, stating, “If we get a President or Provost who champions this, it’s going to
pick up at light speed.” These comments reinforced a perception, similar to other SC members’
responses, that grass roots efforts at USSS need support from senior leadership in order to gain
traction. SC member 8 highlighted the SC’s challenge in attempting to bring in stakeholders
from outside the Division without senior leadership support, an effort that had taken place just
three months prior to their interview.
When [the Collective Impact initiative] was within the division, and it was primarily
people within the division, we knew that we were supposed to be doing this work. It was
my work. The way it is structured now, I don’t know that individual people see it as their
work. They see it as, oh, this meeting I have to go to…. I think that senior level buy-in is
critically important, even if it’s the president saying that student wellbeing is critically
important, and highlighting the work of the collective, and asking it to look into these
issues. Without that, I’m not sure how effective it is. I’m not sure if it’s a collective
anymore.
SC member 8 touched on the collective efficacy of the group when the members see the work as
important at the university level and have a level of responsibility to carry out the work together;
SC member 8 further contended that commitment and communication from the president were
“critically important” to group cohesion and sufficiently comprehensive efforts around student
health and wellbeing. Given this finding around the importance of senior leadership commitment
and prioritization, the researcher explored the mechanism by which SC members perceive senior
leadership priorities.
72
University-wide priorities conveyed by accountability
To understand how SC members formed their perceptions of senior leadership priorities
and to qualify “commitment” or “buy-in” as described above, the researcher inquired about the
mechanism by which SC members “know” the university’s priorities. When responding to this
question initially, SC members offered literal and visible mechanisms, such as what the President
talks about at public events; what is discussed in meetings with senior leadership; and news that
circulates in the university’s internal and external news outlets (e.g., departmental newsletters
and the public facing news website). However, when answering other questions such as “What
would it take align your unit’s activities with the common agenda?” or “What organizational
barriers prevent the Division from fully implementing this framework?” an underlying theme of
accountability emerged as an important communicator of senior leadership priorities. SC
member 8 provided an example that illuminated how they “hear” what is important to senior
leadership.
And the charge to me, from cabinet level folks, is ‘Take care of this’, and ‘Can I get a
report on this?’ If the questions are about [other priorities], but they’re not about student
wellbeing, then the message … there’s my prioritization.
As demonstrated by this quote, SC member 8 described the message they receive about priorities
when senior leadership regularly tasks them with projects that are unrelated to student wellbeing.
When following up on this comment, the researcher inquired if the SC member was referring to
accountability, to which SC member 8 answered affirmatively. SC member 1 echoed this
challenge, describing the tension they felt between supporting student mental health and
simultaneously running their unit, stating,
When the President is coming to me or when the Provost is coming in, or a parent or a
trustee, they’re not going to come to me and say, ‘[SC member 1] …. You’re not helping
our students thrive.’ They’re going to come to me because a student didn’t get [a
particular service from my unit].
73
These data points provide a deeper insight in to how the university, in this case senior leaders,
conveys messages on priorities that aren’t public or obvious.
Across all nine interviewees and in varying ways, SC members primarily expressed the
theme of accountability as (a) “what senior leadership asks of them”; (b) “what is in their job
description”; and (c) “what they are evaluated on.” Even in light of the alignment between
Student Affairs professional values and the common agenda of student wellbeing, there appeared
to be general agreement across SC members that the work of the Collective Impact effort was
separate from, and additive to the work of their “actual job.” SC member 3 recalled a colleague
saying, “This isn’t the work. My job is to do these things and collective impact is an aside…. My
position description says this…. I get evaluated on these things here, but you’re telling me to do
the secondary thing here.” Said SC member 5, referring to aligning unit activities to the common
agenda, “It’s not connected to HR. It’s not connected to your file.” SC member 7 described the
shift they felt was necessary in order to leverage the Collective Impact framework: those
involved, including SC members and staff and faculty around the university needed to shift from,
“‘This is one thing I have to do.’ to, ‘This is part of what you do.’ … ‘This is your job.’” Based
on their own declarations, SC members expressed a tension between what they were held
accountable for at the university level and the “additional” work of the Collective Impact
initiative in the DSA. While at the unit level, or at the level of the Division, SC members
perceived student health and wellbeing to be aligned with the values of their work, SC members
did not perceive student health and wellbeing to be a priority for the university at-large. This
perception was based on the less obvious communication mechanism of accountability. From the
vantage point of the SC members, this finding validates a gap in the cultural setting influence
74
that the organization needs to communicate to all its stakeholders that student wellbeing is a
priority.
Finding 3: Progress Made and Opportunities Exist to Fully Establish the Five Conditions of
Collective Impact to Advance Student Wellbeing
Although the SC believed that the Collective Impact initiative needed to be a university-
wide effort, requiring the support and engagement of senior leadership as well as key
stakeholders outside the DSA, they made substantial progress in establishing the five conditions
of Collective Impact within the Division. In the early years of a Collective Impact initiative, it is
important to note that much of the effort and attention is directed towards design and
implementation, while tending to changes in context and environment (Preskill et al., n.d.). In
essence, those involved are working to establish the five conditions of Collective Impact and
identify contextual factors that are supporting or hindering their efforts in order to adapt strategy
and structure. Given that the DSA was in year two of their Collective Impact effort at the time of
this study, a developmental evaluation approach was appropriate to assess progress and identify
successes and challenges with utilizing the framework. The findings in this section are therefore
intended to provide insight on the extent to which the five conditions of Collective Impact had
been established within the DSA Collective Impact initiative, and to highlight opportunities for
improvement based on SC member insights.
Common Agenda: Calls Attention to a Focal Point and Creates Sustainability
By identifying a common agenda¸ SC members noted the benefit of focused attention and
awareness across campus, as well as the likelihood for sustainability of the efforts. As described
by Kania and Kramer (2011), a common agenda is a shared understanding of the problem and an
agreed-upon approach to solving it. The USSS DSA common agenda was a product of the 2017-
75
2020 DSA Strategic Plan, which aimed to advance student wellbeing through the pursuit of four
public health goals. Those goals were to 1) create a more equitable and inclusive campus
environment 2) improve student mental health 3) decrease at-risk consumption of alcohol and
other drugs and 4) decrease the incidence of sexual misconduct. As SC member 1 described,
“Well, I think the overall benefit to the Collective Impact is that we have people from across the
campus coming together and thinking about these things.” SC member 6 expressed how the
identification of a common agenda had wide-reaching implications, stating,
The benefit is that I think it has put [the DSA 2017-2020] strategic plan on the map for
the university…. if we had not put this framework in place, we would not be anywhere.
The strategic plan might exist, but it would just be a plan [and] it would end up changing
at the end of 2020 into something else.
As exemplified by these two quotes, the articulation of a common agenda created a grass roots
type elevation of the goals of the Division, calling attention to a focal point across multiple
stakeholders, while also perpetuating the importance of the four goals. A scoping literature
review on successful organization change supports the importance of a clear and compelling
vision for change, in that shared goals and beliefs have group level effects and impact collective
outcomes (Touten et al., 2018). This finding suggests that the SC had clearly articulated the
“problem” (i.e., the four goals), and involved diverse stakeholders in addressing the problem.
However, as will be detailed below, lack of an agreed upon solution (i.e. plan of action) may
have been hindering the DSA from fully meeting its goal of implementing a Collective Impact
framework. Furthermore, this finding should be considered in the context of finding 2, in which
the SC highlighted the need for the Collective Impact effort to formally expand outside the DSA.
Mutually Reinforcing Strategies: Unit-Specific Alignment without a Coordinated Plan
Mutually reinforcing activities are the differentiated activities of participating partners,
coordinated through a comprehensive plan of action (Kania & Kramer, 2011). The concept of
76
mutually reinforcing activities was described by SC members in a number of different ways,
including aligning their unit’s mission or goals to the common agenda, allocating human and
financial resources to Collective Impact efforts, gathering data related to established metrics for
the Collective Impact effort, and engaging campus partners to participate. However, across all
SC members, there was no mention of an overarching action plan that coordinated efforts and
responsible parties; rather it appeared that SC members were responsible for identifying their
own internal strategies for aligning their units’ efforts to the common agenda. Although these
grass roots or bottom up alignment strategies are in line with those identified in the literature,
without a comprehensive and coordinated plan, they could lead to isolated impact as opposed to
collective impact.
The DSA SC identified a diverse range of activities that aligned to one or more of the
four goals of the common agenda, but lacked a differentiated and coordinated action plan.
Mutually reinforcing activities are intended to be differentiated, tapping into the diverse assets
that each partner unit is able to provide, and coordinated via a strategic approach (Kania &
Kramer, 2011). Across the nine interviewees, the following themes emerged as the primary
mechanisms for mutually reinforcing activities: SC members 1) aligned the goals of their
unit/cluster to the goals of the division (n=5), 2) dedicated human and financial resources to
programming, subcommittee work, or initiatives that stem from the Collective Impact effort
(n=5), 3) intentionally engaged and built relationships with campus partners to collaborate on
programs (n= 7), and 4) gathered data within their units related to established metrics for the
Collective Impact effort (n=5). SC member 5 detailed a planning conversation they had with
their cluster, saying,
77
We talked about our mission and its alignment to [the four goals]. We also talked about
creating learning outcomes that connect to their [sub-unit] missions and to the specific
goals. Each of the units committed to targeting one of the four goals.
SC member 6 whose unit did not provide student facing services shared their contribution in the
form of resource support, saying, “My unit supports [the Collective Impact effort] by ensuring
that we have the human and financial resources for these goals, [for example] how we’ve set
aside the strategic initiatives funding every year.” These strategies illuminate the varied ways
that SC members strove to contribute to the common agenda, and the unique opportunities that
SC members can pursue depending on the scope and function of their unit. Reinforcing activities
can include policy decisions, environmental modifications, resource allocation, awareness-
raising campaigns, advocacy or sponsorship, and data and knowledge sharing among others that
contribute to joint action (Amed, 2015; Butterfoss & Kegler, 2012; Emerson et al., 2012). The
examples provided by SC members in this study were indicative of the early and formative years
of the Collective Impact effort, but likely were not reinforcing of one another given the lack of
an overarching strategy.
With regard to a comprehensive strategy, only two SC members (1 and 4) referenced the
2017-2020 Strategic Plan and their comments indicated that the initiatives/ objectives were too
narrow and not effective for addressing the four public health goals. Similarly, SC members 2
and 3 argued that complex problems like those of the common agenda need complex solutions. A
comprehensive, coordinated, and collaborative plan “enhances problem solving capacity, which
in turn leads to enhanced system-level outcomes, which eventually leads to improvement in
individual-level outcomes” (Sridharan & Gillespie, 2006). A coordinated plan of action can help
shift a group of units from isolated impact to collective impact. SC member 2 provided the
following example of how individual unit actions are unlikely to solve campus-wide problems,
78
and how Collective Impact offers a way to connect those entities if a plan and structure are in
place.
I think we really have to get to a place where…. people aren’t trying to take these on
individually. How do we change that mindset among people that we’re not going to look
for a single solution for this problem … but rather, how do we affect this on a policy,
systems, environment level?
When asked if SC member 2 could give an example, they offered the following:
[Campus Police] came to me because they wanted [our unit] to open a drunk tank for
game day … and [Fire Safety] wanted a drunk tent… They were like, “We have drunk
people that [City Fire Department] won’t transport anymore. They say we’ve got too
many drunk people here, so we’re going to have a tent.” So, we’re “solving the problem”
of drunk people passing out and having nowhere to go, but no one is saying, “Okay guys,
let’s get together as a campus and figure out what in the world are we doing on game
day.” … I think they were ‘solving the problem,’ but I don’t think they had any
expectation that anything could change … or [there was] a structure in which to do
anything other than just put up a drunk tent.
As evidenced by this example, individual units can identify and mobilize strategies and resources
to address components of a larger problem, but this likely won’t “solve the problem.” Without a
comprehensive plan that identifies cross-sector strategies, organizations will focus on what is in
their locus of control – their budgets, their internal processes, and their own isolated impact
(Barberg, n.d.).
Continuous Communication: Important and Challenging
Across SC members, the consensus was that continuous communication was important
for reinforcing the message of the common agenda, but challenging in a decentralized institution.
Continuous communication is described as consistent and purposeful communication across key
partners in a Collective Impact effort via a variety of methods (i.e. meetings, web-based tools,
etc) (Kania & Kramer, 2011). A main purpose of continuous communication is to build trust,
maintain partner engagement, and stimulate collective motivation (Preskill et al., n.d.). The
examples of continuous communication that emerged were communication via working group
79
meetings (n=3), infusing discussion of the four goals into meetings outside the DSA (n=4), and
sharing updates at staff meetings (n=7). SC member 6 captured a breadth of ways that
communication happened.
I see [continuous communication] as the [working group] meetings, the follow up
minutes, the one-on-ones between the particular participating unit heads…. And then also
where the [strategic] plan is discussed in meetings that are not necessarily just about the
plan. Maybe I discuss the division strategic plan in my manager’s meeting…. Or it’s
talked about at the [cluster] all staff meeting, or in some other executive committee
meeting.
As illustrated in this example, continuous communication requires time and intentional effort to
ensure that the common agenda is infused through a variety of interactions and across multiple
levels of the organization. Early indicators of successful communication infrastructure include
regular meetings via working groups, active participation of members during meetings,
engagement of external stakeholders, and routine coordination and communication of efforts
(Preskill et al., n.d.). SC members provided varied examples of communication via meetings,
including external stakeholder engagement and multi-level communication. This finding
indicates early signs of success with this condition of Collective Impact, yet SC members
provided no other examples of mechanisms to foster communication across the network of
participating units. Lack of collaboration infrastructure across a decentralized university could be
associated.
Although not the same, communication and collaboration are inextricable, and are
supported by facilitation and infrastructure. In a large, decentralized organization such as USSS,
it is likely that organizational effectiveness is impacted by the lack of infrastructure for
communication and collaboration across silos. As described by Bohn (2010) in a study on
organizational efficacy, “silos in organizations lead to ineffective communications impeding
organizational outcomes.” The decentralized structure of USSS and lack of infrastructure for
80
collaboration emerged in seven of the nine interviews. Further, SC members attributed the
decentralized structure of the university with creating competition among partners and hindering
progress of the Collective Impact initiative. Six of the nine SC members specifically mentioned
competition and territorialism as an issue that gets in the way of advancing the Collective Impact
effort. When asked to describe organizational challenges to implementing the Collective Impact
framework, SC member 1 said,
We have the issue of this decentralized situation and competing individuals… The thing
that gets in the way again are [sic] humans because some people are going to have
territorial issues. Some people aren’t going to want to do something because they don’t
feel they’re getting credit for it.
Said SC member 3, “I don’t know how to refer to this eloquently, but somehow, there needs to
be more incentive to work collaboratively…. So there’s not so much fighting about ‘this is my
territory and you can’t be here.’” These two examples capture the sentiment of other SC
members, in that the siloed nature of the university creates a sense of competition that hinders
collaboration, possibly because of the desire for credit. Competition across the silos could impact
trust, communication, and the motivation to work collaboratively, which would work in
opposition of the Collective Impact effort.
Shared Measurement System: Data Used for Equity and Engagement
SC members need data to guide and support their individual unit efforts, as well as the
collective effort. In this study, SC members acknowledged the benefit of data in adapting
strategy for their unit’s programs and services. Reflection on disparities in outcomes appeared to
be a driving force for adapting programs and services, as evidenced by the fact that seven of the
nine SC members addressed this. SC member 5 discussed how data from a national survey was
used to guide their unit’s efforts, saying “We are looking at the student in a holistic way, so
we’ve been using Healthy Minds data just to see how each of these goals kind of impact our
81
student communities, especially diverse student communities.” SC member 1 commented,
“We’ve used data to look at which populations we are serving by percentage. And looking at
how does that match with the overall population to see where we need to do additional outreach
or additional programming.” Similar to other SC members, the availability of data disaggregated
by key demographics motivated these SC members to address disparities. The importance of
equity work has been written about extensively in the Collective Impact literature (McAfee,
Blackwell & Bell, 2015), specifically the need to focus response to inequities on policy and
environment changes as opposed to education or programs for the impacted group (Wolff et al.,
2016). In addressing policies, environments and systems, it is therefore important to engage the
appropriate individuals across the network of the university.
Data can be a powerful tool when the right people are exposed to it. In this study, SC
members reported the value of data in engaging campus partners to participate in the Collective
Impact initiative. In six of the nine interviewees, data was described as one of the most valuable
assets in creating buy-in for the Collective Impact initiative. When asked about the benefits of
Collective Impact, SC member 8 stated, “I think the shared data…. is probably, it’s a bit much to
say, the most powerful piece, but the data is what really gets people’s attention and allows for
engagement with the topics.” Another, SC member 9, shared about the importance of data in
higher education, saying,
Data is an incredible motivator, especially since we work in a research university.
Assessment, evaluation and research is [sic] highly regarded. In fact, it’s expected…. that
folks are making data driven decisions. So the data allowed us to make informed
decisions, but also allows us to educate folks across campus about opportunities, gaps,
areas that need attention.
As illustrated by these two sentiments, data was used to share knowledge with campus partners
and it was perceived by SC members that the data motivated partners to engage in problem
82
solving around the goal areas. It is unclear from the data in this study if SC members used shared
data to intentionally engage individuals or units in positions of authority to make policy and
systems level changes; however, given the finding above that a coordinated plan of action had
not yet been employed, it was likely that data were used in more opportunistic situations, i.e. as
needed. This finding does support that data were used throughout the Collective Impact network
for knowledge exchange. In conjunction with the previous finding that data were used to adapt
unit strategy and services, these two sub-findings indicate that a gap in the metacognitive
knowledge influence of “data as a tool for reflection and adaptation” was not validated. A gap
that emerged though in the shared measurement system category was the lack of usage and
familiarity with another data technique known as asset or density mapping.
Shared Measurement System: Role of the Backbone
When asked about asset and density mapping, the majority of SC members were not
familiar with the terminology, reflecting the technical nature of the procedure. Asset and density
mapping are visual data mapping tools that can assist SC members in capturing resources within
a community or the concentration of a particular phenomenon or item of interest, respectively
(Altaweel, 2017; Kansas University Work Group on Health Promotion and Community
Development, 2000). Once explained however, SC members had varying levels of familiarity
with the technique and its purpose. Two SC members, 1 and 8, indicated that they had no
experience or familiarity with either technique or recollection of its use in the SC or associated
subcommittee work. Another two (SC members 6 and 7) stated that they were not familiar with
the technique, but recalled the use of density mapping in one of the four subcommittees (i.e.
alcohol prevention subcommittee). Of the remaining five SC members, four (2, 3, 4 and 9) were
familiar with its use and recalled the use of density mapping in the alcohol prevention and the
83
sexual assault prevention subcommittees; the other one (SC member 5) was familiar with asset
and density mapping but not from any subcommittee or SC-related efforts. None of the SC
members had experience with actually performing the techniques, but the five who were familiar
with the technique referred to its value and the need for it to inform the SC efforts. SC member 2
described it in the following way, stating,
I’ve not done it, but I’m certainly pretty familiar with it and how it can be part of our
planning process, how it can inform our strategies. I think it’s probably a critical step that
we need to do, which we actually didn’t do in the beginning.
This comment illuminates the difference between knowing how to perform the technique(s)
versus familiarity with the technique(s) and understanding their value. SC member 4, who was
familiar with the technique but had not performed the technique(s), described how the density
mapping of alcohol outlets around campus was valuable when presenting to the Board of
Trustees.
I’ve been able to use [density mapping] in some of our arguments actually …. It was the
alcohol one, but specifically using that information when it came to presenting to the
Student Affairs Board of Trustees, and noting how many vendors in the [campus area]
sell alcohol, their jaws dropped. Because they never thought about it like that…. So yeah,
when used right, that could help hit a point home. And it did. It was effective.
As demonstrated by this example, mapping techniques can be a powerful tool to make a case and
motivate important stakeholders to engage in solving the problem(s). The data suggest that SC
members were not familiar with actually performing the mapping procedures themselves, but
instead, need the product of the mapping techniques (i.e. the assets and density maps) made
available to them to inform their efforts in addressing the problem(s) and to engage key
stakeholders. Some SC members could also use additional exposure to value of these mapping
techniques and how to utilize them. This finding suggests that a partial gap exists in SC
members’ procedural knowledge of how to apply mapping techniques. It is possible that this type
84
of procedural knowledge is more important for the coordinating structure of the Collective
Impact initiative, the backbone support organization (see Backbone Support Organization
below).
Shared Measurement System: Organizational Context Matters
Finally, it is important to frame the discussion of data and its use in the larger
organizational context. In this study, SC members reported that within the DSA, they believe the
culture supports learning through failure; however, within the larger context of the university, SC
members felt that the Division was constrained in terms of its ability to fail and adapt because of
a lack of resources. Among the nine SC members, the DSA was described as accepting of or
fostering learning through failure 11 times, with an additional four mentions each of the adaptive
spirit of the Division and the use of Design Thinking for innovation. When asked to describe
how the DSA views failure, the following two quotes capture the essence of how most SC
members responded. Said SC member 1, “I think the division actually accepts failure fine. I think
they use failure to learn…. Under [Vice President of Student Affairs], he encouraged trying
things, and if you failed, there weren’t repercussions…. I encouraged that too [in my unit].” SC
member 5 shared,
I think we view [failure] well. I think we’re a very, again, I’m thinking of my supervisor,
but we’re very flexible. We have to navigate so many entities on campus that I think we
always have a plan B, C, and D. We’ll make it work. That’s one of our little mottoes.
“Let’s make it work” …. I think we’re very adaptive.
The senior leaders within the DSA, many of whom are SC members, attempted to model
adaptive leadership, and encouraged trying and failure throughout their units, supporting their
employees’ ability to learn and adapt to change and challenge. When discussing the positionality
of the DSA relative to the university at large however, the ability to fail, learn and adapt was
perceived differently by SC members.
85
The DSA is a non-academic unit, centrally situated in the university’s organizational
structure comprised of 21 academic units plus an operational arm, and is traditionally known for
providing student services and addressing student crises. Although the setting of a research one
university which highly regards academic pursuits and innovation might foster a culture of
adaptive learning in the academic side of the university, SC members in this study perceived that
the Division had less ability to “fail and learn” due to a lack of resources. As SC member 4
shared,
I feel like they [the Division as a whole] are afraid, because if they fail with something
they feel like it’s going to be a loss of resources for them or it’ll make them look bad….
and [some units] don’t want to fail because they don’t want money taken away…. and
that’s always a risk, right? We’re not funded well as a division, so that’s very much a
fear.
This SC member’s comment echoed the perception of other SC members, in that the resource
strained environment of the Division was directly related to the willingness of various units
within the division to take risks and fail. SC member 9 expanded on the effects of working in
such an environment.
I think there is some interest in the concept [of failure] as it relates to creativity because
we have in previous years focused a lot on design thinking …. fostering creativity….
trying things out, just learning from things that “fail.” And then the flip side to that is our
financial environment where most of the units and division is [sic] constantly looking for
additional resources. And in that state of limited resources, there is not much room for
failure because you have to be selective about how you invest.
These two examples illuminate how the USSS organizational context interacted with SC
members’ perceptions of the Division’s ability to fail. The DSA senior leadership attempted to
internally foster a culture of “learning through failure,” yet this was at odds with a perceived
resource constrained environment of the DSA as nested within USSS. This organizational
construct could also interact with the knowledge influence of using data as tool for reflection and
adaptation: if Steering Committee members perceive that they do not have the ability to fail and
86
adapt within the larger setting of USSS, this could hinder the creativity needed for the necessary
emergent solutions for complex problems. It is feasible that the lack of resources related to the
Division’s resilient spirit and adaptive approach to problem solving, though this would warrant
additional analysis. In conjunction with the previous finding that the DSA internally fosters a
culture of learning and adaptability, a gap in the cultural model influence (i.e., the organization
needs to embrace a culture of inquiry as part of the collective learning process) is partially
validated.
Backbone Organization Supports the Four Conditions
The backbone organization plays a unique role in the Collective Impact framework, and
is often characterized as the distinguishing difference between contemporary cross-sector
collaboration and Collective Impact (Henig et al., 2015). The backbone is an identified and often
separate group of individuals who drive the coordination of the Collective Impact process
through logistical, administrative and facilitation support (Kania & Kramer, 2011). It was agreed
to early on when the DSA decided to utilize the Collective Impact framework that a pre-existing
office within the DSA would function as the backbone. After weighing various options and pros
and cons of each option, the Vice President’s leadership team decided that the USSS DSA’s
Health Promotion office would take on the role of backbone, based on their background and
expertise with the four public health goals and generally neutral position within the organization.
Given the backbone structure was established at the onset of the Collective Impact effort, this
aspect of the evaluation reflects SC members’ perception of the role of the backbone.
The backbone is a unique feature of Collective Impact, and serves the collective in a
variety of ways. In this study, SC members most often described the backbone as the connecting
entity that coordinates and drives the Collective Impact effort. Early indicators of success with
87
the backbone infrastructure include strategic guidance, project management, engagement of
partners and key stakeholders to support the advancement of mutually reinforcing activities, and
regular review of data to assess progress and inform decision making (Preskill et al., n.d.). When
asked to describe the backbone, SC member 6 gave the analogy of a four sided building, with the
backbone as the foundation that supports the four walls of the building, i.e., the other four
conditions of a common agenda, shared measurements, reinforcing activities, and continuous
communication. Another, SC member 7, described the central position of the backbone, and the
delicate balance between driving the initiative and empowering others to do so.
The backbone is really, I think, the key to all of this, and the key to success. And the
backbone’s job is to identify maybe those areas that need more support, how to reinforce
the message [and] provide the resources to [partners]. The backbone is really the one
that’s pushing this thing forward. And again, I think that the backbone, in some ways, has
the most information and then its job it to communicate constantly up and down the
spectrum. And really, be the ones that are both front and center, and somehow at the back
of the room all at the same time.
As evidenced by this quote, the backbone serves many purposes and occupies a unique position
within a Collective Impact initiative. Due to the nature of the backbone entity, it is recommended
that a separate evaluation be conducted to evaluate the progress and opportunities for
improvement within the backbone entity. Furthermore, the researcher’s position as part of the
backbone may have limited SC members’ full disclosure on the extent to which the backbone
serves the needs of the Collective Impact effort.
Summary of Influences
The findings presented in this chapter address the three research questions which guided
this study. From the perspective of the SC members, who held senior leadership positions within
the DSA and had been closely involved with the Collective Impact initiative for at least a year,
three main findings emerged: 1) Collective Impact creates an infrastructure for collaboration and
88
has motivational effects when aligned to unit goals; 2) To advance the wellbeing of the student
population via Collective Impact, the common agenda needs to be a university-wide initiative,
including commitment and unit specific expertise from campus partners, buy-in and support
from senior leadership, and communication of wellbeing as a priority via accountability; and 3)
progress has been made on establishing the five conditions of Collective Impact, and
opportunities exist for improvement. Embedded across those three findings was an assessment of
the knowledge, motivation, and organization influences identified in the literature, and
interactions among those influences, thereby answering the second and third research questions.
The third main finding provided an evaluation of the extent to which the DSA had achieved its
goal of fully implementing a Collective Impact framework to advance student wellbeing, thereby
answering the first research question. Table 6 summarizes the findings organized by research
question, including whether the KMO influences were validated, partially validated, or not
validated.
89
Table 6
Summary of Findings Organized by Research Question including KMO Analysis
Research Question 1: To what extent is the USSS DSA meeting its goal of implementing a Collective
Impact framework?
The DSA has made progress in fully establishing the five conditions of Collective Impact to advance
student wellbeing, and opportunities exist for advancement.
• Common agenda: calls attention to a focal point; lacks agreed-upon solutions
• Mutually reinforcing strategies: unit-specific alignment exists without a coordinated plan
• Continuous communication: important and challenging in a decentralized organization
• Shared measurement system: data used for equity and engagement
• Backbone organization: supports the other four conditions and requires additional analysis
Research Question 2: What is the Steering Committee members’ knowledge and motivation related to
achieving this organizational goal?
• Knowledge Influence 1: Steering Committee members demonstrate systems thinking by
differentiating the role of campus partners in accomplishing their goals. (conceptual
knowledge- gap partially validated)
• Knowledge Influence 2: Steering Committee members know how to apply Mapping
Techniques to their specific initiative. (procedural knowledge – gap validated)
• Knowledge Influence 3: Steering Committee members utilize data as a tool for reflection and
adaptation throughout the strategic planning and implementation process. (metacognitive
knowledge – gap not validated)
• Motivation Influence 1: Individual Steering Committee members see the usefulness of the
common agenda of student wellbeing to their unit or departmental goals. (utility value – gap
partially validated)
• Motivation Influence 2: Steering Committee members believe that the collective efforts of the
group are necessary to accomplish the strategic goals. (collective efficacy- gap validated)
Research Question 3: What is the interaction between organizational culture and context and Steering
Committee members’ knowledge and motivation?
Organizational Influence 1: The organization fosters a culture of inquiry, one that supports failing,
learning, and adapting as part of the collective learning process. (cultural model- gap partially
validated)
• SC members perceive that internally the DSA embraces a culture of inquiry but do not
perceive that same ability to fail and adapt within the larger context of USSS due to resource
constraints. This could impact Steering Committee use of data as a tool for adaptation
(knowledge influence) since the perceived inability to “fail” may hinder the emergent solutions
needed to solve complex problems.
Organizational Influence 1: The organization conveys that student wellbeing is a priority through
institutional structures, roles, and resources. (cultural setting- gap validated)
• SC members do not perceive that USSS conveys to its stakeholders that student wellbeing is a
priority, which impacts Steering Committee collective efficacy. To advance the wellbeing of
the student population via Collective Impact, Steering Committee members argue that the
common agenda needs to be a university-wide initiative, and do not perceive that the collective
efforts of the Steering Committee alone are sufficient to accomplish the four public health
goals.
90
Since the Collective Impact framework is a relatively new conceptualization of cross-
sector collaboration originally described by Kania and Kramer (2011), the researcher relied on
subsequent publications describing the implementation and evaluation of Collective Impact to
assess the degree to which the five conditions of Collective Impact were established in the USSS
DSA (Preskill et al., n.d.). Based on such publications and the findings from this study, the SC
demonstrated early signs of progress on establishing the five conditions and also had opportunity
for advancement. Members of the SC articulated a shared understanding of the “problem” or
common agenda (i.e. the four goals of the 2017-2020 Strategic Plan), yet struggled to identify
agreed upon strategies to address those goals. This corresponded with the finding regarding
mutually reinforcing activities: SC members largely described strategies and activities by which
they or their individual units contributed to the four goals or common agenda, but they did not
mention a coordinated and comprehensive action plan that crossed unit boundaries. This could be
due to the fact that a coordinated action plan had not yet been created or that the 2017-2020
Strategic Plan did not effectively coordinate activities across units or to another reason not
known by the researcher. However, it was clear from the data that such a plan was not “front of
mind” for the SC. SC members described the third condition of Collective Impact, continuous
communication, as important but challenging in a decentralized institution such as USSS.
Communication primarily occurred in the form of updates and information sharing in meetings.
Yet, the findings indicated that the siloed nature of the institution could be stifling collaboration,
including trust building and the willingness of various partners to communicate across units or
divisions. For the fourth condition of Collective Impact, SC members highlighted the benefits of
shared measurement in terms of its use in adapting strategy for their unit’s programs and
services and in engaging campus partners. SC members used data to identify inequitable
91
outcomes across the population of students, to motivate campus partners to join in the
collaborative effort and to visually locate the prevalence or incidence of certain public health
issues (i.e. incidence of sexual assault and locations of outlets that sell alcohol). Based on the
responses of the SC members, the researcher deduced that the ability to conduct the specialized
procedures for asset and density mapping might not be a skillset needed by the SC; rather, the SC
needs to be able to utilize the product of those mapping techniques (i.e. the actual visual maps) to
inform their efforts. Finally, SC members highlighted the importance of the fifth condition of
Collective Impact, the backbone organization, in advancing and supporting the progress of the
other four conditions. Based on the findings from Chapter 4, Chapter 5 will offer
recommendations for closing the gaps in validated and partially validate influences, as well as
organizational supports necessary to support SC members in advancing student wellbeing
through the Collective Impact framework.
92
Chapter Five: Discussion
The purpose of this project was to evaluate the degree to which the University of Siloed
Student Services (USSS) Division of Student Affairs (DSA) was meeting its goal of fully
implementing a Collective Impact framework to enhance student wellbeing. A qualitative case
study research design was employed to address three research questions.
1. To what extent is the DSA meeting its goal of fully implementing a Collective Impact
framework?
2. What is the Steering Committee members’ knowledge and motivation related to
achieving this organizational goal?
3. What is the interaction between organizational culture and context and Steering
Committee members’ knowledge and motivation?
Chapter 4 presented the findings from the study, and validated which knowledge, motivation, and
organizational (KMO) influences are related to Steering Committee (SC) member’s ability to
fully implement the Collective Impact framework with student wellbeing as the organizational
goal. Data were analyzed and findings presented and organized by themes. This chapter offers
recommendations to address gaps in or bolster the KMO influences, as well as an integrated
implementation and evaluation plan to move the university forward into the next phase of
Collective Impact. The knowledge, motivation, and organization recommendations are based on
learning principles and theories from the literature and are context-specific to the study site.
Next, an integrated implementation and evaluation plan based on the New World Kirkpatrick
Model is described, which includes four levels of evaluation starting with the end goal in mind
(Kirkpatrick & Kirkpatrick, 2016). Level 4: Results depicts the desired organizational outcomes
as well as individual indicators to reach that outcome. Level 3: Behavior offers the critical
93
behaviors and associated reinforcements to drive the outcome. Level 2: Learning details a
training program including the timeline, delivery, content, and associated measurement
instruments for the training component of the recommendations. Level 1: Reaction examines
trainee engagement, relevance of the material, and satisfaction. Finally, a discussion of how to
report data and analysis from the integrated implementation and evaluation plan is offered, as
well as a sample report in the appendices.
Recommendations for Practice to Address Knowledge Influences
The research validates that knowledge gaps exist in SC members’ conceptual knowledge
of identifying the specific role of campus partners and procedural knowledge in applying
mapping techniques to a student health issue. A gap was not found in SC members’
metacognitive knowledge of utilizing data as a tool for reflection and adaption of strategies.
Table 7 displays a complete list of assumed knowledge influences and context-specific
recommendations, based on an informal understanding of the organization, the stakeholders, and
the literature review. Discussion of the knowledge influences and the gap analysis process is
guided primarily by Clark and Estes (2008) and Rueda (2011), and recommendations are based
on learning theories and principles. The recommendations for the knowledge influences include
job aids and training, and reflect the organization-specific context and activities that should be
conducted in conjunction with other recommendations to create an implementation package of
activities, interventions, or programs.
94
Table 7
Summary of Knowledge Influences and Recommendations
Assumed Influence Gap
Validated
Priority Principle and Citation Context-Specific
Recommendation
SC members need
to differentiate the
role of campus
partners in
accomplishing their
goal (CK).
Y Y Creating schemata helps
learners organize DK in a
domain (Schraw et al., 2009).
Managing intrinsic load by
segmenting complex
material into simpler parts and
pre-training, among other
strategies, enables learning to
be enhanced (Kirschner et al.,
2006).
Provide a clearly
structured chart (job aid)
that identifies each
campus partner and their
respective role in
addressing student health
goals.
SC members know
how to apply
mapping
techniques to their
specific initiative
(PK).
Y Y To develop mastery, individuals
must acquire component skills,
practice integrating them, and
know when to apply what they
have learned (Schraw &
McCrudden, 2006).
Provide training via case
studies on how to utilize
mapping techniques.
SC members utilize
data as a tool for
reflection and
adaptation
throughout the
strategic planning
and
implementation
process (MK).
N Y Metacognitive strategies
support the application of
knowledge from one context to
another (transfer) and
individual learning through peer
discussion in social contexts
(Mayer, 2011)
Facilitate peer modeling
of using data as a tool for
reflection and adaptation,
and opportunities to
discuss how SC members
plan to use data for their
specific initiatives.
Note. CK = conceptual knowledge, PK = procedural knowledge, MK = metacognitive knowledge, DK = declarative
knowledge, Y = yes, N = no
Provide a Job Aid to Identify Campus Partners and their Roles
The results and findings of this study indicated that SC members need more in-depth
conceptual knowledge to differentiate the role of campus partners in accomplishing the
organizational goal of implementing the Collective Impact framework. To close this knowledge
gap, a recommendation from cognitive load theory is provided. Kirshner et al. (2006) theorize
that learning is enhanced when intrinsic load is managed through breaking down complex
material into simpler parts, and Schraw et al. (2009) suggest the use of schemata to help organize
declarative knowledge in a domain. In line with these theories, a job aid with a structured chart
95
or visual organizer would ease cognitive load and enhance learning. The recommendation then is
to provide SC members with a clearly structured chart that aids in identifying each campus
partner and their respective role in accomplishing the organizational goal.
Provide Training on Applying Mapping Techniques
The second finding from this study identified that SC members need support in the
procedural knowledge of applying mapping techniques to their specific initiative. Mapping
techniques such as asset mapping and density mapping provide visual representation of a feature
of interest, and a recommendation from information processing theory has been selected to
address this knowledge gap. To develop mastery of a technique, individuals must develop
component skills, practice integrating them, and know when to apply what they have learned
(Schraw & McCrudden, 2006). Additionally, procedural knowledge increases when declarative
knowledge required to perform the skill is available or known (Clark & Estes, 2008). Based on
this finding and the accompanying research literature, the recommendation is to provide training
via the use of case studies in order to offer real-life practice with the procedures required to
conduct mapping techniques.
Facilitate Peer Modeling and Discussion of Data as a Tool
The findings support that SC members have sufficient metacognitive knowledge
in utilizing data as a tool for reflection and adaptation throughout the strategic planning process.
From information processing theory, it has been shown that metacognitive strategies support the
application of knowledge from one context to another (transfer) and also support individual
learning through peer discussion in social contexts (Mayer, 2011). Although a gap was not found
in this domain, for future reference, peer modeling could assist in supporting metacognitive
practices. Specifically, an SC member or colleague who is familiar with data could talk through
96
their use of data as a tool for reflection and adaptation, thereby providing opportunities for the
SC as a group to discuss how they plan to use data for their specific initiatives.
Recommendations for Practice to Address Motivation Influences
The research validates that a motivation gap exists in SC members’ collective efficacy
and partially exists in their utility value of the common agenda, based on the perceived lack of
organizational prioritization of student wellbeing. Discussion of the motivation influences is
guided by Clark and Estes (2008) who suggest three categories of motivation in task
performance - choice, persistence, and mental effort. Utility value - the usefulness or relevance
of the task - can impact any of the three categories (Eccles, 2006). Collective efficacy, a group’s
shared belief in its ability to pursue and act jointly for a common purpose (Bandura, 2000) is also
connected to an individual’s motivation when working in group settings. In line with these
theories, recommendations for improving motivation include opportunities for discussion of the
common agenda of student wellbeing, and opportunities for the identification of collective
success and needs. Table 8 displays the motivation influences and context-specific
recommendations that should be conducted in conjunction with other recommendations to create
a comprehensive implementation package.
97
Table 8
Summary of Motivation Influences and Recommendations
Assumed
Influence
Gap
Validated
Priority Principle and Citation Context-Specific
Recommendation
SC members see
the usefulness of
the common
agenda (i.e.,
student wellbeing)
to their
departmental or
unit goals. (UV)
Y Y Rationales that include a
discussion of the importance
and utility of the work or
learning can help
learners develop positive
values (Eccles, 2006;
Pintrich, 2003)
Providing tasks, materials
and activities that are
relevant and useful to
learners enhance their value
for the work (Rueda, 2011)
Design an activity to
help SC members
identify concrete
examples of 1) how the
common agenda is
relevant to departmental
goals, 2) how it
contributes to university
goals, 3) their specific
role in the goal, and 4)
benefits of collaboration
SC members have
a shared belief in
the collective
action of the
group to solve
complex
problems. (CE)
Y Y When individuals exercise
agency at the group level
and perceive a collective
effort, motivation is
enhanced through resilience
and performance (Bandura,
2000).
Learning and motivation are
enhanced when learners
have positive expectancies
for success (Pajares, 2006).
Create a coordinated
action plan document
to identify specific
contributions from
each SC member, and
needed contributions
from external partners.
Facilitate discussion at
SC meetings for
members to provide
progress updates.
Facilitate a discussion
to identify 1-2
powerful sponsors.
Note. UV = utility value, CE = collective efficacy, Y = yes
Increase Utility Value of the Common Agenda for SC Members
The findings of this study indicated that SC members found the common agenda of
student wellbeing as useful to their department’s goals. However, student wellbeing was not
conveyed as a priority at the university level, which could decrease utility value of the common
agenda of student wellbeing. Expectancy value theory as it pertains to utility value indicates that
rationales that include a discussion of the importance and utility of work or learning can help
individuals develop positive values (Eccles, 2006; Pintrich, 2003). This suggests that identifying
specific and concrete examples of how student wellbeing is important to and related to SC
98
members’ work within their unit could enhance utility value. Additionally, identifying examples
of how the common agenda of student wellbeing is connected to the larger university goals could
also substantiate an investment of time and effort. The recommendation then is to provide an
opportunity for SC members to identify the relevance of student wellbeing to their individual
unit/departmental goals and identify specific activities that align to their department’s work.
In the motivation literature, Rueda (2011) states that “the higher an individual values an
activity, the more likely he or she chooses, persists, and engages in it” (pg. 43). Further, Wigfield
and Eccles (2000, 2002) highlight the importance of how useful an activity is in achieving a
future goal, also known as task value. This theoretical evidence suggests that individuals are
motivated to actively choose and engage in an activity when they can identify the “why” and see
a relevant connection to accomplishing something larger. Emerson et al (2012) posit that partners
who engage in collaboration develop a shared “why” and a shared plan for accomplishing their
goal. They go on to discuss that discovery and definition are part of the development of shared
purpose and action: discovery includes “the revealing of individual and shared interests,
concerns, and values” while definition refers to “continuous efforts to build shared meaning by
articulating common purpose and objectives” among other characteristics (pg. 12). Butterfoss
and Kegler (2012), two researchers on collaboration and coalition building, argue that
collaboration is sparked when organizations perceive benefits such as pooling of resources and
minimization of risk or uncertainty. Considering these theoretical perspectives together, it would
appear that SC members who 1) articulate their shared “why” or purpose and 2) collectively
develop a plan that locates each member’s contribution to the goal(s) and 3) identify the benefits
of collaboration could experience increased engagement and motivation to pursue activities
aligned to student wellbeing.
99
Increase Collective Efficacy of the Steering Committee via a Coordinated Action Plan and
Powerful Sponsor
This study found that SC members hold a shared belief that collective action is necessary
at the university level to accomplish the strategic goals related to student well-being; however,
within the boundary of the DSA, SC members as a group did not perceive that collective action
of the SC would be sufficient to address such large scale public health problems.
Recommendations from the literature on cross-sector collaborations and from collective-efficacy
theory have been selected to address this gap. Bandura (2000) discusses the motivation effects of
collective efficacy, highlighting the resilience and performance effects that take place when
individuals exercise agency at the group level and perceive a collective effort. Pajares (2006)
adds that learning and motivation are enhanced when learners have positive expectancies for
success. From the literature on cross-sector collaboration, Bryson et al. (2006) proffer that cross-
sector collaborations will form when individual sector efforts actually fail to solve a public
problem, or when policy makers believe that problems cannot be solved by individual sectors
acting alone. Additionally, Crosby and Bryson (2005) contend that powerful sponsors or
conveners can draw attention to public problems and garner credibility and legitimacy across a
group of stakeholders. The recommendation then is to facilitate opportunities for the SC
members to identify specific examples of how the SC is working collaboratively and making
progress on various initiatives, and to also identify specific campus partners that are needed to
accomplish a goal/ initiative that the SC cannot accomplish on its own. This could be facilitated
via a coordinated action plan document, which the findings from Chapter 4 indicated had not
been utilized. Furthermore, the SC should identify one to two powerful sponsors within the
group who can advocate for the assets and needs of the SC to external stakeholders.
100
Collective efficacy is much more than the sum of individual efficacy. Bandura (1997,
2000), one of the foremost experts on efficacy, describes two components of collective efficacy:
individuals working together believe that other members of the team bring the necessary skills to
accomplish the team’s goal or common objective, and they believe that the group will be more
effective working together than if they were to each work on their own. Furthermore, when
multiple organizations agree to work together towards a common goal, they are more effective
when they intentionally recruit diverse stakeholders who provide specific skills and disciplinary
lenses and represent diverse constituencies and sectors (Butterfoss & Kegler, 2012).
Furthermore, the authors go on to discuss the importance of open and frequent communication in
collaborations, in that it leads to enhanced engagement and pooling of resources. In line with
these principles and theories, it follows that motivation can be enhanced when individuals are
provided the opportunity to openly and continuously discuss diverse stakeholder contributions to
the larger goal. This could be facilitated at regular SC meetings with a portion of time dedicated
for SC to report progress on their aligned activities.
Recommendations for Practice to Address Organizational Influences
The research suggests that the USSS lacks the requisite cultural model and cultural
settings to support the end goal of advancing student wellbeing via the Collective Impact
framework. Discussion of the organization influences is guided by Clark and Estes (2008) and
Gallimore and Goldenberg (2001). Clark and Estes (2008) suggest that individuals’ knowledge,
skills, and motivation interact with the work environment, and successful organizations align
their culture with their organizational goals, policies, and procedures. Gallimore and Goldenberg
(2001) clarify culture further by differentiating cultural models and settings. Cultural models
include the shared mental models and values of a group of individuals, while cultural settings are
101
the more visible interactions where people come together to do something they value. Table 9
displays two validated gaps of organizational influences and context-specific recommendations,
based on an informal understanding of the organization, the stakeholders, and the literature
review.
Table 9
Summary of Organization Influences and Recommendations
Assumed Influence Gap
Validated
Priority Principle and Citation Context-Specific
Recommendation
Cultural Model –
The organization
fosters a culture of
inquiry, one that
supports failing,
learning, and
adapting as part of
the collective
learning process.
Y Y Effective change efforts
utilize feedback to
determine when/if
improvement is
happening (Clark &
Estes, 2008)
Effective change efforts
use evidence based
solutions and adapt
them, where necessary
to the organization’s
culture (Clark and
Estes, 2008).
Create feedback loop
mechanisms including
1) designated time in
SC meetings to share
best practices and
highlight “failures” to
allow for learning; 2)
biannual retreats and 3)
hiring an evaluator to
provide process
evaluation
Cultural Setting –
The organization
conveys that student
well-being is a
priority through
institutional
structures, roles, and
resources.
Y Y Effective organizations
ensure that
organizational
messages, rewards,
policies and procedures
are aligned with or
supportive of
organizational goals
and values (Clark and
Estes, 2008).
Effective change efforts
are communicated
regularly and frequently
to key stakeholders
(Clark & Estes, 2008).
Create a university-
wide governance
structure with
embedded
accountability, rewards,
policies and procedures
aligned to student
wellbeing goals;
Create a formal
communication and
feedback mechanism to
engage all stakeholders,
including students, in
the change process.
102
Create Feedback Loops to Foster a Culture of Collective Learning and Adaptability
The findings of this study indicated that SC members perceive that the DSA embraces a
culture of inquiry internally (i.e., one that supports failing, learning, and adapting as part of the
collective learning process), but they do not perceive that same culture within the larger context
of USSS due to resource constraints. Although a cultural model gap was not found to be present
in the DSA, two principles rooted in organizational change theory have been selected to bolster
or advance a culture of inquiry within the DSA. First, evidence-based solutions are used and
adapted when necessary to the organization’s culture, and second, feedback is utilized to adapt
strategy and create agile organizations (Clark & Estes, 2008; Senge, 1990). These principles
suggest that best-practices should be identified and shared among SC members, and feedback
loops utilized to celebrate successes, normalize failures, and support adaptive learning.
Therefore, the recommendation is to create a variety of feedback loops including 1) designated
time in SC meetings for members to share best practices that have demonstrated success within
the USSS organizational context and to highlight “failures” that allow for learning; 2) biannual
retreats/ forums to discuss process improvement and explore future opportunities at a macro-
scale and 3) hiring a process evaluator to provide formative/ development assessment on how the
collective is functioning. Through these feedback mechanisms, SC members can engage in
iterative cycles of continuous and adaptive learning that will support a culture of inquiry.
Embed Wellbeing throughout University Structures and Communication Mechanisms
The results and findings of this study indicated that SC members do not perceive that the
university prioritizes student well-being. Thus, the recommendations in this section are directed
towards the university at large, though they could also apply internally to the DSA. Two
principles rooted in organizational change and communication theory have been identified to
103
address this cultural setting gap. Clark and Estes (2008) describe that effective organizations
ensure that organizational messages, rewards, policies and procedures must be aligned with or
supportive of organizational values and goals. Furthermore, effective change efforts are
communicated regularly and frequently to all key stakeholders. These principles suggest that the
organization’s structure and leadership need to convey in a variety of ways that student
wellbeing is a priority. Therefore, two recommendations follow: a university-wide governance
structure should be created with embedded accountability, rewards, policies and procedures
aligned to student wellbeing goals; second, a formal communication and feedback mechanism
should be created to engage all stakeholders in the change process. For example, quarterly
meetings could be held among senior leaders, staff, and students to report out on initiatives,
gather feedback from respective stakeholders, and foster a culture of engagement.
To promote collective effort towards population health and well-being, Newton et al.
(2016) argue that high-level commitment to health and well-being needs to be communicated
through dedicated institutional structures, roles, and resources. The authors go on to explain that
this commitment could look like additional staff dedicated to health and well-being in favor of a
ecological and settings-based approach to student, staff, and faculty wellbeing; bi-directional
communication including opportunities for engagement in health and well-being efforts across
the multiple levels of the university system; and the integration of health and well-being with
other university business agendas, such as sustainability or fiscal responsibility. In this way,
communication of well-being as a priority goes beyond the traditional methods of health fairs
and mental health days; the importance of well-being as a university value is communicated
through the structures, resources, and policy decisions that align well-being to all aspects of the
university system.
104
Summary of KMO Recommendations
Evidence-based recommendations were presented for all validated knowledge,
motivation, and organizational gaps that could hinder the SC from achieving its goal of fully
establishing the five conditions of Collective Impact. Additional recommendations were offered
for those influences not validated to have a gap, but could be bolstered. Table 10 outlines all
influences and their respective context-specific recommendations.
Table 10
Summary of KMO Influences and Recommendations
Assumed KMO Influences Context- Specific Recommendations
Knowledge Influences
SC members can differentiate the role of campus
partners in accomplishing their goal (CK).
SC members know how to apply mapping
techniques to their specific initiative (PK).
SC members utilize data as a tool for reflection and
adaptation throughout the strategic planning and
implementation process (MK).
Motivation Influences
SC members see the usefulness of the common
agenda (i.e., student wellbeing) to their
departmental or unit goals. (UV)
SC members have a shared belief in the collective
action of the group to solve complex problems.
(CE)
Provide a structured chart that identifies campus partners’
respective roles in addressing student health goals.
Provide training via case studies on how to utilize
mapping techniques.
Facilitate peer modeling of using data as a tool for
reflection and adaptation, and opportunities to discuss how
SC members plan to use data for their specific initiatives.
Design an activity to help SC members identify concrete
examples of 1) how the common agenda is relevant to
departmental goals, 2) how it contributes to university
goals, 3) their specific role in the goal, and 4) benefits of
collaboration.
Create a coordinated action plan document to identify
specific contributions from each SC member, and needed
contributions from external partners. Facilitate discussion
at SC meetings for members to provide progress updates.
Facilitate a discussion to identify 1-2 powerful sponsors.
Organization Influences
The organization fosters a culture of inquiry, one
that supports failing, learning, and adapting as part
of the collective learning process. (CM)
Create feedback loop mechanisms including 1) designated
time in SC meetings to share best practices and highlight
“failures” to allow for learning; 2) biannual retreats, and 3)
hiring an evaluator to provide process evaluation.
Create a university-wide governance structure with
embedded accountability, rewards, policies and procedures
105
The organization conveys that student well-being is
a priority through institutional structures, roles, and
resources. (CS)
aligned to student wellbeing goals. Create a formal
communication and feedback mechanism to engage all
stakeholders, including students, in the change process.
Note. Knowledge influence types: CK = conceptual knowledge, PK = procedural knowledge, MK = metacognitive
knowledge. Motivation influence types: UV = utility value, CE = collective efficacy. Organizational influence types:
CM = cultural model, CS = cultural setting.
Integrated Implementation and Evaluation Plan
The USSS DSA’s mission is to advance student well-being by addressing campus-wide
public health concerns. Given the many independent units and siloed efforts within the Division,
the DSA’s goal was to utilize and implement a Collective Impact framework to orient the work
of each unit towards promoting student health and wellbeing, and coordinate, track, and evaluate
the efforts across the Division. The degree to which the DSA implements the collective impact
framework functions as an intermediate goal towards the ultimate outcome of advancing student
well-being. Since specific recommendations were offered previously to address gaps and bolster
opportunities for implementing Collective Impact, the following section provides
recommendations of what the university and the DSA can do next to advance the Collective
Impact effort. It is recommended that the organization adopt the plan or components of the plan
as outlined below to help the DSA and USSS move forward with long-term evaluation strategies.
Based on the New World Kirkpatrick Model and the four levels of training evaluation
(Kirkpatrick & Kirkpatrick, 2016), evaluation is built in to the implementation of the
recommendations to assess efficacy at multiple stages. A key feature of the New World
Kirkpatrick Model is starting with the end in mind when designing a training program
(Kirkpatrick & Kirkpatrick, 2016). According to this feature, Level 4 is the desired
organizational outcome (i.e., mission) and short term indicators that bridge behavior to
organizational results. Level 3 is the application of learning, evidenced by critical behaviors,
drivers of behavior, and on-the-job learning. Level 2 is the learning component, including
106
acquired knowledge, skills, and motivational aspects. Level 1 is the trainee’s reaction to or
satisfaction with the training itself. By planning a training program with an evaluation strategy in
this reverse order, the execution of the program is more likely to deliver a return on expectations
(Kirkpatrick & Kirkpatrick, 2016). As such, the four levels are presented below from Level 4 to
Level 1.
Level 4: Results and Leading Indicators
Table 11 refers to the Level 4: Results and Leading Indicators, based on the Kirkpatrick
New World Model (Kirkpatrick & Kirkpatrick, 2016) in the form of outcomes, metrics, and
methods. The table presents external and internal outcomes, which refer respectively to long-
term student health and wellbeing outcomes, and the team, departmental and/or organizational
outcomes. External outcomes can also be thought of as the ultimate organizational result(s), and
internal outcomes as the short term observations and measurements that indicate progress toward
the result(s). Identifying and tracking both external and internal outcomes provides helpful
information on the progression of training to performance to results.
Table 11
Outcomes, Metrics, and Methods for External and Internal Outcomes
Outcomes Metrics Methods
External Outcomes
1. Improved student health and
well-being
1a. Number of students reporting
positive/negative mental health
Annual data from the Healthy
Minds Study survey
1b. Number of students reporting fair
and equitable treatment in the
classroom
Annual data from the NACCC and
Healthy Minds Study surveys
1c. Number of students reporting at-
risk substance abuse
Annual data from the AlcoholEdu
for College and Healthy Minds
Study surveys
1d. Number of students reporting
incidents of sexual assault, and
reporting healthy relationships
Annual data from the AACU
Campus Climate Survey
107
2. Decreased student attrition
due to one of the four public
health concerns
Number of students reporting their
reason for leaving school as one of the
four public health reasons
Solicit data from Academic Review
and Retention office at the end of
each semester
3. Increased perception of
student well-being as a priority
across the university
Number of faculty and staff external to
the Division, and students who
perceive student well-being as a
priority
Create a faculty and staff survey,
and student survey to assess
perceptions
Internal Outcomes
1. Common agenda established
by the SC in conjunction with
campus partners
1a. Number of DSA and campus
partners who can identify the common
agenda of student well-being
1b. Number of DSA and campus
partners who agree that the common
agenda is relevant to the mission of
their department/unit
1c. Number of students and campus
partners involved in shaping agreed
upon solutions
Staff survey (1)
Staff survey (1)
Solicit data from backbone and/or
SC
2. Shared measurement system
created (SMS) by the backbone
2a. Data dashboard is created and
published on the web
2b. Level of SC agreement on the
student health and well-being
indicators and data collection methods
2c. Number of DSA staff and campus
partners who know where to find the
SMS and how to use it
2d. Number of DSA units and campus
partners who commit to sharing their
data on the SMS
2e. Amount of time designated at each
SC meeting for discussion of/
reflection on data
Solicit information from the
backbone and/or SC
Data from interviews regarding
Collective Impact process
Staff survey (1)
Staff survey (1)
SC meeting minutes
3. DSA unit activities are
aligned to and mutually
reinforcing of student well-
being (i.e., a coordinated plan
of action)
3a. Number of DSA units and campus
partners who can identify their
contribution to a coordinated action
plan
3b. Level of agreement from DSA unit
leaders and campus partners that unit
activities have changed to align to
coordinated action plan
Staff survey (2)
Staff survey (2)
108
3c. Number of working groups/ action
teams carrying out the strategic actions
of the coordinated action plan
Solicit from backbone and/
or SC
4. Strategies and mechanisms
are utilized across the DSA to
support continuous
communication
4a. Number of SC meetings and
working group meetings each semester
4b. Attendance at SC and working
group meetings
4c. Number of DSA or university wide
communications focused on student
wellbeing each year
4d. Number of DSA and university job
descriptions that designate a
percentage of FTE to student wellbeing
(For 4a-4c) Solicit information
from backbone support
organization (i.e., administrative
support team) by requesting/
reviewing meeting minutes
Solicit from Human Resources
5. Backbone support
organization is established and
providing administrative
support
5a. Number of DSA units who can
identify the backbone support
organization and its role
5b. Number of meetings the backbone
support organization has with the SC
and partner units
5c. Number and diversity of new units/
departments engaged in the SC each
year
Staff survey (1)
Solicit information from backbone
support organization
Solicit information from backbone
support organization
Level 3: Critical Behaviors
Given that the Collective Impact initiative was in the “early years” of implementation
(Preskill et al., n.d.), having been in existence for 18 months at the time of this evaluation, the SC
had progressed through much of the initial design and implementation phase. This section
therefore provides suggestions for critical behaviors to address the deficits identified in the
findings, and advance the initiative into the next phase. Table 12 illustrates these critical
behaviors along with a metric, method for implementation, and timeline. The first critical
behavior is that SC members engage in a participatory process through focus groups, surveys,
and town halls with students, staff, and external stakeholders (i.e., other campus partners,
parents, alumni, etc.) to highlight why the common agenda is important to the university mission,
and to solicit contextual feedback to design agreed upon solutions. The second critical behavior
109
is that SC members engage in a participatory process to identify a common set of indicators, data
collection methods to determine progress, and regularly review data for reflection and adaptation
of strategies. The third critical behavior is that SC members need to create a coordinated action
plan, with identified partners for each activity. The action plan should be based on best-practices
in addressing student health and wellbeing, contextualized to the setting, regularly updated and
readily available for partner units to view. The fourth critical behavior is that SC members foster
continuous communication of student well-being efforts throughout and outside the DSA, which
includes regular meetings as a steering committee to share updates and best-practices, providing
regular updates to unit staff and to external partners as appropriate and dedicating a percentage
of staff time (i.e. in the form of job descriptions) to advancing student wellbeing. The fifth and
final critical behavior is that SC members engage partners work collaboratively with the
backbone and consistently connect campus partners and DSA units with the backbone to advance
the common agenda. The specific metrics, methods, and timing for these critical behaviors are
described in Table 12.
Table 12
Critical Behaviors, Metrics, Methods, and Timing for Evaluation
Critical Behavior Metric(s)
Method(s)
Timing
1. SC members will
engage in a participatory
process with students,
staff, and external
stakeholders to highlight
importance of common
agenda and solicit
feedback to design
solutions
Number of students, staff,
and external stakeholders
engaged
SC will conduct a series of
focus groups, surveys, and a
convening with campus
partners, students, and staff
to determine priorities
SC members will review
existing data to identify
priority areas
Year 1 of the strategic
planning process (i.e.,
year 0), and again at
strategic intervals (every
2-3 years)
2. SC members will
engage in a participatory
process to identify a
common set of indicators
and data collection
methods that determine
Number of DSA staff
engaged in the process of
determining indicators
SC members will convene
DSA leaders and staff to
identify key performance
indicators
Year 1 of the
implementation phase of
the Strategic Plan
110
progress towards student
well-being
Frequency that SC
members access,
reference, and reflect on
the indicators and
available data
SC members will revisit
indicators and data
throughout the initiative to
assess progress
Ongoing, at least once
each semester
3. SC members will create
a coordinated action plan
to align unit activities
towards student well-being
Number of units identified
in the action plan
SC members will work
collaboratively with the
backbone support
organization to create an
action plan for alignment
Year 1 of
implementation of the
Strategic Plan
4. SC members will foster
continuous communication
of student well-being
efforts throughout and
outside the Division
Number of reports or
communication methods
that SC members establish
or utilize to disseminate
information
SC members will conduct
regular meetings with their
unit(s) to disseminate
pertinent information
SC members will meet
regularly as a committee to
disseminate information of
various efforts and share
best-practices
SC members will engage
external stakeholders through
other meeting structures to
share student well-being
efforts
SC members will dedicate
staff time and resources to
create an internal
communication system
Once per month during
all years of
implementation
Once per month during
all years of
implementation
Ongoing as meetings
with external
stakeholders occur
Established year 1 of the
implementation phase of
the Strategic Plan, and
continuously throughout
all years of
implementation
5. SC members will
identify an appropriate
backbone support
organization and work
collaboratively with them
to advance the common
agenda
Level of expertise of
backbone members with
the common agenda
Agreement across DSA
leaders and staff of which
individuals, units, or
external partners should
function in the backbone
role
Representativeness of
backbone members of the
involved sectors and
constituencies
SC will convene DSA leaders
and staff to discuss the role
of the backbone support
organization and the common
agenda, and facilitate
conversation about which
individuals, units, or external
partner would best function
in the role and represent the
involved sectors and
constituencies
Year 1 of the strategic
planning process (i.e.,
year 0)
111
Required Drivers
As described in sociocultural theory, one’s ability to sustain a behavior or behavior
change must be supported and reinforced by motivational aspects and the surrounding culture or
environment (Scott & Palinscar, 2006). Thus, SC member behavior can be supported by drivers
such as processes and systems that reinforce, encourage and reward performance, and also
through accountability measures that monitor performance (Kirkpatrick & Kirkpatrick, 2016).
The critical behaviors revolve around implementation of the five conditions of Collective
Impact, and as shown below, are reinforced by a newly created job aid that identifies campus
partners and their role in the Collective Impact initiative, as well as peer modeling and time set
aside during meetings to discuss the use of data for reflection and strategic planning.
Incentivizing the behaviors through rewards and encouragement include opportunities to
strengthen collective efficacy within the SC and provide public recognition for advancing the
work of the Collective Impact initiative. Table 13 shows recommended drivers to support critical
behaviors of SC members as they strive to implement the Collective Impact framework.
112
Table 13
Required Drivers to Support Critical Behaviors
Method(s) Timing
Critical Behaviors
Supported
Reinforcing
Job aid with a clearly structured chart that identifies each
campus partner and their respective role in accomplishing the
organizational goal.
During year 1 and
ongoing as new
partners identified
1, 3, 4
Peer modeling of the use of data as a tool for reflection and
adaptation during SC meetings.
Ongoing
2, 4
Time set aside during meetings for SC members to discuss
how they plan to use data for their specific initiatives.
Ongoing 2, 4
Encouraging
Opportunities during SC meetings for members to discuss
how the common agenda and associated activities are relevant
to their individual unit/departmental goals.
Quarterly 1, 3, 4
Opportunities during SC meetings for members to identify
ways in which they are working collaboratively, and discuss
how their respective efforts contribute to the overarching goal
of student well-being.
Monthly 2, 3, 4
Rewarding
Public recognition of SC in memo from the Provost or
President when one or more of Collective Impact conditions
are established/ advanced.
Each semester 1, 2, 3, 4, 5
Performance incentive for new members who join the
Steering Committee and align their units’ activities to the
action plan.
As completed 3
Monitoring
Conduct interviews and process evaluation to determine
progress on and barriers to each condition of Collective
Impact.
Annually 1, 2, 3, 4, 5
Conduct whole organization town halls to communicate status
updates, successes, and challenges.
Quarterly 1, 2, 3, 4, 5
Organizational support
The aforementioned critical behaviors and required drivers for SC member success are
also supported by organizational constructs, processes, and resources. In the context of
Collective Impact, in order for SC members to achieve their goals, the organization would need
to convey the importance of student well-being through institutional structures, roles, and
113
resources. Senior administrators within the DSA (including SC members), but also senior
administrators from the university (i.e., Provost and President) would examine current resource
allocations and organizational structures and redistribute as necessary to ensure that student
health and well-being are communicated across the university as a priority. For example, the SC
is supported by the backbone support organization. For health and wellbeing-related Collective
Impact efforts such as the one at USSS, the SC and senior administrators could consider forming
an independent unit comprised of health promotion/public health professionals, project
managers, public policy experts, community engagement specialists, learning and development
specialists, and researchers/program evaluators. This cross-functional team could support the
various needs of the SC through data and research, administrative support, and policy
recommendations. Another example of organizational support would be to build in
accountability for student health and wellbeing to job descriptions of key positions across the
university, e.g. Director of Housing, Transportation, Hospitality, Public Safety, and senior level
administrators within academic units. Combined these changes could communicate to
stakeholders across the university that the organization is supporting the SC and internal units in
carrying out the common agenda of student well-being.
The organization would also need to support SC members through fostering a culture of
continuous learning and adaptability. In order to continuously learn, SC members need data and
research support to be able to reflect on student health and well-being problems and successes;
they also need capacity-building opportunities (i.e., time and training) in order to make informed
decisions on how to proceed. The organization would need to invest in survey instruments to
collect data, and provide resources in the form of personnel to conduct research and program
evaluation and regularly offer training on best-practices. This support role for the SC and
114
Collective Impact effort could also be performed by the backbone organization, and would
reinforce the message as mentioned above that student well-being is a priority.
Level 2: Learning
Following completion of the recommended solutions, specifically the training for SC
members, stakeholders will be able to:
1. Differentiate the role of campus partners in accomplishing the organizational goal (i.e.,
student well-being), (conceptual knowledge)
2. Utilize mapping techniques specific to public health problems, (procedural knowledge)
3. Utilize data throughout the strategic planning process as a tool for reflection and
adaptation, (metacognitive knowledge)
4. Identify an example of how student well-being is relevant to their respective
unit/departmental goals, (utility value)
5. Compare and contrast individual unit efforts to collective efforts of the group (SC) in
relation to accomplishing the strategic goals (collective efficacy)
Program
The learning goals listed above could be achieved through a two-pronged training
approach. The first component (part 1) would occur at the start of a Collective Impact effort, and
annually thereafter at an SC reflection and planning retreat; the second (part 2) would occur
monthly during the first year and quarterly over subsequent years as the SC membership evolves.
The participants, in this case SC members or other senior leaders in the DSA, engage in a variety
of face-to-face activities and learning experiences with their colleagues, content experts, and/or
the backbone support organization. Part 1 of the program engages all SC members in a half day
115
(~4 hour) Student Well-being Convening to provide foundational knowledge and motivation,
while part 2 provides ongoing training and organizational support.
At the start of a Collective Impact effort, and annually thereafter as part of a reflection
and planning process, a convening of SC members (part 1) could provide and reinforce key
aspects of a successful Collective Impact process. The learners, in this case the SC members,
would come together with the backbone support organization in a face-to-face retreat style
setting to explore knowledge, motivation, and organizational aspects necessary to advance the
agenda of student well-being. To emphasize cross-sector leadership and collective efficacy, the
retreat begins with a discussion of existing student health and well-being data, with the
opportunity for SC members to reflect on how problems related to student well-being materialize
in their respective unit (i.e., Residential Education, Careers Services, etc.). SC members with
expertise in the use of data and/or student well-being (e.g., Student Health or Health Promotion
professionals) provide peer modeling of how data can be utilized as a reflection tool or to inform
a specific initiative. This is a key point when disaggregation of data is demonstrated to illustrate
potential disparities in health outcomes. Training is provided via case studies utilizing mapping
techniques which identify geographic locations or density of health-related problems, as well as
asset mapping techniques to identify where potential assets exist in the community of focus. SC
members also receive a job aid in the form of an action plan for wellbeing and discuss their role
and campus partners’ role(s) in enhancing student well-being. The SC then has the opportunity to
bring their local knowledge and expertise to expand and modify that chart as necessary. Finally,
the SC engages in an initial discussion of how to communicate the student well-being agenda
and various change efforts across the Division. They discuss how to build in rewards or incentive
116
structures for the front-line staff who will potentially be adapting their daily roles to align
towards student well-being, and the type, timing, and delivery mechanism for the content.
Part 2 of the training, which occurs monthly during the first year and quarterly thereafter,
provides ongoing learning opportunities for the SC. While the content areas provided are by no
means comprehensive, they offer a starting point for a structured training program that meets the
needs of a SC in the early years of a Collective Impact initiative. After the part 1 convening,
additional training is provided on how to conduct mapping techniques. This is facilitated through
the use of case studies and examples from the literature on how mapping techniques have been
used in other Collective Impact initiatives. As a next step in the training curriculum, the SC
discusses and strategizes a formal communication and feedback mechanism to engage their
front-line staff in the effort. This feedback mechanism for DSA staff is context- and culture-
specific to the DSA organization and heavily involves the backbone organization to help
facilitate the process (e.g., conducting interviews and focus groups with staff). As a third
opportunity for training, the SC conducts a 90-180 minute workshop with content experts to
discuss each student health and well-being priority area that surfaced from the convening, and
how their specific unit could align or contribute to the goals set for that priority area. For
example, if 4 health and well-being priority areas were determined, each would be discussed
during its own workshop. Finally, in another monthly meeting, the SC discusses how they can
implement a formal feedback loop mechanism in their SC meetings in order to share best-
practices and lessons learned while proceeding through the Collective Impact process. Again, the
backbone organization assists with facilitating this process in order to provide core
administrative support for the SC throughout the process.
Evaluation of the components of learning
117
Once the learning goals are established, Kirkpatrick and Kirkpatrick suggest that
learning, in the form of knowledge, skills, attitudes, confidence and commitment should be
evaluated in a formative way (2016). Evaluation tools include assessments, discussions,
individual and group activities, and role plays and simulations, which can be built directly into
the training or take place immediately after. Aligned with the Kirkpatrick and Kirkpatrick New
World Model (2016), Table 14 details the evaluation plan for the aforementioned program.
118
Table 14
Evaluation of the Components of Learning for the Program
Methods or Activities Timing
Declarative Knowledge “I know it.”
Knowledge checks through discussions, “think, pair,
share” and other individual/ group activities.
During part 1 after the job aid on campus partners has
been distributed
Demonstration in groups and individually using the job
aid to successfully identify and differentiate campus
partners.
During the part 2 meeting specific to identifying campus
partners
Procedural Skills “I can do it right now.”
Demonstration in groups and individually using the job
aid to successfully perform the mapping techniques.
During part 1 and part 2 (monthly meeting)
Retrospective pre- and post- test assessment survey
asking SC members about their level of proficiency in
conducting mapping techniques before and after the
convening/monthly session.
Pre- and post- convening (part 1) and again after the
monthly session on mapping (part 2)
Individual application of the data use skills (reflection)
in discussions of health and well-being priority areas.
During the monthly meetings (part 2)
Attitude “I believe this is worthwhile.”
Discussions of the value of student well-being to their
respective unit/department.
During the convening (part 1) and again at the monthly
sessions (part 2)
Pre- and post-test assessment item (+ additional
formative assessments) on the value of working
collaboratively to solve student health and well-being
issues, and the importance of aligning their unit/
departmental work to the common agenda.
Pre- and post during the convening (part 1) and
subsequent assessments every 6 months at a monthly
meeting
Confidence “I think I can do it on the job.”
Retrospective pre- and post-test assessment item on the
value of working collaboratively to solve student health
and well-being issues.
Pre- and post during the convening (part 1) and
subsequent assessments every 6 months at a monthly
meeting
Discussions on how SC members are aligning their units
to the common agenda.
During the monthly meetings (part 2)
Commitment “I will do it on the job.”
Creation of unit/departmental alignment plans. During the monthly sessions (part 2)
Discussions following practice and feedback.
During the monthly sessions (part 2)
119
Level 1: Reaction
Level 1 evaluation examines participant engagement, relevance of the material, and
satisfaction. Data should be collected through both formative and summative tools, and can
include observations and pulse checks, as well as post-program surveys. Table 15 outlines the
components to measure level 1 reactions to the program.
Table 15
Components to Measure Reactions to the Program
Method(s) or Tool(s) Timing
Engagement
Presenter observations of SC members’ statements and
actions related to student well-being and the
collaborative process (CI).
During Student Well-being Convening (part 1)
Pre- and post- test assessment of interest and value of
student health and well-being.
Before and after each annual Student Well-being
Convening (part 1)
Workshop evaluations. Following each monthly session in year 1 and quarterly
sessions thereafter (part 2)
SC member attendance (in terms of total number of
meetings attended and amount of time attended at each
meeting).
During the convening (part 1) and at monthly/ quarterly
sessions thereafter (part 2)
Relevance
Brief check-ins with SC members via discussions. During each monthly/quarterly session (part 2)
Topic and content evaluation. Following the Student Well-being Convening (part 1)
and monthly sessions thereafter (part 2)
Satisfaction
Program evaluation. At the end of each academic school year
Ongoing check-ins/ interviews with SC members. Once a semester
Evaluation Tools
As outlined by Kirkpatrick and Kirkpatrick’s Blended Evaluation Approach (2016), an
integrated implementation and evaluation plan is created concurrently so that evaluation tools
120
and methods are relevant and timely. Level 1 Reaction and Level 2 learning are simultaneously
assessed during and immediately after the program via formative and summative tools. Level 3
behavior and Level 4 results are evaluated after the training and again at distanced intervals to
capture longer term outcomes. This section identifies tools for the immediate evaluation after the
training, and after a period of delay.
Immediately following the program implementation
In addition to the methods listed above, level 1 (reaction) and level 2 (learning) are
assessed through the use of a survey administered by the backbone support organization. To
assess the quality of the training and participant commitment and confidence to implement what
they learned, a blended survey gathers data after Part 1 of the program (See Appendix B for the
survey). For level 1, immediately after the Student Well-being Convening (part 1 of the training),
the survey asks SC members about the value of student health and well-being to their work, and
the relevance of the information presented. For level 2, SC members answer questions related to
their skills and motivation, specifically their level of proficiency in conducting mapping
techniques, the degree to which they value working collaboratively to solve student health and
well-being issues, and the importance of aligning their unit or departmental work to the common
agenda of student well-being.
Delayed for a period after the program implementation
Approximately 8 weeks after the implementation of part 1 of the training, and then again
at intervals matched to part 2 sessions, the backbone organization administers a survey to
measure SC member satisfaction and relevance of part 1 and part 2 trainings (level 1); their
confidence and commitment to applying what they learned in their work (level 2); how they are
transferring knowledge and skills from the training to their work (level 3); and the extent to
121
which their training has contributed to the full establishment of the 5 conditions of Collective
Impact (level 4). See Appendix C for the delayed survey instrument.
Data Analysis and Reporting
The level 4 goal is measured by the degree of implementation of the five conditions of
Collective Impact. Evaluation of the early years (i.e., first 1-3 years) of a Collective Impact
initiative should provide indicators of progress towards establishing the infrastructure of the
Collective Impact effort, while the middle years (i.e., 3-7 years) should be evaluated for
intermediate outcomes such as behavioral and systems level changes, and then population-level
impact during the later years (i.e., 7-10 years) (Preskill et al., n.d.). Thus, as mentioned
previously, it should be noted that the ultimate results related to student health and well-being
will likely not materialize or be measurable until many years after the initiation of the Collective
Impact effort. However, the SC and backbone can track progress on the design and
implementation of the Collective Impact infrastructure on an annual basis during the early years
to determine the development of each condition. This input should be gathered qualitatively via
interviews, surveys, and focus groups with SC members, and each SC member and the backbone
organization should participate in order to identify the collective perception on how the
Collective Impact initiative is advancing. Furthermore, this information should be shared with
the community, specifically campus partners and students, via a campus website to foster a
culture of transparency and accountability. Appendix D demonstrates a dashboard displaying
example data for leading indicators of the implementation of Collective Impact. The colors
indicate the level of performance on each component, as well as an indication of trend. It is also
recommended that the SC and/or backbone include data from Level 1, 2, and 3 evaluations as
well for a more comprehensive representation of the Collective Impact effort.
122
Summary
The New World Kirkpatrick Model (Kirkpatrick & Kirkpatrick, 2016) provides an
evidence-based method for the planning, delivery and evaluation of a training program, thereby
increasing the odds that targeted outcomes will be reached. With a complex, learning-centered
framework such as Collective Impact, the New World Kirkpatrick Model creates a structure for
those involved in the effort to assess progress during the developmental phases (i.e., the early
years) and similarly allows for flexibility as the training components and needs evolve. The
utilization of the Collective Impact framework is not an end, but a means to the end of
addressing a large, complex social issue, such as the advancement of student health and well-
being. Thus, the stakeholder goal of implementing the five conditions of Collective Impact
requires process-related objectives and evaluation of those objectives to guide those involved. By
integrating the New World Kirkpatrick Model with the Collective Impact process, the formative
information can facilitate quick learning and adaptation for those who are implementing the
framework (i.e., SC and backbone organization), while the summative data collected on an
annual basis can offer insight into the development of the infrastructure towards the desired
outcomes.
The mission of the USSS DSA is to support and enhance student well-being. While this is
the desired organizational impact (Level 4: Results), student well-being is often defined
differently by different people as discussed in Chapter 2, and unfortunately is not measurable by
a single construct at this point. Thus, population level metrics that indicate progress on critical
public health outcomes such as substance abuse and mental health can assist in “defining”
student well-being. Furthermore, identifying leading indicators for those who are engaged in the
work provides “metrics that personalize Level 4 results for departments and individuals, helping
123
them to see how they contribute to the whole” (Kirkpatrick Partners, n.d.). In this case, the
leading indicators are those related to the implementation of the Collective Impact framework,
specifically for SC members to improve the likelihood that they will be able to enact change at
the systems level as opposed to creating disconnected programming or policies. Discussing and
identifying critical behaviors based on the literature (Level 3) defines the specific roles of the SC
members, while the required drivers reinforce, monitor, encourage and reward SC members for
carrying out the behaviors. If it is determined that SC members require knowledge, skills or
attitudes to facilitate the critical behaviors, this is where a training component is necessary
(Level 2). It should also be noted that continuous learning is important to the Collective Impact
process, and should be framed as a value add as the SC and Collective Impact effort evolves. As
such, level 3 and 4 outcomes should be monitored consistently so that SC members can identify
when required drivers, critical behaviors, and leading indicators are advancing, plateauing, or
falling behind target. This will allow for agility and adaptation as SC members continue to
navigate the ever-changing landscape of a complex process towards a complex outcome.
Limitations and Delimitations
Limitations and delimitations should be considered in the interpretation of this study.
Limitations, those factors which are out of the researcher’s locus of control, include 1) the USSS
was in the midst of extreme change and leadership turnover at the time of this study, 2)
availability of extant literature on collective impact in higher education, and 3) the willingness of
the research participants to fully disclose their thoughts and opinions to the researcher. Without
disclosing the extent and detail of the wide-reaching culture change taking place at USSS, the
researcher was aware that participants’ answers were often framed in a “before” and “after” X
incidents and turnover of key university administrators. The researcher addressed this factor by
124
asking participants for their current assessment of a given construct, as well as their recollection
of their prior assessments. Even with this tactic, the data reflect the SC member’s thoughts and
opinions in a unique time in the university’s history. Although very limited literature existed on
Collective Impact in higher education at the time of this study, adequate research was available
on cross-sector collaboration, collective efficacy, shared governance, and the settings approach
to health promotion in order to ground the study. There was also relatively novel research
emerging on Collective Impact in other settings, though very few were empirical studies given
the complexity of evaluating Collective Impact. Further, and as mentioned previously, since the
researcher was colleagues with the SC members and functioned as part of the backbone,
responses could have been biased, in particular on questions regarding the backbone so as not to
offend the researcher. When weighing the advantages and disadvantages to conducting their own
data collection, the researcher decided that the advantages of established trust and informality
with the SC members outweighed the potential disadvantages. It is also important to note that the
researcher was not in a position of authority to any of the SC members.
Delimitations, or those factors which are within the researcher’s control, include: 1) a
single stakeholder group, 2) a singular data collection method, and 3) generalizability. Given the
limited data collection period and scope of this study, a single stakeholder group (i.e., the
Steering Committee) was selected for this research. Although this factor provided a unique lens
from an important stakeholder group, a more comprehensive study would have also included
front-line and middle management staff, the individuals in the backbone organization, and even
key student leaders. Second, the researcher intended to collect and triangulate data from three
sources – interviews, a focus group, and document analysis. Interviews were conducted with
100% of stakeholders, though it proved incredibly difficult to gather all individuals for a focus
125
group due to availability. For document analysis, the researcher originally intended to gather and
analyze all memos from USSS senior leadership (i.e., President and Provost) to assess
communication of student wellbeing as a priority. It was decided that document analysis would
not supplement the data collection from the interviews since the study intended to study insights
from the lens of Steering Committee members, none of which would have been involved in the
writing of such memos. Further, data from the interviews supported that communication took
place in a variety of methods other than the traditional format of memos. Finally, due to the
specificity of this study, in particular the structure of the Collective Impact effort, its early phase,
and the organizational current at the time, the results are not necessarily generalizable to other
institutions of higher education. The results however could provide insight into lessons learned
and guide other institutions on one method for evaluating Collective Impact (i.e., the KMO gap
analysis framework).
Future Research
This study provided insight on the implementation of Collective Impact from the
perspectives of senior Student Affairs Administrators whose common agenda was to advance the
wellbeing of the student population. The research also identified gaps and opportunities in terms
of stakeholder knowledge and motivation, and the organizational factors that affected their
ability to meet their organizational goal. Future research would benefit from the following: 1)
evaluating the implementation of Collective Impact in higher education from a broader
stakeholder group, 2) evaluating Collective Impact in higher education with different governance
structures, and 3) evaluating Collective Impact in higher education with a different common
agenda.
126
Considering that the literature on Collective Impact in higher education is limited, the
research community would benefit from exploring and evaluating Collective Impact in ways that
expand from the research in this study. For example, this study examined the implementation of
Collective Impact from one stakeholder group, specifically senior Student Affairs administrators
that functioned collectively as the Steering Committee for the initiative. Although this
stakeholder group largely guided the Collective Impact effort, a number of mid-level
professionals and front-line staff were also involved, and therefore likely affected the trajectory
of the effort. Thus, conducting research on the implementation of Collective Impact from the
perspective of those aforementioned stakeholders, specifically those who are on the receiving
end of communication and direction from the Steering Committee members, would provide a
more robust conceptualization of the full Collective Impact effort. Additional research should
also be conducted on Collective Impact efforts that have a different structure within the higher
education setting. For example, at the time of conducting this study, the researcher became aware
of other institutions in which the Collective Impact effort had a similar common agenda, but was
spearheaded by faculty members, primarily in the health care and public health disciplines, or by
senior administrators outside of the Student Affairs profession. Evaluating these types of
Collective Impact initiatives could provide valuable comparisons and unique insights not found
in this research. Finally, given that this research focused on an organizational goal related to
wellbeing, which as described earlier has multiple dimensions and an evolving definition,
additional research is warranted on Collective Impact with a different, and possibly more discrete
common agenda. For example, a large focus of the community of Collective Impact practitioners
has been on advancing equity. While equitable health outcomes are inherent in the outcome of
wellbeing, evaluating Collective Impact with the common agenda of equitable outcomes in other
127
disciplines (i.e. educational access, retention, etc.) can support the body of literature on
Collective Impact and can help to advance the missions of many institutions of higher education.
Conclusion
Institutions of higher education are facing increasingly complex public health problems
among their student populations and are struggling to find solutions that leverage the necessary
and complimentary assets of the broad array of actors in the university settings. As one
participant from this study proclaimed, “Complex problems require complex solutions” (SC
member 2). As such, in 2017, the University of Siloed Student Services Division of Student
Affairs set out on a mission to advance student wellbeing utilizing a framework that had shown
promising results in addressing complex issues in other settings such as cities, municipalities and
the K-12 sector. Collective Impact, a framework first described by Kania and Kramer in 2011,
calls for diverse actors from a variety of sectors to agree to a common agenda, establish and
share metrics to measure progress, align their respective activities in a coordinated way towards
their common goal(s), communicate continuously across multiple mediums, and identify a
coordinating body of individuals to function as the backbone support organization. The common
agenda of advancing student wellbeing via four public health goals was a distal goal that would
likely take many years to materialize, given the nature and pace of change related to public
health interventions. However, this study offered a formative evaluation and insight into the
early years of implementation of Collective Impact, specifically from the perspective of the
Steering Committee (SC), with the hope that the findings and recommendations could help guide
the future decisions of both the stakeholder group and the university at large.
Steering Committee members play an important role in guiding the Collective Impact
effort. They are involved in the identification of the common agenda, contributing to the shared
128
metrics that will measure success, adapting or adding activities within their unit or department to
align to the coordinated strategies necessary to accomplish the goal(s), communicating both
within and outside their respective departments, and collaborating consistently with the backbone
organization. Working in such a collaborative manner and within the above structure may or may
not be familiar or aligned with the values of SC members. Further, organizational influences may
bolster or impede SC members from pursuing work in such a collective approach. Utilizing the
Clark and Estes (2008) gap analysis framework, this study examined Steering Committee
member knowledge and motivation, and the organizational influences that affected the
accomplishment of their goal. Based on the analysis, three main findings emerged: 1) Collective
Impact creates an infrastructure for collaboration and has motivational effects when aligned to
unit goals; 2) To advance the wellbeing of the student population via Collective Impact, the
common agenda needs to be a university-wide initiative; and 3) Progress had been made and
opportunities existed to fully establish the five conditions of Collective Impact to advance
student wellbeing. Specific knowledge, motivation, and organizational factors were also
identified across the three main findings, and recommendations were offered to address gaps and
provide opportunities for advancing and evaluating the implementation of Collective Impact at
USSS.
Collective Impact is new to the higher education setting, and as such, much remains to be
seen in terms of its effectiveness. One promising feature is that Collective Impact can be adapted
and contextualized to the setting in order to provide a framework for collaboration towards a
common goal. As it pertains to a complex goal such as advancing student wellbeing in the
university setting, which requires the commitment, input, and structural support from
individuals, groups, and the organization at large, there is no “silver bullet” solution. Whether it
129
is Collective Impact or another framework for multi-sectoral collaborative work, wellbeing is a
communal construct requiring communal efforts. Investing in such efforts is not only important,
but necessary to continue to advance the mission of higher education and prepare our future
leaders to address the complex issues of their future world.
130
References
Alatartseva, E., & Barysheva, G. (2015). Well-being: Subjective and Objective Aspects.
Procedia - Social and Behavioral Sciences, 166(C), 36–42.
https://doi.org/10.1016/j.sbspro.2014.12.479
Altaweel, M. (2017). Density mapping. https://www.gislounge.com/density-mapping
Amed, S., Naylor, P., Pinkney, S., Shea, S., Masse, L., Berg, S., . . . Higgins, J. (2015). Creating
a collective impact on childhood obesity: Lessons from the scope initiative. Canadian
Journal of Public Health, 106(6), E426-E433.
American College Health Association. (2012). Healthy campus 2020: History and development.
History and Development. http://www.acha.org/HealthyCampus/About/
History_and_Development/HealthyCampus/History_and_Development.aspx?hkey=6a2c
1c14-2a48-466a-af4e-acd9faa5b5da.
Antonovsky, A. (1979). Health, Stress and Coping. (1st ed.). Jossey-Bass, San Francisco
Antonovsky, A. (1987). Unraveling the mystery of health: how people manage stress and stay
well (1st ed.). San Francisco: Jossey-Bass.
Antonovsky, A. (1998). The salutogenic model as a theory to guide health promotion. Megamot,
XXXIX(1-2), 170-182. http://libproxy.usc.edu/login?url=https://search
.proquest.com/docview/38691824?accountid=14749
Baddeley, A. D. (1992). Working memory. Science, 255(5044), 556–559.
Bandura, A. (2000). Exercise of human agency through collective efficacy. Current Directions in
Psychological Science, 9(3), 75–78. https://doi.org/10.1111/1467-8721.00064
131
Baum, F., & Fisher, M. (2014). Why behavioural health promotion endures despite its failure to
reduce health inequities. Sociology of Health & Illness, 36(2), 213–225.
https://doi.org/10.1111/1467-9566.12112
Best, A., Stokols, D., Green, L. W., Leischow, S., Holmes, B., & Buchholz, K. (2003). An
integrative framework for community partnering to translate theory into effective health
promotion strategy. American Journal of Health Promotion, 18(2), 168-176.
https://doi.org/10.4278/0890-1171-18.2.168
Bogdan, R., & Biklen, S. (2007). Qualitative research for education: an introduction to theories
and methods (5th ed.). Boston, Mass.: Pearson A & B.
Bowen, G, (2009). Document Analysis as a Qualitative Research Method. Qualitative Research
Journal, 9(2), 27-40.
Brock M, Carr J. W., Todd, M.K. (2015). An examination of campus recreation usage, academic
performance, and selected health indices of college freshmen. Recreational Sports Journal.
39(1), 27–36. https://doi.org/10.1123/rsj.2014-0061
Bronfenbrenner, U. (1989). Ecological systems theory. In R. Vasta (Ed.), Annals of child
development, Vol. 6 (pp. 187–249). Greenwich, CT: JAI Press.
Cabaj, M., & Weaver, L. (2016). Collective Impact 3.0: An evolving framework for community
change. Canada: Tamarack Institute.
Cabrera, D. & Cabrera, L. (2015). Systems thinking made simple: New hope for solving wicked
problems. Odyssean Press.
Cawood, J., Dooris, M., & Powell, S. (2010). Healthy universities: Shaping the
future. Perspectives in Public Health, 130(6), 259-260.
https://doi.org/10.1177/1757913910384055
132
Christens, B., & Inzeo, P. (2015). Widening the view: Situating collective impact among
frameworks for community-led change. Community Development, 36(1), 1-16.
https://doi.org/10.1080/15575330.2015.1061680
Clark, R. E., & Estes, F. (2008). Turning research into results: A guide to selecting the right
performance solutions. Charlotte, NC: Information Age Publishing, Inc.
Connell, J. I., Kubisch, A. C., Schorr, L. B., & Weiss, C. H. (1995). New approaches to
evaluating community initiatives: Concepts, methods, and contexts. (Connell, J.I., Kubisch,
A.C., Schorr, L. B. & Weiss, C. H. Eds.) Cycle, 65. Washington, D.C.: The Aspen Institute.
Cooke, P.J., Melchert, T.P., & Connor, K. (2016). Measuring Well-being: A review of
instruments. The Counseling Psychologist, 44(5), 730-757.
https://doi.org/10.1177/0011000016633507
Cowan, N. (2000). The magical number 4 in short-term memory: A reconsideration of mental
storage capacity. Behavioral and Brain Sciences, 24(87-185).
Creswell, J. W. (2014). Research design: Qualitative, quantitative, and mixed methods
approaches (4th ed.). Thousand Oaks, CA: Sage Publications.
De Montigny, J., Desjardins, S. & Bouchard, L. (2017). The fundamentals of cross-sector
collaboration for social change to promote population health. Global Health Promotion,
16(1). https://doi.org/10.1177/1757975917714036
Dooris, M. (2001). The "health promoting university": A critical exploration of theory and
practice. Health Education, 101(2), 51-60. https://doi.org/10.1108/09654280110384108
Dooris, M. (2004). Joining up settings for health: A valuable investment for strategic
partnerships? Critical Public Health, 14(1), 49-61.
https://doi.org/10.1080/09581590310001647506
133
Dooris, M. (2006). Healthy settings: Challenges to generating evidence of effectiveness. Health
Promotion International, 21(1), 55-65. https://doi.org/10.1093/heapro/dai030
Dooris, M. (2009). Holistic and sustainable health improvement: the contribution of the settings-
based approach to health promotion. Perspective in Public Health, 129(1), 29-36.
https://doi.org/10.1177/1757913908098881
Dooris, M., & Doherty, S. (2010). Healthy universities: Current activity and future directions -
findings and reflections from a national-level qualitative research study. Global Health
Promotion, 17(3), 6. https://doi.org/10.1177/1757975910375165
Dooris, M. (2013). Expert voices for change: Bridging the silos-towards healthy and sustainable
settings for the 21st century. Health & Place, 20, 39-50.
https://doi.org/10.1016/j.healthplace.2012.11.009
Dooris, M., Wills, J., & Newton, J. (2014). Theorizing healthy settings: A critical discussion
with reference to healthy universities. Scandinavian Journal of Public
Health, 42(15_suppl), 7-16. https://doi.org/10.1177/1403494814544495
Dooris, M., Wills, J., Newton, J., Torp, S., Kokko, S. & Ringsberg, K. (2014). Theorizing
healthy settings: a critical discussion with reference to Healthy Universities. Scandinavian
Journal of Public Health, 42(15), 7–16. https://doi.org/10.1177/1403494814544495
Eccles, J. (2006). Expectancy value motivational theory. http://www.
education.com/reference/article/expectancy-value-motivational-theory/.
Eifert, E. K., Hall, M. E., Gropper, S. S., & Kondor, M. (2017). Health promotion and
institutions of higher education: One university's experience. American Journal of Health
Education, 48(4), 219. https://doi.org/10.1080/19325037.2017.1316691
134
Erez, M. & Gati, E. (2004). A dynamic, multi-level model of culture: From the micro level of the
individual to the macro level of a global culture. Applied Psychology: An International
Review, 53(4), 583–598. https://doi.org/10.1111/j.1464-0597.2004.00190.x
Flood, J., Minkler, M., Hennessey Lavery, S., Estrada, J. & Falbe, J. (2015). The Collective
Impact Model and Its Potential for Health Promotion. Health Education & Behavior, 42(5),
654-668. https://doi.org/10.1177/1090198115577372
Gallimore, R. & Goldenberg, C. (2001). Analyzing cultural models and settings to connect
minority achievement and school improvement research. Educational Psychologist, 36(1),
45–56. https://doi.org/10.1207/S15326985EP3601_5
Gill, R. (2002). Change management – or change leadership? Journal of Change Management,
3(4), 307-318. https://doi.org/10.1080/714023845
Glesne, C., & Peshkin, A. (2011). Chapter 6: But is it ethical? Considering what is “right.” In
Becoming qualitative researchers: An introduction (4th ed.) (pp. 162-183). White Plains,
N.Y: Longman.
Green, J., and Tones, K. (2010). Health Promotion: Planning and Strategies, 3rd ed. Sage,
London.
Grizzell, J., & McNeill, M. Linking health to academic success and retention. Student Health
Spectrum, 20-24.
Grumbach, K., Vargas, R., Fleisher, P., Aragon, T., Chung, L., Chawla, C. … Schmidt, L.
(2017). Achieving Health Equity Through Community Engagement in Translating Evidence
to Policy: The San Francisco Health Improvement Partnership, 2010-2016. Preventing
Chronic Disease: Health Research, Practice, and Policy, 14(27).
https://doi.org/10.5888/pcd14.160469
135
Gruttadaro, D., & Crudo, D. (2012). College students speak: A survey report on mental health.
National Alliance on Mental Illness. https://www.nami.org/collegesurvey
Hancock, T. (1985). Beyond health care: From public health policy to healthy public policy.
Canadian Journal of Public Health = Revue Canadienne de Sante Publique, 76 Suppl 1, 9–
11.
Hanleybrown, F., Kania, J., & Kramer, M. (2012). Channeling change: Making collective impact
work. Stanford Social Innovation Review. https://ssir.org/pdf/Channeling_Change_PDF.pdf.
Henig, J. R., Riehl, C. J., Rebell, M. A., & Wolff, J. R. (2015). Putting collective impact in
context: A review of the literature on local cross-sector collaboration to improve education.
New York, NY: Teachers College, Columbia University, Department of Education Policy
and Social Analysis.
International Conference on Health Promoting Universities and Colleges (2015). Okanagan
Charter: An International Charter for Health Promoting Universities and Colleges (2015).
https://doi.org/10.14288/1.0132754
Institute of Medicine. (2000). Promoting health: intervention strategies from social and
behavioral research. Washington, D.C. National Academies Press.
International Conference on Health Promoting Universities & Colleges (7
th
: 2015 : Kelowna, (B
C )). (2015). Okanagan charter: An international charter for health promoting universities &
colleges. http://hdl.handle.net/2429/54938
Jackson, M.L. & Weinstein, H. (1997). The importance of healthy communities of higher
education. Journal of American College Health, 45(6), 237-241.
https://doi.org/10.1080/07448481.1997.9936893
136
Johnson, R. B., & Christensen, L. B. (2015). Educational research: Quantitative, qualitative,
and mixed approaches. (5th ed.). Thousand Oaks: SAGE.
Kania, J., & Kramer, M. (2011). Collective Impact. Stanford Social Innovation Review, 9(1), 36-
41.
Kania, J. H., & Kramer, M. (2013). Embracing emergence: How collective impact addresses
complexity. Stanford Social Innovation Review, 1-8.
https://ssir.org/pdf/Embracing_Emergence_PDF.pdf.
Kansas University Work Group on Health Promotion and Community Development. (2000).
Section 8. Developing a Strategic Plan. https://ctb.ku.edu/en/table-of-
contents/assessment/assessing-community-needs-and-resources/identify-community-
assets/tools.
Keyes, C. (2002). The mental health continuum: From languishing to flourishing in life. Journal
of Health and Social Behavior, 43(2), 207-222. https://doi.org/10.2307/3090197
Kezar, A. (2001). Research-based principles of change. Understanding and facilitating
organizational change in the 21st century: Recent research and conceptualizations. ASHE-
ERIC Higher Education Report Volume 28(4), 113-123.
Kickbusch, I. (1997). Health–promoting environments: The next steps. Australian and New
Zealand Journal of Public Health, 21(4), 431–434. https://doi.org/10.1111/j.1467-
842X.1997.tb01729.x
Kickbusch, I. (2003). The contribution of the world health organization to a new public health
and health promotion. American Journal of Public Health, 93(3), 383-388.
https://doi.org/10.2105/AJPH.93.3.383
137
Kickbusch, I. (2007). Health and Modernity: The Role of Theory in Health Promotion. In
McQueen, Kickbusch, Potvin, McQueen, David V, Kickbusch, Ilona, & Potvin, Louise.
(2007). Health and modernity: The role of theory in health promotion (pp. 144–161). New
York: Springer.
Kirschner, P., Kirschner, F., & Paas, F. (2006). Cognitive load theory.
http://www.education.com/reference/article/cognitive-load-theory/.
Knight, A., & La Placa, V. (2013). Healthy Universities: taking the University of Greenwich
healthy universities initiative forward. International Journal of Health Promotion and
Education. 51, 41–49.
Krathwohl, D. R. (2002). A revision of Bloom’s Taxonomy: An overview. Theory Into Practice,
41(4), 212–218. https://doi.org/10.1207/s15430421tip4104_2
Lechasseur, K. (2016). Re-examining power and privilege in collective impact. Community
Development, 47(2), 1–16. https://doi.org/10.1080/15575330.2016.1140664
Lederer, A. M., & Oswalt, S. B. (2017). The value of college health promotion: A critical
population and setting for improving the public's health. American Journal of Health
Education, 48(4), 215. https://doi.org/10.1080/19325037.2017.1316692
Lent, R. (2004). Toward a Unifying Theoretical and Practical Perspective on Well-Being and
Psychosocial Adjustment. Journal of Counseling Psychology, 51(4), 482–509.
https://doi.org/10.1037/0022-0167.51.4.482
Martinez, J. A., Roth, M. G., Johnson, D. N., & Jones, J. A. (2015). How robustly does cannabis
use associate to college grades? Findings from two cohorts. Journal of Drug
Education, 45(1), 56-67. https://doi.org/10.1177/0047237915596606
138
Maxwell, J. A. (2013). Qualitative research design: An interactive approach. (3rd ed.).
Thousand Oaks: SAGE.
Mayer, R. E. (2011). Applying the science of learning. Boston, MA: Pearson Education.
McQueen, D.V., Wismar, M., Lin, V., & Jones, M. (2012). Introduction: health in all policies,
the social determinants of health and governance. In D.V. McQueen, M. Wismar, V. Lin, C.
Jones, & M. Davies (EDs). Intersectoral governance for health in all policies: structures,
actions and experiences: Structures actions and experiences. (pp. 3-22). Copenhagen:
WHO Regional Office for Europe.
http://ebookcentral.proquest.com/lib/[SITE_ID]/detail.action?docID=1582964
Melville-Ross, T. (2010). Leadership, governance and management. Perspectives: Policy and
Practice in Higher Education, 14(1), 3-6. https://doi.org/10.1080/13603100903450569
Merriam, S. B., & Tisdell, E. J. (2016). Qualitative research: A guide to design and
implementation. (4th ed.). San Francisco: Jossey-Bass.
Miles, M. B., Huberman, A. M., & Saldaña, J. (2014). Qualitative data analysis: A methods
sourcebook (3rd ed.). Thousand Oaks, CA: Sage Publishers.
Miller, G. A. (1956). The magical number seven, ± two: Some limits on our capacity for
processing information. Psychological Review, 63(2), 81–97.
Mittlemark, M. (2007). Shaping the future of health promotion: priorities for action. Health
Promotion International, 23(1), 98-102. https://doi.org/10.1093/heapro/dam033
Mittelmark, M. B. & Bull, T. (2013). The salutogenic model of health in health promotion
research. Global Health Promotion, 20(2), 30-38, 83, 91.
https://doi.org/10.1177/1757975913486684
139
National Association of County & City Health Officials (NACCHO). (2013). Local health
department job losses and program cuts: Findings from the 2013 profile study.
http://archived.naccho.org/topics/infrastructure/lhdbudget/upload/Survey-Findings-Brief-8-
13-13-3.pdf
National Center for Education Statistics. (2012). Digest of education statistics 2011. Washington,
DC: Department of Education.
Newton, J., Dooris, M. & Wills, J. (2016). Healthy universities: an example of a whole-system
health-promoting setting. Global Health Promotion, 23(1), 57–65.
https://doi.org/10.1177/1757975915601037
Northouse, P. (2016). Leadership: Theory and practice (7th ed.) Thousand Oaks, CA: Sage
Publishers.
Nutbeam, D. (2006). Health literacy as a public health goal: a challenge for contemporary health
education and communication strategies into the 21st century. Health Promotion
International, 15(3), 259–267. https://doi.org/10.1093/heapro/15.3.259
Pajares, F. (2006). Self-efficacy theory. The Gale Group.
Patton, M. Q. (2015). Qualitative research and evaluation methods (4th ed.). Thousand Oaks,
CA: SAGE Publications.
Pawson R. & Tilley N. (1997) Realistic Evaluation. London: Sage Publications
Pennington, D. (2008). Cross-disicplinary collaboration and learning. Ecology and Society,
13(2), 8. https://www.ecologyandsociety.org/vol13/iss2/art8/
Pintrich, P. R. (2003). A motivational science perspective on the role of student motivation in
learning and teaching contexts. Journal of Educational Psychology, 95(4), 667-686.
140
Piazza-Gardner, A. K., Barry, A. E., & Merianos, A. L. (2016). Assessing drinking and academic
performance among a nationally representative sample of college students. Journal of Drug
Issues, 46(4), 347-353. https://doi.org/10.1177/0022042616659757
Potvin, L. & Jones, C. M. (2011). Twenty-five years after the Ottawa Charter: the critical role of
health promotion for public health. Canadian Journal of Public Health, 102(4), 244-248.
Preskill, Parkhurst, & Splansky Juster (n.d.) Guide to Evaluating Collective Impact.
https://www.fsg.org/publications/guide-evaluating-collective-impact
Probst, B. & Berenson, L. (2014). The double arrow: How qualitative social work researchers
use reflexivity. Qualitative Social Work, 13(6), 813-827. DOI: 10.1177/1473325013506248
Rames, M. P. (2000). Effect of Financial Constraints on Student Affairs Services. [Information
Analyses Reports - Research]. NASPA Journal, 38(1), 70-81.
Reason, P. (1988). Human Inquiry in Action: Developments in New Paradigm Research.
Thousand Oaks, CA: Sage Publications
Rocco, T. S. & Plakhotnik, M. S. (2009). Literature reviews, conceptual frameworks, and
theoretical frameworks: Terms, functions, and distinctions. Human Resource Development
Review, 8(1), 120-130. https://doi.org/10.1177/1534484309332617
Roscoe, L. (2009). Wellness: A Review of Theory and Measurement for Counselors. Journal of
Counseling & Development, 87(2), 216–226. https://doi.org/10.1002/j.1556-
6678.2009.tb00570.x
Rueda, R. (2011). The 3 dimensions of improving student performance. New York: Teachers
College Press.
Ryan, R., & Deci, E. (2001). On Happiness and Human Potentials: A Review of Research on
Hedonic and Eudaimonic Well-Being. Annual Review of Psychology.
141
Sarmiento, J.P. (2017). Healthy universities: mapping health promotion interventions. Health
Education, 117(2), 162-175. https://doi.org/10.1108/HE-07-2016-0028
Schein, E.H. (2004) Organizational culture and leadership, 5th Edition. San Francisco, CA:
Jossey-Bass.
Schraw, G., & McCrudden, M. (2006). Information processing theory. http://
www.education.com/reference/article/information-processing-theory/.
Senge, P. M. (1990). The leader’s new work: Building learning organizations. Sloan
Management Review, 32(1).
Stake, R. E. (2005). Qualitative Case Studies. In N. K. Denzin & Y. S. Lincoln (Eds.), The Sage
handbook of qualitative research (pp. 443-466). Thousand Oaks, CA: Sage Publications
Ltd.
Suárez-Reyes, M. & Van Den Broucke, S. (2016). Implementing the Health Promoting
University approach in culturally different contexts: a systematic review. Global Health
Promotion, 23(1), 46–56. https://doi.org/10.1177/1757975915623933
Surman, T., & Surman, M. (2008). Listening to the stars: the constellation model of collaborative
social change. Social Space. (pp. 24-29). Toronto: Centre for Social Innovation.
Tsouros, A., Dowding, G., Thompson, J., & Dooris, M. (1998). Health promoting universities:
Concept, experience and framework for action. Copenhagen: WHO Regional Office for
Europe. http://www.euro.who.int/__data/assets/pdf_file/0012/101640/E60163.pdf
142
Tsouros, A.D. (1998). From the healthy city to the healthy university: project development and
networking. In Tsouros, A.D., Dowding, G., Thompson, J., and Dooris, M. (Eds). Health
Promoting Universities: Concept, Experience and Framework for Action. Copenhagen:
WHO Regional Office for Europe.
http://www.euro.who.int/__data/assets/pdf_file/0012/101640/E60163.pdf.
United States. (1978). The Belmont report: Ethical principles and guidelines for the protection of
human subjects of research. Bethesda, Md.: The Commission.
Weiss, R. S. (1995). Learning from strangers: The art and method of qualitative interview
studies. Simon and Schuster.
Wimbush, E. & Watson, J. (2000). An evaluation framework for health promotion: theory,
quality and effectiveness. Evaluation, 6(3), 301-321.
https://doi.org/10.1177/135638900000600302
World Health Organization (1986). Ottawa Charter for Health Promotion. Ottawa, ON.
World Health Organization. (2005). Edmonton Charter for Health Promoting Universities and
Institutions of Higher Education.
http://www.paho.org/hq/index.php?option=com_content&view=article&id=10755%3A201
5-edmonton-charter-health-institutions&catid=7793%3Aresources&lang=en
143
Appendices
Appendix A: Interview Protocol
Introduction
Mrs./Mr./Dr. X – Thank you so much for taking the time to talk with me today. As you know, I
am a doctoral student with the Rossier School of Education, and I greatly appreciate the time you
are spending with me to conduct this interview. Your perspective and insight will be a valuable
asset to the overall study. As we go through the interview, I’ll be jotting down notes to help
recall what we discussed, but I’d also like to record our conversation. The recording will only be
listened to by me, and it will not be linked with your name at all. The purpose of the recording is
to allow me to go back and ensure that I captured the information accurately; does that sound ok
to you?
So to begin, I’d like to start the conversation around the university and its priorities.
Interview Questions
1. What do you think are the university’s top priorities?
Probe: Where do you think student health and well-being fall in that list of priorities?
Transition: OK, so priorities are XYZ. (Go to question 2)
2. How are those priorities communicated to various university stakeholders?
Transition: I’d like to shift now to the Division of Student Affairs (Go to question 3)
3. Can you describe, in your opinion, the Division’s shared values? How did the DSA arrive at
those values?
Transition: OK, so they do XYZ to identify shared values. Now…(go to question 4)
4. I’d like you to think about the four public health goals of the Division. How well do those
align or not align with the goals and priorities of the university we discussed previously?
144
5. Referring to those goals of the Division, what internal and external organizational barriers
exist that prevent the Division from achieving its goal of enhancing student well-being?
6. How do the four goals of the Division fit with the goals of your department, if at all?
Probes: Can you provide specific examples of how your department goals align with the
DSA goals?
Transition: Continuing with that focus on your department’s goals and the DSA goals,
(go to question 6)
7. What are your thoughts on aligning your department/unit’s efforts with another
department/unit to accomplish the four goals of the strategic plan?
Probes: What would it take to accomplish that?
Transition: I’m going to shift gears here a bit (go to question 5)
8. How does the organization, in this case the DSA, view failure? (Pause, allow time to think)
Could you think of a time when your unit or department failed?
Probes: From your perspective, how does the university view failure?
Transition: Shifting now to Collective Impact specifically… (go to question 9)
9. Can you describe your understanding of Collective Impact?
Probes: Could you share more on your understanding of the five conditions? Could you
explain further what you mean by…?
Summarizing: Ok, so to summarize, your understanding of Collective Impact includes
XYZ.
Transition: Now, (go to question 10)…
10. How do you feel about the Collective Impact framework in general?
145
Probes: What are some of the benefits you’ve noticed? What successes have you seen
thus far? What factors contributed to those successes? How will you personally measure
success?
What are some challenges, as a steering committee and with the framework in general?
What do you think should be done differently moving forward?
Transition: Keeping the CI framework in mind… (go to question 11)
11. Can you provide an example of how your organization is contributing to each of the five
conditions of Collective Impact?
Probes: Here is a reference sheet you can use to refer to each of the five conditions.
Transition: So your organization is doing XYZ. From your perspective, (go to question
12)…
12. How would you describe the role of the various campus partners in achieving the goal of
enhancing student well-being?
Probes: Ok so campus partners are critical in doing XYZ; can you describe what
resources or assets these partners bring?
Transition: OK, we are going to pivot here towards some specific questions about the
work of the steering committee (Go to Question 13)
13. Can you provide an example of how you are using data to inform the strategic planning
process and accomplishing the public health goals?
Probe: Both “you” as an individual, and “you” as a steering committee…
14. How would you describe your level of familiarity with performing asset mapping and/or
density mapping?
Probe: Could you provide an example of when you used this technique?
146
Can you provide an example of when the steering committee used this technique?
Transition: And for our last question, (go to question 15)
15. Thinking about your role as a steering committee member, what additional resources do you
feel you need?
Probe: What internal and external organizational barriers exist that prevent the Division
from fully implementing the Collective Impact framework?
Conclusion: That concludes our interview for today. I’d like to thank you again for taking the
time to share your insights with me and to contribute to this study. Do you have any questions
for me before we close? (Allow time for questions) If any questions come up after we go our
separate ways, you can feel free to contact me via email.
147
Appendix B: Survey Protocol - Immediate
Student Well-being Convening
Strongly
Disagree
Strongly
Agree
After today’s session, I see how student health
and well-being is relevant to my unit’s work.
(L1: relevance)
❂ 1 ❂ 2 ❂ 3 ❂ 4
I am interested in learning more about student
health and well-being. (L1: engagement)
❂ 1 ❂ 2 ❂ 3 ❂ 4
The presentation style of the facilitator
contributed to my learning experience. (L1:
satisfaction)
❂ 1 ❂ 2 ❂ 3 ❂ 4
After today’s session, I could demonstrate the
use of mapping techniques to a peer or
colleague. (L2: Procedural)
❂ 1 ❂ 2 ❂ 3 ❂ 4
After today’s session, I am confident in using
mapping techniques to identify assets in the
community. (L2: confidence )
❂ 1 ❂ 2 ❂ 3 ❂ 4
After today’s session, I believe it is necessary
to work collaboratively to solve student health
and well-being issues. (L2: attitude)
❂ 1 ❂ 2 ❂ 3 ❂ 4
After today’s session, I believe it is important
to align my unit/ department’s work to the
common agenda of student well-being. (L2:
attitude)
❂ 1 ❂ 2 ❂ 3 ❂ 4
148
Open-ended questions:
What additional student health and well-being topics would you like to discuss in the
future?
What additional skills or training do you think are necessary for the SC?
What additional support do you need to meet the goals of the Collective Impact effort?
149
Appendix C: Survey Protocol - Delayed
Monthly Sessions
Strongly
Disagree
Strongly
Agree
The content provided was relevant to my work
as a SC member. (L1: relevance)
❂ 1 ❂ 2 ❂ 3 ❂ 4
The presentation style of the facilitator
contributed to my learning experience (L1:
engagement)
❂ 1 ❂ 2 ❂ 3 ❂ 4
After today’s session, I could demonstrate the
use of ____ technique to a peer or colleague.
(L2: Procedural)
❂ 1 ❂ 2 ❂ 3 ❂ 4
After today’s session, I am confident in using
_____ technique to do _____. (L2: confidence )
❂ 1 ❂ 2 ❂ 3 ❂ 4
After today’s session, I believe it is necessary
to work collaboratively to solve student health
and well-being issues. (L2: attitude)
❂ 1 ❂ 2 ❂ 3 ❂ 4
After today’s session, I believe it is important
to align my unit/ department’s work to the
common agenda of student well-being. (L2:
attitude)
❂ 1 ❂ 2 ❂ 3 ❂ 4
The handouts provided today contributed to my
understanding of the material presented. (L2:
Knowledge)
❂ 1 ❂ 2 ❂ 3 ❂ 4
I know what action to take to align my unit’s
activities towards student well-being. (L3)
❂ 1
❂ 2 ❂ 3 ❂ 4
150
I communicate regularly with my staff about
the Collective Impact efforts. (L3)
❂ 1 ❂ 2 ❂ 3 ❂ 4
I communicate regularly with external
stakeholders about the Collective Impact
efforts. (L3)
❂ 1 ❂ 2 ❂ 3 ❂ 4
I regularly use data to inform my decision-
making process. (L3)
❂ 1 ❂ 2 ❂ 3 ❂ 4
The SC has agreed to a common agenda. (L4)
❂ 1 ❂ 2 ❂ 3 ❂ 4
A shared measurement system has been created
(L4)
❂ 1 ❂ 2 ❂ 3 ❂ 4
An appropriate backbone support organization
has been selected/ formed. (L4)
❂ 1 ❂ 2 ❂ 3 ❂ 4
Open-ended questions:
What percentage of your unit’s activities is aligned to the common agenda of student
well-being? ___% Please provide examples of how your unit has aligned. (L4)
What percentage of your time do you communicate about the work of the Collective
Impact effort? ___% In what ways do you communicate about the Collective Impact
effort? (L4)
151
Appendix D: Data Analysis Chart
Data Analysis Chart
Appendix D. Collective Impact Sample Dashboard. Adapted from Living Cities, T. Boyea-
Robinson and Jeff Raderstrong, 2016, Retrieved from https://www.livingcities.org/resources/317-
collective-impact-element-dashboard. Creative Commons Attribution 4.0 License. Adapted with
permission.
Abstract (if available)
Abstract
In the United States, institutions of higher education are increasingly faced with complex public health issues among their student populations, yet have failed to devise solutions that account for the dynamic interplay of individual, community, and environmental factors. This dissertation examined one university’s implementation of a framework for collaboration—Collective Impact—that has shown promise in addressing complex social issues in other settings. The organization of study is a large, urban university in the Western United States that seeks to address four student-focused public health issues utilizing the Collective Impact framework. A qualitative case study research design was used. Data collection included in-depth interviews with key stakeholders who functioned together as the Steering Committee. Three research questions guided this study and addressed 1) the extent to which the Steering Committee had met its goal of establishing all five conditions of Collective Impact, 2) the knowledge and motivation influences related to achieving this goal, and 3) the interactive effects of organizational culture and context. Three main findings emerged and suggest that 1) progress was made, though further opportunities existed to establish the five conditions of Collective Impact, 2) Collective Impact creates an infrastructure for collaboration and has motivational effects when aligned to unit goals, and 3) to leverage its effectiveness in the higher education setting, the common agenda needs to be a university-wide initiative. Furthermore, organization context could impede the Steering Committee from accomplishing its goal. Subsequent recommendations are based on the New World Kirkpatrick model for program implementation and evaluation.
Linked assets
University of Southern California Dissertations and Theses
Conceptually similar
PDF
Preparing student affairs administrators to support college students of color with mental health needs
PDF
Women of color senior leaders: pathways to increasing representation in higher education
PDF
Collaborative instructional practice for student achievement: an evaluation study
PDF
Developing physician trainees leadership skills: an innovation study
PDF
All hands on deck! Collective impact for systemic change
PDF
Measuring the impact of short-term campus exchange programs: an evaluation study
PDF
Attendance interventions to address chronic absenteeism
PDF
School-based interventions for chronically absent students in poverty
PDF
Readiness factors influencing the ability of institutions of higher education to align the budget with organizational goals: an evaluation study of a college
PDF
Building capacity to increase community member involvement at a clinical and translational science award (CTSA)
PDF
Developing partnerships between education providers and Fortune 500 companies to increase the utilization of tuition assistance and quality digital education: an innovation study
PDF
Raising tenure: a case study of fundraiser retention in higher education
PDF
Closing the completion gap for African American students at California community colleges: a research study
PDF
When they teach us: recruiting teacher candidates of color for the next generation of students, an evaluation study
PDF
Opening interpretations: visual literacy and teaching for inclusion, first-year studio faculty innovate for improvement
PDF
The impact of high turnover and burnout among behavioral health's clinical workforce (clinicians)
PDF
Factors that impact the retention of low-income community college students
PDF
Organizational design for embedding corporate social responsibility
PDF
Closing the compliance gap: an evaluation of influences impacting appropriate compliance risk response among pharmaceutical company managers
PDF
Promoting well-being amid a global pandemic: evaluating the impact on nonprofit employees
Asset Metadata
Creator
Vanni, Amanda Katherine
(author)
Core Title
Collective Impact: a framework to advance health promotion in higher education
School
Rossier School of Education
Degree
Doctor of Education
Degree Program
Organizational Change and Leadership (On Line)
Publication Date
05/03/2020
Defense Date
02/13/2020
Publisher
University of Southern California
(original),
University of Southern California. Libraries
(digital)
Tag
collaboration,collective impact,college,Health promotion,Higher education,OAI-PMH Harvest,Public Health,student wellbeing,student well-being,University
Language
English
Contributor
Electronically uploaded by the author
(provenance)
Advisor
Stowe, Kathy (
committee chair
), Kumar, Shubha (
committee member
), Malloy, Courtney (
committee member
)
Creator Email
amanda.vanni@gmail.com,vanni@usc.edu
Permanent Link (DOI)
https://doi.org/10.25549/usctheses-c89-296892
Unique identifier
UC11664038
Identifier
etd-VanniAmand-8412.pdf (filename),usctheses-c89-296892 (legacy record id)
Legacy Identifier
etd-VanniAmand-8412.pdf
Dmrecord
296892
Document Type
Dissertation
Rights
Vanni, Amanda Katherine
Type
texts
Source
University of Southern California
(contributing entity),
University of Southern California Dissertations and Theses
(collection)
Access Conditions
The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the a...
Repository Name
University of Southern California Digital Library
Repository Location
USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA
Tags
collaboration
collective impact
student wellbeing
student well-being