Close
About
FAQ
Home
Collections
Login
USC Login
Register
0
Selected
Invert selection
Deselect all
Deselect all
Click here to refresh results
Click here to refresh results
USC
/
Digital Library
/
University of Southern California Dissertations and Theses
/
The implementation of a multi-tiered system of support in Downtown Unified School District: an analysis of site administrator needs
(USC Thesis Other)
The implementation of a multi-tiered system of support in Downtown Unified School District: an analysis of site administrator needs
PDF
Download
Share
Open document
Flip pages
Contact Us
Contact Us
Copy asset link
Request this asset
Transcript (if available)
Content
Running head: IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 1
THE IMPLEMENTATION OF A MULTI-TIERED SYSTEM OF SUPPORT IN DOWNTOWN
UNIFIED SCHOOL DISTRICT – AN ANALYSIS OF SITE ADMINISTRATOR NEEDS
by
Matthew R. Chambers
A Dissertation Presented to the
FACULTY OF THE USC ROSSIER SCHOOL OF EDUCATION
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulfillment of the
Requirements for the Degree
DOCTOR OF EDUCATION
May 2019
Copyright 2019 Matthew R. Chambers
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 2
Acknowledgements
To my beautiful wife Jacqui, thank you for your unwavering love and support these past
three years. I am grateful for the countless sacrifices you made to help me complete this degree.
Thank you for continuing to challenge and inspire me to a better person.
To my son Elliott and Baby #2, so many of my early parenting memories will be
associated with this program. Elliott, I found out that you were going to be born on a Thursday
night after class during my first semester at USC. Baby #2, I was writing Chapter Five of this
dissertation when I learned we were going to have you. I hope that one day this degree inspires
you to push yourselves beyond what you think you are capable of, and to give back to your
community and look out for those less fortunate than you.
To my family and dearest friends, thank you for all your encouragement throughout this
process. I am blessed to be surrounded by so many positive people.
To my committee members, Dr. Kenneth Yates, Dr. Briana Hinga, and Dr. John Paramo,
my heartfelt gratitude for your guidance and feedback through this process. Dr. Yates, thank you
for your energetic commitment and accessibility. No matter what I do in my career, because of
you, I will always remember to ask the most important question, “what is the goal?”. Dr. Hinga,
thank you for pushing me to think big and to question the educational and societal systems at
large. Dr. Paramo, thank you for mentoring me and giving me the confidence to believe I can do
more. I wouldn’t be in this program without you.
To my writing partner Dr. Bhavini Bhakta, I couldn’t have asked for a better person to
collaborate with. I learned so much from you in this process. Let’s do it again.
Lastly, to my Trojan family, the friends I made in this program were worth the price of
admission.
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 3
Table of Contents
Acknowledgements………………………………………………………………………………..2
List of Tables……………………………………………………………………………………...6
Abstract……………………………………………………………………………………………7
CHAPTER ONE: INTRODUCTION……………………………………………………………..8
Introduction of the Problem of Practice…………………………………………………...8
Organizational Context and Mission……………………………………………………...9
Organizational Performance Status………………………………………………………10
Related Literature………………………………………………………………………...10
Importance of the Problem to the Organization………………………………………….11
Organizational Performance Goal………………………………………………………..12
Description of the Stakeholders………………………………………………………….12
Stakeholders’ Performance Goals………………………………………………………..12
Stakeholder Group for the Study………………………………………………………...13
Stakeholders of Focus Critical Behaviors………………………………………………..13
Purpose of the Project and Questions……………………………………………………14
Conceptual and Methodological Framework…………………………………………….15
Definitions………………………………………………………………………………..15
Organization of the Study………………………………………………………………..16
CHAPTER TWO: REVIEW OF THE LITERATURE………………………………………….17
Introduction……………………………………………………………………………....17
Graduating from High School College Ready…………………………………………...17
Challenges Preventing Students from Graduating College Ready………………………24
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 4
Intervention Systems to Support Struggling Students…………………………………...28
Stakeholder Role in MTSS………………………………………………………………34
Conceptual Framework…………………………………………………………………..35
Stakeholder Knowledge, Motivation and Organizational Factors……………………….36
Summary…………………………………………………………………………………53
CHAPTER THREE: METHODOLOGY………………………………………………………..55
Purpose of the Project and Questions……………………………………………………55
Conceptual and Methodological Framework…………………………………………….55
Assessment of Performance Influences………………………………………………….57
Participating Stakeholders and Sample Selection………………………………………..70
Instrumentation…………………………………………………………………………..70
Data Collection…………………………………………………………………………..71
Data Analysis…………………………………………………………………………….72
Trustworthiness of Data………………………………………………………………….73
Role of Investigator………………………………………………………………………74
Limitations……………………………………………………………………………….74
CHAPTER FOUR: RESULTS AND FINDINGS……………………………………………….76
Participating Stakeholders……………………………………………………………….76
Data Validation…………………………………………………………………………..77
Results and Findings for Knowledge Causes…………………………………………….78
Results and Findings for Motivation Causes…………………………………………….90
Results and Findings for Organization Causes……………………………………….….95
Summary of Validated Influences……………………………………………………...102
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 5
CHAPTER FIVE: RECOMMENDATIONS AND EVALUATION…………………………..107
Purpose of the Project and Questions…………………………………………………..107
Recommendations to Address Knowledge, Motivation, and Organization Influences...107
Summary of Knowledge, Motivation, and Organization Recommendations…………..123
Integrated Implementation and Evaluation Plan………………………………………..124
Limitations and Delimitations…………………………………………………………..138
Recommendations for Future Research………………………………………………...139
Conclusion……………………………………………………………………………...139
References…………………………………………………………………………………..…..141
Appendix A: Interview Protocol………………………………………………………………..151
Appendix B: Immediate Evaluation Tool………………………..……………………………..155
Appendix C: Delayed Evaluation Tool…………………………..……………………………..157
Appendix D: Digital Dashboard…………………………………………………………….….160
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 6
List of Tables
Table 1: Organizational Goal and Mission………………………………………………..13
Table 2: Summary of Assumed Knowledge Influences on Stakeholder’s Ability to Achieve
the Performance Goal……………………………………………………………42
Table 3: Summary of Assumed Motivation Influences on Stakeholder’s Ability to Achieve
the Performance Goal……………………………………………………………47
Table 4: Summary of Assumed Organization Influences on Stakeholder’s Ability to
Achieve the Performance Goal………………………………………………….52
Table 5: Summary of Knowledge Influences and Method of Assessment……………….60
Table 6: Summary of Motivation Influences and Method of Assessment………………..65
Table 7: Summary of Organizational Influences and Method of Assessment……………68
Table 8: Interview Results for Factual Knowledge of Teacher Roles in the RTI
2
and PBIS
process…………………………………………………………………………...80
Table 9: Summary of Assumed Knowledge Influences as Assets or Needs…………….102
Table 10: Summary of Assumed Motivation Influences as Assets or Needs……………..104
Table 11: Summary of Assumed Organization Influences as Assets or Needs…………...105
Table 12: Summary of Knowledge Influences and Recommendations…………………...108
Table 13: Summary of Motivation Influences and Recommendations……………………112
Table 14: Summary of Organization Influences and Recommendations………………….116
Table 15: Outcomes, Metrics, and Methods for External and Internal Outcomes………...126
Table 16: Critical Behaviors, Metrics, Methods, and Timing for Evaluation……………..128
Table 17: Required Drivers to Support Critical Behaviors………………………………..129
Table 18: Evaluation of the Components of Learning for the Program…………………...134
Table 19: Components to Measure Reactions to the Program…………………………….135
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 7
Abstract
This study utilizes Clark and Estes’ (2008) performance analysis model, which
systematically and analytically clarifies organizational goals to identify the current and preferred
performance level within an organization. The purpose of this study was to identify the
knowledge, motivation and organizational (KMO) needs of site administrators to effectively
implement a Multi-Tiered System of Support (MTSS) at Westside High School in Downtown
Unified School District. All six site administrators from Westside High School were interviewed
and documents were analyzed to determine which of the twenty assumed KMO influences
identified in this study were actual needs. Findings from this study showed that site
administrators at WHS are somewhat motivated and capable of effectively implementing MTSS
at their sites. However, ten of the twenty influences that were assessed were determined to be
actual needs including four knowledge influences, two motivation influences, and all six
organizational influences. This study makes recommendations for improving organizational
performance by utilizing The New World Kirkpatrick Model to plan, implement, and evaluate
the effectiveness of an MTSS training program designed to provide site administrators with the
knowledge and skills, motivation, and organizational tools necessary to achieve their
performance goals.
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 8
CHAPTER ONE: INTRODUCTION
(Co-authored by Bhavini Bhakta and Matthew Chambers)
Introduction of the Problem of Practice
The California Department of Education (2017)
defines a Multi-Tiered System of Support
(MTSS) as an integrated, comprehensive framework that focuses on Common Core State
Standards (CCSS), core instruction, differentiated learning, student-centered learning,
individualized student needs, and the alignment of systems necessary for all students’ academic,
behavioral, and social success. The components of MTSS include both Response to Instruction
and Intervention (RtI²), and Positive Behavioral Interventions and Supports (PBIS).
Combining academic and social-emotional behavioral supports in an integrated model is
more effective and efficient at raising student outcomes than implementing academic and
behavior supports independently of one another (Cook, et al., 2012; McIntosh, et al., 2006;
McIntosh and Goodman, 2016). Studying the implementation of MTSS within the high school
setting aides in understanding precisely how student social and academic outcomes can be
heightened. This particular project represents the larger national problem that at-risk students
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 9
who do not receive effective comprehensive interventions struggle both socially and
academically (McIntosh and Goodman, 2016).
Organizational Context and Mission
The Downtown Unified School District (DUSD, a pseudonym) is a comprehensive local
education agency (LEA) offering educational programs from Kindergarten through Adult
School. DUSD enrolls more than 32,000 students in one of their twenty-one public school
programs.
The mission of the Downtown Unified School District is to develop all students to be
self-motivated learners and productive, responsible and compassionate members of an ever-
changing global society. As part of its vision, DUSD seeks to provide every student with an
opportunity to graduate with a 21st Century education that ensures they are college and career
ready, globally competitive and citizens of strong character.
In the fall of 2017, the Downtown Unified Board of Education voted to align district
graduation requirements with California college admission requirements, also known as the A-G
requirements. This change would ensure that DUSD are eligible to apply to California State
University (CSU) or University of California (UC) schools upon graduation. The higher
requirements will be implemented in three stages beginning with the graduating class of 2022.
The first graduating class required to meet the A-G requirements will be the class of 2026.
The Downtown Board of Education is setting the bar very high and should be
commended for it. However, it will not be easy to meet their new goal since the district is
currently graduating around 42% of students CSU/UC eligible. Therefore, this policy change
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 10
will require a significant increase in academic and social-emotional supports if the district wants
to meet their new goal of graduating more student’s college ready.
Organizational Performance Status
DUSD desires to examine the implementation of MTSS within its high schools.
Currently, DUSD has many effective intervention programs in place to support the
academic and behavioral needs of their students. These programs include tier one interventions
to support students struggling with academics and tier one PBIS behavioral supports. However,
DUSD has yet to implement both PBIS and RTI as an integrated whole under the name MTSS.
Moreover, not all 9th grade students are on track to meet A-G graduation requirements.
Implementing RTI and PBIS as functions of an effective MTSS program are crucial to
early identification of learning and behavioral challenges and timely intervention for students at
risk for poor learning outcomes. A fully and effectively implemented MTSS program creates
opportunities for students to access college and career readiness instruction through tiers of
services and supports that vary in intensity.
Implementing MTSS cohesively throughout the District impacts DUSD’s goal of
developing all students to be self-motivated learners and productive, responsible and
compassionate members of an ever-changing global society because not all students who are
receiving intervention services are meeting A-G requirements.
Related Literature
A multi-tiered system of support (MTSS) is a methodical and careful integration of the
academic RTI and PBIS systems to support students academically and behaviorally in school
(California Department of Education, 2017; Freeman et al., 2017; McIntosh & Goodman, 2016).
Combining RTI and PBIS into an integrated model known as MTSS makes sense because of the
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 11
strong relationship between academic skills and problem behavior, similarities in structure and
process, and integrating both models together leads to an efficient use of resources. (McIntosh &
Goodman, 2016). Three core components of the MTSS framework that guide educators are data-
driven decision making, evidence-based practices to support student needs, and support systems
in place to ensure continuous improvement (Freeman et al., 2017).
Implementing this type of a systematic intervention system requires the commitment and
expertise of all school professionals, not just a handful of special education teachers and school
psychologists. Successful implementation of MTSS centers on the coordination and
collaboration of district and school staff to ensure the most effective instructional approaches are
used to meet the needs of students. (Duffy, 2007). However, the general background literature
about this problem suggests that many teachers do not feel adequately prepared to work within a
multi-tiered system of support due to large and varied teaching loads, a lack of resources for
professional development, and insufficient training in teacher preparation programs (Lancaster &
Hougen, 2017; Prasse et al., 2012).
Importance of the Problem to the Organization
It is important for DUSD to effectively implement MTSS because this will enhance
instructional quality and support for students at risk of poor educational outcomes.
Implementing MTSS cohesively at the high school level impacts DUSD’s goal of developing all
students to be self-motivated learners and productive, responsible and compassionate members
of an ever-changing global society. Furthermore, it is imperative to effectively implement MTSS
because the recent policy change to increase graduation requirements will require a
comprehensive systematic approach to provide at-risk students with academic and social-
emotional supports and interventions.
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 12
Organizational Performance Goal
The goal of the organization is for DUSD’s Westside High School (WHS, a pseudonym)
to effectively implement MTSS to ensure that all ninth-grade students are on track to meet A-G
requirements.
Description of the Stakeholders
The three main stakeholders who are responsible for implementing MTSS and were
considered for this study are teachers, site administrators, and district office leadership. Each of
these stakeholders plays a key role in the process of raising student achievement through a
systematic approach to interventions and support services. Teachers have an integral role in the
effective implementation of any intervention program, particularly MTSS. This role requires
teachers to be trained in the program, work collectively with other education professionals, and
alter their day-to-day instructional practices. Site administrators play a key role in leading the
implementation of MTSS at their sites by communicating goals, providing resources and
training, and fostering a supportive organization environment. District office leadership must
stay current on research, make sure implementation is aligned with district goals, and provide
resources and training for school-based teams to ensure fidelity of implementation.
Stakeholders’ Performance Goals
The goals for the organization and the three primary stakeholders to implement MTSS
are shown in Table 1.
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 13
Table 1
Organizational Goal and Mission
Organizational Mission
Downtown Unified School District (DUSD) will develop all students to be self-motivated
learners and productive, responsible and compassionate members of an ever-changing global
society. As part of its vision, DUSD will provide every student with an opportunity to graduate
with a 21st Century education that ensures they are college and career ready, globally
competitive and citizens of strong character
Organizational Global Goal
By June 2020, DUSD’s Westside High School will effectively implement MTSS to ensure that
all 9th grade students remain on track to meet A-G graduation requirements.
Stakeholder Goal:
Teachers
By June 2020, all teachers
will effectively implement
established MTSS
intervention strategies with
identified students so that all
students are achieving A-G
graduation requirements.
Stakeholder Goal:
Site Administrators
By June 2020, the
administrative team will
implement and monitor a
systematic MTSS
intervention program for
identified students so that all
students are achieving A-G
graduation requirements.
Stakeholder Goal:
District Office Leadership
By June 2020, District Office
leaders will put into place the
standards and processes that
will reinforce, monitor,
encourage, and reward
effective implementation of
MTSS at Westside High
School so that all students are
achieving A-G graduation
requirements.
Stakeholder Group for the Study
The stakeholders of focus for this study are site administrators in DUSD at Westside
High School (WHS). (See Bhakta (2019) for an examination of teachers as the key stakeholders
in this study.)
Stakeholders of Focus Critical Behaviors
Critical behaviors are the few, specific actions, which, if performed consistently on the
job, will have the biggest impact on desired results and achieving organizational success
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 14
(Kirkpatrick and Kirkpatrick, 2016). Critical behaviors are used to guide the literature review
and the application of the Clark & Estes (2008) Knowledge, Motivation, and Organization
framework.
Three preliminary critical behaviors have been identified for site administrators to
achieve their goal. The review of the literature may reveal other critical behaviors.
1. Define observable and measurable organizational goals and targeted results for student
success.
2. Create processes that reinforce, monitor, encourage, and reward teacher critical
behaviors.
3. Use processes to partner with district office leaders to ensure proper support and/or
recognition of teachers and sites in their implementation of MTSS.
Purpose of the Project and Questions
The purpose of this study is to examine the implementation of MTSS in Downtown
Unified School District (DUSD) high schools from the perspective of site administrators. More
specifically, the purpose of this project is to conduct a needs analysis in the areas of knowledge
and skills, motivation, and organization, which are necessary to reach the organizational
performance goal of effectively implementing MTSS at Westside High School by June 2020.
The analysis will begin by generating a list of possible needs and will then move to examining
them systematically to focus on actual or validated needs. While a complete needs analysis
would focus on all stakeholders, for practical purposes the stakeholder to be focused on in this
analysis is DUSD site administrators.
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 15
Two questions guide this study.
1. What are the knowledge and skills, motivation, and organizational needs for site
administrators to effectively implement MTSS at Westside High School?
2. What are the knowledge and skills, motivation, and organizational recommendations for
meeting the needs of site administrators in achieving the DUSD goal of effectively
implementing MTSS at Westside High School?
Conceptual and Methodological Framework
This is a collaborative action research project in which we will utilize Clark and Estes’
(2008) performance analysis model, which is a systematic, analytical method that helps to clarify
organizational goals and identify the current and preferred performance level within an
organization.
Assumed knowledge, motivation and organizational needs will be generated based on
district information, personal knowledge, and related literature. These needs will be validated
and triangulated by data, including knowledge assessments, motivation and organization scales,
and document analysis.
Data collection methods include individual interviews and document analysis.
Research-based solutions will be recommended and accompanied by an implementation
and evaluation plan.
Definitions
A-G Requirements: The A-G requirements are that California high school students must
complete the A-G course sequence with a grade of “C” or higher to qualify for direct admission
to one of the 23 California State Universities (CSUs) or 10 University of California (UC)
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 16
campuses across the state (Betts, et al., 2013). The A-G sequence is comprised of 15 courses in
the subject areas of (a) History / social science, (b) English, (c) Mathematics, (d) Laboratory
Science, (e) Language other than English, (f) Visual and performing arts, and (g) College-
preparatory elective (University of California, n.d.).
Multi-Tiered System of Support: The California Department of Education
defines a Multi-Tiered
System of Support (MTSS) as an integrated, comprehensive framework that focuses on CCSS,
core instruction, differentiated learning, student-centered learning, individualized student needs,
and the alignment of systems necessary for all students’ academic, behavioral, and social
success.
Response to Intervention and Instruction (RtI
2
): The California Department of Education (CDE)
states that RtI
2
is a systematic, data-driven approach to instruction that benefits every student.
RtI
2
integrates resources from general education, categorical programs, and special education
through a comprehensive system of core instruction and tiered levels of interventions to benefit
every student. The CDE work group expanded the notion of RtI to RtI
2
, Instruction and
Intervention, to emphasize the full spectrum of instruction, from general to intensive.
Organization of the Study
This study is organized into five chapters. Chapter One is an introduction to the study.
Chapter Two will be a review of the literature. Chapter Three will cover the research
methodology. Chapter Four will contain the research findings of the study. Lastly, Chapter Five
will include the conclusions, discussions, and suggestions for future research.
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 17
CHAPTER TWO: REVIEW OF THE LITERATURE
(Co-authored by Bhavini Bhakta and Matthew Chambers)
Introduction
The purpose of this literature review will be to explore what it means for students to
graduate college ready, to examine some of the academic, behavioral, and social-emotional
challenges preventing students from reaching that goal, and to identify intervention systems
supporting struggling students. This review of the literature will also analyze the knowledge and
skills, motivation, and organizational factors influencing site administrators through the lens of
the critical behaviors identified in chapter one.
Graduating from High School College Ready
College Readiness
College readiness is most commonly described as the level of preparation a student needs
to enroll and succeed in a college program without requiring remediation (Conley, 2007; Venezia
& Jaeger, 2013). However, college readiness can also encompass a wide range of domains and
contexts including curricular content, academic behaviors, cognitive strategies, high school GPA,
high school class rank, or standardized college entrance exam scores such as the SAT or ACT
(Tierney & Sablan, 2014). There are many ways that governments and educational institutions
prioritize what makes a student college and/or career ready. For example, an analysis of thirty-
seven state definitions of college and career readiness found the most common elements to be
academic knowledge, critical thinking, social and emotional learning, grit, resilience or
perseverance, and citizenship and/or community involvement (Mishkind, 2014). Furthermore,
according to Conley (2007), a college-ready student is able to understand the expectations of
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 18
college courses, process the content knowledge that is presented, and synthesize the key
intellectual lessons and dispositions the course was designed to convey and develop.
A criticism of our education system is that high school hardly prepares students with the
skills required of college level courses, which are faster paced and more rigorous (Conley, 2007).
College level courses, unlike many high school courses, require students to make inferences,
interpret results, analyze conflicting explanations of phenomena, support arguments with
evidence, solve complex problems that have no obvious answer, reach conclusions, offer
explanations and conduct research (Conley, 2007). This is troubling when you consider that
forty percent of new college students need to take remedial courses because they are not
adequately prepared for university level work (United States Department of Education, 2010).
On the same note, college professors estimate that four out of ten students are not adequately
prepared for college (Wyatt, Wiley, Camara & Proestler, 2011). Furthermore, only 28% believe
that public high schools adequately prepare students for the challenges of college (Wyatt et al.,
2011). Data from the National Assessment of Educational Progress (NAEP) suggests that many
students are likely falling short of being college ready upon graduation (Venezia & Jaeger,
2013). In 2009, 38% of twelfth-grade students performed at or above the proficient level on the
NAEP reading assessment, while only 26% percent of test takers were at or above the proficient
level in mathematics (Venezia & Jaeger, 2013).
The Common Core State Standards (CCSS) have been adopted by forty-two states with
the main goal of graduating students’ college and career ready (Common Core State Standards
Initiative, n.d.). Many states have college readiness indicators embedded in their school
accountability systems. Indicators vary from state to state and include such factors as the number
of students taking AP courses, the number of students enrolled in dual enrollment courses, the
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 19
percentage of graduates who go to college, college remediation rates, SAT, ACT, and AP scores,
and one-year college retention rates (United States Department of Education, n.d.).
In 2017, the California Department of Education (CDE) implemented a new
accountability system called the California School Dashboard (California Department of
Education, 2018). The California School Dashboard includes a College/Career Indicator (CCI),
which measures postsecondary preparedness (California Department of Education, 2018). To be
considered CCI prepared, students must earn a high school diploma and successfully achieve one
of the following: (a) score at least a Level 3 "Standard Met" on both ELA and Mathematics on
Smarter Balanced Summative Assessments, (b) complete two semesters of Dual Enrollment with
a passing grade, (c) earn a passing score on two Advanced Placement (AP) Exams or two
International Baccalaureate (IB) Exams or (d) complete the A-G requirements (California
Department of Education, 2018).
Many states including California measure college readiness as a key indicator of student
achievement and school success. College ready students are developed at the secondary level, so
how are high school graduation requirements helping student prepare for university level work?
High School Graduation Requirements
High school graduation requirements have been historically designed to help prepare
students for the rigors of college (Plunk, Tate, Bierut & Grucza, 2014; Shaw & Walker, 1981).
For instance, the last few decades of the twentieth century brought forth increased high school
graduation requirements in response to calls for more cognitively demanding high school
coursework to better prepare students for the demands of college and to improve the quality of
the labor force (Murnane, 2013; Plunk et al., 2014). The origin of high school graduation
requirements as we know them today can be traced back to the late nineteenth century.
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 20
College admission requirements were in place in the United States before high school
graduation requirements. Harvard College established admission standards as early as 1642
(Shaw & Walker, 1981). By around 1870, most colleges entrance requirements included algebra,
geometry, ancient history, English grammar, and ancient and modern geography for admission
(Shaw & Walker, 1981). However, high school courses in music, art, agriculture, commercial
education, industrial education, and home science were not recognized by any of the leading
colleges at the time (Shaw & Walker, 1981). The late nineteenth century marked a period where
high schools and colleges lacked a reasonable articulation due to major differences in college
entrance requirements, time students spent in secondary schools, and the function and scope of
high schools (Shaw & Walker, 1981). However, by 1909 many high schools and colleges in the
U.S. had officially adopted educational standards recommended by the Carnegie Foundation
(Shaw & Walker, 1981). The Carnegie Foundation, which was composed of a group of college
presidents and prominent bankers of the time, defined the idea of four years of high school and
what it meant to separate high school-type courses from college curricula (Shaw & Walker,
1981). The Carnegie Foundation recommended the adoption of a Standard Unit, defined as a
year’s study in some major subject in a secondary school. The Standard Unit, or Carnegie Unit
as it is often referred to today, assumes that schools are in session at least 36 weeks a year, a
subject is studied for four or five periods a week, and a period is 40 to 60 minutes in length
(Shaw & Walker, 1981).The Carnegie Foundation also set the standard of 14 units of high school
work as the basic minimum for admission to colleges which aligned with the practices of the
nation’s top schools at the time (Shaw & Walker, 1981).
Moving ahead to the 21st century, approximately 3 million students graduate from high
school in the United States per year (Venezia & Jaeger, 2013). As the largest state in the union,
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 21
California produces more than ten percent of the country’s total graduates with over 400,000
students earning diplomas from the Golden State each year. (California Department of Education
DataQuest, 2018). The state of California requires students, while in grades nine to twelve, to
complete at minimum of thirteen courses in the following subject areas to obtain a high school
diploma: three courses in English, two courses in mathematics, two courses in science, three
courses in social studies, two courses in physical education, and one course in either visual or
performing arts, foreign language, or career technical education (California Department of
Education, 2017).
In comparison to other states in the nation, California’s graduation requirements lag
(Gao, Lopes, & Lee, 2017). Over the past decade, 18 states have added at least one year of
instruction to their math requirement resulting in California now being one of only three states
requiring only two years of math (Gao et al., 2017). In English, California is one of only two
states that require three years, instead of four years of instruction (Gao et al., 2017). Lastly,
California also lags in science requiring only two years of instruction, while forty-two states in
the union require at least three years of science for high school graduation (Gao et al., 2017).
Many school districts in the state of California supplement the statewide graduation requirements
and raise the bar in their local communities (Gao et al., 2017). A recent Public Policy Institute of
California (PPIC) survey found that during the 2015–16 school year, 63% of unified and high
school districts required a third year of math for high school graduation (Gao et al., 2017).
Additionally, the PPIC survey showed that about 40% of districts require a third year of science
(Gao et al., 2017). Many districts have incorporated the A-G course sequence in their graduation
requirements (Gao et al., 2017). A 2017 survey of districts shows that 51% aligned their
graduation requirements with the A-G requirements for CSU/UC eligibility (Gao et al., 2017).
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 22
About seven in ten districts with graduation requirements aligned with the A-G requirements
require a C or better in each course, while the rest require at least a D mark. Furthermore, these
districts have put in place exemptions for students with learning disabilities and/or those on
alternative graduation pathways (Gao et al., 2017).
In recent years, California has been complacent by not raising their minimum high school
graduation requirements, which are far below other states in the union. Although the state of
California has a low threshold for graduation requirements, the A-G requirements for CSU/UC
eligibility are a much higher standard that many districts, schools, and students are reaching for.
A-G Requirements for CSU/UC Eligibility
The past decade has seen an increased number of California school districts emphasize
A-G completion rates as the primary goal for high school students to attain upon graduation
(Betts, Zau & Bachofer, 2013; Gao et al., 2017). This is significant because California high
school students must complete the A-G course sequence to qualify for direct admission to one of
the 23 California State Universities (CSUs) or 10 University of California (UC) campuses across
the state (Betts, et al., 2013). The A-G sequence is comprised of 15 courses in the subject areas
of (a) History / social science, (b) English, (c) Mathematics, (d) Laboratory Science, (e)
Language other than English, (f) Visual and performing arts, and (g) College-preparatory
elective (University of California, n.d.). Schools must get courses A-G approved by submitting
coursework to the UCOP office for approval. Once courses are approved, students must
complete 30 semesters of A-G coursework with grades of C or higher to be eligible for CSU/UC
admission (Betts et al., 2013). There has been a significant increase in the number of high school
graduates who have met the A-G requirements in the past decade with 43% of graduates meeting
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 23
the requirement in 2015 compared to 35% ten years earlier in 2005 (Gordon, 2017). Students
who do not qualify for CSU/UC eligibility directly out of high school and attend a community
college are statistically less likely to earn a bachelor’s degree than those who go directly to a
state university (Finkelstein & Fong, 2008).
In recent years, school districts across the state have focused on increasing their A-G
completion rate (Betts et al., 2013). This focus may be attributed to the competitive nature of
school accountability systems and the outside influence of civil rights groups such as the
American Civil Liberties Union (ACLU) (Betts et al., 2013). For example, the largest school
districts in the states including Los Angeles, San Diego, San Francisco, and Oakland have all
aligned their high school graduation requirements to include the A-G course sequence (Betts et
al., 2013). It is still unclear how these transformational changes to diploma requirements will
address three potential side effects of the new policies including a potential decrease in
graduation rates, a watering down of A-G course content, and possible academic grade inflation
(Betts et al., 2013).
The Public Policy Institute of California studied how this graduation requirement change
would have affected San Diego Unified School District (SDUSD) students who completed their
secondary education in 2011 (Betts, et al., 2013). Of this group of SDUSD graduates, a
whopping 39% of the cohort would not have obtained their diplomas had the A-G course
requirements been in place the year they graduated (Betts et al., 2013). As school districts shift
their graduation requirements to increase their CSU/UC eligibility rates, it will be interesting to
see how they serve their students who cannot meet the rigorous academic requirements of the A-
G coursework.
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 24
In 2017, the California Department of Education (CDE) implemented a new
accountability system called the California School Dashboard (California Department of
Education, 2018). The new accountability system is a multiple measures approach that factors in
graduation, suspension, English Learner, and college and career data, while replacing the
Academic Performance Index, which accounted for only test scores in its formula (California
Department of Education, 2018). The California School Dashboard includes a College/Career
Indicator (CCI), which measures college and career readiness factors such as Advanced
Placement exam data, Early Assessment Program results, and A-G completion rates (California
Department of Education, 2018). As we approach the next decade, California high schools must
adapt to the new measurements set forth by the CDE and respond accordingly. This will be the
first time that the A-G completion rate is included in how the state measures its schools.
Challenges Preventing Students from Graduating College Ready
Academic Challenges
The College Board, which administers the AP program, reports that only 30 percent of
2011 public high school graduates participated in AP courses and only 18.1 percent succeeded in
scoring 3 or higher (“qualified” to receive college credit or placement into advanced courses) on
at least one AP exam (Venezia & Jaeger, 2013). Other common assessments used to determine
college readiness are the ACT and SAT exams, which are typically administered to high school
juniors and seniors (Venezia & Jaeger, 2013). In 2012, only 25 percent of all ACT-tested high
school graduates met the College Readiness Benchmarks in all four subjects, meaning that they
earned the minimum score needed to have a 50 percent chance of obtaining a “B” or higher in a
corresponding first-year college courses (Venezia & Jaeger, 2013). Looking at SAT data, among
the high school graduating class of 2012, only 43 percent of all SAT takers met the SAT College
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 25
& Career Readiness Benchmark, which indicates a 65 percent likelihood of obtaining a “B-”
average or higher during the first year of college (Venezia & Jaeger, 2013). Furthermore,
although there has been an increase in the number of high school graduates who have met the A-
G requirements in the past decade, this increase is not significant enough to keep up with the
growing number of school district’s that are aligning high school graduation requirements with
A-G requirements (Gordon, 2017).
As such, high school students in California face varying academic challenges in meeting
graduation requirements and entering college with the knowledge and skills necessary to be
successful (Hickman, Bartholomew, Mathwig & Heinrich, 2008).These students often struggle
academically in core content areas primarily because of the disparity between their reading level
and the required reading in high school (Mastropieri, Scruggs & Graetz, 2003). Content specific
textbooks lack good structure, cover a vast amount of material with insufficient specifics and do
not include leveled reading content (Mastropieri et al., 2003). Such textbooks are written with
density and complexity without accessibility to struggling students (Mastropieri et al., 2003).
Another academic challenge prohibiting students from being successful is the pace at
which teachers cover material (Mastropieri et al., 2003). To keep pace with year-long content
coverage, most teachers move on to the next chapter of the textbook without ensuring all
students have mastered the current material (Mastropieri et al., 2003). This is a vicious cycle
that leads to students falling further behind as the school year progresses (Mastropieri et al.,
2003). Systemically, teachers are not sufficiently trained to make learning accessible for
struggling students, thus, do not exhibit the knowledge and expertise required to support these
students. Instead, teachers move through covering content in a timely manner, so that their
students are prepared for high stakes standardized testing (Mastropieri et al., 2003).
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 26
Due to these systemic academic support challenges, students who do not meet high
school graduation requirements fall behind academically even prior to high school, inhibiting
their chances to ‘catch up’ in the four years they have in high school (Hickman, Bartholomew,
Mathwig & Heinrich, 2008). An abundance of research indicates that by third grade, if
appropriate and effective actions are not implemented to correct academic deficits, students can
enter an academic downward spiral that eventually puts them on an unnavigable pathway that
eventually leads them to drop out of school (Hickman et al., 2008). In addition to these systemic
academic challenges that plague both students and schools, social emotional and behavioral
issues must also be considered to understand the obstacles struggling students must overcome.
Social, Emotional & Behavioral Challenges
A comprehensive mission for schools is to educate students to be knowledgeable,
responsible, socially skilled, healthy, caring, and contributing citizens. However, large numbers
of students with mental health problems and deficits in social–emotional competence have
difficulty learning or disrupt the educational experiences of their peers (Benson, Scales, Leffert,
& Roehlkepartain, 1999). There is a myriad of factors that contribute to the social, emotional and
behavioral challenges a student in school faces today.
One such factor is poverty and its impact on overall student success. Students living in
poverty are less likely to meet graduation requirements than their more affluent counterparts
because children who grow up in poverty encounter more arduous challenges (Baydu, Kaplan &
Bayar, 2013). These challenges lead to having fewer role models, less opportunities for building
networks of trust, and more pressure to secure income as soon as possible (Raffo et al., 2009).
Because of this fluctuation in adult role models, there generally exists a lack of parental and
societal support, which in turn, leads to a higher dropout rate among students living in poverty
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 27
(Baydu et al., 2013). In addition, because minority students have a higher chance of being born
into poverty, Hispanic, African American, and Native American minorities exhibit a higher
dropout rate than their Caucasian peers (Baydu et al., 2013).
Beyond statistics related to the struggles of students born into poverty, a separate cadre of
research indicates that negative school experiences, such as behavioral issues within the
classroom, truancy, or lack of motivation can be a result of an engagement in high-risk behaviors
(Joe, Joe, & Rowley, 2009). High-risk behaviors can include, but are not limited to, substance
abuse and cigarette smoking. Students who engage in these high-risk behaviors tend to have an
increased number of negative school-related events, which then increases their rate of suspension
and failure of classes (Joe et al., 2009).
Another increasingly important factor that contributes to the social, emotional and
behavioral well-being of students is both their individual and collective mental health. The
number of incidents of suicidal behavior amongst students has nearly doubled in Los Angeles
Unified School (LAUSD) since they began tracking this data in 2010-11 (Schrobsdorff, 2016).
According to data from the 2010-12 National Health Interview Survey, about 4% of adolescents
aged 12-17 had a serious emotional or behavioral difficulty and received non-medication mental
health services in the past six months (Jones, Pastor, Simon & Reuben, 2014). Additionally,
approximately seven in ten adolescents with serious emotional or behavioral difficulties received
non-medication mental health services in the past six months (Jone et al., 2014). About 20% of
adolescents with serious emotional or behavioral difficulties receive school services for mental
health (Jones et al., 2014). As the data indicates, mental health issues can be a paramount
inhibitor of overall student success.
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 28
One last challenge in students’ social, emotional and behavioral well-being is how
schools address these problems (Greenberg et al., 2003; Zins, Weissberg, Wang & Walberg,
2004). Frequently, schools address student behavior problems by employing consequences such
as detention, suspension, and expulsion (Saeki et al., 2011). A disproportionate amount of time
and energy is spent on serving a small number of students with social-emotional and behavior
problems (Saeki et al., 2011).
The prevalence of social, emotional and behavioral issues among today’s high school
students is indicative of the supports individual schools must have in place to ensure students
graduate college ready. Implementing schoolwide systems of support at the high school level can
ensure that all students are provided with the degree of assistance necessary so that students will
graduate with the knowledge and skills to be successful in college.
Intervention Systems to Support Struggling Students
Multi-Tiered Systems of Support
Multi-tiered Systems of Support (MTSS) is a careful integration of the Response to
Instruction and Intervention (RTI
2
) and Positive Behavioral Interventions and Support (PBIS)
models to support students academically and behaviorally in school (Brown-Chidsey &
Bickford, 2016; California Department of Education, 2017; Freeman, Sugai, Simonsen &
Everett, 2017; McIntosh & Goodman, 2016). In the past few decades, the RTI
2
and PBIS
approaches to school reform, have been considered a huge success based on academic and
behavioral improvements and large-scale implementation (McIntosh & Goodman, 2016). The
goal of an integrated MTSS model is to provide all students with the best opportunities to
succeed both academically and behaviorally in school (McIntosh & Goodman, 2016).
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 29
MTSS integrates common components found in both RTI
2
and PBIS models including
preventive screening, the use of data to drive decisions, a focus on teaming, and a tiered
continuum of supports (Freeman et al., 2017; McIntosh & Goodman, 2016). MTSS focuses on
providing high-quality instruction and interventions matched to student need across domains
(e.g. literacy and social-emotional competence) and monitoring progress frequently to make
decisions about changes in instruction or goals (Freeman et al., 2017; McIntosh & Goodman,
2016). MTSS targets improvement in academic and behavioral outcomes for all students though
building leadership capacity and fostering a culture and system of continuous improvement
(Freeman et al., 2017).
There are three main reasons as to why combining RTI
2
and PBIS into an integrated
model known as MTSS make sense: First, there is a strong relationship between academic skills
and problem behavior (McIntosh & Goodman, 2016). Second, it’s efficient because RTI
2
and
PBIS share so many similarities (McIntosh & Goodman, 2016). Third, integrating both RTI
2
and
PBIS together may lead to an efficient use of resources and protect against competing initiatives
(McIntosh & Goodman, 2016).
According to McIntosh and Goodman (2016), the origins of integrating academic and
behavioral schoolwide intervention models can be traced back to the year 2000 when four
districts applied and received grants from the U.S. Office of Special Education Programs
(OSEP). These successful projects led to further research and exploration on how to implement
integrated models. Today many states such as California have included MTSS as one of their
model educational programs. The California Department of Education (CDE) (2017) defines
MTSS an integrated, comprehensive framework that focuses on CCSS, core instruction,
differentiated learning, student-centered learning, individualized student needs, and the
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 30
alignment of systems necessary for all students’ academic, behavioral, and social success.
Furthermore, the CDE (2017) describes MTSS as aligning initiatives, supports, and resources,
addressing the needs of all students, endorsing Universal Design for Learning (UDL) strategies,
integrating a data collection and assessment system, implementing appropriate research-based
interventions for improving student learning, using schoolwide and classroom research-based
positive behavioral supports for achieving important social and learning outcomes, and
implementing a collaborative approach to analyze student data and working together in the
intervention process.
MTSS addresses the academic as well as social, emotional and behavioral needs of
students through multiple levels of support for all learners (struggling through advanced). A
major ingredient of MTSS is the academic response to intervention piece, which includes data
gathering through universal screening, data-driven decision making, and problem-solving teams.
Response to Instruction and Intervention
Response to Instruction and Intervention (RtI
2
)
is a preventative systems approach to
improving individual and schoolwide achievement through high quality universal instruction
including needs-based tiered supports (McIntosh & Goodman, 2016). The system includes
collaborative teaming across general and special education (McIntosh & Goodman, 2016).
According to McIntosh and Goodman (2016), approximately 68% of schools are in some stage
of RTI implementation, 24% of those schools claim to implement RTI as a regular, systemic
practice, and. 13% of those schools implement RTI in reading/ELA at the secondary level.
This data-based approach to decision-making can influence the nature of instruction, early
intervention, and learning disability identification (Duffy, 2007). RtI
2
can be applied in schools
as a diagnostic approach that shapes instruction and informs decisions about intervention,
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 31
eligibility for special programs, design of individualized education plans (IEPs), and
effectiveness of special education programs (Duffy, 2007). Students who arrive in high school
performing below grade level in reading, writing, or mathematics can benefit from the increased
attention to instructional interventions and progress monitoring offered by RtI
2
constructs (Duffy,
2007).
More specifically, RtI
2
has been characterized as “the science and practice of assessment
and intervention” and is measured by the change in behavior as a byproduct of intervention. RtI
2
is typically comprised of five core components (Saeki et al., 2011):
1. a continuum of evidence-based services available to all students.
2. ongoing monitoring of student progress.
3. a systematic decision-making process of determining student progress in the academic or
behavioral domain.
4. implementation of increasingly intensive interventions when students do not demonstrate
improvements in response to other interventions.
5. evaluation of special education services for students who do not demonstrate
improvements despite implementation of increasingly intensive interventions.
The United States Public Health Service delineates three levels of prevention outcomes:
primary prevention, secondary prevention, and tertiary prevention. Primary prevention seeks to
prevent harm and secondary prevention seeks to reverse harm for those students at-risk for
school problems. Tertiary prevention also seeks to reduce harm but is aimed at students with the
most severe difficulties (Saeki et al., 2011). The more specific details of the steps are to identify
and then define the problem, explore solutions to the problem, implement the solution, and
subsequently examine the effects of the application (Saeki et al., 2011). This three-tiered
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 32
framework for academic intervention affords an opportunity to provide additional, meaningful
supports for students who are at-risk. In addition to instituting school wide academic supports for
struggling students, interventions to support the social, emotional and behavioral well-being of
students must also be considered.
Positive Behavioral Interventions and Support
An approach to support the social, emotional and behavior well-being of students is
school-wide Positive Behavioral Interventions and Supports (PBIS), in which the school focuses
on developing a predictable, efficient, and effective school climate. PBIS is a three-tiered
intervention approach for social and emotional behavior that emphasizes an instructional
approach to behavior support, prevention through environmental change, adaptation to the local
context, and using the science of applied behavior analysis to achieve outcomes that are valued
by staff, students, and families (McIntosh & Goodman, 2016). Research has proven that the
implementation of PBIS results in reduced disruptive behavior, reduced bullying, increased
academic achievement, improved school safety, improved teacher climate, increased social
competence, and increased emotional regulation (McIntosh & Goodman, 2016).
This multi-tiered behavioral framework is used to improve the integration and
implementation of behavioral practices, data-driven decision-making systems, professional
development opportunities, school leadership, supportive state and local education policies, and
evidence-based instructional strategies (Positive Behavioral Interventions and Supports, 2018).
The PBIS framework helps to “improve behavioral and academic outcomes by improving school
climate, preventing problem behavior, increasing learning time, promoting positive social skills,
and delivering effective behavioral interventions and supports (Positive Behavioral Interventions
and Supports, 2018).
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 33
Primary (universal), Secondary (targeted) and Tertiary (intensive) levels of support
implemented through the following guiding principles (Positive Behavioral Interventions and
Supports, 2018):
1. Foundation of social and behavior support (Tier I)
2. Implement PBIS across the whole school.
3. Invest in prevention first.
4. Establish a continuum of behavior support tailored to address the needs of ALL students.
5. Select and use evidence-based practices.
6. Build local capacity with high fidelity technical assistance and support.
7. Document high fidelity of practice implementation.
8. Decide with data.
9. Enhance implementation to be culturally relevant.
Through the implementation of school-wide PBIS, students are taught expected behaviors
and provided support for success on a prevention-oriented basis (Flannery, Fenning, Kato &
McIntosh, 2014). Data are used to evaluate outcomes and make decisions regarding student
needs and school practices (Flannery et al., 2014). The systematic implementation of PBIS
significantly reduces problem behavior for students in high schools, and the degree of reduction
is significantly related to the degree to which the critical features of PBIS are implemented
(Flannery et al., 2014). It is evident that the strategic implementation of PBIS reduces problem
behavior in high schools while promoting the social, emotional, and behavioral well-being of all
students. In order to examine the most effective intervention supports for students, the site
administrators’ direct role in the system must be studied.
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 34
Stakeholder Role in MTSS
Authored by Matthew Chambers
Implementing a systematic intervention system requires the commitment and expertise of
all school professionals, not just a handful of special education teachers and school
psychologists. Successful implementation of MTSS centers on the coordination and
collaboration of district and school staff to ensure the most effective instructional approaches are
used to meet the needs of students. (Duffy, 2007).
Site Administrators
Site administrators play a key role in leading the implementation of MTSS at their sites
by communicating goals, providing resources and training, and fostering a supportive
organization environment. Site administrators, especially the principal, have the ability to
develop a supportive organizational environment for MTSS to be implemented (Forman &
Crystal, 2015).
In interviews conducted with site administrators who have previously implemented
MTSS programs at their schools, they shared the role they played in getting MTSS off the
ground at their schools (Vekaria, 2017). For example, site administrators attended professional
development conferences on MTSS, they visited schools already implementing MTSS, and they
invited speakers to their school sites to teach their staffs about MTSS (Vekaria, 2017). Site
administrators emphasized a slow rollout with staff members including communicating
constantly throughout implementation, allowing staff members to share input and ideas, and
emphasizing support for all learners rather than at-risk students (Vekaria, 2017). Site
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 35
administrators reported that the biggest challenges in implementation were related to managing
time, scheduling, and resources (Vekaria, 2017).
Administrators reported that their presence, leadership, and ongoing commitment was
key to implementation of the MTSS process (Vekaria, 2017). Some of the key tasks that site
administrators accomplished during the implementation process were providing staff members
with resources for instruction, intervention, and data-based decision making, and allocating time
to problem solve and learn about MTSS (Vekaria, 2017. Site administrators play an important
role in cultivating staff buy-in by forming strong relationships with teachers and encouraging
shared leadership within their buildings.
Conceptual Framework
This is a collaborative action research project in which we will utilize Clark and Estes’
(2008) performance analysis model, which is a systematic, analytical method that helps to clarify
organizational goals and identify the current and preferred performance level within an
organization.
Assumed knowledge, motivation, and organizational needs will be generated based on
district information, personal knowledge, and related literature. These needs will be validated
and triangulated by data, including knowledge assessments, motivation and organization scales,
and document analysis. Data collection methods include individual interviews and document
analysis. Research-based solutions will be recommended accompanied by an implementation and
evaluation plan.
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 36
Stakeholder Knowledge, Motivation and Organizational Factors
Knowledge and Skills
According to Clark and Estes (2008), performance gaps are likely the result of three
barriers: knowledge and skills, motivation, and organizational factors. Knowledge and skill
barriers may exist, resulting in stakeholders not knowing how to achieve the goal. In the
educational setting, Rueda (2011) has cautioned the use of fragmented tactics, improper
implementation, and unrealistic solutions. It will be the goal of this gap analysis to recognize the
importance of understanding and validating the causes of the performance gap before developing
and implementing a principal-based solution (Clark & Estes, 2008; Rueda, 2011). Furthermore,
this study will identify and assess the knowledge and skill influences of site administrators
including declarative factual, conceptual, procedural, and metacognitive (Clark & Estes, 2008).
Declarative factual knowledge influences. Krathwohl (2002) states that factual
knowledge contains the basic elements that pupils must know to be acquainted with a discipline
or solve problems in it. This includes terminology or details that one must have knowledge about
in order to understand and effectively solve a problem in a given domain (Rueda, 2011).
Site administrator’s declarative factual knowledge influences
Site administrators know current student performance data and what observable and
measurable goals are. Site administrators must be aware of student achievement data and how to
use it in the goal-setting process (Schmoker, 1999). Schmoker (1999) identified a combination
of three concepts, which include teamwork, creating measurable goals, and performance data
analysis as the ongoing recipe for positive school improvement. Knowledge of data and current
and desired outcomes is essential for student success (Schmoker, 1999).
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 37
Site administrators know teacher roles in the RTI
2
and PBIS process. Implementing a
systematic intervention system requires changes in teacher roles as well as the culture of
classrooms and schools (Duffy, 2007). A core component of an MTSS model is integrated
teaming at the student, classroom, grade-level, and school, which require teacher participation in
analyzing data, developing goals, reviewing progress, and planning Tier 2 and Tier 3
interventions (McIntosh & Goodman, 2016).
Site administrators know district office leader roles in the MTSS process. McIntosh
and Goodman (2016) recommend that district office leaders establish three teams to support
teachers implementing MTSS at the site level. These three teams include a cabinet team, an
implementation support team, and a training and coaching group. Additionally, it is
recommended that existing organizational structures are far more effective and long lasting then
newly formed groups (McIntosh & Goodman, 2016).
Conceptual knowledge influences. Conceptual knowledge refers to a person’s
knowledge of categories, classifications, principles, generalizations, theories, models, or
structures pertinent to a specific domain (Krathwohl, 2002). In other words, conceptual
knowledge is understanding the interrelationships among the smaller components within a larger
structure that enable them to function together (Krathwohl, 2002).
Site administrator’s conceptual knowledge influences
Site administrators know how to classify students as either on track or off track of
meeting organizational goals and targeted results for student success. School leaders who stay
focused on monitoring and supporting established school goals are more likely to have a positive
impact on student achievement (Marzano, Waters, & McNulty, 2005). Using data provides
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 38
objective and quantifiable proof as to whether students are on track to meet performance goals
(American Association of School Administrators, 2002).
Site administrators know how to categorize teacher behaviors as either appropriate
RTI
2
and PBIS strategies or not appropriate RTI
2
and PBIS strategies. Site administrators
from schools that have implemented MTSS recommend that school leaders attend professional
development trainings and observe schools where MTSS has already been implemented or is in
the process of implementation to better understand what appropriate RTI
2
and PBIS strategies
should look like (Vekaria, 2017).
In their 2017 study on MTSS implementation, Lancaster and Hougen (2017) identified
appropriate RTI
2
and PBIS strategies that teachers must have knowledge and awareness of.
These teacher behaviors include but are not limited to administering and interpreting universal
screening techniques, enacting high-leverage teaching practices such as Universal Design for
Learning (UDL), and monitoring student progress (Lancaster & Hougen, 2017).
Site administrators know full MTSS implementation. McIntosh and Goodman (2016)
identify five stages of MTSS implementation, which include exploration/adoption, installation,
initial implementation, elaboration, and continuous regeneration. The exploration/adoption
phase is where schools determine whether the MTSS model fits with current initiatives, will
serve a student need, and there are resources in place to support the implementation (McIntosh &
Goodman, 2016). The installation and initial implementation phases involve creating leadership
teams, planning for implementation, and piloting the program on a smaller scale (McIntosh &
Goodman, 2016). Lastly, the elaboration and continuous regeneration phases require the school
to modify practices based on initial implementation, make the system more effective and
efficient, and evolve with changing school contexts (McIntosh & Goodman, 2016).
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 39
Procedural knowledge influences. According to Krathwohl (2002), procedural
knowledge involves having the skills necessary to apply learned concepts. Learning is increased
when learners acquire component skills, practice integrating them on a consistent basis, and over
time, learn when to apply the skill-set they have acquired (Clark & Estes, 2008).
Site administrator’s procedural knowledge influences
Site administrators know steps to develop organizational goals and targeted results for
student success and how to use data to monitor their progress. Organizational goals should be
designed to provide direction and articulate what a group or entity wants to accomplish (Rueda,
2011). There are three general levels of goals that vary by the length of time in which you wish
to reach them: global (1-5 years), intermediate (weeks to months), and performance (hours to
days) (Rueda, 2011). The most effective organizational goals are measurable and are set against
a benchmark entity such as the success of a competing or similar school (Rueda, 2011).
Additionally, intermediate and performance goals need to be aligned with global goals in order
to maximize their effectiveness and organizational results (Clark & Estes, 2008; Rueda, 2011).
Using data to monitor organizational goals and targeted results for students is an essential
element of the MTSS model (Brown-Chidsey & Bickford, 2016; McIntosh & Goodman, 2016).
Monitoring progress with data in the MTSS model is done to audit current practices, identify
students requiring additional support (e.g. screening students for Tier II), assess individual needs
(e.g. Tier III level), assess continuous student growth (e.g. formative assessments), and assess
student outcomes on a broader scale (e.g. summative assessments) (McIntosh & Goodman,
2016).
Site administrators know how to observe and identify effective RTI
2
and PBIS
strategies. Brown-Chidsey & Bickford (2016) recommend that schools create an MTSS
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 40
handbook with a section for academics (RTI
2
) and a section for behavior (PBIS) to assist site
administrators in observing and identifying effective intervention practices. The academic
portion of the handbook should include summaries and overviews of Tier I materials and
programs, universal screening assessment tools, Tier II interventions available at the school,
progress monitoring tools and tables, approved Tier III interventions, and evaluation tools for
special education referral (Brown-Chidsey & Bickford, 2016). The behavior portion of the
school’s MTSS handbook should include schoolwide expectations and plans for teaching the
expectations to students, procedures for teachers to follow the acknowledgement/reward system,
the flowchart for how staff document and respond to challenging behaviors, the process for
providing Tier II and Tier III interventions for struggling students, and details on the data
management process for decision making (Brown-Chidsey & Bickford, 2016).
Site administrators develop a system for reinforcing, monitoring, encouraging, and
rewarding teachers for implementing components of MTSS, while providing supports for
teachers who are not. In a meta-analysis of 69 studies on principal leadership, it was discovered
that the site administrator responsibility of affirmation was most closely correlated with student
achievement (Marzano, Water, & McNulty, 2005). Affirmation as described in the study is the
extent to which a principal recognizes and celebrates accomplishments of the school, staff, and
students (Marzano, et al., 2005). Principals who implemented MTSS at their sites reported
higher success rates of teacher buy-in when colleagues shared success stories and methods of
implementation (Vekaria, 2017). In addition to recognizing the successful components of MTSS
implementation, the affirmation process also includes the difficult task of acknowledging failures
(Marzano, et a., 2005). Acknowledging setbacks and celebrating successes is a balanced
approach to accounting for a school’s successes and failures (Cottrell, 2002).
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 41
It is inevitable that teachers will require training and support when implementing a tiered
support model such as MTSS (O’Connor and Freeman, 2012). For example, teachers will need
training and support in the areas of assessment and data management (O’Connor and Freeman,
2012). Site administrators need to be prepared with resources to provide teachers in need of
supports such as referring them to conferences or trainings, sending them on site visits to schools
who have already implemented, or partnering them with more successful teachers on their own
site (McIntosh & Goodman, 2016).
Metacognitive knowledge influences. Metacognitive knowledge is the awareness of
one’s own cognition; it is strategic or reflective knowledge about how to go about solving
problems and cognitive tasks, which include contextual and conditional knowledge and
knowledge of self (Krathwohl, 2002). Cognitive processes include remembering relevant
information from long-term memory, understanding and building meaning from information,
carrying out a procedure or process in a given situation, evaluating and making judgements
based on criteria and standards, and forming patterns based on distinct elements from given
situations (Mayer, 2008).
Site administrator’s metacognitive knowledge influences
Site administrators reflect on the process of defining organizational goals and targeted
results for student success, reflect on the process of encouraging and rewarding teacher
critical behaviors, and reflect on full MTSS implementation to monitor and adjust
organizational goals. Baker (2006) emphasizes that learners can more readily apply knowledge
acquired in one context to another context if they have more awareness of themselves as learners,
if they monitor their strategies and resources, and if they assess their readiness for performances.
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 42
Site administrators who reflect on the process they use for setting goals, encouraging teacher
critical behaviors, and monitoring progress will likely be more efficient and effective leaders.
Table 2
Summary of Assumed Knowledge Influences on Stakeholder’s Ability to Achieve the
Performance Goal
Assumed Knowledge Influences Research Literature
Author, Year; Author, Year.
Declarative Factual (terms, facts, concepts)
Stakeholder knows…
Site administrators know current student
performance data and what observable and
measurable goals are.
Schmoker, 1999
Site administrators know teacher roles in the RTI
2
and PBIS process.
Duffy, 2007
McIntosh & Goodman, 2016
Site administrators know district office leader roles
in the MTSS process.
McIntosh & Goodman, 2016
Declarative Conceptual (categories, process
models, principles, relationships)
Stakeholder knows…
Site administrators classify students as either on
track or off track of meeting organizational goals
and targeted results for student success.
Marzano, Waters, & McNulty, 2005
American Association of School
Administrators, 2002
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 43
Site administrators know how to categorize teacher
behaviors as either appropriate RTI
2
and PBIS
strategies or not appropriate RTI
2
and PBIS
strategies.
Vekaria, 2017
Lancaster & Hougen, 2017
Site administrators know full MTSS
implementation.
McIntosh & Goodman, 2016
Procedural
Stakeholder knows how to…
Site administrators know steps to develop
organizational goals and targeted results for student
success and how to use data to monitor their
progress.
Rueda, 2011
Clark & Estes, 2008
Brown-Chidsey & Bickford, 2016
McIntosh & Goodman, 2016
Site administrators observe and identify effective
RTI
2
and PBIS strategies.
Brown-Chidsey & Bickford, 2016
Site administrators develop a system for
reinforcing, monitoring, encouraging and rewarding
teachers for implementing components of MTSS,
while providing supports for teachers who are not.
Marzano, Water, & McNulty, 2005
Vekaria, 2017
Cottrell, 2002
O’Connor and Freeman, 2012
McIntosh & Goodman, 2016
Metacognitive
Stakeholders know how to reflect on…
Site administrators reflect on the process of
defining organizational goals and targeted results
for student success, reflect on the process of
encouraging and rewarding teacher critical
behaviors, and reflect on full MTSS implementation
to monitor and adjust organizational goals.
Baker, 2006
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 44
Motivation
General theory. A learner’s success is heavily reliant on the learner’s motivation.
Motivation refers to the personal investment that an individual has in reaching a desired state or
outcome (Ambrose et al., 2010). Clark & Estes (2008) assert that active choice, persistence, and
mental effort are three indices of a learner’s motivation. Active choice occurs when the learner’s
intention to pursue a goal is replaced by action, persistence transpires when the learner continues
the action in the face of distractions, and mental effort includes the strategies the learner chooses
to work smarter and develop novel solutions to achieve the initial goal.
Value. Value refers to a learned belief about the effectiveness or benefits of a task or goal
which can change without self-awareness (Clark & Estes, 2008). Eccles (2006) suggests that
modeling value and interest in a given content can foster positive values within the learner.
Site administrators value influences
Site administrators value defining observable and measurable organizational goals and
targeted results for student success, creating processes that reinforce, monitor, encourage, and
reward teacher critical behaviors, and using processes to partner with district office leaders to
ensure proper support and/or recognition of teachers and sites in their implementation of
MTSS. According to Clark and Estes (2008), people determine whether they value an idea or
task based on whether they perceive it will help them or not. Furthermore, values can be likened
to preferences that lead individuals such as site administrators to embark on a mission and persist
in the face of adversity or distraction (Clark & Estes, 2008). It is necessary to assess whether site
administrators will accomplish any of the responsibilities associated with successfully
implementing an MTSS model by first determining whether they value the tasks associated with
it.
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 45
Self-Efficacy. Self-efficacy refers to one’s confidence in their ability to complete a
specific task or performance goal, and their belief that by completing these tasks or goals they
can make a difference in the long term (Bandura, 1997; Clark & Estes, 2008).
Site administrator’s self-efficacy influences
Site administrators are confident about defining observable and measurable
organizational goals and targeted results for student success, creating processes that reinforce,
monitor, encourage, and reward teacher critical behaviors, and using processes to partner
with district office leaders to ensure proper support and/or recognition of teachers and sites in
their implementation of MTSS. Bandura (1997) asserts that task-specific confidence is critical
for feeling motivated in the work setting. According to Rueda (2011), self-efficacy is most
closely connected to the motivational indicator of persistence. Therefore, site administrators will
need high levels of self-efficacy to implement MTSS at their sites in the face of challenges and
obstacles.
Emotions. Clark and Estes (2008) refer to emotions as the mood or attitude one has
toward pursuing and achieving a goal. This means that one’s attitude about achieving a goal will
affect how they handle a given task. If they feel the task is not possible or do not believe it is
worthwhile, they could develop a negative attitude which would have an adverse effect on the
given task (Clark & Estes, 2008).
Site administrator’s emotion influences
Site administrators feel positive about defining observable and measurable
organizational goals and targeted results for student success, creating processes that reinforce,
monitor, encourage, and reward teacher critical behaviors, and using processes to partner
with district office leaders to ensure proper support and/or recognition of teachers and sites in
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 46
their implementation of MTSS. Anderman & Anderman (2006) focus on emotions as they relate
to being prideful and having a stronger sense of self-esteem when an achievement related event
is at play. If site administrators within a given school setting feel positive about a successfully
accomplished task, the outcome of the task will motivate site administrators to continue the
completing task in the future. As such, site administrators who feel positive about setting goals,
creating processes for rewarding teachers, and working with the district office to implement
MTSS will continue to do so with a strong sense of motivation, which again, will optimize
learning and behavioral outcomes for students.
Attributions. According to Anderman & Anderman (2006), Attribution Theory
examines individuals' beliefs about why certain events occur and correlates those beliefs to
subsequent motivation. Characteristics of a given attribution are classified along three causal
dimensions: locus, stability, and controllability.
Site administrator’s attribution influences
Site administrators believe that the success or failure of defining observable and
measurable organizational goals and targeted results for student success, creating processes
that reinforce, monitor, encourage, and reward teacher critical behaviors, and using processes
to partner with district office leaders to ensure proper support and/or recognition of teachers
and sites in their implementation of MTSS is in their control. When site administrators can
process that the success or failure of any given task is within their locus of control, they attribute
that success or failure to an internal or external factor (Yough & Anderman, 2006). Dependent
on how effectively a school implements its MTSS program, a site administrator can attribute
their success, or lack thereof, to the program itself or to their individual work related to the
program.
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 47
Table 3
Summary of Assumed Motivation Influences on Stakeholder’s Ability to Achieve the Performance
Goal
Assumed Motivation Influences Research Literature
Author, Year; Author, Year.
Value
Stakeholder values…
Site administrators value defining
observable and measurable organizational
goals and targeted results for student
success, creating processes that reinforce,
monitor, encourage, and reward teacher
critical behaviors, and using processes to
partner with district office leaders to ensure
proper support and/or recognition of
teachers and sites in their implementation
of MTSS.
Clark & Estes, 2008
Self-Efficacy
Stakeholder is confident that….
Site administrators are confident about
defining observable and measurable
organizational goals and targeted results
for student success, creating processes that
reinforce, monitor, encourage, and reward
teacher critical behaviors, and using
processes to partner with district office
leaders to ensure proper support and/or
recognition of teachers and sites in their
implementation of MTSS.
Bandura, 1997
Rueda, 2011
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 48
Emotions
Stakeholder feels positive about…
Site administrators feel positive about
defining observable and measurable
organizational goals and targeted results
for student success, creating processes that
reinforce, monitor, encourage, and reward
teacher critical behaviors, and using
processes to partner with district office
leaders to ensure proper support and/or
recognition of teachers and sites in their
implementation of MTSS.
Anderman & Anderman, 2006
Attributions
Stakeholders believes that ....
Site administrators believe that the success
or failure of defining observable and
measurable organizational goals and
targeted results for student success,
creating processes that reinforce, monitor,
encourage, and reward teacher critical
behaviors, and using processes to partner
with district office leaders to ensure proper
support and/or recognition of teachers and
sites in their implementation of MTSS is in
their control.
Yough & Anderman, 2006
Organization
Organizational factors such as culture, structure, resources, policies, and practices are one
of the three barriers that performance gaps can be attributed to (Clark & Estes, 2008; Rueda,
2011). Organizational culture can be described as the group norms, rules of the game, habits of
thinking, formal rituals, or philosophies of the company (Rueda, 2011).
Resources. Employees need resources such as time, money, or people to meet
performance goals (Clark & Estes, 2008).
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 49
Site administrators resource influences
Site administrators have the funding to be trained on implementing an MTSS program,
and to train, support, and recognize teachers in the MTSS implementation process. Site
administrators and teachers need ongoing training and resources to ensure proper MTSS
implementation (Vekaria, 2017). Besides attending professional development trainings, this
requires site administrators to provide teachers with resources for instruction, intervention, and
data-based decision making and allocate time to learn about the MTSS framework (Vekaria,
2017). Lancaster & Hougen (2017) recognize that the biggest barrier to schools successfully
implementing MTSS is that there is a lack of resources for teacher training.
Site administrators have the time to plan and implement an MTSS program and meet
with district office personnel. It is valuable for district office personnel to make time to
communicate with site administrators and give them feedback throughout the MTSS
implementation process (Vekaria, 2017). Site administrators from the state of Michigan who
implemented MTSS reported that their biggest challenges were related to time, scheduling, and
resources at the site level (Vekaria, 2017). District office personnel need to work with school
personnel to create a library of materials and tools for successful implementation of the MTSS
model (McIntosh & Goodman, 2016).
Policies and procedures. Policies refer to rules put in place by an organization, while
procedures guide individuals on how to complete a task (Clark & Estes, 2008). Performance
problems are common when policies and procedures conflict with organizational culture (Clark
& Estes, 2008). Organizational policies and procedures can influence whether individuals such
as site administrators or teachers meet their individual performance goals (Rueda, 2011).
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 50
Site administrator’s policies and procedures influences
Policies and procedures are in place for site administrators to create processes that
reinforce, monitor, encourage, and reward teacher critical behaviors and are aligned with
school and district policies. Integrating academic and behavior intervention models is a large-
scale change to an organizational structure that shifts responsibilities and staff roles (McIntosh &
Goodman, 2016). Site administrators must be sensitive of this when they are working with
teachers. They must consider teacher contracts when modifying staff roles and duties throughout
the MTSS implementation process. Furthermore, any evaluation tool must be presented to
teachers, so they can learn new expectation, attend trainings, and practice implementation
(Brown-Chidsey & Bickford, 2016).
Policies and procedures are in place for site administrators to partner with district
office leaders to ensure proper support and/or recognition of teachers and sites in their
implementation of MTSS (RTI
2
and PBIS strategies). The implementation of an MTSS model
requires organizational changes that often challenge the current policies and procedures in place
(McIntosh & Goodman, 2016). Therefore, effective MTSS programs include a district-level
team with members capable of making policy changes. (McIntosh & Goodman). If site
administrators attempt to implement an MTSS model without district members on the team, the
project will likely come to a halt.
Cultural setting. Cultural settings are the visible locations where the policies and
practices of an organization intersect (Gallimore & Goldenberg, 2001; Rueda, 2011). A cultural
setting describes people coming together over time to accomplish a goal (Gallimore &
Goldenberg, 2001).
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 51
Site administrator’s cultural setting influences
The school has systems in place to motivate site administrators to define observable
and measurable organizational goals and targeted results for student success, create processes
that reinforce, monitor, encourage, and reward teacher critical behaviors, and use processes to
partner with district office leaders to ensure proper support and/or recognition of teachers and
sites in their implementation of MTSS (RTI
2
and PBIS strategies). Schools are complex social
settings that are in a constant state of change (Rueda, 2011). Because of these dynamics, the
systems put in place are often a hindrance to achieving goals or improving performance (Rueda,
2011). As school leaders, site administrators must understand the characteristics that make up
their organizations if they wish to have influence on them.
Cultural models. Cultural models are the invisible aspects of an organization that might
include the behavior, artifacts, rules, values, practices, policies, or reward structures of an
organization (Gallimore & Goldenberg, 2001; Rueda, 2011). Any organization can be viewed as
exhibiting a cultural model, which is defined as the shared perception of the organization at large
(Gallimore & Goldenberg, 2010).
Site administrator’s cultural model influences
There is a culture of cooperation among site administrators who define observable and
measurable organizational goals and targeted results for student success, create processes that
reinforce, monitor, encourage, and reward teacher critical behaviors, and use processes to
partner with district office leaders to ensure proper support and/or recognition of teachers and
sites in their implementation of MTSS (RTI
2
and PBIS strategies). Fostering a culture of
cooperation and a sense of community among staff members is one of most significant ways that
school leaders can influence student achievement (Marzano, et al., 2005). Developing a shared
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 52
vision for the school and setting goals as a staff or team are ways that a culture of cooperation is
modeled (Marzano, et al, 2005). A culture of cooperation among site administrators will more
likely lead to the successful implementation of MTSS.
Table 4
Summary of Assumed Organization Influences on Stakeholder’s Ability to Achieve the
Performance Goal
Assumed Organization Influences Research Literature
Author, Year; Author, Year.
Resources (time; finances; people)
Stakeholder have the resources (time,
money, people) to …
Site administrators have the funding to be
trained on implementing an MTSS
program, and to train, support, and
recognize teachers in the MTSS
implementation process.
Vekaria, 2017
Lancaster & Hougen 2017
Site administrators have the time to plan
and implement an MTSS program and
meet with district office personnel.
Vekaria, 2017
McIntosh & Goodman, 2016
Policies, Processes, & Procedures
Stakeholder have policies that align with
…
Policies and procedures are in place for site
administrators to create processes that
reinforce, monitor, encourage, and reward
teacher critical behaviors and are aligned
with school and district policies.
McIntosh & Goodman, 2016
Brown-Chidsey & Bickford, 2016
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 53
Policies and procedures are in place for site
administrators to partner with district office
leaders to ensure proper support and/or
recognition of teachers and sites in their
implementation of MTSS (RTI
2
and PBIS
strategies).
McIntosh & Goodman, 2016
Culture
Stakeholder feels like they are part of an
organization that aligns with …
The school has systems in place to
motivate site administrators to define
observable and measurable organizational
goals and targeted results for student
success, create processes that reinforce,
monitor, encourage, and reward teacher
critical behaviors, and use processes to
partner with district office leaders to ensure
proper support and/or recognition of
teachers and sites in their implementation
of MTSS (RTI
2
and PBIS strategies).
Rueda, 2011
There is a culture of cooperation among
site administrators who define observable
and measurable organizational goals and
targeted results for student success, create
processes that reinforce, monitor,
encourage, and reward teacher critical
behaviors, and use processes to partner
with district office leaders to ensure proper
support and/or recognition of teachers and
sites in their implementation of MTSS
(RTI
2
and PBIS strategies).
Marzano, Water, and McNulty,
2005
Summary
It is apparent from the literature that there are numerous knowledge, motivation, and
organizational (KMO) factors that influence site administrators as they pursue schoolwide goals.
The literature demonstrates specific KMO areas that site administrators should consider when
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 54
looking to increase the number of college ready students who graduate from their schools. For
instance, the academic, behavior, and social-emotional challenges that prevent students from
succeeding in schools can be addressed by providing a Multi-Tiered System of Support (MTSS)
to students (McIntosh & Goodman, 2016). Based on the findings from the literature review, the
methodology for this study was designed to learn about the KMO needs of site administrators
implementing an MTSS program at their school.
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 55
CHAPTER THREE: METHODOLOGY
Authored by Matthew Chambers
Purpose of the Project and Questions
The purpose of this study was to examine the implementation of MTSS in the Downtown
Unified School District (DUSD) high schools from the perspective of site administrators. More
specifically, the purpose of this project was to conduct a needs analysis in the areas of
knowledge and skill, motivation, and organizational resources necessary to reach the
organizational performance goal of effectively implementing MTSS at Westside High School by
June 2020. The analysis began by generating a list of possible needs and then moved on to
examining these systematically to focus on actual or validated needs. While a complete needs
analysis would focus on all stakeholders, for practical purposes the stakeholder that was focused
on in this analysis was DUSD site administrators.
Two questions guided this study.
1. What are the knowledge and skills, motivation, and organizational needs for site
administrators to effectively implement MTSS at Westside High School?
2. What are the knowledge and skills, motivation, and organizational recommendations for
meeting the needs of site administrators in achieving the DUSD goal of effectively
implementing MTSS at Westside High School?
Conceptual and Methodological Framework
This was a collaborative action research project in which Clark and Estes’ (2008)
performance analysis model was utilized. The Clark and Estes (2008) model is a systematic,
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 56
analytical method that helps to clarify organizational goals and identify the current and preferred
performance level within an organization. Achieving organizational goals and closing
performance gaps is not possible without identifying the root cause of the problem (Clark &
Estes, 2008). To diagnose the problem, we must understand the human causes and the three
critical areas that they might fall under including people’s knowledge and skills, their motivation
to achieve the goal, or any organizational factors. (Clark & Estes, 2008; Rueda, 2011).
The Clark & Estes (2008) performance analysis model that was used in this study is
known as an innovation model. The innovation model is used when organizations such as the
Downtown Unified School District strive to do something new such as implement a new
program or process. As part of the innovation model, a knowledge and skills, motivation, and
organizational needs assessment was conducted to determine how the organization can reach
their performance goal.
The cyclical flow model below illustrates the steps of the Gap Analysis Process.
Figure 1. Gap analysis process adapted from Clark and Estes (2008).
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 57
Assessment of Performance Influences
The literature review in Chapter Two identified the knowledge, motivation, and
organizational (KMO) influences that will affect site administrators who are implementing
comprehensive interventional models in their schools. The purpose of this project was to
conduct a needs analysis in the areas of knowledge and skill, motivation, and organizational
resources necessary to reach the organizational performance goal of effectively implementing
MTSS at Westside High School by June 2020. The Clark and Estes (2008) gap analysis model
was utilized to determine the KMO needs to meet the desired performance goal. This section of
the study outlines the qualitative methods that were used to assess the KMO needs including
interview questions and document analysis. Once the assessments were concluded, a
comprehensive evaluation plan was created to address the KMO influences that are lacking.
Knowledge Assessment
The literature review in Chapter Two helped establish a list of assumed knowledge
influences that affect site administrators as they implement an MTSS program at their school.
These knowledge influences were established based on Anderson and Krathwohl’s (2001) four
knowledge types of factual, conceptual, procedural, and metacognitive as outlined in A
Taxonomy for Learning, Teaching, and Assessing. This list is presented in Table 5 along with
assessment methods for collecting data specific to the knowledge areas of factual, conceptual,
procedural, and metacognitive knowledge.
Factual knowledge. To perform their critical behaviors, site administrators must have
knowledge of current student performance data and what observable and measurable goals are,
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 58
teacher roles in the RTI
2
and PBIS process, and district office leader roles in the MTSS process.
Anderson and Krathwohl (2001) recommend using inquiry methods that require participants to
state, explain, discuss, or declare what they know to assess an individual’s factual knowledge.
This requires knowledge of basic facts, specific details, and terminology related to MTSS. In
order to validate factual knowledge, site administrators were asked interview questions and a
document analysis was completed to review artifacts for evidence. Table 5 provides an overview
of the methods that were used, along with the sample interview items and documents that were
analyzed.
Conceptual knowledge. Based on the literature, it is assumed that site administrators
have the knowledge to classify students as either on track or off track of meeting organizational
goals and targeted results for student success, the knowledge to categorize teacher behaviors as
either appropriate RTI
2
and PBIS strategies or not appropriate RTI
2
and PBIS strategies, and the
knowledge to understand what full MTSS implementation looks like. Anderson and Krathwohl
(2001) recommend using inquiry methods that require participants to demonstrate
interrelationships among basic elements within a larger structure that enable them to function
together. This requires knowledge of underlying categories, principles, or structures of MTSS. In
order to validate conceptual knowledge, site administrators were asked interview questions and a
document analysis was completed to review artifacts for evidence. Table 5 provides an overview
of the methods that were used, along with the sample interview items and documents that were
analyzed.
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 59
Procedural knowledge. The procedural knowledge that site administrators need to
perform their critical behaviors are knowing steps to develop organizational goals and targeted
results for student success and how to use data to monitor their progress, knowing how to
observe and identify effective RTI
2
and PBIS strategies, and knowing how to develop a system
for reinforcing, monitoring, encouraging, and rewarding teachers for implementing components
of MTSS, while providing supports for teachers who are not. Anderson and Krathwohl (2001)
recommend using inquiry methods that require participants to demonstrate how to do
something. This requires knowledge of skills and procedures involved with a task, including
techniques, methods of inquiry, and criteria for using skills, techniques, and
methods. In order to validate procedural knowledge, site administrators were asked interview
questions and a document analysis was completed to review artifacts for evidence. Table 5
provides an overview of the methods that were used, along with the sample interview items and
documents that were analyzed.
Metacognitive knowledge. To perform their critical behaviors, site administrators reflect
on the process of defining organizational goals and targeted results for student success, reflect on
the process of encouraging and rewarding teacher critical behaviors, and reflect on full MTSS
implementation to monitor and adjust organizational goals. Anderson and Krathwohl (2001)
recommend using inquiry methods that require participants to demonstrate knowledge of
cognition as well as awareness and knowledge of their own strategies for learning and thinking.
This requires the knowledge to reflect on and become more aware of one’s own beliefs,
knowledge of planning strategies, and knowledge of comprehension-monitoring strategies.
In order to validate metacognitive knowledge as an asset or actual need, site administrators were
asked interview questions and a document analysis was completed to review artifacts for
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 60
evidence. Table 5 provides an overview of the methods that were used, along with the sample
interview items and documents that were analyzed.
Table 5
Summary of Knowledge Influences and Method of Assessment
Assumed Knowledge Influences Document
Item
Interview Item
Declarative Factual (terms, facts, concepts)
Stakeholder knows…
Site administrators know current student
performance data and what observable and
measurable goals are.
2015 WASC
Self-Study
Report
Give me an
example of some
current student
performance data
at your school.
Site administrators know teacher roles in the
RTI
2
and PBIS process.
Describe the
teacher role in the
MTSS process.
Site administrators know district office
leader roles in the MTSS process.
Describe for me
district office
leader roles in the
MTSS process.
Declarative Conceptual (categories,
process models, principles, relationships)
Stakeholder knows…
Site administrators classify students as either
on track or off track of meeting
2015 WASC
Self-Study
Report
How do you
classify students
as being on track
or off track for
meeting their
academic goals?
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 61
organizational goals and targeted results for
student success.
Site administrators know how to categorize
teacher behaviors as either appropriate RTI
2
and PBIS strategies or not appropriate RTI
2
and PBIS strategies.
Give me an
example of an
appropriate
MTSS strategy?
Give me an
example of an
inappropriate
MTSS strategy?
Site administrators know full MTSS
implementation.
What is your
understanding of
a Multi-Tiered
System of
Support (MTSS)?
As a site
administrator,
what do you see
your role being in
the MTSS
process?
What do you
think that full
implementation
of MTSS looks
like?
Procedural
Stakeholder knows how to…
Site administrators know steps to develop
organizational goals and targeted results for
student success and how to use data to
monitor their progress.
2015 WASC
Self-Study
Report
Describe the
steps you take to
develop
organizational
goals for student
success.
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 62
Site administrators observe and identify
effective RTI
2
and PBIS strategies.
2018 WHS
Transition
Guide
Tell me the steps
you would take
for observing and
identifying
effective MTSS
strategies.
Site administrators develop a system for
reinforcing, monitoring, encouraging and
rewarding teachers for implementing
components of MTSS, while providing
supports for teachers who are not.
Tell me how you
reinforce,
encourage, and
reward teachers
implementing
MTSS strategies.
Describe the
supports you
would provide
teachers who are
not implementing
components of
MTSS.
Metacognitive
Stakeholders know how to reflect on…
Site administrators reflect on the process of
defining organizational goals and targeted
results for student success, reflect on the
process of encouraging and rewarding
teacher critical behaviors, and reflect on full
MTSS implementation to monitor and adjust
organizational goals.
How do you
evaluate the
effectiveness of
your role in
implementing
new programs?
What leads you to
change your
practices?
Motivation Assessment
Motivation is an internal process that initiates and sustains goal-directed behavior (Clark
& Estes, 2008; Mayer, 2011; Rueda, 2011). The literature review in Chapter Two helped
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 63
establish a list of assumed motivation influences that affect site administrators as they implement
an MTSS program at their school. This list is presented in Table 6 along with assessment
methods for collecting data specific to the motivational areas of value, self-efficacy, emotions,
and attribution.
Value. To perform their critical behaviors, site administrators need to value the tasks
associated with implementing MTSS at their site. The motivational factor of value was assessed
by asking site administrators open-ended interview questions to solicit the importance they attach
to the tasks related to MTSS (Clark & Estes, 2008; Rueda, 2011). Table 6 provides an overview
of the methods that were used, along with the sample interview items and documents that were
analyzed.
Self-efficacy. Assessing the motivational influence of self-efficacy requires an appraisal
of one’s personal capabilities to execute the functions they perform in the group (Bandura,
2006). In this study, site administrators were assessed to determine their confidence level with
regards to the critical tasks of defining observable and measurable organizational goals and
targeted results for student success, creating processes that reinforce, monitor, encourage, and
reward teacher critical behaviors, and using processes to partner with district office leaders to
ensure proper support and/or recognition of teachers and sites in their implementation of MTSS.
To determine if the motivational factor of self-efficacy was an actual need, site administrators
were asked interview questions and a document analysis was completed to review artifacts for
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 64
evidence. Table 6 provides an overview of the methods that were used, along with the sample
interview items and documents that were analyzed.
Emotions. Open-ended interview questions that focus on emotional reactions and solicit
personal experience were used to assess emotions. This needs assessment was meant to
determine how positive site administrators feel about performing the critical behaviors required
to implement MTSS. Table 6 provides an overview of the methods that were used, along with the
sample interview items and documents that were analyzed.
Attributions. The last motivational influence that was assessed in this study was
attributions or the amount of control an individual or group believes they have on affecting an
outcome (Rueda, 2011). If individuals perceive a performance goal to be out of their control,
regardless of the effort they put into an activity or task, then they will choose not to work toward
that goal (Clark & Estes, 2008; Rueda, 2011). This study assessed whether site administrators
believe that the success or failure of defining observable and measurable organizational goals
and targeted results for student success, creating processes that reinforce, monitor, encourage,
and reward teacher critical behaviors, and using processes to partner with district office leaders
to ensure proper support and/or recognition of teachers and sites in their implementation of
MTSS is in their control. To assess the motivational factor of attributions, site administrators
were asked interview questions and a document analysis was completed to review artifacts for
evidence. Table 6 provides an overview of the methods that will be used, along with the sample
interview items and documents that were analyzed.
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 65
Table 6
Summary of Motivation Influences and Method of Assessment
Assumed Motivation Influences Document
Item
Interview
Item
Value
Stakeholder values…
Site administrators value defining observable
and measurable organizational goals and
targeted results for student success, creating
processes that reinforce, monitor, encourage,
and reward teacher critical behaviors, and
using processes to partner with district office
leaders to ensure proper support and/or
recognition of teachers and sites in their
implementation of MTSS.
Do you find value
in integrating
academic,
behavioral, and
social emotional
support into one
comprehensive
intervention
model?
How valuable is it
to partner with
district office
leaders when
implementing
MTSS?
Self-Efficacy
Stakeholder is confident that….
Site administrators are confident about
defining observable and measurable
organizational goals and targeted results for
student success, creating processes that
reinforce, monitor, encourage, and reward
teacher critical behaviors, and using
processes to partner with district office
leaders to ensure proper support and/or
recognition of teachers and sites in their
implementation of MTSS.
How confident are
you in your
abilities to monitor
and support
teachers as they
implement MTSS
strategies?
How confident are
you in your
abilities to partner
with district office
leaders to
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 66
implement MTSS
at your site?
Emotions
Stakeholder feels positive about…
Site administrators feel positive about
defining observable and measurable
organizational goals and targeted results for
student success, creating processes that
reinforce, monitor, encourage, and reward
teacher critical behaviors, and using
processes to partner with district office
leaders to ensure proper support and/or
recognition of teachers and sites in their
implementation of MTSS.
How positive do
you feel about
implementing
MTSS at your
school?
How positive do
you feel about
partnering with
district office
leaders to
implement MTSS
at your site?
Attributions
Stakeholders believes that ....
Site administrators believe that the success
or failure of defining observable and
measurable organizational goals and targeted
results for student success, creating
processes that reinforce, monitor, encourage,
and reward teacher critical behaviors, and
using processes to partner with district office
leaders to ensure proper support and/or
recognition of teachers and sites in their
implementation of MTSS is in their control.
In what ways, if
any, do you think
teacher success in
implementing
effective MTSS
strategies will be
in your control?
Organization Assessment
The third cause of performance gaps can be attributed to organizational factors such as
culture, structure, resources, policies, and practices (Clark & Estes, 2008; Rueda, 2011). The
literature review in Chapter Two helped establish a list of assumed organizational influences that
affect site administrators as they implement an MTSS program at their school. This list is
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 67
presented in Table 7 along with assessment methods that were used for collecting data specific to
the organizational areas of resources, policies and procedures, and culture.
Resources. Site administrators need resources such as time, money, and people to
perform the critical behaviors associated with implementing MTSS at their site. To determine if
the assumed organizational influence of resources is an actual need, site administrators will be
asked interview questions and a document analysis will be completed to review artifacts for
evidence. Table 7 provides an overview of the methods that will be used, along with the sample
interview items and documents that will be analyzed.
Policies, process, and procedures. This needs assessment investigated whether site
administrators perceive their current policies and procedures are in alignment with the goal of
implementing an MTSS program at their school. To assess this, site administrators were asked
interview questions and a document analysis was completed to review artifacts for evidence.
Table 7 provides an overview of the methods that were used, along with the sample interview
items and documents that were analyzed.
Culture. It is important to assess how the culture of an organization (e.g. values, beliefs,
and attitudes) influences site administrators as they implement an MTSS program. Therefore,
open-ended interview questions were utilized to seek site administrator perceptions regarding the
systems in place to motivate them and the cooperation level among teachers, themselves, and
district leadership. Table 7 provides an overview of the methods that were used, along with the
sample interview items and documents that were analyzed.
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 68
Table 7
Summary of Organizational Influences and Method of Assessment
Assumed Organization Influences Document
Item
Interview
Item
Resources (time; finances; people)
Stakeholder have the resources (time, money,
people) to …
Site administrators have the funding to be trained
on implementing an MTSS program, and to train,
support, and recognize teachers in the MTSS
implementation process.
Ideally, what
resources would
you need to
implement MTSS
effectively?
Tell me about the
financial challenges
you may face
implementing
MTSS at your site.
Site administrators have the time to plan and
implement an MTSS program and meet with
district office personnel.
With everything
else going on at
your school, how
are you going to
make the time to
plan and implement
MTSS?
Describe any
adjustments to
people’s roles and
responsibilities you
see necessary to
implement MTSS
Describe any
structural changes
to the school
schedule you see
necessary to
implement MTSS.
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 69
Policies, Processes, & Procedures
Stakeholder has policies that align with …
Policies and procedures are in place for site
administrators to create processes that reinforce,
monitor, encourage, and reward teacher critical
behaviors and are aligned with district policies.
2018 WHS
Transition
Guide
Emails to
Faculty and
Staff
What policies or
procedures would
you see having to
be changed or
created to
implement MTSS?
Policies and procedures that are in place for site
administrators to partner with district office
leaders to ensure proper support and/or
recognition of teachers and sites in their
implementation of MTSS (RTI
2
and PBIS
strategies) align with district policies.
Systematic
Process of
Intervention
Task Force
Agendas
*See above.
Culture
Stakeholder feels like they are part of an
organization that…
The school has systems in place to motivate site
administrators to define observable and
measurable organizational goals and targeted
results for student success, create processes that
reinforce, monitor, encourage, and reward teacher
critical behaviors, and use processes to partner
with district office leaders to ensure proper
support and/or recognition of teachers and sites in
their implementation of MTSS (RTI
2
and PBIS
strategies).
Tell me how the
culture of your
school is going to
influence the
implementation of
MTSS at your site.
There is a culture of cooperation among site
administrators who define observable and
measurable organizational goals and targeted
results for student success, create processes that
reinforce, monitor, encourage, and reward teacher
critical behaviors, and use processes to partner
with district office leaders to ensure proper
support and/or recognition of teachers and sites in
their implementation of MTSS (RTI
2
and PBIS
strategies).
Describe how the
culture of
cooperation among
teachers, site
administrators, and
district office
leaders will impact
the implementation
of MTSS.
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 70
Participating Stakeholders and Sample Selection
The stakeholder of focus for this study was site administrators at Westside High School
in the Downtown Unified School District.
Sampling
The stakeholder group of focus for this study was site administrators at Westside High
School (WHS). There are six site administrators at WHS including the principal and five
assistant principals. Every site administrator at WHS participated in this study to maximize the
data that was collected. All six site administrators were interviewed as part of the process.
Lastly, each site administrator was de-identified in the study as a numbered respondent from Site
Administrator 1 to Site Administrator 6.
Recruitment
Site administrators at WHS volunteered to participate in this collaborative study. Site
administrators agreed to be interviewed for this study to help gain insight into the knowledge,
motivation, and organizational needs for implementing an MTSS program at their school site.
Instrumentation
The instruments that were used for this study were interviews and document analysis.
These data collection instruments are described below.
Survey Design
No surveys were used due to the small sample size.
Interview Protocol Design
The interview questions that were developed and implemented in this research study
followed Maxwell’s (2013) belief that creativity and insight guide protocols, rather than a direct
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 71
conversion of your research questions into interview form. A semi structured interview protocol
was utilized in this study, which creates a framework for asking predetermined questions, but
having the flexibility to reword the questions or ask follow-up questions if necessary (Merriam &
Tisdell, 2016).
The interview protocol consisted of 26 open-ended questions to determine whether
assumed KMO influences were assets or actual needs. The questions were also designed to
allow for opportunities for other influences to emerge. There were 18 primary questions asked to
all participants and 8 secondary questions that were asked if time permitted. See Appendix A for
the Interview Protocol and Questions used in this study.
Observation Checklist Design
No observations were used in this study.
Document Analysis Design
After conducting interviews, it was necessary to conduct a document analysis to
triangulate the data collected from the interview process. Documents such as professional
development agendas, district and site intervention meeting minutes and agendas, school
transition guides, and WASC reports were utilized to further assess KMO influences.
Data Collection
Following University of Southern California Institutional Review Board (IRB) approval,
participants were solicited verbally for interviews. All communications regarding interviews and
the document review process came directly from the investigator or the co-investigator (Bhakta,
2019).
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 72
Surveys
No surveys for the site administrator portion of this study.
Interviews
Five of the six interviews were conducted over the course of two days at Westside High
School. The sixth interview was conducted over the phone the following week. The five in-
person interviews were conducted in a quiet setting at the participant’s school campus, which
created a sense of comfort. All interviews began with the investigator following the official
interview protocol, which included describing the purpose of the study and gaining consent from
the participant to record the interview. Each interview lasted approximately 30-40 minutes in
length. During the interview, participants were asked questions from the interview protocol (see
Appendix A). Upon the completion of the interviews, participants were thanked. Interviews
were then transcribed and kept on a password secured computer.
Observations
No observations for this study.
Document Analysis
An analysis was conducted to review artifacts, documents, meeting agendas, and other
useful resources that were mentioned by interview respondents. During each interview, the
investigator requested documents from participants to verify responses. Some of these items
were provided on the spot and others were emailed to the investigator later. These documents
provided additional evidence for the KMO needs assessment.
Data Analysis
For interviews, the first step in the data analysis process was to read the transcripts from
the six interviews and compare them with the handwritten notes taken during the interview.
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 73
During this review process, data from the transcripts was coded using symbols that represent the
categories of knowledge and skills, motivation, and organization. Coding the KMO categories
allowed the researcher to capture relevant information for analysis in Chapter Four of this study
(Merriam & Tisdell, 2016). The coding process brought to light a series of patterns and
relationships in the data that were translated into themes and presented in the findings of this
report.
For document analysis, the process of open coding was utilized to go through the raw
data and develop categories based on what data seemed most relevant and important (Maxwell,
2013). Coding categories using an excel spreadsheet allowed the researcher to sort data based on
items that were in the descriptive data set (Maxwell, 2013). The results of the data collected and
analyzed are presented in the findings section, Chapter Four.
Trustworthiness of Data
Multiple precautions were taken to ensure the credibility and trustworthiness of this
study. First, the data was triangulated since multiple sources of data and data collection methods
were used (Merriam & Tisdell, 2016). Six interviews were conducted, and multiple documents
were analyzed. These interviews and the document review process were a result of adequate
engagement in the field, which led to possible data saturation (Merriam & Tisdell, 2016).
Throughout the process, the researchers design, and methods went through a peer review with
fellow classmates (Merriam & Tisdell, 2016). Additionally, interviews were professionally
transcribed to ensure validation and accuracy of data collection (Maxwell, 2013). Lastly, the
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 74
nature of this study’s process led to an audit trail, or detailed account of the methods and
procedures used to carry out the study (Merriam & Tisdell, 2016).
Role of Investigator
The investigator in this study is not an employee of the Downtown Unified School
District. The investigator worked in partnership on this study with Bhavini Bhakta (2019). Both
investigators shared the responsibilities of picking the sample for the study, recruiting
participants, collecting consent forms, administering interviews, distributing surveys, and
conducting document analysis. The investigator obtained permission from the IRB at the
University of Southern California in order to obtain data and conduct the study. The investigator
responded to all requests for information or materials solicited by the IRB and conducted the
study in strict accordance with IRB-approved research protocol. The investigator will also report
the findings and discussion in Chapter Four and Chapter Five.
Limitations
The limitations of this study are that only one high school in the Downtown Unified
School District was able to participate. Increasing the number of schools participating may have
yielded more valuable results for the District Office. Additionally, some of the site
administrators in this study were promoted to principal positions in other districts shortly after
participating. Therefore, the new members of the site administrative team may have different
KMO needs that were not addressed in this study. Lastly, this gap analysis study would have
benefited from more time to investigate the KMO needs of all three stakeholders. Bhakta (2019)
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 75
studied teachers, but neither of the co-investigators in this study were able to interview district
office leaders.
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 76
CHAPTER FOUR: RESULTS AND FINDINGS
Authored by Matthew Chambers
Findings from data collection will be reported in this chapter as they relate to the
knowledge, motivation, and organizational (KMO) influences that impact Westside High School
(WHS) site administrators as they implement MTSS at their school. The gap analysis approach
(Clark & Estes, 2008: Rueda, 2011) will guide the analysis of challenges and barriers in the areas
of knowledge and skills, motivation, and organizational culture. This approach will lead to the
identification and implementation of solutions that WHS site administrators can use to
effectively implement MTSS as a systematic school-wide program.
The following data collection process was utilized to determine the KMO needs of site
administrators seeking to implement MTSS at their schools. First, a total of twenty assumed
influences were developed from the literature review in Chapter Two and categorized into the
areas of knowledge, motivation, and organization. Multiple sources of qualitative data from
interviews and documents were then collected to assess the validity of the assumed knowledge,
motivation, and organizational factors. Following the interviews, documents were collected
from the site to verify statements made during the interviews and provide further evidence of
current on-site practices and results.
Participating Stakeholders
The stakeholder group that participated in this study were site administrators at Westside
High School (WHS). All six site administrators at WHS, including the principal and five
assistant principals were interviewed for this study resulting in 100% participation for this
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 77
stakeholder group. There was no individual demographic info collected for each participant (e.g.
age, race, years of experience, etc.).
Data Validation
This study utilized two sources of data including interviews and document analysis.
Interviews were the primary source of data used to assess most assumed KMO influences. All six
30-40 minute interviews were transcribed upon completion and coded to determine which
influences were addressed. The criteria for determining the validity of assumed influences was as
follows: Assumed influences were validated as an asset for knowledge when at least five out of
six (83.3%) site administrators gave responses confirming the correct influence. For motivation
and organization, the influence was validated as an asset if five out of six (83.3%) site
administrators agreed with the assumed influence. For example, if at least five out of six
(83.3%) site administrators found value in an intervention model that combines academic,
behavioral, and social emotional support, then that assumed motivation influence was validated
as an asset. Likewise, if at least five out of six (83.3%) site administrators agreed that there is a
culture of cooperation amongst teachers, site administrators, and district office leaders, then that
assumed organizational influence was validated as an asset. If less than five out of six (83.3%)
site administrators provided answers that put an assumed influence in doubt, then that KMO
influence was not validated, and therefore, determined to be a need. Documents were analyzed to
either verify statements made during interviews or to validate assumed influences that were not
addressed during the interview process.
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 78
Results and Findings for Knowledge Causes
Site administrator knowledge causes were assessed through interviews and document
analysis. Results are presented in the following section for each assumed influence in the areas
of factual knowledge, conceptual knowledge, procedural knowledge, and metacognitive
knowledge. Each assumed influence was determined to be an asset (validated) or determined to
be a need (not validated) based on the findings.
Factual Knowledge
Influence 1 - Site administrators know current student performance data and what
observable and measurable goals are.
Survey results. Surveys were not conducted for this influence.
Interview findings. Interviews were not conducted for this influence.
Observation. Observations were not conducted for this influence.
Document analysis. The 2015 Western Association of Schools and Colleges (WASC)
Self-Study Report for Westside High School contains multiple examples of student performance
data. For example, the report includes a myriad of data ranging from English scores on state
exams (72% of 9th graders testing proficient or above) to the Average Daily Attendance Rate
(97.26%). The WASC Self-Study Report also includes a Schoolwide Action Plan, which
consists of numerous examples of observable and measurable goals created by site
administrators. One such goal is to increase the percent of 9th grade general education students
meeting at least 3 “a-g” requirements by the end of their freshman year by 5% each year.
Another example of a measurable goal found in the report would be to increase the percent of
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 79
students meeting college-readiness on the SAT tests by scoring a minimum of 500 in Reading to
50% over six years, 550 in math to 40%, and 560 in writing to 25% over six years.
Summary. The assumed influence that site administrators know current student
performance data and know the observable and measurable goals was validated as an asset.
There was an adequate amount of documented evidence in the WASC Self-Study Report to
affirm that site administrators work with current performance data to create measurable goals.
Influence 2 - Site administrators know teacher roles in the RTI
2
and PBIS process.
Survey results. Surveys were not conducted for this influence.
Interview findings. No site administrators were able to accurately describe all three of
the main teacher roles in the RTI
2
and PBIS (MTSS) process. As shown in Table 8 below, more
than half (4 out of 6) of the respondents were able to describe some components of the teacher
role such as identifying students who need academic, behavioral, or social emotional
interventions. For instance, Site Administrator #3 gave a hospital analogy to describe their role,
“Teachers are like intake at the emergency room. They're triaging patients. They notice when
something is not right with students.” Additionally, two out of six site administrators also
mentioned that one of the teacher roles in the MTSS process is to implement RTI
2
and PBIS
strategies in their classroom. For example, Site Administrator #4 said that teachers “identify and
report things getting in the way of students making progress, provide supports within their
classroom, implement PBIS strategies, deliver academic interventions, and provide feedback to
administrators and counselors.” However, not one respondent mentioned that a teacher’s role in
the MTSS process includes working in teams to analyze data, create goals, and develop and refer
students to Tier 2 and Tier 3 interventions.
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 80
Observation. Observations were not conducted for this influence.
Document analysis. Document analysis was not conducted for this influence.
Summary. The assumed influence that site administrators know teacher roles in the
MTSS process was not validated; therefore, it was determined to be a need. No site
administrators interviewed for this study stated that the teacher role in the MTSS process
includes being a part of integrated teaming, which requires teacher participation in analyzing
data, developing goals, reviewing progress, and planning Tier 2 and Tier 3 interventions.
Table 8
Interview Results for Factual Knowledge of Teacher Roles in the RTI
2
and PBIS process.
Teacher Role in the RTI
2
and PBIS Process % of Site Administrators who correctly
identified the teacher role
Identify students as recipients of academic,
behavioral, or social emotional support.
66.7%
Implement RTI
2
and PBIS strategies in the
classroom to support students.
33.3%
Follow established protocols to work in teams
to analyze data, create goals, and develop and
refer students to Tier 2 and Tier 3
interventions.
0%
Influence 3 - Site administrators know district office leader roles in the MTSS
process.
Survey results. Surveys were not conducted for this influence.
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 81
Interview findings. One out of six (16.7%) of site administrators were able to accurately
describe district office leader roles in the MTSS process. Site Administrator #4 gave a thorough
response that corresponded with the literature on this topic, “[district office leaders] make sure
that sites have the support that they need by facilitating professional development. Additionally,
they make sure that there is something built in the funding that can support the implementation
of the support system. They deal with the unions, if it requires. They are supportive and make
sure that parents and the community are informed that the district supports those structures and
interventions. They support what the site needs by talking to site administrators and teachers
about the work and adjusting as needed.”
However, most site administrators (66.7%) interviewed responded that district office
leader roles should make sure that MTSS is being implemented district wide and not just at their
school site. For example, Site Administrator #1 stated that “district leaders role is to ensure
consistency amongst their schools”, Site Administrator #2 responded that “the system should be
rolled out district wide and supported district wide with financial resources, staffing, etc.”, and
Site Administrator #5 believes that district office leaders should make sure it is district wide by
“setting minimums for the program, but allow for flexibility to be customized for every site.”
Observation. Observations were not conducted for this influence.
Document analysis. Document analysis was not conducted for this influence.
Summary. The assumed influence that site administrators know district office leader
roles in the MTSS process was not validated; therefore, it was determined to be a need. The
findings show that two thirds of site administrators interviewed mistakenly think that the role of
district office leaders is to ensure implementation district wide. However, the literature describes
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 82
district office leader roles being more focused on creating teams to support the management,
implementation, and training needs of the program.
Conceptual Knowledge
Influence 4 - Site administrators know how to classify students as either on track or
off track of meeting organizational goals and targeted results for student success.
Survey results. Surveys were not conducted for this influence.
Interview findings. Interviews were not conducted for this influence.
Observation. Observations were not conducted for this influence.
Document analysis. The 2015 WASC Self-Study Report for Westside High School
contains multiple examples of site administrators classifying students as either on track or off
track of meeting organizational goals or targeted results for student success. For example, the
report includes a description of how 9th grade students who are more than 10 credits behind due
to failing classes are considered off track, while all other students are considered on track.
Another example in the report of site administrators classifying students as either on track or off
track for a goal is with the A-G completion rates.
Summary. The assumed influence that site administrators know how to classify students
as either on track or off track of meeting organizational goals and targeted results for student
success was validated as an asset. There was an adequate amount of evidence in the WASC
Self-Study Report to demonstrate that site administrators monitor student progress with
quantifiable data to determine whether students are on track to meet performance goals.
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 83
Influence 5 - Site administrators know how to categorize teacher behaviors as either
appropriate RTI
2
and PBIS strategies or not appropriate RTI
2
and PBIS strategies.
Survey results. Surveys were not conducted for this influence.
Interview findings. When asked to give an example of an appropriate MTSS strategy,
100% of site administrators interviewed were able to give accurate responses. Every site
administrator interviewed was able to identify either an appropriate RTI
2
or PBIS strategy.
Five out of six administrators mentioned PBIS strategies in their response. Site
Administrator #4 gave an example of a Tier 3 strategy in which the “teacher utilizes a behavior
support plan for a student who has trouble staying focused in class. The plan is developed with
the help of the counselor and might include strategies such as allowing the student to step out for
a minute, go grab a drink of water, and then come back to class. Or maybe they can have a
fidget tool.”
Three out of six site administrators mentioned RTI
2
strategies in their response. Site
Administrator #2 gave an example of an RTI Tier 1 strategy that might be seen in a math class,
“there would be time set aside for me to re-teach the concepts that maybe a portion of the
students didn't comprehend, but also it could be enrichment for those students that are ahead of
the game. It's really differentiation within the classroom to meet the needs of every student."
Although most examples given by site administrators could be classified as either an
academic or behavioral strategy, Site Administrator #6 shared an example that blended the two,
“A math teacher notices a student not doing well, so they put the student in a group with higher
learners for extra help. If the student is still not doing well, they have the student come in to go
over test results, they help the student and then gives them a chance to retest. If after they’ve
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 84
done all these strategies; they’ve retaught, they’ve retested, they might refer the student to their
counselor for a social-emotional check-in because they realize their heart is not there.”
Observation. Observations were not conducted for this influence.
Document analysis. Document analysis was not conducted for this influence.
Summary. The assumed influence that site administrators know how to categorize
teacher behaviors as either appropriate or inappropriate RTI
2
and PBIS strategies was validated
as an asset. The findings show that every site administrator interviewed was able to give at least
one examples of an appropriate RTI
2
and PBIS strategy.
Influence 6 - Site administrators know full MTSS implementation.
Survey results. Surveys were not conducted for this influence.
Interview findings. Two of six (33.3%) site administrators interviewed accurately
described many components that makeup a Multi-tiered System of Support (MTSS) in full
implementation. For instance, Site Administrator #2 described MTSS as having “different levels
of support from low levels to individual supports that have to do with academic support, as well
as social emotional supports for the whole child. There's supposed to be an entry and exit criteria
that's clearly defined and that's based on the data, students are placed in programs that meet their
needs." Site Administrator #4 described similar components, but also stated that MTSS is "a
systematic way for identifying the support for students and having some sort of response, activity
or intervention that will address those particular needs whether academic or social/emotional.”
However, the remaining four (66.7%) site administrators interviewed were unable to identify
specific components of MTSS. Site Administrators #3, 5, and 6 responded that MTSS is a model
that supports academics and social emotional support, but none of them described how the model
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 85
functioned. For example, Site Administrator #3 stated, “"It's a coherent system across the school
or district that's there to support our students academically, socially, emotionally.”
Observation. Observations were not conducted for this influence.
Document analysis. Document analysis was not conducted for this influence.
Summary. The assumed influence that site administrators know full MTSS
implementation was not validated; therefore, it was determined to be a need. The findings show
that site administrators were unable to describe key components that make up the MTSS model
such as it being a blend of RTI and PBIS, there being tiers of service, or that there are teams
analyzing data to guide decisions about student support. Lastly, not one site administrator was
able to describe what the stages of implementation look like according to the literature.
Procedural Knowledge
Influence 7 - Site administrators know steps to develop organizational goals and
targeted results for student success and how to use data to monitor their progress.
Survey results. Surveys were not conducted for this influence.
Interview findings. Interviews were not conducted for this influence.
Observation. Observations were not conducted for this influence.
Document analysis. The 2015 Western Association of Schools and Colleges (WASC)
Self-Study Report for Westside High School contains multiple examples of organizational goals
developed by WHS site administrators. For example, the document includes a goal to increase
the percent of students who complete the “a-g” requirements by an average of 5% every year
over two years. This goal also includes a plan to re-evaluate and set new targets by the third year.
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 86
The WASC document also includes a goal to increase the percent of students who pass AP
exams to more than 50% of exams administered, while maintaining or increasing the number of
exams administered and a goal to increase enrollment in Career Tech Ed classes by 1% each
year.
Summary. The assumed influence that site administrators know steps to develop
organizational goals and how to use data to monitor their progress was validated as an asset.
There was an adequate amount of documented evidence in the WASC Self-Study Report to
affirm that site administrators develop organizational goals and monitor student progress based
on those goals.
Influence 8 - Site administrators know how to observe and identify effective RTI
2
and PBIS strategies.
Survey results. Surveys were not conducted for this influence.
Interview findings. When asked to give an example of an appropriate MTSS strategy,
every site administrator (100%) interviewed was able to identify either an appropriate RTI
2
or
PBIS strategy. Five out of six site administrators (83.3%) identified effective PBIS strategies.
However, only 50% of respondents mentioned specific RTI
2
strategies, so it is unclear whether
the other 50% of site administrators know how to identify them. Additionally, only 50% of site
administrators mentioned observing teachers in the classroom at some point during their
interview.
Observation. Observations were not conducted for this influence.
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 87
Document analysis. The PBIS Pyramid of Interventions found in the 2018 WHS
Transition Guide is evidence that the site administrators at WHS have a guide/handbook to help
them identify effective PBIS strategies.
Summary. The assumed influence that site administrators know how to observe and
identify RTI
2
and PBIS strategies was not validated; therefore, it was determined to be a need.
There was not enough evidence to prove that at least five out of six site administrators at WHS
are able to identify effective RTI
2
strategies.
Influence 9 - Site administrators develop a system for reinforcing, monitoring,
encouraging and rewarding teachers for implementing components of MTSS, while
providing supports for teachers who are not.
Survey results. Surveys were not conducted for this influence.
Interview findings. When asked to tell me how you would reinforce, encourage, and
reward teachers implementing MTSS strategies, 100% of site administrators gave effective
strategies according to the literature. Multiple respondents gave public and private examples of
reinforcing and celebrating teacher behavior by providing thank you notes to teachers or giving
them ‘pats on the back.’ For example, Site Administrator #2 stated that it’s important to “let
teachers know what they're doing is highly effective and it's really important that we share their
strategies with other teachers.” Furthermore, Site Administrator #5 mentioned that you need to
“celebrate wins and give pats on the back. It doesn't necessarily have to be something monetary
where they're being compensated, but that thank you for your efforts, or for your work,
recognizing when they're collaborating with their colleagues." Site Administrator #1 shared an
example of celebrating teacher successes at faculty meetings, “recognizing teachers at a staff
meeting by saying ‘hey have you seen so-and-so's class? You should try stopping by." Lastly,
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 88
Site Administrator #4 recommended sharing and highlighting success stories with teachers,
“you're doing an amazing job with Jonny. He's now able to sit through the whole class without
having to leave the room and just acknowledge that, so they know that we're noticing some of
those things.”
When asked to describe the supports you would provide teachers who are not
implementing components of MTSS, 100% of site administrators gave effective strategies
according to the literature. Responses including sending teachers to conferences and trainings,
conducting instructional rounds, and connecting teachers with peer coaches. For instance, Site
Administrator #2 would “have teachers observe the classrooms of model teachers where the site
admin and teacher observe together to make sure they're seeing same things and expectations are
clear and then have teacher reflect on where they are and where they want to be.” Site
Administrator #4 would provide “specific training or professional development teachers in need
of support, connect them with their peers that are being successful, and provide period coverage
for teachers to observe other classrooms.”
Observation. Observations were not conducted for this influence.
Document analysis. Document analysis was not conducted for this influence.
Summary. The assumed influence that site administrators develop a system for
reinforcing, monitoring, encouraging, and rewarding teachers for implementing components of
MTSS, while providing supports for teachers who are not was validated; therefore, it was
determined to be a need. The findings show that although site administrator have yet to develop
a formal systematic approach to monitor, support, and reward teachers, they have the knowledge
to do it. Every site administrator interviewed demonstrated knowledge of strategies for affirming
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 89
and celebrating appropriate teacher behavior. Additionally, all site administrators responded
with research-based strategies for supporting struggling teachers.
Metacognitive Knowledge
Influence 10 - Site administrators reflect on the process of defining organizational
goals and targeted results for student success, reflect on the process of encouraging and
rewarding teacher critical behaviors, and reflect on full MTSS implementation to monitor
and adjust organizational goals.
Survey results. Surveys were not conducted for this influence.
Interview findings. When asked how they evaluate the effectiveness of their role when
implementing a new program, five out of six site administrators (83.3%) responded that they
would use student performance data to assess their effectiveness. Additionally, soliciting teacher
feedback through informal conversations or survey responses was mentioned by 83.3% of site
administrators as a way that they assess their individual performance. Site Administrator #6
summed up these sentiments in this very concise statement, "It's a lot of self-reflection, it's
looking at data, it's talking to teachers."
All the respondents stated that teacher feedback is the main factor that leads them to
change their practices. For example, Site Administrator #2 stated that "you need to really listen
to teachers. They know the kids, they're in the trenches." Furthermore, Site Administrator #1
noted that teacher feedback is often more timely and informative than performance data, which is
often a lagging indicator of whether a program is working or not, "Before you get the data, you
can get a good idea of what is going on just by talking to teachers, watching the classes, popping
in, and seeing what's going on."
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 90
Observation. Observations were not conducted for this influence.
Document analysis. Document analysis was not conducted for this influence.
Summary. The assumed influence that site administrators reflect on the essential
components of implementing a MTSS program at their school was validated as an asset. The
findings show that site administrators at WHS reflect on the effectiveness of their role when
implementing new programs and are willing to monitor and adjust organizational goals based on
a combination of teacher feedback, student performance data, and classroom observations.
Results and Findings for Motivation Causes
Site administrator motivation causes were assessed by utilizing interviews and document
analysis. Results are presented in the following section for each assumed influence in the areas
of value, self-efficacy, emotions, and attributions. Each assumed influence was determined to be
an asset (validated) or determined to be a need (not validated) based on the findings.
Value
Influence 11 - Site administrators value defining observable and measurable
organizational goals and targeted results for student success, creating processes that
reinforce, monitor, encourage, and reward teacher critical behaviors, and using processes
to partner with district office leaders to ensure proper support and/or recognition of
teachers and sites in their implementation of MTSS.
Survey results. Surveys were not conducted for this influence.
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 91
Interview findings. All six site administrators interviewed for this study affirmed that
they find value in integrating academic, behavioral, and social emotional support into one
comprehensive intervention model. Responses from each subject varied slightly, but all revolved
around the idea that students are human beings with emotional challenges that impact their
learning. For example, Site Administrator #3 responded that “Academic, behavior, and social
emotional aspects are all interconnected and don't operate in a vacuum. I mean this is a human
being." In addition, Site Administrator #6 stated that "In order for us to service our students, we
need to look at the whole child. We're not just brains, we're emotional human beings." Site
Administrator #1 stated that "If kids aren't in the right place with mental health, they won't
learn." Similarly, Site Administrator #2 emphasized that "If there's something blocking that
student emotionally, it doesn't matter how many academic interventions you have in place."
Observation. Observations were not conducted for this influence.
Document analysis. Document analysis was not conducted for this influence.
Summary. The assumed influence that site administrators find value in integrating an
MTSS intervention model that combines academic, behavioral, and social emotional support at
their school was validated as an asset. The findings show that site administrators at WHS
perceive academics to be interconnected with behavioral and social emotional challenges and
must be addressed collectively.
Self-Efficacy
Influence 12 - Site administrators are confident about defining observable and
measurable organizational goals and targeted results for student success, creating
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 92
processes that reinforce, monitor, encourage, and reward teacher critical behaviors, and
using processes to partner with district office leaders to ensure proper support and/or
recognition of teachers and sites in their implementation of MTSS.
Survey results. Surveys were not conducted for this influence.
Interview findings. Only one site administrator responded that they were confident in
their abilities to monitor and support teachers as they implement MTSS in their classrooms. The
other five site administrators admitted feeling less than confident. Some of the reasons that they
shared were not having “the amount of time needed to collaborate and work with teachers” and
needing to be “more familiar with my role” in the process.
When asked if they were confident in their abilities to partner with district office leaders
to implement MTSS at their site, 67% of site administrators responded that they were confident.
Although 4 out of 6 site administrators shared that they were confident they could partner with
the district office leaders, half of all respondents stated concerns that district office leaders were
too slow or unfocused to successfully support MTSS implementation. For example, Site
Administrator #2 said "I am confident that I can partner with them, but I worry about my
patience because the process is taking so long to roll out."
Observation. Observations were not conducted for this influence.
Document analysis. Document analysis was not conducted for this influence.
Summary. The assumed influence that site administrators are confident in their abilities
to implement MTSS at their site was not validated; therefore, it was determined to be a need.
Site administrator responses demonstrated an overall lack of confidence supporting teachers and
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 93
partnering with district office leaders, which are critical behaviors necessary to implement
MTSS.
Emotions
Influence 13 - Site administrators feel positive about defining observable and
measurable organizational goals and targeted results for student success, creating
processes that reinforce, monitor, encourage, and reward teacher critical behaviors, and
using processes to partner with district office leaders to ensure proper support and/or
recognition of teachers and sites in their implementation of MTSS.
Survey results. Surveys were not conducted for this influence.
Interview findings. Five out of six site administrators responded with either a negative
or neutral answer when asked how positive they feel implementing MTSS at their site. Of these
responses, the negative sentiments revolved around site administrators feeling frustrated about
past experiences working with district office leaders on this project. For example, Site
Administrator #3 stated, “District leadership has handcuffed us and said, ‘you can't do it, but
we're not going to do it either.’” Additionally, Site Administrator #6 said "I feel that it's going to
be awhile. I feel like we still have a few years before it's implemented. I don't feel that it's a
priority.”
Observation. Observations were not conducted for this influence.
Document analysis. Document analysis was not conducted for this influence.
Summary. The assumed influence that site administrators feel positive about
implementing MTSS at their site was not validated; therefore, it was determined to be a need.
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 94
The findings show that a majority (83%) of site administrators at WHS have negative emotions
associated with MTSS implementation including frustration, doubt, and impatience.
Attributions
Influence 14 - Site administrators believe that the success or failure of defining
observable and measurable organizational goals and targeted results for student success,
creating processes that reinforce, monitor, encourage, and reward teacher critical
behaviors, and using processes to partner with district office leaders to ensure proper
support and/or recognition of teachers and sites in their implementation of MTSS is in
their control.
Survey results. Surveys were not conducted for this influence.
Interview findings. When asked in what ways, if any, do you think teacher success in
implementing MTSS is in your control, 100% of site administrators affirmed the assumption that
success or failure is in their control. All respondents gave examples of how their actions might
impact the program’s success. For instance, Site Administrator #1 said that their role is “to
support those who are really excited about it, get other people to see it, show them how it works,
and you can build from there." Other responses emphasized site administrator actions such as
monitoring teachers, managing resources, and developing teacher efficacy.
Observation. Observations were not conducted for this influence.
Document analysis. Document analysis was not conducted for this influence.
Summary. The assumed influence that site administrators believe the success or failure
of implementing MTSS at their site is in their control was validated as an asset. The findings
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 95
show that every site administrator at WHS believes that they play in a significant role in whether
teachers implementing MTSS are successful.
Results and Findings for Organization Causes
Site administrator organization causes were assessed through interviews and document
analysis. Results are presented in the following section for each assumed influence in the areas
of resources, policies and procedures, cultural settings, and cultural models. Each assumed
influence was determined to be an asset (validated) or determined to be a need (not validated)
based on the findings.
Resources
Influence 15 - Site administrators have the funding to be trained on implementing
an MTSS program, and to train, support, and recognize teachers in the MTSS
implementation process.
Survey results. Surveys were not conducted for this influence.
Interview findings. Two out of six (33%) of site administrators believe that the district
and site have enough funding to support the implementation of MTSS. Site Administrator #5
indicated that there is enough money, but that it would “need to be reallocated” to support
additional staffing needs. Likewise, Site Administrator #6 stated that “funding is not an issue
because of low socioeconomic status (SES) numbers, so our LCAP funding is high and that
targets specifically at-risk students. We have funding that we can move from technology and
other areas." However, the 67% of site administrators who disagreed expressed concerns about
the cost of additional staffing and the investment in a training program for such a large staff.
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 96
Additionally, Site Administrator #3 cautioned that there is talk in the district “about cutting
positions to free up money and I think a lot of those positions would be necessary to accomplish
MTSS.”
Observation. Observations were not conducted for this influence.
Document analysis. Document analysis was not conducted for this influence.
Summary. The assumed influence that site administrators have funding to train and
support their staff in order to effectively implement MTSS was not validated; therefore, it was
determined to be a need. Westside High School has a faculty and staff of over 150 that will need
ongoing training and resources to successfully implement MTSS. There is not enough evidence
to prove that site administrators or the district have a financial plan to achieve this goal.
Influence 16 - Site administrators have the time to plan and implement an MTSS
program and meet with district office personnel.
Survey results. Surveys were not conducted for this influence.
Interview findings. When asked with everything else going on at your school, how are
you going to make time to plan and implement MTSS, 100% of site administrators responded
with various concerns and apprehension. For example, Site Administrator #6 said “we need to
put a lot more time into it than we're doing now for it to be successful.” Furthermore, Site
Administrator #3 added “there's not enough time to train the staff with so many initiatives.” Site
Administrator #4 felt the investment in time was so large that "at the initial start you need
additional personnel to be able to manage the implementation. Otherwise, something else is
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 97
going to give.” Lastly, Site Administrator #2 concisely summed up the time challenge by stating
"we're going to have to shift our priorities."
Observation. Observations were not conducted for this influence.
Document analysis. Document analysis was not conducted for this influence.
Summary. The assumed influence that site administrators have time to plan and
implement an MTSS program at their site was not validated; therefore, it was determined to be a
need. The findings show that every site administrator believes that something needs to change
(e.g. adjusting roles, shifting priorities, or adding staff) for them to have enough time to plan and
implement MTSS at their site.
Policies and Procedures
Influence 17 - Policies and procedures are in place for site administrators to create
processes that reinforce, monitor, encourage, and reward teacher critical behaviors and are
aligned with school and district policies.
Survey results. Surveys were not conducted for this influence.
Interview findings. When asked what policies or procedures you would need to be
changed or created to implement MTSS, site administrators gave varied responses that included
revising the bell schedule, renegotiating the teacher contract, refining the mental health referral
process, and more. Site Administrator #2 stated that “we need a different bell schedule, so that
would be a different policy that the union would have to vote on, and the district would have to
support.” Site Administrator #4 would like to rethink the way students are scheduled, “if
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 98
[students] don’t get a C, then they have to retake the class. In lieu of retaking the whole class,
they might just need a few concepts to get back on track.”
Observation. Observations were not conducted for this influence.
Document analysis. During the interview, Site Administrator #6 stated that “we need to
update our policies similar to the way PBIS shifted the way we discipline student.” The PBIS
Pyramid of Interventions and Behavior Expectations found in the 2018 WHS Transition Guide
are evidence that WHS has policies and procedures in place for PBIS. The document includes a
list of services that the school offers in Tier I (Universal), Tier II (Strategic), and Tier III
(Intensive). Additional evidence of the Character Counts program was found in emails and
faculty presentations.
Summary. The assumed influence that policies and procedures are in place for site
administrators to create processes that reinforce, monitor, encourage, and reward teacher critical
behaviors and are aligned with school and district policies was not validated; therefore, it was
determined to be a need. The findings showed that policies and procedures are in place to
support teachers with the implementation of the PBIS component of MTSS. However, there
were not any interview responses or documented evidence to prove that policies and procedures
are in place to support full MTSS implementation, which includes an RTI
2
component.
Influence 18 - Policies and procedures are in place for site administrators to partner
with district office leaders to ensure proper support and/or recognition of teachers and
sites in their implementation of MTSS (RTI
2
and PBIS strategies).
Survey results. Surveys were not conducted for this influence.
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 99
Interview findings. According to interview responses, the Systematic Process of
Intervention (SPI) Task Force was created to help ensure teachers and site administrators
received proper support when implementing MTSS. Site Administrator #2 described it this way,
“MTSS is a district wide initiative that they were calling "Systematic Process of Intervention
(SPI). We had a few meetings, but now it is on pause and we are waiting for the district.”
Observation. Observations were not conducted for this influence.
Document analysis. Agendas from the Systematic Process of Intervention (SPI) Task
Force were reviewed for this study. The SPI Task Force was a group of elementary, secondary,
and district level representatives that met during the 2015-16 school year to explore the
implementation of a systematic intervention program. Documents show the investigated training
needs for a three-tiered level of support, alternative structures for bell schedules, and universal
screeners. There is no evidence that the group met after the 2015-16 school year.
Summary. The assumed influence that policies and procedures are in place for site
administrators to partner with district office leaders to ensure proper support during MTSS
implementation was not validated; therefore, it was determined to be a need. Although a district
wide group (SPI Task Force) had been formed to achieve the goal of providing proper support
during implementation, there was no evidence that this group is currently active. The
implementation of an MTSS model requires organizational changes that often challenge the
current policies and procedures in place. The site administrators at WHS will need district level
policy makers on their team in order to be successful.
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 100
Cultural Settings
Influence 19 - The school has systems in place to motivate site administrators to
define observable and measurable organizational goals and targeted results for student
success, create processes that reinforce, monitor, encourage, and reward teacher critical
behaviors, and use processes to partner with district office leaders to ensure proper
support and/or recognition of teachers and sites in their implementation of MTSS (RTI
2
and PBIS strategies).
Survey results. Surveys were not conducted for this influence.
Interview findings. Site administrators were asked to tell me how the culture of your
school is going to influence the implementation of MTSS at your site. Sixty seven percent of
respondents gave an answer that indicated the culture would have a negative influence on
implementation. For example, Site Administrator #4 made several statements that summed up
the school cultural, "I think we still need to do a lot of work in changing the mindset of adults. A
pocket of teachers still prescribes to the idea that we have a pacing guide and kids have to learn it
at the same time and if they don't it's on them, not me. There's still a lack of ownership for the
success of kids." Site Administrator #2 added that “some of our staff are opposed to the new
graduation requirements. They will probably see this as one more thing they have to do, versus
it's what's good for all students." Lastly, Site Administrator #3 commented that "our teachers are
highly intellectual and highbrow about rigor, so any time you talk about taking time out of a
classroom, it's like touching the third rail." On the other hand, 33% of site administrators felt
that the culture would have a positive influence. According to Site Administrator #6, “we have
built a culture where our teachers are more open to trying things and more open to
administration. Before it was us versus them.”
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 101
Observation. Observations were not conducted for this influence.
Document analysis. Document analysis was not conducted for this influence.
Summary. The assumed influence that the school has systems and a culture in place to
motivate site administrators to implement MTSS was not validated; therefore, it was determined
to be a need. The findings show that most site administrators believe the culture will have a
negative influence on the implementation of MTSS.
Cultural Models
Influence 20 - There is a culture of cooperation among site administrators who
define observable and measurable organizational goals and targeted results for student
success, create processes that reinforce, monitor, encourage, and reward teacher critical
behaviors, and use processes to partner with district office leaders to ensure proper
support and/or recognition of teachers and sites in their implementation of MTSS (RTI
2
and PBIS strategies).
Survey results. Surveys were not conducted for this influence.
Interview findings. When asked to describe how the culture of cooperation among
teachers, site administrators, and district office leaders will impact the implementation of MTSS,
only one out of six (16.7%) site administrators gave a response that the culture of cooperation
would have a positive impact. Five out of six (83.3%) of site administrators gave answers that
described a culture of cooperation, or lack thereof, that would have a negative impact on MTSS
implementation. For instance, Site Administrator #5 stated that “it’s common to get two out of
three groups on the same page but getting all three together requires time and effort.”
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 102
Additionally, Site Administrator #6 shared that “district office leaders do not have a pulse on
what’s going on at the site and they don’t really listen to site principals. They don’t really
communicate well or listen to the sites needs and wants.” Lastly, Site Administrator #3
mentioned that “there’s always tension” between site administrators and teachers.
Observation. Observations were not conducted for this influence.
Document analysis. Document analysis was not conducted for this influence.
Summary. The assumed influence that the school has a culture of cooperation between
teachers, site administrators, and district office leaders to implement MTSS was not validated;
therefore, it was determined to be a need. The findings show that almost every site administrator
(5 out of 6) believes the culture of cooperation will have a negative influence on the
implementation of MTSS.
Summary of Validated Influences
Knowledge
As shown in Table 9, six out of ten assumed knowledge influences were validated as
assets through interviews and document analysis. Recommendations to improve the knowledge
influences that were determined to be needs will be discussed in Chapter Five.
Table 9
Summary of Assumed Knowledge Influences as Assets or Needs
Assumed Knowledge Influences Validated (Asset) or Not Validated (Need)
Declarative Factual
Site administrators know current student
performance data and what observable and
measurable goals are.
Validated (Asset)
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 103
Site administrators know teacher roles in the
RTI
2
and PBIS process.
Not Validated (Need)
Site administrators know district office leader
roles in the MTSS process.
Not Validated (Need)
Declarative Conceptual
Site administrators classify students as either
on track or off track of meeting organizational
goals and targeted results for student success.
Validated (Asset)
Site administrators know how to categorize
teacher behaviors as either appropriate RTI
2
and PBIS strategies or not appropriate RTI
2
and PBIS strategies.
Validated (Asset)
Site administrators know full MTSS
implementation.
Not Validated (Need)
Procedural
Site administrators know steps to develop
organizational goals and targeted results for
student success and how to use data to
monitor their progress.
Validated (Asset)
Site administrators observe and identify
effective RTI
2
and PBIS strategies.
Not Validated (Need)
Site administrators develop a system for
reinforcing, monitoring, encouraging and
rewarding teachers for implementing
components of MTSS, while providing
supports for teachers who are not.
Validated (Asset)
Metacognitive
Site administrators reflect on the process of
defining organizational goals and targeted
results for student success, reflect on the
process of encouraging and rewarding teacher
critical behaviors, and reflect on full MTSS
implementation to monitor and adjust
organizational goals.
Validated (Asset)
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 104
Motivation
As shown in Table 10, two out of four assumed motivation influences were validated as
assets through interviews and document analysis. Recommendations to improve the motivation
influences that were determined to be needs will be discussed in Chapter Five.
Table 10
Summary of Assumed Motivation Influences as Assets or Needs
Assumed Motivation Influences Validated (Asset) or Not Validated (Need)
Value
Site administrators value defining observable
and measurable organizational goals and
targeted results for student success, creating
processes that reinforce, monitor, encourage,
and reward teacher critical behaviors, and
using processes to partner with district office
leaders to ensure proper support and/or
recognition of teachers and sites in their
implementation of MTSS.
Validated (Asset)
Self-Efficacy
Site administrators are confident about
defining observable and measurable
organizational goals and targeted results for
student success, creating processes that
reinforce, monitor, encourage, and reward
teacher critical behaviors, and using processes
to partner with district office leaders to ensure
proper support and/or recognition of teachers
and sites in their implementation of MTSS.
Not Validated (Need)
Emotions
Site administrators feel positive about
defining observable and measurable
organizational goals and targeted results for
student success, creating processes that
reinforce, monitor, encourage, and reward
Not Validated (Need)
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 105
teacher critical behaviors, and using processes
to partner with district office leaders to ensure
proper support and/or recognition of teachers
and sites in their implementation of MTSS.
Attributions
Site administrators believe that the success or
failure of defining observable and measurable
organizational goals and targeted results for
student success, creating processes that
reinforce, monitor, encourage, and reward
teacher critical behaviors, and using processes
to partner with district office leaders to ensure
proper support and/or recognition of teachers
and sites in their implementation of MTSS is
in their control.
Validated (Asset)
Organization
As shown in Table 11, zero out of six assumed organization influences were validated as
assets through interviews and document analysis. Recommendations to improve the organization
influences that were determined to be needs will be discussed in Chapter Five.
Table 11
Summary of Organization Influences as Assets or Needs
Assumed Organization Influences Validated (Asset) or Not Validated (Need)
Resources
Site administrators have the funding to be
trained on implementing an MTSS program,
and to train, support, and recognize teachers
in the MTSS implementation process.
Not Validated (Need)
Site administrators have the time to plan and
implement an MTSS program and meet with
district office personnel.
Not Validated (Need)
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 106
Policies and Procedures
Policies and procedures are in place for site
administrators to create processes that
reinforce, monitor, encourage, and reward
teacher critical behaviors and are aligned with
school and district policies.
Not Validated (Need)
Policies and procedures are in place for site
administrators to partner with district office
leaders to ensure proper support and/or
recognition of teachers and sites in their
implementation of MTSS (RTI
2
and PBIS
strategies).
Not Validated (Need)
Cultural Settings
The school has systems in place to motivate
site administrators to define observable and
measurable organizational goals and targeted
results for student success, create processes
that reinforce, monitor, encourage, and reward
teacher critical behaviors, and use processes
to partner with district office leaders to ensure
proper support and/or recognition of teachers
and sites in their implementation of MTSS
(RTI
2
and PBIS strategies).
Not Validated (Need)
Cultural Models
There is a culture of cooperation among site
administrators who define observable and
measurable organizational goals and targeted
results for student success, create processes
that reinforce, monitor, encourage, and reward
teacher critical behaviors, and use processes
to partner with district office leaders to ensure
proper support and/or recognition of teachers
and sites in their implementation of MTSS
(RTI
2
and PBIS strategies).
Not Validated (Need)
Chapter Five will include recommendations for how to improve the assumed influences
that were not validated in the findings of this study. Proposed solutions and recommendations
will be shared with district office leaders and site administrators in Downtown Unified School
District.
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 107
CHAPTER FIVE: RECOMMENDATIONS AND EVALUATION
Authored by Matthew Chambers
Purpose of the Project and Questions
The purpose of this study was to examine the implementation of MTSS in the Downtown
Unified School District (DUSD) high schools from the perspective site administrators. More
specifically, the purpose of this project was to conduct a needs analysis in the areas of
knowledge and skill, motivation, and organizational factors necessary to reach the organizational
performance goal of effectively implementing MTSS at Westside High School by June 2020.
The analysis began by generating a list of possible needs and then followed with an examination
to determine whether they were actual needs or not. While a complete needs analysis would
focus on all stakeholders, for practical purposes the stakeholders of focus were the DUSD site
administrators.
Two questions guided this study.
1. What are the knowledge, and skills, motivation, and organizational needs for site
administrators to effectively implement MTSS at Westside High School?
2. What are the knowledge and skills, motivation, and organizational recommendations for
meeting the needs of site administrators in achieving the DUSD goal of effectively
implementing MTSS at Westside High School?
Recommendations to Address Knowledge, Motivation, and Organization Influences
The assumed knowledge, motivation, and organization (KMO) influences in the
following tables were determined to be actual needs during data collection. Each one of these
KMO influences has been given a high priority for achieving the organization's goal.
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 108
Additionally, an evidence-based principle has been identified to guide context-based
recommendations for improving performance in these areas. Following the tables, a detailed
discussion is provided for each influence including solutions based in the literature.
Knowledge Recommendations
Introduction. As shown in Table 12, four out of ten assumed knowledge influences
were determined to be actual needs during data collection. More specifically, two factual
knowledge influences, one conceptual knowledge influence, and one procedural knowledge
influence were determined to be needs. However, all metacognitive knowledge influences were
determined to be assets in which improvement is not needed. For the knowledge influences that
remain classified as needs, evidence-based principles have been identified to guide context-based
recommendations for improving performance in these areas.
Table 12
Summary of Knowledge Influences and Recommendations
Assumed Knowledge
Influence
Priority
High
Low
Principle and
Citation
Context-Specific
Recommendation
Factual
Site administrators know
teacher roles in the RTI
2
and PBIS process.
High How individuals
organize
knowledge
influences how
they learn and
apply what they
know (Schraw &
McCrudden,
2006).
Site administrators are
provided information that
organizes teacher roles in the
RTI
2
and PBIS process.
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 109
Site administrators know
district office leader
roles in the MTSS
process.
High How individuals
organize
knowledge
influences how
they learn and
apply what they
know (Schraw &
McCrudden,
2006).
Site administrators are
provided information that
organizes district office roles
in the MTSS process.
Conceptual
Site administrators know
full MTSS
implementation.
High Integrating
auditory and
visual
information
maximizes
working memory
capacity (Mayer,
2011).
Site administrators are
provided a visual diagram
that labels the different
stages of MTSS
implementation.
Procedural
Site administrators
observe and identify
effective RTI
2
and PBIS
strategies.
High Learning is
enhanced when
individuals are
provided
guidance,
modeling,
coaching, and
other scaffolding
during
performance
(Mayer, 2011).
Site administrators will
receive training from an
MTSS specialist/coach in
which they see a
demonstration and have
opportunities for practice and
feedback.
Metacognitive
There were no
deficiencies found in this
area of influence.
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 110
Factual knowledge solutions. Site administrators do not know teacher or district office
leader roles in the RTI
2
and PBIS (MTSS) process. Schraw & McCrudden (2006) found that
how individuals organize knowledge influences how they learn and apply what they know.
Based on this principle, learners are likely to benefit from developing visual organizers to
support their acquisition of new knowledge. Therefore, it is recommended that site
administrators are provided information that organizes teacher and district office leader roles in
the RTI
2
and PBIS (MTSS) process
Clark & Estes (2008) describe information as new knowledge that reduces people’s
uncertainty about how to achieve a performance goal. To retain new knowledge, Schraw &
McCrudden (2006) recommend providing experiences that help people make sense of the
material rather than just focus on memorization. One way that learners can organize new
knowledge is by creating their own concept map (Ambrose et al., 2010). Concept mapping is a
technique that helps people represent their knowledge organizations visually (Ambrose et al.,
2010). A concept mapping activity can be used to reveal a learner’s prior knowledge and monitor
knowledge acquisition (Ambrose et al., 2010).
Conceptual knowledge solutions. Site administrators do not know full MTSS
implementation. According to Mayer (2011), integrating auditory and visual information
maximizes working memory capacity. This suggests that providing a visual representation of
MTSS implementation would help site administrators in the learning process. Therefore, it is
recommended that site administrators are provided a visual diagram that labels the different
stages of MTSS implementation.
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 111
The basic premise of cognitive load theory is that humans have a limited cognitive
capacity in working memory, which impacts the volume of what we can learn (Kirschner,
Kirschner, & Paas, 2006). Providing visual representation helps learners organize new concepts
and information, so that they can retain it in working memory (Ambrose, 2010). Mayer (2011)
found that new information should be presented in manageable parts to reduce cognitive
overload. Additionally, Mayer’s (2011) research on the science of learning concluded that
frequent practice spread out over shorter learning sessions is more effective for learning than one
longer session.
Procedural knowledge solutions. Site administrators do not know how to observe and
identify effective RTI
2
and PBIS strategies. Mayer (2011) notes that learning is enhanced when
individuals are provided guidance, modeling, coaching, and other scaffolding during
performance. This suggests that learners need multiple opportunities in various settings to
acquire knowledge and practice implementing what they learned. Therefore, the recommendation
is for site administrators to receive training from an MTSS specialist/coach in which they see a
demonstration and have opportunities for practice and feedback.
Training is any situation where people can practice “how to” knowledge and skills and
receive corrective feedback in the process. (Clark & Estes, 2008). To develop mastery, Schraw &
McCrudden (2006), recommend that individuals must acquire component skills, practice
integrating them, and know when to apply what they have learned. The training program being
recommended will give learners opportunities to apply their knowledge and skills in diverse
contexts, which will help them overcome context dependence and prepare them better to transfer
their skills to multiple settings (Ambrose et al., 2010).
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 112
Metacognitive knowledge solutions. All metacognitive knowledge influences were
determined to be assets in which improvement is not needed.
Motivation Recommendations
Introduction. As shown in Table 13, two out of four assumed motivation influences
were determined to be actual needs during data collection. More specifically, the motivational
influences of self-efficacy and emotions were determined to be areas where improvement is
needed. However, the motivational influences of value and attributions were determined to be
assets in which improvement is not needed. For the motivational influences that remain
classified as needs, evidence-based principles have been identified to guide context-based
recommendations for improving performance in these areas.
Table 13
Summary of Motivation Influences and Recommendations
Assumed Motivation
Influence
Priority
High
Low
Principle and
Citation
Context-Specific
Recommendation
Value
There were no
deficiencies found in this
area of influence.
Self-Efficacy
Site administrators are
confident about defining
observable and
measurable
organizational goals and
targeted results for
High Achieving small
successes
increases one’s
confidence to
achieve more
Give site administrators
opportunities to practice
completing portions of the
full MTSS implementation
tasks, while providing them
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 113
student success, creating
processes that reinforce,
monitor, encourage, and
reward teacher critical
behaviors, and using
processes to partner with
district office leaders to
ensure proper support
and/or recognition of
teachers and sites in their
implementation of
MTSS.
difficult tasks
(Bandura, 1997).
Learning and
motivation are
enhanced when
learners have
positive
expectancies for
success (Eccles,
2006).
with feedback about their
performance.
Provide models of successful
administrators who have
achieved similar goals.
Emotions
Site administrators feel
positive about defining
observable and
measurable
organizational goals and
targeted results for
student success, creating
processes that reinforce,
monitor, encourage, and
reward teacher critical
behaviors, and using
processes to partner with
district office leaders to
ensure proper support
and/or recognition of
teachers and sites in their
implementation of
MTSS.
High Positive
emotions, such
as happiness and
joy, support
work
commitment
(Clark & Estes,
2008).
District office leaders
encourage site administrators
to share policies that they
would eliminate or modify to
increase their work
enjoyment.
Attributions
There were no
deficiencies found in this
area of influence.
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 114
Value solutions. All value influences were determined to be assets in which
improvement is not needed.
Self-Efficacy solutions. Site administrators are not confident that they can complete the
complex tasks necessary to implement a Multi-Tiered System of Support at their school.
Bandura (1997) found that achieving small successes increases one’s confidence to achieve more
difficult tasks (Bandura, 1997). In addition, Eccles’ (2006) study on expectancy-value theory
discovered that learning and motivation are enhanced when learners have positive expectancies
for success. These theories would suggest that providing site administrators with opportunities to
be successful and see others who have been successful would enhance their confidence.
Therefore, the recommendation is to give site administrators opportunities to practice completing
portions of the full MTSS implementation tasks, while providing them with feedback about their
performance. Furthermore, it is recommended that site administrators are provided models of
successful administrators who have achieved similar goals.
People are more likely to pursue goals when they have task-specific confidence
(Bandura, 1997; Clark & Estes, 2008). Parajes (2006) recommends setting close, concrete and
challenging goals that allow the learner to experience success at the task. Therefore, site
administrators will need to have positive experiences to increase their confidence and develop a
sense of self-efficacy.
It is especially critical that district leaders recognize and support site administrators as
they go through this process because district-level support is a predictor of site administrator
self-efficacy (Osterman & Sullivan, 1996; Tschannen-Moran & Gareis, 2007). In their study,
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 115
Osterman & Sullivan (1996), discovered that the most efficacious principals felt highly
supported by their superintendent and district office leaders. In addition, Tschannen-Moran &
Gareis (2007) studied the beliefs of 558 principals in Virginia and found that superintendent and
district office support were highly correlated with self-efficacy levels.
Emotions solutions. Site administrators do not feel positive about creating goals,
supporting teachers, and partnering with district office leaders to implement MTSS at their site.
This is significant because positive emotions, such as happiness and joy, support work
commitment (Clark & Estes, 2008). In contrast, an excess of negative emotions such as
unhappiness or anger in the workplace can reduce productivity. (Clark & Estes, 2008). A
recommendation to increase the mood of site administrators is to have district office leaders
encourage site administrators to share policies that they would eliminate or modify to increase
their work enjoyment.
It is not necessary for everyone to be happy in order to be committed, however, excessive
negative emotions lead individuals to focus on past negative events instead of future goals (Clark
& Estes, 2008). According to Bower (1995), eliminating rules and policies that reduce work
enjoyment without providing a measured benefit is one way to create a positive emotional
environment at work. Johnson (2005) found that many principals leave the role due to the
struggles with the bureaucracy of the position including district politics, union relationships, and
legislative changes. Therefore, providing site administrators with an opportunity to share their
perspective on policies with district leaders also has the benefit of making them feel supported in
an area that is often a major source of frustration.
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 116
Attributions solutions. All attributions influences were determined to be assets in which
improvement is not needed.
Organization Recommendations
Introduction. As shown in Table 14, six out of six assumed organization influences
were determined to be actual needs during data collection. More specifically, the organizational
influences of resources, policies and procedures, cultural settings, and cultural models are areas
of need in which improvement is necessary. For these influences that remain classified as needs,
evidence-based principles have been identified to guide context-based recommendations for
improving performance in these areas.
Table 14
Summary of Organization Influences and Recommendations
Assumed Organization
Influence
Priority
High
Low
Principle and
Citation
Context-Specific
Recommendation
Resources
Site administrators have
the funding to be trained
on implementing an
MTSS program, and to
train, support, and
recognize teachers in the
MTSS implementation
process.
High Effective change
efforts ensure
that everyone has
the resources
(equipment,
personnel, time,
etc.) needed to
do their job, and
that if there are
resource
shortages, then
resources are
Site administrators complete
a budget analysis where they
are forced to prioritize
expenditures by their
alignment with
organizational goals.
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 117
aligned with
organizational
priorities (Clark
and Estes, 2008).
Site administrators have
the time to plan and
implement an MTSS
program and meet with
district office personnel.
High Effective change
efforts ensure
that everyone has
the resources
(equipment,
personnel, time,
etc) needed to do
their job, and
that if there are
resource
shortages, then
resources are
aligned with
organizational
priorities (Clark
and Estes, 2008).
Site administrators attend a
time management training
(e.g. Breakthrough Coach).
Policies and Procedures
Policies and procedures
are in place for site
administrators to create
processes that reinforce,
monitor, encourage, and
reward teacher critical
behaviors and are aligned
with school and district
policies.
High Effective
organizations
insure that
organizational
messages,
rewards, policies
and procedures
that govern the
work of the
organization are
aligned with or
are supportive of
organizational
goals and values
(Clark and Estes,
2008).
A policy review committee
consisting of district office
leaders, site administrators,
and teachers will conduct a
comprehensive policy review
that will focus on the
implementation of MTSS.
Policies and procedures
are in place for site
administrators to partner
High Effective
organizations
insure that
A policy review committee
consisting of district office
leaders, site administrators,
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 118
with district office
leaders to ensure proper
support and/or
recognition of teachers
and sites in their
implementation of MTSS
(RTI
2
and PBIS
strategies).
organizational
messages,
rewards, policies
and procedures
that govern the
work of the
organization are
aligned with or
are supportive of
organizational
goals and values
(Clark and Estes,
2008).
and teachers will conduct a
comprehensive policy review
that will focus on the
implementation of MTSS.
Cultural Settings
The school has systems
in place to motivate site
administrators to define
observable and
measurable
organizational goals and
targeted results for
student success, create
processes that reinforce,
monitor, encourage, and
reward teacher critical
behaviors, and use
processes to partner with
district office leaders to
ensure proper support
and/or recognition of
teachers and sites in their
implementation of MTSS
(RTI
2
and PBIS
strategies).
High Organizational
performance
increases when
individuals
communicate
constantly and
candidly to those
involved about
plans and
process (Clark &
Estes, 2008).
Site administrators to partner
with district office leaders
and teachers to develop a
communication plan for
providing ongoing
information to all
stakeholders about the
implementation of MTSS.
Cultural Models
There is a culture of
cooperation among site
administrators who
define observable and
High Effective change
efforts insure
that all key
stakeholders’
Create an MTSS
implementation team that
includes district office
leaders, site administrators,
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 119
measurable
organizational goals and
targeted results for
student success, create
processes that reinforce,
monitor, encourage, and
reward teacher critical
behaviors, and use
processes to partner with
district office leaders to
ensure proper support
and/or recognition of
teachers and sites in their
implementation of MTSS
(RTI
2
and PBIS
strategies).
perspectives
inform the
design and
decision-making
process leading
to the change
(Clark & Estes,
2008).
and teachers to monitor the
change process by collecting
feedback and adjusting as
necessary.
Resources solutions. Site administrators do not have the funding to be trained on
implementing an MTSS program, and to train, support, and recognize teachers in the MTSS
implementation process. According to Clark & Estes (2008), effective change efforts ensure that
everyone has the resources (equipment, personnel, time, etc.) needed to do their job, and that if
there are resource shortages, then resources are aligned with organizational priorities. This
suggests that site administrators need to analyze their budget and prioritize funding for projects
that align with organizational goals. Hence, the recommendation is for site administrators
complete a budget analysis where they are forced to prioritize expenditures by their alignment
with organizational goals.
Site administrators and teachers need ongoing training and resources to ensure proper
MTSS implementation (Vekaria, 2017). Lancaster & Hougen (2017) found that the biggest
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 120
barrier to schools successfully implementing MTSS is that there is a lack of resources for teacher
training. Pan et. al’s (2003) study on resource allocation in education discovered that effective
resource allocation starts with the alignment of goals, priorities, and activities of education
decision makers. Additionally, the Pan et al. (2003) study recommended that school leaders take
steps to understand what resources they have available (e.g. money, time, staff) that can be used
more efficiently or pooled for greater effectiveness. Based on this evidence, conducting a budget
analysis would be a good first step for site administrators.
Site administrators do not have the time to plan and implement an MTSS program and
meet with district office personnel. According to Clark & Estes (2008), effective change efforts
ensure that everyone has the resources (equipment, personnel, time, etc.) needed to do their job,
and that if there are resource shortages, then resources are aligned with organizational priorities.
This suggests that site administrators must learn more effective ways to managing their time, so
that their energy is spent working to achieve organizational goals. Therefore, a recommendation
is for site administrators to attend a time management training such as Breakthrough Coach.
Site administrators from the state of Michigan who implemented MTSS reported that
their biggest challenges were related to time, scheduling, and resources at the site level (Vekaria,
2017). Clark & Estes (2008) suggest that changing organizational goals requires a review of the
way resources such as time are allocated. An examination of how site administrators can best
use their time is especially essential for them since they operate within significant time
constraints (Grissom, Loeb, & Masters, 2013).
Policies and procedures solutions. Policies and procedures are not in place for site
administrators to create processes to support teachers as they implement MTSS. Additionally,
policies and procedures are not in place for site administrators to partner with district office
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 121
leaders as they implement MTSS at their school. Effective organizations insure that
organizational messages, rewards, policies and procedures that govern the work of the
organization are aligned with or are supportive of organizational goals and values (Clark and
Estes, 2008). This suggests that district office leaders, site administrators, and teachers must
work collaboratively to review current policies and procedures to see if they align with the
organization’s goals. The recommendation is to have a policy review committee consisting of
district office leaders, site administrators, and teachers will conduct a comprehensive policy
review that will focus on the implementation of MTSS.
The implementation of an MTSS model requires organizational changes that often
challenge the current policies and procedures in place (McIntosh & Goodman, 2016). Clark &
Estes (2008) recommend that top management (e.g. district office leaders) should be continually
involved in the improvement process. Furthermore, Clark & Estes (2008) suggest that policies
and process should align with organizational goals to prevent performance gaps. Therefore, a
review of all current policies, procedures, and processes by all stakeholder groups is essential for
making improvements.
Cultural settings solutions. The school does not have systems in place to motivate site
administrators to perform the critical behaviors necessary to implement MTSS at their site.
According to Clark & Estes (2008), organizational performance increases when individuals
communicate constantly and candidly to those involved about plans and processes (Clark &
Estes, 2008). This suggests that site administrators must find ways to continuously get their
message to all stakeholders. Based on this principle, the recommendation is for site
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 122
administrators to partner with district office leaders and teachers to develop a communication
plan for providing ongoing information to all stakeholders about the implementation of MTSS.
Schools are complex social settings that are in a constant state of change (Rueda, 2011).
Because of these dynamics, the systems put in place are often a hindrance to achieving goals or
improving performance (Rueda, 2011). Clark & Estes (2008) found that clear and candid
communication cultivate a culture of trust that leads to commitment. Moreover, as trust
increases, employees are more likely to adapt to change goals by acquiring new knowledge and
skills necessary to increase performance (Clark & Estes, 2008). Based on this information,
communication is the key to building a culture of trust necessary for the change process.
Cultural model solutions. There is not a culture of cooperation for site administrators to
complete the critical behaviors necessary to implement MTSS at their school. Effective change
efforts insure that all key stakeholders’ perspectives inform the design and decision-making
process leading to the change (Clark & Estes, 2008). This principle would indicate that site
administrators, teachers, and district office leaders need to work collaboratively to design and
implement a successful MTSS program. Hence, the context-based recommendation is to create
an MTSS implementation team that includes district office leaders, site administrators, and
teachers to monitor the change process by collecting feedback and adjusting as necessary.
Developing a shared vision for the school and setting goals as a staff or team are ways
that a culture of cooperation is modeled (Marzano, et al, 2005). Clark & Estes (2008)
recommend conducting regularly scheduled meeting that include individuals from all areas of the
organization. These meetings should be designed to generate ways to receive feedback that
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 123
encourage participation from members of the organization that would not otherwise want to
share.
Summary of Knowledge, Motivation and Organization Recommendations
The following recommendations were guided by evidence-based principles and chosen to
address the outstanding knowledge and motivation needs identified in this study. It is
recommended that site administrators are provided information that organizes teacher and district
office leader roles in the RTI
2
and PBIS (MTSS) process. It is also recommended that site
administrators are provided a visual diagram that labels the different stages of MTSS
implementation and that site administrators receive training from an MTSS specialist/coach in
which they see a demonstration and have opportunities for practice and feedback. Another
recommendation is to give site administrators opportunities to practice completing portions of
the full MTSS implementation tasks, while providing them with feedback about their
performance. In addition, to provide them with models of successful administrators who have
achieved similar goals. Lastly, it is recommended that district office leaders encourage site
administrators to share policies that they would eliminate or modify to increase their work
enjoyment.
The following recommendations were chosen to address the outstanding organizational
needs identified in this study. It is recommended that site administrators complete a budget
analysis where they are forced to prioritize expenditures by their alignment with organizational
goals. It is also recommended that site administrators attend a time management training (e.g.
Breakthrough Coach). Additionally, a policy review committee consisting of district office
leaders, site administrators, and teachers will conduct a comprehensive policy review that will
focus on the implementation of MTSS has been recommended. It is also recommended that site
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 124
administrators partner with district office leaders and teachers to develop a communication plan
for providing ongoing information to all stakeholders about the implementation of MTSS. And
lastly, a recommendation is to create an MTSS implementation team that includes district office
leaders, site administrators, and teachers to monitor the change process by collecting feedback
and adjusting as necessary.
Integrated Implementation and Evaluation Plan
Organizational Purpose, Need and Expectations
The mission of the Downtown Unified School District is to provide every student with an
opportunity to graduate with a 21st Century education that ensures they are college and career
ready, globally competitive, and citizens of strong character. In the fall of 2017, the Downtown
Unified Board of Education voted to align their graduation requirements with the California A-G
requirements, so that all DUSD graduates are eligible to apply to a California State University
(CSU) or University of California (UC) school upon graduation. Achieving this goal will be a
challenge since the district is currently graduating around 42% of students CSU/UC eligible.
Therefore, this policy change will require a significant increase in academic and social-emotional
supports if the district wants to meet their new goal of graduating more student’s college ready.
The organizational goal to support the new graduation requirements is for DUSD’s
Westside High School (WHS) to effectively implement a Multi-Tiered System of Support
(MTSS) program to ensure that all ninth-grade students are on track to meet their A-G
requirements. To support this organizational goal, site administrators at WHS will implement
and monitor a systematic MTSS intervention program, so that all ninth-grade students are on
track to meet A-G graduation requirements by June 2020.
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 125
The desired outcome for this project is to improve organizational performance by
providing site administrators at WHS with the necessary knowledge and skills, motivation, and
organizational tools to successfully achieve their performance goal of implementing an MTSS
program to keep students on track to meet their A-G requirements.
Implementation and Evaluation Framework
The New World Kirkpatrick Model will be utilized to design an integrated
implementation and evaluation plan for the training program recommended in this study
(Kirkpatrick & Kirkpatrick, 2016). The goal of the model is to measure the effectiveness of a
training program, maximize transfer of learning to behavior and subsequent organization results,
and demonstrate value of training to the organization (Kirkpatrick & Kirkpatrick, 2016). The
New World Kirkpatrick Model is an evaluation framework consisting of four levels: Level 1:
Reaction, Level 2: Learning, Level 3: Behavior, and Level 4: Results (Kirkpatrick & Kirkpatrick
2016). Clark & Estes (2008) support the four-level model developed by Kirkpatrick and
recognize it as the best and almost universally used performance evaluation system since its
design in the 1950s.
The New World Kirkpatrick Model is an updated version of the old Kirkpatrick Model
and differs significantly in how instruction for training is designed and planned (Kirkpatrick &
Kirkpatrick, 2016). In the updated model, the four levels are planned in reverse order with Level
4: Results coming first and Level 1: Reaction coming last (Kirkpatrick & Kirkpatrick, 2016).
Planning from the beginning with an emphasis on what will be accomplished through improved
on-the-job performance of training graduates keeps the focus on what’s important (Kirkpatrick &
Kirkpatrick, 2016).
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 126
Level 4: Results and Leading Indicators
Kirkpatrick and Kirkpatrick (2016) describe Level 4: Results as the main reason why
trainings are performed, and without connecting the contributions of training to outcomes they
can become essentially meaningless. Leading indicators are measurable and short-term
personalized targets that suggest whether critical behaviors are on track to reach desired results
(Kirkpatrick & Kirkpatrick, 2016). As shown in Table 15 below, leading indicators (internal and
external) are listed below along with the metrics and methods for how they will be measured
and/or observed. These leading indicators will inform DUSD as to whether their site
administrators are reaching the goal of implementing MTSS at their sites.
Table 15
Outcomes, Metrics, and Methods for External and Internal Outcomes
Outcome Metric(s) Method(s)
External Outcomes
Increase in A-G
completion rate.
Number of graduates who met
their A-G requirements.
Reported annually to the state
and posted publicly on the
California School Dashboard.
Reduction in annual
suspension rate.
Number of suspensions for
the school year.
Reported annually to the state
and posted publicly on the
California School Dashboard.
Increase in positive
press regarding school
programs and
performance.
Number of positive articles
written in the press related to
the implementation of the
MTSS program.
Bi-annual search of online
records.
Increase in the number
of site visits from
external school
leaders/organizations to
observe MTSS program.
Number of documented site
visits from external school
leaders/organizations to
observe MTSS program in
action.
Annual report shared with
PTSA.
Internal Outcomes
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 127
Increase in the number
of 9th grade students
who are on track to meet
their A-G requirements.
Number of students who
complete their 9th grade year
on track to meet their A-G
requirements.
Annual review of 9th grade
transcripts by counselors and
site administrators.
Decrease in number of
students with at least
one “D” or “F” grade at
the semester.
Number of students with at
least one “D” or “F” grade at
the end of the semester.
Bi-annual report ran from
Student Information System
(SIS).
Reduction in office
referrals for behavior
issues.
Number of student referrals
sent by teachers to the office
for behavior issues.
Monthly review of report ran
by the Discipline Office from
the SIS.
Increase in the number
of students being
referred to counselor for
social-emotional
support.
Number of students being
referred by teachers to
counselors for social-
emotional support.
Monthly review of report ran
by the Counseling Office
from the SIS.
Increase in the number
of students receiving
Tier 2 academic support.
Number of students assigned
to a Tier 2 academic support
period.
Quarterly review of report ran
by the Intervention Office
from the SIS.
Increase the number of
times that teachers are
using MTSS strategies
in the classroom.
Number of times that teachers
are observed using MTSS
strategies by site
administrators on informal
walkthroughs.
Central tracking system for
informal walkthroughs
maintained by the principal’s
office.
Level 3: Behavior
Critical behaviors. Kirkpatrick and Kirkpatrick (2016) describe Level 3: Behaviors as
the degree to which participants apply what they learned during training when they are back on
the job. Critical behaviors are the few, specific actions, which, if performed consistently on the
job, will have the biggest impact on desired results and achieving organizational success
(Kirkpatrick and Kirkpatrick, 2016). As seen in Table 16, three critical behaviors were identified
that site administrators must be able to demonstrate in order to achieve their performance goals.
First, site administrators must define observable and measurable organizational goals and
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 128
targeted results for student success. Second, site administrators must create processes that
reinforce, monitor, encourage, and reward teacher critical behaviors. Lastly, site administrators
must use processes to partner with district office leaders to ensure proper support and/or
recognition of teachers and sites in their implementation of MTSS. The specific metrics,
methods, and timing for these critical behaviors can be found in Table 16 below.
Table 16
Critical Behaviors, Metrics, Methods, and Timing for Evaluation
Critical Behavior Metric(s) Method(s) Timing
Site administrators
define observable
and measurable
organizational goals
and targeted results
for student success.
Number of
organizational goals
developed to
measure the
effectiveness of the
MTSS program.
The site MTSS
implementation team
reviews the newly
developed
organizational goals.
Monthly status
report on
organizational
goals is reported
out by principal to
MTSS
implementation
team members.
Site administrators
create processes that
reinforce, monitor,
encourage, and
reward teacher
critical behaviors.
Number of processes
created to reinforce,
monitor, encourage,
and reward teachers
as they identify
students in need of
Tier 2 and 3 support,
implement MTSS
strategies, and
follow MTSS
protocols.
District office leaders
review the processes
created by the site
administrative team.
Monthly status
report reviewed at
admin team
meeting
classifying
teachers as either
“in need of
immediate
support”, “further
evidence needed”,
or “needs to be
recognized for a
job well done.”
Site administrators
use processes to
partner with district
office leaders to
Number of times
that site
administrators meet
with district office
Status of MTSS
implementation at
WHS is shared at
monthly school board
Monthly.
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 129
ensure proper
support and/or
recognition of
teachers and sites in
their implementation
of MTSS.
leaders to partner on
MTSS
implementation.
meeting by district
office leaders.
Required drivers. Required drivers are the processes and systems that reinforce,
monitor, encourage, and reward performance of critical behaviors (Kirkpatrick & Kirkpatrick,
2016). It is important for organizations to identify required drivers, so that trainees are provided
necessary support and held accountable for expectations after training is conducted (Kirkpatrick
& Kirkpatrick, 2016). Table 17 lists the recommended drivers that support critical behaviors of
site administrators as they seek to build their capacity as they lead the implementation of an
MTSS program at their school.
Table 17
Required Drivers to Support Critical Behaviors
Method(s) Timing
Critical Behaviors
Supported
1, 2, 3 Etc.
Reinforcing
Provide a refresher training
that organizes teacher and
district office leader roles in
the RTI
2
and PBIS (MTSS)
process.
Bi-annually 2,3
Provide a job aid that contains
a visual diagram that labels
the different stages of MTSS
implementation.
Ongoing 1,2,3
Encouraging
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 130
Provide site administrators
the opportunity to meet with
models of successful
administrators who have
achieved similar goals.
Bi-annually 1,2,3
Site administrators receive
ongoing training from an
MTSS specialist/coach in
which they see a
demonstration and have
opportunities for practice and
feedback.
Monthly 1,2,3
Rewarding
Through their monitoring of
site level implementation of
MTSS, district office leaders
will recognize site
administrators at district level
cabinet meetings.
Bi-annually 1,2,3
Through their monitoring of
site level implementation of
MTSS, district office leaders
will share site administrator
successes at school board
meetings.
Bi-annually 1,2,3
Monitoring
Site administrators must
report progress on
organizational goal to MTSS
implementation team that
includes district office
leaders, site administrators,
and teachers.
Monthly 1,2,3
Site administrators will report
on progress of organizational
goals and status of MTSS
implementation to
superintendent and district
office leaders.
Annually 1,2,3
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 131
Organizational support. District office leaders will play an essential role in supporting
site administrators and holding them accountable for executing their critical behaviors on the job.
District office leaders will need to actively participate and engage on the site’s MTSS
implementation team to monitor the change process. They will need to provide funding and time
for site administrators to be trained on implementing MTSS, time management strategies, and
providing an MTSS coach/specialist. District office leaders will need to facilitate a setting for
site administrators and teachers to review district policies that are out of alignment with
organizational goals. Lastly, district office leaders will need to partner with site administrators
and teachers to develop a communication plan for providing ongoing information to all
stakeholders about the implementation of MTSS.
Level 2: Learning
Kirkpatrick and Kirkpatrick (2016) describe Level 2: Learning as the degree to which
participants acquire the knowledge, skills, attitude, confidence, and commitment based on their
participating in training. Knowledge can be characterized by the phrase, “I can do it”, skills by
the phrase, “I can do it right now”, attitude by the phrase “I believe it will be worthwhile”,
confidence by the phrase, “I think I can do it on the job”, and commitment by the phrase, “I will
do it on the job.” (Kirkpatrick & Kirkpatrick, 2016).
Learning goals. The following learning objectives are recommended solutions based on
the KMO needs identified at the end of Chapter Four. Upon completion of the recommended
solutions the site administrators will be able to do the following:
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 132
1. Describe the teacher and district office leader roles in the RTI
2
and PBIS process.
(Factual Knowledge)
2. Describe what full MTSS implementation looks like. (Conceptual Knowledge)
3. Observe and identify effective RTI
2
and PBIS strategies. (Procedural Knowledge)
4. Be confident in their abilities to complete the critical behaviors necessary to
implement an MTSS program at their school. (Self-efficacy)
5. Feel positive about implementing MTSS at their school. (Emotions)
Program. The following program is recommended to achieve the five learning objectives
listed above that address knowledge and motivation needs and the organizational improvements
identified in Chapter Four.
It is recommended that the district hires an MTSS specialist/coach to lead trainings and
provide ongoing support for one school year. The MTSS specialist/coach would begin in the
summer by providing site administrators with a two-day summer training that breaks down
MTSS concepts into manageable parts. At this training, site administrators will be asked to create
a concept map to demonstrate their learning. Site administrators will be provided a visual
diagram that labels the different stages of MTSS implementation and site administrators will
receive training from an MTSS specialist/coach in which they see a demonstration and have
opportunities for practice and feedback. Early in the fall semester, site administrators will visit
multiple schools that have implemented RTI
2
and PBIS and conduct classroom walkthroughs
alongside an MTSS specialist/coach. Site administrators will also sit in on MTSS team meetings
alongside the specialist/coach. Site administrators will debrief with coach about the RTI
2
and
PBIS strategies observed and the specialist/coach will provide feedback. This will take place
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 133
over a five-week period. On these site visits, the MTSS specialist/coach will provide site
administrators with models of successful administrators who have achieved similar goals.
Site administrators will do two activities in the summer to address some of the
organizational concerns regarding the resources needed to implement MTSS at their site. First,
site administrators will attend a time management conference (e.g. Breakthrough Coaching) to
address the concern that they do not have enough time, and secondly, site administrators will
complete a budget analysis where they are forced to prioritize expenditures by their alignment
with organizational goals to address the concern that they do not have the funding to implement
an MTSS program.
The final comprehensive program recommendation is for the school district to create an
MTSS implementation team that includes district office leaders, site administrators, and teachers
to monitor the change process by collecting feedback and adjusting as necessary. This team
should be created near the beginning of the fall semester and initially focus on developing a
communication plan for providing ongoing information to all stakeholders about the
implementation of MTSS. Additionally, the MTSS implementation team should mandate the
creation of a policy review committee consisting of district office leaders, site administrators,
and teachers to conduct a comprehensive policy review that will focus on the implementation of
MTSS. As part of this policy review committee, it is recommended that district office leaders
encourage site administrators to share policies that they would eliminate or modify to increase
their work enjoyment.
Evaluation of the components of learning. Table 18 lists the methods and activities that
will be used to evaluate the declarative knowledge, procedural skills, attitude, confidence, and
commitment of site administrators participating in the recommended training program.
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 134
Table 18
Evaluation of the Components of Learning for the Program.
Method(s) or Activity(ies) Timing
Declarative Knowledge “I know it.”
Knowledge check using concept map to
demonstrate learning.
During training.
Group activity where site administrators teach
and explain newly acquired concepts.
During training.
Pre and post assessment. Before and after training.
Procedural Skills “I can do it right now.”
During classroom walkthroughs, an
observation checklist is completed identifying
RTI
2
and PBIS strategies.
During training.
During MTSS team meeting observation, an
observation checklist is completed identifying
RTI
2
and PBIS strategies.
During training.
Feedback from MTSS coach and peers
provided after practicing implementing
components of MTSS.
During training.
Attitude “I believe this is worthwhile.”
Likert scale survey completed by participants
at the end of each training session.
Before, during, and after training.
Discussion about the value and relevance of
the training.
During training.
Confidence “I think I can do it on the job.”
Likert scale survey completed by participants
at the end of each training session.
Before, during, and after training.
Discussion about challenges and barriers to
implement new knowledge and skills.
During training.
Commitment “I will do it on the job.”
Likert scale survey completed by participants
at the end of each training session.
Before, during, and after training.
Site administrators develop an action plan,
which includes SMART goals to benchmark
progress.
Near the end of training.
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 135
Level 1: Reaction
Kirkpatrick and Kirkpatrick (2016) describe Level 1: Reaction as the degree to which
participants find the training favorable, engaging, and relevant to their job. The goal of the Level
1 is to quickly and efficiently determine whether the training program and instructor were
effective (Kirkpatrick and Kirkpatrick, 2016). Table 19 below lists the methods that will be used
to determine whether site administrators find their training favorable, engaging, and relevant.
Table 19
Components to Measure Reactions to the Program.
Method(s) or Tool(s) Timing
Engagement
Instructor observation. During each training session.
Active interaction during group activities. During each training session.
Asking meaningful questions. During each training session.
Relevance
Pulse check via group discussion. During each training session.
Anonymous survey. After each training session.
Customer Satisfaction
Pulse check via group discussion. During and near the end of the training
session.
Anonymous survey. After each training session.
Evaluation Tools
Kirkpatrick and Kirkpatrick (2016) recommend using a blended evaluation methodology
to gather valuable data about the effectiveness of a training program. This blended approach
includes using immediate and delayed evaluation tools to assess program effectiveness
(Kirkpatrick & Kirkpatrick, 2016). The purpose of immediate evaluation tools is to assess Levels
1 and 2 and measure anticipated application and outcomes. On the other hand, the purpose of
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 136
delayed evaluation tools is to focus on the behaviors and results that training graduates are
experiencing on the job, while also revisiting the reaction and learning levels.
Immediately following the program implementation. For the recommended training
program in this study, an immediate evaluation tool has been created to assess Level 1
(engagement, relevance, and customer satisfaction) and Level 2 (declarative knowledge,
procedural skills, attitude, confidence, commitment) based on site administrators experiences
from the training. The immediate evaluation tool will utilize survey questions with rating scales
items (see Appendix B).
Delayed for a period after the program implementation. Kirkpatrick and Kirkpatrick
(2016) suggest focusing delayed evaluations on how training graduates have applied what they
learned on the job including the support they are receiving (Level 3) and the results they have
accomplished (Level 4). Additionally, delayed evaluations also reexamine the effectiveness of
the training by measuring the reaction (Level 1) and learning (Level 2) of the program once
again (Kirkpatrick & Kirkpatrick, 2016). In this case, the delayed evaluation will take place
about a month after the launch of the training program to the assess the impact of the training
program at all four levels. Evaluating the program at this time will give the organization an
opportunity to see what’s working and troubleshoot areas where the program needs to be revised.
Site administrators will be evaluated through an observation form and a survey including rating
scale items, multiple choice, and open-ended questions (see Appendix C).
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 137
Data Analysis and Reporting
Kirkpatrick and Kirkpatrick (2016) recommend being selective about the data used to
communicate progress toward program goals to stakeholders. For this program, it is
recommended that site administrators continuously share and communicate data that emphasizes
progress toward the number of teachers successfully implementing MTSS in their classrooms,
the number of times individual teachers are observed utilizing appropriate MTSS strategies, and
the number of times that site administrators observe teachers through informal walkthroughs.
This data will be collected by site administrators on informal MTSS walkthroughs and organized
by a central tracking system for informal walkthroughs maintained by the principal's office.
Developing a digital dashboard to communicate the status of these data points will communicate
to internal stakeholders that site administrators are serious about reinforcing, encouraging,
rewarding, and monitoring teacher critical behaviors to support the implementation of MTSS
(See Appendix D).
Summary of the Implementation and Evaluation
The New World Kirkpatrick Model was used to plan, implement, and evaluate the
effectiveness of an MTSS training program designed to provide site administrators with the
knowledge and skills, motivation, and organizational tools necessary to achieve their
performance goals. The Kirkpatrick’s four levels of training and evaluation model was used to
maximize transfer of learning to behavior, and subsequently lead to organizational results. The
New World Kirkpatrick Model was used to backwards plan the MTSS program for site
administrators, so that outcomes were identified from the beginning. Furthermore, each level in
the four-level model was embedded with measurements to determine if the training was doing
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 138
what it was intended to do. In other words, were site administrators finding the training
favorable, engaging, and relevant to their jobs (Level 1), were they acquiring the intended
knowledge, skills, attitude, confidence, and commitment based on their participating in the
training (Level 2), were site administrators applying what they learned on the job (Level 3), and
were targeted outcomes occurring as a result of the training (Level 4).
Limitations and Delimitations
A limitation to this study was that it focused only on site administrators when there were
three stakeholders identified as being essential to achieving the organization’s global goal of
effectively implement MTSS to ensure that all 9th grade students remain on track to meet A-G
graduation requirements. Teachers were the stakeholder of focus in Bhakta’s (2019) examination
of this study, so there is some research on this group. However, district office leaders were not
examined in any capacity limiting the scope of this research topic.
Studying site administrators also limited this study because they are such a small
stakeholder group. Six participants are a small sample even though 100% of site administrators
participated in this study from Westside High School. Another limitation with studying site
administrators was the concern with high turnover in the profession. A recent survey of over
100,000 principals found that over 20% of school leaders left their site after one year (Goldring
et at., 2014). The problem is especially bad at the high school level where principal tenure is a
year shorter on average than at the elementary level (Fuller & Young, 2009). Based on these
statistics, the knowledge, motivation, and organizational factors influencing site administrators
would need to be continuously assessed in the future.
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 139
Recommendations for Future Research
Recommendations for future research on this topic should be initially focused on
collecting data from district office leaders, the third stakeholder in this study. Since the co-
investigator in this study, (Bhakta, 2019), already gathered data from teachers, a more thorough
and complete gap analysis would be possible if research were focused on district office leaders.
Expanding this research topic to multiple schools in the Downtown Unified School
District would also yield additional results for the district to have a better understanding of how
to implement MTSS on a larger scale. This may benefit the district with the efficient use of its
resources. Furthermore, expanding to multiple schools in multiple districts will be ideal for
consumers of this research to have a better grasp of common trends and themes that might
emerge from the studies. Lastly, future research at Westside High School later would be
valuable to study the impact of the MTSS implementation. This research would provide more
opportunities for observations, document review, and data analysis.
Conclusion
In 2018, the Downtown Unified School District developed a goal for Westside High
School to effectively implement MTSS to ensure that all ninth-grade students are on track to
meet A-G requirements. This goal was established in response to a recent policy change to
increase graduation requirements that will require a comprehensive systematic approach to
provide at-risk students with academic, behavioral, and social-emotional supports and
interventions.
This study examined the knowledge and skills, motivational, and organizational needs of
site administrators to effectively implement an MTSS program at their school. Interviews and
document analysis determined what KMOs were assets to the organization and what KMOs were
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 140
needs. Based on the findings, a recommended training program was designed using the New
World Kirkpatrick Model to equip site administrators with the KMOs that they were lacking to
successfully implement MTSS.
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 141
References
Ambrose, S. A., Bridges, M. W., DiPietro, M., Lovett, M. C., & Norman, M. K. (2010). How
learning works. San Francisco: Jossey-Bass.
American Association of School Administrators (2002). Using data to improve schools: What’s
working. Retrieved from
http://www.aasa.org/uploadedFiles/Policy_and_Advocacy/files/UsingDataToImproveSch
ools.pdf
Anderman, E. & Anderman, L. (2006). Attributions. Retrieved from http://www.education.
com/reference/article/attribution-theory/
Anderson, L.W. & Krathwohl, D.R. (2001). A taxonomy for learning, teaching, and assessing: A
revision of Bloom’s taxonomy of educational objectives. New York: Longman.
Baker, L. (2006). Metacognition. Retrieved from http://www.education.com/reference/article/
metacognition/
Bandura, A. (1997). Self-efficacy: The exercise of control. New York: W.H. Freeman.
Baydu, M. M., Kaplan, O., & Bayar, A. (2013). Facing the influence of poverty on graduation
rates in public high schools. Procedia - Social and Behavioral Sciences,84, 233-237.
doi:10.1016/j.sbspro.2013.06.541
Betts, J. Zau, A. & Bachofer, K.V. (2013). College readiness as a graduation requirement: An
assessment of San Diego’s challenges. Public Policy Institute of California. Retrieved
from https://sandera.ucsd.edu/_files/college-readiness-final.pdf
Bhakta, B. (2019). The Implementation of a multi-tiered system of support in Downtown Unified
School District: An analysis of teacher needs. (Unpublished doctoral dissertation).
University of Southern California, Los Angeles, California.
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 142
Bono, J., Foldes, H., Vinson, G. & P Muros, J. (2007). Workplace emotions: The role of
supervision and leadership. The Journal of applied psychology. 92. 1357-67.
10.1037/0021-9010.92.5.1357.
Bower, G. (1995). Emotion and cognition. Proceedings of the 13th Congress of Psychology, Vol.
1., edited by X. Gallegos and L. Hernandez. Mexico City: Trillas Co.
Brown-Chidsey, R., & Bickford, R. (2016). Practical handbook of multi-tiered systems of
support: Building academic and behavioral success in schools. New York: Guilford.
California Department of Education (2017). Multi-Tiered System of Supports. Retrieved from
https://www.cde.ca.gov/ci/cr/ri/.
California Department of Education (2017). Definition of MTSS: The California Department of
Education's definition of multi-tiered system of support and a comparison between it and
response to instruction and intervention. Retrieved from
https://www.cde.ca.gov/ci/cr/ri/mtsscomprti2.asp.
California Department of Education (2017). State minimum course requirements. Retrieved from
https://www.cde.ca.gov/ci/gs/hs/hsgrmin.asp
California Department of Education (2018). California accountability model & school
dashboard. Retrieved from http://www.cde.ca.gov/ta/ac/cm/index.asp
California Department of Education (2018). College/Career indicator. Retrieved from
https://www.cde.ca.gov/ta/ac/cm/cci.asp
California Department of Education DataQuest. (2018). Enrollment, graduates, and dropout in
California public school, 1974-75 through 2013-14. Retrieved from
https://dq.cde.ca.gov/dataquest/EnrGradDrop.asp
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 143
Clark, R. E. & Estes, F. (2008). Turning research into results: A guide to selecting the right
performance solutions. Charlotte, NC: Information Age Publishing, Inc.
Common Core State Standards Initiative (n.d.) About the Standards. Retrieved from
http://www.corestandards.org/about-the-standards/
Conley, D. T. (2007). Redefining college readiness. Educational Policy Improvement Center.
Retrived from http://aypf.org/documents/RedefiningCollegeReadiness.pdf
Cook, C. R., Dart, E., Collins, T., Restori, A., Daikos, C., & Delport, J. (2012). Preliminary
study of the confined collateral, and combined effects of reading and behavioral
interventions: Evidence for a transactional relationship. Behavioral Disorders, 38, 38-56.
Cottrell, D. (2002). Monday morning leadership. Dallas, TX: Cornerstone Leadership Institute.
Duffy, H. (2007). Meeting the needs of significantly struggling learners in high school: A look at
approaches to tiered intervention. Washington, DC: National High School Center at
American Institutes for Research. Retrieved from
http://buildingrti.utexas.org/sites/default/files/booklets/Better_High_Schools_HDuffy.pdf
Eccles, J. (2006). Expectancy value motivational theory. Retrieved from http://www.education.
com/reference/article/expectancy-value-motivational-theory/
Finkelstein, N. D., & Fong, A. B. (Eds.) (2008). Course-taking patterns and preparation for
postsecondary education in California’s public university systems among minority youth.
(Issues & Answers Report, REL 2008–No. 035). Washington, DC: U.S. Department of
Education, Institute of Education Sciences, National Center for Education Evaluation and
Regional Assistance, Regional Educational Laboratory West. Retrieved from
http://ies.ed.gov/ncee/edlabs/regions/west/pdf/REL_2008035_sum.pdf
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 144
Flannery, B. K., Fenning, P., Kato, M. M., & McIntosh, K. (2014). Effects of school-wide
positive behavioral interventions and supports and fidelity of implementation on problem
behavior in high schools. School Psychology Quarterly, 29(2), 111-124.
Forman, S., & Crystal, C. (2015). Systems Consultation for Multi-Tiered Systems of Supports
(MTSS): Implementation Issues. Journal of Educational and Psychological Consultation,
25(2-3), 1-10.
Freeman, J., Sugai, G., Simonsen, B. & Everett, S. (2017). MTSS Coaching: Bridging Knowing
to Doing, Theory Into Practice, 56:1, 29-37, DOI: 10.1080/00405841.2016.1241946
Fuller, E., Young, M. (Summer 2009). Tenure and retention of newly hired principals in Texas.
Texas High School Project: Leadership Initiative Issue Brief 1. Department of
Educational Administration. The University of Texas at Austin. Austin. Retrieved from
http://www.academia.edu/571836/Tenure_and_retention_of_newly_hired_principals_in_
Texas
Gallimore, R., & Goldenberg, C (2001). Analyzing cultural models and settings to connect
minority achievement and school improvement research. Educational Psychologist,
36(1), 45-56.
Gao, N., Lopes, L. & Lee, G. (2017). Just the facts: California’s high school graduation
requirements. Public Policy Institute of California. Retrieved from
http://www.ppic.org/wp-content/uploads/hs-graduation-requirements.pdf
Goldring, R., Taie, W., & Owens, C. (2014). Principal attrition and mobility: Results from the
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 145
2012-13 Principal follow-up survey. National Center for Education Statistics (Ed.):
Institute of Education Sciences. Retrieved from
http://nces.ed.gov/pubs2014/2014064rev.pdf
Gordon, L. (2017). California’s public universities grapple with uncertain future student
enrollments. Daily News. Retrieved from http://www.dailynews.com/social-
affairs/20170204/californias-public-universities-grapple-with-uncertain-future-student-
enrollments
Greenberg, M. T., Weissberg, R. P., O'Brien, M. U., Zins, J. E., Fredericks, L., Resnik, H., &
Elias, M. J. (2003). Enhancing school-based prevention and youth development through
coordinated social, emotional, and academic learning. American Psychologist, 58(6-7),
466-474.
Grissom, J. A., Loeb, S., & Master, B. (2013). Effective Instructional Time Use for School
Leaders: Longitudinal Evidence From Observations of Principals. Educational
Researcher, 42(8), 433–444. https://doi.org/10.3102/0013189X13510020
Hickman, G. P., Bartholomew, M., Mathwig, J., & Heinrich, R. S. (2008). Differential
developmental pathways of high school dropouts and graduates.The Journal of
Educational Research, 102(1), 3-14.
Joe, S., Joe, E., & Rowley, L. L. (2009). Consequences of physical health and mental illness
risks for academic achievement in grades K-12. Review of Research in Education, 33(1),
283-309.
Johnson, L. (2005). Why principals quit. Principal, 84(3), 21 – 23. Retrieved from
http://www.naesp.org/resources/2/Principal/2005/J-Fp21.pdf
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 146
Jones, L.I., Pastor, P.N., Simon A.E. & Reuben C.A. (2014). Use of selected nonmedication
mental health services by adolescent boys and girls with serious emotional or behavioral
difficulties: United States, 2010–2012. NCHS data brief, no 163. Hyattsville, MD:
National Center for Health Statics. Retrieved from
https://www.cdc.gov/nchs/data/databriefs/db163.pdf
Kirkpatrick, J. D., & Kirkpatrick, W. K. (2016). Kirkpatrick's four levels of training evaluation.
Alexandria, VA: ATD Press.
Kirschner, P., Kirschner, F., & Paas, F. (2006). Cognitive load theory. Retrieved from
http://www.education.com/reference/article/cognitive-load-theory/
Krathwohl, D. R. (2002). A revision of Bloom's taxonomy: An overview. Theory into practice,
41(4), 212-218.
Lancaster, P. & Hougen, M. (2017. Preparing educators to work in an MTSS model: Making a
case for partnerships. Perspectives on languages and literacy, 43 (4), 41-46.
Marzano, R, Water, T. & McNulty B.(2005). School Leadership That Works. Alexandria,
Virginia; ASCD.
Mastropieri, M. A., Scruggs, T. E., & Graetz, J. E. (2003). Reading comprehension instruction
for secondary students: Challenges for struggling students and teachers. Learning
Disability Quarterly, 26(2), 103-116.
Maxwell, J. A. (2013). Qualitative research design: An interactive approach (3
rd
Ed.).
Thousand Oaks, CA: SAGE Publications, Inc.
Mayer, R. E. (2011). Applying the science of learning. Boston, MA: Pearson Education.
McIntosh, K., Chard, D.J., Boland, J.B., & Horner, R.H., (2006). Demonstration of
combined efforts in school-wide academic and behavioral systems and incidence of
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 147
reading and behavior challenges in early elementary grades. Journal of Positive Behavior
Interventions, 8, 146-154.
McIntosh, K., & Goodman, S. (2016). Integrated multi-tiered systems of support: blending RTI
and PBIS. New York: The Guilford Press.
Merriam, S. B., & Tisdell, E. J. (2016). Qualitative research: A guide to design and
implementation (4
th
Ed.). San Francisco, CA: Josey-Bass.
Mishkind, A. (2014). Overview: State definitions of college and career readiness. College &
Career Readiness & Success Center at American Institutes for Research. Retrieved from
https://ccrscenter.org/sites/default/files/CCRS%20Defintions%20Brief_REV_1.pdf
Murnane, R. J. (2013). U.S. high school graduation rates: Patterns and explanations. Journal of
Economic Literature, 51(2), 370-422.
O’Connor, E. P., & Freeman, E. (2012). District-level considerations in supporting and
sustaining RTI implementation. Psychology in the Schools, 49, 297–310.
doi:10.1002/pits.21598.
Osterman, K. & Sullivan, S. (1996). New principals in an urban bureaucracy: A sense of
efficacy. Journal of School Leadership, 6, 661-690.
Pan, D., Rudo, Z. H., Schneider, C. L., Smith Hansen, L. (2003). Examination of resource
allocation in education: Connecting spending to student performance. Research Report.
Southwest Educational Development Lab., Austin, TX. Institute of Education Sciences
(ED), Washington, DC.
Pajares, F. (2006). Self-efficacy theory. Retrieved from http://www.education.com/reference/
article/self-efficacy-theory/
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 148
Plunk, A., Tate, W., Bierut, L., & Grucza, R. (2014). Intended and unintended effects of state-
mandated high school science and mathematics course graduation requirements on
educational attainment. Educational Researcher, 43(5), 230-241.
Positive Behavioral Supports and Interventions (2018). Implementation blueprint. Retrieved
from http://www.pbis.org/blueprintguidestools/blueprint/implementation-blueprint
Prasse, D. P., Breunlin, R. J., Giroux, D., Hunt, J., Morrison, D., & Thier, K. (2012). Embedding
multi-tiered system of supports/response to intervention into teacher preparation.
Learning Disabilities: A Contemporary Journal, 10(2), 75-93.
Raffo, C., Dyson, A., Gunter, H., Hall, D., Jones, L., & Kalambouka, A. (2009). Educationand
poverty: Mapping the terrain and making the links to educational poverty. International
Journal of Inclusive Education, 13(4), 341-358.
Rueda, R. (2011). The 3 dimensions of improving student performance. New York: Teachers
College Press.
Saeki, E., Jimerson, S. R., Earhart, J., Hart, S. R., Renshaw, T., Singh, R. D., & Stewart, K.
(2011). Response to intervention (RtI) in the social, emotional, and behavioral domains:
Current challenges and emerging possibilities.Contemporary School Psychology, 15, 43-
52.
Schmoker, M. (1999). Results: The key to continuous school improvement. Alexandria, VA:
Association of Supervision and Curriculum Development.
Schraw, G., & McCrudden, M. (2006). Information processing theory. Retrieved from http://
www.education.com/reference/article/information-processing-theory/.
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 149
Schrobsdorff, S. (2016). There’s a startling increase in major depression among teens in the U.S.
Time. Retrieved from http://time.com/4572593/increase-depression-teens-teenage-
mental-health/
Shaw, R., & Walker, W. (1981). High school graduation requirements—From whence did they
come? NASSP Bulletin, 65(447), 96-102.
Tschannen-Moran, M., & Gareis, C. (2007). Cultivating Principals’ Self-Efficacy: Supports that
Matter. Journal of School Leadership, 17(1), 89–114.
Tierney, W., & Sablan, J. (2014). Examining college readiness. American Behavioral Scientist,
58(8), 943-946.
United States Department of Education (2010). A blueprint for reform: The reauthorization of
the elementary and secondary education act. Retrieved from
https://www2.ed.gov/policy/elsec/leg/blueprint/blueprint.pdf
United States Department of Education (n.d.). College and career ready students. Retrieved from
https://www2.ed.gov/policy/elsec/leg/blueprint/college-career-ready.pdf
University of California (n.d.). A-G Guide. Retrieved from http://www.ucop.edu/agguide/
Vekaria, H. (2017). Ask the Administrators: Interviews Provide Guidance for MTSS
Implementation. Perspectives on Language and Literacy, 43(4), 37-39
Venezia, A., & Jaeger, L. (2013). Transitions from high school to college. The Future of
Children, 23(1), 117-136.
Wyatt, J.N., Wiley, A., Camara, W.J. & Proestler, N. (2011). The development of an index of
academic rigor for college readiness. Retrieved from
https://research.collegeboard.org/sites/default/files/publications/2012/7/researchreport-
2011-11- development-index-academic-rigor-college-success.pdf
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 150
Yough, M., & Anderman, E. (2006). Goal orientation theory. Retrieved from http://www.
education.com/reference/article/goal-orientation-theory/
Zins, J.E., Weissberg, R.P., Wang, M.C., & Walberg. H. J. (Eds.). (2004). Building academic
success on social and emotional learning: what does the research say? New York:
Teachers College Press.
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 151
Appendix A: Interview Protocol
Site Administrator Interview Protocol Introduction:
Thank for you joining me today to participate in this dissertation study. My name is Matt
Chambers and I am an Educational Doctorate student at USC’s Rossier School of Education.
The purpose of this study is to examine the organizational needs for implementing an integrated
and comprehensive intervention program at your school site. All participant names and
information including the district, school, and individuals will be de-identified and kept
confidential. The interview should take about 30-45 minutes to complete.
Before we begin, I would like to make sure that you are aware that we can stop the
interview at any time if you feel uncomfortable with the questions or process. You can choose
not to answer any questions. I would like to confirm that you consent to participating in this
study. Do you consent to participating in this pilot study? Wait for response. Lastly, I would like
to record the audio (no video) of our conversation for maintaining accurate records of participant
responses. This is not necessary if you feel uncomfortable with being recorded. I am the only
person that will listen to this recording. Do you consent to being recorded for this interview?
Wait for response. Once again, thank you so much for agreeing to participate in this pilot study,
we will not begin.
*Turn on Recording Device on iPhone
My name is Matt Chambers and I am an Educational Doctorate student at USC’s Rossier
School of Education. This is an interview for my dissertation study. I am sitting here with Insert
Participant’s Name on Date. Mr. or Mrs. Insert Participant’s Name, do you consent to being
audio recorded for this study? Wait for response. [If no, stop recording / If yes, process]. Thank
you, let us begin with the first question.
Site Administrator Interview Questions
1. What is your understanding of a Multi-Tiered System of Support (MTSS)? - Factual
Knowledge
Interview Note: If response to #1 demonstrates that participant does not fully understand
what MTSS is, interviewer will provide participant with the following definition:
A Multi-tiered system of support (MTSS) is an integrated and comprehensive intervention model
that combines academic, behavioral, and social-emotional support. The MTSS framework
combines Response to Intervention and Instruction (RTI²) and Positive Behavioral Interventions
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 152
and Supports (PBIS) to create a more efficient and effective intervention model that serves the
whole student.
1. As a site administrator, what do you see your role being in the MTSS process? - Factual
Knowledge
2. Describe the teacher role in the MTSS process. - Factual Knowledge
3. Give me an example of an appropriate MTSS teacher strategy. - Conceptual Knowledge
4. How confident are you in your abilities to monitor and support teachers as they
implement MTSS strategies? - Self-Efficacy
5. In what ways, if any, do you think teacher success in implementing effective MTSS
strategies will be in your control? - Attributions
6. Tell me how you would reinforce, encourage, and reward teachers implementing MTSS
strategies. - Procedural Knowledge
7. Describe the supports you would provide teachers who are not implementing components
of MTSS. - Procedural Knowledge
8. Describe for me district office leader roles in the MTSS process - Factual Knowledge
9. How confident are you in your abilities to partner with district office leaders to
implement MTSS at your site? - Self-Efficacy
10. Ideally, what resources would you need to implement MTSS effectively? - Resources
Interviewer Note: If response to #11 does not include finances, time, roles or structure of
schedule ask the following questions:
a. FINANCES: Tell me about the financial challenges you may face implementing MTSS
at your site. - Resources
b. TIME: With everything else going on at your school, how are you going to make the
time to plan and implement MTSS? - Resources
c. ROLES: Describe any adjustments to people’s roles and responsibilities you see
necessary to implement MTSS? - Resources
d. STRUCTURE OF SCHEDULE: Describe any structural changes to the school schedule
you see necessary to implement MTSS. - Resources
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 153
1. What policies or procedures would you see having to be changed or created to implement
MTSS? - Policies, Processes, and Procedures
2. Tell me how the culture of your school is going to influence the implementation of MTSS
at your site. - Culture
3. Describe how the culture of cooperation among teachers, site administrators, and district
office leaders will impact the implementation of MTSS. - Culture
4. How do you evaluate the effectiveness of your role in implementing new programs? -
Metacognitive Knowledge
a. What leads you to change your practices? - Metacognitive Knowledge
5. Do you find value in integrating academic, behavioral, and social emotional support into
one comprehensive intervention model? - Value
6. How positive do you feel about implementing MTSS at your school? - Emotions
7. Describe your feelings about teacher bias regarding over identifying specific subgroups
as having academic or behavioral deficiencies. How might these biases impact the
implementation of an academic and behavioral intervention program? - Diversity
End of Interview
#19-26 are questions that will most likely will be answered in the questions above. I will
reserve them as extra questions to scan at the end of the interview and ask if necessary
(also depending on time of interview):
1. Give me an example of some current student performance data at your school. - Factual
Knowledge
2. What do you think that full implementation of MTSS looks like? - Factual Knowledge
3. Give me an example of an inappropriate MTSS teacher strategy? - Conceptual
Knowledge
4. How do you classify students as being on track or off track for meeting their academic
goals? - Conceptual Knowledge
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 154
5. Tell me the steps you would take for observing and identifying effective MTSS
strategies. - Procedural Knowledge
6. Describe the steps you take to develop organizational goals for student success. -
Procedural Knowledge
7. How valuable is it to partner with district office leaders when implementing MTSS? -
Value
8. How positive do you feel about partnering with district office leaders to implement
MTSS at your site? - Emotions
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 155
Appendix B: Immediate Evaluation Tool
Immediate Evaluation Tool (Levels 1 and 2)
Please use the following rating scale to circle the number that best correlates with how you
feel about each statement.
1 = Strongly Disagree
2 = Disagree
3 = Neutral
4 = Agree
5 = Strongly Agree
The training held my interest. (L1 - Engagement) 1 2 3 4 5
The information in this program is applicable to my work. (L1 -
Relevance)
1 2 3 4 5
I was satisfied with the training. (L1 - Customer Service) 1 2 3 4 5
What I learned will help me do my job better. (L1 - Attitude). 1 2 3 4 5
I am confident that I can apply what I learned from this training on
the job. (L1 - Confidence and L2 - Self-efficacy)
1 2 3 4 5
I am committed to implement what I learned from this training on the
job. (L1 - Commitment).
1 2 3 4 5
I feel positive about implementing MTSS at my school. (L2 -
Emotions)
1 2 3 4 5
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 156
Mark an X next to the following statements that best describe teacher roles in the MTSS process.
(L2)
_____Identify students as recipients of academic, behavioral, or social emotional support.*
_____Creating individualized goals for all students and grouping students based on their
performance levels.
_____Implement RTI
2
and PBIS strategies in the classroom to support students.*
_____Follow established protocols to work in teams to analyze data, create goals, and
develop and refer students to Tier 2 and Tier 3 interventions.*
_____Using data to drive instruction, while focusing all social emotional and behavioral
issues to counselors and administrators for Tier 2 and 3 support.
Which of the following is NOT a district office leader role in the MTSS process? (L2)
a. Ensuring implementation is district wide*
b. Supporting training needs of the program
c. Creating teams to support the management of the program
d. Creating teams to support the implementation of the program.
e. All of the above are district office leader roles in the MTSS process.
Which of the following best describes what full MTSS implementation looks like? (L2)
a. A blended intervention model that includes RTI and PBIS.
b. An intervention model that includes three tiers of service.
c. An intervention model that includes teams analyzing data to guide
decisions about student support.
d. An ongoing evaluation of the intervention model.
e. All of the above.*
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 157
Appendix C: Delayed Evaluation Tool
Delayed Evaluation Tool (Levels 1, 2, 3, and 4)
Please use the following rating scale to circle the number that best correlates with how you
feel about each statement regarding last month’s MTSS training program.
1 = Strongly Disagree
2 = Disagree
3 = Neutral
4 = Agree
5 = Strongly Agree
I have applied what I learned at last month's MTSS training to my
current role as a site administrator . (L1, L2, and L3)
1 2 3
4 5
Looking back, the MTSS training was a good use of my time. (L1) 1 2 3
4 5
I am still confident in applying what I learned at the MTSS training last
month in my current role as a site administrator (L1 and L2).
1 2 3
4 5
I am still committed to implementing what I learned from last month’s
MTSS training on the job. (L1).
1 2 3
4 5
I still feel positive about implementing MTSS at my school. (L2) 1 2 3
4 5
This program has positively impacted my school. (L1, L2, L4) 1 2 3
4 5
I am already seeing positive results from the training (L4). 1 2 3
4 5
Mark an X next to the following statements that best describe teacher roles in the MTSS process.
(L2)
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 158
_____Identify students as recipients of academic, behavioral, or social emotional support.*
_____Creating individualized goals for all students and grouping students based on their
performance levels.
_____Implement RTI
2
and PBIS strategies in the classroom to support students.*
_____Follow established protocols to work in teams to analyze data, create goals, and
develop and refer students to Tier 2 and Tier 3 interventions.*
_____Using data to drive instruction, while focusing all social emotional and behavioral
issues to counselors and administrators for Tier 2 and 3 support.
Which of the following is NOT a district office leader role in the MTSS process? (L2)
a. Ensuring implementation is district wide*
b. Supporting training needs of the program
c. Creating teams to support the management of the program
d. Creating teams to support the implementation of the program.
e. All of the above are district office leader roles in the MTSS process.
Which of the following best describes what full MTSS implementation looks like? (L2)
a. A blended intervention model that includes RTI and PBIS.
b. An intervention model that includes three tiers of service.
c. An intervention model that includes teams analyzing data to guide
decisions about student support.
d. An ongoing evaluation of the intervention model.
e. All of the above.*
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 159
The following observation checklist will be completed to evaluate the training program
(L3):
Critical Behavior Observed Not
Observed
Comment
Site administrators define observable and measurable
organizational goals and targeted results for student
success.
Site administrators create processes that reinforce,
monitor, encourage, and reward teacher critical behaviors.
Site administrators use processes to partner with district
office leaders to ensure proper support and/or recognition
of teachers and sites in their implementation of MTSS.
IMPLEMENTING A MULTI-TIERED SYSTEM OF SUPPORT 160
Appendix D: Digital Dashboard
Example of Digital Dashboard to Communicate Progress of Goals
Progress Tool #1 Progress Tool #2 Progress Tool #3
Number of teachers
successfully implementing
MTSS in their classrooms.
Number of times individual
teachers are observed
utilizing appropriate MTSS
strategies.
Number of times that site
administrators observe
teachers through informal
walkthroughs.
Abstract (if available)
Abstract
This study utilizes Clark and Estes’ (2008) performance analysis model, which systematically and analytically clarifies organizational goals to identify the current and preferred performance level within an organization. The purpose of this study was to identify the knowledge, motivation and organizational (KMO) needs of site administrators to effectively implement a Multi-Tiered System of Support (MTSS) at Westside High School in Downtown Unified School District. All six site administrators from Westside High School were interviewed and documents were analyzed to determine which of the twenty assumed KMO influences identified in this study were actual needs. Findings from this study showed that site administrators at WHS are somewhat motivated and capable of effectively implementing MTSS at their sites. However, ten of the twenty influences that were assessed were determined to be actual needs including four knowledge influences, two motivation influences, and all six organizational influences. This study makes recommendations for improving organizational performance by utilizing The New World Kirkpatrick Model to plan, implement, and evaluate the effectiveness of an MTSS training program designed to provide site administrators with the knowledge and skills, motivation, and organizational tools necessary to achieve their performance goals.
Linked assets
University of Southern California Dissertations and Theses
Conceptually similar
PDF
The implementation of a multi-tiered system of support at Downtown Unified School District: an analysis of teacher needs
PDF
Multi-tiered system of support as the overarching umbrella: an improvement model using KMO gap analysis to address the problem of school districts struggling to implement MTSS
PDF
Promoting a positive school culture from three perspectives: a promising practices study from the administrator perspective
PDF
Positive behavior intervention support plan: a gap analysis
PDF
The interaction of teacher knowledge and motivation with organizational influences on the implementation of a hybrid reading intervention model taught in elementary grades
PDF
Improving foundational reading skills growth in middle school: a promising practices study
PDF
Multi-tiered system of supports to address significant disproportionality in special education
PDF
Implementing an equity-based multi-tiered system of support in a large urban school district: the grows and glows
PDF
Embedded academic support for high school student success: an innovation study
PDF
Implementation of the Social Justice Anchor Standards in the West Coast Unified School District: a gap analysis
PDF
The implementation of response to intervention: an adapted gap analysis
PDF
Winning the organizational leadership game through engagement: a gap analysis
PDF
Establishing a systematic evaluation of positive behavioral interventions and supports to improve implementation and accountability approaches using a gap analysis framework
PDF
Promoting equity in discipline practices for Latino students: a gap analysis
PDF
Reducing suspensions through implementation of schoolwide PBIS
PDF
Increasing student persistence at a community college from an administration perspective
PDF
Teacher perception on positive behavior interventions and supports’ (PBIS) cultivation for positive teacher-student relationships in high schools: an evaluation study
PDF
The role of positive behavior systems in reducing exclusionary school discipline
PDF
A qualitative examination of PBIS team members' perceptions of urban high school teachers' role in implementing tier 2 schoolwide positive behavior supports
PDF
An examination of tri-level collaboration around student achievement using the gap analysis approach: School site leadership factors
Asset Metadata
Creator
Chambers, Matthew Ryan
(author)
Core Title
The implementation of a multi-tiered system of support in Downtown Unified School District: an analysis of site administrator needs
School
Rossier School of Education
Degree
Doctor of Education
Degree Program
Education (Leadership)
Publication Date
03/01/2019
Defense Date
03/01/2019
Publisher
University of Southern California
(original),
University of Southern California. Libraries
(digital)
Tag
academic supports,behavioral supports,gap analysis,intervention,intervention system,Knowledge,Motivation,MTSS,multi-tiered system of support,OAI-PMH Harvest,organization,PBIS,RTI,site administrators,social-emotional supports,training program
Format
application/pdf
(imt)
Language
English
Contributor
Electronically uploaded by the author
(provenance)
Advisor
Yates, Kenneth (
committee chair
), Hinga, Briana (
committee member
), Paramo, John (
committee member
)
Creator Email
mattchambers41@gmail.com,mrchambe@usc.edu
Permanent Link (DOI)
https://doi.org/10.25549/usctheses-c89-129317
Unique identifier
UC11675641
Identifier
etd-ChambersMa-7127.pdf (filename),usctheses-c89-129317 (legacy record id)
Legacy Identifier
etd-ChambersMa-7127.pdf
Dmrecord
129317
Document Type
Dissertation
Format
application/pdf (imt)
Rights
Chambers, Matthew Ryan
Type
texts
Source
University of Southern California
(contributing entity),
University of Southern California Dissertations and Theses
(collection)
Access Conditions
The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the a...
Repository Name
University of Southern California Digital Library
Repository Location
USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA
Tags
academic supports
behavioral supports
gap analysis
intervention
intervention system
MTSS
multi-tiered system of support
organization
PBIS
RTI
site administrators
social-emotional supports
training program