University of Southern California Dissertations and Theses

Evaluating the implementation of 21st century skills and learning

(USC Thesis Other)

Evaluating the implementation of 21st century skills and learning

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content Running head: EVALUATING THE IMPLEMENTATION OF 21
ST
1

Evaluating the Implementation of 21
st
Century Skills and Learning
by
Marsha A. Coloma-Flores

A Dissertation Presented to the
FACULTY OF THE USC ROSSIER SCHOOL OF EDUCATION
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulfillment of the
Requirements for the Degree
DOCTOR OF EDUCATION

December 2018

Copyright 2018 Marsha A. Coloma-Flores
EVALUATING THE IMPLEMENTATION OF 21
ST
2

DEDICATION
This dissertation is dedicated to the memory of my beloved grandparents, Mr. & Mrs.
Alfonso and Elena Abucay and Mr. & Mrs. Gerobuano and Emilia Coloma. Without their
enormous personal sacrifices and unconditional love, I would have never become the individual I
am today.
I hope that I have made them proud by the achievement of this monumental academic
goal. I wish that they were here today to share with me the celebration and completion of
receiving a Doctor of Education degree.

EVALUATING THE IMPLEMENTATION OF 21
ST
3

ACKNOWLEDGEMENTS
Completion of this study would not have been achieved without the kindness and support
of many key figures in my life, all of whom have made this a very humbling experience. To the
University of Southern California’s Rossier School of Education, thank you for letting me fulfill
my dream of being a student at this school. I would like to express my deepest and sincerest
gratitude to my chair and advisor, Dr. Eugenia Mora-Flores. No dissertation is complete without
the guidance and genius of one’s chair and for this I am thankful. Dr. Mora-Flores shared her
insight, expertise, and provided unwavering support for me, to see me through my challenges in
this journey to completion. Without her advice and persistent help, this dissertation would not
have been possible. To my committee members, Dr. Margo Pensavalle and Dr. Kimberly
Ferrario, I am extremely grateful for your assistance and recommendations throughout my study,
as well as serving on my committee.
I would like to acknowledge the Rossier School of Education OCL staff for their
feedback, instructional expertise, and their willingness to share their knowledge and experiences
towards developing our growth and abilities as Doctor of Education graduates. Thank you,
especially, to the professional leadership of Dr. Monique Datta, Dr. Doug Lynch, Dr. Kathy
Stowe, Dr. Daniel Chatham, Dr. Ester Kim, Dr. Julie Slayton, and Dr. Kenneth Yates for their
words of encouragement and especially their patience to help me understand the complex lessons
and material. I am deeply indebted to their invaluable help in preparing this study. To my
classmates in Cohort 4 and some of the world’s most intelligent people walking the Earth – WE
MADE IT!!!
To my family and friends, especially my sisters, Mylene and Emmie, my dad, Meneo,
and my mom, Nenita - thank you for helping me survive all the stress from the past three years
EVALUATING THE IMPLEMENTATION OF 21
ST
4

and not allowing me to give up. You should know that your support was worth more than I can
express on paper. The Lord has definitely blessed me, and I am forever grateful.
Above all, I wish to thank my loving wife, Danielle. You are a constant source of
strength, energy, and motivation that has helped me get through this bold and arduous
undertaking towards crossing the finish line. I am blessed to be able to share this journey with
you. You are my greatest inspiration.

EVALUATING THE IMPLEMENTATION OF 21
ST
5

ABSTRACT
This dissertation evaluated the knowledge, motivation and organizational influences
surrounding the implementation of 21
st
century skills and learning among educators in the
inclusive classroom at Canyon Valley Elementary School, a pre-kindergarten through 5
th
grade
campus. The impetus of this study was centered on the organization’s goal of 100%
implementation of 21
st
century education, and the challenges and learning opportunities teachers
face as educators. Specifically, this study explored teachers’ experiences and perception of 21
st

century education at their respective sites, and the impact of 21
st
century learning mandates. The
study utilized Clark and Estes (2008) Gap Analytical Framework and a review of the literature to
identify factors effecting the implementation of 21
st
century skills and learning including
knowledge and motivation inconsistencies (confidence) and organizational influences
(instructional time and on-site support, program initiatives and support, and assessment and
mastery).
The methodological approach adopted a qualitative design using semi-structured
interviews, a survey and document analysis. Six teachers participated in the qualitative
interviews and survey, drawing on their experiences, current challenges, and their projected
visions for further implementation of 21
st
century education. Documents and artifacts examined
involved student and parent handbooks, district websites, lesson plans, and the Local Control
Accountability Plan.
Seven KMO influencers identified through the literature review and validated through the
data analysis provided the basis for the recommendations. Knowledge elements focus primarily
on teacher related goals and what is needed for full implementation of 21
st
Century Education;
motivation elements surround teacher self-efficacy and goal orientation; and organizational
EVALUATING THE IMPLEMENTATION OF 21
ST
6

elements address what resources are used to achieve the teacher’s goal to raise student
achievement through 21
st
century skills. The research study found that in order to move closer to
100% implementation of 21
st
century education, teachers would flourish through participation of
pre-service trainings, evaluation and feedback, and creating job aids at the start of each school
year.
The recommendations and implementation model presented includes collaboration time
with the goal of increasing and impacting student achievement. This study was presented with
the intent of helping CVES continue to improve the implementation of 21
st
century education in
all inclusion classrooms.
EVALUATING THE IMPLEMENTATION OF 21
ST
7

TABLE OF CONTENTS

Dedication 2
Acknowledgements 3
Abstract 5
Table of Contents 7
List of Tables 10
List of Figures 11
Introduction to Problem of Practice 12
Organizational Context and Mission 13
Importance of Addressing the Problem 13
Purpose of the Project and Questions 15
Organizational Performance Goal 16
Description of Stakeholder Groups 17
Stakeholder Group of Focus 18
Stakeholder Performance Goals 18
Review of the Literature 20
21
st
Century Education 21
21
st
Century Education in the Inclusive Classroom 23
Legal Implications in Inclusive Education 25
Addressing Teacher Readiness 25
Individualized Education Plan 26
Teacher Self-Efficacy 27
EVALUATING THE IMPLEMENTATION OF 21
ST
8

Mainstreaming in the 21
st
Century Classroom 27
Stakeholder Knowledge, Motivation, and Organizational Influences 28
Knowledge Influences 29
Motivation 34
Organizational 40
The Clark and Estes (2008) Gap Analysis Conceptual Framework 45
Data Collection and Instrumentation 47
Qualitative Data Collection and Instrumentation 48
Interviews 49
Documents and Artifacts 50
Findings 50
Participating Stakeholders 51
RQ1: What are the teachers’ knowledge and motivational factors that contribute to CVES
achieving its goal of 100% of 21
st
Century Education? 52
Knowledge Inconsistencies 52
Motivation Inconsistencies 54
Instructional Confidence 55
RQ2: What are the organizational elements related to CVES achieving its goal of 100% of 21
st

Century Education? 58
Instructional Time and On-Site Support 58
Program Initiatives and Support 60
Assessment and Mastery of Implementation Inconsistency 65
Recommendations for Practice to Address KMO Influences 68
EVALUATING THE IMPLEMENTATION OF 21
ST
9

Knowledge Recommendations 69
Motivation Recommendations 74
Organization Recommendations 78
Conclusion 84
References 86
Appendix A: Participating Stakeholders with Sampling Criteria for Interviews 94
Appendix B: Ethics 96
Appendix C: Interview Instrument Protocols 97
Appendix D: Informed Consent Protocols 103
Appendix E: Integrated Implementation and Evalutation Plan 104
Level 4: Results and Leading Indicators 106
Level 3: Behavior 107
Level 2: Learning 110
Level 1: Reaction 113
Appendix F: Level 1-4 Survey (Delayed After Training) 118
Appendix G: Level 1-4 Survey (Delayed After Training) 120

EVALUATING THE IMPLEMENTATION OF 21
ST
10

LIST OF TABLES

Table 1: Organizational Mission and Global Goal…….………………………….……….….... 19
Table 2: Knowledge Influences, Types, and Assessments for Knowledge Gap Analysis……... 33
Table 3: Motivational Influences and Motivational Influence Assessment…..………………… 39
Table 4: Organizational Influences.…………………………………………..………………… 43
Table 5: Participant Demographics….……………………………………..…………………… 51
Table 6: Survey Responses for Self-Efficacy in Implementing 21
st
Century Instructional
Strategies…………………………………………………………………..…………………… 56
Table 7: Teacher Needs for Full Implementation of 21
st
Century Skills..…………………...… 64
Table 8: Needs to Meet Mastery of 21
st
Century Instruction…………..……………………… 67
Table 9: Summary of Knowledge Influences and Recommendations…………………………. 69
Table 10: Summary of Motivational Influences and Recommendations..……………………... 75
Table 11: Summary of Organizational Influences and Recommendations..…………………… 79
Table 12: Outcomes, Metrics, and Methods for External and Internal Outcomes…………… 106
Table 13: Critical Behaviors, Metrics, Methods, and Timing for Evaluation.……………….. 107
Table 14: Required Drivers to Support Critical Behaviors.………………………………….. 108
Table 15: Evaluation of the Components of Learning for the Program..…………………….. 112
Table 16: Components to Measure Reactions to the Program……………………………….. 113
Table 17: Dashboard of Data Measures..…………………………………………………….. 115

EVALUATING THE IMPLEMENTATION OF 21
ST
11

LIST OF FIGURES

Figure 1: Twenty-first Century Student Outcomes and Support…...…………………………... 23
Figure 2: Collaborative Influences of Knowledge, Motivation, and Organizational Factors....... 47

EVALUATING THE IMPLEMENTATION OF 21
ST
12

Introduction to Problem of Practice
Providing all students with a rigorous and current education by teaching 21
st
century skills
has become an integral part of today’s school system. Twenty-first century instruction, also
known as, authentic instruction, stresses higher order thinking, application of knowledge and
learning, conversation, and global citizenship for all learners (Preus, 2012). In today’s inclusive
classroom, preparing all students of various learning abilities to develop 21
st
century skills is
imperative to compete, survive, and succeed in the world. Inclusive education is when children
with a variety of learning needs, including language and mild to moderate disabilities, are
enrolled together in the same classes. Inclusion is an attempt to respond to all individual
students by structuring a curriculum that enhances equitable opportunities of learning within the
general education classroom, and reduce the exclusion of pupils (Grayson, 1990). Educational
leaders in the field have made attempts to research and employ various methods to work with the
growing diverse student population, yet there is little evidence to the most effective practices for
working with all students in the inclusive classroom (Heflin & Simpson, 1998; Morrier et al.,
2011; Strong 2014). All students are entitled to an education that prepares them with the skills
needed in an ever changing, global and technologically rich society. The challenge lies in the
preparation of teachers to meet the needs of a diverse classroom as well as learn about and
implement 21
st
century skills. Therefore, there is a need to support teachers in the inclusive
classroom to successfully implement 21
st
century skills. This problem is important to address
because the preparation of general educators is essential if equitable learning opportunities and
outcomes, for all students, are to be improved through 21
st
century skills.

EVALUATING THE IMPLEMENTATION OF 21
ST
13

Organizational Context and Mission
Canyon Valley Elementary School (CVES) is one of several pre-kindergarten through
fifth grade schools on a modified traditional program in the South Calzone County School
District. All classrooms at CVES practice full inclusion supporting various student learning
needs, including students with mild to moderate disabilities. The mission of the school is to
transform public education and guarantee high levels of student achievement by applying
innovative 21
st
century practices; leverage resources; develop strategic alliances; inspire
powerful leadership; and provide exemplary customized services to the community and students
in all programs.
Canyon Valley Elementary School consists of twenty-two teachers, (Pre-K (2),
Kindergarten (4), 1
st
(4), 2
nd
(3), 3
rd
(3), 4
th
(3), 5
th
(3)), a full-time school psychologist, speech -
language pathologist, resource director, and four support staff. These specialized staff positions
push in to support classroom teachers and students identified with learning needs. Identified
students meet with specialized staff for a set block of time either one-on-one, or in small group
settings to receive specific or modified instruction. The site leadership at CVES includes one
full-time administrator. Instructional leadership teams meet once a month to make collaborative
decisions regarding curriculum and school site concerns. The instructional leadership team
consists of the administrator, speech-language pathologist, resource director, and a grade level
representative. The school also provides after-school programs that consists of daily homework
help as well as athletic and enrichment activities.
Importance of Addressing the Problem
It is important to investigate the challenge of preparing teachers to implement 21
st

century skills because the needs of all students are highly dependent on the instructional
EVALUATING THE IMPLEMENTATION OF 21
ST
14

effectiveness of their school’s educational professionals. The effectiveness must align to the
social and global demands of how to be a successful citizen in the 21
st
century. This is important
because all students must be provided an education that prepares them for the everchanging
demands of the work force in the 21
st
century. Under-utilization of one’s potential along with
the demands of globalization have pushed education to readdress their policies and practices to
be in alignment with the Partnership for 21
st
century skills (P21) framework, which has been
adopted by numerous school district in the country (Uduigwome, 2012). The 21
st
century skills
approach is designed to academically challenge students and invite them to engage in issues that
have personal or social significance (Phelps, 2005; Preus, 2012). There is currently a wide
global achievement gap in the United States in comparison to its international counterparts, and
the country cannot continue to lag in this area. We need all educational settings, including the
inclusive classroom to ensure that all students in such a diverse setting are provided access to 21
st

century curriculum
Some of the negative perceptions expressed by educators, as well as the natural
challenges the inclusive classroom poses, include fostering a culturally responsive classroom to
meet the needs of all the students in the classroom, and limited trainings provided to teachers.
The inclusion model is a construct of the American school system that will not be going away
and has become the current education standard. Teachers at CVES need to be prepared to meet
the task in order to meet their school and district 21
st
century education goals. The goal of a 21
st

century inclusive educational setting is to construct a learning environment in which all people
are cognizant about and compassionate of one another (Steinback et al., 1994). Ultimately, a call
to educational leaders is urgent, to seek methods towards cultivating inclusive school
communities that recognize differences and meet diverse needs within a common framework
EVALUATING THE IMPLEMENTATION OF 21
ST
15

towards achieving the goal of receiving a 21
st
century education. Teachers at CVES can begin to
have a better understanding of these issues by evaluating the knowledge, motivation, and
organizational factors related to their own self-efficacy in inclusive instruction and 21
st
century
teaching and learning. Through this, teachers can realize instructional changes that need to be
addressed in order to invite all learners in the inclusive classroom to gain the knowledge 21
st

century Education proposes.
Purpose of the Project and Questions
The purpose of this project is to investigate the challenges educational professionals face
in implementing 21
st
century instruction in the inclusive classroom at Canyon Valley Elementary
School. The analysis will focus on knowledge, motivation, and organizational influences related
to achieving the organizational goals. While a complete performance evaluation would focus on
all stakeholders, for practical purposes the stakeholder to be focused on in this analysis are
teachers in the inclusive classrooms. There are social and academic benefits to running an
inclusive education system and understanding a teacher’s knowledge and motivation to
instructional implementation are required in order to apply an effective inclusive environment
that includes the implementation of 21
st
century skills. As such, the questions that guide the
study are the following:
1. What are the teachers’ knowledge and motivational factors that contribute to CVES
achieving its goal of 100% implementation of 21
st
Century education?
2. What are the organizational elements related to CVES achieving its goal of 100%
implementation of 21
st
Century education.

EVALUATING THE IMPLEMENTATION OF 21
ST
16

Organizational Performance Goal
CVES has set a three-year goal involving the achievement of students in the inclusive
classroom written in the district’s Local Control and Accountability Plan (LCAP). The learning
goal is that the district will implement and assess a rigorous, standards aligned, 21
st
century
educational program that promotes the development of the whole child. The expected annual
measurable outcomes of student achievement will improve by the following: (1) 100% of ELA,
ELD, and Math lessons will be aligned to the rigor of California’s Common Core State Standards
as measured by observation and walkthrough data. (2) All students will read at grade level by the
end of third grade as measured by district measures. (3) The achievement gap for students with
special needs will close by 3% annually. (4) Collect and analyze special education referral and
SST data as a baseline. (5) 100% of students will have access to standards aligned instructional
materials in core subjects, as measured by Williams Settlement compliance data and internal
monitoring. The districts planned actions and services to meet these outcomes include providing
grade level and cross grade level collaboration for teachers, coaches, and administrators to plan
California Common Core Standards aligned lessons which follow a 21
st
century skill curriculum,
design formative assessments, analyze student work and participate in lesson study. Professional
development on instructional practices and pedagogy for all certificated staff and administrators
will be integrated in the district’s budget, as well as an academic coach for every site to ensure
site-based professional learning and high-quality 21
st
century instruction for all students is taking
place.
The LCAP is intended to be a comprehensive planning tool, accordingly, in developing
goals, specific actions, and expenditures. There are eight areas identified as educational
priorities. For the purpose of this research paper, we will specifically address the district goals
EVALUATING THE IMPLEMENTATION OF 21
ST
17

for the inclusive classroom because the site selected for this study is 100% inclusive instruction.
Specifically, this study will focus on the first goal of preparing teachers to plan and implement a
rigorous CCSS aligned, 21
st
century skills-based curriculum. Understanding where the district is
in meeting the first goal will help the district approach data from an informed position. Over the
course of the 2015-2016 school year, the district has worked collaboratively with the LCAP
advisory group to inform, review, and revise the goals and action steps that are in the LCAP.
This group of stakeholders include, parents, staff, and community members who have met
several times throughout the school year. In addition to the LCAP advisory team, the district
sought input from the English Language Advisory Committee, The District English Language
Advisory Committee, District Parent Teacher Association Council, certificated and classified
bargaining units, and management teams. The LCAP must be updated each year so that (1)
programs and goals in the LCAP align with the district budget, (2) that progress is made for each
stated goal and assesses the effectiveness of actions and services provided, (3) updates and
changes can be made in the LCAP, for the following year, based on the review and assessment.
These updates are necessary towards positively affecting student performance.
Description of Stakeholder Groups
To make inclusive education happen in the 21
st
century classroom, key stakeholders
contribute and benefit from the inclusive classroom, including the classroom teacher, special
education/resource teachers, parents, school administrators, and the students. In 21
st
century
inclusive education, both the classroom teacher and resource teacher are expected to provide the
necessary support for the identified students in the program, whether it be a learning or language
need. Teachers’ input are vital in offering direct feedback on the instructional needs of students.
Parents also play crucial roles as a key stakeholder. They have an opportunity to voice their
EVALUATING THE IMPLEMENTATION OF 21
ST
18

opinions or give comments that can change or affect their child’s education. Administrators as
stakeholders are particularly important. Through assessment and data analysis, school and
community observations and feedback, administrators evaluate and determine the impact of the
school program and collaborate with all primary and secondary stakeholders to address ongoing
needs. Finally, students benefit extensively from the inclusive classroom. Udeme (2016)
explains that this educational format provides equitable opportunities for students to receive
effective educational services, allows all children to belong to the same learning community, and
aids in developing acceptance and friendship of classmates. According to the school district
LCAP, stakeholder engagement and input sessions from each school site via School Site Council
(SSC), staff meetings, English Learner Advisory Committee (ELAC) and the Parent Teacher
Association (PTA) should result in clear priorities for all students. Stakeholders should continue
to validate current goals and determine clear action steps needed to meet those goals. The
success of inclusive education lies in the collaborative efforts of all stakeholders.
Stakeholder Group of Focus
While collaboration by many educators takes place at CVES, classroom teachers were
chosen as the focus stakeholder group for this study. If the teachers at CVES do not build on
their confidence, knowledge, and instructional practices, students in the inclusive setting will
continue to be underserved in the era of 21
st
century Education.
Stakeholder Performance Goals
While the joint efforts of all stakeholders will contribute to the achievement of the overall
organizational goal of implementing a rigorous CCSS aligned, 21
st
century curriculum, it is
important to evaluate where the CVES teachers are currently regarding their performance goal.
Therefore, the stakeholders of focus for this study will be all CVES teachers. The stakeholders’
EVALUATING THE IMPLEMENTATION OF 21
ST
19

goal, supported by the school administrator and instructional coach, will be for teachers to
engage in professional development and implement a standards-aligned 21
st
century skills-based
curriculum in the inclusive classroom. Professional development and training is implemented in
the school’s Single Plan for Student Achievement document to certificated and classified
teachers. Instructional strategies and differentiation trainings to better support the inclusive
classroom, refers to a methodical style to knowledge and development to progress individual,
team, and organization effectiveness (Goldstein & Ford, 2002; Aguinis and Kraiger, 2009).
Table 1
Organizational mission and global goal
Organizational Mission
To develop a well-rounded child who is proficient in all academic areas, including the English
language and who has an appreciation for diversity, a positive self-image, and an ability to
incorporate respect, perseverance, and resiliency into their everyday lives.
Organizational Global Goal
Learning for All: Implement and assess a rigorous, standards aligned, 21
st
century educational
program that promotes the development of the whole child.
Stakeholder Goal
Teachers will engage in professional development and implement a standards-aligned 21
st

century skills-based curriculum in the inclusive classroom.

This study seeks to understand the extent to which teachers know how to support the
academic development of 21
st
century skills for all students in the inclusive classroom. It is
important to understand the current and lack of knowledge or skills preventing CVES from
EVALUATING THE IMPLEMENTATION OF 21
ST
20

meeting its goal. Clark and Estes (2008) explain that increasing knowledge, skills, and
motivation are key contributors in the success of any new world organization. Their research
found that organizations will continue to face complex problems and change, and that there is no
one single universal method to improve performance skills, from poor morale to weak skills.
However, studying whether people know how to achieve their performance goals and have
adequate knowledge, determines the type of support stakeholders will need for the organization
to positively flourish (Clark & Estes, 2008). Analyzing the performance of stakeholders in an
organization includes the course of exploring what knowledge and skills influence the ability of
key players to complete organizational goals (Rueda, 2011). The primary purpose is the positive
transfer of training towards potentially leading to changes in work performance (Goldstein &
Ford, 2002; Grossman & Salas, 2011). This study seeks to learn about the teacher’s knowledge
and motivation along with organizational factors that are impacting the school’s ability to meet
their 21
st
century learning goal. This information can be used to better support the teachers and
potentially lead to change in the right direction.
Review of the Literature

This literature review will examine the root causes of authentic learning in the 21
st

century and the challenges educational professionals face in closing the academic achievement
gap of students in the inclusive classroom at Canyon Valley Elementary School. The review
begins with general research on the importance of improving instructional practices through
authentic instruction. This is followed by an overview of literature on inclusive education,
blended learning, and teacher challenges from grades kindergarten through sixth grade. The
review will present a discussion on research based instructional practices for students in the
inclusive classroom. This section includes current research on teacher preparation and
EVALUATING THE IMPLEMENTATION OF 21
ST
21

professional development practices in the elementary school setting in the United States.
Following the general research literature, the review turns to the Clark and Estes Gap Analytical
Conceptual Framework and specifically, knowledge, motivation, and organization influences on
the classroom teacher’s ability to foster a learning environment that includes utilizing authentic
instruction that meets the academic needs of all students in the inclusive classroom.
21
st
Century Education
All students, regardless of their economic situation, must receive an education that will
prepare them for a successful future. Over the past several decades, employment demands, and
economic shifts have changed the skills businesses seek from their employees. The Partnership
for 21
st
Century Skills Organization (2011) designed the framework for American students to be
21
st
century learners for their future, and globally prepared citizens. Twenty-first Century Skills
and Learning are the fundamental skills that students, kindergarten through twelfth grade, must
master to ensure their future success in the workforce. These skills include, critical thinking,
problem solving, communication, and collaboration with the intent that once students exhibit
these skills through information, media, and technology, they will be well prepared to be
effective working citizens. Yet, while other nations are years ahead incorporating 21
st
century
education, 15-year old students in the United States rank 14
th
in literacy and 25
th
in math against
countries such as South Korea, Finland, and Singapore (National Center for Education Statistic,
2009). American students are underprepared in comparison to students in other countries and do
not have the capacity to be contenders in future job markets (Wagner, 2008; Girlando, 2013). In
his study, Girlando (2013) found that, in order for 21
st
century skills to be effective, there needs
to be a shift in teaching practice. The researcher believes that clear communication regarding
instructional effectiveness in 21
st
century skills and learning is instrumental to the success of
EVALUATING THE IMPLEMENTATION OF 21
ST
22

both the student and the teacher. Abilock et. al., (2013) indicated that, while professional
development remains to be the heart of teacher improvement, effective professional development
begins within the school. Rotherham & Willingham (2009) state that it is crucial for educators in
America to become master practitioners in 21
st
century education so that students in the United
States are receiving highly effective instruction.
The Partnership for 21
st
Century Skills (2009) redirected education on the significance of
students conceptualizing skills and content knowledge and applying the knowledge. Mastery of
core subject areas, such as English, foreign language, visual and performing arts, math, science,
and geography are vital to student success. However, it is crucial for educational leaders to look
more closely at the way teachers are unfolding the core content and combining 21
st
century
content within their curricula (Girlando, 2013). Content include themes such as global
awareness, entrepreneurial literacy, civic literacy, health literacy, and environmental literacy
(The Partnership for 21
st
Century Skills, 2009). Developers of the framework believe that when
schools provide the building blocks for students, they will be more engaged in the learning
process and have a better opportunity to thrive in the global economy.
Still, many educational researchers and leaders argue that the skills teachers are expected
to have, to teach students in the 21
st
century era, are not new or different concepts than before.
Dewey (1944) explains that instructional strategies and techniques are best gained through
experience. Experience with problem solving, critical thinking, collaboration, and
communication enrich the 21
st
century learning classroom and prepare students for the future.
Larson & Miller (2011) suggest that teachers in the classroom need to make the most of the
opportunities they receive to engage students in these experiences by developing lessons that
incorporate technology and focus on problem solving skills. Borsheim (2008) believes that
EVALUATING THE IMPLEMENTATION OF 21
ST
23

engaging students with the authentic 21
st
century experience will offer them a greater sense of
meaning and value towards what they are learning. And while it is still important to teach
academic content, 21
st
century education helps students learn how to become creative innovators
in what they are learning. Larson & Miller (2011) express that through applying, analyzing,
creating, and evaluating new knowledge, students are engaging in high-order thinking and are
learning how to transfer the process outside the world and beyond multiple choice tests.
21
st
Century Education in the Inclusive Classroom
Twenty-first century learning and skills is a model in education that shifts the way society
has commonly viewed teaching. The teacher-led approach has moved to a cooperative learning
Figure 1
model where a student is encouraged to use their voice to express their understanding of what is
being taught. In 2002, The Partner for 21
st
Century Skills (P21) was developed by the U.S.
Department of Education and a collection of companies to engrain more twenty-first century
EVALUATING THE IMPLEMENTATION OF 21
ST
24

skills into education. The team designed a framework to help guide educators to meet student
outcomes. Most importantly, the 21
st
century classroom invited all students of learning abilities
to grasp the goals set in the framework. Figure 1 provides the framework for the P21 design.
Figure 1. twenty-first century student outcomes and support
Determining whether all students of all learning abilities were capable of successfully
meeting 21
st
century learning goals was not a new idea. During the 1990’s, Newman (1995)
investigated 21
st
century education and developed standards for what he coined as, authentic
instruction and assessment and evaluated the result of teaching higher-order thinking, in-depth
conversation, deep knowledge, and relevance to the outside world. In their study, they examined
24 elementary and secondary schools and evaluated teacher instruction and student work, and the
access diverse students have to such instruction (Newman, Marks, & Gamoran, 1996). Where
authentic instruction was employed, student performance improved by 30
th
to 60
th
percentile.
The Research Institute on Secondary Education Reform (RISER) furthered the study and
researched what would occur if students with learning needs experience similar benefits from
authentic instruction (Hanley-Maxwell & Phelps, 2003). Their investigation included selecting
four secondary schools that showed a “high degree of inclusive practices” and highlighted both
authentic learning and assessment. In investigating inclusive practices, the RISER study sought
to focus on whether or not the schools’ core beliefs reveal that students with all abilities have
equal access to authentic instruction through adequate support strategies and resources. The
RISER study revealed a correlation between authentic learning and academic success. Their
research proved that authentic instruction practices were great indicators of student outcomes
regardless of learning abilities. However, the most important factor revealed from the RISER
study was the impact of instructional quality to student work in the inclusive classroom.
EVALUATING THE IMPLEMENTATION OF 21
ST
25

Legal Implications in Inclusive Education
The legislative influence on academic preparedness and rigor for all student had
undergone a paradigm shift when American education introduced the implementation of the
Individuals with Disabilities Act (IDEA) of 1975. This law stated that children with disabilities
have the legal right to receive an education in the least restrictive environment (LRE) (Villa &
Thousand, 2003). By the early 1990’s, the interpretation progressed into the approach now
known as inclusion. This method relies on general education’s principles and practices as being
the first choice for all learners with supplemental support and aids; rather than removing students
from their classroom to receive those services. Tremendous problems exist among schools,
districts, and states, particularly when the nature of inclusion varies. In some schools, inclusion
means the physical or social inclusion of students with disabilities in regular classrooms. In
other schools, it means modified instruction and assessment practices to engage students in core
curriculum towards a positive experience and successful learning outcomes. While both
methods, in tangent, work best together to support SWD, a differentiated approached to support
individual learners and improved 21
st
century inclusive pedagogy is necessary for students to
really flourish.
Addressing Teacher Readiness
Presently, meeting the academic needs of all children in the 21
st
century education
inclusive classroom settings pose many challenges for educators who indicate they lack the
knowledge and skills on mainstreaming and including children with special needs. Busby et. al.,
surveyed 31 graduate students in southwest Alabama to assess the level of teacher preparation
graduate students received to work with students in the inclusive setting. In their findings, the
EVALUATING THE IMPLEMENTATION OF 21
ST
26

participants expressed the dire need for colleges to incorporate classes of the most current
teaching practices for children with various learning needs. Participants also believe,
identification and early intervention are crucial for teachers-general and special educators alike
to acquire the adequate training and support services to aid all students of varying learning needs.
The researchers noted that instructors reported feeling poorly prepared to meet the academic
needs of children in the inclusive setting. The researchers explain that these issues are a
challenge to teachers, especially from rural areas, where the incidence of learning disabilities are
found to be lower resulting in teachers who have minimal experience with students and limited
access to training, funding, and resources. Furthermore, general educators, nationally, have only
enrolled in courses when necessary, therefore, have little specialized training in the field.
Individualized Education Plan
Wrestling to meet the academic needs of students is not the only concern within the
inclusive setting in the 21
st
century classroom. Wernsing (2014) found half a million students in
the United States are served under the category of emotional disturbance (ED) and often
experience high rates of academic failure. The researcher concluded that ED students
significantly benefit from Individualized Education Plan (IEP) based on effective methods
implemented by well-trained teachers. In Wernsing’s investigation, a variety of best practices
have been identified in the SPED setting to provide academic support for ED students.
However, research has been limited to the general education inclusive setting on effective
strategies that best serve ED students when they are included. Teachers in SPED have also
shared their beliefs that better progress could be made by students with autism, ED, or mild
disabilities more often when placed in special classes, therefore, removing them from general
education classrooms (Cox, 2009). Teacher education for both special education teachers and
EVALUATING THE IMPLEMENTATION OF 21
ST
27

general education teachers play vital roles in preparing teachers for the challenges of educating
students with learning needs in the inclusive classroom. Studies have investigated teacher
attitudes in regards to inclusive settings and point to the necessity of garnering a positive attitude
towards the practice of inclusion and all students with diverse learning needs.
Teacher Self-Efficacy
In addition to the lack of 21
st
century teacher training, regular education teachers have
also been apprehensive towards applying inclusive instructional practices because of the lack of
time given to plan and supplement the curriculum in order to accurately accommodate students
in their classrooms. Jordan and Stanovich (2001) and Van Reusen et. al., (2000) found that
teachers with high confidence levels have a higher sense of self-efficacy, while teachers with
lower confidence levels feel unable to address the dynamics of the inclusive classroom.
Although there is insufficient empirical evidence to address the needs of all children in the
general education classroom, Singh (2007) found evidence that a majority of general education
teachers believe they are ill prepared for inclusive education. To address teacher confidence, it is
imperative that a training would better prepare general educators for the inclusion setting in the
21
st
century classroom.
Mainstreaming in the 21
st
Century Classroom
In an effort to provide greater access and progress, for all students, school districts have
favored the inclusive setting as the means to meeting their 21
st
century goals. Therefore,
challenging frustrated general education teachers, due to their perception of incompetence
towards meeting the needs of all students (Ashley, 2009). Henceforth, when teachers are not
well prepared to provide an enriching learning environment for all learners, the educator’s lack
of 21
st
century instructional pedagogy and confidence can negatively impede the teacher’s ability
EVALUATING THE IMPLEMENTATION OF 21
ST
28

to foster a nurturing classroom environment. As the percent of students will learning needs
continue to rise in inclusive class settings, teacher training must also increase towards
establishing an effective academic environment for the success of all students in the classroom.
As a whole, students with identified specific learning needs trail considerably behind their
general education peers in graduation rates, standardized measures of achievement,
postsecondary participation rates, and employment (Blanton et. al., 2011). As a result, leaders of
school districts often face difficulty when trying to provide the necessary supports to meet the
instructional needs of all students while ensuring increased student learning, for all, in the
inclusive classroom (Kniveton, 2004).
Stakeholder Knowledge, Motivation, and Organizational Influences
The purpose of this section is to review relevant knowledge and motivation related
literature for 21
st
century inclusive education practices at Canyon Valley Elementary School.
CVES shares an internal problem to see test score gains among its students in the inclusive
classroom setting.
To access the extent to which school site stakeholders know how to support the academic
development of students in the inclusive classroom, it is important to understand the current and
lack of knowledge or skills preventing the performance gap to close. Clark and Estes (2008)
explain that increasing knowledge, skills, and motivation are key contributors in the success of
any new world organization. Their research found that organizations will continue to face
complex problems and change, and that there is no one single universal tonic to improve
performance skills, from poor morale to weak skills. However, studying whether people know
how to achieve their performance goals and have adequate knowledge, determines the type of
support stakeholders will need for the organization to positively flourish (Clark & Estes, 2008).
EVALUATING THE IMPLEMENTATION OF 21
ST
29

Analyzing the performance of stakeholders in an organization includes the course of exploring
what knowledge and skills influence the ability of key players to complete organizational goals
(Rueda, 2011). The primary importance through those finding is the positive transfer of training
towards potentially leading to changes in work performance (Goldstein & Ford, 2002; Grossman
& Salas, 2011).
Literature reviewed in this paper will focus on certain aspects of knowledge and skills
that influence the academic achievement of students in the inclusive classroom. Alongside the
literature review will be an investigation of the forms of knowledge needed to reach the goal.
Knowledge influences. In this section, literature for knowledge influence will be
reviewed relevant to the stakeholder goals of CVES.
Mayer (2011) wrote, “Knowledge is at the heart of learning, instruction, and
assessment.” It is worthwhile to distinguish the types of knowledge (factual, conceptual,
procedural, and metacognitive) towards the purpose of learning before proceeding (Mayer,
2011). Researchers, Anderson and Krathwohl (2001), describe the four types of knowledge, as
followed: (a) factual – the basic elements to be acquainted with a discipline or solve problems in
it; (b) conceptual knowledge – the interrelations among the basic fundamentals within a larger
construct that allows them to function collectively; (c) procedural knowledge – consists of the
understanding to do something, having methods of inquiry, and measures for using skills,
algorithms, techniques, and methods; and (d) metacognitive knowledge – over-all cognitive
knowledge as well as an awareness of one’s own cognition (Anderson & Krathwohl, 2001).
Distinguishing knowledge is critical in evaluating the causes of gaps. For CVES faculty,
knowing conceptual and procedural types of information will best drive the organization towards
EVALUATING THE IMPLEMENTATION OF 21
ST
30

achieving their goal. Specific literature supporting the types of knowledge will be discussed
further in the next section.
Understanding and Implementation of Individualized Educational Plans. Conceptual
knowledge is important when understanding patterns and relationships of ideas, topic, or
themes. In this case, for staff at CVES, raising test scores of students and meeting 21
st
century
education goals in the inclusive classroom begins with comprehension of executing the goals set
in a student’s Individualized Education Plan (IEP). IEP’s and IEP meetings are convened to
develop individual service execution plans for students (McCombs-Tolis, 2002). The IEP has
been called the heart of imparting free appropriate public education (Rotter, 2014). As the
foundation of the Individuals with Disabilities Education Improvement Act (IDEIA) of 2004, no
document is more crucial for ensuring a compliant program design, implementation, monitoring,
and enforcement of the law. The IEP serves as a roadmap for educational services (Conroy, Yell,
& Katsiyannis, 2008; Diliberto & Brewer, 2012; Gartin & Murdick, 2005). School districts are
accountable for ensuring that teachers are well-versed of their duties to review and implement
the IEP (NICHCY, 2010).
According to the school district Local Control Accountability Plan (LCAP), stakeholder
engagement and input sessions from each school site via School Site Council, staff meetings,
English Language Leaner Advisory Committee (ELAC) and the Parent Teacher Association
(PTA) should result in clear priorities for all students. Stakeholders should continue to validate
current goals and many action steps. In short, all sections of society who have a stake – directly
and indirectly – in the success of inclusive education lies in the collaborative efforts of all the
stakeholders. Through a student’s IEP and their IEP meeting, teachers are expected to deliver
21
st
century instruction based on the objectives and goals written in these documents. General
EVALUATING THE IMPLEMENTATION OF 21
ST
31

education teachers, in the inclusive classroom, are ever more responsible for the implementation
and accommodations for all students of various learning needs (Neal, 2012). A study found that,
teachers’ participation in individualized education plan (IEP) meetings is important, because
those teachers are responsible for the implementation of IEP meeting decisions. This study
performed by Neal (2012) conclude that an understanding of teachers’ cognizance of
accommodations and contribution in the decision-making process, will give district personnel
and policy makers suitable training and sufficient resources for schools’ and teachers’ provision
of accommodations for all. To make meaning of conceptual knowledge relies on the individual
staff member to make meaning of themes, ideas, and topics to determine the relationship
between multiple pieces of information (Krathwohl, 2002; Rueda, 2011). As a curricular
objective to better the academic achievement of low performing students in the inclusive setting,
conceptual knowledge of the principles and structures of a student’s IEP helps teachers
understand how the process works.
Professional Development. Through the collective effort of administrators, teachers, and
district leaders, gathering and organizing data will need to be synthesized and prioritized to make
decisions (Marsh et al., 2006) and professionals must know what to do with it (Weiss, 2007). The
9
th
principal of leadership indicates that organizational effectiveness increases when leaders are
knowledgeable about and are consistently learning about themselves and their business. Past
research has identified several problems with IEP implementation. Procedural knowledge is
vital for faculty to know how a skill or a task is performed in respect to their job responsibilities
(Krathwohl, 2002; Rueda, 2011). Research has found that operational inclusive schools
recognize that a staff pledge to generate and preserve a climate beneficial to learning for all
students entails policies, leadership and long-term professional development (Rouse and Florian,
EVALUATING THE IMPLEMENTATION OF 21
ST
32

1996). Having procedural knowledge makes execution of IEP documents, much easier, because
staff will have the knowledge of subject-specific skills and techniques for determining when to
use appropriate procedures (Krathwohl, 2010).
When IEP’s are written, and delivered to classroom teachers, teachers have indicated
feeling secluded and solely accountable for performing directives that were established without
attention to present weaknesses to the execution of them (Carrington,2000). According to
research, the ability of the employee to attain and transfer their knowledge and skills is important
to the achievement of the individual, as well as, the organization (Alexander, Schallert, &
Reynolds, 2009; Carpenter, 2012; Mayer, 2011). Often, policies become adopted without the
formal training necessary to see them in fruition. Thompson and Ross (2000) cautioned that
student diversity and an emphasized shift from knowledge worker will involve expert teachers
who are projected to guarantee all students learn and perform at appropriate levels. Robinson
and Carrington (2000) conducted a study of teachers in Queensland, Australia. Interviews of
teachers revealed that, among four areas in need, professional development found to impact
teacher knowledge and practice in improving student outcomes the most. In conclusion, this
study shows that professional development accepted in a climate of school reform and
collaboration boosts a teacher's sense of ownership and relevance from productive professional
development trainings. Elmore, (2002), believes that improved student learning is the cause of
professional developments aligned specifically to the school’s mission statement. Additionally,
Elmore (2002) illustrates that an effective training or professional development design includes
large amounts of collaboration between teacher peers.
Table 2 below provides the organizational mission, organizational goal, and information
specific to knowledge influences, knowledge types, and knowledge influence assessments. As
EVALUATING THE IMPLEMENTATION OF 21
ST
33

Table 1 indicates, two knowledge influences will be used to gain insight about the knowledge
that classroom teachers have in regards to 21
st
century instructional needs in the inclusive
classroom.
Table 2
Knowledge Influences, Types, and Assessments for Knowledge Gap Analysis
Organizational Mission
To develop a well-rounded child who is proficient in all academic areas, including the English
language and who has an appreciation for diversity, a positive self-image, and an ability to
incorporate respect, perseverance, and resiliency into their everyday lives.
Organizational Global Goal
Learning for All: Implement and assess a rigorous, standards aligned, 21
st
century educational
program that promotes the development of the whole child.
Stakeholder Goal
Teachers will engage in professional development and implement a standards aligned 21
st

century skills-based curriculum in the inclusive classroom.
Knowledge Influence Knowledge Type
(i.e., declarative
(factual or
conceptual),
procedural, or
metacognitive)
Knowledge Influence Assessment
Teachers will need to know what it
means to implement 21
st
century
skill-based curriculum.
Conceptual Teachers were interviewed by
principal, academic coach, and
resource specialist to determine if
teachers had the knowledge to
differentiate instruction to meet
the needs of the diverse learning
needs in the classroom.
Teachers will need to know how to
support all students in 21
st
century
skills.
Procedural Teachers were given a
Professional Development on the
current ELA and Math Common
EVALUATING THE IMPLEMENTATION OF 21
ST
34

Core aligned programs which
contain 21
st
century instructional
strategies. Evaluations of the
program began as early as the
second week of school by informal
walk throughs from principal and
academic coach.
Teachers will need to know how to
execute the 21
st
century strategies
for differentiated instruction in the
inclusive classroom.
Procedural Teachers were given a list
specifying which data is expected
to have before the students’ IEP
meeting.

Motivation
While 21
st
century instructional training and professional development is a part of the
school’s goals, evaluation of the effectiveness supports our judgement and motivation on
whether something is worthwhile (Clark and Estes, 2008). Understanding the motivational gaps
that prevent increased student achievement is pertinent to seeing any change. For changes to
occur, it requires stakeholders to obtain a level of motivation. High quality student performance
is contingent on first-class school leadership. School site stakeholders, such as classroom
teachers, must operate as instructional leaders, where their every action and decision is focused
on the goal of making sure that all students achieve high academic standards. In general,
motivation is the internal, psychological process that gets people going, keeps them moving, and
motions to get the job done (Pintrich and Schunk, 1996). Research suggests that members of
faculty are motivated to demonstrate skills at high levels to achieve personal and organizational
goals, however motivation determines when a person starts, continues, and completes a task
(Rueda, 2011). For teachers at CVES, an increased level of motivation can be the difference in
closing performance gaps through removing needless “demotivators” and exchanging them with
motivational positive support (Clark and Estes, 2008).
EVALUATING THE IMPLEMENTATION OF 21
ST
35

Although there are numerous motivational theories, for CVES, two specific theories are
crucial to the academic success of students in meeting 21
st
century goals in the inclusive
classroom and raising CAASPP test scores. First, an examination of beliefs regarding teacher
ability, effort, and 21
st
century instructional capacity will be the focus through the theory of
attribution. For example, teachers should feel that low English Language Arts and Literacy
scores of students in the inclusive classroom, is a problem. In the classroom, the understanding
teachers have about the causes of low test scores in the past could positively influences their
ability to control what happens in the future (Anderman and Anderman, 2009). Another theory
of focus will be the theory of self-efficacy, which is the theory that helps foster the outcome one
expects (Pajares, 2009). For example, teachers need to feel they are effective educators and have
the professional capacity to work with all students and implementing 21
st
century skills in the
inclusive classroom.
This literature review will focus on attribution and self-efficacy that impact the teacher’s
motivation to improving CAASPP test scores of all learners in the inclusive classroom.
Attribution theory. Many theories have been made to understand how human
motivation works. Clark and Estes (2008) have discovered that motivation results from our
experiences and beliefs, interaction with coworkers, and our prospect for being effective.
Attribution theory deals with how the social observer uses evidence to explain outcomes for
events. It examines what information is collected and how it is brought together to form a causal
judgment (Fiske, & Taylor, 1991) Attribution theory gives an important method for examining
and understanding motivation in school settings (Anderman and Anderman, 2009). Two
constructs of this theory include the internal locus and external locus. Weiner’s Model of
attributions contains three casual dimension that explains the characteristics of attributions. The
EVALUATING THE IMPLEMENTATION OF 21
ST
36

locus, being one of the dimensions, refers to whether the cause of the event is perceived as
internal or external to the individual (Anderman and Anderman, 2009). The internal locus refers
to whether causes are perceived to be contingent on their own behavior or belief’s in one’s
ability. For example, Bandura (1977, 1997), explains that people who are positive and believe
they are capable and effective will achieve significantly more than those who are just as capable
but tend to doubt their own abilities. External locus refers to the causes outside of the
individual’s control. For example, our motivation is gauged by whether we believe the
environment prepares us with goals and instructional means that ensues in a reasonable amount
of effectiveness (Clark and Estes, 2008).
Teachers and Internal Locus. Among the various constructs of the attribution theory,
the only theory controllable by the person is the internal locus (Anderman and Anderman, 2009).
When people are doubtful of their effectiveness, they will achieve significantly less than their
peers who uphold greater confidence in their abilities (Clark and Estes, 2008). A study
performed by Busby et. al., (2012) surveyed 31 graduate students in southwest Alabama to
assess the level of teacher preparation graduate students received to work with children of
various learning needs. In their findings, the participants expressed the dire need for colleges to
incorporate classes of the most current teaching practices. Participants also believe,
identification and early intervention are crucial for teachers-general and special educators alike
to acquire the adequate training and support services to aid children with targeted instructional
support. The researchers noted that instructors reported feeling poorly prepared to meet the
academic needs of children in the inclusive setting. The researchers explain that these issues are
a challenge to teachers, especially from rural areas, who have minimal experience and limited
access to training, funding, and resources. Furthermore, general educators, nationally, have only
EVALUATING THE IMPLEMENTATION OF 21
ST
37

enrolled in differentiation courses when necessary, therefore, have little specialized training in
the field. Jordan and Stanovich (2001) and Van Reusen et. al., (2000), found that teachers with
high confidence levels have a greater sense of self-efficacy while teachers with lower confidence
levels feel unable to address and support all learners the inclusive classroom.
Self-Efficacy Theory. Self-efficacy is the theory that people hold ideas about
themselves which affect their day-to-day functioning and capabilities (Pajares, 2009). According
to Bandura (1991), self-efficacy is a foundational component of motivation. This means that
performance outcomes depend largely on the judgments that individuals hold about their abilities
to learn or take action. In self-efficacy, the environment impacts the behavior. For CVES
teachers, faculty need to begin believing in their effectiveness and skills to implement 21
st

century pedagogy in the inclusive classroom. Two constructs of self-efficacy theory that will be
focused on are individual and collective self-efficacy. Individual self-efficacy shares both a
personal and social construct. For example, confident individuals foresee successful outcomes,
in the same way as students who carry confidence in their social skills expect successful social
encounters (Pajares, 2009). In collective efficacy, the groups collective beliefs about the abilities
of their learners can potentially affect student outcomes. For example, Pajares (2009) found that
a strong collective sense of efficacy and a shared belief that students have the capacity to learn,
can result in a school environment that is conducive to high-academic achievement and teachers
focused on enhancing the lives of their students.
Teachers and Individual Efficacy. Teacher efficacy is the belief that they can influence
how well students learn, regardless of the student’s level of motivation (Guskey and Passaro,
1994; Guskey, T.R. and Passaro, P.D. 1994). The importance of teacher efficacy arises from the
recurrent quality that high-performance aptitude produces a new mastery experience, which then
EVALUATING THE IMPLEMENTATION OF 21
ST
38

influences the efficacy beliefs. (Tschannen-Moran et al., 1998). In an effort to provide greater
access and progress in general education curriculum, school districts have embraced the idea of
inclusive classrooms. Consequently, this movement has significantly challenged frustrated
general education teachers due to their perception of incompetence towards meeting the needs of
students with various learning abilities (Ashley, 2009). Ashley performed a mixed-methods
approach to explore differences in teachers’ self-efficacy beliefs based on identified students. In
her observation, the researcher found that teachers with high self-efficacy provided their full
focus to students needing assistance, while teachers with low self-efficacy appeared to be
emotionally distant and displayed little sense of humor. The study also showed variations of
student behavior. High self-efficacy teachers’ students behaved well, while low self-efficacy
teachers were generally negative and provided a disclaimer if students behavior differed from a
typical situation. Henceforth, when teachers are not well prepared to provide an enriching
learning environment for all learners, the educator’s lack of 21
st
century instructional pedagogy
and confidence can negatively impede the teacher’s ability to foster a nurturing classroom
environment. As the percent of students with various learning needs continues to rise in
inclusive class settings, teacher training in 21
st
century instruction must also increase towards
establishing an effective academic environment for the success of all students in the classroom.
Understanding individual self-efficacy amongst staff at CVES will support teachers to be more
confident in their instructional practice that support their classrooms.
Understanding the gaps in knowledge and motivational theory amongst stakeholders are
stepping stones to closing the gap and increasing student academic outcomes. Table 3 shows the
organizational mission, global goal, stakeholder goal, motivational influences and motivational
influence assessments identified in this literature review.
EVALUATING THE IMPLEMENTATION OF 21
ST
39

Table 3
Motivational Influences and Motivational Influence Assessment
An Organizational Mission
To develop a well-rounded child who is proficient in all academic areas, including the English
language and who has an appreciation for diversity, a positive self-image, and an ability to
incorporate respect, perseverance, and resiliency into their everyday lives.
Organizational Global Goal
Learning for All: Implement and assess a rigorous, standards aligned, 21
st
century educational
program that promotes the development of the whole child.
Stakeholder Goal
Teachers will engage in professional development and implement a standards aligned 21
st

century skills-based curriculum in the inclusive classroom.
Assumed Motivation Influences

Motivational Influence Assessment

Attribution – Teachers need to believe that all
students are capable of high outcomes in the
21st century based curricular program.
Written survey item “Student achievement is
strongly influenced by the amount of effort I
put into the lesson.” (strongly disagree –
strongly agree)
Interview item:
“What are some of the causes for learners’
success or failure in your class?”

Self-Efficacy – Teachers need to feel they are
effective educators and have the skills to
teach all learners in the mainstream
classroom.
Written survey item “I feel confident about
my ability to differentiate 21
st
century
instruction for all students” (strongly
disagree-strongly agree)

Interview item:
“How do you feel about
your ability to
differentiate instruction?”

EVALUATING THE IMPLEMENTATION OF 21
ST
40

Organizational
While 21
st
century instructional training, professional development, evaluation of
effective support systems, and understanding of the knowledge and motivational gaps are critical
to the success of all schools; the culture of an organization is the foundation of the order and
rules which are lived by (Schein, 2010). As changes in an organization attempt to take root,
possible spirals of dysfunction commonly occur resulting in failure. According to Schneider
et.al., (1996) changes in structure and systems are only effective to a certain degree. Change
begins with the people in the organization and the altering of the fundamental psychology. For
the CVES organization to flourish, organizational gaps will need reworking, including
recognizing and remedying the lack of processes and materials that can prevent the achievement
of the performance gap (Clark and Estes, 2008). While acknowledging and improving
organizational processes and material resources are vital, Clark and Estes (2011) explain that
there are more elements that involve a culture shift. Schein (2010) believes that changes in any
organization require a connection between culture and leader, and ultimately manipulated by the
organization’s leader.
In a school or classroom setting, Rueda (2011) explains that there are organizational
factors and dynamics that can influence the behavior of a school staff. Rueda believes that there
are always reasons why morale suffer and employees will behave the way they do. In fact,
educational leaders take in a large deal of criticism when schools fail to make good on their
promises to the school community (Fiore, 2011); however, truth is, the effectiveness of a school
relies heavily on all school stakeholder, and not just the leader, and their ability to achieve the
school’s mission (Rueda, 2011). For teachers at CVES, organizational processes and clarity of
21
st
Century goals and communication can be the difference in closing organizational gaps
EVALUATING THE IMPLEMENTATION OF 21
ST
41

through providing the right types of performance support necessary to see improvement (Clark
and Estes, 2008).
Student Support. For staff at CVES, seeing a shift in the achievement of all learners in
the inclusive classroom requires there to be a level of understanding between the roles of primary
teacher, student support teacher, and administration. Researchers have suggested a need for
qualified teachers and a focus on preparation and teacher retention as areas of concern in
producing positive student outcomes. Most importantly, Lockwood (2004) found that
administrative support may be the one component to positively impact student outcomes and
impact teacher decisions. Lack of administrative support was also found to be a reason for
dissatisfaction amongst teachers who work in the inclusive setting (Lockwood, 2004).
According to a 1999 report, 11% of public school teachers were satisfied with their position
(National Center for Educational Statics, 1999; Lockwood, 2004) and lack of administrative
support as a leading factor to dissatisfaction with their positions (Kreger, 1999; Bruton, 2001;
Lockwood, 2004).
The need for understanding how organizations learn and accelerate learning is critical
today than ever before (Senge, 1990). While the old model of producing outcomes falls on the
figure at the top, thinking and acting is critical among all players in all levels of the organization.
However, while the top-down approach is highly frowned upon in modern times, leadership
literature explain that leaders play the role of designer, teacher, and steward. These new roles
are requiring administrators to involve staff in the decision-making processes where leaders are
learning and working together towards a shared vision with the school community (Senge,
1990).
EVALUATING THE IMPLEMENTATION OF 21
ST
42

Producing student outcomes is the mounting concern of any educator. This outcome,
however, cannot be fully achieved without the support and guidance of its school site leader.
Especially in the department of special education, students are often left abandoned not only by
their teachers, but the specific support expected from school administrators. In a questionnaire
conducted by Balfour (2001) a random sample of 240 special education teachers in Virginia; 140
surveys were returned to evaluate the perceived levels of administrative support by both
elementary and secondary special education teachers. The survey was also used to determine the
factors associated with the decision special educators make in staying or leaving the field. While
no significant differences were found in the amount of support received in either elementary or
secondary education, administrative support was listed as a primary factor in either boosting the
morale of the special education department or the decision for teachers to leave the profession.
Cultural Processes and Communication. When organizational procedures and processes
are unclear, it can create an organizational barrier for even the most motivated and exceptional
employees. Rueda (2011) explains that communication through the cultural models help define
what is customary and normal, such as the classroom, playground, office, and cafeteria where the
process and systems are important to solidify. Communication of procedures and processes, also
known as organizational communication, has existed for centuries and occurs to ensure that the
functions of an organizations are communicated (Sharma, 2015). Research found that effective
communication touches an extensive assortment of mechanisms in an organization and can help
in achieving more success for the organization. Effective internal communication can assist in
developing a strong atmosphere of motivation, trust, engagement, and sharing of thoughts and
ideas freely. Lack of effective communication may cause miscommunication and adversely
affect the smooth functioning of the organization (Sharma, 2015).
EVALUATING THE IMPLEMENTATION OF 21
ST
43

Clark and Estes (2008) explain that when organizational goals, policies, or procedures
conflict with organizational culture, performance problems are to be expected. CVES has
developed a plan through the Single Plan for Student Achievement (SPSA) which illustrates that
a committee will meet to review the expectations of the school and revise plans of 21
st
century
implementation of lessons and incentives for the entire school year. This group will meet three
times annually to assess, plan, and implement next steps in improving school culture and student
success and whether all necessary support is provided for all learners in the inclusive classroom.
Schneider et. al., (1996) define school culture as “the actions they have taken”, they refer to this
as the practices and behavior that connect climate and culture. In a study performed by Sharma
(2015) of 463 staff members in education, a solid constructive relationship and significant
correlation was found between overall communication and job satisfaction indicating that staff
members feel satisfied with communication in their workplace when task related information or
relational information is communicated well. Her research found that effective communication
strengthened trust, engagement, and sharing of thoughts and ideas freely.
Table 4 below provides the organizational mission, organizational goal, and information
specific to organizational influences. As Table 4 indicates, two knowledge influences will be
used to gain insight about the knowledge that classroom teachers have in regard to 21
st
century
instructional needs of all learners in the inclusive classroom.
Table 4
Organizational Influences
Organizational Mission
EVALUATING THE IMPLEMENTATION OF 21
ST
44

To develop a well-rounded child who is proficient in all academic areas, including the English
language and who has an appreciation for diversity, a positive self-image, and an ability to
incorporate respect, perseverance, and resiliency into their everyday lives.
Organizational Global Goal
Learning for All: Implement and assess a rigorous, standards aligned, 21
st
century educational
program that promotes the development of the whole child.
Stakeholder Goal
Teachers will engage in professional development and implement a standards aligned 21
st

century skills-based curriculum in the inclusive classroom.
Assumed Organizational Influences

There is a consensus from faculty expressing a lack of support from administration regarding
the support necessary to assist students in meeting 21
st
century standards.
There is a culture within the faculty that relies heavily on the student support services
department to provide primary instruction to students identified with learning needs through
the pull-out method.
Faculty are overwhelmed by teaching responsibilities that keep them from investing more time
and effort to understanding 21
st
century instructional practices necessary towards supporting
each student in the inclusive classroom.
There is no general school wide information on the co-teaching model which weakens the
effectiveness of inclusion.
Figure Y. Interactive Knowledge, Motivation, and Organizational Influences

EVALUATING THE IMPLEMENTATION OF 21
ST
45

The Clark and Estes (2008) Gap Analysis Conceptual Framework

The purpose of the conceptual framework is to address the system of concepts,
assumptions, expectations, beliefs, and theories that validate the research (Miles & Huberman,
1994; Robson, 2011; Maxwell, 2013). The conceptual framework provides the visual and
written narrative of the key factors to be studied and the relationship between the variables
among them (Miles & Huberman, 1994; Maxwell, 2013). A conceptual framework is the agent
that provides the road map of how concepts will be measured and presented in the methodology
section of a quantitative study (Merriam & Tisdell, 2016). This section is included to justify to
the readers why the research and study is important. Furthermore, the conceptual framework is
the concept or model already in existence that a researcher plans to study, to find out why things
are happening in the study, and the theory or phenomena of the subject you are investigating
(Maxwell, 2013). The conceptual framework informs the rest of the research design, assesses the
potential outcome of the study, helps to develop the research questions, select the study design,
and recognize any possible threats to the conclusion of the study (Maxwell, 2013).
The conceptual framework in this study was used as a guide to organize and examine the
knowledge, motivation, and organizational influences that effect the study’s main stakeholder
group and the project’s goal that direct the gathering and analysis of the data. To investigate the
role of educators, the conceptual framework utilized relevant research, as well as ideas outside of
the educational field that could potentially bring in new ideas and allow new approaches to be
integrated in the study (Maxwell, 213). To make 21
st
century inclusive education transpire,
important school site and district figures contribute to and profit from the achievement of the
inclusive program. In order for a school site to grasp the needs of 21
st
century implementation in
an inclusive classroom, it requires stakeholders to be appropriately equipped and involved.
EVALUATING THE IMPLEMENTATION OF 21
ST
46

To understand the degree to which school site stakeholders properly support the academic
development of all learners in the 21
st
century inclusive classroom, it is imperative to recognize
the lack of knowledge, skills, or organizational structures preventing the performance gap to
close. Clark and Estes (2008) explain that increasing knowledge, skills, and motivation are key
contributors in the success of any new world organization. Their research suggests that
organizations will inherently face complex problems and explain that there is no specific solution
to improve performance skills, from poor morale to weak skills. However, investigating whether
students can achieve their performance goals and have adequate knowledge, relies heavily on the
form of support stakeholders will need for the organization to prosper (Clark & Estes, 2008).
Analyzing the performance of stakeholders in an organization includes the course of exploring
what knowledge and skills influence the ability of key players to complete organizational goals
(Rueda, 2011).
Figure 2. depicts the collaborative influences of knowledge, motivation, and
organizational factors. In this diagram, organizational factors in blue represent the most
dominant controls over the focus stakeholder group and their goals. Cultural setting, models, the
leadership community, environment, instructional practice, academic rigor, expectations, and
accountability are the overlaying forces that often dictate the actions of the focus group.
Stakeholder knowledge and motivational influences are represented in red at the center of the
figure. Finally, the figure in green is the expected outcome of the two-primary section
interactively working together towards the organizational goal. Understanding the gaps in
knowledge and motivational theory amongst stakeholders are stepping stones to closing the gap
and increasing student academic outcomes.

EVALUATING THE IMPLEMENTATION OF 21
ST
47

Classroom Teachers
Factual, Conceptual, Procedural, and Metacognitive
knowledge and skills related to self-efficacy and
instructional improvement
District
Cultural Setting and Cultural Models
Leadership community, learning environment, instructional
practice, academic rigor, high expectations, and
accountability

Figure 2

Data Collection and Instrumentation
The primary purpose of this qualitative case study is to understand general education
teachers’ knowledge, perceptions and self-efficacy in implementing a 21
st
century-based
curriculum in the inclusive classroom. A pivotal goal of this study is to inform school and
district administration, student services staff and directors, para-educators, and classroom
teachers, in the area of professional development. These goals were intended to develop an
100% implementation of high quality instruction and
academic support in the inclusive classroom through
professional development and clearly outlined 21
st
century
teaching standards.
EVALUATING THE IMPLEMENTATION OF 21
ST
48

understanding of the factors that can contribute to higher levels of teacher self-efficacy and
improved instructional practice at CVES.
In this section, a description of the methodology and procedures utilized in the study is
provided, along with the rationale and justification of its design. The data collection and
participant selection processes are described as well as an explanation of how participants’ rights
and privacy were protected. At the conclusion of this section, a description of how validity and
reliability were ensured for this research study.
Qualitative Data Collection and Instrumentation
The methodological approach I used for my research is a qualitative design. Crewsell
(2014) describes five qualitative approaches to inquiry in which the researcher explores a case or
multiple case over time, through detailed, in-depth data collection which involves multiple
sources, such as interview, observations, audiovisual and documents. In order to capture more
fully the participants’ perceptions of the problem, as well as the human experience, multiple
sources of evidence was utilized to attain information from the participants, such as in-depth
interviews and probing questions during individual questioning (Hatch, 2002; Maxwell, 2013).
Within my study, purposeful selection was employed and followed Creswell’s (2014) five
goals illustrated by Maxwell (2007). The first goal to utilizing purposeful sampling, according
to Creswell (2014), is towards achieving proper representation of the setting. Because my study
involved few participants, random sampling would not have made sense since random sampling
involves a larger sample size. The second goal of utilizing purposeful selection was to capture a
more precise conclusion that represented the entire range of members, referred to as maximum
variation. The third goal addressed selecting appropriate members of the organization of study.
This involved members of the school community whom are responsible in fully implementing
EVALUATING THE IMPLEMENTATION OF 21
ST
49

21
st
century instructional strategies in the inclusive classroom. These particular members make
district or school wide decisions on how students will be educated in the inclusive classroom
setting. The fourth goal in purposeful selection was to highlight the comparisons between
settings or individuals in order to draw conclusions. Teaching lesson plans were examined to
find strengths and weaknesses, in lesson planning, to target possible gaps in instructional
practices that are not helping to increase academic gains. Lastly, the final goal in purposeful
selection was to develop close relationships to those who will genuinely and honestly provide
insight in the investigator’s research question. Those selected to participate in the study
understood the importance of addressing the organizational goal and are hopeful that this study
guides the organization to address the gaps found that are impacting the organizations ability to
meet their goal.
Interviews
For this qualitative study six general education teachers from one elementary school were
purposefully selected. The specific demographic of the participants involved in the study
revolved around the participants years of teaching. Teacher experience within the inclusive
environment varied with each participant. These interviews were informal, completed
individually, and audio recorded. According to Patton (2002), this type of approach is the most
open-ended and gives more flexibility to obtain the information through whatever emerges from
talking with the interviewee. The only weakness to this type of approach, Patton adds, is that is
can require a greater amount of time because the informal approach relies heavily on the
conversation (Patton, 2002). My interview approach was semi-structured, having 10-12
questions to guide the interview. This method allowed me to have a semi-structured interview
where my questions were carefully crafted, and interviewees were free to answer them in their
EVALUATING THE IMPLEMENTATION OF 21
ST
50

own words (Weiss, 1994). As Creswell (2013) asserts, keeping questions in realistic terms, by
gathering perception and beliefs, rather than an instrumentalism approach, will provide answers
more closely related to the phenomena of the study by the participants.
Documents and Artifacts
In addition to interviews, artifacts from general education teachers, were collected, to
document the planned strategies that were utilized to support 21
st
century learning, written in
their year-long grade plans. In a qualitative study, artifacts are supplemental material that can
“enrich” the understanding of the subject (Glesne, 2011). The purpose of collecting teacher
year-long plans was to gain rich descriptions of how teachers support these teaching challenges
and how they think about their execution of their work. My purpose for collecting documents
and artifacts of this nature was to examine how the documents were being translated by the stake
holders and the audience (Bogdan & Biklen, 2007).
Findings
The purpose of this project was to investigate the challenges educational professionals
face implementing 21
st
century instruction in the inclusive classroom at Canyon Valley
Elementary School. The outcome of the findings suggest that additional school wide
communication, time, and training is needed to have school wide implementation of 21
st
century
teaching and learning. The three main areas requiring increased attention are school wide
expectations, modeling and training of 21
st
century techniques, and effective planning for
instructional implementation. Findings involving these areas are covered in the knowledge,
motivation and organizational (KMO) needs and analysis captured in the conceptual framework
influenced by Clark & Estes (2008) gap analysis model. This section will present the findings
using the research questions which reflect the KMO conceptual framework. Subthemes will be
EVALUATING THE IMPLEMENTATION OF 21
ST
51

provided to better illustrate information regarding research questions and the three areas (K,M,
and O) stated above regarding 21
st
century teaching and learning. Supportive data is in the form
of faculty interviews, surveys and documents and artifacts. Pseudonyms are used for the
participating school, faculty, as well as documents which are generalized to protect the
organization’s identity.
Participating Stakeholders
While a complete performance evaluation would focus on all stakeholders, for practical
purposes the stakeholders focused on in this analysis were teachers in the inclusive classroom.
The participants were comprised of six full time instructional faculty members grades
kindergarten through fifth grade. The staff members teaching experiences range from a first-year
probationary teacher to a teacher who has been in the field for thirty years. Two teachers have
experience between one to five years, one teacher is in her ninth year, two teachers are in their
fifteenth year, and another in her 29
th
year in the classroom. All teachers instruct in the inclusive
classroom setting serving students of different learning accommodations, including language
immersion support, resource specialist support, and emotional and behavioral disorders. Table 5
provides an over of the participating teachers in the study.
Table 5
Participant Demographics
Grades Taught Teaching
Experience
(Years)
Students with
Identified
Learning Needs
(Current Class)
Dual Language
Learners
(Current Class)
Participant 1 4
th
1 Y Y
Participant 2 5
th
4 Y Y
Participant 3 K, 2
nd
, 5
th
5 Y Y
EVALUATING THE IMPLEMENTATION OF 21
ST
52

Participant 4 3
rd
, 4
th
, 5
th
8 Y Y
Participant 5 1
st
, 2
nd
, 3
rd
15 Y Y
Participant 6 K, 1
st
, 2
nd
, 3
rd
,
4
th
, 5
th

23 Y Y
Participant 7 K, 1
st
, 2
nd
, 3
rd
,
4
th
, 5
th

30 Y Y

RQ1: What are the teachers’ knowledge and motivational factors that contribute to CVES
achieving its goal of 100% of 21
st
century education??
Two major gaps were found related to Research Question 1; teachers knowledge and
motivation influences towards 21
st
century skills teaching and learning in the inclusive
classroom. The gaps focus on the participants’ varying responses to define 21
st
century
Education and the utility value of 21
st
century instruction as a strategy to achieve 21
st
century
learning outcomes per district goals. An overarching theme in utility value was confidence or
lack thereof amongst the participants. The findings are detailed below.
Knowledge Inconsistencies
Knowledge interview questions were designed to cover factual (F), conceptual (C),
procedural (P), and metacognitive (M) influences related to participants overarching
understanding of 21
st
century education. Not all respondents had clear knowledge of 21
st
century
education as seen through their inability to define it thoroughly. As part of collecting conceptual
knowledge questions, all participants were asked to define 21
st
century education and learning in
their own words. Of the six participants who had been interviewed, two were unable to offer
definitions. One participant clearly stated, “…I have no clue.” While the other participant
offered a response in the form of a questions, “…it has something to do with technology, right?”
While definitions varied amongst those participants who had experience with 21
st
century
EVALUATING THE IMPLEMENTATION OF 21
ST
53

education, there was a consensus amongst those four participants in which they referenced a term
coined the Four-C’s. One participant simply stated, “That’s the Four-C’s we learned at a PD
(Professional Development) some time ago. Collaborations, Critical Thinking, Creativity, and
Communication.” The other three participants shared a similar response, using more elaborate
language, “21
st
century education involved allowing students to think and work together”.
Another participant stated, “The students become the facilitators. They have more creative
freedom and express their learning through various means.” The fourth participant expressed her
understanding of 21
st
century education as “allowing students to discuss the learning, work
together, use technology, and creatively share their understanding.” After a thorough
investigation of the organization’s student handbook, parent handbook, school site plans, and
school website, it was found that there were no current documents that provide a clear definition
of 21
st
century education. The interview participants did not refer to any specific departmental or
school provided definition.
Esteves (2011) explains that while 21
st
century education is essential for student success,
there still seems to be no global definition that is being consistently used by school districts and
states, nor what curriculum programs are best suited to prepare students for 21
st
century
education. The wide range of responses by the participants reflect the disparities and confusion
in the way global education should be addressed in the classroom. Various definitions of a 21
st

century learning environment, according to Matsuda (2016) revolve around access to technology,
internet, and one-on-one devices, which is a limited understanding of 21
st
century education,
which causes a serious disconnect when leaders and classroom teaches go about implementation.
While one of the six participants was able to recall the 4 C’s of 21
st
century education
(communication, creativity, critical thinking, and collaboration), the majority of the participants
EVALUATING THE IMPLEMENTATION OF 21
ST
54

were unable to provide a clearer definition where technology is regarded as a tool to enhance
reading, writing, listening, and speaking to communicate and problem solve in a 21
st
century
environment. Therefore, with the lack of a thorough definition, teachers are given instructional
freedom to address how 21
st
century instruction will be delivered in the classroom.
Motivation Inconsistencies
Motivation interview questions were directed towards theories of expectancy-value and
self-efficacy. Of particular interest was the response variation in the area of self-efficacy
regarding 21
st
century education. Pajares (2006) defines self-efficacy as having the ability and
confidence to positively influence individuals towards successfully mastering a task. Denler et
al., (2009) explain that self-efficacy is when modeled behavior is more likely to be adopted if the
model is credible, similar (e.g., gender, culturally appropriate), and the behavior has functional
value. As long as teachers hold their beliefs in high regard, their actions will result in positive
outcomes. Self-efficacy relates to the confidence an individual possesses. Positive outcomes
include feedback and the opportunity for teachers to observe trustworthy and similar
instructional styles, such as understanding and executing 21
st
century instruction. In this case,
the participants equated self-efficacy of learning 21
st
century education to successful attainment
of 21
st
century pedagogy and skills and the ability to utilize strategies towards classroom
instruction.
One teacher, whom did not have 21
st
century teaching experience, did not perceive
having a command of 21
st
century instruction. While not opposed to mastering 21
st
century
pedagogy and skills, the participant believed to be better adapted to traditional approaches to
instruction, such as teacher facilitated instruction and whole group instruction. By contrast, the
instructional strategies focused in this study include integrated technology, cooperative learning
EVALUATING THE IMPLEMENTATION OF 21
ST
55

structures, project-based learning, critical thinking, and collaboration. The five other participants
who have had 21
st
century education experiences commented in favor of implementing project-
based learning and collaboration yet revealed doubts of utilizing other 21
st
century strategies that
can help support student learning. Participant #3 stated, “I would nowhere say I have mastered
all strategies. I do feel like I have a grasp on technology. What I don’t know, I am willing to try
with an open mind and flexibility. I am willing to try and fail.” Participant #5 supported the
mastery of technology, explaining, “I can deliver instruction using technology. I’m capable of
that. As far as using other 21
st
century pedagogy and skills, my confidence level is not very
high. I could do it, I just don’t know how to, on a daily basis.” The results showed 5 out of 6
participants felt capable and willing to learn more strategies for 21
st
century instruction.
Instructional Confidence
Survey responses provided data related to the motivational influences. When the
questions were stated specific to the context of 21
st
century strategies and skills implementation
the results illustrated a split 50% strength and 50% weakness in rating the level of confidence in
executing 21
st
century instructional strategies. Of the six respondents, those who had over fifteen
years of teaching experience felt confident in 21
st
century instruction, in contrast, teachers who
had less than 10 years of teaching experience felt less confident in their ability to execute 21
st

century instruction. Participants were also asked about their ability to implement guided inquiry
instruction to students and the results show 66% of teachers found this area to be a weakness.
Consistent with the findings of Rotherham and Willingham (2010), the issues in school reform
and 21
st
century skills implementation is meeting the challenges of delivering content and skills
in a way that genuinely improves academic outcomes for students. Table 6 provides a visual of
EVALUATING THE IMPLEMENTATION OF 21
ST
56

the results of the survey questions about teacher self-efficacy in implementing 21
st
century
instructional strategies.
Table 6
Survey Responses for Self-Efficacy in Implementing 21
st
Century Instructional Strategies
Survey Item Strength Weakness
Effectively executes 21
st
Century
Instructional Strategies
50% 50%
Allow student time for
collaboration
50% 50%
Implementation of Guided Inquiry 33% 66%
Models 21
st
Century strategy to
students and the expected desired
outcomes.
83% 16%
Utilization of small group
workshop or teacher-student
collaboration
33% 66%
Utilization of Real World
Problems
33% 66%
Short research or Self Exploration 33% 66%

Interviews further showed a gap in teachers’ confidence and beliefs about the need for
21
st
century teaching and learning. The participants who have had extensive exposure with 21
st

century education placed student learning at the forefront. Five out of 6 participants felt student
learning outcomes is the primary objective to enforcing 21
st
century learning in the inclusive
classroom. Participant #3 stated, “I think 21
st
century education is a positive direction in
education. The greatest disadvantage to implementation is just transitioning the kids and making
EVALUATING THE IMPLEMENTATION OF 21
ST
57

sure to have the handle on how to support them during the learning process.” The 21
st
century
model focuses on student centered rather than teacher centered teaching and learning. Classroom
assignments and tasks are led by encouraging students to problem solve through collaboration
and critical thinking. In this way, students are taught various strategies to meet the expected
outcome. Participant #5 explained,
“I feel, honestly, that we’re in a good place. I think we’re currently doing a really good
job encouraging students to be thinkers. As far as mastery, as an educator, I believe
mastering 21
st
century instructional strategies is going to be a continuous process. As
long as the kids are further motivated to learn using the strategies, that’s successful
learning. My frustrations only build when I can’t figure out how to get students to
execute a strategy for their own good.”
The participant with one year of 21
st
century teaching experience explained, “I would love to
move students in my classroom from being just a technology group to a critical thinking group.
It just seems this is their comfort level of learning right now.” This finding agrees with previous
research conducted by Rotherham and Willingham (2010) indicating that 21
st
century skills need
to be taught intentionally, deliberately, and effectively. The idea of implementing 21
st
century
skills is not enough, explaining that teachers must know how to plan and succeed where
generations before them failed.
Overall, participants agreed that a level of confidence is necessary towards applying 21
st

century education. All six participants wished to be effective educators with 21
st
century skills.
Niemann (2011) asserts that the current teaching system needs to be reimagined so that skills
needed to thrive in the 21
st
century are taught in combination with core academic subjects.
Wagner (2008) believes that while there is a shift in instruction, he believes that both the skills
EVALUATING THE IMPLEMENTATION OF 21
ST
58

and the core content can be taught with the right teacher preparation and training on 21
st
century
strategies. Unfortunately, more empirical evidence is lacking in regard to teacher self-efficacy in
the 21
st
century classroom. Further evaluation of the 21
st
century teaching model is needed.
RQ2: What are the organizational elements related to CVES achieving its goal of
100% of 21
st
century education?
The section reports on the results and findings in response to research question 2 and the
organizational influences within the school’s cultural setting and cultural model in response to
21
st
century learning outcomes per district goals. Overarching themes in the following section
include instructional time and support, program initiatives and support, and assessment and
mastery of implementation inconsistency. The findings are detailed below.
Instructional Time and On-Site Support
Time in a teaching day was thought to be a contributing factor in the full implementation
of 21
st
century strategies. All participants were in favor of finding ways to incorporate more
time for student collaboration and technology within the core curriculum and various school
adopted programs. Participant #4 explained, “I would love to have more time to allow students
to collaborate and problem solve, not just time to meet the standards.” Participant #6 expressed,
“I can fit the technology piece in my day, and some collaboration amongst students. I would
love to be able to provide more opportunities for my students to explore more project-based
learning.” Participant #2 suggests that there be a clear-cut vision statement and specific goals
that teachers will meet, and an outline of strategies to use that will help ensure we are meeting
the 21
st
century learning goal. She shared,
“We spent a lot of time developing a lesson that touched on the 4 C’s of 21
st
century
education. We were supposed to do one trimester or something like that… We really sat
EVALUATING THE IMPLEMENTATION OF 21
ST
59

down. And we used to do projects all the time, it just doesn’t seem there is any kind of
time for that now with other programs we are asked to add to our day.”
These results indicate that while teachers find significant value in 21
st
century instruction, time
does not allow teachers to fully immerse their students in 21
st
century specific learning strategies
that would enhance the teacher’s lesson and student learning outcomes.
Twenty-first century skills were deemed significantly important to all participants. In
addition, all participating teachers felt that school administrators have an obligation to also serve
as an instructional leader in order to improve implementation. Participant #6 explained, “I know
that there have been available resources. I don’t know that I’ve tapped into it as much as I could
have. I do not feel like there has been much support on the site level to build my confidence.”
Participant #2 added, “I don’t know that we’ve had an administrator as an instructional leader. I
have never felt that. It’s just because there’s so much time to spend on managing a school and
behavior and things like that.” Participant #3 stressed,
“I recall when they [the district] sent facilitators to the schools, and we [teachers] were
like the kids, and they were sent to observe 21
st
century instruction in the classroom and
to address if we were doing it right. Our frustrations became very high when they had no
answers to our questions regarding improvement.”
Further evaluation of the administrator’s role is needed to address the instructional leadership
expectancy component of the school’s organization. This study found that the teachers believe
there is a gap in the organizational influence of administrative support and instructional
leadership in 21
st
century education.

EVALUATING THE IMPLEMENTATION OF 21
ST
60

Program Initiatives and Support
Cajon Valley Elementary School (CVES), has a district and school goal of implementing
and assessing a rigorous, standards aligned, 21
st
century educational program that is created to
develop the whole child. The Local Control Accountability Plan (LCAP), which is a
comprehensive planning tool, has a three-year goal to see student achievement improved though
the implementation of 21
st
century learning and skills. As part of the initiative written in the
LCAP, programs were adopted to support global learning, as well as funding to create school site
instructional trainer positions. While participants acknowledge the availability of resources and
support, there was a common gap between the functionality of the new programs and support.
Responses regarding new 21
st
century program adoption ranged from no knowledge of
available programs to limited use of programs. When asked about the implementation process of
newly adopted 21
st
century programs, participants shared, “I have a program in my room that I
have yet to use…” “our administrator allows us to push programs in as we see fit.” Participant
#2 explained, “The administrators are as unfamiliar with the material as we are. Creating an
overarching year-long plan to where we are achieving those LCAP goals would be nice. A map
to get us where we need to be would be good to sort of have that framework.” Participant #3
stated, “Without clear expectation or an outline of goals and how we will meet 21
st
century goals,
it creates less buy-in from all staff to get full engagement.” Participant #6 said,
“Administrative support would be nice, but it’s still coming about. I would like to see
more administrators in the classroom maybe doing more, [be]cause I do think that they
get a feel for what it really is like as opposed to just saying, ‘You’re going to implement
this,’ and not knowing what that looks like. I think that we still need that because most of
EVALUATING THE IMPLEMENTATION OF 21
ST
61

our administrators have been removed from the classroom for a while, and I think it
would be key to continue their education in this as well.”
These findings agree with researchers, Rotherham and Willingham (2010) explaining that change
needs to move beyond the administrators’ offices and into the classroom. What teacher’s need
from their school leader is much more robust training and support, including lesson plans that
deal with the high demands and potential classroom management issues that come with applying
student-centered methods.
When asked about support from either the school site or opportunities for training,
responses varied. Four out of the six participants felt extremely supported through the various
professional development opportunities. The school district offers teachers opportunities to
enroll in online training for instructors in the areas of technology and instructional strategies. A
review of PD’s offered by the school district on their website, modules surrounding 21
st
century
instruction and strategies are available, yet teachers gravitate more towards PD’s regarding
understanding newly adopted programs and online assessment tools. These training opportunities
are offered all year long and teachers can enroll at their own leisure. Participants #1, however,
explained that,
“We have these online education programs that are available to use, but I’m still not
comfortable applying the learning with the classroom. There was a PD on tools and
things, but I haven’t really had any technology training, which is important because they
use their Chromebooks all day. Again, I’m confused on what to do with the PD I did
receive.”
Participant #2 explains that, while available professional development is a plus,
EVALUATING THE IMPLEMENTATION OF 21
ST
62

“Online training lacks the opportunity to learn and discuss the learning in a collaborative
setting. Unlike years ago, when training was done together, this modality of learning
doesn’t allow for teachers to be on the same page. It definitely is missing that component
of asking, ‘what’s working, what isn’t working, what’s not working’.”
Participant #5 believes that “collaboration during staff meetings, or any collaborative support,
especially with our social emotional learners would be more cohesive in applying the adopted
programs into the day.” Overall the teachers agreed that there was training available but that it
was ineffective because it lacked authentic application to the classroom and collaborative
learning opportunities among teachers. The findings suggest that more direct staff
communication is needed.
When asked about the advantages and disadvantages of a school wide adoption of 21
st

century learning and skills, responses varied. Participant #4 explained,
“I don’t think there’s any disadvantages. I think there’s only advantages, even if it’s not
something that somebody is total versed in, then they can discuss that with someone who
is and instructs them. I feel like any implementation can be tweaked to a certain
personality or teaching style, so I don’t think there’s a disadvantage. If it’s not something
that you like or are familiar with, you could adapt it. I think adaption is huge and I think
modeling that for teachers is an excellent way to go about motivating staff.”
Participant #1 placed the student transition into 21
st
century learning at the forefront by
explaining, “Advantages? It is preparing them for the future, but disadvantages, it’s also not
preparing them for the future because they are missing that traditional style of learning, which I
still think it important, so again, it’s that balance and I feel like maybe the implementation for
EVALUATING THE IMPLEMENTATION OF 21
ST
63

both students and teachers are not as balanced as it should be.” Participant #2 felt that school
wide adoption provided an opportunity for the staff to learn together.
“This is a journey that we are all working on together to make happen. The advantages
of school-wide adoption is that we’re all on the same page together, and we can learn
together. As long as we’re provided regular collaboration time to meet to discuss what’s
happening, school-wide adoption allows for the possibility of creating a strong team.
Cohesion is critical. I know that I’m on a strong [grade-level] team where we are so
cohesive and we definitely have a meeting of the minds when we come together, there’s a
respect for ideas, we come together in everything, it just works so beautifully, and I think
that’s really important.”
Participant #3 felt self-led in school-wide implantation,
“I don’t believe we have an implementation process for 21
st
century education. I started
doing it on my own when I first heard about it. I do believe that for full implementation
of 21
st
century skills, we need to come up with a process. You can’t just jump into all of
it all at once. For my Masters classes, we had to come up with a plan to implement 21
st

century skills to the school. So, I developed one with the previous principal, and it just
sort of never came up again, and the plan was a step by step implementation taking the 4
C’s year to year so that eventually, everyone has a better understanding of each and every
one is working towards applying them together. So, I have my own framework that I use
that I think works.”
This confirms with previous research conducted by Rotherham and Willingham (2010), the
process of implementation requires three primary components (1) educators and policy makers
must ensure the program and content is complete and not performed for ephemeral pursuit of
EVALUATING THE IMPLEMENTATION OF 21
ST
64

skills (2) school districts and schools need to revamp the training process for teachers and (3)
need new methods that can measure learning and complex tasks. Table 7 gives a list of
instructional needs discovered in the interviews.
Table 7
Teacher Needs for Full Implementation of 21
st
Century Skills
Teacher Needs for Full Implementation of 21
st
Century Skills
• Administrative Support
• On-Site Teacher Training
• School-Wide Procedures and Expectations
• Lesson Plans
• Collaboration Time
• Modeling of Strategies
• Time for Teaching, Modeling, and Evaluating 21
st
Century Instruction and Skills with
the current adopted curriculum and program.

In addressing organizational inconsistencies, data collection instrumentation offered a
scan of each grade level’s year-long scope and sequence. Scope and sequence plans, according to
Arafeh (2015), are important when ensuring course or program content, instructional activities,
assessments, and student support are present. Similar to the teachers request for more
collaborative learning opportunities in professional development, it was further confirmed from
the review of the scope and sequences that collaboration was not taking place. According to the
collected data, individual teachers from the same grade level team had differing year-long plan
outlines. The participants’ plans provided varying glimpses of their academic understanding and
EVALUATING THE IMPLEMENTATION OF 21
ST
65

implementation of the 21
st
century skills indicated in their core curriculum. Hartoonian et. al.,
(1989) explains that designing a scope and sequence for the 21
st
century creates congruency
between educational theories and instructional practices and how it is unfolded in the classroom.
Components of teacher plans ranged from a brief list of units to be taught throughout the year to
plans that listed focus standards, unit worksheets, and assessment. Only two digital year-long
plans included teacher made videos to be shared with students, that contained step by step
strategies for students to apply the math concept being taught. Cluphf and William (2008) found
that collaborative planning results in a positive dynamic of teacher engagement, decision
making, and effective lessons that produce high levels of learner involvement. Additionally, the
act of collaborative planning involves formal conversations between veteran and novice teachers
to assist in helping them reach mature levels of teaching.
Overall responses in the area of program initiatives and support highlight a gap between
participating teachers and system wide practices regarding planning, support, and execution of
21
st
century education implementation. Consistent with self-efficacy attitudes, participants with
more teaching experience felt more confident about 21
st
century adoption, while teaches with
less amount of years felt less confident and unfamiliar with 21
st
century teaching expectations
and goals.
Assessment and Mastery of Implementation Inconsistency
When the more experienced participants were questioned about how they are assessing
the effectiveness or ineffectiveness in implementing 21
st
century skills in their classrooms the
results varied, but 5 out of 6 utilized project-based learning as their mode of assessments.
Participant #4 explained, “I can assess whether the strategy or skills were working based on
authentic assessment.” The participant went on to explain that authentic assessments is presented
EVALUATING THE IMPLEMENTATION OF 21
ST
66

through further research and project presentations where students share their findings. “This
gives students the communication piece of 21
st
century education where they can share what
they’ve discovered on their own to their classmates. I can also assess my implementation when I
see that the kids are happy and excited about their learning.” Participant #2 referred to a
student’s ability to problem solve, as an assessment tool. “When students are problem solving, it
shows self-reliance and motivation. It makes me have to work harder in my instruction to say,
‘What strategy can I use to motivate them harder, to keep them to persevere.’” Participant #2
also added, “If I did a lesson and asked students to collaborate or deliver results to their
classmates, and they are struggling I use that as well as an assessment. ‘Did I just blow through
the lesson?’ So then, I have to reflect and make sure I adjust.” Assessing the effectiveness of 21
st

century strategies appears to give the participants more of an opportunity to evaluate their own
instruction and whether students are fully engaged in the learning process.
When asked about meeting instructional mastery of 21
st
century skills, all teachers did not
feel they had a full grasp of delivering 21
st
century strategies and skills to their students.
Participant #5 explained,
“I think to know the basics of 21
st
century instruction is easy. It’s easy to say, ‘Yeah.
The kids need to know how to do that, especially to be globally ready. I think that it
takes maybe one or two times, open a discussion with teachers, and continue to have
PD’s or something. I think to meet mastery, however, that takes years. Five minimum,
just to be proficient at teaching it.”
Participant #2 recommended, “Curriculum for core subjects would be nice and suggestions on
best 21
st
century strategies would be helpful, rather than guessing all the time whether I should
have student collaborate, or present.” Participant #5 explained,
EVALUATING THE IMPLEMENTATION OF 21
ST
67

“Mastery of instruction requires professional development, coaching, on-site coaching,
collaboration, in a non-evaluative form. So, it’s not my administrator coming in and
telling me what I’m doing wrong, but actually offering strategies. Whether you’re a
first-year teacher or a teacher with twenty-nine years of experience, it’s hard to meet
mastery when an evaluation isn’t providing any real recommendation, but just finding
faults. That’s not how kids learn, and that’s definitely not how adults learn.”
As the majority of participants were teachers who have been in the classroom for more than ten
years, experience did not differ in how teachers can attain full understanding of 21
st
century
education. The participant with the least experience cited recommended on-site professional
development, as opposed to online training classes. “Having those PD’s is the best way to move
forward and feel like you’ve mastered something.” Table 8 gives a list of needs identified in the
interviews for teachers to meet mastery of 21
st
century instruction.
Table 8
Needs to Meet Mastery of 21
st
Century Instruction
Needs to Meet Mastery of 21
st
Century Instruction
• On-Site Professional Development
• Coaching
• On-Site Coaching
• Collaboration
• Curriculum
• Non-Evaluative Walk-Throughs

EVALUATING THE IMPLEMENTATION OF 21
ST
68

The findings related to organizational factors impacting the school’s ability to meet its
goal focused heavily on the impact of administrative support and program initiatives adopted by
CVES. The first theme that emerged from the findings was teachers’ perception of the impact of
implementation and support of 21
st
century skills and techniques. Goddard et. al., (2015) explain
that the principles of instructional leadership may be the factor that encourages teachers to
collaborate and discern data towards strengthening school effectiveness and teacher self-efficacy
beliefs. Positive experiences were primarily connected to the teacher’s chosen strategies used to
achieve instructional success. Even those participants who largely perceived the impacts as
negative believed they have the capacity to improve in their own understanding and rolling out
21
st
century learning goals in their classroom. School leadership and collaboration were cited as
significantly important factors in the overall success in supporting all learners of the school.
Recommendations for Practice to Address KMO Influences
Based on the findings, recommendations addressing the identified gaps affecting
implementation and teacher self-efficacy are addressed in this section. Addressing teacher self-
efficacy in the 21
st
century inclusive classroom are categorized by knowledge, motivation, and
organizational influences. Tables outline the presentation of influencer, typology, principle
from the literature supporting the proposed solution, and a brief description of the recommended
action. For each recommendation, details outline the expected results, as well as the behaviors,
learning, and reactions.
The approach corresponds with the New World Kirkpatrick Model (Kirkpatrick &
Kirkpatrick, 2016), starting with the desired outcomes and working backwards toward the
suggested interventions. As the Clark and Estes (2008) gap analytical framework provided the
foundation for this study, it calls to the research by Kirkpatrick (1998) as the model for
EVALUATING THE IMPLEMENTATION OF 21
ST
69

implementing change. Utilization of the New World Kirkpatrick Model (Kirkpatrick &
Kirkpatrick, 2016) provides a continuation of the original framework. Thus, by designing
interventions addressing identified gaps built upon a standardized framework, change initiatives,
even implemented separately or consecutively, deliver a programmatic approach to affecting
change.
Knowledge Recommendations
Introduction. The knowledge influence in Table 9 includes all assumed knowledge
influences and their probability of being validated. The knowledge influences used to achieve
the teachers’ goal will be validated based on the most frequently mentioned conceptual,
procedural, and motivational knowledge influences to achieving the teachers goal during semi-
structured interviews, teacher surveys, as well as the literature review. The conceptual
framework for this study is Clark and Estes (2008) gap analysis. The knowledge influences
include the conceptual dimension that addresses the what, the procedural dimensions that address
the how, and the metacognitive dimensions that address the self (Krathwohl, 2002). As indicated
in Table 9, it is anticipated that these influences have a high probability of being validated and a
high priority for achieving the teachers goal. Table 9 show the recommendations for these
highly probable influences based on theoretical principles.
Table 9
Summary of Knowledge Influences and Recommendations
Assumed Knowledge
Influence
Validated?
Yes, High
Probability
or No
(V, HP, N)
Priority
Yes, No
(Y, N)
Principle and
Citation
Context-Specific
Recommendation
Teachers will need to
know what it means to
implement 21
st
century
skill-based curriculum.
HP Y
Conceptual
knowledge is
important when
understanding
District will provide
pre-service trainings
on the current 21
st

century initiatives
EVALUATING THE IMPLEMENTATION OF 21
ST
70

patterns and
relationships of
ideas, topic, or
themes
(Anderson &
Krathwohl,
2001).

Identify specific
behavioral
objectives for
learning. (Daly,
2009)

and teachers will
collaboratively
identify specific
methods and
strategies for
implementation.
Teacher will need to know
how to support all
students in 21
st
century
skills.
HP Y
Procedural
knowledge –
Providing
scaffolding and
assisted
performance
in a person’s
ZPD promotes
developmentally
appropriate
instruction (Scott
& Palincsar,
2006).algorithms
, techniques, and
methods
(Anderson &
Krathwohl,
2001).

District will support
and provide
scaffolding
techniques on the
current ELA and
Math program to
specifically target
and address the
needs of all learners.

Evaluations and
feedback of the
program will begin
as early as the
second trimester of
the school year by
informal
walkthroughs from
principal and
academic coach.
Teacher’s will need to
know how to compile data
prior to each Individual
Educational Plan for each
of their identified students
with learning needs.
HP Y
Procedural
knowledge -
Goals motivate
and direct
students
(Pintrich, 2003).

Training
individuals
Teachers will apply
what they know
about collecting data
for IEP meetings.

Include mock
advising sessions
as guided practice
to allow teachers
to practice “how
EVALUATING THE IMPLEMENTATION OF 21
ST
71

with the “how
to”
knowledge and
providing guided
practice and
feedback
helps individuals
achieve their
goal
(Clark & Estes,
2008).

to” empower
students and
provide feedback.
Teachers will need to
know how to execute the
21
st
century strategies for
differentiated instruction
in the inclusive classroom.
HP Y
Procedural
knowledge -
Feedback as well
as actual success
on challenging
tasks
positively
influences
people’s
perceptions of
competence
(Borgogni et al.,
2011).

Job aids are used
to
provide self-help
information to
individuals who
need
“how to”
information
without guided
practice (Clark &
Estes, 2008).
Teachers will train
on how to execute
plans to enhance
student outcomes
training on
Individualized
Educational Plans
(IEP) and 21
st

century instructional
strategies for each
student by the
school’s director of
student services.

*Indicate knowledge type for each influence listed using these abbreviations: (D)eclarative;
(P)rocedural; (M)etacognitive

EVALUATING THE IMPLEMENTATION OF 21
ST
72

Declarative knowledge solutions, or description of needs or assets. Teachers will need
to know what it means to implement 21
st
century skills-based curriculum (C). Anderson &
Krathwohl (2001) discuss how conceptual knowledge is important when understanding patterns
and relationships of ideas, topics, or themes. Job aides are used to identify specific behavioral
objectives for learning (Daly, 2009). Therefore, it is recommended that districts will provide
pre-service trainings on the current 21
st
century initiatives and teachers will collaboratively
identify specific methods and strategies for implementation.
Mayer (2011) and Rueda (2011) emphasize what people should know or how people
learn, is only a fraction of the problem. Equally important, educators should know how to assist
in learning. Rueda (2011) describes this as the science of instruction. Educators should have a
clear idea of what instructional methods works for each learner under each individual
circumstance (Mayer, 2011). Scott and Palinscars’ (2006) principle of sociocultural theory
suggests, the creation of new knowledge occurs when individuals engage in social
interaction, cooperative learning, and teaching. In addition, job aids help connect educators to
previous knowledge. Grossman and Salas (2011) describe job aids as a method that make it
easier to transfer knowledge to the work environment. From a theoretical perspective, teachers
learning and creating together is necessary to achieve optimal performance and 21
st
century
performance goals as it is recommended that teachers work together to create job aids.
Administrative and teaching staff can begin developing job aids at the beginning of each school
year. Job aids can be in the form of checklists, necessary worksheets, and flow charts that make
implementation of the process simple.
Procedural knowledge solutions, or description of needs or assets. Teachers will need
to know how to support Students with Disabilities in 21
st
century skills. Anderson & Krathwohl
EVALUATING THE IMPLEMENTATION OF 21
ST
73

(2001) and Scott & Palincsar (2006) suggest that procedural knowledge provides individuals
with the ability to scaffold and assist performance in a person’s Zone of Proximal Development
promotes developmentally appropriate instruction, algorithms, techniques, and methods.
Therefore, it is recommended that evaluations and feedback of the program will begin as early as
the second trimester of the school year in the means of informal walkthroughs from the school
principal and academic coach. Pintrich (2003) suggests that procedural knowledge gives
individuals an opportunity to set goals to motivate and direct students. Thus, it is recommended
that teachers apply what they know about collecting data for Individualized Education Plan
meetings. This would include mock advising sessions as guided practice to allow teachers to
practice “how to” empost students and provide feedback. Borgogni et al. (2011) explains that in
procedural knowledge, feedback, as well as actual success on challenging asks, positively
influences people’s perceptions of competence. Henceforth, it is recommended that teachers will
train on how to execute plans to enhance student outcomes and receive training on Individualized
Education Plans for each student by the school’s director of student services. Clark and Estes
(2008) suggests that training individuals with the “how to” knowledge and providing guided
practice and feedback helps individuals achieve their goals.
Mayer (2011) explains that individuals that practice tasks are more capable of doing the
task better and encounter specific transfer. Schraw and McCrudden (2006) recommend that
individuals should first acquire the knowledge, practice the learned skill, and apply it often to
successfully master what was taught. Researchers agree that in order for students to be
successful, it is imperative that educators have the skills to empower students and advise them
appropriately (Allen, Smith, & Muehleck, 2013; CAS, 2015; Young-Jones, Burt, Dixon, &
Hawthorne, 2013). From a theoretical perspective, it would appear that training teacher on how
EVALUATING THE IMPLEMENTATION OF 21
ST
74

to support, target specific learning needs, collect data, and execute plans to enhance student
outcomes would allow teachers to reach their performance goal. As such, it is recommended that
teachers be trained on the skills and techniques needed support Students with Disabilities and
participate in instructional modeling sessions. According to Pintrich (2003), training individuals
with the “how to” include mock advising sessions as guided practice to allow teachers to practice
“how to” empower students and provide feedback. Evaluations and feedback of the program
should begin as early as the second trimester of the school year beginning with informal
walkthrough from the principal and academic coaching staff.
Motivation Recommendations
Introduction. The motivation influences in Table 3 includes all assumed motivation
influences and their probability of being validated. The motivation influences of value and self-
efficacy used to achieve the teachers’ goal will be validated based on the frequently mentioned
motivational influences to achieving their goal during semi-structured interviews, observations,
and collection of artifacts. Clark and Estes (2008) suggest that individuals make a choice, use
mental effort, and persist in undertakings that make a difference. The teacher’s motivation to
value the training surrounding 21
st
century education and their belief that they can execute and
provide positive instructional outcomes will be assessed to determine any gaps in their
motivation. Within the Clark and Estes (2008) framework, complication of teacher instructional
motivation continues through the lens of self-efficacy theory (Bandura, 1977, 1997) and goal
orientation theory (Pintrich, 2000, 2003; VandeWalle, 1997, 2004) to provide a clearer
understanding of motivation-based teaching deficiencies affecting 21
st
century instructional
performance in the inclusive classroom.
EVALUATING THE IMPLEMENTATION OF 21
ST
75

The classroom teachers’ motivation to value their ability, effort, and instructional
capacity and their belief that they can provide optimal learning outcomes will be evaluated to
determine any gaps in their motivation. As indicated in Table 10, it is anticipated that these
influences have a high probability of being validated and a high priority for achieving teacher
goals. Table 10 also shows the recommendations for the highly probable influences based on
theoretical principles.
Table 10
Summary of Motivation Influences and Recommendations
Assumed Motivation
Influence

Validated
Yes, High
Probability
, No
(V, HP, N)
Priorit
y
Yes,
No
(Y, N)
Principle and
Citation
Context-Specific
Recommendation
Teachers need to
believe that all
students are
capable of high
outcomes in the
21
st
century based
curricular program.
(Expectancy
Value)
HP Y Rationales that
include a discussion
of the importance
and utility value of
the work or learning
can help learners
develop positive
values (Eccles, 2006;
Pintrich, 2003).

Feedback as well as
actual success on
challenging tasks
positively influences
people’s perceptions
of competence
(Borgogni et al.,
2011).

Teachers will run
formative
assessments, after
each lesson, as an
opportunity to give
students specific,
targeted feedback
on how they are
performing 21
st

century skills, so
that students will
understand how to
improve their level
of learning and
opportunities to
practice the skills.
Teachers need to
feel they are
effective educators
and have the skills
to teach all
students in the
inclusive
HP Y High self-efficacy
can positively
influence motivation
(Pajares, 2006).

Pajares (200^) found
that when
individuals are able
Principals and peer
teachers will sit in
during classroom
instruction and
observe. After
instruction, the
principal and peer
advisor will focus
on offering
EVALUATING THE IMPLEMENTATION OF 21
ST
76

classroom. (Self-
Efficacy)
to successfully
master a task, self
efficacy increases.

Pajares (2006) points
out that the self-
efficacy
philosophies that
people hold are the
basis for motivation
and as long as they
believe that their
actions will result in
a positive outcome
they will attempt the
task at hand.
corrective feedback
and providing
positive
encouragement on
the teacher’s ability
to provide
developmental
advising.

Expectancy Value. Teachers need to believe that all students are capable of high
outcomes in the 21
st
century based curricular program. Pintrich (2003) and Eccles (2006)
suggests rationales that include a discussion of the importance and utility value of the work or
learning can help learners develop positive values. According to Borgogni et al., (2011)
feedback as well as actual success on challenging tasks positively influences peoples’
perceptions of competence. This would suggest that providing teachers with evidence that their
instructional methods were successful and is modifying student learning in a positive direction,
would increase their expectancy value. Therefore, it is recommended that instructional staff
develop a process in which teachers will run formative assessments, such as think-pair-share,
strategic questioning, analyzing student work, 3-Way summaries, presentations, classroom polls,
and/or exit tickets after each lesson, as an opportunity to give students specific, targeted feedback
on how they are performing 21
st
century skills. This will give teachers an opportunity to
understand how to improve their level of learning and opportunities to practice the skill.
According to Eccles (2006) a principal of the expectancy value theory suggests that when
individuals set higher expectations for success and perceptions of confidence can positively
influence learning and motivation. Researchers suggest students’ beliefs about their academic
EVALUATING THE IMPLEMENTATION OF 21
ST
77

experiences have significant implications for their school adjustment and achievement (Mueller
& Dweck, 1998; Wigfield & Eccles, 2002). Eccles (2006) also adds that when individuals value
a task, their motivation increases. From a theoretical perspective, it would appear that increasing
expectancy value in teachers and students would increase their value to provide data on all
perceived barriers to developing teaching strategies to ensure students are meeting 21
st
century
skills and learning expectations; as such, it is recommended that students’ experiences with
teachers and teacher instruction be used to provide teaching staff with feedback about how their
teaching strategies and methods are aligned with the school’s vision and mission statement.
Feedback will be gathered at the end of each trimester using an online survey that allows
students to rate the effectiveness of their teachers.
Self-Efficacy. Teachers need to feel they are effective educators and have the skills to
teach students with disabilities in the inclusive classroom. Pajares (2006) suggests, high self-
efficacy can positively influence motivation. Additionally, Pajares (2006) found that when
individuals are able to successfully master a task, self-efficacy increases. In addition, Pajares
(2006) points out that the self-efficacy philosophies that people hold are the basis for motivation
and as long as they believe that their actions will result in a positive outcome, the will attempt
the task at hand. This would suggest that providing individuals with feedback and the
opportunity to observe their teaching peers would increase their self-efficacy. Therefore, is is
recommended that the school principal and peer teachers sit in during classroom instruction and
observe. After instruction, the principal and peer advisor will focus on offering corrective
feedback and providing positive encouragement on the teacher’s ability to provide 21
st
century
instruction.
EVALUATING THE IMPLEMENTATION OF 21
ST
78

Clark and Estes (2008) explain that the skills necessary to succeed are the most critical
factor in the amount of mental effort an individual will expend for a task. If individuals lack the
confidence and do not hold a strong belief in their abilities, they will avoid the task altogether
(Clark & Estes, 2008). According to Pajares (2006) when individuals are given an opportunity
to observe a credible, similar model, self-efficacy will grow. Additionally, Denler et. al., (2009)
suggests, when peers with credible instructional skills model behaviors, it is more likely that the
observer will adopt the behavior. Finally, by providing feedback, in a one-on-one setting, an
individual's performance has a likelier chance to improve (Shute, 2008). From a theoretical
perspective it would appear that providing instructional models and constructive criticism,
teacher’s self-efficacy would positively increase, as well as their, beliefs about their instructional
capacity in 21
st
century skills and learning, as such it is recommended that teachers observe peers
and receive feedback from the school principal to positively influence motivation.
Organization Recommendations
Introduction. The organizational influences in Table 11 include all assumed
organizational influences and their probability of being validated. The organizational influences
used to achieve the teachers’ goal will be validated based on the most frequently mentioned
organizational influences to achieving the teachers goal during semi-structured interviews,
observations, artifacts, and the literature review. Even when stakeholders obtain the knowledge
and skills and are motivated, Clark and Estes (2008) believe that the lack of policies, procedures,
cultural setting models, and effective organizational resources hold back stakeholders from
achieving their desired performance goals. Gallimore (2001) states that cultural settings are the
noticeable characteristics of the who, what, when, where, why, and how of the inner workings in
an organization, whereas the cultural model are often times undetectable. The cultural model
EVALUATING THE IMPLEMENTATION OF 21
ST
79

influence of the organization’s capacity to ensure an environment that supports change, also
includes an environment that is trustworthy. The cultural setting influence of the organization’s
capability to provide teachers with enough time to discern data on the perceived barriers to
teaching 21
st
century skills and learning in the inclusive classroom will be evaluated to determine
any organizational gaps.
Table 11
Summary of Organizational Influences and Recommendations
Assumed
Organization
Influence

Validated
Yes, High
Probability,
No
(V, HP, N)
Priority
Yes, No
(Y, N)
Principle and
Citation

Context-Specific
Recommendatio
n
Cultural Model
Influence 1:
There is a
consensus from
faculty
expressing a
lack of support
from
administration
regarding
instructional
techniques for
all learners in
the 21
st
century
classroom
HP Y Open
communication and
concern for fellow
colleagues builds
trust (Korsgaard,
Brodt, & Whitener,
2002; Schein,
2010)

Organizational
performance
increases and trust
is promoted when
individuals and
leaders
communicate
openly and
constantly about
plans and progress
(Clark & Estes,
2008).
Time will be set
aside in regularly
scheduled staff
meetings to allow
teachers and
administrators an
opportunity to
communicate
their plans,
progress, and
support for one
another.
Cultural Model
Influence 2:
There is a
culture within
the faculty that
relies heavily on
the student
support services
department to
provide primary
N N Positive emotional
environments
support motivation
(Clark & Estes,
2008).

Organizational
effectiveness
increases when
leaders ensure that
employees have the
resources needed to
Review the
current workload
with resource
specialists to
collaboratively
develop a plan to
redesign the pull-
out process, so
that teachers and
resource specialist
understand their
individual roles in
EVALUATING THE IMPLEMENTATION OF 21
ST
80

21
st
century
learning and
instruction to
students with
identified
learning needs
through the pull-
out method.
achieve the
organization’s
goals (Clark &
Estes, 2008).
supporting
students to meet
the learning goals.
Cultural Model
Influence 4:
There is no
general school
wide
information on
the co-teaching
model which
then weakens
the effectiveness
of inclusion in
the 21
st
century
classroom.
HP Y Organizational
performance
increases when a
change produces
improvement in
how the job is done
and the results have
a lasting impact
(Langley, Moen,
Nolan, Nolan,
Norman & Provost,
2009

Social interaction,
cooperative learning,
and
cognitive
apprenticeships (such
as reciprocal
teaching) facilitate
construction of new
knowledge (Scott &
Palincsar, 2006).
Collaborate with
school staff at the
very beginning of
each schools year
to communicate
how the co-
teaching model
will be
implemented with
the goal of
increasing and
impacting student
achievement.
Cultural Setting
Influence 1:
Faculty are
overwhelmed by
teaching
responsibilities
that keep them
from investing
more time and
effort to
understanding
21
st
century
instructional the
inclusive
classroom.
HP Y Organizational
effectiveness
decreases if the
workload increases
more than 10% when
adopting change
(Sirkin, Keenan, &
Jackson, 2005)

Effective change
begins by addressing
motivation
influencers; it ensures
the group
knows why it needs
to change. It then
addresses
organizational
barriers and then
knowledge and skills
needs (Clark and
Estes,
2008).
Provide monthly
opportunities for
teachers to
collaborate on
which 21
st
century
strategies have
been effective,
building upon
staff’s interest in
their individual
growth as
teachers with the
intent to meeting
the school’s
vision and
mission.
EVALUATING THE IMPLEMENTATION OF 21
ST
81

Cultural models. This influence was chosen because teachers will be unable to meet
instructional 21
st
century pedagogy and skills goals if they are feeling a lack of support from
administration. The teachers need a supportive environment to support student success rates for
the 21
st
century classroom. Clark and Estes (2008) suggests that organizational performance
increases and trust is promoted when individual leaders communicate openly and constantly
about plans and progress. Through open dialogue and transparency between staff and
administrators, trust for one another will be increased, which allows teams to have constructive
conversations (Korsgaard, Brodt, & Whitener, 2002; Lencioni, 2002; Schein, 2010). This would
suggest that communication between stakeholders would enhance organizational effectiveness.
Therefore, it is recommended that time be set aside in regularly scheduled staff meetings to allow
teachers and administrators an opportunity to communicate their plans, progress, and support for
one another.
This influence was also chosen because teachers will be unable to support the needs of all
students unless general school wide information on the co-teaching model is clearly articulated.
Teachers need to have clarity on the objective of the co-teaching model with support staff in
order for instruction of 21
st
century pedagogy and skills to be fully implemented.
Langley et al. (2009) suggest that organizational performance increases when a change
produces improvement in how the job is done and the results have a lasting impact. Teaching
social interaction, cooperative learning, and cognitive apprenticeship (such as reciprocal
teaching), facilitate new knowledge (Scott & Palincsar, 2006). This would suggest that clear
information on the strategy, cooperative teaching, to ensure all students are receiving 21
st
century
education, will create an environment that will result in positive outcomes for years to come.
Therefore, it is recommended that staff should be given time to collaborate with one another at
EVALUATING THE IMPLEMENTATION OF 21
ST
82

the very beginning of each school year to communicate how the co-teaching model will be
implemented with the goal of increasing and impacting student achievement.
Cultural settings. This influence was chosen because learning will not be improved
unless faculty are provided time to understand new and current curriculum as it pertains to 21
st

century learning in the inclusive classroom. Faculty are overwhelmed by numerous teaching
responsibilities that keep them from investing more time and effort to understanding 21
st
century
instructional practices necessary towards supporting all learners in the general education
classroom. Sirkin et. al., (2005) suggest that organizational effectiveness decreases if the
workload increases more than 10% when adopting change. Clark and Estes (2008) state that
effective change begins by addressing motivation influences; it ensures the group knows why it
needs to change. It then addresses organizational barriers, following knowledge and skills needs.
This would suggest, when change is implemented, time and resources need to be provided to
support the teaching community. Therefore, it is recommended that administrators provide
opportunities during each monthly staff meeting for teachers to collaborate on which 21
st
century
strategies have been effective, building upon staff’s interest in their individual growth as teachers
with intent to meeting the school’s vision and mission.
Teachers will need additional time to assess data on current 21
st
century instructional
strategies towards improving, developing, and recommending teaching techniques alongside
their current teaching duties. Successful change in an organization, such as creating new
processes, involves including the right resources for the organization to flourish (Fernandez and
Rainey, 2006). Clark and Estes (2008) believe that resources can be added or restructured when
work processes are redesigned to increase effectiveness. Additionally, when promoting
employee effort, for transformation to be successful, Sirkin, et. al., (2005) states that leaders need
EVALUATING THE IMPLEMENTATION OF 21
ST
83

to consider removing unnecessary “regular” work. From a theoretical perspective, it appears that
in redesigning the work processes, currently followed by instructional staff members, the school
organization would increase its effectiveness as such it is recommended that a review of the
current workload be analyzed and a plan developed to accommodate the additional effort needed
for the change.
In summary, recommendations were developed according to knowledge, motivation, and
organizational influences. These implementation and evaluation plans can be found in the
Appendix. The knowledge solutions involve pre-service trainings on 21
st
century initiatives,
support and techniques using the current adopted core curriculum, and specific 21
st
century
instructional support for all learner (Table 9). The motivational solutions include identifying 21
st

century learning needs through evaluating assessments and data, and improving teacher self-
efficacy through instructional feedback through observations and peer recommendations (Table
10). The organizational suggestions are comprised of improving 21
st
century learning and skills
through administrative support using regular staff meeting and working collaboratively with all
instructional staff members at the school site (Table 11). The internal and external outcomes,
critical behaviors, drivers, course metrics, and evaluations are contained in Appendix E:
Integrated Implementation and Evaluation Plan.
Successfully implementing change in any school setting is not easy and will come with
its own challenges. Reeves (200) explains that the process of changing schools is tedious and
exhaustive, but is crucial in the pursuit of change, especially for schools identified as low
performing and in need of turning around. For a teacher leader, implementing change requires
many steps, including sacrifice of time. Teacher leaders will need to use their voice as change
makers and to be confident to speak up for what is needed for the kids. From the start of the
EVALUATING THE IMPLEMENTATION OF 21
ST
84

school year, leaders interested in change will need to begin by building relationships with all
staff members, more importantly, with the administrator, in order to begin having difficult
conversations about applying new ideas and concepts that are worth trying at the school site.
These new ideas will need to be backed, so teacher leaders will need to make sure they provide
evidence that proves specific methods and strategies work to improve the academic success of
students. It is also imperative that in order for implementation to take off, change makers will
need to invite members of the staff to contribute their expertise to planning, mapping the year in
a collaborative setting, and provide guided support to newer teachers. When efforts to make
change is done collaboratively, teachers are more likely encouraged to do their best.
Conclusion
This study explored the critical challenges classroom teachers face when meeting 21
st

century learning goals in the inclusive classroom at CVES. Specifically, this study explored
teachers experiences and perception of 21
st
century education at their respective site, the
challenges and learning opportunities they face as educators, and the impacts of 21
st
century
learning mandates. This study also explored the perception participating teachers have on the
institutes greatest strengths and implications for change. Drawing on the experience of these
participants, this study sought to investigate current challenges, and their projected visions for
further implementation of 21
st
century education success in the organization.
Analysis of assumed influences resulted in particular gaps found in knowledge,
motivation, and organizational influences. Root causes were found in the lack of 21
st
century
education implementation in the inclusive classroom. Confidence, time, administrative support,
and mastery of implementation were found to be significant issues amongst the participants.
Recommendations include designing interventions to address the knowledge, motivation, and
EVALUATING THE IMPLEMENTATION OF 21
ST
85

organizational gaps. Declarative and Procedural Knowledge development is recommended to
effectively address instructional challenges. The analysis resulted in recommendations for pre-
service trainings, evaluation and feedback, and creating job aids at the start of each school year.
Use of Expectancy Value and Self-efficacy theories influenced recommendations that included
student and administration feedback to positively influence motivation. Organizational
recommendations to achieve the teachers’ goals focused on the cultural model and setting
influences. The recommendations and implementation model presented include collaboration
time with the goal of increasing and impacting student achievement. This study was presented
with the intent of helping CVES continue to improve the implementation of 21
st
century
education in all inclusion classrooms.
EVALUATING THE IMPLEMENTATION OF 21
ST
86

References
Aguinis, H., & Kraiger, K. (2009). Benefits of training and development for individuals and
teams, organizations, and society. Annual Review of Psychology, 60(1), 451-474.
doi:10.1146/annurev.psych.60.110707.163505
Alexander, P. A., Schallert, D. L., & Reynolds, R. E. (2009). What is learning anyway? A
topographical perspective considered. Educational Psychologist, 44(3), 176-192.
doi:10.1080/00461520903029006
Ashley, S. S. (2009). Self-efficacy beliefs of elementary general education teachers in inclusive
classrooms and the role of professional development (Order No. 3357529). Available
from ProQuest Central; ProQuest Dissertations & Theses Full Text; ProQuest
Dissertations & Theses Global; ProQuest Education Journals. (305028424). Retrieved
from
http://libproxy.usc.edu/login?url=http://search.proquest.com.libproxy1.usc.edu/docview/3
05028424?accountid=14749
Bandura, A., 1925. (1997). Self-efficacy: The exercise of control. New York: W.H. Freeman.
Blanton, L. P., Pugach, M. C., & Florian, L. (2011, May 9). Preparing General Education
Teachers to Improve Outcomes ... Retrieved April 28, 2016, from
http://www.ncld.org/wp-content/uploads/2014/11/aacte_ncld_recommendation.pdf
Busby, R., Ingram, R., Bowron, R., Oliver, J., & Lyons, B. (2012). Teaching elementary children
with autism: Addressing teacher challenges and preparation needs. The Rural Educator,
33(2), 27.
Carpenter, S. K. (2012). Testing enhances the transfer of learning. Current Directions in
Psychological Science, 21(5), 279-283. doi:10.1177/0963721412452728
EVALUATING THE IMPLEMENTATION OF 21
ST
87

Clark, Richard E. 1940- (Richard Edward), & Estes, F., 1950. (2008). Turning research into
results: A guide to selecting the right performance solutions. Charlotte, N.C: Information
Age Pub Inc.
Competencies, skills, and knowledge teacher education programs need to teach the inclusion
teacher. (1994). (). Retrieved from
http://libproxy.usc.edu/login?url=http://search.proquest.com.libproxy2.usc.edu/docview/6
2812165?accountid=14749
Conroy T., Yell M. L., Katsiyannis A. (2008). The Supreme Court on the burden of persuasion
when challenging IEPs. Remedial and Special Education, 29, 108-117.
Cox, M. (2009). The commitment of elementary school teachers to inclusive education for
children with disabilities
Diliberto J. A., Brewer D. (2012). Six tips for successful IEP Meetings. Teaching Exceptional
Children, 44, 30-37.
Elmore, R. F. (2001). Psychiatrists and light bulbs: Educational accountability and the problem
of capacity. Paper presented at the annual meeting of the American Educational
Research Association, Seattle, Washington.
Fiore, D. (2000). Positive school cultures: The importance of visible leaders. Contemporary
Education, 71(2), 11. Retrieved from
http://libproxy.usc.edu/login?url=http://search.proquest.com.libproxy1.usc.edu/docview/1
291789189?accountid=14749
Fiske, S. T., & Taylor, S. E. (1991). Social cognition (2nd ed.). New York: McGraw-Hill
EVALUATING THE IMPLEMENTATION OF 21
ST
88

Florian, L. (1998). An examination of the practical problems associated with the implementation
of inclusive education policies. Support for Learning, 13(3), 105-108. doi:10.1111/1467-
9604.00069
Ford, Bridgie A., Stuart, Denise H., & Vakil, Shernavaz. (2014). Culturally Responsive Teaching
in the 21st Century Inclusive Classroom. Journal of the International Association of
Special Education, 15(2), 56-62.
Gallimore, R., & Goldenberg, C. (2001). Analyzing cultural models and settings to connect
minority achievement and school improvement research. Educational Psychologist,
36(1), 45–56. https://doi.org/10.1207/S15326985EP3601_5
Gartin B. C., Murdick N. L. (2005). IDEA 2004: The IEP. Remedial and Special Education, 26,
327-331.
Goldstein, I. L., & Ford, J. K. (2002). Training in organizations. Wadsworth, 4
th
ed. Belmont,
California.
Grayson, D. (1990). Designing and Implementing a Research-based Intervention Program to
Transform Education for the 21st Century: An Inclusive Approach to Excellence,
ProQuest Dissertations and Theses.
Guskey, T.R. and Passaro, P.D. 1994. Teacher efficacy: A study of construct dimensions.
American Educational Research Journal, 31(3): 627–643.
Heflin, L., & Simpson, R. (1998). Interventions for children and youth with autism: Prudent
choices in a world of exaggerated claims and empty promises. part II: Legal/Policy
analysis and recommendations for selecting interventions and treatments. Focus on
Autism and Other Developmental Disabilities, 13(4), 212-220.
doi:10.1177/108835769801300402
EVALUATING THE IMPLEMENTATION OF 21
ST
89

Jordan, A., & Stanovich, P. (2001). Patterns of teacher-student interaction in inclusive
elementary classrooms and correlates with student self-concept. International Journal of
Disability, Development and Education, 48(1), 33-52. Retrieved from
http://libproxy.usc.edu/login?url=http://search.proquest.com.libproxy1.usc.edu/docview/6
2354902?accountid=14749
Kirkpatrick, D. L., Kirkpatrick, J. D., & Ebooks Corporation. (2006). Evaluating training
programs: The four levels (3rd ed.). San Francisco, CA: Berrett-Koehler.
Kirkpatrick, J. D., & Kirkpatrick, W. K. (2016). Kirkpatrick's four levels of training evaluation.
Alexandria, VA: Association for Talent Development.
Kniveton, B. H. (2004). A study of perceptions that significant others hold of the inclusion of
children with difficulties in mainstream classes. Educational Studies, 30(3), 331-343.
doi:10.1080/0305569042000224260
Krathwohl, D. R. (2002). A revision of bloom's taxonomy: An overview. Theory into Practice,
41(4), 212-218. doi:10.1207/s15430421tip4104_2
Lockwood, B. S. (2004). The value of administrative supports in relation to elementary and
secondary special education teachers (Order No. 3126198). Available from ProQuest
Dissertations & Theses Full Text; ProQuest Dissertations & Theses Global. (305189118).
Retrieved from
http://libproxy.usc.edu/login?url=http://search.proquest.com.libproxy1.usc.edu/docview/3
05189118?accountid=14749
Loschen, R. S. (2015). Parents of children with autism: Empowerment issues related to school
services
EVALUATING THE IMPLEMENTATION OF 21
ST
90

Marsh, J. A., Pane, J. F., & Hamilton, L. S. (2006). Making sense of data-driven decision making
in education: Evidence from recent RAND research. ().RAND Corporation.
Mayer, R. E., 1947. (2011). Applying the science of learning. Boston, MA: Pearson/Allyn &
Bacon.
McCombs-Tolis, J. (2002). Serving students with disabilities via individualized education plan
(IEP) meetings: Employing a self-organizing systems perspective as a philosophical
agent of change. Educational Horizons, 81(1), 33-37.
Neal, T. (2012). Teachers' knowledge and participation in choosing student accommodations
(Order No. 3498214). Available from ProQuest Dissertations & Theses Full Text.
(926427627). Retrieved from
http://libproxy.usc.edu/login?url=http://search.proquest.com.libproxy1.usc.edu/docview/9
26427627?accountid=14749
NICHCY. (2010). When the IEP team meets. Retrieved from
http://nichcy.org/schoolage/iep/meetings#implement
Pintrich, P. R., & Schunk, D. H. (1996). Motivation in education: Theory, research, and
applications. Englewood Cliffs, N.J: Merrill.
Pintrich, P. R. (2003). A motivational science perspective on the role of student motivation in
learning and teaching contexts. Journal of Educational Psychology, 95(4), 667–686.
https://doi.org/10.1037/0022-0663.95.4.667
Reeves, D. (2009). Leading change in your school: How to conquer myths, build
commitment, and get results. Alexandria, VA: Association for Supervision &
Curriculum Development.
EVALUATING THE IMPLEMENTATION OF 21
ST
91

Robinson, C. M. (2012). The influence of inclusion on the academic performance of general
education students on the new jersey assessment of skills and knowledge in grades 6, 7,
and 8
Robinson, R., & Carrington, S. (2002). Professional development for inclusive schooling.
International Journal of Educational Management, 16(5), 239-247.
doi:10.1108/09513540210434612
Rotter, K. (2014). IEP use by general and special education teachers. SAGE Open, 4(2)
doi:10.1177/2158244014530410
Rouse, M., & Florian, L. (1996). Effective inclusive schools: A study in two countries.
Cambridge Journal of Education, 26(1), 71-85. doi:10.1080/0305764960260106
Rueda, R. (2011). The 3 dimensions of improving student performance: Finding the right
solutions to the right problems. New York: Teachers College Press.
Savolainen, H., Engelbrecht, P., Nel, M., & Malinen, O. (2012;2011;). Understanding teachers'
attitudes and self-efficacy in inclusive education: Implications for pre-service and in-
service teacher education. European Journal of Special Needs Education, 27(1), 51-18.
doi:10.1080/08856257.2011.613603
Scott, S., & Palinscar, A. (2006). Sociocultural theory. Retrieved from http://www.education.
com/ reference/article/sociocultural-theory/
Sharma, P. R. (2015). Organizational communication: Perceptions of staff members' level of
communication satisfaction and job satisfaction (Order No. 3708217). Available from
ProQuest Dissertations & Theses Full Text; ProQuest Dissertations & Theses Global.
(1691070109). Retrieved from
EVALUATING THE IMPLEMENTATION OF 21
ST
92

http://libproxy.usc.edu/login?url=http://search.proquest.com.libproxy1.usc.edu/docview/1
691070109?accountid=14749
Singh, D. K. (2007). Preparing general education teachers for inclusion. (). Retrieved from
http://libproxy.usc.edu/login?url=http://search.proquest.com.libproxy1.usc.edu/docview/6
2051160?accountid=14749
Strong, J. E. (2014). Preparing teachers of students with autism spectrum disorder: Evidence-
based practices and teacher self-efficacy
Thompson, S. & Ross, F. (2000), ``Becoming a teacher in a professional development
school'', Teaching and Change, Vol .8 No .1, pp .31-50
Tschannen-Moran, M., Woolfolk Hoy, A. and Hoy, W.K. 1998. Teacher efficacy: Its meaning
and measure. Review of Educational Research, 68(2): 202–248.
Uduigwome, G. E. (2012). Twenty-first century skills and school programs: Implementation at a
suburban elementary school (Order No. 3514332). Available from Dissertations &
Theses @ University of Southern California; ProQuest Dissertations & Theses Full Text;
ProQuest Dissertations & Theses Global. (1027592486). Retrieved from
http://libproxy.usc.edu/login?url=https://search-proquest-
com.libproxy1.usc.edu/docview/1027592486?accountid=14749
Van Reusen, A.,K., Shoho, A. R., & Barker, K. S. (2001). High school teacher attitudes toward
inclusion. The High School Journal, 84(2), 7. Retrieved from
http://libproxy.usc.edu/login?url=http://search.proquest.com.libproxy1.usc.edu/docview/2
20219269?accountid=14749
Villa, R. A., & Thousand, J. S. (2003). Making inclusive education work. Alexandria:
Association for Supervision and Curriculum Development.
EVALUATING THE IMPLEMENTATION OF 21
ST
93

Wernsing, K. M. (2014). General and special education co-teachers' perspectives: The
experiences of students with emotional disturbance included in the general education
classroom
Whitworth, J. W., Ed.D. (1999). A Model for Inclusive Teacher Preparation. Electronic Journal
for Inclusive Education, 1(2).
doi:http://corescholar.libraries.wright.edu/cgi/viewcontent.cgi?article=1012&context=eji
e

EVALUATING THE IMPLEMENTATION OF 21
ST
94

Appendix A: Participating Stakeholders with Sampling Criteria for Interviews
Criterion 1. Participants must have knowledge or be familiar with the topic that way
they have had experience with the subjects identified as learning disabled.
Criterion 2. Participants must have attended teacher credentialing courses, professional
development, and workshop(s) in order to respond to questions regarding the improvement or
need of more professional development.
Criterion 3. Participants must have students identified as learning disabled and are
currently receiving student support services as documented in the Individual Education Plan
Interview Sampling (Recruitment) Strategy and Rationale
Interviews will be a second method of data collection. These interviews will be
conducted face-to-face, one on one, with the purpose of obtaining a conversation where the
participant can share what is going on in their mind (Merriam and Tisdale, 2016). Since most
participants have taught between 5-10 years, many of the selected participants were involved
with teaching in the inclusive classroom. Interviews give participants freedom to share their past
experiences as well as provide historical information (Creswell, 2014) that a multiple-choice
survey lacks. In this research study, a semi-structured interview will be employed. This
approach includes a formal interview using guiding questions specific to the topic (Creswell,
2012). According to Creswell (2016), while the interview is somewhat structured, this method
of interviewing gives the researcher more flexibility to provide less structured questions, and ask
the next question based on the flow of the interview. Because this will be a qualitative study,
purposeful selection of sampling recruitment and strategy will be chosen method. Maxwell
(2013) explains this strategy is particularly useful when settings or persons are deliberately
selected and provide information relevant to the questions and goals. Weiss (1994) uses the term
EVALUATING THE IMPLEMENTATION OF 21
ST
95

“panels” to define a group of experts in the field to whom are privileged witnesses to the
situation or event and can provide information to answer the research questions.

EVALUATING THE IMPLEMENTATION OF 21
ST
96

Appendix B: Ethics
Some general education teachers have a difficult time managing and implementing the
goals and accommodations directed in a student IEP. As a result, many teachers become hesitant
to share their current classroom teaching practices if they are not executed in a manner they feel
confident in. Due to this factor, extra awareness will be given so that, in the end, the
conversations could potentially lead to more candid and honest answers. A series of interviews
will be utilized to understand the perceptions of classroom teachers regarding their understanding
of the students IEP, goals, instructional practices, modifications, and their self-efficacy in
carrying out these tasks.
In looking over potential participants for the study, many important factors can impact
the results of the study. Teachers differ in teacher of teaching experience, ethnicity, age, socio-
economic status, marital status, physical health, and mental well-being. While these aspects do
not impact the teachers participating in the study, ethically, it is imperative to understand that
each participant will come in with a unique background that may affect their personal
understanding of a situation.
Before beginning the interview process, each participant will receive a consent form
which will describe the title, purpose, nature, procedures, risk, benefits, and confidentiality of the
research study. All participants will be informed that they may opt out, at any time, from
participating in the study. Additionally, participants will be informed on how the study will be
used in the future. Participants will be allowed to ask questions, as well as, obtain a copy of the
results at the end of the study

EVALUATING THE IMPLEMENTATION OF 21
ST
97

Appendix C: Protocols

Interview Instrument
The purpose of the interview is to explore educators’ self-efficacy towards the
implementation of 21
st
Century Pedagogy and Skills in the inclusive classroom.
Interview Form
Interviewee: (Name / Grade Level)______________________________________
Interview Sections: Demographics/Background

Post Interview Comments or Leads: ____Can you recommend another faculty member
who might participate in this study?
______________________________________________________________________
______________________________________________________________________
Introductory Protocol
To facilitate my notes, I would like to record our conversation today. I am in receipt of
your signed consent form. For your confidentiality, only I will be involved with the study will
have access to the recording which will be eventually destroyed after transcription. Also, you
signed a form designed to protect research participants. Essentially, that document stated that: (1)
all information will be held confidential, (2) your participation is voluntary, and you may stop at
any time if you feel uncomfortable, and (3) the research does not intend to inflict any harm.
Thank you for agreeing to participate.
I have planned this interview to last no longer than one hour. During this time, there are
several questions to cover. If time begins to run short, I would be interested in meeting once
more for a follow up.
EVALUATING THE IMPLEMENTATION OF 21
ST
98

Introduction
You have been selected to participate in this research because you have been identified as
someone who has a great deal of knowledge regarding in inclusive education. This research
project focuses on the improvement of the implementation of 21
st
Century Skills and Learning in
the inclusive classroom, with interest in understanding how teaching staff is engaged in learning
and using technology, how they view the advantages and disadvantages of using innovative
methods towards student learning outcomes, and what issues are of most importance to the
organization in considering the implementation of 21
st
Century education. This study is not
designed to evaluate your techniques or experiences but is for learning more about areas of
support needed in teaching 21
st
Century Skills and Learning so that we can hopefully help
organizations support their educators and students better.
Interview Questions
Demographics
1. What are you teaching right now?
2. How long have you taught?
3. What grade levels have you taught?
4. Have you ever taught in an inclusive classroom?
5. What type of preparation have you had in the area of 21
st
Century Pedagogy and Skills?
For example, classes or Professional Development?

Knowledge - Factual
6. In your own words, define 21
st
Century Pedagogy and Skills.
EVALUATING THE IMPLEMENTATION OF 21
ST
99

7. What type of instructional practices or strategies do believe best support the development
of 21
st
century skills?
8. Which pedagogical practices are helpful in developing 21
st
century skills in students?
9. Can you explain an experience you had implementing 21
st
Century instruction, if
applicable? Something that maybe worked well, and you saw students using 21
st
century
skills?
Knowledge - Conceptual
10. What are your own expectations of teaching 21
st
Century instruction?
11. What advantages exist, if any, in using 21
st
Century instruction as opposed to traditional
teaching instruction?
12. What are the advantages and disadvantages you have experienced using 21
st
Century
instruction, if applicable?
Knowledge – Procedural
13. Describe the process used to implement 21
st
Century Pedagogy and Skills within your
grade level or describe how other teachers might take steps to begin implementation of
21
st
Century Pedagogy and Skills.
14. What 21
st
Century Pedagogy and Skills teaching strategies do you think you have
successfully implemented?
15. Describe some 21
st
Century Pedagogy and Skills strategies used to further enhance core
subject instruction.
Knowledge - Metacognitive
16. Reflecting on a past experience, how do you know that your 21
st
Century instructional
techniques were working.
EVALUATING THE IMPLEMENTATION OF 21
ST
100

17. What are you looking for to ensure students are developing 21
st
Century Skills?
18. Walk me through your preparation developing a lesson that includes 21
st
Century
Pedagogy and Skills. What challenges, if any, do you face?
19. What would you change or modify in your preparation after your experience?
Motivation – Expectancy Value Theory
20. What are your expected outcomes in regards to 21
st
Century Pedagogy and Skills,
through improving student learning outcomes, such as mandated state testing?
21. How important do you feel it is for you to use 21
st
Century Pedagogy and Skills for
classroom instruction?
22. How much time do you think it will take to learn 21
st
Century Pedagogy and Skills for
instruction? How much time to meet mastery? What do you think is needed to be
successful at 21
st
Century Pedagogy and Skills instruction?
Motivation - Attributes
23. What would be an assessment of a successful instructional outcome using 21
st
Century
Pedagogy and Skills? Unsuccessful learning outcomes?
24. What would encourage or discourage you from further implementing 21
st
Century
Pedagogy and Skills?
Motivation – Self-efficacy
23. How do you feel about your ability to use 21
st
Century Pedagogy and Skills for
instruction in the classroom?
24. To what degree do you feel confident about your ability to use 21
st
Century Skills and
Learning for instruction in the classroom?
Motivation – Goals
EVALUATING THE IMPLEMENTATION OF 21
ST
101

25. What might be some specific goals you are wanting to achieve in terms of 21
st
Century
Pedagogy and Skills instruction techniques?
Organizational – Cultural Settings
26. Describe the support you would like to have or have had for your 21
st
Century Pedagogy
and Skills experience in the classroom.
27. Describe any experiences you have had collaborating with either administration or
faculty in adopting 21
st
Century Pedagogy and Skills techniques in your classroom.
28. What do you think are the advantages and disadvantages to school-wide adoption of 21
st

Century Skills and Learning?
29. What would your organization need to ensure full implementation of 21
st
Century
Pedagogy and Skills instruction school-wide?
30. Describe the process used to implement 21
st
Century Pedagogy and Skills school-wide or
describe how your school might take steps to fully implement 21
st
Century Pedagogy and
Skills in the classroom.
31. What support do you need in your work for implementing 21
st
century pedagogy in the
classroom? What can your school do or offer to support you in the classroom?
32. What recommendations do you have for the school to move forward in full
implementation of 21
st
century pedagogical practices?
Ending Protocol
We have completed all the questions that relate to 21
st
Century Pedagogy and Skills
teaching, assessment, and improvement. Is there anything else you would like to add?
EVALUATING THE IMPLEMENTATION OF 21
ST
102

I might be in contact to schedule a follow-up interview as needed. Thank you very much for your
time. You provided some useful information that will greatly benefit this study. Your
participation is appreciated.

EVALUATING THE IMPLEMENTATION OF 21
ST
103

Appendix D: Protocols

Informed Consent/Information Sheet

Title of Study
Evaluating 21
st
Century Skills and Learning in the Inclusive Classroom

Researcher
Marsha A. Coloma-Flores
Doctoral Student at the University of Southern California
Dr. Mora-Flores, Dissertation Chair

Purpose of the Study
The purpose of this study is to evaluate the Knowledge, Motivation, and Organizational
influences of 21
st
Century Skills and Learning practices in the inclusive classroom at Canyon
Valley Elementary School. This study’s goal is to evaluate and improve current instructional
practices and teacher self-efficacy in guiding students to meet 21
st
Century learning academic
goals written in the district plan. This study will examine the knowledge and motivation of the
teachers, as primary stakeholders. The data collected will be analyzed in order to make
recommendations for implementation to improve teacher efficacy, instructional readiness, and
support for all students enrolled in an inclusive classroom at Canyon Valley Elementary School.

Study Procedures
I will survey all teachers at Canyon Valley Elementary School. I will then interview 5 teachers,
one-on-one, from a variety of grade levels.

Participant Rights and Confidentiality
Your participation in this study is voluntary. You have the right to decide whether you would
like to participate in this study or not with no risk to you. Your responses will remain
confidential and you have the right to discontinue your participation in this study without
concern of any adverse effects.

Consent
I understand that my participation in this study is voluntary and that I have the right to
discontinue my participation at any time. I also understand that no harm will come to me through
my participation in this study. My responses will remain confidential. I understand that I have
the right to ask questions at any time throughout this process.
Participant’s Signature: ___________________________ Date: _______________

Investigator’s Signature: __________________________ Date: _______________
EVALUATING THE IMPLEMENTATION OF 21
ST
104

Appendix E
Integrated Implementation and Evaluation Plan
Implementation and Evaluation Framework
The New World Kirkpatrick Model informs the implementation and evaluation plan of
this study (Kirkpatrick & Kirkpatrick 2016). This model is based off of Kirkpatrick and
Kirkpatrick’s four levels of training and evaluation. This New World Kirkpatrick Model
recommends that the four levels of training and evaluation be planned in reverse order starting
with: Level 4 (Results), Level 3 (Behavior), Level 2 (Learning) and Level 1(Reaction). This
model suggests that Level 4 measures the results target outcomes by leading indicators to ensure
that critical behaviors are on track to achieve the desired results. Through Level 4 , the
organization can evaluate how much the individuals transfer what they learned in training once
they are back in the department. Level 3 consists of critical behaviors, required drivers, and on-
the-job learning (Kirkpatrick & Kirkpatrick, 2016). The critical behaviors are the key behaviors
that the individuals must consistently be able to perform and required drivers are ways to
monitor, encourage, reinforce, and reward the continued use of the critical behaviors. With level
2, participants are evaluated on the degree of knowledge, skills, attitude, confidence, and
commitment they learned from training. Finally, with Level 1, the organization can evaluate the
participants’ reaction to the training, such as satisfaction, engagement, and relevance
(Kirkpatrick & Kirkpatrick, 2016). Creating the implementation and evaluation framework
using this model requires the organizational goal to be integrated with the recommendation for
solutions to increase the support needed to successfully implement change (Kirkpatrick &
Kirkpatrick, 2016).

EVALUATING THE IMPLEMENTATION OF 21
ST
105

Organizational Purpose, Need and Expectations
CVES is an elementary school with a mission to transform public education and provide
students with high quality instruction by applying 21st Century education practices. CVES has a
goal of 100% ELA, ELD, and Math lesson implementation using 21st Century education. While
the district has worked to provide a Common Core aligned program in all core areas, it shares an
internal problem to see test score gains among its student population in the inclusive classroom.
The districts planned actions and services to meet these goals include utilization of 21st Century
instructional practices, including grade level and cross grade level collaboration for certificated
teaching staff, design ongoing formative assessments, analyze student data, and participate in
modeled lessons.
The recommendations within this chapter address the knowledge, motivation, and
organizational influences pertaining to teacher self-efficacy in implementing 21st Century skills
and learning in the inclusive classroom. Although the recommendations target individual KMO
elements, three overarching themes describe the expectations for the outcomes of meeting the
organizational goal of 100% implementation of 21st Century Skills and Learning in the inclusive
classroom. First, teacher metacognitive awareness of the interactions and influences of self-
efficacy and goal orientation on 21st Century pedagogy. Such awareness may produce
heightened performance in areas critical to instructional delivery. Second, administration’s
awareness of empowering and developing staff through trust and collaboration and
recommended strategies. Finally, the recommendations and inclusion of the New World
Kirkpatrick Model (Kirkpatrick & Kirkpatrick, 2016) will serve as a standard for evaluations and
training in the future, as well as for continuous examining of the process and procedures at
CVES.
EVALUATING THE IMPLEMENTATION OF 21
ST
106

Level 4: Results and Leading Indicators

The leading indicators are used to measure accomplishments and/or undesirable
outcomes by tracking the critical behaviors impact on the desired outcomes (Kirkpatrick &
Kirkpatrick, 2016). The proposed leading indicators external and internal outcomes, metrics, and
methods are shown in Table 12 that indicates teachers are achieving their desired results. It is
expected that with training, job aid, and organizational support, the internal outcomes will be
met. The external outcomes should also be met as the internal outcomes are accomplished.

Table 12

Outcomes, Metrics, and Methods for External and Internal Outcomes

Outcome Metric(s) Method(s)
Internal Outcomes
1) Increase teachers
pedagogical practices on the
implementation of 21st Century
teaching strategies that support
student learning in the inclusive
classroom
One to two Positive/Negative
examples, per teacher, of experiences
of current teaching situations.
Compare annual student data and
teaching examples of instructional
situations.
2) Increase teachers
understanding of how to create
collaborative 21st Century
Pedagogy and Skills session
through empowering and
developing teacher self-
efficacy.
One to two Positive/Negative feedback
comments from teaching staff.
Set aside regular time to practice
model instruction using 21st Century
teaching strategies and feedback
between site administrators or lead
teachers.
3) Increased teachers’
awareness of their teaching
effectiveness of 21st Century
Pedagogy and Skills.
The frequency of self-reflection by
teachers on effectiveness.
Share out annual data on instructional
effectiveness during grade level
meetings and/or staff meetings.
4) Increased use of teacher
development and collaboration.
One to two Positive/Negative feedback
comments from lead teachers and site
administrators during observations.
Set aside regular time for 1:1
conversations between staff and site
administrators.
5) Increased staff trust of
teacher instructional knowledge
between one another
On a monthly basis, each teacher will
describe one teaching situation and
seek or be provided support. Each
teacher will provide at least one form
of Feedback in meetings to support one
another.
On a monthly basis, during staff
meetings, all teachers will share
requests for support.
EVALUATING THE IMPLEMENTATION OF 21
ST
107

Increased time for developing
and recommending teaching
strategies.
Each trimester, teachers will meet to
determine the workload and develop a
collaborative plan to redesign the work
process.
Set aside regular time to develop a
plan to redesign work. Set aside time
to implement the plan.
External Outcomes
Increase student academic
outcomes in core subjects.
Collect data annually from student
assessments.
Compare data, school wide, within
each trimester.

Level 3: Behavior
Critical behaviors. The stakeholders of focus are the classroom teachers for supporting
students in the 21st Century classroom. The first critical behavior is that teachers will meet to
document their current experiences in implementing 21st Century instruction in the classroom.
The second critical behavior is that observations of 21st Century instruction must be conducted
during real time classroom instruction with a follow up meeting regarding lead teacher and
school administration feedback on the teacher’s ability to engage all students in 21st Century
learning strategies. Third critical behaviors is that teachers will self-reflect during weekly
collaboration days about their teaching performance, how well they support and assess student
needs throughout their instruction, and how well students are responding to 21st Century
learning strategies. The specific metrics, methods, and timing for each outcome behaviors
appear in Table 13.
Table 13

Critical Behaviors, Metrics, Methods, and Timing for Evaluation
Critical Behavior Metric(s)
Method(s)
Timing
1) Teachers will meet to
document their
experiences in
implementing 21st
Century instruction in the
classroom.
The number of positive
experiences where a
students needs were met
by the 21st Century
instructional strategies.
Teachers will compare
methods that have been
successful in the classroom
and choose one or two
specifically weaker strategies
to improve.
Monthly.
2) Lead teachers and site
administrator will conduct
observations of teaching
sessions that will include
The number or
instructional issues
observed during the
teacher evaluation session.
The lead teacher and site
administrator will provide
feedback during the post
evaluation sessions.
Monthly for the first
trimester, thereafter,
annually, as long as
EVALUATING THE IMPLEMENTATION OF 21
ST
108

post feedback on the
teacher’s ability to roll out
the 21st Century teaching
strategy of focus.
positive outcomes are
being made.
3) Teachers will self-
reflect during weekly
collaboration time in
regard to their
instructional performance,
how well they support and
assess student needs, and
how students are
responding to 21st Century
learning strategies.
The number of successes
during reflection.
Teachers will report out at
each staff meeting the overall
outcomes and effectiveness
of the chosen strategy of
focus.
Collaborations are done
weekly. Feedback from
site administrators will
be provided monthly or
after any informal
walkthrough.

Required drivers. Teacher require the support and collaboration of administrative
leadership, instructional peers, and the entire school organization to reinforce, encourage,
reward, and monitor the instructional strategies implemented in the classroom. Reinforcement is
used to remind teachers of what was learned and provide constructive feedback and extra
training, if necessary. Encouragement allows leadership ways to provide coaching and
mentoring. Rewarding is used to provide incentives for critical behaviors. Finally, monitoring
ensures that leaders are being held accountable for evaluating teacher performance. Table 14
describes the recommended drivers to support critical behavior of the support staff team.
Table 14
Required Drivers to Support Critical Behaviors
Method(s) Timing
Critical Behaviors Supported
1, 2, 3 Etc.
Reinforcing
Create a job aid chart to
connect previous experiences
implementing 21st Century
pedagogy with current
understanding.
Ongoing 1, 2, 3
Provide teachers and school
administrators with 21st
Century pedagogical training
on how to implement
instructional strategies and
Ongoing 1, 2, 3
EVALUATING THE IMPLEMENTATION OF 21
ST
109

techniques that meet all
students needs.
Encouraging
Peer modeling instructional
sessions as a tool to allow
instructional peers and school
administrators an opportunity
to empower one another and
provide constructive
feedback.
Annually 1, 2, 3
Feedback from grade level
peers and administration to be
discussed at grade level
meetings, post evaluations, or
post modeling sessions.
Monthly 1, 2, 3
Rewarding
Provide monthly opportunities for
teachers to collaborate on which
21st Century Strategies have been
effective, building upon staff’s
interest in their individual growth as
teachers with the intent to meeting
the school’s vision and mission.
Monthly 1, 2, 3
Monitoring
Time will be set aside in regularly
scheduled staff meetings to allow
teachers and administrators an
opportunity to communicate their
plans, progress, and support for one
another.
Ongoing 1, 2, 3
Administrators and teaching staff
will sit in instructional modeling
sessions and observe.
Annually 1, 2

Organizational support. To ensure that the required drivers are implemented, the
organization will provide the following support. First, review the current workload and available
resources, and collaboratively develop a plan to restructure the implementation of 21st Century
instruction, which will take into account the organization’s goal. Furthermore, time will be set
aside in regularly scheduled staff meetings to provide time for staff and administration the
opportunity to share their plans, progress, and support for one another to foster a cohesive
instructional environment. Moreover, the organization will provide monthly updates and
EVALUATING THE IMPLEMENTATION OF 21
ST
110

assessment on current student progress, and identify rates of growth and change that connect
with the vision and goal of the school. Finally, annually, the organizational will share out
teacher self-efficacy on the implementation of the discussed 21st Century strategies and skills
discussed at collaboration meetings.
Level 2: Learning
Learning goals. Upon completion of the recommended solutions, the teachers will be
able to:
1. Teachers know what it means to implement 21
st
Century skill-based curriculum.
2. Teachers will know how to support all students in 21st Century skills. (P)
3. Teacher’s will know how to compile data prior to each Individual Educational Plan for
each of their identified. (P)
4. Teachers will need to know how to execute the plans written for all learners in the
inclusive classroom. (P)
5. Teachers will believe that all students are capable of high outcomes in the 21st Century
based on curricular program (Value)
6. Teachers will feel they are effective educators and have the skills to teach all learners in
the inclusive classroom. (Self-Efficacy)
Program. The learning goals provided in the previous section will be achieved through
training and exercises that will increase the knowledge and motivation of the teachers to provide
data on perceived learning barriers to developing and recommending strategies to support student
achievement through increased instructional pedagogy of 21st Century skills. To develop
teacher’ knowledge and skills they will be provided with training, peer observations, and job
aids. The program will be ongoing throughout the year. First, prior to the beginning of each
EVALUATING THE IMPLEMENTATION OF 21
ST
111

school year, teachers will receive an in-depth one week training on how to scaffold instruction
and assist in student performance in order for all students to receive appropriate instruction.
These trainings will provide teachers with opportunities to learn new methods and techniques,
and discuss with fellow staff members when 21st Century strategies were used where students’
needs were assessed and met. Through this method, teachers will be able to develop a list of
successful and unsuccessful methods, and utilize teachers’ skills to decide which teachers will
model instruction to fellow staff members. Next, teachers will design a schedule, for the first
semester, that will release teachers from class to observe 21st Century instructional methods
performed by fellow staff members. These observations will also include time for teachers to
debrief what was observed and how they will incorporate the teaching skill in their classroom
lessons. Furthermore, teachers will be asked to decide on a teaching strategy that will be used in
a core lesson, and they will then collect pre- (before the strategy is used) and post- (after the
strategy is used) data and share amongst grade level team members to reflect on effectiveness.
The teachers, on an ongoing basis, will also learn 21st Century topics that will increase
their motivation to provide data on perceived barriers to developing and recommending
strategies to support student learning for all types of learners. To increase teacher motivation,
they will participate in advising sessions with the school site administrator and instructional
coach, and provide data on their use of 21st Century pedagogy. First, teachers will participate in
administrative evaluations once every nine weeks for either a language arts or math lesson. After
that, a second evaluation will take place during the second trimester, or as long as the
administrator sees necessary. During each session, the administrator will complete a checklist to
evaluate the teacher’s use of 21st Century techniques. Following each evaluation, school
administrator and instructional coach will provide corrective feedback and positive
EVALUATING THE IMPLEMENTATION OF 21
ST
112

encouragement about the teacher’s ability to effectively utilize 21st Century instructional
techniques. In addition, teacher’s will be asked to share out their instructional successes among
the teaching staff, during staff meetings, and the positive impact it has had on their students’
learning.
Evaluation of the Components of learning. To apply what is learned to solve problems
and to meet performance goals, the teachers must have the knowledge and skills and motivation
to achieve their goals. Therefore, it is important to assess learning for both the conceptual and
procedural knowledge being taught. It is also important that the teachers value training, are
committed, and confident so that they can apply what they have learned on a daily basis. As
such, Table 15 lists the evaluation methods and timing for these learning components.
Table 15
Evaluation of the Components of Learning for the Program.
Method(s) or Activity(ies) Timing
Declarative Knowledge “I know it.”
Knowledge checks through discussion during
group tabling.
Periodically, every nine weeks, during staff
meetings, and documented via job aid chart.
Report out on table discussions. Throughout instructional training, tracking is
implemented to ensure that all attendees are
participating and reporting out.

Procedural Skills “I can do it right now.”
Feedback from peers during instructional
modeling sessions.
Immediately after the instructional modeling
session.
Use of 21st Century instructional strategies
with class.
During the instructional modeling session.

Attitude “I believe this is worthwhile.”
Pre- and Post- survey to determine if the value
of the instructional modeling has increased
teacher knowledge and efficacy to increase
student understanding of core concepts.
At the end of the workshop.
EVALUATING THE IMPLEMENTATION OF 21
ST
113

Determine the 21st Century instructional
strategies to be used in future lessons.
During learning event.

Confidence “I think I can do it on the job.”
Feedback from school administrator,
instructional coach, and grade level peers.
Immediately after the learning event.
Allotted time set aside for school administrator
and instructional coach to address concerns
and developmental advising.
After the learning event.

Commitment “I will do it on the job.”
1:1 Discussions following observations After the learning event.
Ask the attendees to share how they will
implement what they have learned on the job.
After the learning event.

Level 1: Reaction
It is important to determine how the participants react to the learning event. Thus, it is
essential to confirm that the quality of the learning event was acceptable by the participants. As
such, Table 16 lists the reactions of the participants to the learning event being favorable,
engaging, and relevant.
Table 16
Components to Measure Reactions to the Program.
Method(s) or Tool(s) Timing
Engagement
Checklist of rating observation completed by
grade level peers, instructional coach, and site
administrator.
On-going after training event.
Workshop Evaluation At the following grade level meeting.
Relevance
Pulse check with school site administrator via
1:1 discussion
After each observation.
Workshop evaluation At the following grade level meeting.
Customer Satisfaction
Pulse check with instructional model attendees
via satisfaction survey
After each training event
EVALUATING THE IMPLEMENTATION OF 21
ST
114

Workshop Evaluation At the following grade level meeting.

Evaluation Tools
Immediately following the program implementation. Following the learning event,
participants will complete a survey (see Appendix A for the survey questions). During the
instructional model observations, the school site administrator, instructional coach, and grade
level peers will fill out a checklist (see Appendix B for the checklist). The survey will indicate
the relevance of the material to the job, participant satisfaction, commitment, attitude, and
confidence in apply what has been modeled and learned.
For Level 1 and Level 2, during the observations, the school site administrator,
instructional coach, and grade level peers will fill out a checklist that rates the effectiveness of
the classroom teacher in several areas and then will provide feedback. During in person learning
events, the classroom teacher will conduct pulse checks by asking the participants if the content
is relevant to their work in addressing realistic classroom instruction issues. The classroom
teacher will ask about the environment and if it is creating any barriers to the students’ learning.
Level 2 will include checks for understanding what is being presented. Level 2 will also utilize
group discussions and reporting out on the 21st Century instructional strategies being
implemented to gauge participant understanding.
Delayed for a period after the program implementation. Approximately six weeks
after the learning event, the organization will administer a survey (see Appendix C for survey
questions) containing open and scaled items using the blended evaluation approach to measure,
from the trainer’s perspective, satisfaction and relevance of the training to the teacher’s ability to
provide data on perceived barriers to developing and recommending strategies to support 21st
Century pedagogy and instruction to the inclusive classroom (Level 1), knowledge, skills,
EVALUATING THE IMPLEMENTATION OF 21
ST
115

confidence, attitude, commitment and value of applying their training (Level 2), application of
the learning event to the teacher’s ability to assess student understanding, and execution of 21st
Century learning skills (Level 3), and the extent to which they are able to provide 21st Century
instruction in the inclusive classroom, on a regular basis (Level 4).
Data Analysis and Reporting:
The Level 4 goal for teachers, in the inclusive classroom, is measured by how the
students, of various learning needs, are able to perform 21st Century learning skills. The
teachers must have the knowledge, skills, and motivation to provide data on perceived barriers to
developing 21st Century instruction and support in the inclusive classroom. Each trimester,
teachers will compile data from informative assessment to track which 21st Century instructional
methods are supporting student learning in a way that has made an impact on student
achievement. Every six weeks, teachers will track student progress to see if there has been an
increase in student learning in both ELA and Math. To monitor student progress, and hold
teachers accountable, the dashboard below will report the data on these measures. Similar
dashboards will be created to monitor Levels 1, 2, and 3.
Table 17
Dashboard Goal Fall
2018
Winter
2018
Spring
2019
2018-2019
Annual Totals
Teachers are knowledgeable about how to
implement various 21st Century
instructional strategies.
100% XX XX XX XX
Teachers 21st Century instructional skill in
the inclusive classroom.
95% XX XX XX XX
Student success rate - Fall 2018 80% XX XX XX XX
Student success rate - Winter 2018 80% XX XX XX XX
Student success rate - Spring 2019 85% XX XX XX XX
EVALUATING THE IMPLEMENTATION OF 21
ST
116

Summary:

The New World Kirkpatrick Model provides information on the implementation and
evaluation of this study (Kirkpatrick & Kirkpatrick, 2016). The four levels of training and
evaluation are applied to this study to ensure that teachers have the knowledge, motivation, and
organizational skills to provide data on perceived barriers to developing and providing
recommendations on 21st Century instructional strategies to support the inclusive classroom.
With this model, the 21st Century instructional training program starts with the identification of
outcomes that are binded within the organizational goals. Next, the program sets the critical
behaviors to evaluate whether or not the participants are applying what they have learned, once
they are back in the classroom. Furthermore, learning outcomes are identified and the
participants are then evaluated on their learning, knowledge, attitude, commitment, and self-
efficacy during the training. Finally, methods to evaluate participant reactions on the training
were created to determine participants’ satisfaction, engagement, and relevance. To implement
change and gain maximum results, it is crucial to evaluate and analyze data collection during
program implementation. For the levels of training evaluation, this section will answer the
questions: “Does the level of …. meet expectations?” If so, then why? and if not, then why not?”
(Kirkpatrick & Kirkpatrick, 2016).
During the training when the level of reaction and learning does not meet expectations,
then the trainer and administrators need to identify the issue and changes need to be made to the
program. If the teachers are not learning or reacting as expected, it is highly advised that the
trainers make checks to answer any participant questions (Kirkpatrick & Kirkpatrick, 2016).
EVALUATING THE IMPLEMENTATION OF 21
ST
117

When the level of reaction and learning meets expectations, the trainer should check for
understanding and what helped to increase engagement (Kirkpatrick & Kirkpatrick, 2016).
After training, when instructional outcomes do not meet expectations, then it is
imperative to communicate with the participants to investigate what of the level 3 behaviors are
not being applied, and how these indicators are affecting level 4 results (Kirkpatrick &
Kirkpatrick, 2016). The trainer can offer feedback through surveys and interviews by asking the
participants which behaviors would allow them to progress further to achieve the desired goals.
Lastly, when the levels of behavior and results are met, it is crucial that participants begin
identifying which instructional strategies they are doing to continue to increase their performance
and share out within the organization (Kirkpatrick & Kirkpatrick, 2016).
Finally, it would be key to provide a finalized report on training outcomes to the
participating staff and administration. Support from the organization determines the success of
any training program (Kirkpatrick & Kirkpatrick, 2016). Kirkpatrick and Kirkpatrick (2016)
recommend organizations to utilize reports and touch points to drive performance throughout the
process of implementation. Reports should address areas such as relevance of training,
credibility, and efficiency of the training program (Kirkpatrick & Kirkpatrick, 2016).

EVALUATING THE IMPLEMENTATION OF 21
ST
118

Appendix F
For each of the questions below, circle the response that best characterized how you feel about
the statement.

Strongly
Disagree

Strongly
Agree

1. The training held my interest 1 2 3 4 5 6 7
2. During training we discussed how
to apply what we learned.
1 2 3 4 5 6 7
3. I will recommend this training
program to other schools in the
district.
1 2 3 4 5 6 7
4. I believe it will be worthwhile for
me to build my understanding of
21
st
Century pedagogy.
1 2 3 4 5 6 7
5. The feedback has given me the
confidence to apply what I learned
when I return to the classroom.
1 2 3 4 5 6 7
EVALUATING THE IMPLEMENTATION OF 21
ST
119

6. I am committed to applying what
I learned during my post training
evaluation.
1 2 3 4 5 6 7
7. I found the feedback during the
instructional model session valuable
for empowering and developing 21
st

Century instruction in the inclusive
classroom.
1 2 3 4 5 6 7
8. I was satisfied with the training
on 21
st
Century instructional
strategies and techniques.
1 2 3 4 5 6 7
Please provide feedback for the following questions
1. What part of the training did you find irrelevant for your instructional needs?
2. What were the major instructional strategies you learned today?

EVALUATING THE IMPLEMENTATION OF 21
ST
120

Appendix G
For each of the questions below, circle the response that best characterized how you feel
about the statement.

Strongly
Disagree

Strongly
Agree

1. I have had the opportunity to use
what I learned in the classroom.
1 2 3 4 5 6 7
2. Reflecting on the training, I
believe that the training was a good
use of my time.
1 2 3 4 5 6 7
3. After the training, I have
successfully applied what I learned
towards the inclusive classroom.
1 2 3 4 5 6 7
4. I have received support from my
administrator to apply what I have
learned in the inclusive classroom.
1 2 3 4 5 6 7
5. I have received support from my
instructional coach to apply what I
have learned in the inclusive
classroom.
1 2 3 4 5 6 7
EVALUATING THE IMPLEMENTATION OF 21
ST
121

6. This training has positively
impacted teachers understanding
and execution of 21
st
Century
instruction in the inclusive
classroom.
1 2 3 4 5 6 7
Please provide feedback for the following questions:
1. Describe any challenges you are facing implementing what you learned about 21st
Century instruction in the inclusive classroom, and possible solutions to the challenges.

2. Reflecting on this program, how could it have been improved?

Abstract (if available)

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

Strategies California superintendents use to implement 21st century skills programs

PDF

The process secondary administrators use to implement twenty‐first century learning skills in secondary schools

PDF

A study of California public school district superintendents and their implementation of 21st century skills

PDF

STEM teacher education: An evaluation study

PDF

The influence of leadership on the development of 21st century skills throughout Irish schools

PDF

A study of California public school district superintendents and their implementation of 21st-century skills

PDF

The impact of globalization on the development of educational policy, 21st century learning, and education in science, technology, engineering, and mathematics in Costa Rican schools

PDF

The key essentials for learning in the 21st century: programs and practices

PDF

A study of California public school district superintendents and their implementation of 21st century skills

PDF

Teaching academic and behavioral skills through arts-based programming: an evaluation study

PDF

Building 21st century skills for school-age children in Colombia: lessons from a promising practice

PDF

The role of the school district toward preparing students for the 21st century

PDF

Globalization and the need for 21st-century skills: implications for policy education in science, technology, engineering, mathematics, and project-based learning in schools in Ireland

PDF

Factors impacting the effectiveness of mentor teachers in a national teacher residency

PDF

The development of 21st century skills among incarcerated youth: a needs assessment of teachers

PDF

Instructional leadership: the practices employed by elementary school principals to lead the Common Core State Standards and 21st century learning skills

PDF

STEM integration: a case study of a 21st century skills professional development program for math and science teachers in an urban school

PDF

Preparing students for the future - 21st century skills

PDF

Principals’ leadership influence on teachers’ capacity to enact 21st-century skills curriculum

PDF

College and career readiness through independent study: an innovation study

Asset Metadata

Creator Coloma-Flores, Marsha A. (author)

Core Title Evaluating the implementation of 21st century skills and learning

School Rossier School of Education

Degree Doctor of Education

Degree Program Organizational Change and Leadership (On Line)

Publication Date 12/06/2018

Defense Date 08/31/2018

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag 21st century education,21st century learning,evaluating 21st century learning,evaluating 21st century skills,OAI-PMH Harvest

Format application/pdf (imt)

Language English

Contributor Electronically uploaded by the author (provenance)

Advisor Mora-Flores, Eugenia (committee chair), Ferrario, Kimberly (committee member), Pensavalle, Margo (committee member)

Creator Email macoloma@usc.edu,macoloma7@gmail.com

Permanent Link (DOI) https://doi.org/10.25549/usctheses-c89-111085

Unique identifier UC11676953

Identifier etd-ColomaFlor-6996.pdf (filename),usctheses-c89-111085 (legacy record id)

Legacy Identifier etd-ColomaFlor-6996.pdf

Dmrecord 111085

Document Type Dissertation

Format application/pdf (imt)

Rights Coloma-Flores, Marsha A.

Type texts

Source University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Access Conditions The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the a...

Repository Name University of Southern California Digital Library

Repository Location USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA