University of Southern California Dissertations and Theses

College and career readiness through high school STEM programs: an evaluation study

(USC Thesis Other)

College and career readiness through high school STEM programs: an evaluation study

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content

College and Career Readiness Through High School STEM Programs: An Evaluation
Study
by
Praisy Poluan

Rossier School of Education
University of Southern California
A dissertation submitted to the faculty
in partial fulfillment of the requirements for the degree of
Doctor of Education

May 2021

© Copyright by Praisy Poluan 2021
All Rights Reserved

The Committee for Praisy Poluan certifies the approval of this Dissertation

Frederick Freking
Rocky Murray
Kenneth Yates, Committee Chair

Rossier School of Education
University of Southern California
2021

iv
Abstract
This study analyzes college and career readiness through high school STEM programs. The
purpose of this case study is to evaluate the degree to which a STEM-focused college and career
readiness program is achieving its goal of 100% program completion. Clark and Estes’ (2008)
gap analysis framework is adapted to identify the knowledge, motivation, and organizational
influences related to achieving said goal, with the program teachers as the stakeholder of focus.
Qualitative methods, specifically interviews and document analysis, were used to identify and
assess the influences. Interview data were triangulated with web pages and lesson plans to
identify and confirm whether each knowledge, motivation, and organizational influence is an
asset or a need. These findings inform the solutions recommended to bridge the gaps in
influences. Then, the New World Kirkpatrick Model (Kirkpatrick & Kirkpatrick, 2016) is used to
create a plan for implementation and evaluation of the recommended solutions. This study
outlines the expected outcomes of the training program presented as part of the recommended
solutions. Lastly, this study provides recommendations for future research regarding college and
career readiness, STEM Programs, teacher training, and the impact of COVID-19 on high school
STEM Programs.

v
Dedication
To my mother, Stherry Tangkilisan, you are the reason I am here today. I have you to thank for
my never-ending pursuit of education. Like everyone always says, I take after you in many ways,
especially in my constant yearning to return to school and take in all the knowledge and
experience that I can. “Memang anak Stherry.”

To my siblings, Jason, Monique, Illona, and Darlene, thank you for your endless excitement and
support. These past few years have put us through so many life changes both individually and as
a family, and I am so grateful to have had you with me through it all.

To my friends, you are my family. I could not imagine where I would be without your constant
encouragement. Please accept all the mi goreng in return. Thank you for always returning me to
my streets.

To my Trojan squad, I truly could not have gotten through this program without you. I cannot
even begin to express how much I appreciate how much you have supported me throughout this
experience. Bryan, thank you for reminding me that a strong, independent dementor such as
myself can be vulnerable, rely on others, and still be more Barney.

vi
Acknowledgements
To my dissertation chair, Dr. Kenneth Yates, I cannot thank you enough for all the time
and effort you have provided me with throughout this journey. Your wisdom and support have
ignited a passion in me to continue to grow and make a difference in my community. I am sorry I
did not stick with Ed Psych, but I am so glad I was still able to have you as my chair. Your
influence will continue to inspire me.
To my committee member, Dr. Frederick Freking, you fueled the fire in me to
pursue the connections between culture and STEM. I appreciate the scientific lens you brought to
our conversations. I truly am privileged to have been able to work with someone who strives so
hard to bridge STEM and society.
To my committee member, Dr. Rocky Murray, you challenged me to grow and
seek to understand all perspectives. You saw my voice not as an inexperienced one, but as a new
asset. Thank you for pushing me to stay open-minded as I strive to recognize and appreciate all
voices.

vii
Table of Contents
Abstract .......................................................................................................................................... iv
Dedication ....................................................................................................................................... v
Acknowledgements ........................................................................................................................ vi
List of Tables .................................................................................................................................. x
List of Figures ................................................................................................................................ xi
Chapter One: Introduction and Statement of the Problem .............................................................. 1
Background of the Problem ................................................................................................ 1
Importance of Addressing the Problem .............................................................................. 2
Organizational Context and Mission .................................................................................. 3
Organizational Goal ............................................................................................................ 4
Description of Stakeholder Groups ..................................................................................... 5
Stakeholder’ Performance Goals ........................................................................................ 6
Stakeholder Group for the Study ........................................................................................ 7
Stakeholder of Focus Critical Behaviors ............................................................................ 7
Purpose of the Project and Questions ................................................................................. 7
Conceptual and Methodological Framework ...................................................................... 8
Definitions ........................................................................................................................... 8
Organization of the Project ................................................................................................. 9
Chapter Two: Review of the Literature ........................................................................................ 11
Influences on the Problem of Practice .............................................................................. 11
Conceptual Framework ..................................................................................................... 22
Stakeholder Knowledge, Motivation, and Organizational Influences .............................. 23
Chapter Three: Methodology ........................................................................................................ 40
Conceptual and Methodological Framework .................................................................... 40

viii
Assessment of Performance Influences ............................................................................ 42
Participating Stakeholders and Sample Selection ............................................................. 50
Instrumentation ................................................................................................................. 51
Data Collection ................................................................................................................. 52
Data Analysis .................................................................................................................... 53
Trustworthiness of Data .................................................................................................... 53
Role of Investigator ........................................................................................................... 54
Limitations and Delimitations ........................................................................................... 54
Chapter Four: Results and Findings .............................................................................................. 55
Participating Stakeholders ................................................................................................ 55
Determination of Assets and Needs .................................................................................. 56
Results and Findings for Knowledge Causes .................................................................... 57
Results and Findings for Motivation Causes .................................................................... 62
Results and Findings for Organization Causes ................................................................. 67
Chapter Five: Recommendations and Evaluation ......................................................................... 76
Recommendations to Address Knowledge, Motivation, and Organization Influences .... 77
Integrated Implementation and Evaluation Plan ............................................................... 93
Limitations and Delimitations ......................................................................................... 108
Recommendations for Future Research .......................................................................... 108
Conclusion ...................................................................................................................... 109
References ................................................................................................................................... 110
Appendix A: Interview Protocol ................................................................................................ 117
Appendix B: Immediate Evaluation Tool (Levels 1 and 2) ....................................................... 120
Appendix C: Delayed Evaluation Tool (Levels 1, 2, 3, and 4) .................................................. 122
Appendix D: Exempt Research Information Sheet .................................................................... 124

ix
Appendix E: Recruitment Letter ................................................................................................ 126

x
List of Tables
Table 1: Organizational Mission, Global Goal and Stakeholder Performance
Goals
6
Table 2: Summary of Assumed Knowledge Influences on Teachers’ Ability to
Achieve the Performance Goal
27
Table 3: Summary of Assumed Motivation Influences on Teachers’ Ability to
Achieve the Performance Goal
33
Table 4: Summary of Assumed Organization Influences on Teachers’ Ability to
Achieve the Performance Goal
37
Table 5: Summary of Knowledge Influences and Method of Assessment 43
Table 6: Summary of Motivation Influences and Method of Assessment 45
Table 7: Summary of Organization Influences and Method of Assessment 48
Table 8: Knowledge Assets or Needs as Determined by the Data 73
Table 9: Motivation Assets or Needs as Determined by the Data 74
Table 10: Organizational Assets or Needs as Determined by the Data 75
Table 11: Summary of Knowledge Influences and Recommendations 78
Table 12: Summary of Motivation Influences and Recommendations 84
Table 13: Summary of Organization Influences and Recommendations 88
Table 14: Outcomes, Metrics, and Methods for External and Internal Outcomes 96
Table 15: Critical Behaviors, Metrics, Methods, and Timing for Evaluation 97
Table 16: Required Drivers to Support Critical Behaviors 98
Table 17: Evaluation of the Components of Learning for the Program 104
Table 18: Components to Measure Reactions to the Program 105

xi
List of Figures
Figure 1: Gap Analysis Process 41
Figure 2: Sample Dashboard for Level 4 Internal Outcomes 107

1

Chapter One: Introduction and Statement of the Problem
An overwhelming amount of high school graduates lack the college and career readiness
(CCR) skills required for success in postsecondary education and the workforce (Conley &
Darling-Hammond, 2013). Education reformers, business spokespeople, and leaders in higher
education pushed schools to foster the skills needed for the evolving world. However, students
continue to underperform upon entering college and the workforce. On the American College
Testing (ACT) exam, which measures the knowledge and skills acquired up to the beginning of
12th grade and encountered in entry level college courses, high school graduates from 2017 to
2019 averaged between 20.2 and 21.3 out of 36 points (ACT, 2020). On the Scholastic Aptitude
Test (SAT), which evaluates the knowledge and skills learned in high school and needed to
succeed in college, 41% of students in 2019 scored below the college-readiness benchmark for
reading and writing, while 61% scored below the math benchmark (College Board, 2019). These
scores indicate that students are unprepared for college and the workforce, an issue that must be
addressed in K-12 schools. Various attempts have been made to better address college and career
readiness through STEM programs. This study evaluates one such STEM program at a particular
school.
Background of the Problem
The lack of college and career readiness in graduating high school students is problematic
since it could lead to college and workforce attrition. Students who lack CCR skills are
unprepared for post-secondary life, which contributes to low retention rates in college and the
workforce. In a study of non-cognitive, environmental, academic, and other factors relating to
college attrition, Thomas and Tatum (2016) found a correlation between not only college
readiness skills and college retention but also career readiness and college retention.
2

Undergraduate participants viewed their career outlooks after college graduation as motivation to
complete their programs (Thomas & Tatum, 2016). As a result, they recommended the
development of college and career preparation programs for all students in order to foster college
and career readiness skills, therefore encouraging college retention.
College and career readiness programs have a similar effect on workforce attrition. In a
study of the perceptions of CTE programs, Fitzgerald (2018) found that due to a shortage of
skilled tradespeople, workforce leaders believed that career programs fostering necessary entry-
level skills were essential in adequately preparing high school students for the workforce.
Students graduating with entry-level skills is especially important in helping to address the
shortage of skilled tradespeople more immediately (Fitzgerald, 2018). Similarly, Fayer, Lacey,
and Watson (2017) studied past, present, and future STEM occupations and found that the
growth of STEM programs were necessary to satisfy the projected increase in job openings in
STEM fields. Notwithstanding, drafters, engineering technicians, and mapping technicians, all
STEM careers are expected to grow and through 2024 (Fayer et al., 2017). Thus, college and
career readiness programs designed to prepare students for these growing fields are necessary for
fostering success in the workforce. Failure to do so could result in a lack of preparation in high
school, or a lack of college and career readiness, which would result in high attrition rates.
Quality college and career readiness programs at the high school level are necessary for
producing graduates who are prepared for college and the workforce.
Importance of Addressing the Problem
Producing college- and career-ready students is essential for a variety of reasons. Failing
to do so ultimately results in graduates being ill-prepared for success in college or the workforce.
Low college retention can, in part, be attributed to a lack of college and career readiness
3

knowledge and skills. Thomas and Tatum (2016) found a correlation between not only college
readiness skills and retention, but also career readiness and retention. In their study of non-
cognitive factors, college readiness, career readiness, student engagement, natural connected
notes, usefulness of college retention practices, college attrition level, and intention to stay in
college, Thomas and Tatum (2016) found that undergraduate students viewed their post-
graduation career outlook as motivation to stay in college and complete their programs. Thus,
college readiness skills required to complete their programs and career readiness skills necessary
for succeeding in the workforce were both essential for college retention.
Brand (2009) also found connections between labor market outcomes and high school
career academies, which were developed to improve college and career readiness programs.
They found that, for at-risk males especially, enrollment and successful completion in career
academies correlated with improved workforce outcomes such as increased post-high school
rates and earnings, without reducing educational outcomes (Brand, 2009). These career
academies, geared towards preparing students for college and the workforce, exposed students to
real labor market experiences and provided them with the knowledge and skills expected of them
in postsecondary settings. Failure to implement programs, such as career academies, and
procedures to increase college and career readiness could therefore contribute to low workforce
outcomes.
Organizational Context and Mission
The organization of focus is the science, technology, engineering, and mathematics
(STEM) Program at Lights High School (both names are pseudonyms). Lights High School
(LHS), which serves over 2,300 students, is one of six comprehensive high schools in the Sea
Cliffs Union High School District (a pseudonym). The mission of the LHS is to foster college-
4

and career-ready students through academic and career technical education (CTE) pathways.
They offer 25 Advanced Placement (AP), Honors, and Accelerated courses, and three CTE,
including Design, Visual, and Media Arts, Food Services and Hospitality, and Production and
Managerial Arts (Anonymous, 2018). During the 2017-2018 school year, over 700 students
enrolled in at least one AP, Honors, and Accelerated courses, and over 300 students enrolled in
at least one CTE course (Anonymous, 2018). Of last year’s graduates, 33% attend a four-year
university and 61% attend community college or trade school (Anonymous, 2018).
The STEM Program at LHS is a new four-year program dedicated to preparing students
for future careers in STEM through hands-on and minds-on project-based learning. The 2019-
2020 school year was the fourth year of the program, serving almost 200 students as it continues
to grow. A team of seven math and science teachers lead the program, offering freshman through
senior year STEM courses. Many of these courses include University of California Curriculum
Integration (UCCI) courses, designed by California teachers in collaboration with University of
California (UC) faculty to help students prepare for college, explore potential career paths, and
fulfill requirements for freshman admission to UC and CSU campuses. Students enter the
program during their freshman or sophomore year, sometimes junior year in unique
circumstances, and engage in the program learning community activities including, but not
limited to, coursework, special trips, competition teams, community interactions, stewardship,
and projects.
Organizational Goal
The STEM Program teachers decided that, to help prepare their students for
postsecondary lives and also aid LHS in fulfilling its goal of graduating college- and career-
ready students, they would also work to increase college and career readiness through the STEM
5

Program. Since 2020 is the first year of graduating seniors in the program, the STEM program
currently has no graduates. Thus, the STEM Program teachers set a goal of having 100%
program completion from seniors graduating in 2022, one school year after this study is
complete and recommendations have been made towards program improvement. Successful
program completion will be measured by students (a) earning C’s or higher in all STEM classes
throughout the years they are enrolled in the STEM Program; (b) completing stewardship hours
and assignments geared towards giving back to the community and preparing for future careers;
(c) participation in project week each semester as measured by completion of each Project Week
Engineering Notebook; (d) completing a senior STEM research project or paper.
Furthermore, it is important to evaluate the STEM Program’s enrollment and completion.
The evaluation will help indicate to STEM Program teachers how to improve the program in
order to increase enrollment and completion, therefore increasing students’ college and career
readiness through essential knowledge and skills taught in the courses.
Description of Stakeholder Groups
As the goal of the organization is to foster college and career ready students through the
completion of program requirements, the stakeholders are (1) students, who enroll in the STEM
Program and will ultimately graduate with college and career skills fostered through the STEM
courses; (2) teachers, who teach the STEM courses and are accountable to their students’
success; (3) counselors, who must ensure each student satisfies college and career requirements
and then place students into the appropriate courses year after year to ensure they satisfy
requirements for each step in the program; (4) administrators, who will attain funding for
teachers’ professional development and student enrollment in the STEM Program.
6

Stakeholder’ Performance Goals
Each stakeholder in the STEM Program serves an essential function for the success of the
program. Thus, each stakeholder has their own goal they must achieve in order to fulfill the
overarching mission and goal of the program. Table 1 lists the mission and goal of the STEM
Program, as well as one initial goal for each of the stakeholders.

Table 1
Organizational Mission, Global Goal and Stakeholder Performance Goals
Organizational Mission
The mission of the STEM Program is to prepare students for future careers in STEM through
hands-on and minds-on project-based learning in their communities.
Organizational Performance Goal
By June 2022, the STEM Program at LHS will have 100% program completion.
Stakeholder 1
Students
Stakeholder 2
Teachers
Stakeholder 3
Counselors
Stakeholder 4
Administrators
By August 2021,
100% of graduating
STEM Program
students will be on
track for fulfilling all
of the requirements
for program
completion.
By August 2021,
100% of STEM
Program teachers will
have implemented
recommendations
made towards
program
improvement.
By August 2021,
counselors will
ensure that 100% of
their students are
placed in appropriate
courses to satisfy
college and career
requirements.
By August 2021,
administrators will
have provided the
organizational culture
and setting necessary
to achieve 100%
STEM Program
completion.

7

Stakeholder Group for the Study
While contributions from all stakeholders are essential in fulfilling LHS Stem Program’s
goal of graduating college- and career-ready students through 100% program completion, it is
important to evaluate where the STEM Program teachers currently stand in attaining their
performance goal. Thus, the stakeholder of focus for this study will be all STEM Program
teachers. The STEM Program teachers’ goal is to implement the recommendations made towards
program improvement. Failure to accomplish this goal puts students at risk for not graduating
with essential college and career readiness, which hinders the STEM Program in fulfilling its
mission of preparing its students for their future careers in STEM.
Stakeholder of Focus Critical Behaviors
To achieve their goal of implementing recommendations made towards program
improvement, the STEM Program teachers, as the stakeholder of focus, must perform the
following critical behaviors:
1. Conduct biweekly meetings to discuss STEM Program completion requirements.
2. Implement pedagogical strategies in classes daily that support students in
completing STEM Program requirements.
3. Evaluate, in biweekly meetings, whether and to what extent steps taken towards
STEM Program improvement are effective.
By performing each of these critical behaviors, STEM Program teachers will contribute to
achieving STEM Program goals.
Purpose of the Project and Questions
The purpose of this project is to evaluate the degree to which LHS’s STEM Program is
achieving its goal of 100% program completion. The analysis will focus on knowledge,
8

motivation and organizational influences related to achieving this organizational goal. While a
complete evaluation project would focus on all STEM Program stakeholders, for practical
purposes, the stakeholders to be focused on in this analysis are all STEM Program teachers.
As such, the questions that guide this study are the following:
1. To what extent is LHS’s STEM Program meeting its goal of 100% program
completion by June 2022?
2. What are the knowledge, motivation and organizational influences related to
STEM Program teachers achieving their goal of implementing recommendations
made towards program improvement to achieve 100% student completion?
3. What are the recommendations for organizational practice in the areas of
knowledge, motivation, and organizational resources needed for STEM Program
teachers to achieve their goal?
Conceptual and Methodological Framework
Clark and Estes (2008) detail three critical factors that contribute to stakeholders meeting
their goals: stakeholders’ knowledge and skills, motivation to achieve their goals, and
organizational influences. These three factors will be adapted into an evaluation model and
implemented as the conceptual framework of this paper. The knowledge, motivation, and
organizational influences will be assessed as a qualitative case study using open-ended
questionnaires, focus groups, and document analysis. Solutions and recommendations will be
proposed accompanied by an evaluation plan.
Definitions
In this study, specific terms will have the following meaning:
9

• Project Week: A recurring week-long and program-wide project that occurs
towards the middle of each semester. All students in the program come together
during their STEM Program classes and work in groups that combine students in
different classes and grade levels. Groups work to build a device that fulfills a
new objective each Project Week (STEM Program, 2018).
• Senior Stewardship Capstone: A senior project where students design and
implement a college-level STEM research project or paper aimed for publishing.
Students present their projects or papers at an end-of-year STEM Program
banquet and symposium (STEM Program, 2019).
• Stewardship Experience: An experience where students lead or participate in a
STEM event in the community. Every experience is followed up by a post to
social media, presentation and/or submission to the STEM Program page (STEM
Program, 2016).
Organization of the Project
This study is organized into five chapters. This first chapter provides readers with the key
concepts and terminology commonly found in a discussion regarding college and career
readiness through high school STEM programs. The organization’s mission, goals, and
stakeholders and the framework for the project were introduced. Chapter Two provides a review
of the current literature surrounding the study. Topics of college and career readiness and STEM
programs will be addressed. Chapter Three details Clark and Estes’ (2008) gap analysis
framework, which are the knowledge, motivational, and organizational influences on the
stakeholders of the STEM Program. It also discusses research methods, in terms of participant
and data collection and analysis. Chapter Four analyzes the results of the study. Lastly, Chapter
10

Five provides solutions, based on data and literature, for closing the perceived gaps in attaining
LHS’s STEM Program’s goals. It recommends an implementation and evaluation plan for the
presented solutions.

11

Chapter Two: Review of the Literature
College and career readiness skills are essential for success in postsecondary life. Leaders
in the workforce and at higher education institutions assert the need for these skills, especially in
response to the lack of college and career readiness in recent graduates. As a result, academic
programs and career technical education pathways were developed to better prepare students for
life after high school. This chapter reviews the literature surrounding college and career
readiness programs. It begins by discussing the factors that influence college and career
readiness programs. Then, it describes the conceptual framework of this study: Clark and Estes’
(2008) gap analysis framework. The gap analysis framework includes the knowledge and skills,
motivation, and organizational influences on a problem of practice. This framework has been
adapted for the evaluation of college and career readiness programs, particularly the STEM
Program at LHS.
Influences on the Problem of Practice
While Chapter One focused on the impact of the problem of graduated students who are
ill-prepared for college and the workforce, Chapter Two discusses the factors that influence the
issue of college and career readiness. This section begins with a history of college and career
readiness programs. Then, it discusses Conley’s (2012) Four Keys Model of college and career
readiness knowledge and skills. This section then transitions into the relationship between
college and career readiness programs and outcomes in high school, college, and the workforce.
Finally, before transitioning into Clark and Estes’ (2008) gap analysis framework in the
following section, this section concludes with existing literature surrounding STEM-specific
college and career readiness programs.
12

History of College and Career Readiness Programs
High school graduates are expected to enter college or their careers with the skills
necessary to effectively study, communicate, and make decisions that affect themselves and their
peers. The 1940s and 1950s ushered in a movement for high school graduates to attend college
(Nishimura, 2014); however, many of these students did not stay in college and struggled to find
work as they did not have the skills to succeed. As a result, educators realized the need for
college and career readiness. Both academic and vocational courses were created to help students
become both college- and career-ready (Nishimura, 2014).
A study of individual perceptions of CCR through a blended learning model and a
project-based learning model in Hawaii showed positive results surrounding college and career
readiness programs. Nishimura (2014) assessed the senior project and a blended learning
environment at one school in Hawaii to see if students, staff, and community members perceived
the programs as successful in fostering CCR skills. The blended learning classroom was created
to incorporate technology that students would encounter in the workforce. The curriculum was
created to expose students to a social, self-directed, and cost-effective environment while also
increasing their access to knowledge through technology. Analysis of this curriculum showed
that technology skills are an essential component of CCR, although not specifically through
blended learning, but rather, in general (Nishimura, 2014). Blended learning also showed to be
an ideal school element for fostering CCR skills. The project-based learning model was
encompassed in the senior project, where students had to use skills from academic careers and
implement them in a social setting. Students had to create a project with driving questions and
constructive investigations in a realistic workforce setting. After interviewing 17 participants,
including community members, former educators, and recent high school graduates, Nishimura
13

(2014) found an intersection between perceptions of CCR and school practices. Specifically, the
senior project was an ideal application of CCR skills. Furthermore, Nishimura (2014) concluded
that high school systems must further define CCR skills and consider how they can produce
college- and career-ready graduates.
Four Keys Model of College and Career Readiness
After 18 years of study and research, Conley (2012) narrowed down a complete
definition of college and career readiness:
A student who is ready for college and career can qualify for and succeed in
entry-level, credit-bearing college courses leading to a baccalaureate or
certificate, or career pathway-oriented training programs without the need for
remedial or developmental coursework. (para. 4)
Based on empirical evidence and interactions with practitioners attempting to improve their
existing college and career readiness programs, Conley (2012) categorized college and career
readiness knowledge and skills into four components, or Four Keys: (1) key cognitive strategies,
which include problem formulation, research, interpretation, communication, and precision and
accuracy, (2) key content knowledge, or the structure of knowledge and technical knowledge and
skills, (3) key learning skills and techniques, encompassed by ownership of learning and learning
techniques, and (4) key transition knowledge and skills, which include contextual, procedural,
financial, cultural and personal knowledge and skills. Students who can demonstrate these
knowledge and skills are considered college- and career-ready.
The Four Keys can be expressed through multiple ways in academic performance. There
is a wide range of literacies which include, but are not limited to, text, visuals, and quantitative
modes (Conley, 2012). Students can also demonstrate their awareness of social systems, engage
14

in career planning, illustrate proficiency in a second language and the cultures that languages
reflect, and show their experiences in and appreciation of arts. Program instructors and leaders
can encourage students to take advantage of these various forms of expression in demonstrating
their college and career readiness skills. As such, this Four Keys model of college and career
readiness has been used by high school program leaders in developing program courses and
requirements geared towards preparing their students for college and the workforce.
College and Career Readiness Programs and Student Outcomes
High School Outcomes
In a mixed methods study of CCR program implementation and student outcomes in
Illinois, Bragg et al. (2014) found that these programs yielded positive short-term outcomes in
math and English performance. Bragg et al. (2014) analyzed two CCR pilot initiatives based on
Conley’s (2012) four dimensions of CCR and saw a significant increase in math and English
score reports. In the first model, the Prairie College model, this college worked with 11 district
high schools to administer a placement test, a self-paced semester- or year-long math course, a
Freshmen Orientation Night, and alignment activities which allowed students to work directly
with instructors to encourage individualized communication between the high school and college
level students and faculty. The second model, the River College model, had low achieving
students attend seminars on the college campus and distributed “study sheets” that included
study strategies, time management, and learning styles. Students used these resources in
conjunction with instructional academic intervention. An analysis of placement tests at the
beginning and end of the program year showed improvements in math and English scores and
performance. Thus, Bragg et al. (2014) concluded that CCR efforts should be moved forward
15

and even extended to science, technology, engineering, and math (STEM) education in order to
improve academic and career preparation in all fields.
A study on educational outcomes of students in career academies also yielded positive
results at the high school level. Brand (2009) studied a program in which students experienced
real workforce experiences beyond the classroom and worked in teams on real-life projects.
These program experiences were geared towards helping students understand the importance of
everyday career skills such as professionalism, reliability, team work, and clear oral
communication skills (Brand, 2009). This program design also allowed students to draw
connections between how their education was connected to their post-secondary fields (Brand,
2009). Furthermore, these career academies yielded positive high school outcomes for students.
Brand (2009) found positive impacts on attendance, credit completion, and graduation rates.
Students were more likely to complete required credits and graduate high school, and the rate at
which high-risk students completed core curriculum was doubled. Overall, college and career
programs such as career academies demonstrate positive impacts on high school outcomes.
Yet another CCR program, Advancement Via Individual Determination (AVID), shows
higher student outcomes when compared to students in a traditional academic program of study.
Day et al. (2012) studied CCR outcomes by comparing student grade point averages, curriculum
rigor, and student ACT scores between AVID students and traditional academic program (TAP)
students. They found that AVID students were more successful in complete honors and AP
courses and that the students’ twelfth grade grade point averages were higher than TAP students
(Day et al., 2012). Like many other college and career readiness programs, AVID fosters higher
high school outcomes for its students.
16

In a study of achievement outcomes and CCR program completion, Castellano et al.
(2017) also found greater high school achievement. They specifically studied the relationship
between completing CCR programs and high school achievement outcomes. Participants
included graduates who completed a program of study, concentrated in a specific occupational
area through a number of CTE courses, and may have taken one or two CTE courses but did not
satisfy the first two categories (Castellano et al., 2017). Castellano et al. (2017) found that
graduates who completed a program of study were more likely to have a higher overall grade
point average, have a higher CTE grade point average, and earn more STEM credits than all
other students. Compared to CTE concentrators, program of study completers were also more
likely to have a higher overall grade point average and earn more STEM credits (Castellano et
al., 2017). This study further supports the idea that college and career readiness programs
increase high school achievement Given that college and career programs yielded such positive
high school outcomes, research then moved to analyze the effects of these programs on college
outcomes.
College Outcomes
In addition to increased high school outcomes, Brand’s (2009) study on career academies
analyzed students’ college outcomes, yielding positive results. Students in career academies were
more likely to complete entrance requirements for college admissions eligibility and attend
postsecondary institutions and four-year colleges compared to general and vocational track
students (Brand, 2009). Additionally, more likely to complete their bachelor’s degrees than non-
academy graduates (Brand, 2009). Thus, college and career readiness programs such as career
academies support students in higher college outcomes. These results emphasize the importance
17

of college and career readiness programs such as career academies in producing college-ready
students.
High school CCR programs that include partnerships with local colleges also show
increased college outcomes. Creech and Clouse (2013) studied the outcomes of partnership
between four high schools and one university, which was developed to reduce the need for
remediation in reading writing for students who did not meet ACT benchmarks. After taking a
pretest, formative tests, and a post-exam throughout the program, Creech and Clouse (2013)
found that students had significant gains in reading and writing achievement, along with
achieving college readiness in reading comprehension. These increases in reading and writing
achievement further emphasize the positive role of college and career readiness programs in
improving students’ college outcomes. High school and college partnerships prove to be an
effective form of CCR programs that bolsters students’ college readiness.
Workforce Outcomes
Students’ workforce outcomes must also be taken into consideration when evaluating
CCR programs. Brand’s (2009) study of career academies found significant improvements in
workforce outcomes for at-risk males. They discovered increased post-high school rates and
earnings without reducing the chances of students attending college or earning post-secondary
credits (Brand, 2009). Brand (2009) also found significant, sustained increase in earnings and
overall months and hours of employment. These positive results for at-risk males show that
college and career readiness programs such as career academies help to foster students’ career
readiness skills and workforce outcomes.
CTE programs, dedicated mainly towards preparing students for the workforce, are
essential to addressing the shortage of skilled tradespeople. Fitzgerald’s (2018) study of the
18

perceptions of CTE programs and their ability to address the issues of an ill-prepared workforce
found that educators and workforce leaders believed that CTE programs provided students with
most of the skills necessary to address the shortage of skilled tradespeople. Workforce
participants of the study noted that their new tradespeople often lacked the necessary entry-level
skills required to perform their roles effectively (Fitzgerald, 2018); thus, CTE programs provided
graduates with these entry-level career readiness skills. Having discussed workforce outcomes
resulting from CCR programs, it is essential to examine STEM-focused CCR programs in
preparation for analyzing one in a case study.
STEM-focused College and Career Readiness Programs
Program Qualities
Because college and career readiness programs are geared towards preparing students for
college and the workforce, many STEM programs incorporate project-based instruction to hone
students’ communication, collaboration, and problem-solving skills. For example, Bass and
Panahandeh (2016) studied the Digital Pathways program, a STEM-focused CCR program
focused on providing low-income, underrepresented minority youth with hands-on activities in
digital media production in order to support their pursuit of higher education and technology-
based careers. After conducting staff interviews, student observations, and interviews, they found
that the best practices for empowering students to move from consuming to producing digital
media included attending to noncognitive elements of learning (Bass & Panahandeh, 2016).
Students engaged in creative expression, personal agency, and collaboration in a supportive
community environment, and even teachers were placed in internships with industry personnel,
career panels, and career counselors (Bass & Panahandeh, 2016). Thus, immersing both students
and teachers in STEM workforce environments is a key aspect of fostering college- and career-
19

ready graduates. While the Digital Pathways program is one example of a STEM-focused CCR
program focused on developing students’ communication, collaboration, and problem-solving
skills, other STEM programs also focus their instruction on honing these skills.
The Clean Technologies & Sustainable Industries Early College High School Program
(CTSI ECHS) is another STEM program that fosters students’ CCR skills and decision-making
through activities that promote communication, collaboration, and problem-solving. Students in
the program were co-enrolled in high school and college, with a collaborative environment that
included one-to-on computing (Irwin et al., 2013). Program courses combined environmental
science and sustainability, mathematics, and English during students’ junior year, as well as
nanotechnology, public policy, and economics during their senior year (Irwin et al, 2013).
English was included to strengthen students’ communication, organization, and research skills;
public policy was included to expose students to local and global policies and laws related to
STEM (Irwin et al., 2013). The experiences and skills taught in the combination of all of these
courses allowed students to engage in problem-based learning and narrate their approaches to
successfully tackling real-world issues, thus preparing them for similar issues they might
encounter in their postsecondary lives.
Another STEM program used small-group instruction, individual instruction, feedback,
and corrective action as strategies for developing students’ CCR skills. This was a STEM
intervention program designed using Conley’s (2012) Four Keys framework (Lane at al., 2020).
Students initially struggled in mathematics due to the lack of experience in advanced content,
poor math teaching, or having a “shaky math foundation, leaving them ill-equipped for rigorous
academic environments and unaware of the tools for success (Lane et al., 2020). Lane et al.
(2020) found that through small-group instruction, individual instruction, feedback, and
20

corrective action, help-seeking behaviors and group work was normalized. These instructional
strategies also allowed students to develop metacognition and contextual awareness, which
helped students develop time management and other skills necessary to succeed in college and
the workforce (Lane et al., 2020).
Program Outcomes
Conley (2012) outlines the four key categories of knowledge and skills that CCR
programs are designed to foster in students: key cognitive strategies, key content knowledge, key
learning skills and techniques, and key transition knowledge and skills. Student outcomes from
CCR programs can thus be categorized into one of these Four Keys. At CTSI ECHS, students
felt they had better self-confidence and communication, organization, and problem-solving skills
as a result of the program, which fall into all four categories of CCR skills (Irwin et al., 2013).
Through a summer bridge STEM program, Lane et al. (2020) also found that students were able
to address knowledge gaps and develop metacognition and contextual awareness, each of which
also falls into Conley’s (2012) Four Keys. Students in the Digital Pathways program exhibited
growth in communication, collaboration, and problem-solving skills (Bass & Panahan, 2016).
Each of these STEM-focused CCR programs serve as strong evidence for positive program
outcomes; students who enroll in and complete STEM programs have the opportunity to develop
their college and career readiness skills in preparation for the workforce.
College Outcomes
STEM-focused CCR programs show long-term positive outcomes for students as they
complete secondary education and move on to college. Irwin et al. (2013) found that participants
were able to earn over 20 college credits upon completing high school requirements. All students
earned college credits, with some even majoring in STEM fields (Irwin, 2013). The High School
21

Summer Research Program, a multi-disciplinary engineering program, showed similar results,
where a large percentage of alumni pursue STEM degrees in college (Kittur et al., 2017).
Students also report feeling better prepared for STEM curricula and college expectations, as
reported in Lane et al. (2020). As evident in the results of each of these programs, STEM-
focused CCR programs support students’ outcomes into college, beyond their completion of the
programs in high school.
Workforce Outcomes
Program outcomes continue to extend into students’ post-secondary lives and into the
workforce. Shively and Weiss (2017) conducted a study of seven career academies under the
National Academy Foundation Organization. Research questions analyzed the perceived skills
gained as well as how participation shaped students’ feelings of readiness (Shively and Weiss,
2017). They found that students felt more engaged, with increased feelings of college and career
readiness (Shively & Weiss, 2017). Similarly, Kittur et al. (2017) studied a High School Summer
Research Program (HSSRP), an eight-week engineering research program taking students from
diverse backgrounds to promote diversity in college applicants and the STEM workforce. Both
program students and staff were provided with opportunities to network and support one another
in researching and communicating their research through “learn-by-teaching” systems (Kittur et
al., 2017). As a result, students reported high satisfaction with the program, and a majority of
alumni work in the STEM fields. Thus, the positive effects of STEM-focused CCR programs
extend beyond high school and college and into the workforce.
Program Needs
While general and STEM-focused programs have shown to produce positive outcomes in high
school, college, and the workforce, stakeholders have identified program needs and areas for
22

improvement. Ejiwale (2013) identified the ten barriers to the successful implementation of
STEM education as (1) poor preparation and shortage of qualified STEM teachers; (2) lack of
investment in professional development for teachers; (3) poor preparation and inspiration of
students; (4) lack of connection with individual learners, such as through afterschool programs,
contests, designing and building, and summer programs; (5) lack of support from the school
system; (6) lack of research collaboration across STEM fields; (7) poor content preparation,
particularly clear guidelines and outcomes in teaching materials; (8) poor content delivery and
method of assessment; (9) poor condition of facilities and instructional media; (10) lack of
hands-on, career-based, cooperative training for students. These program needs outline what
must be practiced by and provided to teachers on a daily basis. For this study, the daily
implementation of these programs needs is represented by the knowledge, motivation, and
organizational variables contributing to the success of the STEM program. This study will use
Clark and Estes’ (2008) gap analysis framework to narrow the list of program needs down to the
knowledge, motivation, and organizational variables based on STEM Program teachers’ critical
behaviors.
Conceptual Framework
Clark and Estes’ (2008) gap analysis framework describes the three main factors that
influence stakeholder performance within an organization: knowledge and skills, motivation, and
organizational culture. By identifying the areas for improvement within each of these three
factors, stakeholders of the organization can create an effective performance solution for their
organization. Per Clark and Estes (2008), it is essential to contextualize stakeholder goals with
respect to the overarching organizational goals in order to identify the assumed performance
influences in the areas of knowledge, motivation, and the organization. This problem-solving
23

process must be conducted based on general theory, context-specific literature, and an
understanding of the organization.
In this study, the Clark and Estes (2008) gap analysis framework will be adapted as an
evaluation of the STEM Program at LHS, with regard to increasing program enrollment and
completion. The evaluation will allow for STEM Program teachers to determine areas for
improvement in their program in order to devise a performance solution. This study will then
provide recommendations for teachers to implement in order to increase enrollment and
completion numbers, thus promoting college and career readiness in their students through the
knowledge and skills taught in STEM Program courses.
Stakeholder Knowledge, Motivation, and Organizational Influences
In this study, STEM Program teachers are the stakeholders of focus. Thus, this section
analyzes and discusses the teachers’ knowledge and skills, motivation, and organizational
influences that affect performance, based on the daily critical behaviors for success. Each
influence is identified and supported by research on various existing STEM programs in regards
to how they contribute to successful teacher performance.
Knowledge and Skills
Krathwohl (2002) describes four main types of knowledge needed to learn and achieve
goals: factual, conceptual, procedural, and metacognitive. Factual knowledge is pieces of
information; conceptual knowledge includes concepts, processes, and principles; procedural
knowledge is knowing how to accomplish the task; and metacognitive knowledge is the
awareness of one’s own cognition. In the context of STEM Program teachers’ critical behaviors
for improving the program, the assumed knowledge influences for each category are identified
24

and described next. Each has to do with program completion requirements, pedagogical
strategies or steps to program improvement.
Declarative Factual Knowledge Influences
Declarative factual knowledge describes discrete, isolated bits of information (Anderson
& Krathwohl, 2001). With respect to the STEM Program teachers’ critical behaviors, this type of
knowledge includes isolated pieces of knowledge about the program completion requirements
and steps to program improvement.
Per Bruce-Davis et al. (2014), teachers must know (a) program completion requirements
and (b) steps to program improvement in order to perform their critical behaviors towards
increasing enrollment and completion and improving the STEM Program. Bruce-Davis et al.
(2014) found that a successful STEM program required a shared, common vision of what the
program should look like, or program requirements, and how to achieve said vision, or steps to
program improvement. Kanipes et al. (2019) also supports the assertion that stakeholders of
focus must know the steps to program improvement. Their study of the STEM Center of
Excellence for Active Learning worked to transform pedagogy and institutional teaching and
learning, which required stakeholders to know the steps in developing the STEM Center.
Conceptual Knowledge Influences
Conceptual knowledge begins to bridge the gaps between isolated bits of declarative
factual knowledge; it describes more complex, organized knowledge forms such as concepts,
processes, and principles (Anderson & Krathwohl, 2001). To improve the STEM Program,
teachers must know the three categories of tasks for students to carry out in order to complete the
program. Having this conceptual knowledge allows teachers to direct students towards program
completion.
25

STEM Program teachers must have a common vision of categories of program
completion requirements in order to guide students in completing the program. Bruce-Davis et
al. (2014) emphasizes the importance of the common vision in order for all stakeholders to be in
accordance with requirements.
Procedural Knowledge Influences
Once teachers have conceptual knowledge, they are well-equipped to move on to the
skills and criteria for context- and domain-specific decision-making, or procedural knowledge
(Anderson & Krathwohl, 2001). STEM Program teachers must know (1) how to identify and
manage students’ progress on program completion requirements and (2) how to take steps
towards program improvement.
How to Identify and Manage Student Progress. Knowing how to identify and manage
students’ progress on program completion requirements allows STEM Program teachers to
ensure that their students are on track to successfully completing the program. Cheeks and
McDowelle (2018) found that effective program measures include awareness of resources and
the coordination of efforts towards managing students’ goals and progress towards achieving
goals such as program completion.
How to Take Steps Towards Program Improvement. STEM Program teachers must
also be familiar with how to take steps towards program improvement. Per Conley and Darling-
Hammond (2013), a system of assessment and accountability can include a series of steps
strategically geared towards continuous program improvement, which is necessary for program
stakeholders. Kanipes et al. (2019) also emphasize the importance of knowing how to take steps
towards program improvement, such as course redesign efforts and transforming pedagogy in
order to increase the production of high-achieving STEM students.
26

Metacognitive Knowledge Influences
Per Krathwohl (2002), metacognitive knowledge is the deepest form of understanding
such that it requires an awareness of cognition in general as well as an awareness of one’s own
cognition. Thus, STEM Program teachers must exercise their metacognitive in order to
successfully implement daily critical behaviors. Specifically, teachers must monitor their roles in
(1) managing students’ progress towards program completion and (2) taking steps towards
program improvement.
In their analysis of a Digital Pathways program, Bass and Panahandeh (2016) emphasize
the importance of teachers understanding their role in (1) students’ learning and program
completion progress and (2) program improvement, particularly through instructional strategies,
teacher internships in the STEM industry, and attending to noncognitive elements of learning.
Bragg et al. (2014) also assert that teachers must monitor their roles in re-envisioning academic
interventions, endorsing curriculum alignment between the high school and college levels, and
integrating and outsourcing additional student supports. From a transformative lens, Smith et al.
(2019) argue that teachers must reflect on their cultural approaches towards supporting and
managing a diverse group of students in order. Altogether, these articles posit that teachers must
monitor their roles in managing students’ progress and taking steps towards program
improvement through an empathetic, transformative lens. Table 2 shows the teachers’ top
knowledge and skill influences and the related literature.
The asterisked knowledge influences are the priorities for teachers to perform critical
behaviors and serve to manage the scope of this study. These influences are considered to be
priorities since they have the greatest effects on teachers and the program, as well as the greatest
perceived challenges for program improvement.
27

Table 2
Summary of Assumed Knowledge Influences on Teachers’ Ability to Achieve the Performance
Goal
Assumed Knowledge Influences Research Literature
Declarative Factual
Teachers know the program completion
requirements.*
Bruce-Davis et al., 2014
Teachers know pedagogical strategies that
support students in completing STEM
Program requirements.

Teachers know steps to program
improvement.*
Bruce-Davis et al., 2014; Kanipes et al., 2019
Conceptual
Teachers know the relationship between
program completion requirements and
achieving the goal of program
improvement.

Teachers know the three categories of tasks
students must carry out to complete the
program.*
Bruce-Davis et al., 2014
Teachers know the relationship between
pedagogical strategies and program
completion requirements.

Teachers know why steps must be taken
towards program improvement.

Procedural
Teachers know how to identify and manage
students’ progress on program completion
requirements.*
Cheeks & McDowell, 2018
Teachers know how to implement
pedagogical strategies in classes daily that
support students in completing STEM
Program requirements.

28

Teachers know how to take steps towards
program improvement.*
Conley & Darling-Hammond, 2013; Kanipes
et al., 2019
Metacognitive
Teachers will know how they contribute to
managing students’ progress on program
completion.

Teachers monitor their roles in managing
students’ progress towards program
completion.*
Bass & Panahandeh, 2016; Bragg et al., 2014;
Smith et al., 2019
Teachers monitor their role in implementing
pedagogical strategies daily to support
students in completing STEM Program
requirements.

Teachers know how they contribute to
program improvement.

Teachers monitor their roles in taking steps
towards program improvement.*
Bass & Panahandeh, 2016; Bragg et al., 2014;
Smith et al., 2019

Motivation
Mayer (2011) posits three indices which indicate motivation: choice, persistence, and
mental effort. Issues with choice include intending to start a task without actually acting on it. A
gap in motivation may also be seen through low persistence, or starting a task but not continuing
it. Low mental effort is characterized by inaccurate or inappropriate knowledge of the task.
Regarding teachers encouraging students to enroll in and complete the program as well as
improving the program, persistence and mental effort appear to be the greatest issues for
motivation. Teachers have a grand vision for how they want the program to look and what they
want to require of the students; thus, they often choose to initiate tasks, just as they chose to take
on a feat as large as starting or joining the program to begin with. However, persisting in the task
29

of developing and improving the program has become a challenge, especially since teachers have
not yet come to an agreement on program qualities such as requirements and structure. In this
case, low mental effort contributes to low persistence, which makes teachers reluctant to push for
program completion and take steps towards program improvement.
To address the issues in persistence and mental effort needed to encourage students to
enroll in and complete the program and take steps towards program improvement, teachers must
first address each specific motivational variable attributed to their low mental effort. Mayer
(2011) describes the following as potential factors affecting motivation: (a) value, which refers to
how much interest, importance, and utility one sees in a task; (b) self-efficacy, or the expectation
of success and confidence in one’s skills; (c) mood, or one’s own emotions; (d) attributions,
which is the belief about control over an event; (e) goal orientation, where one sets achievable
goals towards accomplishing a task. The potential motivational variables for teachers
encouraging the enrollment in and completion of the program as well as taking steps towards
program improvement are as follows.
Value
The amount of value a stakeholder places in a task has to do with their (a) interest in the
task; (b) perceived importance of the task; (c) utility of the task (Mayer, 2011). A lack of any one
or more of these three factors could contribute to a decreased value of the task. In particular, the
greatest influence affecting STEM Program teachers’ values is conducting meetings to discuss
program completion.
To advance STEM learning, Kanipes et al. (2019) assert the importance of transforming
institutional education. Thus, the task of discussing and agreeing upon clear program
requirements as a form of transforming institutional education should be of high value. Smith et
30

al. (2019) also emphasize the need for a transformative cultural approach to advising STEM
students in their educational paths. In order to have a transformative approach, teachers must be
in agreement regarding program completion requirements, which requires holding value in
having meetings to discuss the requirements.
Self-efficacy
Self-efficacy describes whether and to what extent a stakeholder expects to be successful
and feels confident in their ability to be successful. According to Pajares (2006), high self-
efficacy comes from experience, observation, verbal messages and social persuasions, and
physiological and emotional states. Having high self-efficacy increases stakeholder motivation,
pushing them to be more engaged in their tasks and in more effective self-regulation strategies
geared towards completing said tasks (Pajares, 2006). In LHS’s STEM Program, teachers must
feel positively in regards to their daily critical behaviors: (1) conducting meetings to discuss
program requirements; (2) implementing pedagogical strategies to support program completion;
(3) taking steps towards program improvement. Since biweekly meetings are already conducted
to discuss the program in general and teachers already implement pedagogical strategies on a
daily basis, taking steps towards program improvement is the critical behavior for which teachers
may need greater self-efficacy.
Fakayode et al. (2019) argues that to transform STEM, program leaders require well-
coordinated alliances with all stakeholder groups in order to encourage inclusion and
participation. These alliances are necessary for leaders such as STEM Program teachers to feel
confident in their abilities to transform the program via the necessary steps towards program
improvement.
31

Emotions
A stakeholder’s mood, or emotions, is intertwined with their cognition (Immordino-
Yang, 2011). If stakeholders feel negatively about their critical behaviors, they are less motivated
to complete them and may not do so; whereas, if stakeholders feel positive emotions towards
their action items, they have higher motivation towards accomplishing them. In order to perform
their critical behaviors, of aiding students in their completion of the program and taking steps
towards improving the program, teachers must feel positively about doing so.
STEM Program teachers must feel positively about conducting meetings to discuss
program requirements and taking steps towards program improvement. Bruce-Davis et al. (2014)
found that the administrators, teachers, and students in their study all shared an appreciation for
academic and affective support in the challenging learning environment, which contributed to
their combined efforts for a successful program.
Attributions
Mayer (2011) describes attribution as one’s belief about their control over the stability of
an event. Attributions can be affected by internal and external factors, as well as whether or not
each factor is stable (Anderman & Anderman, 2006). If a stakeholder’s attributions are internal
and stable, their control has to do with their own ability to complete the task; however, if the
locus is internal and unstable, then the stakeholder’s control has to do with the effort they put
into the task. If the locus is external and stable, then the stakeholder’s control likely has to do
with the task difficulty. Whereas, if the external cause is also unstable, then successful task
completion is up to luck. In order to help students in program completion and to take steps
towards program improvement, teachers must attribute their control over critical behaviors
towards their own ability and effort to complete reasonable tasks.
32

Teachers must have control over conducting meetings to discuss program requirements
and taking steps towards program improvement, whether it be due to their own ability or effort to
complete said tasks. Peters et al. (2019) describe high-impact practices that lead to higher levels
of student performance in STEM programs and courses, all of which teachers have control over
integrating in their classrooms. These practices are unique to the traditional classroom
experiences and can support STEM Program teachers in taking control of conducting meetings to
discuss program requirements and taking steps towards program improvement.
Goal Orientation
Goals have to do with mastery, or mastering the task at hand, or performance, which is
demonstrating one’s ability compared to others (Yough & Anderman, 2006). Stakeholders either
approach or avoid these goals. STEM Program teachers must set goals to master their critical
behaviors in order to improve the program.
Teachers must set goals to address the challenges in setting program completion
requirements and taking steps towards program improvement. Yavuz et al. (2019) found that
effective strategies for counseling students and addressing program needs required intentional
actions towards strategic program design. Thus, teachers must intentionally set goals to address
the challenges in counseling students towards program completion and taking strategic steps
towards program improvement.
Table 3 shows the teachers’ primary motivational influences and the related literature.
The highlighted motivational influences are the priorities for teachers to perform critical
behaviors and serve to manage the scope of this study. These motivational influences are
priorities since they seem to affect teachers’ performance most significantly.

33

Table 3
Summary of Assumed Motivation Influences on Teachers’ Ability to Achieve the Performance
Goal
Assumed Motivation Influences Research Literature
Value
Teachers value conducting meetings to
discuss program completion requirements.
Kanipes et al., 2019; Smith et al., 2019
Teachers value implementing pedagogical
strategies to support students in completing
the program.

Teachers value taking steps towards program
improvement.

Self-Efficacy
Teachers are confident that they can conduct
biweekly meetings to discuss program
completion requirements.

Teachers are confident that they can
implement pedagogical strategies daily to
support students in completing the program.

Teachers are confident that they can take steps
towards improving the program.
Fakayode et al., 2019
Emotions
Teachers feel positively about conducting
biweekly meetings to discuss STEM
Program requirements.
Bruce-Davis et al., 2014
Teachers feel positive about implementing
pedagogical strategies in class daily to
support students in completing the program.

Teachers feel positive about taking steps
towards program improvement.
Bruce-Davis et al., 2014
Attributions
34

Teachers have control over conducting
biweekly meetings to discuss program
requirements.
Peters et al., 2019
Teachers attribute implementing pedagogical
strategies in classes daily to their control
over supporting students in completing
STEM program requirements

Teachers have control over taking steps
towards program improvement.
Peters et al., 2019
Goal Orientation
Teachers address all challenges in setting
program completion requirements.
Yavuz et al., 2019
Teachers master and implement all
pedagogical strategies daily.

Teachers address all challenges in taking
steps toward program improvement.
Yavuz et al., 2019

Organization
Organizational factors that affect stakeholder performance are: (1) culture; (2) structure,
practices, and policies; (3) reciprocal relationships of cultural models and settings (Rueda, 2011).
The environment, groups, and individuals in an organization contribute to the organization’s
culture (Clark & Estes, 2008). Structures, practices, and policies are put into place upon the
formation of the organization and as it continues to run and grow. The factors must intertwine in
such a way that allows for stakeholders to have the necessary supports for organizational change
processes. Stakeholders must also have available time, supplies, personnel, and other resources
in order to perform their critical behaviors and achieve their goals.
Cultural Models
Cultural models refer to shared mental understandings of how the world works or ought
35

to work (Gallimore & Goldenberg, 2001). The organization must create a cultural model in
which stakeholders experience a culture of trust and perceive shared values in order to have buy-
in with the organization. STEM Program teachers must experience a culture of trust in carrying
out their critical behaviors and perceive that they are part of an organizational culture that shares
their values in program improvement.
In order to align with the STEM Programs cultural model, teachers must perceive that
they are part of a culture that values program improvement. Fakayode et al. (2019) also
emphasize the importance of raising an “enlightened citizenry” that is critical to understanding a
STEM program’s challenges and opportunities in order to adapt and work collaboratively. STEM
Program teachers must feel that their value of program improvement aligns with the program
culture.
Cultural Setting
Whereas cultural models describe shared understandings, cultural setting is the culture
that exists as people come together to work towards a common goal (Gallimore & Goldenberg,
2001). Since the STEM Program is the newest academic program at LHS, stakeholders must
come together, along with the veteran academic programs as model programs, in order to work
towards the goal of student completion and improvement of the program.
The most useful aspect of cultural setting for STEM Program teachers to perform their
critical behaviors is having role models from other existing programs. These role models must
also conduct regular meetings to discuss program requirements and take steps for program
improvement so that STEM Program teachers have guidance as to how to follow suit. Fletcher
Claville et al. (2019) posits that the Nanoscience Project at Hampton University serves as an
innovative model for STEM programs to follow in engaging stakeholders. Having such an
36

example allows for other STEM programs to revolutionize their existing programs.
Policies and Procedures
Organizational policies and procedures must support stakeholders in performing their
critical behaviors to achieve their goals. These policies and procedures should also be structured
in such a way that promotes a positive cultural setting through reciprocal relationships with
cultural models. Policies at LHS should support teachers in conducting biweekly meetings to
discuss program requirements, implementing pedagogical strategies daily to support students in
completing the program, and taking steps towards program improvement.
The policies of priority for STEM Program teachers are ones that support them in
evaluating and taking steps towards program improvement. Without such policies, teachers
would be unable to improve their program and achieve their goal of all students completing the
program. Fakayode et al. (2019) stresses the importance of alliances that promote broad and
inclusive STEM program participation. Such alliances allow for policies and agreements that are
conducive to program improvement.
Resources
Resources refer to any time, monetary, supplies, tools, and personnel, and other tangible
factors necessary for an organization to take steps towards achieving their goals. Time is
essential for planning organizational structure and qualities and carrying out critical behaviors.
Personnel are essential for running the organization and implementing critical behaviors.
Material resources serve as tools for completing tasks.
STEM Program teachers need time and material resources to conduct meetings,
implement pedagogical strategies, and take steps towards program improvement. Foroozesh et al.
(2019) attribute student success to a combination of student support initiatives and externally
37

funded programs. These programs work to leverage resources in order to provide students with
the necessary experiences to be successful. Similarly, LHS’s STEM Program requires time and
resources to help their teachers provide students with the learning experiences to develop college
and career readiness skills for the postsecondary world.
Table 4 shows the assumed organizational influences and the related literature. The
asterisked organizational influences are the priorities for teachers to perform critical behaviors
and serve to manage the scope of this study. Organizational influences are prioritized according
to the perceived level of effects on teacher performance.

Table 4
Summary of Assumed Organization Influences on Teachers’ Ability to Achieve the Performance
Goal
Assumed Organizational Influences Research Literature
Cultural Models
Teachers experience a culture of trust in
which they can conduct biweekly meetings
to discuss STEM Program requirements.

Teachers experience a culture of trust in
which they feel comfortable implementing
pedagogical strategies in classes daily that
support students in completing STEM
Program requirements.

Teachers perceive that they are part of an
organizational culture that values program
improvement.*
Fakayode et al., 2019
Cultural Setting
38

Teachers have role models from other
academic and career programs at LHS who
also conduct regular meetings to discuss
program completion requirements.*
Fletcher Claville et al., 2019
Teachers have role models from other
academic and career programs at LHS who
also implement pedagogical strategies in
class daily to support students in program
completion.

Teachers have role models from other
academic and career programs at LHS who
evaluate steps taken towards program
improvement.*
Fletcher Claville et al., 2019
Policies/Procedures
Policies at LHS support teachers in
conducting biweekly meetings to discuss
STEM program completion requirements.

Policies at LHS support teachers in
implementing pedagogical strategies in
classes daily that support students in
completing program requirements.

Policies at LHS support teachers in evaluating
steps taken towards program improvement.*
Fakayode et al., 2019
Resources
Teachers have time to conduct biweekly
meetings to discuss STEM Program
completion requirements.*
Foroozesh et al., 2019
Teachers have resources to implement
pedagogical strategies in class daily that
support students in program completion.*
Foroozesh et al., 2019; Peters et al., 2019
Teachers have time and resources to evaluate
steps taken towards program improvement.*
Foroozesh et al., 2019

39

The knowledge, motivation, and organizational influences identified in this chapter will
serve as a foundation for data collection outlined in the next chapter.

40

Chapter Three: Methodology
The purpose of this project is to evaluate the degree to which LHS’s STEM Program is
achieving its goal of 100% program completion. The questions that guide this gap analysis are
the following:
1. To what extent is LHS’s STEM Program meeting its goal of 100% program
completion by June 2022?
2. What are the knowledge, motivation and organizational influences related to
STEM Program teachers achieving their goal of implementing recommendations
made towards program improvement to achieve 100% student completion?
3. What are the recommendations for organizational practice in the areas of
knowledge, motivation, and organizational resources needed for STEM Program
teachers to achieve their goal?
Conceptual and Methodological Framework
Clark and Estes’ (2008) gap analysis framework deconstructs the knowledge,
motivational, and organizational influences that affect a stakeholder in performing their critical
behaviors as they work to achieve their goals in the organization. After identifying factors within
each of these three influences, stakeholders can evaluate to what extent they have the necessary
components for implementing their critical behaviors. From there, stakeholders can determine
the gaps in knowledge, motivation, and organizational factors and work in order to create an
effective performance solution.
This study adapts the gap analysis framework into an evaluation model. Through the
adaptation, this study determines whether or not LHS’s STEM Program teachers are meeting
their goals and why. The results of the evaluation are used to build and recommend a
41

performance solution. From there, teachers can take the recommended steps towards student
completion of the program through program improvement.
Figure 1 below outlines the gap analysis process that will be adapted into an evaluation
model.

Figure 1
Gap Analysis Process

42

The gap analysis process begins by identifying the stakeholders’ goals of the organization
and evaluating to what extent the stakeholders are achieving their goals. The next step is to
identify the gaps in attaining said goals, as well as the knowledge, motivation, and organizational
factors influencing the stakeholders. Once the root causes are identified, a performance plan and
solution is created for implementation. Then, the cycle starts again, with an updated evaluation
of the extent to which stakeholders are achieving their goals.
Assessment of Performance Influences
The STEM Program teachers’ knowledge, motivation, and organizational influences
discussed in Chapter Two were assessed in order to determine the extent to which teachers are
achieving their goal of 100% program completion. The influences were analyzed in the context
of the three critical behaviors:
1. Conduct biweekly meetings to discuss STEM Program completion requirements.
2. Implement pedagogical strategies in classes daily that support students in
completing STEM Program requirements.
3. Evaluate, in biweekly meetings, whether and to what extent steps taken towards
STEM Program improvement are effective.
Knowledge Assessment
The assumed knowledge influences described in Chapter Two serve as the influences that
were assessed in order to create a performance solution. This section describes how each of the
influences were assessed. More specifically, interview questions and artifacts collected are listed
here. These items helped to assess STEM Program teachers’ knowledge and skills surrounding
program completion requirements and steps to program improvement. The interview items
consisted of questions and prompts derived from Anderson and Krathwohl’s (2001) guide for
43

writing items that extract knowledge and skills. Artifacts for assessing knowledge and skills were
derived from the STEM Program’s webpage and teacher meeting notes. Table 5 lists the
interview items and specific artifacts collected for each assumed knowledge influence of priority.

Table 5
Summary of Knowledge Influences and Method of Assessment
Assumed Knowledge Influences Interview Items Artifacts
Declarative Factual
Teachers know the program
completion requirements.*
What requirements must
students fulfill in order to
complete the program?

Teachers know pedagogical
strategies that support students
in completing STEM Program
requirements.

Teachers know steps to program
improvement.*
What do teachers need to
do to help improve the
program?

Conceptual
Teachers know the relationship
between program completion
requirements and achieving the
goal of program improvement.

Teachers know the three
categories of tasks students
must carry out to complete the
program.*
What are the three
categories of tasks that
students must carry out in
order to complete the
program?
Program Web Page
Teachers know the relationship
between pedagogical strategies
and program completion
requirements.

44

Teachers know why steps must
be taken towards program
improvement.

Procedural
Teachers know how to identify
and manage students’ progress
on program completion
requirements.*
How do you identify and
manage students’ progress
on program completion
requirements?

Teachers know how to implement
pedagogical strategies in classes
daily that support students in
completing STEM Program
requirements.*
How do you incorporate
culturally relevant
pedagogy in your
instruction?
Lesson Plans
Teachers know how to take steps
towards program
improvement.*
What steps do you take for
program improvement?

Metacognitive
Teachers will know how they
contribute to managing
students’ progress on program
completion.

Teachers monitor their roles in
managing students’ progress
towards program completion.*
Tell me how you reflect
upon student progress in
completing the program.

Teachers monitor their role in
implementing pedagogical
strategies daily to support
students in completing STEM
Program requirements.

Teachers know how they
contribute to program
improvement.

Teachers monitor their roles in
taking steps towards program
improvement.*
Tell me how you reflect
upon progress towards
program improvement.

45

Motivation Assessment
Chapter 2 describes the assumed motivation influences affecting STEM Program
teachers. These influences include value, self-efficacy, emotions, attributions, and goal
orientation, which were assessed using interview items and artifact analysis. Literature that
served as the framework for the motivation assessment interview questions are: (1) Eccles (2006)
and Pintrich (2003) for value; (2) Pajares (2006) for self-efficacy; (3) Clark and Estes (2008) for
emotions; (4) Pintrich (2003) and Anderman and Anderman (2009) for attributions; Pintrich
(2003) and Yough and Anderman (2006) for goal orientation. The main artifacts used for
assessing teacher motivation were various meeting notes. Table 6 lists each interview item and
artifacts that were used to assess each assumed and prioritized influence on teacher motivation.

Table 6
Summary of Motivation Influences and Method of Assessment
Assumed Motivation Influences Interview Items Artifacts
Value
Teachers value conducting
meetings to discuss program
completion requirements.*
How important is it for
teachers to conduct
meetings to discuss program
completion requirements?

Teachers value implementing
pedagogical strategies to
support students in completing
the program.*
Tell me about the
importance you place on
providing culturally relevant
pedagogy in your
instruction.

Teachers value taking steps
towards program
improvement.

Self-Efficacy
46

Teachers are confident that they
can conduct biweekly
meetings to discuss program
completion requirements.

Teachers are confident that they
can implement pedagogical
strategies daily to support
students in completing the
program.

Teachers are confident that they
can take steps towards
improving the program.*
How confident are you in
taking towards program
improvement right now?

Emotions
Teachers feel positively about
conducting biweekly meetings
to discuss STEM Program
requirements.*
How do you feel about
conducting biweekly
meetings to discuss program
completion requirements?

Teachers feel positive about
implementing pedagogical
strategies in class daily to
support students in completing
the program.

Teachers feel positive about
taking steps towards program
improvement.*
How do you feel about
taking steps towards
program improvement?

Attributions
Teachers have control over
conducting biweekly meetings
to discuss program
requirements.*
To what extent is
conducting biweekly
meetings in your own
control?

Teachers attribute implementing
pedagogical strategies in
classes daily to their control
over supporting students in
completing STEM program
requirements.

47

Teachers have control over
taking steps towards program
improvement.*
To what extent is taking
steps towards program
improvement in your own
control?

Goal Orientation
Teachers address all challenges
in setting program completion
requirements.*
What are the challenges in
setting program completion
requirements?

How have you addressed
these challenges?

Teachers master and implement
all pedagogical strategies
daily.

Teachers address all challenges
in taking steps towards
program improvement.*
What are the challenges in
taking steps towards
program improvement?

How have you addressed
these challenges?

Organization/Culture/Context Assessment
The STEM Program teachers’ organizational influences that were assessed are described
in Chapter Two. This section specifies how each of the organizational factors were assessed.
Gallimore and Goldenberg’s (2001) research served as the basis for the interview questions that
evaluate the organization’s cultural model, policies and procedures, and resources. The cultural
setting was evaluated through interview items composed using Scott and Palinscar’s (2008)
research on sociocultural theory. Artifacts analyzed for organizational influences include a wide
range of sources, such as program, site, and district webpages, program meeting notes, lesson
48

plans, and observations. Table 7 shows which interview items and artifacts were used to assess
each prioritized organizational influence on STEM Program teachers.

Table 7
Summary of Organization Influences and Method of Assessment
Assumed Organizational Influences Interview Items Artifacts
Cultural Models
Teachers experience a culture of
trust in which they can conduct
biweekly meetings to discuss
STEM Program requirements.

Teachers experience a culture of
trust in which they feel
comfortable implementing
pedagogical strategies in classes
daily that support students in
completing STEM Program
requirements.

Teachers perceive that they are part
of an organizational culture that
values program improvement.*
Do you feel like the
teachers, counselors, and
administrators value
taking steps towards
program improvement?

Cultural Setting
Teachers have role models from
other academic and career
programs at LHS who also
conduct regular meetings to
discuss program completion
requirements.*
Do you have role models
or other successful
programs to review in
conducting regular
meetings?

49

Teachers have role models from
other academic and career
programs at LHS who also
implement pedagogical strategies
in class daily to support students
in program completion.

Teachers have role models from
other academic and career
programs at LHS who evaluate
steps taken towards program
improvement.
Do you have role models
or other successful
programs to review in
evaluating steps taken
towards program
improvement?

Policies/Procedures
Policies at LHS support teachers in
conducting biweekly meetings to
discuss STEM program
completion requirements.

Policies at LHS support teachers in
implementing pedagogical
strategies in classes daily that
support students in completing
program requirements.

Policies at LHS support teachers in
evaluating steps taken towards
program improvement.
What policies affect your
ability to take steps
towards program
improvement?

To what extent do the
policies support or deter
your ability to take steps
towards program
improvement?

Resources
Teachers have time to conduct
biweekly meetings to discuss
STEM Program completion
requirements.

How much time do you
have for conducting
meetings to discuss
program completion
requirements?

50

Teachers have resources to
implement pedagogical strategies
in class daily that support students
in program completion.
What sorts of resources do
you have to implement
pedagogical strategies in
class to support students
in program completion?

What sorts of resources do
you need to implement
pedagogical strategies in
class to support students
in program completion?
Lesson Plans
Teachers have time and resources to
evaluate steps taken towards
program improvement.
How much time do you
have to take steps
towards program
improvement?

What sorts of resources do
you have to take steps
towards program
improvement?

What sorts of resources do
you need to take steps
towards program
improvement?

Participating Stakeholders and Sample Selection
The stakeholder group of focus for this study is the STEM Program teachers. The
program is run by seven teachers, four of which teach the math courses in the program, while the
remaining three teach the program’s science courses.
Sampling
The goal was to interview and collect artifacts from all teachers since there are so few.
The small population size provides the opportunity to collect artifacts and thorough interview
51

responses from the entire program’s teachers. Gathering data from the entire group of
stakeholders provided all of the data required and even available to conduct the study.
Recruitment
Upon approval from the University of Southern California (USC) Institutional Review
Board (IRB), all STEM Program teachers were contacted via email for all interviews and
artifacts. The email detailed the parameters of the study and what was asked of each teacher. It
also included a digital consent form, which teachers were able to sign electronically. After
teachers signed said consent form, they responded to the email to provide the artifacts and
schedule a time and place for the interview.
Instrumentation
The instrumentation used for this study were a semi-structured interview protocol and
analysis of artifacts collected.
Interview Protocol Design
This study utilized Creswell’s (2009) guide for semi-structured interviews. The interview
protocol included instructions, interview questions, probes for questions to elicit more detailed
responses, and a final thank you statement. Interview questions assessed the assumed knowledge,
motivation, and organizational influences affecting STEM Program teachers’ implementation of
critical behaviors. Interviews were transcribed under pseudonyms and kept in a password-
protected drive.
Artifact Analysis Design
Artifacts collected were analyzed as data points to help assess teachers’ knowledge,
motivation, and organizational influences. STEM Program teachers used Canvas as their online
learning management system. Each teacher’s Canvas page, the program’s Canvas page, site and
52

district webpages, and meeting notes were all scanned for key information that indicated whether
or not, or to what extent, the knowledge, motivation, and organizational factors affect teachers in
implementing their critical behaviors. Lesson plans and classroom walkthroughs were also
analyzed for evidence of time and supply resources teachers have or need in order to carry out
their critical behaviors.
Data Collection
Interviews
Per Creswell (2009) and the global pandemic COVID-19 restrictions, STEM Program
teachers were interviewed via online video conferencing, in a quiet and private setting such as
their classrooms. Each interview lasted approximately 30-45 minutes. Interviews followed the
interview protocol, in which the purpose of the interview was stated, consent for an audio
recording was verbally requested, and interview questions were asked. Upon completion of the
interviews, participants were asked for permission to be contacted regarding follow-ups if
necessary. They were thanked for their time as well. Interviews were transcribed and kept on a
password-protected drive. After transcription, audio recordings were deleted.
Artifacts
Artifacts were collected from teachers via email, unless publicly available. Teachers
needed to share their class Canvas pages and the STEM Program Canvas page, but the site and
district pages are publicly available. Digital meeting notes and lesson plans needed to be shared.
Classroom walkthroughs were conducted before or after interviews, since interviews were
already occurring in STEM Program teachers’ classrooms.
53

Data Analysis
Interviews
Interview analysis began with reading and coding the digital transcripts created from the
audio recordings. Codes represent each of the knowledge, motivation, and organizational
influences being analyzed. Patterns of effective strategies and areas of need that emerged from
coding were used to guide the solutions presented at the end of the study. Interview data were
kept confidential and personal information was de-identified so as to protect teacher identities.
Artifacts
The program Canvas page and teacher meeting notes were assessed for key words and
phrases that indicated teachers’ knowledge, motivation, and organizational influences. Site and
district webpages were analyzed for organizational influences, such as policies, that affected
teachers in implementing their critical behaviors. Lesson plans, class Canvas pages, and
classroom walkthroughs were also used to assess organizational influences, specifically time and
supply resources available or needed.
Trustworthiness of Data
In order to maintain the credibility and trustworthiness of this study, all of the data
collected were from different sources. The collection and analysis of multiple data points
allowed for saturation of data, indicating that the quality and quantity of data were high enough
that no new information would present itself (Merriam & Tisdell, 2016). A data transcribing
service was used to digitally transcribe the interviews in order to ensure the validity and accuracy
of interview data (Creswell, 2009). Interviews were conducted by a non-district affiliated peer so
as to avoid any potential reactions from participants that could alter their responses and pose a
threat to validity (Creswell & Creswell, 2018).
54

Role of Investigator
The investigator was one of the teachers in the STEM Program. The investigator obtained
permission to conduct this study from USC IRB, LHS’s principal, and the STEM Program co-
directors. Artifact collection and data analysis was conducted by the investigator, who served as
the main point of contact for the study. Interviews were conducted virtually by an independent
non-district-affiliated USC Rossier graduate student.
Limitations and Delimitations
As this is a case study, its limitation is generalizability. The study is not as representative
of all high school STEM programs, and the solution presented may not be as effective or
applicable to other programs. Since the investigator is one of the teachers in the program,
reliability and validity also present themselves as limitations. Interview responses and data
analysis may be skewed due to investigator bias.
Delimitations of this study include the sample. Since there are only seven teachers in the
program, the sample is truly representative of the population of LHS’s STEM Program teachers.
Furthermore, the specificity of a case study allows for a holistic investigation of the research
questions in context. Thus, the solution presented will likely be more effective in providing
program-specific strategies for program improvement.
55

Chapter Four: Results and Findings
The factors affecting the STEM Program teachers were determined, as described in
Chapter Two, from the three critical behaviors required to achieve 100% program completion:
1. Conduct biweekly meetings to discuss STEM Program completion requirements.
2. Implement pedagogical strategies in classes daily that support students in
completing STEM Program requirements.
3. Evaluate, in biweekly meetings, whether and to what extent steps taken towards
STEM Program improvement are effective.
Each factor was categorized into knowledge, motivation, and organizational influences per Clark
and Estes’ (2008) gap analysis framework. Thus, the results of this study will be presented by
knowledge type, motivational construct, and organizational dimension.
As described in Chapter Three, qualitative data - interviews, webpages, lesson plans,
pacing guides - were collected to explore the knowledge, motivation, and organizational factors
that influenced teachers in performing the critical behaviors. First, the interviews were conducted
by a facilitator not associated with the project. Following the interviews, the interviewer reached
out to participants to request lesson plans and pacing guides. Finally, data such as program
information and assignments were collected from publicly available site, district, and program
web pages.
Participating Stakeholders
Interviews were conducted with four of the seven program teachers. A school site
administrator unaffiliated with this project reached out to all program teachers to protect the
privacy of the teachers. The administrator forwarded a recruitment email from a Data
Collector/Manager affiliated with the school, providing instructions on how to reach out to the
56

interviewer if they were interested in participating in the program. After multiple reminder
emails from the administrator, a total of four of the seven teachers reached out to the interviewer
to participate. After conducting all four interviews, the interviewer followed up with each of the
teachers for lesson plans, pacing guides, and other curriculum resources specific to the program.
Two of the four teachers sent documents back.
Determination of Assets and Needs
Interviews and document analysis (web pages, and curriculum resource documents) serve
as the sources of data for this study. Interviews were conducted with four program teachers.
Following transcription, interview responses were categorized under knowledge, motivation, and
organizational factors as outlined in Chapter Three. After coding, interview responses were
categorized as “needs” or “assets.” If all four teachers had consistent responses, then the
corresponding knowledge type, motivational construct, or organizational dimension for that
critical behavior was marked as an asset. If one or more of the teachers’ responses was
inconsistent from the other teachers’ responses or indicated any uncertainty, then the
corresponding factor was marked as a need.
Interview data was then triangulated with web pages and curriculum resource documents.
These additional data sources confirmed whether or not each knowledge, motivation, or
organizational factor was indeed a need or an asset. If interview responses and coding were
consistent with characteristics of the web pages and curriculum resource documents, then each
factor remained either a need or an asset. If the coding was inconsistent with data from the
additional sources, then the factor was identified as a need.
57

Results and Findings for Knowledge Causes
Participants’ knowledge, motivation, and organizational influences were assessed via
interviews and document artifacts. These data analyses were used to determine whether each
factor was considered an asset or a need for supporting teachers in engaging in critical behaviors
required to achieve their goal of 100% program completion. This section describes the results
and findings using the assumed causes for each knowledge category.
Factual Knowledge
Influence 1
Teachers know the program completion requirements.
Interview Findings. Participants were asked, “What requirements must students fulfill in
order to complete the program?” Interview responses were inconsistent; one participant
described three major requirements, two participants listed two requirements, and the last
participant only mentioned one requirement. Participant One listed and described the
requirements as (1) enrollment in at least one math and one science each year, (2) active
participation in Project Week, and (3) stewardship hours and a senior stewardship capstone.
Participants Two and Three also described stewardships and program-related math and science
courses but did not mention Project Week. Participant Four only discussed math and science
course enrollment. Since interview responses were inconsistent, this influence is considered a
need.
Document Analysis. No documents were analyzed for this influence.
Summary. For teachers to achieve the threshold goal of 100% program completion, they
must know the program completion requirements. Interview data indicated that teachers do not
58

know said requirements, as their responses were inconsistent relative to one another. Thus, this
influence is deemed a need.
Influence 2
Teachers know steps to program improvement.
Interview Findings. Participants were asked, “What do teachers need to do to help
improve the program?” Again, interview responses were inconsistent. Participant One described
“developing clear guidelines” and “consistency” about student expectations. Participant Two
suggested having more time and experience to gain curricular expertise, while Participant Three
also responded with time, in addition to collaboration. Participant Four listed (1) opportunities
for students to speak to the community and delve into the STEM industries, (2) teachers
conducting reflections, and (3) connections between the curriculum and applications. Since
responses were inconsistent, this influence is considered a need.
Document Analysis. No documents were analyzed for this influence.
Summary. In order to achieve 100% program completion, teachers must know steps to
program improvement. When asked about the steps to improving the program, responses were
inconsistent across participants. Inconsistency in responses indicates that this influence is a need.
Conceptual Knowledge
Influence 1
Teachers know the three categories of tasks students must carry out to complete the
program.
Interview Findings. To analyze this influence, teachers were asked, “What requirements
must students fulfill in order to complete the program?” Participant responses were inconsistent.
Participant One listed all three categories as (1) math and science course enrollment, (2) Project
59

Week participation, and (3) stewardship. Participants Two and Three also listed (1) math and
science course enrollment and (2) stewardship, and Participant Four only listed math and science
course enrollment. Since teacher responses were inconsistent, this influence is deemed a need.
Document Analysis. The STEM Program web page is also lacking in clear guidelines for
program expectations. Requirements are not listed anywhere on the web page. There is a general
overview and links to applications, although the program no longer requires students to apply.
Summary. Interview responses were inconsistent across each participant. Analysis of the
program web page also lacked any of the three existing program completion requirements and
included past applications that are no longer considered program requirements. The conflicting
responses indicate that this influence is a need.
Procedural Knowledge
Influence 1
Teachers know how to identify and manage students’ progress on program completion
requirements.
Interview Findings. For this influence, teachers were asked, “How do you identify and
manage students’ progress on program completion requirements?” All four participants referred
to the learning management system as the tool used to identify and manage student progress.
Since all of the interview responses were consistent, this influence is considered an asset.
Document Analysis. No documents were analyzed for this influence.
Summary. Interview responses were analyzed to determine whether or not this influence
is an asset or need for teachers to achieve their goal of 100% program completion. Each
participant identified the program’s learning management system as their method for tracking
student progress. Consistency across all responses indicates that this influence is an asset.
60

Influence 2
Teachers know how to implement pedagogical strategies in classes daily that support
students in completing STEM Program requirements.
Interview Findings. For this influence, the instructional strategy of focus is culturally
relevant pedagogy. Participants were asked, “How do you incorporate culturally relevant
pedagogy in your instruction?” Responses to this question conflicted. Participant One described
the use of culturally relevant pedagogy as challenging since the demographics of the school and
program were different from what the participant grew up in, trained for, and previously
experienced. Participant Two indicated that they tried to relate content to current events but
experienced difficulty in drawing connections to current events outside of STEM-related topics.
Participant Three described having a diverse group of student leaders to represent the program
and working to make the curriculum accessible to any and all students. Participant Four
described tying the sciences in with a diverse group of contributing members of society. The
inconsistency of responses indicates that this influence is a need.
Document Analysis. Lesson plans from Participants One and Two were analyzed for
evidence of culturally relevant pedagogy. Participants Three and Four did not provide lesson
plans. Neither of the lesson plans provided allude to the use of students’ cultures, experiences, or
perspectives to teach students cultural competence or even STEM-related concepts. The lack of
culturally relevant pedagogy suggests that this influence is a need.
Summary. Teachers must know how to implement pedagogical strategies in classes daily
in order to achieve their goal of 100% program completion. This study looked specifically at
culturally relevant pedagogy as a necessary strategy. Inconsistency across all four participants’
61

interview responses and analysis of lesson plans from two participants indicate that this influence
is a need.
Metacognitive Knowledge
Influence 1
Teachers monitor their roles in managing students’ progress towards program
completion.
Interview Findings. Participants were asked, “Tell me how you reflect upon student
progress in completing the program.” All four participants discussed communicating with other
program teachers and checking in with students as ways to reflect on their roles in managing
student progress. Since all interview responses were consistent, this influence is considered an
asset.
Document Analysis. No documents were analyzed for this influence.
Summary. To achieve the goal of 100% program completion, teachers must monitor
their roles in managing student progress. Each of the participants described collaborating with
one another and checking in with students in order to reflect on student progress; thus, this
influence is considered an asset.
Influence 2
Teachers monitor their roles in taking steps towards program improvement.
Interview Findings. Participants were asked, “Tell me how you reflect upon progress
towards program improvement.” Teacher responses were consistent. Each participant described
taking steps towards program growth and collecting feedback from stakeholders. Participant One
discussed growth in enrollment and collecting feedback from students in order to inform any
changes in the program. Participants Two and Four referred to program growth in terms of
62

improving program activities, as well as feedback from students. Participant Three described
program changes and growth based on parents and student feedback. Interview responses were
consistent, which indicates that this influence is an asset.
Document Analysis. No documents were analyzed for this influence.
Summary. Teachers must monitor their roles in taking steps towards improving the
STEM Program. All four participants described some form of program growth through
stakeholder feedback, suggesting that this influence is an asset.
Results and Findings for Motivation Causes
Value
Influence 1
Teachers value conducting meetings to discuss program completion requirements.
Interview Findings. Teachers were asked, “How important is it for teachers to conduct
meetings to discuss program completion requirements?” All four participants said that
conducting meetings to discuss program completion requirements was of high importance. Since
interview responses were consistent across all participants in that all participants value
conducting meetings, this influence is considered an asset.
Document Analysis. No documents were analyzed for this influence.
Summary. Program teachers must value conducting meetings to discuss program
completion requirements in order to achieve their goal of 100% program completion. Interview
responses indicate that all participants value conducting said meetings; thus, this influence is
deemed an asset.
63

Influence 2
Teachers value implementing pedagogical strategies to support students in completing the
program.
Interview Findings. Participants were prompted, “Tell me about the importance you
place on providing culturally relevant pedagogy in your instruction.” Each participant indicated
that they place high value in implementing culturally relevant pedagogy in the program. All
interview responses were similar, which indicates that this influence is an asset.
Document Analysis. No documents were analyzed for this influence.
Summary. Teachers must value implementing pedagogical strategies, especially
culturally relevant pedagogy, that support their students in completing the program. In their
interview responses, all four program teachers said that they believe culturally relevant pedagogy
is extremely important, which suggests that this influence is an asset.
Self-Efficacy
Influence 1
Teachers are confident that they can take steps towards improving the program.
Influence 1 Interview Findings
In their interviews, participants were asked, “How confident are you in taking steps
towards program improvement right now?” Three of the four participants said they were not
confident. Participant One attributed the lack of confidence to the shorter class times that
resulted from the COVID-19 pandemic, saying that teachers were more focused on adjusting the
curriculum to satisfy content standards and requirements in the shorter amount of time.
Participant Two also described a greater focus on curriculum and policy changes rather than
taking steps towards program improvement. Participant Three responded, “I can't say I'm too
64

confident in it. Only because so much of what we're doing at school is just how do we get by and
care for our students.” The only confident teacher was Participant Four, who indicated that
continuing to consistently meet, collaborate, and make improvements would contribute to
moving forward. The inconsistency in teacher responses and lack of confidence from three of the
four participants indicates that this influence is a need.
Influence 1 Document Analysis
No documents were analyzed for this influence.
Influence 1 Summary
In order to achieve the goal of 100% program completion, teachers must feel confident
that they can take steps towards improving the program. Three of the four participants were not
confident, for reasons that varied; the fourth participant was confident. The lack of confidence
from most of the participants, as well as the varying reasons as to why they lack confidence,
shows that this influence must be considered a need.
Goal Orientation
Influence 1
Teachers address all challenges in setting program completion requirements.
Interview Findings. To assess this influence, participants were asked the following
questions:
1. What are the challenges in setting program completion requirements?
2. How have you addressed these challenges?
Responses to the first question conflicted. Participant One described the main challenge
in setting program completion requirements as developing consistency across the program by
bridging the visions for the program with implementation of said visions by defining specific
65

requirements that are clear to both the teachers and the students. Participant Two discussed
course scheduling conflicts as the greatest obstacle in setting program completion requirements.
Similarly, Participant Three offered monitoring or balancing courses and extracurricular work as
a challenge. Participant Four described the challenge of getting students who “fall through the
cracks” back on track with program requirements.
While responses to the first question conflicted across each interview, teachers did offer
solutions to each obstacle. Participant One said that while there was room for improvement
regarding setting clear expectations, they did try to address the challenge by having a balance
between clear, bulleted lists of instructions and prompts that fostered critical thinking. Participant
Two offered flexibility in course scheduling requirements as a solution to scheduling conflicts.
Participant Three discussed communicating and reflecting with student representatives, and
Participant Four described annual teacher retreats as opportunities for teachers to discuss
program improvements.
While each participant did offer up solutions to the challenges they presented, responses
were inconsistent in regard to the challenges they faced in setting program completion
requirements. Conflicting responses ultimately indicate that this influence is a need.
Document Analysis. No documents were analyzed for this influence.
Summary. Program teachers must address all challenges in setting program completion
requirements. Per their interview responses, participants did offer solutions for addressing each
of the challenges they presented; however, each participant listed different challenges from one
another. The inconsistency in challenges described by each teacher suggests that this influence
must be considered a need.
66

Influence 2
Teachers address all challenges in taking steps towards program improvement.
Interview Findings. For this influence, participants were asked the following two
questions:
1. What are the challenges in taking steps towards program improvement?
2. How have you addressed these challenges?
Each teacher listed a different challenge in taking steps towards program improvement.
Participant One described time, resources, and the effects of the COVID-19 pandemic on modes
of learning as the main challenges. Participant Two said that producing relevant new curriculum
was an obstacle, while Participant Three discussed policies and procedures related to course
logistics and certifications. Participant Four listed collaboration and consensus across all teachers
as challenges.
Since each participant listed a different challenge, they also provided various solutions to
addressing each challenge. Participant One described grants and support from the program’s
booster club as some solutions while also mentioning that teachers have no control over the
challenge of having enough time in their classrooms. Participant Two observed other STEM
programs in an attempt to model their curriculum after other courses but lost traction with
adopting special curriculum. Participant Three also discussed creativity with curriculum and
teacher credentialing, and Participant Four offered teacher resolutions as solutions to challenges
with collaboration and consensus.
Again, teacher responses regarding program challenges were inconsistent. Furthermore,
participants indicated that some of the challenges they listed had not yet been addressed. Thus,
67

the influence of addressing all challenges in taking steps towards program improvement must be
considered a need.
Document Analysis. No documents were analyzed for this influence.
Summary. In order to achieve the goal of 100% program completion, STEM Program
teachers must address all challenges in taking steps towards program improvement. When asked
to identify the challenges in taking steps towards improvement, participants’ responses were
inconsistent in that each teacher listed a different challenge. Teachers also indicated that some of
the challenges they listed had not yet been fully addressed. The conflicting responses, in
combination with a lack of viable solutions to all challenges, indicates that this influence is a
need.
Results and Findings for Organization Causes
Cultural Models
Influence 1
Teachers perceive that they are part of an organizational culture that values program
improvement.
Influence 1 Interview Findings
Participants were asked, “Do you feel like faculty value taking steps towards program
improvement?” All four participants shared similar responses, faculty outside of the program
value and support the program. Since all interview responses were consistent, this influence is
considered an asset.
Influence 1 Document Analysis
No documents were analyzed for this influence.
68

Influence 1 Summary
Program teachers must perceive that other school faculty value program improvement in
order to achieve the goal of 100% program teachers. Participants all indicated that the
organizational culture values program improvement, which suggests this influence is an asset.
Cultural Setting
Influence 1
Teachers have role models from other academic and career programs at LHS who also
implement pedagogical strategies in class daily to support students in program completion.
Influence 1 Interview Findings
To analyze this influence, participants were asked the following questions:
1. What sorts of resources or references do you have to implement ASE-specific
pedagogical strategies?
2. What sorts of resources do you need to implement ASE-specific pedagogical
strategies?
Teacher responses were consistent for the first question. All four participants described supplies
and curriculum resources provided by colleagues and the booster club. For the second question,
however, interview responses varied. Participant One described time as a necessary resource,
while Participants Two and Three indicated a need for defined curricula and project ideas.
Participant Four lab supplies and funding. Although each participant responded similarly to the
first question, responses to the second question were inconsistent. Additionally, none of the
participants mentioned any other programs that could serve as role models for improving the
STEM program. Thus, this influence is a need.
69

Influence 1 Document Analysis
No documents were analyzed for this influence.
Influence 1 Summary
STEM Program teachers must have role models from other academic and career
programs in order to improve and achieve their goal of 100% program completion. Interview
findings indicate that there is a need for role models and references. This influence is considered
a need.
Policies and Procedures
Influence 1
Policies at LHS support teachers in evaluating steps taken towards program
improvement.
Influence 1 Interview Findings
Participants were asked the following two questions:
1. What policies affect your ability to take steps towards program improvement?
2. To what extent do the policies support or deter your ability to take steps towards
program improvement?
In response to the both questions for this influence, Participants One, Two and Three answered
similarly. They indicated that scheduling was a major policy affecting their ability to take steps
towards program improvement. However, these policies were obstacles but not roadblocks.
Scheduling in particular made program improvement challenging, but teachers worked to create
different solutions or target different populations of students. Participant Four seemed to struggle
in answering both questions and did not provide a clear response. Since Participants One, Two,
70

and Three responded similarly to both questions and Participant Four’s confusion was likely an
outlier, this influence is considered an asset.
Influence 1 Document Analysis
No documents were analyzed for this influence.
Influence 1 Summary
In order to achieve their goal of 100% program completion, school policies must support
STEM Program teachers in evaluating steps taken towards program improvement. Interview
responses were consistent across Participants One, Two, and Three, while confusion from
Participant Four is deemed an outlier. Thus, this influence is considered an asset.
Resources
Influence 1
Teachers have time to conduct biweekly meetings to discuss STEM Program completion
requirements.
Interview Findings. Participants were asked, “How much time do you have for
conducting meetings to discuss program completion requirements?” All four participants
responded similarly, indicating that they had about an hour every other week to meet. None of
the participants said they need more time to discuss program completion requirements, so this
influence is considered an asset.
Document Analysis. No documents were analyzed for this influence.
Summary. Program teachers must have time to conduct biweekly meetings to discuss
program completion requirements. Interview findings indicate that teachers do indeed have time
to do so, which suggests that this influence is an asset.
71

Influence 2
Teachers have resources to implement pedagogical strategies in class daily that support
students in program completion.
Interview Findings. For this influence, participants were asked the following two
questions:
1. What sorts of resources do you have to take steps towards program improvement?
2. What sorts of resources do you need to take steps towards program improvement?
All four participants answered the first question similarly, saying they had each other as
resources. Teachers indicated that they were each other’s support system. While this consistency
in responses might indicate that this influence is an asset, responses to the second question
suggest otherwise. Participant One described a need for additional lab supplies, while Participant
Two discussed a need for curriculum. Participant Three said they were unable to answer the
question, and Participant Four said they required funding and new projects. The conflicting
responses to the second question indicate that this influence is considered a need.
Document Analysis. Lesson plans from Participants One and Two were analyzed with
respect to this influence. Participants Three and Four did not provide any lesson plans. Details
listed in both of the lesson plans provided confirm teacher responses to the first question for this
influence. Participants described supplies and curriculum resources, both of which the lesson
plans appear to provide. The lesson plans also confirm the need for time and collaboration. Each
of the lesson plans indicate that time is needed both in and out of the classroom, along with
collaboration amongst teachers in order to plan and implement the projects described in the
lesson plans. Thus, the lesson plans confirm what the interviews suggest - that this influence is a
need.
72

Summary. In order to achieve their goal of 100% program completion, teachers must
have resources to implement pedagogical strategies in class daily. Per the interview responses
and lesson plan analyses, teachers have supplies and curriculum resources but are in need of
time, defined curricula, project ideas, and additional supplies and funding. This influence is
therefore considered a need.
Summary of Validated Influences
Tables 8, 9, and 10 show the knowledge, motivation and organization influences for this
study and their determination as an asset or a need.

73

Table 8
Knowledge Assets or Needs as Determined by the Data
Assumed Knowledge Influence Asset or Need
Factual
Teachers know the program completion
requirements.
Need
Teachers know steps to program
improvement.
Need
Conceptual
Teachers know the three categories of tasks
students must carry out to complete the
program.
Need
Procedural
Teachers know how to identify and manage
students’ progress on program completion
requirements.
Asset
Teachers know how to implement
pedagogical strategies in classes daily that
support students in completing STEM
Program requirements.
Need
Metacognitive
Teachers monitor their roles in managing
students’ progress towards program
completion.
Asset
Teachers monitor their roles in taking steps
towards program improvement.
Asset

74

Table 9
Motivation Assets or Needs as Determined by the Data
Assumed Motivation Influence Asset or Need
Value
Teachers value conducting meetings to
discuss program completion requirements.
Asset
Teachers value implementing pedagogical
strategies to support students in completing
the program.
Asset
Self-Efficacy
Teachers are confident that they can take
steps towards improving the program.
Need
Goal Orientation
Teachers address all challenges in setting
program completion requirements.
Need
Teachers address all challenges in taking
steps towards program improvement.
Need

75

Table 10
Organizational Assets or Needs as Determined by the Data
Assumed Organizational Influence Asset or Need
Cultural Models
Teachers perceive that they are part of an
organizational culture that values program
improvement.
Asset
Cultural Setting
Teachers have role models from other
academic and career programs at LHS who
also implement pedagogical strategies in
class daily to support students in program
completion.
Asset
Policies and Procedures
Policies at LHS support teachers in evaluating
steps taken towards program improvement.
Asset
Resources
Teachers have resources to implement
pedagogical strategies in class daily that
support students in program completion.
Need
Teachers have time to conduct biweekly
meetings to discuss STEM Program
completion requirements.
Asset

The results and findings presented in Chapter Four indicate whether each knowledge,
motivation, and organizational influence is an asset or a need based on interview and artifact data
analysis. The findings inform the recommendations and evaluation presented in Chapter Five.

76

Chapter Five: Recommendations and Evaluation
This project is designed to evaluate the extent to which LHS’s STEM program is
achieving its goal of 100% program completion. The research questions that guide this study in
evaluating the extent to which program teachers, the stakeholder of focus, are achieving this goal
are as follows:
1. To what extent is LHS’s STEM Program meeting its goal of 100% program
completion by June 2022?
2. What are the knowledge, motivation and organizational influences related to
STEM Program teachers achieving their goal of implementing recommendations
made towards program improvement to achieve 100% student completion?
3. What are the recommendations for organizational practice in the areas of
knowledge, motivation, and organizational resources needed for STEM Program
teachers to achieve their goal?
Program teachers must perform the following critical behaviors in order to achieve the goal of
100% program completion:
1. Conduct biweekly meetings to discuss STEM Program completion requirements.
2. Implement pedagogical strategies in classes daily that support students in
completing STEM Program requirements.
3. Evaluate, in biweekly meetings, whether and to what extent steps taken towards
STEM Program improvement are effective.
Per Clark and Estes’ (2008) gap analysis framework, the knowledge, motivation, and
organizational influences affecting whether or not teachers perform these behaviors were
identified. Then, based on these critical behaviors, interviews were conducted and documents
77

such as lesson plans and publicly available web pages were collected as data. These data were
analyzed in order to determine which influences were assets or needs. Finally the identification
of influences as assets or needs to the program serve as rationale in creating the
recommendations, implementation, and evaluation package presented in this chapter. The
recommendations and evaluation presented in this chapter reflect the consistency, continuity, and
commitment necessary for improvement of LHS’s STEM Program with respect to identified
needs, and maintenance of assets to ensure performance.
Recommendations to Address Knowledge, Motivation, and Organization Influences
Based on the results and findings of each knowledge, motivation, and organizational
influence presented in Chapter Four, this section presents recommendations for the STEM
Program in order to help teachers perform their critical behaviors and achieve the goal of 100%
program completion. Each knowledge, motivation, and organizational recommendation begins
includes a brief overview and rationale for prioritizing the validated causes, if any. Each sub-
section also includes a table with the corresponding knowledge, motivation, or organizational
cause, priority level, recommendations, and evidence-based principles to support each
recommendation. Tables are followed by detailed discussions for each high-priority cause,
principle, and solution.
Knowledge Recommendations
Results and findings show that four of the seven assumed knowledge influences are
needs. Factual and conceptual knowledge are needs, which indicates that they are of high
priority. Metacognitive knowledge, though, is an asset, so it is not a high priority. Some
procedural knowledge are assets, while others are needs which are considered high priority.
Table 11 lists each assumed knowledge influence, whether it is an asset or a need, its priority
78

level, the principle that guides the recommendation for improving performance, and the
recommendation itself. Following the table is a discussion of each context-specific solution
recommended and the literature to support the recommendation.

Table 11
Summary of Knowledge Influences and Recommendations
Assumed
Knowledge
Influence
Asset
or
Need
Priority
Yes or
No
Principle and Citation Context-Specific
Recommendation
Factual
Teachers know
the program
completion
requirements.
Need Yes Information learned
meaningfully and
connected with prior
knowledge is stored
more quickly and
remembered more
accurately because it is
elaborated with prior
learning (Schraw &
McCrudden, 2006).
Provide teachers with
opportunities to connect
what they already know
about the program with
program completion
requirements.
Teachers know
steps to
program
improvement.
Need Yes Information learned
meaningfully and
connected with prior
knowledge is stored
more quickly and
remembered more
accurately because it is
elaborated with prior
learning (Schraw &
McCrudden, 2006).
Provide teachers with
opportunities to connect
what they already know
about the program with
steps to program
improvement.
Conceptual
Teachers know
the three
categories of
tasks students
must carry out
to complete the
program.
Need Yes How individuals organize
knowledge influences
how they learn and apply
what they know (Schraw
& McCrudden, 2006).
Provide teachers with
opportunities to organize
what they know about
program completion
requirements into
categories.
Procedural
79

Teachers know
how to identify
and manage
students’
progress on
program
completion
requirements.
Asset Yes Effective observational
learning is achieved by
first organizing and
rehearsing modeled
behaviors, then enacting
them overtly (Mayer,
2011).

Feedback that is private,
specific, and timely
enhances performance
(Shute, 2008).
Model the process for
identifying and
managing student
progress, and then allow
teachers to follow the
procedure while
monitoring and
supporting them as
needed.
Teachers know
how to
implement
pedagogical
strategies in
classes daily
that support
students in
completing
STEM
Program
requirements.
Need Yes Effective observational
learning is achieved by
first organizing and
rehearsing modeled
behaviors, then enacting
them overtly (Mayer,
2011).

Feedback that is private,
specific, and timely
enhances performance
(Shute, 2008).
Provide teachers with
opportunities to observe,
rehearse, and enact the
pedagogical strategy of
culturally responsive
teaching.
Metacognitive
Teachers
monitor their
roles in
managing
students’
progress
towards
program
completion.
Asset Yes Self-regulatory strategies,
including goal setting,
enhance learning and
performance (APA,
2015; Dembo & Eaton,
2000; Denler, et al.,
2009).
Have teachers outline
their roles in managing
student progress towards
program completion, set
goals to satisfy these
roles, periodically reflect
on whether or not these
goals were met, and
create new goals when
previous ones have been
satisfied.
80

Teachers
monitor their
roles in taking
steps towards
program
improvement.
Asset Yes Self-regulatory strategies,
including goal setting,
enhance learning and
performance (APA,
2015; Dembo & Eaton,
2000; Denler, et al.,
2009).
Have teachers outline
their roles in taking steps
towards program
improvement, set goals
to satisfy these roles,
periodically reflect on
whether or not these
goals were met, and
create new goals when
previous ones have been
satisfied.

Declarative Knowledge Solutions
The results and findings indicate that there is a gap in program teachers’ factual
knowledge about (1) program completion requirements and (2) steps to program improvement.
To address this gap in factual knowledge, they should be provided with opportunities to relate
their existing knowledge about the program with both program completion requirements and
steps to program improvement. Schraw and McCrudden (2006) pose that information learned
meaningfully is remembered more accurately. To be presented meaningfully, this information
should be connected with prior knowledge. Since teachers are already familiar with various
aspects of the program, introducing program requirements and steps to program improvement
would be meaningful when related to program qualities.
In a study on college and career readiness through independent study, Brasch (2018)
found that making the content relevant to meaningful practice was necessary for program
success. Similarly, Cheeks and McDowelle (2018) posit that stakeholders must understand the
relevance of their work to prior interests and information. As such, LHS’s STEM Program
teachers should bring the gap in declarative knowledge by connecting the information about
program requirements and steps to program improvement with their existing knowledge.
81

Conceptual Knowledge Solutions
There is a gap in program teachers’ conceptual knowledge about STEM program
completion requirements. In particular, understanding the categories of program requirements is
essential to teachers’ conceptual knowledge of the program. To bridge this gap in conceptual
knowledge, program teachers should be provided with opportunities to organize what they
already know about program completion requirements into categories. Schraw and McCrudden
(2006) assert that how individuals organize their knowledge affects how they learn and apply
their learning. Opportunities to organize existing knowledge into categories allows teachers to
better understand each of the program completion requirements. In turn, they will be better
equipped to perform the critical behaviors required for achieving their goal of 100% program
completion.
Brand (2009) asserts that seeing the relationships between content learned and
application of content are essential for learning. Lane et al. (2020) also suggested that organizing
information supports learners with awareness skills. By providing STEM Program teachers with
the opportunities to organize their knowledge about the program into categories, they will be
better able to develop their contextual knowledge and perform the critical behaviors necessary to
achieve the goal of 100% program completion.
Procedural Knowledge Solutions
Teachers must know how to (1) identify and manage students’ progress on program
completion requirements and (2) implement pedagogical strategies in classes daily to support
students in completing STEM Program requirements. The results and findings indicate that
identifying and managing student progress is an asset, while there is room for improvement with
respect to program teachers’ procedural knowledge of how to implement these strategies,
82

particularly culturally responsive teaching. To address this gap in knowledge, teachers should
have the opportunities to observe, rehearse, and enact said strategies. Mayer (2011) argues that
effective learning is achieved by organizing, rehearsing, and enacting modeled behaviors. By
seeing examples of culturally responsive teaching, teachers have models of successful strategies
they can gain procedural knowledge from. Furthermore, private, specific, and timely feedback
also helps to facilitate learning (Shute, 2008). Therefore, allowing teachers to follow modeled
procedures while monitoring and supporting them is essential to supporting teachers’ procedural
knowledge.
Similar to Mayer’s (2011) and Shute’s (2008) assertions, Lane et al. (2020) also
emphasize the effectiveness of addressing knowledge gaps through support, feedback, and
corrective action. Being informed of these knowledge gaps and working to bridge them allows
teachers the opportunity for reflection and revision, which Conley and Darling-Hammond (2013)
discuss as useful forms of assessment. Providing teachers with the opportunities to follow
modeled procedures paired with feedback and support is essential for addressing their procedural
knowledge gaps.
Metacognitive Knowledge Solutions
Metacognitive knowledge about program completion requirements and program
improvement are both assets for teachers. However, they are still deemed a high priority in order
to sustain them as assets. Thus, teachers should outline their roles in (1) managing student
progress towards program completion and (2) taking steps towards program improvement. Per
the APA (2015), Dembo and Eaton (2000), and Denler et al. (2009), self-regulatory strategies
such as this enhance learning and performance. Teachers should periodically reflect on whether
83

or not their outlined goals were met in order to ensure that their metacognitive knowledge about
the program continues to be assets.
Per DiBenedetto (2015), reflective teaching practices, such as the one recommended as a
metacognitive knowledge solution, are essential for awareness and for solving real-world
problems. Conley and Darling-Hammond (2013) also emphasize that opportunities for reflection
and revision are crucial tools for metacognition and growth. Having teachers outline their roles
in managing students and taking steps towards program improvement will therefore help them
maintain metacognitive knowledge as an asset.
Motivation Recommendations
The results and findings described in Chapter Four indicate that three out of five of the
assumed motivation influences are needs. In particular, self-efficacy and goal orientation were
determined to be needs. As a result, they have been categorized as high priority. Value, however,
is an asset, which indicates that it is not a high priority. Table 12 lists each assumed motivation
influence and summarizes whether it is an asset or a need, high or low priority, recommendations
for addressing the influence if it is a priority, and the principle to support the recommendation.
Each recommended solution is explained in further detail following the table.

84

Table 12
Summary of Motivation Influences and Recommendations
Assumed
Motivation
Influence
Asset
or
Need
Priority
Yes or
No
Principle and Citation Context-Specific
Recommendation
Value
Teachers value
conducting
meetings to
discuss
program
completion
requirements.
Asset Yes Rationales that include a
discussion of the
importance and utility
value of the work or
learning can help
learners develop
positive values (Eccles,
2006; Pintrich, 2003).
Explain the importance of
conducting meetings to
discuss program
completion
requirements.
Teachers value
implementing
pedagogical
strategies to
support
students in
completing the
program.
Asset Yes Rationales that include a
discussion of the
importance and utility
value of the work or
learning can help
learners develop
positive values (Eccles,
2006; Pintrich, 2003).
Explain the importance of
implementing
pedagogical strategies
to support students in
completing the program.
Self-Efficacy
Teachers are
confident that
they can take
steps towards
improving the
program.
Need Yes Feedback and modeling
increases self-efficacy
(Pajares, 2006).
Model the program
improvement process
and provide teachers
with feedback
throughout the process.
Goal Orientation
Teachers address
all challenges
in setting
program
completion
requirements.
Need Yes Designing learning tasks
that are novel, varied,
diverse, interesting, and
reasonably challenging
promotes mastery
orientation (Yough &
Anderman, 2006).
Group and scaffold tasks
for teachers to set
program completion
requirements, and
provide a safe
environment for them to
take risks.
Teachers address
all challenges
in taking steps
towards
program
improvement.
Need Yes Designing learning tasks
that are novel, varied,
diverse, interesting, and
reasonably challenging
promotes mastery
orientation (Yough &
Anderman, 2006).
Group and scaffold tasks
for teachers to improve
the program, and
provide a safe
environment for them to
take risks.
85

Value Solutions
The value teachers place in (1) conducting meetings to discuss program completion
requirements and (2) implementing pedagogical strategies to support students in completing the
program are both high. Thus, value is considered an asset. In order to maintain value as an asset,
it must remain as a high priority influence. Teachers should continue to be able to explain the
importance of (1) conducting meetings to discuss program completion requirements and (2)
implementing pedagogical strategies to support students in completing the program. Eccles
(2006) and Pintrich (2003) post that a discussion of the importance and utility of work supports
the development of positive values. By maintaining discussions about program completion
requirements and culturally responsive pedagogical strategies, teachers will continue to have
positive values surrounding program completion requirements and program improvement.
Cheeks and McDowelle’s (2018) study on a career development program yielded similar
suggestions as they found the awareness of interests, skills and values to be important for
success. Irwin et al. (2013) also found the value in communication for self-confidence and
overall outcomes. Thus, maintaining conversations about meetings to discuss program
completion requirements and steps to program improvement will contribute to teachers’ values
as motivational assets.
Self-Efficacy Solutions
Program teachers must feel confident that they can take steps towards program
improvement. Without this self-efficacy, they cannot perform the critical behaviors required to
achieve their goal of 100% program completion. The results and findings indicate that there is a
gap in teachers’ self-efficacy. To address this gap, teachers should be provided with a model of
the program improvement process which they can practice and receive feedback on. Pajares
86

(2006) posits that feedback and modeling increases self-efficacy. Teachers who are not confident
in their ability to take steps towards program improvement should refer to existing models for
improvement while also accepting feedback throughout the process. By looking to other
successful models, teachers can become confident that they can also successfully take steps
towards program improvement. Accepting constructive feedback throughout the program
improvement process will also help to increase self-efficacy.
Per Brasch (2018), professional development that provides teachers with strategies and
opportunities for practice builds teacher efficacy. Lane et al. (2020) also assert that support,
feedback, and corrective action are essential strategies for success. With these opportunities for
feedback and growth, teachers can become more confident in their abilities to take steps towards
program improvement.
Goal Orientation Solutions
In order to engage in the critical behaviors necessary to achieve the goal of 100%
program completion, teachers must address all challenges in (1) setting program completion
requirements and (2) taking steps towards program improvement. The results and findings
presented in Chapter Four suggest that program teachers’ goal orientation is an area for
improvement. When teachers are reluctant to take risks and address challenging tasks, the tasks
should be grouped and scaffolded, while also providing teachers with a safe environment for
risk-taking. According to Yough and Anderman (2006), tasks that are novel, varied, diverse,
interesting, and reasonably challenging fosters mastery orientation. Having grouped and
scaffolded tasks encourages program teachers to complete the steps necessary for setting
program completion requirements and improving the program. Furthermore, maintaining a
87

positive climate allows teachers to feel comfortable in taking the risks associated with program
completion requirements and program improvement.
Lane et al. (2020) discussed challenging problem-solving approaches, along with
appropriate scaffolding, as key academic, behavioral, and cognitive strategies for fostering
positive habits of mind. These strategies allow stakeholders to safely take risks to address
challenges and grow. Similarly, LHS’s STEM Program teachers need a safe environment for
risk-taking in order to address the challenges in setting program completion requirements as well
as taking steps towards program improvement.
Organization Recommendations
Per the results and findings presented in Chapter Four, one of the five assumed
organizational influences is considered a need. This organizational need is related to resources
required to implement pedagogical strategies in class daily that support students in program
completion. Since this is the only need, it is considered a high priority. The remaining
organizational influences are assets, which suggests that they are not high priorities. Table 13
lists the organization causes, priority, principle and recommendations. Following the table, a
detailed discussion for each high priority cause and recommendation and the literature
supporting the recommendation is provided.

88

Table 13
Summary of Organization Influences and Recommendations
Assumed Organization
Influence
Asset or
Need
Priority
Yes or
No
Principle and
Citation
Context-Specific
Recommendation
Cultural Models
Teachers perceive that
they are part of an
organizational culture
that values program
improvement.
Asset Yes Communicate
constantly and
candidly to those
involved about
plans and progress
(Clark & Estes,
2008).
Openly communicate
with teachers about
plans for program
improvement.
Cultural Setting
Teachers have role
models from other
academic and career
programs at LHS who
also implement
pedagogical strategies
in class daily to
support students in
program completion.
Asset Yes Providing
scaffoldings and
assisted
performance in a
person’s zone of
proximal
developments
promotes
developmentally
appropriate
instruction (Scott
& Palinscar,
2006).
Have other successful
programs on campus
serve as models and
support systems for
STEM Program
teachers.
Policies and Procedures
89

Policies at LHS support
teachers in evaluating
steps taken towards
program
improvement.
Asset Yes Effective
organizations
ensure that
organizational
messages, rewards,
policies and
procedures that
govern the work of
the organization
are aligned with or
are supportive of
organizational
goals and values
(Clark and Estes,
2008).
Provide teachers with
time and
opportunities to
discuss policies and
procedures necessary
for program
improvement with
leaders.
Resources
Teachers have
resources to
implement
pedagogical strategies
in class daily that
support students in
program completion.
Need Yes Effective change
efforts ensure that
everyone has the
resources
(equipment,
personnel, time,
etc) needed to do
their job, and that
if there are
resource shortages,
then resources are
aligned with
organizational
priorities (Clark
and Estes, 2008).
Teachers work together
with program
stakeholders to
establish priorities
and obtain resources.
90

Teachers have time to
conduct biweekly
meetings to discuss
STEM Program
completion
requirements.
Asset Yes Effective change
efforts ensure that
everyone has the
resources
(equipment,
personnel, time,
etc) needed to do
their job, and that
if there are
resource shortages,
then resources are
aligned with
organizational
priorities (Clark
and Estes, 2008).
Teachers work together
with program
stakeholders to
establish priorities
and obtain resources.

Cultural Model Solutions
In order to perform the critical behaviors that support teachers in achieving their goal of
100% program completion, program teachers must perceive that they are part of an
organizational culture that values program improvement. The results and findings indicate that
teachers do indeed believe their organization values program improvement. Thus, this influence
is considered an asset. It is still considered a high priority, though, since it is necessary to
maintain this influence for continual program growth. In order to do so, Clark and Estes (2008)
suggest communicating constantly and candidly about plans and progress. Teachers should
continue to openly have conversations about their plans and progress for program improvement
to maintain a positive cultural model.
Brand’s (2009) study on career academies also emphasizes communication as an
important strategy for success. Maintaining open communication contributes to positive
outcomes. Irwin et al. (2013) describe communication as an important gateway to collaboration
and problem solving, which further support college and career readiness. STEM Program
91

teachers should regularly have discussions about program improvement in order to preserve the
cultural model influence as a program asset.
Cultural Settings Solutions
Teachers must have role models from other academic and career programs who also
implement pedagogical strategies in class daily to support students in program completion. Per
the results and findings presented in Chapter Four, these role models are present and available
for teachers. This influence is considered an asset and a high priority necessary for continued
program improvement. Program teachers should continue to look to other programs on campus
as role models for the STEM Program. Scott and Palinscar (2006) suggest that having other
programs as role models is a source of scaffolding and assisted performance, which promotes
appropriate instruction. Thus, teachers should continue to look at other programs to scaffold their
efforts for improvement.
Lane et al. (2020) emphasized the need for scaffolding as a support for problem-solving
approaches. Scaffolding and assistance serve as key strategies for academic, behavioral, and
cognitive success. Fletcher Claville et al. (2019) presented a model STEM, arguing that an
innovative model such as the one studied can be used to develop and systematically implement a
successful program. Both of these studies suggest that models provide scaffolding and assistance
necessary for program success. LHS’s STEM Program teachers should continue, then, to look to
other programs at the site as models to guide their efforts towards improvement.
Policies and Procedures Solutions
For teachers to perform the critical behaviors that are required in order to achieve their
goal of 100% program completion, the policies at LHS must support teachers in evaluating steps
taken towards program improvement. According to the results and findings, LHS’s policies do
92

indeed support program teachers in improving the program. This organizational influence is an
asset with high priority since it is important for teachers to continue to have policies that support
them. Clark and Estes (2008) posit that effective organizations ensure that policies and
procedures support organizational goals. Program teachers should continue to work with site
leaders to discuss policies and procedures necessary for program growth
Brand’s (2009) study shows that working with policymakers is essential for the success
of college and career readiness programs. Such policies accomplish tasks such as increasing
academic rigor, ensuring students have strong relationships with adults, and making school more
relevant. Similarly, Conley and Darling-Hammond (2013) discuss having buy-in from leaders
and other policymakers in order to develop programs. Thus, STEM Program teachers should
maintain their communication and relationships with key policymakers to ensure that policies
continue to support their program improvement endeavors.
Resources Solutions
In order to achieve the goal of 100% program completion, teachers must have (1)
resources to implement pedagogical strategies in class daily that support students in program
completion and (2) time to conduct biweekly meetings to discuss STEM Program completion
requirements. The results and findings presented in Chapter Four indicate that teachers have time
to conduct biweekly meetings but need resources to implement pedagogical strategies. To
address this need, program teachers should work together with other program stakeholders in
order to establish priorities and obtain resources. Per Clark and Estes (2008), effective change
efforts ensure that everyone has the resources needed to do their job; resources shortages are
aligned with organizational priorities. Any resources shortages indicate that the organization’s
priorities must be revisited in such a way that prioritizes resources needed for teachers to
93

perform their critical behaviors. In particular, these priorities must be realigned to provide
teachers with resources to implement pedagogical strategies to support students in program
completion. Thus, teachers must work with other program stakeholders to revisit and realign
priorities to obtain the necessary resources.
Foroozesh et al. (2019) also discussed coordinating resources to ensure that STEM
programs can provide opportunities and resources for their students to succeed both in and out of
the classroom. They assert that programs must work to help students overcome their unequal
educational backgrounds and challenges in order to ensure all students succeed. Helguera (2018)
examines the topic of resources from the perspective of allocating staff, equipment, and finances
as necessary for effective programs. STEM Program teachers must communicate and collaborate
with others to ensure that they have all resources necessary for program growth.
Summary of Knowledge, Motivation and Organization Recommendations
Each knowledge, motivation, and organizational influence was labeled as either an asset
or a need based on the results and findings presented in Chapter Four. Influences labeled as
needs were considered high priority, while assets were determined to be low priority. For each
high priority influence, a solution was recommended to help address the need. Each context-
specific recommendation was supported by literature and should help bridge any knowledge,
motivation, and organization gaps. The next section of this chapter combines all recommended
solutions into an integrated implementation and evaluation plan.
Integrated Implementation and Evaluation Plan
Organizational Purpose, Need and Expectations
The mission of LHS’s STEM program is to help prepare students for postsecondary lives
by developing their college and career readiness. To accomplish this, program teachers set the
94

goal of 100% program completion from students graduating in 2022. Program teachers must
perform the following three critical behaviors in order to ensure that they are working towards
this goal:
1. Conduct biweekly meetings to discuss STEM Program completion requirements.
2. Implement pedagogical strategies in classes daily that support students in
completing STEM Program requirements.
3. Evaluate, in biweekly meetings, whether and to what extent steps taken towards
STEM Program improvement are effective.
The knowledge, motivation, and organizational influences affecting whether and to what extent
program teachers are able to perform these critical behaviors were analyzed via interview and
document analysis. The results and findings indicate whether each influence is an asset or a need,
and any needs were deemed high priority. In hopes of supporting program teachers in
successfully performing the critical behaviors required to achieve 100% program completion, a
context-specific solution for each high priority item was formed and recommended based on
literature. This section delves deeper into implementing and evaluating the recommended
solutions.
Implementation and Evaluation Framework
Kirkpatrick and Kirkpatrick (2015) categorize program evaluation into four main levels.
The first level addresses base level learner awareness. These assessments measure reactional
characteristics such as motivation and attributions. The second level of evaluation focuses on the
impact of the program while it is being implemented, which measures learning outcomes and
learner performance. Level Three evaluates the transfer of knowledge--in particular, how
learners are able to apply the concepts to various situations. Level Four, which is the leading
95

indicator of program effectiveness, measures the results of the program. The focus of level four
is assessing whether or not the program was beneficial in achieving its overall goals.
Contrary to the traditional Kirkpatrick Model, the New World Kirkpatrick Model posits
that implementing and evaluating a program requires planning that actually starts with Level
Four of the Kirkpatrick and Kirkpatrick’s (2015) model and shifts down to Level One. First,
program teachers will measure Level Four, or the bottom line external and internal results. Then,
they will evaluate Levels Three based on the transfer of teachers’ critical behaviors, Level Two
based on learning during implementation, and Level One based on teachers’ reactions to the
programs. In contrast, implementing the evaluation plan occurs in reverse, from level one up to
level four. This section outlines the methods, timing, outcomes, and other key factors essential
for each level of learning assessment.
Level 4: Results and Leading Indicators
Per Kirkpatrick and Kirkpatrick (2016), Level 4 is results, which measures the extent to
which targeted outcomes occur following training and support. Leading indicators serve as short-
term observations and measurements that indicate whether or not the critical behaviors required
to achieve the targeted outcomes are on track. With respect to LHS’s STEM Program, “Results”
describes the degree to which program teachers are engaging in the three critical behaviors
required to achieve their goal of 100% program completion. The leading indicators, along with
metrics and methods for how they will be measured and observed, are listed in Table 14.

96

Table 14
Outcomes, Metrics, and Methods for External and Internal Outcomes
Outcome Metric(s) Method(s)
External Outcomes
Increase in school rank per
publicly available
dashboard
Position in the ranking
Track end-of-year ranking
from dashboard
Increased district ranking of
school performance
Position in district ranking list
Track end-of-year district
ranking from district web
page
Increased number of
acceptances into post-
secondary institutions
Number of post-secondary
institution acceptances
Collect data from students
regarding post-secondary
acceptances
Increased positive community
perceptions
Number of positive mentions
in public meetings, forums,
and web pages.
Collect data from meetings,
forums, and web pages
Internal Outcomes
Increased program
completion rates
Number of graduates who
complete the program
Track data from program
participants
Increased number of students
who apply to post-
secondary institutions
Number of students who
apply to post-secondary
institutions
Collect data from students
regarding post-secondary
plans
Increased positive
parent/guardian and student
perceptions
Number of positive comments
from parents/guardians and
students
Collect data from program
stakeholders

Level 3: Behavior
Critical Behaviors
Level 3 of Kirkpatrick and Kirkpatrick’s (2016) model is behavior, or the extent to which
participants apply their training. These behaviors are narrowed down to a few key behaviors that
97

the stakeholders of focus must consistently perform. For LHS’s STEM Program, the stakeholders
of focus are the teachers, and the critical behaviors and metrics, methods, and timing for
evaluation are listed in Table 15.

Table 15
Critical Behaviors, Metrics, Methods, and Timing for Evaluation
Critical Behavior Metric(s) Method(s) Timing
1. Conduct biweekly
meetings to discuss
STEM Program
completion
requirements.

Number of
consistently defined
program completion
requirements
List and describe
program completion
requirements
Biweekly
2. Implement
pedagogical
strategies in classes
daily that support
students in
completing STEM
Program
requirements.
Number of
integrations of
culturally relevant
pedagogical
strategies
Explicitly describe
and integrate
culturally relevant
pedagogical
strategies into
lesson plans
Per unit of instruction
3. Evaluate, in
biweekly meetings,
whether and to what
extent steps taken
towards STEM
Program
improvement are
effective.
Number of students
completing the
program
Track completion of
each program
requirement
Biweekly

98

Required Drivers
Required drivers are additions to Level 3. They are the processes and systems that
reinforce, monitor, encourage, and reward the performance of critical behaviors. They drive
stakeholders to engage in critical behaviors. The required drivers for LHS’s STEM Program
teachers to perform the critical behaviors are listed in Table 16, along with the methods and
timing for each.

Table 16
Required Drivers to Support Critical Behaviors
Method(s) Timing
Critical Behavior(s)
Supported
Reinforcing
Create reminders for
conducting biweekly
meetings and discussing
program requirements.
Biweekly 1
Provide resources and
training on culturally
responsive pedagogy
Resources - ongoing
Training - monthly
2
Conduct evaluations of steps
taken towards program
improvement.
Monthly 2, 3
Encouraging
Provide opportunities to
conduct meetings and
discuss program
requirements
Biweekly 1
Provide models of culturally
relevant pedagogy,
opportunities to apply
strategies, and feedback on
use of strategies
Monthly 2, 3
Provide feedback on steps
taken towards program
improvement
Monthly 1, 3
Rewarding
99

Celebrate completion of
program requirements
Quarterly 1
Highlight the use of culturally
relevant pedagogy in
leadership meetings and
Principal’s weekly
newsletter
Quarterly 2, 3
Celebrate program
improvement during
semester reviews
Biannually 3
Monitoring
Identify inconsistencies in
program requirements
Quarterly 1
Identify curricular
opportunities for use of
culturally responsive
teaching
Per unit of instruction 2
Identify steps not taken
towards program
improvement
Monthly 3

Organizational Support
In order to support STEM Program teachers in performing the critical behaviors, school
site leaders must be involved. They must openly communicate with program teachers about plans
for program improvement and provide teachers with the time, opportunities, and resources
needed. Site leaders must also constantly review policies and procedures to ensure that program
teachers are equipped with the means necessary for success. Essentially, school site leaders must
help make certain that the required drivers are in place for program teachers to engage in the
critical behaviors.
Level 2: Learning
Level 2 of the New World Kirkpatrick Model is learning, or the degree to which
participants gain the knowledge, skills, attitude, confidence, and commitment presented in
100

training (Kirkpatrick & Kirkpatrick, 2016). In the context of LHS’s STEM Program, teachers
must achieve each of the goals listed in order to successfully engage in the critical behaviors
required to achieve their goal of 100% program completion.
Learning Goals
The following learning objectives come from the knowledge, motivation, and
organization solutions presented at the end of Chapter Four. Upon engaging in the recommended
solutions, STEM Program teachers will be able to:
1. List, describe, and categorize program completion requirements.
2. List and take steps towards program improvement.
3. Identify and manage student progress on program completion.
4. Implement pedagogical strategies, particularly culturally responsive teaching, in
classes daily.
5. Monitor their roles in managing student progress and taking steps towards
program improvement.
6. Address all challenges in setting program completion requirements and taking
steps towards program improvement.
Program
The following program is recommended in order to fulfill the learning goals listed above.
First, an initial training event would be conducted. The program and training event are designed
to address the knowledge, motivation, and organizational needs of the primary stakeholders,
while drawing in support from other stakeholders. Each component of the program aligns with
the critical behaviors, which is how the program will be deconstructed and presented.
101

Critical Behavior 1: Program Completion Requirements. STEM Program teachers
must begin with program completion requirements. In order to do so, they must revisit the
program vision and goals. In a lengthy paper, the thesis can often become forgotten, overlooked,
or muddled by the details of the writing process. Similarly, the vision and goals for the STEM
Program can become lost among the details. Thus, teachers must constantly revisit the vision for
the program and goals for program students in order to ensure that each assignment, event, and
activity aligns with its original intent. Then, teachers should list the previous and current
requirements and activities in detail. Each item on this list should be cross-referenced with the
goals and vision of the program to ensure that they fulfill the goals and carry out the vision. Any
requirements that do not satisfy these conditions should be removed or adjusted in order to
ensure that they do contribute to program goals.
After refining the list of past and current requirements, program teachers should review
the vision and goals again to see if any were unaddressed. If they were, then teachers must create
new requirements that do address the goals. Teachers should look to other programs on campus
as models and references for developing activities and requirements that align with program
goals.
Program teachers should then review all requirements, answering the following questions
for each requirement:
1. Is the requirement sustainable? Can it be applied to each year of the program,
with only minimal changes, if any?
2. Do the current policies and procedures support the implementation of the
requirement?
102

3. Are the resources to carry out the requirement easily available? Is the requirement
equitable for all students.
Answers to all of these questions should be, “Yes,” for each program requirement. Any
requirement that does not follow suit must again be revised.
Following the review and refinement of all program requirements, teachers must ensure
that all stakeholders are on the same page. Interview responses indicated that teachers were not
in agreement regarding program requirements. Thus, all program teachers must know each of the
requirements.
Upon agreement on the program requirements, must determine how they will consistently
monitor and track student progress in completing said requirements. The results and findings
presented in Chapter Four indicate that tracking student progress is an asset; teachers were all in
agreement in regards to monitoring student progress via the district-wide learning management
system. Therefore, teachers should devise a system in which each requirement is explicitly stated
and easily tracked through the learning management system.
Critical Behavior 2: Culturally Responsive Pedagogy. Teachers must begin by
learning what culturally responsive pedagogy is. From there, they should learn why culturally
responsive pedagogy is important in the context of their program goals. Experts in the topic
should be brought in to run a professional learning plan in which program teachers learn the
importance of culturally relevant pedagogy and how to actively and consistently implement it in
their classrooms daily. After learning the “why” behind this pedagogical strategy, teachers must
learn how to engage in it. Experts can provide tips, resources, and other scaffolds while modeling
culturally responsive pedagogy.
103

Following the modeling of culturally relevant pedagogical strategies, program teachers
need time to integrate them into the curriculum while being monitored for support. Teachers
should begin by integrating culturally relevant pedagogy into one lesson in each unit. They must
implement the improved lesson plan with students, and then engage in critical reflection per Jay
and Johnson’s (2002) typology. They must also receive feedback from the experts in culturally
responsive pedagogy and then have the opportunity and scaffolds to adjust and improve their
instruction. In future training events, this process must be repeated, gradually increasing in the
number of lesson plans per unit as teachers become more comfortable with culturally relevant
teaching, until teachers are implementing strategies in their classes daily.
Critical Behavior 3: Steps to Program Improvement. By following the parts of this
improvement program that address Critical Behavior 1 and 2, teachers will already be engaging
in the improvement protocol for Critical Behavior 3. Following this training event, or in
successive learning events, teachers should again have the opportunity to critically reflect on
their progress and actions. They should also collect feedback from other stakeholders - students
and staff. Based on the feedback provided, teachers can determine the next steps for program
improvement or revisit previous ones to improve even more during future training events.
Evaluation of the Components of Learning
After engaging in the proposed program for improvement, the components of learning
must be evaluated. Table 17 lists each method or activity for evaluation and its timing.

104

Table 17
Evaluation of the Components of Learning for the Program
Method(s) or Activity(ies) Timing
Declarative Knowledge “I know it.”
Pre- and post-knowledge check in the form of
verbal question and answer
Before, during, and after
Update program web page and learning
management system to reflect changes
During and after
Procedural Skills “I can do it right now.”
Task completion checklists Before, during, and after
Attitude “I believe this is worthwhile.”
Likert survey questions Before, during, and after
Discussion about value and rationale During
Discussion of any issues During
Confidence “I think I can do it on the job.”
Likert survey questions Before, during, and after
Discussion of supports and obstacles During
Mentorship with experts on culturally
responsive pedagogy and role models from
other programs
During
Commitment “I will do it on the job.”
Discussion of issues and praises During
Self-progress reports During
Individual performance plan Before, during, and after
105

Level 1: Reaction
Kirkpatrick and Kirkpatrick (2016) describe Level 1, reaction, as the extent to which
participants react positively towards training programs. More specifically, Level 1 measures
engagement, relevance, and customer satisfaction. For LHS’s STEM Program, Level 1 evaluates
whether or not, and to what degree, program teachers find the improvement training to be
favorable. Table 18 lists the methods and tools for measuring each component of teachers’
reactions, along with the timing of each.

Table 18
Components to Measure Reactions to the Program
Method(s) or Tool(s) Timing
Engagement
Likert survey questions After
Task completion checklists Before, during, and after
Asking meaningful questions During
Relevance
Likert survey questions After
Follow-up connecting discussions After
Customer Satisfaction
Anonymous Likert survey questions After
Dedicated observer who gauges teacher
commentary and body language
During

106

Evaluation Tools
Kirkpatrick and Kirkpatrick (2016) dual evaluation of training effectiveness. The first
evaluation immediately follows the training in order to evaluate Levels 1 and 2. The second
evaluation is delayed for a period following the training program in order to evaluate all four
levels.
Immediately Following Program Implementation
The immediate evaluation tool to measure Levels 1 and 2 of the training program will be
conducted via Google Forms. Survey questions for Level 1 measure the engagement, relevance,
and customer satisfaction of the training, and questions for Level 2 evaluate teachers’ declarative
knowledge, procedural skills, attitude, confidence, and commitment (see Appendix B).
Delayed for a Period After Program Implementation
The delayed evaluation tool measures all four levels. Delaying this second evaluation
allows for teachers to further reflect on the learning that occurred during the training program.
Google Forms will again be used to conduct this evaluation, with questions such as Likert Scale
items (see Appendix C).
Data Analysis and Reporting
Data from both evaluation tools will be analyzed and presented to program teachers and
site administrators. A visual speedometer dashboard will be used to help teachers visualize and
monitor their Level 4 internal outcomes. Figure 2 shows a sample of what the dashboard might
look like.

107

Figure 2
Sample Dashboard for Level 4 Internal Outcomes

The data from the dashboard and evaluation tools will be used to evaluate the
effectiveness of the training program and inform the next steps towards program improvement.
STEM Program teachers will again have the opportunity to critically reflect on data, this time
along with the administrators, in order to determine how they should proceed in moving forward
towards engaging in the critical behaviors required to achieve their goal of 100% program
completion. Self-reflection and collaboration will encourage program stakeholders to critically
think and hold one another accountable for the growth of the program.
Summary of the Implementation and Evaluation
The New World Kirkpatrick Model (Kirkpatrick & Kirkpatrick, 2016) was used to plan,
implement, and evaluate the training program created to support STEM Program teachers in
participating in the critical behaviors required to achieve their goal of 100% program
improvement. The training program is designed to bridge the gap in knowledge, motivation, and
organizational influences affecting teachers in engaging in the critical behaviors. Per Kirkpatrick
and Kirkpatrick’s (2016) model, the program addresses each of the four levels of training
evaluation. Program teachers should participate in the training in order to take steps towards
improving the program and achieving program goals. Furthermore, evaluating the training
108

program will also allow teachers to measure its effectiveness and determine how to proceed in
working towards achieving program goals. Following the implementation and evaluation of the
training, teachers will see its contributions to program growth.
Limitations and Delimitations
A limitation of this study is that only four of the seven STEM Program teachers
participated in interviews, and only two of the four participants submitted lesson plans. Thus, the
analysis and recommendations presented are in response to the limited data collected.
Additionally, the strategies and solutions presented are unique to the STEM Program at LHS.
These recommendations may not translate to STEM Programs at other schools. Another
limitation is researcher bias. As a science teacher studying a science-related program, the
primary researcher’s biases may have affected the analysis and recommendations. The final
limitation of the study is that it does not address how the COVID-19 pandemic might affect the
STEM Program. Program requirements and activities, as well as school policies and procedures,
were impacted COVID-19, which influences teachers’ roles in the program.
The delimitation of this study is that the stakeholder of focus is STEM Program teachers.
The decision to focus on STEM Program teachers was based on their control over program
design and decision to volunteer to teach in the program. They are uniquely motivated to take
steps towards program success.
Recommendations for Future Research
Future research on STEM and other college and career readiness programs should focus
on teacher knowledge and skills related to college and career readiness. Such programs typically
focus on developing students’ college and career skills, while overlooking the fact that teachers
themselves may also need support with the skills.
109

Strategies for providing equity is also recommended for future research. Existing research
focuses on supporting students or improving programs as a whole. Further research should be
conducted on universal design for learning and individualized strategies within STEM and other
CCR programs.
Finally, future research on the impact of COVID-19 on STEM and CCR programs would
be beneficial. Since the impact of the global pandemic is new and widely unknown, research
should be conducted into exploring its effects on academic and career programs of different
levels of education.
Conclusion
Such that the ultimate goal of education is to prepare students for their future, LHS aims
to foster students’ college and career readiness. STEM Program teachers set out to attain this
goal through the lens of science, technology, engineering, and math; however, there is room for
improvement in their efforts to improve the program and attain 100% program completion. This
study analyzes how program teachers can engage in critical behaviors necessary for achieving
their goal. It focuses on the knowledge and skills, motivational, and organizational gaps
preventing the key stakeholders, STEM Program teachers, from performing the critical
behaviors. This study then proposes a program solution that addresses the gaps in knowledge,
motivation, and organization through appropriately scaffolded teacher supports. The
effectiveness of this program solution must then be evaluated in order to inform the next steps in
program improvement.
This dissertation serves as a case study gap analysis on a STEM college and career
readiness program. It furthers the research in the fields of STEM programs and CCR programs
and can be used to improve STEM CCR program performance.
110

References
ACT (2020). National Norms for ACT Test Scores Reported During the 2019-2020 Reporting
Year. ACT, Inc. Retrieved from http://www.act.org
Anderman, E. & Anderman, L. (2006). Attributions. Retrieved from http://education.com.
Anderson, L., & Krathwohl, D. (2001). A taxonomy for learning, teaching, and assessing (1st
ed.). Longman.
Anonymous (2018). Local control accountability plan and annual update (LCAP) template. CA
Department of Education.
Bass, K., & Panahandeh, S. (2016). Designing the Game: How a Project-Based Media
Production Program Approaches STEAM Career Readiness for Underrepresented Young
Adults. Journal of Science Education and Technology, 25(6), 1009–1024.
https://doi.org/10.1007/s10956-016-9631-7
Bragg, D., Taylor, J., Tierney, W., & Sablan, J. (2014). Toward college and career readiness:
How different models produce similar short-term outcomes. American Behavioral
Scientist, 58(8), 994–1017. Retrieved from https://doi.org/10.1177/0002764213515231.
Brand, B (2009). High school career academies: A 40-year proven model for improving college
and career readiness. American Youth Policy Forum.
Brasch, D (2018). College and career readiness through independent study: An innovation study.
University of Southern California Libraries.
Bruce-Davis, M., Gubbins, E., Gilson, C., Villanueva, M., Foreman, J., & Rubenstein, L. (2014).
STEM high school administrators’, teachers’, and students’ perceptions of curricular and
instructional strategies and practices. Journal of Advanced Academics, 25(3), 272–306.
https://doi.org/10.1177/1932202X14527952
111

Castellano, M., Ewart Sundell, K., & Richardson, G. (2017). Achievement Outcomes Among
High School Graduates in College and Career Readiness Programs of Study. Peabody
Journal of Education, 92(2), 254–274. https://doi.org/10.1080/0161956X.2017.1302220
Cheeks, I. & McDowelle, J. (2018). The Creation of the Armadillo Career Readiness Skills Gap
Program at Randolph Community College: Closing the Career Gap for High School
Juniors. January 2018. http://search.proquest.com/docview/2239323268/.
Clark, R. E., & Estes, F. (2008). Turning research into results: A guide to selecting the right
performance solutions. CEP Press.
College Board (2019). SAT: Understanding scores. The College Board. Retrieved from
https://collegereadiness.collegeboard.org/sat/scores/understanding-scores/interpreting.
Conley, D. (2012). A complete definition of college and career readiness. Educational Policy
Improvement Center.
Conley, D. (2014). Getting ready for college, careers, and the common core: what every
educator needs to know. 1. San Francisco: Jossey-Bass.
Conley, D.T., & Darling-Hammond, L. (2013). Creating systems of assessment for deeper
learning. Stanford, CA: Stanford Center for Opportunity Policy in Education.
Creech, K., & Clouse, P. (2013). Outcomes of a Partnership for College and Career Readiness
and a Senior English Transition Course. NASSP Bulletin, 97(4), 314–334.
https://doi.org/10.1177/0192636513504451.
Creech, K., & Clouse, P. (2013). Outcomes of a Partnership for College and Career Readiness
and a Senior English Transition Course. NASSP Bulletin, 97(4), 314–334.
https://doi.org/10.1177/0192636513504451.
112

Creswell, J. W. & Creswell, J. D. (2018). Research design: Qualitative, quantitative, and mixed
methods approaches. Sage Publications.
Day, S., Smith, P.J., Keiser, K., Ostler, C., & Surface, J. (2012). Comparison of college/career
readiness outcomes between the Advancement Via Individual Determination (AVID)
program and the traditional high school academic program. Retrieved from
http://search.proquest.com/docview/1266847618/.
DiBenedetto, C. A. (2015). AGSTEM interdisciplinary collaboration: Building bridges from
subject to subject to enhance college and career readiness. The Agricultural Education
Magazine, 87(5), 5-7. Retrieved from http://libproxy.usc.edu/login?url=https://search-
proquest-com.libproxy1.usc.edu/docview/1766243316?accountid=14749
Eccles, J. (2006). Expectancy value motivational theory. Retrieved from http://education.com.
Ejiwale, J. A. (2013). Barriers to successful implementation of STEM education. Journal of
Education and Learning 7(2).
Fakayode, S.O., Jennings Davis, J., Yu, L., Meikle, P.A., Darbeau, R., & Hale, G. (2019).
Transforming STEM departments for inclusion: Creative innovation, challenges,
adaptation, and sustainability at the University of Arkansas-Fort Smith. Diversity in
Higher Education, 22, 73-105.
Fayer, S., Lacey, A., & Watson, A. (2017). STEM occupations: Past, present, and future. U.S.
Bureau of Labor Statistics.
Fitzgerald, K. (2018). The value of career and technical education in addressing college and
career readiness and the ill-prepared workforce: A study of the perceptions of community
college administrators and instructors; district high school administrators, and guidance
113

counselors, and CTE department; and a workforce employer in North Carolina. ProQuest
LLC.
Fletcher Claville, M.O., Babu, S., Parker, B.C., Hill, E.V., Claville, E.W., & Penn-Marshall, M.
(2019). Nanohu: A successful collaborative STEM model preparing African Americans
for engagement in nanoscience, laying the foundation for transformative, institutional
STEAM engagement. Diversity in Higher Education, 22, 107-128.
Foroozesh, M., Giguette, M., Birdwhistell, T., Morgan, K., Johanson, K., Coston, T.S., &
Wilkins-Green, C. (2019). All for one and one for all: Coordinating the resources of
individual student research training initiatives in biomedical sciences at Xavier
University of Louisiana. Diversity in Higher Education, 22, 129-149.
Furuto, L. (2015). STEM Education on a Worldwide Voyage: Curriculum to College, Career,
and Community Readiness. International Journal for Mathematics Teaching and
Learning, 1–20. Retrieved from http://search.proquest.com/docview/1697489746/
Gallimore, R., & Goldenberg, C. (2001). Analyzing cultural models and settings to connect
minority achievement and school improvement research. Educational Psychologist, 36,
45-56.
Helguera, M. (2018). The allocation of resources for career technical education programs that
improve college- and career-readiness. University of Southern California Libraries.
Hooker, S., Brand, B., Legters, N., & Balfanz, R. (2010). College knowledge: A critical
component of college and career readiness. New Directions for Youth Development,
2010(127), 75–85. https://doi.org/10.1002/yd.364
Immordino-Yang, M.H. (2011). Implications of affective and social neuroscience for educational
theory. Educational Philosophy and Theory, 43(1).
114

Irwin, D., Logan-King, L., Lewis, T., & Ackroyd, D. (2013). Collaborations to improve college
and career readiness through trans-disciplinary learning. 2013 IEEE Integrated STEM
Education Conference (ISEC), 1–2. https://doi.org/10.1109/ISECon.2013.6525210
Kanipes M.I., Tang, G., Spencer-Maor, F.E., Wilson-Kennedy, Z.S., & Byrd, G.S. (2019).
Advancing STEM by transforming pedagogy and institutional teaching and learning: The
creation of a STEM center of excellence for active learning. Diversity in Higher
Education, 22, 55-72.
Kirkpatrick, J. & Kirkpatrick, W. (2015). An introduction to the new world Kirkpatrick model.
Kirkpatrick Partners, LLC.
Kittur, H., Shaw, L., & Herrera, W. (2017). A New Model for a Multi-Disciplinary Engineering
Summer Research Program for High School Seniors: Program Overview, Effectiveness,
and Outcomes. Journal of STEM Education : Innovations and Research, 18(4), 25–31.
Retrieved from http://search.proquest.com/docview/1991088308/
Krathwohl, D.R. (2002). A revision of Bloom’s taxonomy: An overview. Theory into Practice,
41(4).
Lane, T., Morgan, K., & Lopez, M. (2020). “A Bridge Between High School and College”: A
Case Study of a STEM Intervention Program Enhancing College Readiness Among
Underserved Students. Journal of College Student Retention: Research, Theory &
Practice, 22(1), 155–179. https://doi.org/10.1177/1521025117729824
Mayer, R. E. (2011). Applying the science of learning. Boston, MA: Pearson Education.
Merriam, S. B., & Tisdell, E. J. (2016). Qualitative research: A guide to design and
implementation (4th ed.). Jossey-Bass.
115

Nishimura, J. (2014). Assessing college and career readiness through the senior project program.
University of Southern California Libraries.
Pajares, F. (2006). Self-efficacy theory. Retrieved from http://education.com.
Perry, J., Wallace, E., Hynes, K., & Hirsch, B. (2012). What schools are doing around career
development: Implications for policy and practice. New Directions for Youth
Development, 2012(134), 33–44. https://doi.org/10.1002/yd.20013
Peters, A.W., Tisdale, V.A., & Swinton, D.J. (2019). High-impact educational practices that
promote student achievement in STEM. Diversity in Higher Education, 22, 183-196
Pintrich, P. R. (2003). A motivational science perspective on the role of student motivation in
learning and teaching contexts. Journal of Educational Psychology, 95, 667-686.
Rueda, R. (2011). The 3 dimensions of improving student performance. Teachers College Press.
Scott, S. & Palinscar, A. (2006). Sociocultural theory. Retrieved from http://education.com.
Shively, J., & Weiss, A. (2017). On the path toward college and career readiness: An evaluation
of the implementation and outcomes of the Career Academy Program in Birmingham,
Alabama. Vanderbilt University. Peabody College.
Smith, M.A., White, A.M., Bernot, K.M., Petty, C.L., Dinitra White, C., Byfield, G.E., Newman,
R.H., Coomans, R.J., & Rorie, C.J. (2019). A cultural shift: A transformative approach to
advising STEM students at an HBCU. Diversity in Higher Education, 22, 291-316.
Thomas, S., & Tatum, S. (2016). The examination of non-cognitive factors, college readiness,
career readiness, student engagement, natural connected nodes, usefulness of college
retention practices, college attrition level, and intention to stay in college. ProQuest
Dissertations Publishing. Retrieved from
http://search.proquest.com/docview/1865451639/.
116

Walker, R., Hillesheim, C., & Coley, A. (2015). Connecting secondary education to the job
market in Mississippi: establishing true measures of college and career readiness.
(Report). Journal of the Mississippi Academy of Sciences, 60(4).
Williams, T. M., Crawford, M. B., Hooper-Bui, L. M., Givens, S., Lavender, H., Watt, S., &
Warner, I.M. (2019). LSU office of strategic initiatives: A great equalizer for broadening
participation in STEM. Diversity in Higher Education, 22. Emerald Publishing United.
Yavuz, O., Parzych, J., & Generali, M. (2019). A Systematic Approach to Exploring College and
Career Readiness Program Needs Within High-Poverty Urban Public Schools. Education
and Urban Society, 51(4), 443–473. https://doi.org/10.1177/0013124517727054.
Yough, M., & Anderman, E. (2006). Goal orientation theory. Retrieved from
http://education.com.
117

Appendix A: Interview Protocol
I. Introduction

Hello, my name is Dr. Wierzchucki, I am a researcher for USC conducting interviews for a study
of high school STEM programs, particularly the ASE Program. We are particularly interested in
the knowledge, motivation, and organizational factors that influence teachers in taking steps
towards program improvement.

Thank you for agreeing to participate today. We appreciate the time that you have set aside for
the interview. As mentioned in the consent form, the interview should take about 30 minutes.
Does that still work for you?

I want to assure you that I am strictly wearing the hat of the researcher today, so my questions
and observations are not evaluative. I am not judging your responses or your performance as a
teacher. Additionally, this interview is confidential. What that means is that your name and
anything you say will not be shared with anyone outside of the research team. I will not share
your responses with other teachers, the principal, or the district. In the study, you will remain
anonymous and any identifiable information will be de-identified. Also, you can stop the
interview at any time.

Our Zoom conference will be recorded to ensure that I do not overlook any details. The audio of
the recording will be turned into a transcript and de-identified. Then, the audio and visual of the
recording will be deleted. Is this okay with you?

I will also be taking notes to help me in the event that I would like to ask follow up questions. Do
you have any questions or concerns before we begin?

I will now begin the recording.

II. Setting the Stage

I’d like to start by asking some background questions.

1. First, could you tell me about your background in education?
a. How did you become interested in the field of education?
b. How long have you worked in the field?
c. What is your content background?
d. What kind of teacher preparation did you experience?
2. How did you become interested in becoming an ASE Program teacher?
a. How long have you taught in the [STEM Program]?
b. Which [STEM Program] course(s) do you teach?

III. Heart of the Interview

118

Now I’d like to ask you some questions about what you know about program requirements.

3. What requirements must students fulfill in order to complete the program?
4. What are the three categories of tasks that students must carry out in order to complete
the program?
5. How do you identify and manage students’ progress on program completion
requirements?
6. Tell me how you reflect upon student progress in completing the program.

These next questions ask about what you know about improving the program.

7. What do teachers need to do to help improve the program?
8. What steps do you take for program improvement?
9. How do you incorporate culturally relevant pedagogy in your instruction?
10. Tell me how you reflect upon progress towards program improvement.

Let’s shift gears into motivational factors associated with program completion requirements.

11. How important is it for teachers to conduct meetings to discuss program completion
requirements?
12. How do you feel about conducting biweekly meetings to discuss program completion
requirements?
13. To what extent is conducting biweekly meetings in your own control?
14. What are the challenges in setting program completion requirements?
a. How have you addressed these challenges?

Now let’s talk about motivational factors associated with program improvement.

15. How confident are you in taking steps towards program improvement right now?
16. To what extent is taking steps towards program improvement in your own control?
17. Tell me about the importance you place on providing culturally relevant pedagogy in
your instruction.
18. What are the challenges in taking steps towards program improvement?
a. How have you addressed these challenges?

I’d like to ask about organizational influences on program completion requirements.

19. What sorts of resources or references do you have to implement ASE-specific
pedagogical strategies in class to support students in program completion?
a. What sorts of resources do you need to implement ASE-specific pedagogical
strategies in class to support students in program completion?
20. Do you have role models or other successful programs to review in conducting regular
meetings?
21. How much time do you have for conducting meetings to discuss program completion
requirements?

119

Lastly, I would like to talk about organizational factors that influence program improvement.

22. Do you feel like the faculty (which includes admin & and other faculty) value taking
steps toward program improvement?
23. Do you have role models or other successful programs to review in evaluating steps taken
towards program improvement?
24. What policies affect your ability to take steps towards program improvement?
a. To what extent do the policies support or deter your ability to take steps towards
program improvement?
25. How much time do you have to take steps towards program improvement?
26. What sorts of resources do you have to take steps towards program improvement?
a. What sorts of resources do you need to take steps towards program improvement?

IV. Closing Comments

Thank you for taking the time to share your thoughts! I will now end the recording.

Everything that you have shared is really helpful for the study. If I have any follow-up questions,
could I contact you? Is email okay? Again, thank you for participating in the study!

120

Appendix B: Immediate Evaluation Tool (Levels 1 and 2)
Please use the following rating scale to circle the number that best correlates with how you
feel about each statement.

1 = Strongly Disagree
2 = Disagree
3 = Neutral
4 = Agree
5 = Strongly Agree
The seminar held my interest.
(L1 - Engagement)
1 2 3 4 5
The information in the seminar is applicable
to my STEM Program goals.
(L1 - Relevance)
1 2 3 4 5
I was satisfied with the seminar.
(L1 - Customer Satisfaction)
1 2 3 4 5
What I learned in the seminar will help me
with program goals.
(L1 - Attitude)
1 2 3 4 5
I am confident I can apply what I learned to
achieving program goals.
(L1 - Self Efficacy)
1 2 3 4 5
I feel positive about achieving program goals.
(L1 - Emotion)
1 2 3 4 5

121

Please answer the questions to the best of your ability regarding the information presented
in the seminar.
1. What are the STEM Program Requirements? (L2)
2. What steps must be taken to improve the STEM Program. (L2)
3. How is student progress managed? (L2)
4. How is culturally responsive pedagogy integrated in your curriculum? (L2)
122

Appendix C: Delayed Evaluation Tool (Levels 1, 2, 3, and 4)
Please use the following rating scale to circle the number that best correlates with how you
feel about each statement regarding the A-G registration seminar two months ago.

1 = Strongly Disagree
2 = Disagree
3 = Neutral
4 = Agree
5 = Strongly Agree
I applied what I learned at the seminar to list
and describe STEM Program completion
requirements.
(L1, L2, L3)
1 2 3 4 5
Looking back, the seminar was a good use of
my time.
(L1)
1 2 3 4 5
I am still confident in what I learned in the
seminar with goals for the STEM Program.
(L1, L2)
1 2 3 4 5
I still feel positive about what I learned in the
seminar.
(L1)
1 2 3 4 5
The seminar had a positive impact on my
ability to achieve program goals.
(L1, L2, L4)
1 2 3 4 5

123

Please answer the questions to the best of your ability regarding the information presented
in the seminar.
1. What are the STEM Program Requirements? (L2)
2. What steps must be taken to improve the STEM Program. (L2)
3. How is student progress managed? (L2)
4. How is culturally responsive pedagogy integrated in your curriculum? (L2)

124

Appendix D: Exempt Research Information Sheet
INFORMATION SHEET FOR EXEMPT RESEARCH

STUDY TITLE: College and Career Readiness Through High School STEM Programs: An
Evaluation Study

PRINCIPAL INVESTIGATOR: Praisy Poluan

FACULTY ADVISOR: Kenneth Yates, EdD

You are invited to participate in a research study. Your participation is voluntary. This document
explains information about this study. You should ask questions about anything that is unclear to
you.

PURPOSE
The purpose of this study is to evaluate the degree to which the ASE Program is achieving its
goal of 100% program completion. We hope to learn the knowledge, motivation, and
organizational influences related to achieving said goal in an effort to make changes to increase
the program completion rate. You are invited as a possible participant because you teach in the
program and can provide unique insights into the program.

PARTICIPANT INVOLVEMENT
It is your choice whether or not to participate. If you choose to participate, you may change your
mind and leave the study at any time.

If you decide to take part, you will be asked to:
1. Participate in an interview with a data collector not affiliated with Edison High School or
the ASE program. The interview will be conducted individually after school on Zoom
and will last approximately 30-45 minutes.
2. Share your Canvas page as a part of a “virtual classroom walkthrough.”
3. Share lesson plans.

CONFIDENTIALITY
The members of the research team and the University of Southern California Institutional
Review Board (IRB) may access the data. The IRB reviews and monitors research studies to
protect the rights and welfare of research subjects.

We will keep your records for this study confidential as far as permitted by law. However, if we
are required to do so by law, we will disclose confidential information about you. Efforts will be
made to limit the use and disclosure of your personal information, including research study and
medical records, to people who are required to review this information. We may publish the
information from this study in journals or present it at meetings. If we do, we will not use your
name.

125

When the results of the research are published or discussed in conferences, no identifiable
information will be used. The interview will be recorded and transcribed. Following the
transcription, the recording will be deleted or erased. Your identity will be removed from the
transcription, so no one in connection with the study will know who you are. Information from
your Canvas page and lesson plans will also be deidentified. Your data will be stored on a
password protected computer with access being limited to the principal investigator and will be
stored for a period of time per institutional policy.

Possible risks and discomforts you could experience during this study include:
1. The risk of a breach of confidentiality. There is a small risk that people who are not
connected with this study will learn your identity or your personal information
2. The risk that you might experience some discomfort about student progress toward
achieving program completion.
3. Some of the questions may make you feel uneasy. You can choose to skip or stop
answering any questions you don’t want to.
4. There may be other risks that are not known at this time.

The investigators are required to report certain cases with the potential of serious harm to you, or
others, such as suicidality or child abuse to the appropriate authorities.

INVESTIGATOR CONTACT INFORMATION
If you have any questions about this study, please contact Ms. Praisy Poluan by either email at
ppoluan@usc.edu or call (714) 392-5561 or Kenneth Yates, EdD at kenneth.yates@usc.edu.

IRB CONTACT INFORMATION
If you have any questions about your rights as a research participant, please contact the
University of Southern California Institutional Review Board at (323) 442-0114 or email
irb@usc.edu.

126

Appendix E: Recruitment Letter

Dear Teachers,

On behalf of the University of Southern California Rossier School of Education, I would like to
invite you, as a teacher in the ASE Program at Edison High School, to participate in a research
study. You are being asked to be interviewed as a part of a Praisy Poluan’s doctoral dissertation
under my supervision.

Our goal is to explore the knowledge, motivation, and organizational influences related to
students completing the program.

Participation is voluntary. All personal information from the interview will be deidentified. The
interview is a one-time session lasting approximately 30-45 minutes. The interview will be
conducted by Bryan Wierzchucki, another doctoral student not affiliated with Edison High
School, via Zoom video conferencing.

For more information, you may contact Ms. Praisy Poluan by either email at ppoluan@usc.edu
or call (714) 392-5561 or me, Dr. Kenneth Yates, at kenneth.yates@usc.edu.

If you would like to participate, please email Bryan at wierzchu@usc.edu and he will set up a
time to conduct the interview. Thank you!

Sincerely,

Dr. Kenneth Yates
Professor of Clinical Education
University of Southern California
Rossier School of Education

Abstract (if available)

Abstract This study analyzes college and career readiness through high school STEM programs. The purpose of this case study is to evaluate the degree to which a STEM-focused college and career readiness program is achieving its goal of 100% program completion. Clark and Estes’ (2008) gap analysis framework is adapted to identify the knowledge, motivation, and organizational influences related to achieving said goal, with the program teachers as the stakeholder of focus. Qualitative methods, specifically interviews and document analysis, were used to identify and assess the influences. Interview data were triangulated with web pages and lesson plans to identify and confirm whether each knowledge, motivation, and organizational influence is an asset or a need. These findings inform the solutions recommended to bridge the gaps in influences. Then, the New World Kirkpatrick Model (Kirkpatrick & Kirkpatrick, 2016) is used to create a plan for implementation and evaluation of the recommended solutions. This study outlines the expected outcomes of the training program presented as part of the recommended solutions. Lastly, this study provides recommendations for future research regarding college and career readiness, STEM Programs, teacher training, and the impact of COVID-19 on high school STEM Programs.

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

College and career readiness through independent study: an innovation study

PDF

Calibrating the college preparation bridgeway for independent study program students at Amazing Independent High School District: a multisite evaluation study through the teachers' lens

PDF

One to one tablet integration in the mathematics classroom: an evaluation study of an international school in China

PDF

A-G course completion percentage upon high school graduation. A gap analysis

PDF

A case study in student retention at a Northern California private Jewish day school: a gap analysis

PDF

Using mastery learning to address gender inequities in the self-efficacy of high school students in math-intensive STEM subjects: an evaluation study

PDF

Creating the conditions for change readiness in higher education: an innovation study

PDF

Challenging stigmas and perceptions in alternative high schools through mentorship: an innovation study

PDF

A knowledge, motivation and organizational gap analysis for integrating the arts with a STEM curriculum

PDF

K-12 college and career readiness guidance: an improvement study

PDF

Perception of alternative education teachers readiness to instruct English language learners: an evaluation study

PDF

The implementation of a multi-tiered system of support in Downtown Unified School District: an analysis of site administrator needs

PDF

Effects of mentoring on public school administrators: an evaluation study

PDF

Job placement outcomes for graduates of a southwestern university school of business: an evaluation study

PDF

Aligned leadership attributes and organizational innovation: an evaluation study

PDF

STEM teacher education: An evaluation study

PDF

Increasing student persistence at a community college from an administration perspective

PDF

Incorporating service learning curriculum to enhance college and career readiness: a professional development for teachers

PDF

Civic learning program policy compliance by a state department of higher education: an evaluation study

PDF

The allocation of resources for career technical education programs that improve college and career readiness

Asset Metadata

Creator Poluan, Praisy (author)

Core Title College and career readiness through high school STEM programs: an evaluation study

School Rossier School of Education

Degree Doctor of Education

Degree Program Education (Leadership)

Publication Date 05/03/2021

Defense Date 03/15/2021

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag career readiness,college and career readiness,college readiness,gap analysis,High School,OAI-PMH Harvest,STEM programs

Language English

Contributor Electronically uploaded by the author (provenance)

Advisor Yates, Kenneth (committee chair), Freking, Frederick (committee member), Murray, Rocky (committee member)

Creator Email ppoluan@usc.edu,praisypoluan@gmail.com

Permanent Link (DOI) https://doi.org/10.25549/usctheses-c89-459539

Unique identifier UC11668947

Identifier etd-PoluanPrai-9588.pdf (filename),usctheses-c89-459539 (legacy record id)

Legacy Identifier etd-PoluanPrai-9588.pdf

Dmrecord 459539

Document Type Dissertation

Rights Poluan, Praisy

Type texts

Source University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Access Conditions The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the a...

Repository Name University of Southern California Digital Library

Repository Location USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA