Close
About
FAQ
Home
Collections
Login
USC Login
Register
0
Selected
Invert selection
Deselect all
Deselect all
Click here to refresh results
Click here to refresh results
USC
/
Digital Library
/
University of Southern California Dissertations and Theses
/
The mentorship of instructors and its impact on computer science interest among middle school girls: an evaluation study
(USC Thesis Other)
The mentorship of instructors and its impact on computer science interest among middle school girls: an evaluation study
PDF
Download
Share
Open document
Flip pages
Contact Us
Contact Us
Copy asset link
Request this asset
Transcript (if available)
Content
THE MENTORSHIP OF INSTRUCTORS AND ITS IMPACT ON COMPUTER SCIENCE
INTEREST AMONG MIDDLE SCHOOL GIRLS: AN EVALUATION STUDY
by
Jennifer Nicole Casey
A Dissertation Proposal Presented to the
FACULTY OF THE USC ROSSIER SCHOOL OF EDUCATION
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulfillment of the
Requirements for the Degree
DOCTOR OF EDUCATION
May 2020
Copyright 2020 Jennifer Nicole Casey
ii
DEDICATION
To every little girl dreaming of a life as a scientist; fear nothing. Find your inspiration, harness
your determination, and live your life as a scientific adventure.
iii
ACKNOWLEDGEMENTS
As a child who was filled with wonderment and avid curiosity, I struggled in the
traditional classroom setting. I learned what it took to push past obstacles, create goals, and cross
the finish line. One of my favorite stories as a child was The Tortoise and Hare. As a fable
centered around perseverance, this type of mantra has stayed with me throughout my education
and professional experience. I have had the opportunity to be surrounded by family and friends
who were steadfast in their belief of my success, allowing me to sacrifice time and prior
commitments to succeed. I have taken chances and risen to the occasion by the encouragement of
leaving my comfort zone.
This dissertation was written while traveling to four continents and numerous plane rides
crisscrossing the United States. While I was tempted to sequester myself at home and not travel
so much for the sake of focusing on my writing, I consciously chose to make it a part of the
journey. This is what this process is all about: learning about the challenge, finding focus, and
achievement among the chaos. What better way to learn more about ones’ perseverance than to
take risks, leave ones’ comfort zone, make progress, and then offer a heartfelt acknowledgment
to those who gave guidance along the way? This process became part of my voyage in learning
how to complete a task similar to the tortoise; no matter the speed, you will still win. To those
who helped in the creation of this dissertation:
To my husband, Ryan. Through this process over the last three years, you have been my
rock and the best partner I could have ever wished for in this life. Thank you for all of the
endless hours of keeping me sane, cheering me on, and never permitting me to quit. Thank you
for making me laugh, wiping away frustrated tears, and giving me space to take time for myself.
The culmination of this dissertation came right on time. Happy 10th anniversary!
iv
To my parents and sister. Though my mother is no longer on this Earth, her fierce spirit
of perseverance lives within me. You all were the first to lay the foundation of helping me
overcome the struggles of learning, always listening to my fears and cheering me on every step
of the way. You have been an inspiration, through your support and belief of my success, I am
going to walk across the stage and collect my terminal degree!
To my in-laws, thank you for understanding why I wanted to take an additional three
years of my marriage to pursue a degree from a university 2000 miles away from my home.
Thank you for asking me about the program and encouraging me along the way. I am forever
grateful for your support. You have given the gift of Ryan and me to grow personally and
professionally.
To my best friends, you are like family and my true inspiration! I am blessed to have met
each of you; know that you came into my life for a reason. More importantly, I am lucky to have
learned from, traveled the world with, and have your undying support: Anna Love, Angela Dunn,
Jessica Jones, Sarah Guy, Chris Burtner, Brad Landon, Stephanie and Shane Delany, Kelly and
Angel Cheek, Shane Tilley, Melanie Clarkson, Amanda Carmichael, Penny Holloway, Erin
Shattuck, and Marcus Wolf. Each of you has your role in helping me learn more about myself
and my strengths; you have impacted me on both a personal and professional level. Thank you
from the bottom of my heart. I have learned, laughed, and loved so much from each of you.
To my committee: Dr. Tracy Tambascia, Dr. Anthony Maddox, and Dr. Nicole
MacCalla. I could not have asked for a more knowledgeable, patient, and genuine group of
people that have supported me and kept me thinking on my toes to reach the goal of completion.
I could not have asked for a more straightforward committee to help guide this work. Thank you
for your probing questions, your expertise, and your guidance along the way.
v
Finally, to the entire USC Organizational Change and Leadership Cohort Nine. Thank
you all for answering my questions, joining my reading groups, answering my texts and emails at
all hours of the day, and encouraging each other throughout this process. We once met as
students with a bit of imposter syndrome, and we are leaving the program as friends and
professionals who will conquer any challenge that may come our way. Thank you again, and
Fight On!
vi
TABLE OF CONTENTS
Dedication ....................................................................................................................................... ii
Acknowledgements ........................................................................................................................ iii
List of Tables ............................................................................................................................... viii
List of Figures ................................................................................................................................ ix
Abstract ............................................................................................................................................x
Chapter One: Introduction ...............................................................................................................1
Introduction to the Problem of Practice ...............................................................................1
Organizational Context and Mission ...................................................................................3
Organizational Goal .............................................................................................................4
Related Literature.................................................................................................................4
Importance of the Evaluation ...............................................................................................7
Description of Stakeholder Groups ......................................................................................8
Stakeholders Groups’ Performance Goals ...........................................................................8
Stakeholder Group for the Study .........................................................................................9
Purpose of the Project and Questions ..................................................................................9
Methodological Framework ...............................................................................................10
Definition of Terms............................................................................................................10
Organization of the Project ................................................................................................11
Chapter Two: Review of the Literature .........................................................................................13
An Overview of After-School Programs ...........................................................................13
After-School Computer Science Instructional Interventions .............................................17
Characteristics of Successful After-School Programs .......................................................19
Impact of After-School Programs ......................................................................................24
Knowledge, Motivation and Organizational Influences Framework .................................29
Stakeholder Knowledge, Motivation and Organizational Influences ................................30
Motivation ..........................................................................................................................34
Organization .......................................................................................................................37
Conceptual Framework: The Interaction of Stakeholders’ Knowledge and Motivation
and the Organizational Context .........................................................................................42
Conclusion .........................................................................................................................45
Chapter Three: Methodology .........................................................................................................46
Participating Stakeholders .................................................................................................47
Interview Sampling Strategy and Rationale ......................................................................48
Data Collection and Instrumentation .................................................................................49
Survey ................................................................................................................................50
Interviews ...........................................................................................................................50
Data Analysis .....................................................................................................................51
vii
Credibility and Trustworthiness .........................................................................................53
Validity and Reliability ......................................................................................................54
Ethics..................................................................................................................................55
Limitations and Delimitations............................................................................................56
Chapter Four: Findings ..................................................................................................................58
Participating Stakeholders .................................................................................................59
Findings..............................................................................................................................62
Knowledge Findings ..........................................................................................................63
Motivation Findings ...........................................................................................................70
Organizational Findings .....................................................................................................76
Themes ...............................................................................................................................80
Summary ............................................................................................................................83
Chapter Five: Discussion ...............................................................................................................85
Organizational Context and Mission .................................................................................85
Purpose of the Project and Questions ................................................................................87
Recommendations for Practice ..........................................................................................87
Implementation and Evaluation Framework ......................................................................93
Implementation and Evaluation Plan .................................................................................94
Strengths and Weaknesses of the Approach ....................................................................106
Limitations and Delimitations..........................................................................................106
Future Research ...............................................................................................................107
Conclusion .......................................................................................................................108
References ....................................................................................................................................109
Appendices ...................................................................................................................................124
Appendix A: Survey Protocol ..........................................................................................124
Appendix B: Interview Protocol ......................................................................................129
Appendix C: Statement of Explanation About the Study to Accompany Email With
Survey ..............................................................................................................................132
Appendix D: Information Sheet/Facts for Exempt Non-Medical Research ....................133
viii
LIST OF TABLES
Table 1 Organizational Mission, Global Goal and Stakeholder Performance Goals.......................8
Table 2 Knowledge Influences, Types, and Assessments for Knowledge Gap Analysis..............33
Table 3 Motivational Influences and Assessments for Motivation Gap Analysis .........................37
Table 4 Organizational Influences .................................................................................................41
Table 5 Sampling Strategy and Timeline.......................................................................................49
Table 7 Knowledge Influences ......................................................................................................64
Table 9 Organizational Influences .................................................................................................77
Table 10 Outcomes, Metrics, and Methods for External and Internal Outcomes..........................95
Table 11 Critical Behaviors, Metrics, Methods, and Timing for Evaluation ................................97
Table 12 Required Drivers to Support Critical Behaviors .............................................................99
Table 13 Evaluation of the Components of Learning for the Program........................................102
Table 14 Components to Measure Reactions to the Program ......................................................103
ix
LIST OF FIGURES
Figure 1. Conceptual framework: Interaction of stakeholder knowledge and motivation within
organizational cultural models and settings. ..................................................................................43
Figure 3: Total number of workshops facilitated since January 2018. ..........................................60
Figure 4. Interview participants composition of gender and terminal degree. ..............................62
Figure 5: Ability to employ culturally responsive learning techniques and skills. ........................65
Figure 6: Survey and response: I can define deficit thinking. .......................................................68
Figure 7: Survey and response: Overall, how satisfied are you with facilitating a STEMaven
workshop? ......................................................................................................................................72
Figure 8: Survey and response: How motivated are you to see students succeed? ......................72
Figure 9: Survey and response: I feel confident in my ability to teach middle school girl’s
computer science. ...........................................................................................................................74
Figure 10: Survey and response: I can navigate the aspects of the workshops with little/no
previous experience. ......................................................................................................................75
Figure 11: Survey question and response: The organization provides professional development
that is centered around new teaching strategies. ............................................................................78
Figure 12: Survey question and result: The professional development is beneficial to both
yourself and the organization. ........................................................................................................78
Figure 13. Elements of professional development that supports the growth of effective and
evidence-based teaching skills. ....................................................................................................105
x
ABSTRACT
The purpose of this study was to evaluate the effectiveness of STEMaven mentors
motivating and retaining middle school girls’ interest in computer science viewed through the
lens of knowledge, motivation, and organizational influences. To better understand the
effectiveness of mentors, I conducted a mixed-methods case study involving mentors at a coding
and technology after-school program. Eighteen participants completed a simple survey that
consisted of thirteen questions. Eight survey participants that met additional criteria were
interviewed using open-ended questions lasting approximately 40-60 minutes in length. Findings
indicate that effective mentor instruction involves the development of evidence-based practices
fostered through professional development centered around opportunities of self-awareness,
evaluation methods, and reflection occurring during and directly after the facilitation of a
workshop. In addition, effective instruction involves the ability to teach at any knowledge level,
using hands-on lessons that are creative, relevant, and engaging. Furthermore, effective mentor
instruction described in this study is based on mentors working to increase their effective and
evidence-based teaching skills that will increase interest of computer science in middle school-
aged girls. The findings of the study aid in developing instructional approaches that enhance
instruction in after-school programs of any discipline.
Keywords: Motivation, effective and evidence-based teaching strategies, mentors, middle school
girls, STEM
1
CHAPTER ONE: INTRODUCTION
Introduction to the Problem of Practice
Women have made gains in many professional fields, but they remain underrepresented
in computer science. This trend of inequality raised awareness of gender disparity in computing
occupations and increased interest and retention strategies for young girls. A quick perusal of a
children’s toy store reveals a movement toward increasing girls’ interest in technology fields. As
cognizance changes, along with increased funding and availability, boys are no longer the
majority of students attending informal learning opportunities involving exposure to and
mentoring in technology outside of school hours (McNally, 2012). However, it is not enough to
teach technology during school hours, and it is essential to promote collaboration, cooperation,
and acceptance through effective teaching and mentoring.
Technology, defined by Lan and Young (1996), is always connected with obtaining a
certain result, resolving certain problems, completing certain tasks using particular skills,
employing knowledge, and exploiting assets. It is comprised of both products and techniques.
Even as technology is more accessible and affordable, the U.S. Department of Education (2016)
stated the K-12 school system was not equipped to prepare students, particularly girls, for
collegiate-level work in computer science. Computer science, as of March 2020, is comprised of
eight disciplines: computer engineering, computer forensics, computer software engineering,
electrical engineering, game and interactive media design, information science, mathematics
teacher education, and neuroscience. With only 18% of computer science degrees being earned
by women, it is imperative to create interest among girls and expose them to computers and
technology at an early age (Sassler, Michelmore, & Smith, 2017). Unfortunately, schools have
barriers that limit efforts to interest girls in computer science, such as funding, partnerships,
2
expertise, and instructional interventions (Williams, 2011). Furthermore, Vekiri (2010) found
girls’ self-efficacy and interest in computer science declined without instructional interventions
to directly target stereotypes and misconceptions.
Tan, Barton, Kang, and O’Neill (2013) found a pattern of reoccurring thoughts in the
girls they studied, suggesting the reason they choose not to pursue a science, technology,
engineering, or mathematics (STEM) career was due to the idea that these fields were too
masculine and did not personify soft, feminine characteristics. For this reason, feelings of not
belonging in the STEM classroom or community or of not being smart enough ultimately
disrupted their interest to pursue STEM, not low-test scores (Campbell, 2011). Similarly, Outlay,
Platt, and Conroy (2017) found instructional interventions designed to promote computer science
were strongly related to student attitudes on technology, and those with a goal of fostering
retention of middle school girls had a retention success rate of over 85%. Likewise,
Bystydzienski, Eisenhart, and Bruning (2015) found efforts to increase interest and proficiency
in STEM among high school girls had promising results, as tools to build confidence in computer
science correlate to graduation rates. In addition, workshop participants have strengths in
different areas, which is more aligned to Gardner’s Theory (1983), and different approaches to
teaching would better align or appeal to the students’ multiple intelligences. When lessons can be
created to differentiate to the senses, there is an increase in proficiency and interest. Further
evidence of the need to address this issue is the increasing number of middle school girls who
desire to obtain a career in computer science but ultimately choose a non-STEM career (Hardin
& Longhurst, 2016). The retention of young girls and women’s interest in computer science must
be addressed to preserve a balanced and adequate talent pool to meet the country’s workforce
needs (Liben & Coyle, 2014).
3
Organizational Context and Mission
STEMaven (pseudonym) is a 501(c)(3) nonprofit corporation founded in January 2017 to
offer a variety of workshops for middle school-aged girls to explore technology, coding, and
science. The mission of STEMaven is to inspire middle school girls in East Tennessee to actively
explore the fields of technology, to close the gender gap in the technology profession, and to
foster participants’ future careers. They provided free coding and technology-related workshops
through hands-on workshops. The workshops were facilitated by volunteers that STEMaven
called “instructional mentors.” These volunteers had degrees ranging from undergraduates to
Ph.D. research scientists in the field of STEM. STEMaven hosted 3-hour workshops at either a
local college campus, a local library, or middle school computer lab every other week throughout
the year.
STEMaven has a framework where each workshop is facilitated by mentors who have the
background knowledge of each lesson, and each is developed and taught through evidence-based
practices. Through this effective set of norms, STEMaven has grown in the number of
participants each year, and, in just two years, hosted 500 to 600 participants. Focusing primarily
on the fifth- through eighth-grade girls, participants come from a range of local area schools,
with coding as the preferred workshop.
The programs’ goals are to provide increased opportunities through effective and
evidence-based instruction and mentorship for middle school girls to engage in workshops that
motivate and retain their interest in computer science. The program hosts between 20 and 30
events annually that offer female students unique opportunities to explore their interests in
various areas of technology and computer science. Similar to other successful intervention
programs that have effective teaching and mentoring, documented attendance rate shows that
4
there has been a steady increase in the number of program participants each year. This evidence
demonstrates the program is reaching a large number of middle school girls interested in learning
more about computer science and other STEM fields.
Organizational Goal
STEMaven provides local middle school girls free hands-on coding, otherwise called
computer programming, and technology-related workshops to inspire and encourage them to
pursue careers related to computer science. Along with computer science, there is an emphasis
on computational thinking skills. Computational thinking is normally associated with computer
science when a student is learning how to code. Computational thinking skills, often referred to
as “21st century skills,” assist in effective problem solving (Shute, Sun, & Asbell-Clarke, 2017).
Computational thinking skills include algorithmic thinking, debugging, iteration, and pattern
recognition. By December 2020, mentors at STEMaven will demonstrate improved effective,
and evidence-based instruction proficiencies. The improvement of effective and evidence-based
instruction proficiencies will increase confidence in facilitating workshops and encourage the
interest of computer science and aid in closing the gender gap in the technology profession. The
director outlined the need for mentors to understand effective and evidence-based teaching
strategies that aid in interest in computer science. STEMaven’s goal was assessed through
mentor surveys as well as interviews collected through January 2020.
Related Literature
The broadening gender gap has a negative impact on the future of diversity, creativity,
and innovation within the STEM workforce. First, stereotype threat directly influences middle
school-aged females’ attrition from STEM fields. Secondly, a lack of professional role
confidence is a direct cause of low graduation rates among college-aged women. Lastly, the lack
5
of interventions and female role models ultimately deters young girls from pursuing a STEM
career.
As early as middle school, the gender gap affects females who aspire to achieve a career
in STEM due to low self-efficacy and stereotype threat. A majority of middle and high school
girls are deterred from pursuing a STEM career due to outdated stereotypes still present in
society (Hughes, Nzekwe, & Molyneaux, 2013). From STEM-related posters in the classroom
depicting mostly male scientists to aisles of toys targeted at boys, girls are silently guided to
believe there is little to no room for them in STEM (Tan et al., 2013). Furthermore, Tan et al.
(2013) conducted a study with middle school and first-year college students and identified three
ways in which a sense of belonging can be found within a classroom: earning good grades, being
surrounded by classmates who share the same interests, and sustaining a high level of effort (Tan
et al., 2013). Lastly, stereotyped beliefs of STEM classes as male-dominated can begin as early
as the primary grades (Blažev, Karabegović, Burušić, & Selimbegović, 2017). Blažev et al.
(2017) conducted a study of 883 primary-aged Croatian students and found that, regardless of
prior school achievement, students with stereotype-consistent interests in school subjects tend to
show stronger stereotype endorsement than others. The long-term effect of stereotype threat in
STEM disciplines adds to the layer of difficulty in preserving women in STEM careers. The
added layer of complexity extends to a lack of professional role confidence in college-aged
women.
Lack of professional role confidence among college-aged women is directly linked to
their ability to see themselves as successful STEM professionals. This lack of confidence in
pursuing STEM disciplines affects the already low rate of women earning STEM degrees
(Hardin & Longhurst, 2016). Litzler, Samuelson, and Lorah (2014) discovered, from a large-
6
scale 2008 online survey of over 10,000 undergraduate students, that encouraging professors and
student community, comparison to peers, major desirability, and GPA have a significant, positive
relationship with professional STEM confidence. Furthermore, the study showed family
members, teachers, peers, and supervisors who encourage college-aged women to create a strong
sense of STEM self-confidence are critical for establishing resiliency. The benefit of an active
community of peers and family from an early age through the completion of a STEM degree
decreases stereotype threat.
Women who earned a STEM degree persevered in a habitually male-dominated arena by
believing they belonged and deserved to be there (Zeldin, Britner, & Pajares, 2006). Over the last
three decades, the number of women who earned these degrees has increased while their number
in STEM careers consistently remains about 15% (Ceci, Williams, & Barnett, 2009). Women
who graduate and pursue these careers may still lack professional STEM confidence. Cech,
Rubineau, Silbey, and Seron (2011) argue this lack of professional STEM confidence is the
primary reason women leave these careers. STEM confidence affects the retention of young girls
and college-aged women within these careers, and the gender gap is linked to the decline in
retention rates of this population, driving the need for intervention programs to foster STEM
interest and self-esteem.
Recognizing the need for interventions, many elementary, middle, and high schools
created after-school programs to engage young girls in STEM activities. Studies show programs
aimed at increasing girls’ interest and self-esteem in STEM are successful at motivating them to
pursue STEM careers. Hyllegard, Rambo-Hernandez, and Ogle (2017) conducted a study of 72
middle school girls and found those with the lowest self-esteem in STEM expressed the highest
interest in mathematics and science, followed closely by girls with the highest self-esteem in
7
STEM sharing the same interest. Furthermore, formal semester-long after-school programs may
result in stronger long-term outcomes (Hyllegard et al., 2017). Research consistently finds more
schoolboys than girls are likely to have a positive perception of STEM subjects (Liben & Coyle,
2014), so there is a need to increase middle school girls’ interest in computer science. According
to a study by Shuen et al. (2011) of 319 female students, hands-on semester- and year-long
workshops have a positive, long-lasting effect on STEM self-esteem and interest.
Long-term mentorship from female faculty, as well as female graduate and undergraduate
students, has been proven to inspire young girls to pursue STEM fields and improve their
attitudes toward these subjects (Shuen et al., 2011). In another study by Halim, Soh, and Arsad
(2018), descriptive analysis revealed a high interest in science among both genders, while
inferential investigation indicated a significant difference in interest in mathematics among girls.
Importance of the Evaluation
Providing increased opportunities through effective and evidence-based instruction and
mentorship for middle school girls that motivate and retain an interest in computer science is
vital for a variety of reasons. STEM helps bridge the gender and ethnicity gap in science and
mathematics (Werner & Denning, 2009). It facilitates economic development, thereby enhancing
job creation. Additionally, a growing body of evidence suggests effective after-school programs
can have positive outcomes for girls in middle school (Campbell, 2011). Thus, the organization
needs to increase retention and cultivation rates to continue bridging this gender gap. Evaluating
the organization’s performance and how it provides increased opportunities through effective
and evidence-based instruction and mentorship for middle school girls provided formative data
to assess the organization’s programming decisions.
8
Description of Stakeholder Groups
Three stakeholder groups play a role in the goal of this study. The STEMaven program
director selects the workshops, recruits the mentors, and manages the program’s overall
performance. The mentors help facilitate the learning of the curriculum, revise workshops to
adapt to the students’ learning level and encourage the development of interest in computer
science to ensure progress and growth of attendance rates. The student participants partake in the
workshops designed to encourage and retain an interest in computer science (Table 1).
Stakeholders Groups ’ Performance Goals
Table 1
Organizational Mission, Global Goal and Stakeholder Performance Goals
Organizational Mission
The mission of the STEMaven is to inspire middle school girls in East Tennessee to actively
explore the fields of technology, to close the gender gap in the technology profession, and to
foster participants’ future careers.
Organizational Performance Goal
By December 2020, mentors at STEMaven will demonstrate improved knowledge and use of
effective and evidence-based instruction proficiencies.
STEMaven Program Director STEMaven mentors Participants
By August 2019, the program
director conducted an
analysis of the FY17-FY18
framework used to determine
content for workshops and
strategies that encompassed
effective and evidence-based
strategies.
By November 2019, the
mentors contributed to the
achievement of the
organization’s performance
goal by facilitating coding
and technology-related
workshops that encourage
development of interest in
computer science.
By December 2020, middle
school girl participants will
have increased engagement
in workshops due to use of
effective and evidence-based
teaching strategies used by
the mentors.
9
Stakeholder Group for the Study
While the joint efforts of all stakeholders contributed to the achievement of the overall
organizational goal, it was essential to evaluate why the majority of mentors had a lack of
knowledge in effective and evidence-based teaching skills. Therefore, the stakeholders of focus
for this study were the mentors at STEMaven.
The mentors contributed to the achievement of the organization’s performance goal by
facilitating workshops that increase interest, confidence, and perceptions of computer science.
The program plays a significant role in bridging the gender gap in science, technology,
engineering, and mathematics and improves the students’ problem-solving skills.
Purpose of the Project and Questions
The purpose of this study was to evaluate the effectiveness of STEMaven mentors
necessary to achieve the goal of instruction that motivates and retains middle school girls’
interest in computer science. Employing the Clark and Estes (2008) gap analysis model, the
analysis focused on knowledge, motivation, and organizational influences related to achieving
this organizational goal. While a complete needs analysis would focus on all STEMaven
stakeholders, for practical purposes, the stakeholders in this analysis were all STEMaven
mentors. Four questions guided this study:
1. To what extent is STEMaven contributing to the development of effective and evidence-
based teaching skills in the mentors?
2. What are the STEMaven mentors’ knowledge and motivation related to improving their
effective and evidence-based teaching skills?
10
3. What is the interaction between STEMaven’s organizational culture and context and the
mentors’ knowledge and motivation to improving effective and evidence-based teaching
skills?
4. What are the recommendations for STEMaven’s practice in the areas of knowledge,
motivation, and organizational resources?
Methodological Framework
A mixed-methods approach to this study allowed for the quantitative and qualitative
assessment of survey and interview results. Information gathered provided insight into the
knowledge, motivation, and organizational factors necessary for STEMaven to inspire middle
school girls in East Tennessee to actively explore the possibilities of technology, to empower
their future careers, and to help close the gender gap in the technology profession. Data were
gathered using a screening survey and semi-structured interviews. A survey was used to validate
the assumed influences outlined in the knowledge, motivation, and organizational factors tables,
and in-person interviews explored the assumed knowledge, motivation, and organizational
causes of effective and evidence-based instruction and mentorship. The survey utilized a
combination of Likert scale and multiple choice. Descriptive analysis was conducted upon
completion of the simple survey results, and both the mean and standard deviation were
identified to find the averages in responses. Interviewees were purposefully recruited for
participation after the screening surveys were submitted.
Definition of Terms
The following terms and acronyms are used for this study:
After-school program: Any organized activity youth can participate in outside of the
traditional school day or classroom.
11
Coding: The process of using a programming language to get a computer.
Computer science: The study of computers and computational systems, mostly related to
software and software systems.
Gender gap: The unequal balance of opportunity or status through social, intellectual, or
economic attainment.
Hands-on: relating to, being, or providing direct practical experience in operation or
functioning of something.
Instructional interventions: Specific program or set of steps to aid in the development of
learning.
Mentor: An experienced advisor who shares information about his or her own career
path, as well as guidance, motivation, emotional support, and role modeling.
Retention: The ability to keep or the continuation of holding an interest in the subject.
Self-efficacy: belief in one’s own power to do things successfully.
STEM: Pertaining to the culture and integration of curriculum on science, technology,
engineering, and mathematics.
Stereotype threat: When someone feels themselves to be at risk of conforming to
stereotypes within their social context.
Technology: The application of scientific knowledge for practical purposes, especially in
industry.
Organization of the Project
Five chapters were used to organize this study. This chapter provided the key concepts
and terminology commonly found in a discussion about the gender gap in STEM-related to
computer science. In addition, STEMaven’s mission, goals, stakeholders, and the framework for
12
the project were introduced. Chapter Two provides a review of current literature on after-school
programs. Topics such as the history, impact, and structure of after-school programs were
addressed. Chapter Three details the knowledge, motivation, and organizational elements of
STEMaven that were examined, as well as methodology related to achieving the organizational
goal. In Chapter Four, the data and results were assessed and analyzed. Finally, Chapter Five
provides solutions, based on data and literature, for closing the perceived gaps as well as
recommendations for an implementation and evaluation plan for the solutions.
13
CHAPTER TWO: REVIEW OF THE LITERATURE
This literature review examined the problem of cultivating and retaining middle school
girls’ interest in computer science. The chapter begins with the definition and history of after-
school programs. It discusses general research on the different types of in-school and after-
school computer science instructional interventions available to middle school girls around the
United States. To understand the benefits of after-school computer-based programs in middle
schools, elements of computer science after-school programs need to be defined. The review
presents an in-depth discussion on the characteristics of successful programs in relation to
evaluation, professional development, sense of community, and rigor of instruction. The
relationship between school programs and gender disparity and self-efficacy, school retention
rate, and inequality in opportunities for employment and education are also examined.
Following the general research literature, this chapter provides an overview of the Clark and
Estes’ (2008) knowledge, motivation, and organizational influences framework used in this
study. This section defines the types of knowledge, motivation, and organizational influences to
increase mentors’ ability to meet the organizational goal. The chapter ends with a presentation of
the conceptual framework guiding this study.
An Overview of After-School Programs
After-school programs, sometimes referred to as instructional interventions, have
increased in popularity over the past 15 years (Durlak, Mahoney, Bohnert, & Parente, 2010). The
majority of these programs provide elementary and middle school students an opportunity to
socialize with peers, interact with positive role models, and receive assistance in completing a
task, such as homework or a project. A high-quality program allows students to construct
relationships with peers and mentors in an informal setting, increasing motivation, and
14
connecting with their interests (Nugent, Barker, Grandgenett, & Adamchuk, 2010). In this
context, students benefit academically and forge new social networks while learning the tools
needed to be a STEM professional.
After-school programs are created for a variety of disciplines and structured to support
different ability levels. These programs allow exposure to unique opportunities not typically
found in the classroom, which enriches and foster interest in STEM (Hollister, 2003). Naizer,
Hawthorne, and Henley (2014) developed a theoretical framework for building after-school
programs to retain middle school girls’ interest in computer science. This model permitted for
participants to build self-efficacy through gender-specific workshops offered within a well-
designed after-school program. Other studies have shown positive affective outcomes from
similar interventions. For example, Grolnick, Farkas, Sohmer, Michaels, and Valsiner (2007)
found a supportive and engaging after-school program can cultivate interest, motivation, and
competence in middle school-aged girls. The quality of these workshops depends on the level of
support, the knowledge and motivation of the facilitators, and the capabilities and proficiency of
the coordinator (Lundh, House, Means, & Harris, 2013).
History and Growth of After-School Programs
The growth of after-school programs has provided an avenue for expanded enrichment
and learning opportunities for all ages and genders, with more recent programs designed
specifically for middle school girls (Mouza, Marzocchi, Pan, & Pollock, 2016). This has allowed
for the cultivation of girls’ interest in computer science and the reduction of the gender gap in
computer science. After-school programs primarily began due to historical changes in the labor
force and the formal school setting (Halpern, 2002). During the late nineteenth century, the need
for children in the labor force dropped dramatically (Schuman, 2017). During this time,
15
expectations of students’ academic success increased and were expedited by the creation of
universal, compulsory education (Halpern, 2002). The first type of after-school club was for
males only, called “boy’s clubs,” and took place after school hours. By the early twentieth
century, researchers noted after-school programs could aid in student’s growth and development
(Mahoney, Lord, & Carryl, 2005).
Current computer science-based after-school programs have grown in both depth and
funding over the last 30 years. Before the 1970s, after-school programs were rare, and, often,
students did not have an outlet to boost interests in areas outside of school (Afterschool Alliance,
2017). Most care was provided at home by parents or family members, and, when family
members were not available, children were cared for by neighbors or family friends. At home,
children typically had an outlet in a nurturing setting and an opportunity to work on homework,
play, have snacks and participate in extracurricular activities (Mahoney et al., 2005). With the
change in American economics and dual-income households, supporting at-home after-school
care is increasingly difficult. After-school care provides the support students need in a well-
organized atmosphere (Junge, 2003). Considerable growth of after-school programs is evident by
an increase in media coverage and funding by government and private organizations (Wade-
Jaimes, Cohen, & Calandra, 2019).
In 1994, Congress established the 21st Century Community Learning Center program
(Zhang & Byrd, 2006), which allowed local schools to open up for broader use by the
community. In 1998, the programs refocused on providing the same type of enrichment during
summer break. Programs ran between three and five years with funding to support a new
program. Three-year programs are fully supported each year, while 5-year programs are
supported in decreasing amounts each year.
16
In the last decade, Congress appropriated over a billion dollars a year to maintain or establish
after-school programs across the country (Zhang & Byrd, 2006). Since 2000, federal, state, and
city governments, as well as communities, have pushed initiatives to create or expand after-
school enrichment programs. An estimated 23% to 30% of American students, from both public
and private schools, attend an after-school program a minimum of three days a week (Zhang &
Byrd, 2006).
Link to Education Reform
After-school programs have seen a jump in attendance due to the increase in divorces,
single-parent homes, and families in which both parents are employed (Mahoney et al., 2005).
After-school programs reduce peer pressure and outside negative influences for those who attend
(Afterschool Alliance, 2017). According to the American Institute for Research (2008), the goals
of after-school programs are to be well-organized and provide quality curriculum,
implementation, supervision, facilities, and evaluation procedures. The most significant
influence on achievement is the quality of the program (U.S. Department of Education, 2016).
Since the inception of No Child Left Behind (NCLB), state and federal accountability
requirements increased expectations for student achievement and engagement in English
language arts, science, and mathematics. NCLB legislation mandated each state set a starting
point for yearly progress and raise this bar annually through 2014 (Seaton & Carr, 2005). In turn,
the school increased and promoted innovative teaching and higher levels of engagement to raise
student interest and achievement. The number of after-school programs increased to meet district
goals. After-school interventions have grown throughout the years by focusing on the different
types of learning formats and support from the community. The history and growth of after-
17
school interventions led to the development and utilization of multiple models (Afterschool
Alliance, 2017).
After-School Computer Science Instructional Interventions
Arizona State University started the Center for Science and Imagination in 1992 to bring
together scientists and artists and combine the two disciplines (Perera et al., 2017). Scientists
reported the kind of teaching involved in the sciences was also part of the arts curriculum (Perera
et al., 2017). Curiosity is fundamental, regardless of whether one is an engineer or artist (Payton,
White, & Mullins, 2017). Combining the disciplines improves understanding of how a set of
research questions can influence research questions asked elsewhere. The National Science
Foundation (NSF) supports blending movement and computer programming, as this supports
building the computation thinking skills of young girls. Also, the Education Testing Services
found hands-on project-based lessons improve retention, augment design thinking, and offer a
different approach to learning computer science (Mesiti, Parkes, Paneto, & Cahill, 2019).
Therefore, hands-on learning is likely to improve retention (Mesiti et al., 2019).
Universities are collaborating with local industry and school systems to improve
students’ preparation through innovative experiences wherein students can network with
professionals and expand on their identities as achievers (Verma, Dickerson, & McKinney,
2011). These after-school programs provide students hands-on learning and are funded over
multiple years (Migus, 2014). Typically, these programs are structured to take place on
weekends with an additional two-week academy in the summer and focus on middle school-aged
students. The curriculum encompasses field trips to museums and career day events. Teachers in
these programs are generally middle school teachers who receive 40 hours of professional
development and 40 more hours of follow-up training and support. Teachers are trained in
18
curriculum implementation and also receive free materials and resources to teach the same
lessons in their classrooms (Migus, 2014).
Nonetheless, middle school after-school programs should be more than an extension of
the classroom or preparation for high school learning. To have an effective impactful and
challenging after-school program, the staff needs to be well-trained and dedicated. There should
be a set of norms, values, and expectations, and asset-based standards to make the teachers more
facilitators than managers (Morehouse, 2009). Morehouse (2009) adds that lessons need to have
meaningful projects to allow students to contribute to a greater good and solve a problem. They
should have a reinforcing structure consistent with supporting youth as they develop
emotionally, socially, and morally. Lastly, all lessons need to have a hands-on approach to
learning with group projects with real-world implications. Making a program challenging
provides individual instruction, learning experiences, and expectations. As such, the program
benefits students by improving retention, creating more interest, and allowing them to establish
connections with peers (Morehouse, 2009). Moreover, in partnership with community
organizations, after-school programs can create challenging and high-quality curriculum (Nugent
et al., 2010).
After-school programs can only be effective if they are funded. Rorie, Gottfredson,
Cross, Wilson, and Connell (2011) found 65% of registered voters believe after-school programs
were necessary for their communities (Afterschool Alliance, 2017). Since 2006, Congress has
invested over $1 billion in these programs. Effective after-school programs include constructive
activities, socialization skills, and promote positive goals. In addition, they encourage positive
self-efficacy in the discipline. According to Girod, Martineau, and Zhao (2004), after-school
programs in rural and inner cities have been funded $250 million from the U.S. Department of
19
Education over the past decade. For instance, KLICK! (Kids Learning In Computer Clubhouses!)
is a federally funded after-school computer program based in Michigan, developed by local
teachers and funded by the Department of Education and the Kellogg Foundation. The program
supports 200 students at 10 schools (Girod et al., 2004). Students build webpages, surf the
internet, chat online, film and edit movies, and play computer games. Students develop a
newsletter and compete in video games and robotics. The program has allowed teens who are
less successful in traditional school classrooms to thrive and become empowered as they learn
new skills and apply them without being judged by their peers. Research shows participation is
active for teens who typically do not value school, have a low GPA, and have little knowledge of
technology.
Characteristics of Successful After-School Programs
Effectiveness of after-school program achievement is assessed based on four categories:
identification of theoretical framework, formulation of initial scale, a test of content validity, and
confirmatory factor analyses. The structure of these programs commonly has four key objectives;
academic development, social behavior, caring environment, and personal inspiration
(Afterschool Alliance, 2017; Fashola, 1998; Little, Wilmer, & Weiss, 2008). Quality programs
usually combine academic, recreational, physical, and artistic elements throughout the
curriculum to engage students (Grogan, Henrich, & Malikina, 2014). In addition, evaluations are
essential in establishing a process to monitor the progress of achievement and allow for
curriculum adjustment, reallocation of funding, improvement of facilities, staff development,
decision making, and accountability (American Institute for Research, 2008). A school’s after-
school program is only useful when administrators support it. Establishing an effective after-
school program requires thinking strategically about the future of the program and
20
collaboratively sharing ideas, missions, and resources in developing the program (McElvain &
Caplan, 2001).
For instance, Omaha, Nebraska, is expected to become a “tech hub” by 2020. Moreover,
it will become a national “STEM Ecosystem” with help from community organizations and by
being part of the STEM pipeline (Leas, Nelson, Grandgenett, Tapprich, & Cutucache, 2017).
Becoming a STEM Ecosystem requires community organizations committed to providing high-
quality STEM training and increasing the workforce for long-term economic stability. To create
the STEM Ecosystem, the city created after-school activities called NE STEM 4U centered on
problem-based learning, which has increased content knowledge (Leas et al., 2017).
Furthermore, it is designed for youth who are forming their opinion about STEM areas and
careers. This program’s model can be duplicated in other cities to increase STEM awareness and
curiosity.
Environment
An environment that promotes a positive learning experience for middle school girls is
one that employs culturally responsive teaching strategies. These strategies center the lessons
around the cultural knowledge, prior experiences, the frame of reference, and performance styles
of how middle school girls learn to make the lessons more relevant and useful for them (Gay,
2000). Positive relationships occur between mentor and participant when culturally responsive
teaching is utilized as it builds interaction between academic and socio-cultural realities (Ladson-
Billings, 1992). Programs that promote mentors to be culturally sensitive create a framework
where a wide variety of instructional strategies build upon multiple intelligences that teach
participants how to incorporate a combination of information from various disciplines in one
lesson (Gay, 2000; Ladson-Billings, 1992).
21
Mentors can be organized into three categories: cultural organizers, cultural mediators,
and orchestrators of social contexts for learning (Diamond & Moore, 1995). When a mentor
understands the vital role culture plays during a lesson and thus provides opportunities for
students with different levels of intelligence to freely express themselves, they are considered
cultural organizers. Cultural mediators are those who offer opportunities for participants to
engage in critical dialogue centered around cultural conflicts and then analyze variances between
reality and those that are from a different cultural mindset. Mentors must recognize the
importance of how culture can influence learning to make their facilitation of lessons compatible
with the social, cultural contexts of middle school girls (Diamond & Moore, 1995).
Cooper, He, and Levin (2011) explained the Critical Cultural Competence or CCC, as a
process that extends beyond basic knowledge of diversity, culture and climate, confidence and
self-efficacy and toward a self-reflection on the biases that can occur in an educational setting.
Ladson-Billings (2001) viewed CCC as the responsibility and action of mentors to learn about
the background of their students and the reasons why they are participating in the program. This
thought process transpired the necessity for educators to participate in quality professional
development opportunities regarding educating diverse learners. Without professional
development, Cooper et al. (2011) proposed, concepts of difference are likely to be left
unattended, while students are left isolated from their diverse thinking while their mentors
abandon the instructional learning process.
Program Approaches and Professional Development
Professional development focused on standards that define computer science is necessary
for better implementation of after-school programs. Effective after-school programs are designed
to meet local needs and combine elements beyond state standards. Fashola (1998) stated after-
22
school programs are successful when they provide structure, a link to the school curriculum, and
employ well-qualified and well-trained staff. Consistent and fluid training for all staff contributes
to programs’ giving engaging experiences for students. An example is the effective research-
based professional development characteristics of the NSF GK-12 Program, which partners
universities with local school districts and places university STEM graduates in the classroom
with teachers and which uses an inter- and intra-reliability testing concept. The NSF supports
teacher professional development, and approximately 20% of all federally supported research at
U.S. higher education institutions (Cormas & Barufaldi, 2011). The objective of the GK-12
professional development program is to increase communication and teaching skills, best
practices for teaching, additional professional development, and improved partnership with local
school districts and higher education institutions. Participants have a shared vision, and, within
these programs, there are key stakeholders, plenty of resources and time for planning, time for
collaboration and support from outside professions, and proximity to universities.
The partnership between the Department of Teacher Education and School of
Engineering at the University of Dayton and a Dayton Regional STEM center created the STEM
Education Quality Framework (SQF). SQF is a tool to guide STEM teachers. The program
includes curriculum development in a 6-week NSF program. Its objective is to increase the
knowledge of teachers, empower teachers to provide an innovative experience to their students,
as well as be informed to give students potential careers and societal needs (Pinnell et al., 2013).
The program is structured with an introductory innovation and design project, followed
by a more in-depth design project provided by an industrial mentor. It includes industry tours and
training related to curriculum design and pedagogy. There are also brainstorming sessions, a
23
five-step process for curriculum development, including midterm edits and assessment,
culminating with a web-based publication of the curriculum.
Teacher Preparation and Engagement
Out-of-school programs also benefit teachers through additional preparation. Teachers in
after-school programs develop a similar language, network, and attitude conducive to the
program (Ulvik & Sunde, 2013). Since teachers need a conceptual understanding of their role in
after-school programs as well as practical skills, these programs equip them with skills to help
them in guiding and teaching.
Moreover, Wahl-Alexander, Schwamberger, and Neels (2017) observed teachers who
participate in after-school clubs are more likely to enrich communication with community
members. They also incorporate new methods of organization and scheduling. Teachers feel a
sense of community with their peers and ownership in designing program lessons. They feel a
sense of affirmation at the close of the program and a deeper understanding of student affiliation.
Teachers increase their collaboration with parents and develop effective communication
strategies to enhance the parent-teacher connection.
Teachers in after-school programs gain knowledge through teaching lessons (Hynes,
2012). They enhance their teaching strategies and content knowledge by using examples or
analogies students understand. After-school programs provide an opportunity for teachers and
mentors to improve their professional skills, content knowledge, and community outreach. These
programs benefit not only students and teachers but help to close the gender gap by increasing
interest in computer science in middle school girls.
24
Rigor of Instruction
Rigor is widely used in the education system to describe instruction, learning
experiences, and academically challenging educational expectations. The rigorous and
motivating curriculum can be successfully infused within high-quality after-school computer
science programs. The Education Trust conducted a study in 2015 and found computer science
was viewed as frustrating and often resulted in modest learning gains and interest. Results
showed after-school programs have the flexibility to reduce this stress through cooperative
learning and meaning as well as through individualized relationships between students and
educators.
Impact of After-School Programs
There are two main types of after-school programs throughout the country: camps that
occur during the summer months and after-school programs that occur during the school year.
Both types of after-school programs allow students to learn and develop interests to increase
their engagement in everyday and academic life. Most of these programs are supported by school
districts and include a range of activities as well as play, snacks, homework, and enrichment.
Most experiences take place during the school year, but outside of the classroom, and
lessons align more closely with the school curriculum. They are driven by students’ authentic
interests, are flexible in time and space, credit-bearing, and aligned with school and community
collaboration. Programs that support computer programs address subject matter, practices, terms,
and instruments not included in the average school day or even covered in advanced grade levels.
Examples of these programs are learning at a local research agency and participating in a youth
research team associated with a local organization. Both policymakers and funders view after-
25
school programs as a viable way of engaging students in a high-quality program to build interest
and commitment to a discipline (Bevan & Michalchik, 2013).
Real-Life Skills
After-school programs permit school districts to add new classes to the traditional school
curriculum. These programs allow students to experience “real-life” skills and develop
knowledge, which can lead to a life-long interest. Patriots Technology Center Training Center at
Kettering Middle School has been offering after-school programs for 15 years (Gay, 2012). The
center hosts exhibit on cybersecurity, video gaming, robotics, flight simulation, and computer
building for students and parents to view. Each team develops a video game which is judged by
gaming professionals. The center provides students exposure to computer science professionals,
and a local video gaming conference continually recruits its students due to the program’s
success. The project manager states the success of the program occurs when former club
members go to college, get jobs in the areas of computer science, and return to share their
experiences with current students. Although there is a wide range of instructional interventions,
there is a common theme of success. Along with describing examples of after-school programs,
crucial elements are highlighting current computer science instructional interventions.
Female Student Self-Efficacy
Current research on characteristics of exceptional after-school programs is abundant and
highlights the qualities, goals, and evaluations used to sustain these programs. Research shows
after-school computer programs have a positive impact on attitude, self-efficacy, and knowledge
for middle school girls. After-school programs present a unique opportunity for middle school
girls academically, socially, and emotionally (Naizer et al., 2014). Naizer et al. (2014) explained
after-school programs have a positive impact on student attitudes toward mathematics and
26
science, with even short bursts of intervention showing immediate benefits. The authors found
consistent exposure in an after-school program resulted in increased interest in STEM subjects.
Exposure to role models has a positive impact on middle school girls by improving attitudes
toward the discipline (Weber, 2011). Middle school girls who begin to think about a career in
computer science and find a role model in the field are more likely to pursue a computer science
degree.
After-school programs allow students to understand concepts, processes, and procedures
better. These activities enhance achievement and interest in computer science and lead to an
increase in inquiry and reasoning skills (Sahin, 2013). After-school programs also motivate
students to work together, share ideas and have a sense of belonging. Studies show students’
participation in a computer science after-school program in their early years enhances and retains
their interest (VanMeter-Adams, Frankenfeld, Bases, Espina, & Liotta, 2014).
Students report a more positive experience in after-school programs when they are
structured, including positive emotions (Shernoff, 2010). Studies also show an increase in social
competence. Students learn how to better work with others through cooperation and teamwork.
They have an increase in empathy and understanding, which is critical to understanding different
perspectives of thought. They have an increase in psychosocial adjustment and social skills and
also learn how to develop relationships with their peers and adults.
After-school programs foster problem-solving skills that transfer to learning in a typical
classroom environment. According to Mayer, Quilici, and Moreno (1999), students learn
content-specific skills taught via educational games, general cognitive abilities, including
learning strategies in an informal setting; and gain exposure to educational computing
environments, which can transfer to improved cognitive processing in a variety of situations. As
27
such, the programs prepare students with the skills to be change agents in the field, and retention
of interest continues to solidify.
Female Student Retention
After-school programs are increasingly recognized as aiding in the retention of girls’
interest in STEM. Schools, both public and private, seek outreach opportunities to prompt STEM
and attract more girls to the field (Ma & Schapira, 2017). Workshops are created in a
collaborative effort to make equipment readily available, train teachers, provide step-by-step
instructions, and design individual tasks to help students more easily understand concepts and
want to practice more. Visits to museums and natural settings, entering science fairs, and joining
STEM-related clubs at their school also help to make a lasting impression on female students’
interest (Cech et al., 2011).
Reducing Stereotyping
There has been a history in the United States of stereotyping women as unable to succeed
in STEM (Campbell, 2011). Werner and Denning (2009) noted the attitude that the field is
competitive and masculine causes the low number of women in computer science. The rate of
female undergraduates in computer science declined from 37% in 1984 to 27% in 1997 (Denner,
Werner, Bean, & Campe, 2005). The decrease may be caused by a lack of confidence and
stereotyping. Furthermore, Denner et al. (2005) found a high need for interventions to increase
the interest and ability of female students in computer science and related careers. Similarly, the
NSF completed a study on the learning environment’s effect on students’ perspective of
stereotyping and found a combination of a sense of belonging or how they were valued played a
role in eliminating this factor.
28
Girls Creating Games is a current example of an after-school and summer program for
sixth- through eighth-grade girls aimed at reducing stereotype threat and activating interest in
information technology. The program centers around girls in the role of a designer of interactive
computer games. A second example is pair programming, which helps to improve women’s
representation in computer science as well as increase practical problem solving in computer
science coursework (Werner & Denning, 2009). Pair programming involves two students sharing
one computer and each having a clear role. One is the driver who manages the keyboard and
mouse while the other navigates, requiring collaboration. Working with a partner in such a
format is appealing to girls because it aligns with their social interests (Werner & Denning,
2009). Computer games are considered an essential component in building children’s problem-
solving skills (Curtis & Lawson, 2002). Notably, they drive girls toward the role of tech leaders,
which helps to break down stereotypes and discrimination.
Closing the Gender Gap
Computer science programs provide unique hands-on learning experiences to middle
school girls. Girls are typically not attracted to computers and computer programming, and a way
to curb this thought process is to introduce young girls to computers and how they work (Shortt,
1998). Through after-school computer clubs, computer camps, or enrichment programs aimed at
girls, girls will become familiar in a non-stressful environment, allowing for them to be
encouraged to use a variety of technologies. These after-school programs should include the
history of computers and the contributions of this technology from the women who created them.
Mayer-Smith, Pedretti, and Woodrow (2000) stated gender stereotypes are decreasing
with computer science instruction. This is due to the increase in after-school programs, which
boost attitudes on the subject and experience at a younger age. Modern culture has added to the
29
rise in computer science interest through social media and video gaming. In addition, the
increase in computer usage in all classrooms starting in the elementary years increased self-
efficacy and pushed technology as a norm.
Girls who participated in a computer-game construction experiment were just as
successful as their male counterparts. Computer-game construction enhances essential discipline-
related high-order thinking skills, teaches valuable computing science abstraction skills, and is
not male-dominated. Computer science is the fastest-growing economic sector, with a 68%
increase in output growth rate projected for the next decade (U.S. Bureau of Labor Statistics,
2017). With computational science is a critical factor in solving current real-world issues, such as
space travel, environmental modeling, cell phones, online social media, and even the
development of medicine, local schools are starting to support out-of-school programs to boost
the retention of interest (Carbonaro, Szafron, Cutumisu, & Schaeffer, 2010). Middle school girls
who participate in after-school computer science programs tend to have a higher self-concept but
also experience gender bias more than their male peers. Gender bias can come from male peers,
teachers, mentors, and even female peers. There is evidence of misappropriation in computer
science and ongoing gender segregation. Having the outlet of the computer science after-school
program allows middle school girls to obtain positive peer connections and promote retention of
interest (Robnett, 2016).
Knowledge, Motivation and Organizational Influences Framework
The Clark and Estes (2008) gap analysis framework was designed to evaluate problems
within an organization through a review of knowledge and skills, motivation, and organization.
When a gap exists within any of these categories, performance goals become unattainable. The
framework allowed for an examination of the root cause of gaps to improve individual and
30
organizational performance (Clark & Estes, 2008). Krathwohl (2002) stated there are four types
of knowledge types needed for employees to solve performance issues and achieve stakeholder
goals: factual, conceptual, procedural, and metacognitive. Clark and Estes explained motivation
is centered on an employee’s personal beliefs about themselves and their coworkers. The authors
emphasized motivation requires three process areas: active choice, when people act upon their
decision to work toward a goal; persistence, when people continue working toward their goal
despite barriers and distractions; and mental effort when people decide the amount of energy to
put into working toward their goal and do so. Additionally, motivation is a critical factor in
achieving the success of a target due to beliefs one develops for one’s self as a learner and
achiever (Rueda, 2011). Finally, organizational influences on stakeholder performance may
include organizational culture, processes, and resources (Clark & Estes, 2008).
The Clark and Estes (2008) gap analysis framework was applied to the STEMaven
computer science instructional intervention in terms of teacher knowledge, motivation, and
organizational needs to meet performance goals. The following sections discuss assumed
influences on the stakeholder performance goal in the context of knowledge and skills. Secondly,
assumed influences from the motivational perspective are described. Finally, assumed
organizational influences are studied. Each of the assumed stakeholder knowledge, motivation,
organizational influences on performance were examined through the methodology discussed in
Chapter Three.
Stakeholder Knowledge, Motivation and Organizational Influences
Based on a review of current research, two knowledge influences on STEMaven’s
mentors are discussed in the next section, followed by the types of knowledge, which helped to
determine the methodology to assess mentors’ knowledge gaps.
31
Knowledge and Skills
Clark and Estes (2008) affirm that, when an organization invests in resources to increase
employees’ knowledge and skills, there is a positive long-term impact on the organization’s
productivity. The authors suggested reinforcing knowledge and skills pertinent to the mission
and vision improved the employees’ aptitude in closing performance gaps related to their
profession. Clark and Estes’s reasoning was based on an assumption of performance being
enriched when employees use the new knowledge and skills to solve performance problems and
close the gaps between their current performance levels and performance goals. When employee
performance gaps are closed, the organization strengthens its ability to achieve its organizational
goals (Clark & Estes, 2008).
To solve performance issues and achieve their goals, employees need four types of
knowledge. Factual knowledge consists of the essential facts and definitions an employee must
know to understand their discipline or solve their performance problem. The second type is
conceptual and includes the categories, principles, models, theories, and structures which allow
employees to differentiate information and analyze correlations to the performance problems.
The third type of knowledge is procedural, which is directly applied to a task to solve a
performance problem. It consists of knowing how to do something, the methods of inquiry, and
the criteria for using those skills, techniques, and methods. The fourth type is metacognitive.
This type of knowledge allows for awareness and knowledge of one’s cognition. The employee
knows how they learn, about the task, and about strategies needed to carry out the task.
The first dimension is the knowledge influences required for mentors to achieve their
goals. To help STEMaven mentor’s close performance gaps, it is necessary to assess their
knowledge influences and corresponding knowledge types. Based on a review of the current
32
research, a procedural and a metacognitive knowledge influence on STEMaven’s mentors are
discussed in the next section. This categorization into the types of knowledge helped to
determine the methodology to assess the STEMaven mentor’s knowledge gaps.
Knowledge Influence 1 (Procedural). STEMaven mentors need to understand effective
and evidence-based strategies in teaching middle school students in computer science (Allison &
Rehm, 2007). Allison and Rehm’s (2007) action plan highlighted the importance of effective
strategies related to increasing middle school girls’ interest in computer science. The first step is
to review the descriptions and selection criteria in each lesson and scanning for verbiage students
may not understand. The second step is for mentors to understand the importance of culture and
climate within the program. The third step is to ensure facilitators have a strong background in
their subject and have knowledge of best practices. The last step is to audit the lesson plans and
check if they are organized explicitly around middle school standards and promote a vested
interest in computer science.
Knowledge Influence 2 (Metacognitive). Mentors need to need knowledge of deficit
thinking so they can promote a positive learning experience for middle school girls and employ
culturally responsive teaching strategies. Mentors must know effective teaching strategies, apply
the knowledge, and subsequently reflect on their effectiveness in teaching. Self-monitoring
effectiveness in education is essential to retaining the girls’ interest in computer science. Akl,
Keathly, and Garlick (2007) found that actively reflecting on an organizations’ effectiveness
ensures ongoing success. The authors used a two-step evaluation plan with both formative and
summative measures. The formative measure provided feedback regarding the development and
implementation of the lesson plans, and the summative assessment addressed the quality and
effectiveness of the activities. Di Stefano, Gino, Pisano, and Staats (2014) reported one becomes
33
more effective at a task after reflecting on the way one has performed it before. The team
conducted three studies based on the dual-process theory of thought. The results showed that,
when a group used reflection and sharing through group talk, they performed an average of 18%
or better. When employees meet to reflect on the who, what, when, where, and why, they can
review previous learning and give learning a new meaning, these meetings have a positive
impact on group morale. Table 2 illustrates an overview of the knowledge influences on
STEMaven’s mentors, corresponding knowledge types, and methods to assess knowledge gaps
about the stakeholder and organizational goals and the mission of the organization.
Table 2
Knowledge Influences, Types, and Assessments for Knowledge Gap Analysis
Organizational Mission
The mission of the STEMaven is to inspire middle school girls in East Tennessee to actively explore
the fields of technology, to close the gender gap in the technology profession, and to foster participants’
future careers.
Organizational Global Goal
By December 2020, mentors at STEMaven will demonstrate improved knowledge and use of effective
and evidence-based instruction proficiencies.
Stakeholder Goal
The mentor goal, supported by the program director, is to facilitate workshops that increase interest,
confidence, and perceptions of computer science.
Knowledge Influence Knowledge Type Knowledge Influence Assessment
Mentors need to know
how to incorporate
effective and evidence-
based strategies of
teaching middle school
girl’s computer science.
Procedural
Survey items to assess his/her knowledge. For
example, “I understand what skills are needed to
have a culturally responsive lesson.” (strongly agree,
agree, disagree, strongly disagree) During my
lesson(s) at STEMaven, I employ culturally
responsive learning techniques.” (strongly agree,
agree, disagree, strongly disagree) “I take the time to
learn about the background of my students and
reasons to why they are participating in the
program.” (strongly agree, agree, disagree, strongly
disagree)
34
Table 2, continued
Knowledge Influence Knowledge Type Knowledge Influence Assessment
Mentors need to know
what deficit thinking is so
they can promote a
positive learning
experience for middle
school girls and employ
culturally responsive
teaching strategies.
Metacognitive Interview questions for mentors to collect
information on their reflective teaching strategies.
For example, “Explain the importance of how
cultural can influence learning.” “How do you
include this in your lesson(s) to make them
compatible with the social cultural contexts of
middle school girls?” “What strategies do you
employ to address deficit thinking with other mentors
and/or the director?” “Are students included in these
strategies?”
Motivation
Motivation is the second dimension required for STEMaven’s mentors to achieve their
stakeholder goals. Clark and Estes (2008) explained motivation is centered on an employee’s
personal beliefs about themselves and their coworkers. As previously mentioned, the authors
state motivation encompasses active choice, persistence, and mental effort. The authors
cautioned employee performance challenges could hinder these three motivational process areas.
By assessing and addressing motivational challenges, organizations can aid employees in closing
the gaps between their current performance levels and their performance goals. When employee
performance gaps are closed, the organization is closer to achieving its organizational goals.
STEMaven can achieve its organizational goal by supporting mentors so that they can
confidently teach computer science to middle school girls. Multiple factors can affect motivation
and goal orientation (Rueda, 2011).
For STEMaven, two specific considerations are critical to success. First, mentors need to
see how effective their teaching is in increasing retention of an interest in computer science.
Second, mentors need increased confidence that they can teach computer science to middle
school girls. For STEMaven to assess motivational challenges, understanding the motivational
35
influences related to mentors’ achieving their stakeholder goal is critical. This literature review
focused on the effect of goal orientation and self-efficacy on mentors’ motivation to effectively
teach middle school girls in computer science. The application of these theories to assess the
STEMaven’s mentors’ motivational levels are discussed. The type of motivation influence
determined the methodology to evaluate mentors’ motivational gaps.
Goal orientation. Goal commitment and implementation objectives are fundamental to
achieving goal-directed behavior. Goal orientation is the degree to which an employee focuses
on a task and the task’s results. As Bandura (2000) wrote, unless people believe they can produce
desired results and omit the lack of motivation, they have little incentive to get things done.
With a strong goal orientation, employees will focus on the whole goal and how it will affect the
organization. At STEMaven, for each new workshop, the educational project manager needs to
describe in detail what the project is, the expectations, and a clear vision of how it is related to
the objective of the program. When managers employ strategies based on strong goal orientation,
productivity will increase with teachers reaching the goal along with utilizing current resources
and skills. Also, a definite goal orientation allows teachers to see their contribution to the overall
goal.
Self-efficacy. As Borgogni, Dello Russo, and Latham (2011) stated, to strengthen ones’
self-confidence, one must work on self-efficacy and perception. Motivation needs to be
supported by competent evidence in line with ones’ conceptual reasoning (Pintrich, 2003). When
an organization focuses on enhancing its collective efficacy, it increases employee self-efficacy,
which leads to an increase in organizational productivity. When managers can recognize the
needs and expectations of the employees, they will be at relating to others in a particular context
or role efficacy (Borgogni et al., 2011). In addition, the researchers stated role efficacy is the
36
potential value of a person in personal and interpersonal effectiveness while occupying a
particular role. Understanding the role of efficacy and its relationship between role uncertainty
and conflict helps to improve an organization’s climate and employees’ self-efficacy. The roles
of employees must be clear, as they are considered an essential component of the group
performing functions and aids in self-efficacy (Pintrich, 2003).
Richter, Hirst, van Knippenberg, and Baer (2012) found creative self-efficacy and
individual creativity increases a team’s resources. When there is shared knowledge and diversity
within groups in an organization, there is an increase in both creativity and self-efficacy. When
an employee harnesses their creative self-efficacy, they will have the knowledge and skills to
produce creative outcomes and realize more creative benefits from team resources.
They will proactively access resources such as team members’ knowledge, expertise, and
insights. When employees have the belief in themselves to be successful in their tasks, their
performance increases, and the organization’s goal is accomplished.
Goal orientation and self-efficacy are the key motivational factors that influence teachers
and mentors in successfully teaching in an after-school program. In addition to motivational
factors, organizational factors need to be addressed when assessing the influence of these factors
on an after-school program. Table 3 below identifies two motivational influences focused on
goal orientation and self-efficacy. These influences were used to more fully understand how
motivation affects the mentors’ engagement in increasing the number of middle school girls’
interest in computer science.
37
Table 3
Motivational Influences and Assessments for Motivation Gap Analysis
Organizational Mission
The mission of the STEMaven is to inspire middle school girls in East Tennessee to actively
explore the fields of technology, to close the gender gap in the technology profession, and to
foster participants’ future careers.
Organizational Global Goal
By December 2020, mentors at STEMaven will demonstrate improved knowledge and use of
effective and evidence-based instruction proficiencies.
Stakeholder Goal
The mentor goal, supported by the program director, is to facilitate workshops that increase
interest, confidence, and perceptions of computer science.
Assumed Motivation Influences Motivational Influence Assessment
Goal Orientation: Intrinsic: Mentors need to see
how effective their teaching is in increasing
retention of interest in computer science.
Survey item, “Overall how satisfied are you
with facilitating a STEMaven lesson(s)?”
(very satisfied, satisfied, neutral, dissatisfied,
very dissatisfied). “How motivated are you
to see students succeed?” (very motivated,
somewhat motivated, not very motivated, not
at all motivated, not sure).
Self-Efficacy: Individual self-efficacy: Mentors need
increased confidence that they can teach computer
science to middle school girls.
Interview item “Tell me how confident you
about my ability to teach middle school girls in
computer science.” (strongly disagree-strongly
agree) Interview item: “
Organization
This section examined the possible organizational barriers or influences (Clark & Estes,
2008) affecting STEMaven mentors’ ability to increase middle school girls’ interest and
retention of computer science.
Both organizational barriers and social barriers need to be discussed as interrelated
constructs affecting the increase in the number of middle school girls’ completion of the
computer science program. Along with the knowledge and motivational gaps, organizational
38
factors such as organizational identity and continuity, trust, and access to best practices modeling
are linked to having the elements to accomplishing the performance goal.
Organizational culture theory. Schein (2017) defined culture as basic conventions
learned by a group as they solve a problem. These conventions can then be transferred to new
group members as the appropriate way to recognize, reason, and feel when the same problem
arises. There is a benefit in the cognizance of these cultural forces.
Therefore, the culture affects this change process and, thus, the ability to maintain the
desired change (Kezar, 2001).
Schein (2017) described culture as an influential construct consisting of three levels:
artifacts, beliefs and values, and basic underlying assumptions. According to Clark and Estes
(2008), culture is multi-dimensional and serves as a channel for describing the goals, beliefs, and
processes gained by employees over a period of time. These basic underlying assumptions are
the most defiant aspects of culture to adopt in the change process, especially in organizations
with a perceived resilient culture (Schein, 2017).
Organizational culture can be deep-seated and lend to stability, as some theorists suggest
flexible and adaptive learning has a more promising position to prepare employees for the
complex and changing world (Senge, 1990). This type of thought can be matured through the
obligation to the inquiry, communication, diversity of thought, and a commitment to
acknowledging that problems are complex and interconnected (Schein, 2004). Change to the
cultural environment cannot improve on its own and must be shared by all employees within an
organization (Senge, 1990). Gallimore and Goldenberg (2001), as described in this paper,
divided culture into two categories: cultural models and cultural settings. There are two cultural
39
models and two cultural settings influencing mentors to facilitate workshops that increase
interest, confidence, and perceptions of computer science.
Cultural models. Cultural models are an organization’s internal beliefs and values. The
socio-culture framework suggests these models are often seen as barriers, but, if addressed along
with knowledge and motivation, they can then become opportunities for growth. The two
cultural models are organizational identity and continuity, along with trust, and the specific
cultural setting is the STEMaven organization.
Organizational identity and continuity. When an organization understands the element of
its own identity, it can then begin to grow (Schein, 2004). Organizations with a collective
identity encompassing the entire workforce create bonds in the employee community and
improve perceptions of effort (Bolman & Deal, 2008). A set mission and vision allow employees
common purpose structured across the organization. Without a clear and shared message, issues
such as retention and continuity will emerge (Bolman & Deal, 2008). Collaboration,
development of shared values and goals, and articulated norms support continuity (Bolman &
Deal, 2008; Krosgaard, Brodt, & Whitener, 2002; Schneider, Brief, & Guzzo, 1996).
Culture of trust and collaboration. Teaching can be inherently stressful, especially
under the pressure of creating individual lessons to retain the interest of middle school girls. This
takes a profound knowledge base of pedagogy and a system of support, including trust and
collaboration with all employees. Schneider et al. (1996) suggest the nature of interpersonal
relationships, hierarchy in decision making, the focus and use of goals, support, rewards, and
recognition all manifest honest and open interactions. Teachers must feel safe to take risks and
be encouraged to be collaborative, as organizations with these practices have more significant
growth (Garmston & Wellman, 1999; Senge, 1990).
40
Cultural settings. Cultural settings are the cultural models’ indicators. The parameters
within an organization can be studied and observed.
This section reviews specific cultural settings at STEMaven, which may be barriers or
assets to stakeholder goal achievement, including mentor support systems, mentors and models,
and communication of instructional values.
Teacher support systems. A learning organization is capable of encouraging employees
to acquire and transfer knowledge (Huffman, Thomas, & Lawrenz, 2003). Teachers are more
useful when they have accessible support systems (Huffman et al., 2003). Examining practice,
curriculum implementation and development, and collaborative work can be challenging to
implement and sustain. Productive support systems targeting the specific levels of individual
teachers increase peer support, roles, and job satisfaction (Kipps-Vaughan, 2013). Along with
emphasizing long-term active support through engagement, connections between teachers’ work
and their students will improve professional practice (Huffman et al., 2003).
Mentors and models. A key support system for improving the depth of knowledge for
new and established teachers are mentors and models. In educational environments, a mentor is a
more experienced coworker who supports teachers in the development and execution of the
instruction (Ulvik & Sunde, 2013). Utilizing mentors is an intuitive practice that provides
teachers the support to solve problems and develop trustful relationships (Fresko & Alhija,
2012). Therefore, providing teachers with opportunities for mentoring supports the conceptual
understanding of their role and helps to develop their practical skills as well as job satisfaction.
Table 4 below identifies the organizational influences. These influences were used to
more fully understand how motivation affects the mentor’s engagement in increasing middle
school girls’ interest in computer science.
41
Table 4
Organizational Influences
Organizational Mission
The mission of the STEMaven is to inspire middle school girls in East Tennessee to actively explore the
fields of technology, to close the gender gap in the technology profession, and to foster participants’
future careers.
Organizational Global Goal
By December 2020, mentors at STEMaven will demonstrate improved knowledge and use of effective
and evidence-based instruction proficiencies.
Stakeholder Goal
The mentor goal, supported by the program director, is to facilitate workshops that increase interest,
confidence, and perceptions of computer science.
Assumed Organizational Influences
Organization Influence Assessment
Cultural Model Influence 1:
Organizational identity and continuity:
The organization needs a culture that supports
change in existing teaching strategies founded on
collective engagement, shared purpose, and
collaboration to aid in motivating and retaining
interest of the participants.
Survey questions about the value of new teaching
strategies. For example, “The organization provides
professional development that is centered around
new teaching strategies.” (A great deal, somewhat,
a little, not at all) “PD is offered that is centered
around new teaching strategies.” (A great deal,
somewhat, a little, not at all).
“The PD is beneficial to both yourself and the
organization.” (very helpful, somewhat helpful,
somewhat unhelpful, not helpful at all).
Cultural Model Influence 2:
Cultural Trust: There needs to be a culture of
trust in the organization between leadership and
mentors.
Interview questions would surround concepts of
transparency and accountability. For example,
“How does the program director hold you
accountable for success or failure of your
workshop(s)?” “Explain how leadership exemplifies
ideal accountability behaviors and transparency for
all mentors.” “The program director is proactive in
identifying internal and external factors that may
derail a workshop before it facilitated.”
Cultural Setting Influence 1:
Support systems: Mentors need a culture that
supports change in existing teaching strategies
founded on collective engagement, shared
purpose, and collaboration.
Interview questions asked whether the mentors feel
they are in a culture that supports change in existing
teaching strategies founded on collective
engagement, shared purpose, and collaboration. For
example, “What barriers may exist that hinder
mentors from creating a new engaging lesson(s).”
“What supports systems would you like to see
STEMaven put in place that could further support
the goal of increasing interest, confidence, and
perceptions of computer science?”
42
Table 4,continued
Assumed Organizational Influences
Organization Influence Assessment
Cultural Settings Influence 2:
Mentors and models: Mentors need effective role
models who have succeeded in increasing
interest, confidence, and perceptions of computer
science.
Interview questions that ask whether mentors have
access to a mentor that has facilitated a successful
after-school program. For example, “Tell me about
when you were able to meet with an effective
mentor(s) that succeeded in increasing interest,
confidence, and perceptions of computer science.”
Conceptual Framework: The Interaction of Stakeholders ’ Knowledge and Motivation and
the Organizational Context
The conceptual framework is a critical component of the research process, as it is the
causal structure guiding the study and includes the system of concepts, beliefs, and theories to
support the research (Maxwell, 2013). Its purpose is to assist the researcher with a visual model
of the essential ideas, the scope, the concepts being investigated, and of gaps in the literature,
which are key to answering research questions (Maxwell, 2013). The conceptual framework
presented below, in both graphical and narrative form, defends the research and aid in identifying
the appropriate methods for investigating the research questions (Maxwell, 2013). Therefore, the
conceptual framework presented in this written work considers previous research on best
practices and methods on how to facilitate workshops that increase interest, confidence, and
perceptions of computer science.
This study employed the knowledge, motivation, and organizational factors gap analysis
model developed by Clark and Estes (2008). The variables are interrelated, work as part of a
system, and align to achieve the organizational goal (Clark & Estes, 2008). The framework
utilized the knowledge, motivation, and organization variables for this study. It emphasized
previous research in context with the ability of after-school programs to increase interest and
retention of middle school girls in computer science. There was also the consideration of
43
previous research in a context that contributed to identifying the methods most applicable to the
organizational needs at STEMaven. This study aimed to find a medium to best support the
organizational goal of providing increased opportunities through effective and evidence-based
instruction and mentorship for middle school girls to engage in workshops that motivate and
retain their interest in computer science by December 2020 (Figure 1).
Figure 1. Conceptual framework: Interaction of stakeholder knowledge and motivation within
organizational cultural models and settings.
44
While this study presented and organized the knowledge, motivation, and organization
factors independent of each other, it was necessary to understand the intersecting relationships
among variables. According to Maxwell (2013), a concept map is a visual representation of the
conceptual framework for the design of a study, and an explanation of the symbols and their
interaction is depicted in Figure 1, the map for this study. This figure outlined the relationships
among the factors influencing best practices and methods with the STEMaven program, both
with each other and within the broader organizational context, leading to the effectiveness of
self-reflection in teaching and the stakeholder goal. In the figure, the more substantial blue circle
represents STEMaven as the organization of the study and cultural settings and models within it.
These cultural influences include organizational culture around issues such as organizational
identity and continuity and supports and structures that affect teacher effectiveness (Schein,
2004, Schneider et al., 1996; Senge, 1990).
Within the organization, the global goal of mentors at STEMaven, demonstrating
improved knowledge and use of effective and evidence-based instruction proficiencies grounds
the subject of the study. They are illustrated in the figure as a green circle. Within the global goal
are the knowledge and motivation influences on teacher effectiveness. The knowledge influences
are declarative and factual in relation to best practices and methods and metacognitive regarding
the instruction to increase retention of an interest in computer science (Akl et al., 2007; Alvarado
& Judson, 2014; Di Stefano et al., 2014). The motivational influences include intrinsic value to
motivation and engagement in the professional development process and individual and
collective self-efficacy with creating an interest in computer science (Bandura, 2000; Borgogni
et al., 2011; Clark & Estes, 2008; Priyadarshini, 2009; Richter et al., 2012; Rueda, 2011). These
influences must also interact with one another for the achievement of the goal (Clark & Estes,
45
2008). Interacting with one another and within the broader organizational context, these
influences are represented to best support growth towards the stakeholder goal (Clark & Estes,
2008).
Conclusion
This evaluation study sought to identify the resources necessary to reach the goal of
mentors at STEMaven demonstrating improved knowledge and the use of effective and
evidence-based instruction proficiencies. To inform this study, this chapter reviewed literature
related to increasing the effectiveness of teaching strategies and reflection. This literature
reviewed the identification of the assumed knowledge, motivation, and organizational influences
specifically related to the achievement of the stakeholder goal and experience at STEMaven. The
knowledge influences included procedural concepts about best practices and methods, along with
effective teaching strategies and metacognitive concerning self-reflection. The motivation
influences included intrinsic value about engagement in the professional development process
and the individual and collective self-efficacy in believing they have practical teaching skills.
Finally, the organizational influences included program culture around issues of identity and
continuity, teacher support systems, and structures to influence teacher effectiveness and
communication of knowledge. Chapter Three describes the process of validating these
influences.
46
CHAPTER THREE: METHODOLOGY
The purpose of this study was to evaluate the effectiveness of STEMaven mentors
necessary to achieve the goal of demonstrating improved knowledge and the use of effective and
evidence-based instruction strategies. This was achieved by employing the Clark and Estes
(2008) gap analysis model with a mixed-methods design. Both were used when the researcher
conducted quantitative research, analyzed the data, and then explained the results along with the
qualitative results (Creswell & Creswell, 2018). While a complete needs analysis would focus on
all STEMaven stakeholders, for practical purposes, the stakeholders to be focused on in this
analysis were the STEMaven mentors. Four questions guide this study:
1. To what extent is STEMaven contributing to the development of effective and evidence-
based teaching skills in the mentors?
2. What are the STEMaven mentors’ knowledge and motivation related to improving their
effective and evidence-based teaching skills?
3. What is the interaction between STEMaven’s organizational culture and context and the
mentors’ knowledge and motivation to improving effective and evidence-based teaching
skills?
4. What are the recommendations for STEMaven’s practice in the areas of knowledge,
motivation, and organizational resources?
The remainder of the chapter outlines a description of the participating stakeholders, data
collection and instrumentation, data analysis, credibility and trustworthiness, ethics, and
limitations and delimitations of this study.
47
Participating Stakeholders
The stakeholder group of focus for this study were 34 STEMaven volunteers and
mentors, some of whom hold terminal degrees in STEM. Mentors at STEMaven are not required
to be of a particular gender, nor to facilitate a workshop, is a mentor required to have a terminal
degree in a STEM field, or currently hold a position in that field. The quantitative portion of this
study involved a screening survey sent to all volunteers and mentors, allowing for purposive
sampling to find out which mentors facilitated a workshop. One of the reasons for quantitative
sampling in this study was to provide descriptive statistics regarding the mentors who have
facilitated workshops and have background knowledge of the organization (Creswell &
Creswell, 2018). To gather as much data as possible, an interview was requested. Ideally, seven
to 10 mentors were needed to be identified for a 45 to 60-minute interview (Johnson &
Christensen, 2015). The interview request was sent by email to all mentors who had facilitated at
least two workshops and held a terminal degree in a STEM field. The interview questions aimed
to describe the attributes and thoughts of the participants as they related to the research questions
and conceptual framework (Robinson & Firth Leonard, 2019).
Survey Sampling Criteria and Rationale
Criterion 1. Approved volunteers and mentors at STEMaven. The study was interested
in those who have been approved to mentor a workshop.
Criterion 2. Mentors who have facilitated a workshop. The emphasis of the mentorship
is a significant part of the study and could provide insight into the research questions and related
influences. There are approximately 75 workshops each fiscal year on eight different topics of
computer science. Each workshop is available to students at three levels based on their prior
knowledge: basic, intermediate, or advanced.
48
Survey Sampling Strategy and Rationale
I surveyed all STEMaven mentors who had facilitated at least one workshop since
January 2018 to get as much data as possible. The survey was administered after the summer
workshops to gauge the participants’ perceptions and feelings of STEMaven workshops related
to the increase in interest of computer science for middle school girls. This was administered
through a link sent via email from Qualtrics to reach the whole audience.
Interview Sampling Criteria and Rationale
Criterion 1. Mentors who have terminal STEM degrees and had or currently have a
career in STEM. These mentors have obtained a degree based on computer science or
technology. They are in a position to recall aspects of the program related to obtaining an interest
in computer science and or technology.
Criterion 2. Mentors who have facilitated at least two workshops since January 2018.
This was to ensure mentors understood the program and what students needed to capture their
interest in STEM.
Criterion 3. The mentors must have completed the survey.
Interview Sampling Strategy and Rationale
Participant selection for the qualitative portion of this study was purposeful, so the
researcher could better understand the problem and answer the research questions (Creswell &
Creswell, 2018). For this reason, purposeful sampling was used to identify participants who met
specific needs for the study (Johnson & Christensen, 2015). In addition to surveys, I interviewed
mentors to gain more depth and understanding of the results. Merriam and Tisdell (2016) stated
interviews as a source of data, allow the researcher to obtain rich information as well as to find
out participants’ perspectives.
49
Eight participants who previously completed the questionnaire were selected and
contacted based on the purposeful criteria related to identifying the extent to which mentors at
STEMaven are demonstrating improved knowledge and the use of effective and evidence-based
instruction proficiencies. This sampling allowed for an adequate representation of diverse
thought, opinion, and belief in addition to insight learned from semi-structured open-ended
questions to aid in the validation of survey data addressing questions on knowledge, motivation,
and organizational influences related to increasing interest in computer science.
Table 5
Sampling Strategy and Timeline
Sampling Strategy
(e.g., census,
purposeful with
max. variation)
Number in
Stakeholder
population
(e.g., There
are 50
mentors)
Number of Proposed
participants from
stakeholder population
(e.g., of the 50, I will
sample 10 mentors, 2
from each of 5
departments)
Start and
End Date
for Data
Collection
Surveys: Screening for
interviews,
purposeful Likert
scale questions to
triangulate with
interviews
Total 34 part-
time mentors.
I sent the survey to all
mentors and hope for a
70% participation rate.
August –
September
2019
Interviews: Screened
participants to be
representative of
the population
demographics
Total 7 part-
time mentors.
A minimum of 7 survey
participants representative
of the stakeholder
demographics.
October-
November
2019
Data Collection and Instrumentation
This study used quantitative sample surveys and qualitative interviews to explore the
interaction of knowledge, motivation, and organizational influences on STEMaven mentors’
performance. The study relied on surveys to collect data and provide simple summaries about the
samples as well as the measures obtained. Furthermore, a descriptive analysis of the samples was
50
formed based on quantitative data analysis. Qualitative methods aided in answering the research
questions in a manner that was meaningful to STEMaven and similar programs.
Survey
Survey instrument. The researcher administered the surveys via an online link
distributed through email. The instrument was created in English. The quantitative survey
consisted of demographic questions and 13 survey items. The survey took approximately 10 to
15 minutes for participants to complete. The last survey question asked if they were willing to
participate in an interview. If they were, they were asked to provide their email or phone number.
The survey protocol can be found in Appendix A.
Survey procedures. Johnson and Christensen (2015) stated quantitative research is
essential to make accurate generalizations about a population using sample data. The survey used
in this study was quantitative to provide for generalization about STEMaven mentors concerning
the knowledge, motivation, and organizational influences related to the mentors at STEMaven,
demonstrating improved knowledge and the use of effective and evidence-based instruction
proficiencies. To gain the most significant sample possible, the survey was sent to all mentors at
STEMaven that fit the survey criteria protocol.
Interviews
The study collected data through interviews. This method provided a qualitative
understanding and insight into how knowledge, motivation, and organizational influences at
STEMaven affected the consistent implementation of mentors at STEMaven, demonstrating
improved knowledge and the use of effective and evidence-based instruction proficiencies. Face-
to-face techniques helped to generate interpretations and theory.
51
Interview protocol. Merriam and Tisdell (2016) stated that interviews are primary
sources of data that allow the researcher to gain rich information as well as the opportunity to
find out the participants’ perspectives. The interviewees were mentors who voluntarily agree to
participate and who have taught at least two workshops in the past year. For convenience, all
interviews took place via phone or video conference, allowing for the comfort of the mentor due
to heavy work schedules. No participant preferred a more private location, so additional
locations were not considered. The interview protocol can be found in Appendix B.
Interview procedures. The interviews were semi-structured and in-depth and occurred
after the survey had been closed. Semi-structured interviews have a higher likelihood of yielding
comparable data. Interviews were planned to occur during October and November 2019, but
additional interviews were held in January 2020 to gain additional data. Each participant was
interviewed individually only once for no more than 60 minutes. Interviews occurred over the
phone or via video conferencing using software of the participants’ choice. The researcher used a
notepad for notetaking. The interview consisted of open-ended questions that allowed
participants to respond in their context and express themselves comfortably (Patton, 2002). The
semi-structured, open-ended questions permitted the interviewer opportunities to explore and
probe (Merriam & Tisdell, 2016). The interview questions addressed knowledge, motivation, and
organizational influences related to mentors at STEMaven, demonstrating improved knowledge
and the use of effective and evidence-based instruction proficiencies. The interview guide
provided the interviewer a consistent set of questions to ask of each interviewee (Patton, 2002).
Data Analysis
Making meaning from data is the process of analysis (Merriam & Tisdell, 2016). Data
analysis included several strategies and tools and started with the sequential analysis of survey
52
questions and one-on-one interviews. The researcher reviewed the quantitative survey data along
with the qualitative transcriptions of the interviews through sequential design. This section
describes the approach of analyzing one phase of data and then looking at another method of
collection (Creswell & Creswell, 2018).
Qualitative Analysis
The qualitative transcriptions of the one-on-one interviews through sequential design
involved an analysis of a single case that helped formulate a theory, after which the rest were
examined for their contribution to the theory. The process involved narrative analysis where the
stories presented by the participants were reformulated based on the contexts and different
experiences of the individuals to inform the study and analyzed for STEMaven perceptions and
descriptions.
Quantitative Analysis
The survey was conducted using the software Qualtrics, and the tool also supported
analysis for triangulation among data points to aid in disaggregation. In analyzing quantitative
data, the researcher calculated the mean, percentages, medians, and modes. The majority of
comparisons were made using multiple item themes. Demographic information was contained in
the survey instrument. These measurements provided a further quantitative understanding of the
participants (Johnson & Christensen, 2015) and were analyzed using descriptive statistics and
frequency tables for the level of education and gender. Tables and figures were analyzed for
participants’ characteristics and perceptions of STEMaven. Survey items are presented in a
narrative setting as well as with visualization and table in Chapter Four.
53
Credibility and Trustworthiness
The researcher is the instrument in the qualitative phase of the research. The qualitative
phase may carry inherent bias (Merriam & Tisdell, 2016) due to the researcher’s serving as a
volunteer mentor in the program. Therefore, this section describes the steps the researcher took
to minimize inherent bias and increase credibility and trustworthiness through all phases of the
research. The researcher spent an adequate amount of time collecting the data until the findings
seemed comprehensive, and no discoveries or themes emerged (Merriam & Tisdell, 2016). To
test the credibility of the results, the researcher keenly pursued both ideas and examples, which
may counter the emerging theme(s), both within each of the qualitative methods and any needed
follow-up.
As mentioned in the ethics section, the researcher was neither an evaluator nor a direct
supervisor of mentors in the program. Therefore, participants were reassured of the
confidentiality of their engagement was vital for credibility. During data collection, the
researcher provided written and verbal statements that solidified the confidentiality of the data.
This confidentiality provided increased credibility and trustworthiness that made mentors feel
comfortable in responding to the survey and interview questions honestly and openly (Creswell
& Creswell, 2018). Additionally, the participants could have opted out of the study at any time.
This explanation and open understanding of data collection increased credibility and
trustworthiness (Rubin & Rubin, 2012).
Throughout the computation of the analysis and during the reporting, the researcher self-
reflected regarding bias, assumptions, and perspective. The researcher kept a digital journal
related to each of the data collection methods to document questions, reactions, and areas that
could have required further clarification. The digital journal allowed the researcher to reflect on
54
the bias. As Merriam and Tisdell (2016) stated, power dynamics will always be present while
gathering data. The researcher paid close attention to the power dynamics throughout the
collection of data and any subsequent reflections.
Finally, the reporting phase included detailed descriptions of all findings. The researcher
included descriptions, quotations, and any other comprehensive information gathered from the
participants as they connected to the research. This detailed description and increased
transparency of the data collection process increased credibility. The reporting includes a
discussion of the findings and recommendations reflective of the literature. This referencing
solidifies the reporting in formerly conducted research and increase reliability in the
recommendations (Merriam & Tisdell, 2016).
In summary, there are multiple ways human beings experience phenomena, and this
study’s resulting narrative related solely to participants’ experience. Research can remind us of
the fundamental unlikelihood of netting absolute truth (Merriam & Tisdell, 2016). Like the
instrument in qualitative research, the researcher ensured trust and conducted the study ethically.
The validity of the findings relied solely on the trustworthiness of the researcher.
Validity and Reliability
The interview and survey questions added to both validity and reliability. The
participants were selected both randomly and purposively, which reduced selection bias and
increase reliability (Krueger & Casey, 2009). The researcher aimed to have 90% of the
population take part in the study. This number of participants was to contribute to the greater
validity of the study and provide sufficient data to create generalization regarding the population
(Johnson & Christensen, 2015).
55
Threats to the validity of this study included historical threat due to the newness of the
program, having a range of creation of fewer than three years, and possible dispersal of
treatment, primarily due to a variety of data collection of over three months. While this period is
relatively short, reducing any threat of development, reversion, or impermanence, it could have
also lead to participants’ cross-contaminating. Nonetheless, an essential reason for using
interviews was that they allowed for ideas to be shared one-on-one (Merriam & Tisdell, 2016).
Therefore, contamination as a threat to validity applied to the quantitative and qualitative
phases of the study. To evade this threat, the researcher provided a statement in the interview and
survey emails requesting participants not discuss the interview or survey questions with
colleagues. Ultimately, reliability depended on participants’ answering honestly and openly
according to interview and survey instructions.
Ethics
This study was substantiated in operating ethically before, during, and following the
study. Therefore, to conduct the research ethically, the researcher considered several
accountability measures. Glesne (2011) stated the overall goal of obtaining new knowledge
might take a subservient role to researcher responsibilities, such as respect for persons,
beneficence, and justice. These expectations included gaining informed consent to participate,
voluntary participation, right to withdraw without penalty, separate permission to record, and
confidentiality.
Informed consent is an essential element of the institutional review board and the study
(Rubin & Rubin, 2012). There is an obligation of all researchers not to harm their participants
(Glesne, 2011). The researcher had the duty to obtain informed consent. Therefore, all
participants were informed of their protection through the University of Southern California’s
56
Human Subjects Protection Program. Additionally, the researcher provided those surveyed an
information fact sheet for exempt non-medical research. The participants gained knowledge that
their participation was voluntary and possible risks related to participating in the study (Glesne,
2011). The researcher declared all participation would remain confidential, nor would names or
answers be shared. Interviewees were also reminded of the confidentiality of all conversations.
All participants were reminded they could cease involvement in the study at any time. The
researcher provided, in writing at the beginning of the survey and verbally before the interviews,
a notice of the option to not participate without an effect on their professional opportunity within
the program. In addition, at the start of the interviews, the researcher requested permission to
record all audio manually. Following the interviews, the researcher utilized software to transcribe
interviews verbatim. The transcripts were coded with pseudonyms for confidentiality. The
researcher destroyed all files following transcription and stored all data temporarily during the
analysis phase.
Finally, while conducting the study, the researcher assumed the mentors at STEMaven
wanted the program to succeed and answered questions in both surveys and interviews honestly
and accurately. There was an inherent bias for which the researcher needed to account. This bias
included the researcher’s role of over 12 years of STEM teaching experience as a middle school
teacher.
Limitations and Delimitations
There were both limitations and delimitations the researcher was aware of as this study
began. Limitations could vary throughout the study and be out of the researcher’s control. The
design of this evaluation study, including instrumentation and sampling methods, constrained the
analysis and interpretation of the data. For instance, the interview questions were not validated
57
prior to the study for clarity or understanding in a similar sample group. Therefore, mentors may
have interpreted the questions differently from the researcher’s intent. In addition, the study was
conducted during a brief period during the fall workshop series, making it dependent on the
respondents’ mindset during this limited and exhaustive time of year for the organization.
Delimitations are the researcher’s choices, which may have had repercussions for the study. The
limitations of this study incorporated data collection from mentors at STEMaven who have
facilitated at least one workshop since January 2018. The study did not include the students. A
sample of this size limited the external validity and called into question the applicability of study
findings to other settings (Creswell & Creswell, 2018).
Further research is needed to determine whether the findings apply at other after-school
programs of mentors demonstrating improved knowledge and the use of effective and evidence-
based instruction proficiencies that, in turn, cultivate girls’ interest in computer science.
Research should examine whether quality programs are more likely to utilize partnerships within
the local community as a non-negotiable element to support learning and foster interest in the
discipline. Lastly, additional research is needed to learn whether after-school programs are
successful when access, sustained participation, program quality, and strong community
relationships are evaluated.
58
CHAPTER FOUR: FINDINGS
The purpose of this study was to evaluate the effectiveness of STEMaven mentors
necessary to achieve the goal of demonstrating improved knowledge and the use of effective and
evidence-based instruction strategies. This study employed the Clark and Estes (2008) gap
analysis model, with an analysis focused on knowledge, motivation, and organizational
influences related to achieving the organizational goal. This chapter first reviews the stakeholder
participants in the study, then outlines the findings and results from the survey and interviews in
relation to the research questions:
1. To what extent is STEMaven contributing to the development of effective and evidence-
based teaching skills in the mentors?
2. What are the STEMaven mentors’ knowledge and motivation related to improving their
effective and evidence-based teaching skills?
3. What is the interaction between STEMaven’s organizational culture and context and the
mentors’ knowledge and motivation to improving effective and evidence-based teaching
skills?
To address these research questions, the researcher developed a 13-item online survey
and conducted eight interviews. The data collection for this study took place over a 3-month
period. The Director of STEMaven supplied the email contacts of all mentors who had facilitated
a workshop since January 2018. A total of 34 emails were supplied. The list of emails was
current at the time of the study and contained both employment and personal addresses. For that
reason, the director sent the first email containing a short explanation of the study to all 34
contacts prior to the release of the survey. An email containing a link to complete the survey was
sent one day later and sent via Qualtrics. Over a 2-week period, surveys were collected via
59
Qualtrics. Upon the closing of the survey, mentors who had provided contact information
agreeing to participate in an interview were sent an email requesting participation. Only four
responded. Not having enough data through interviews, a second email was sent requesting
interviews. This resulted in five additional mentors, with four being interviewed. All interviews
took place using video conferencing.
Participating Stakeholders
The stakeholders of focus for this study were mentors at STEMaven. This population
consisted of 34 mentors, aged 18 and older, with experiences that ranged from current
undergraduate students to working professionals who hold terminal degrees in a STEM field.
This study sought to develop a sample of mentors who represent the entire mentor population at
STEMaven, yet excluded mentors with only a high school degree due to the desire to study those
with a professional STEM background. Therefore, this study, conducted in the fall of 2019,
quantitatively and qualitatively explored the research questions with mentors who had a
minimum of an undergraduate degree. In addition, each had facilitated at least one workshop
since January 2018. Mentors were surveyed confidentially and asked to provide an email address
if they agreed to participate in an individual, semi-structured interview. To qualify for the semi-
structured interview, they needed the additional requirements of a terminal degree in a STEM
discipline, had or currently have a career in a STEM field, and facilitated at least two workshops
since January 2018. This section offers an overview of the mentors who participated in the
survey, followed by a description of the interview participants.
Survey Participants
The quantitative survey, created in Qualtrics, was sent by email to the 34 participants
who held at least an undergraduate degree and facilitated at least one workshop since January
60
2018, followed up by two reminders. Figure 2 shows the highest level of education, followed by
the number of workshops each respondent has facilitated (Figure 3). The final data set included
18 mentors who completed the survey, representing a 53% response rate.
Figure 2: Survey participants’ highest level of formal education.
Figure 3: Total number of workshops facilitated since January 2018.
Among survey participants, 33% were undergraduates, 17% had earned a master’s
degree, and 50% had earned a doctoral degree. Disciplines were represented across survey
33%
17%
50%
Highest level of formal education
Undergraduate Degree Master's Degree Doctoral Degree
50%
17%
17%
10%
6%
Total number of workshops facilitated
1 2 - 4 5 - 7 8 - 10 11 or more
61
participants and consisted of metallurgy and materials engineering at 25%, Physics 25%,
computer science 25%, nuclear engineering 13% and mathematics 12%. In addition, seven of the
18 survey participants, or 39%, responded they had volunteered as a mentor at one or more after-
school programs outside of STEMaven prior to January 2018. The survey did not allow for an
open-ended field to respond regarding the specific type or location where they mentored. Of the
18 participants who completed the survey, 83% were female, while 17% were male.
Interview Participants
As stated in Chapter Three, a total of eight mentors participated in interviews. After the
completion of the survey, participants had the option to offer their email address if they wanted
to be interviewed. The researcher contacted each participant based on the interview sampling
criteria and rationale, contacting those who completed the survey, held a terminal degree in
STEM discipline, had or currently held a position in a STEM field, and facilitated at least two
workshops since January 2018. Nine mentors responded, supplied their email address, and were
emailed requesting a day and time to be interviewed. In total, eight mentors responded and held
an interview, totaling 44% of the survey population. Of the eight participants, seven were female
and one male. In addition, two had employment history as a scientific researcher in a STEM field
that was affiliated with a University, with the remaining six respondents having employment
history as a scientific researcher at a National Laboratory in a STEM field. The interviews took
approximately 45 to 60 minutes each to complete. Figure 4 shows representation by terminal
degree and gender. Table 6 provides interview group composition. All names are pseudonyms.
62
Figure 4. Interview participants composition of gender and terminal degree.
Table 6
Interview Group Composition
Pseudonym Gender Employment Type Number of Workshops
Ryan Male National Laboratory 10+
Melissa Female National Laboratory 3
Anna Female University 2
Jessica Female University 4
Angela Female National Laboratory 7
Sarah Female National Laboratory 2
Debra Female National Laboratory 2
Lisa Female National Laboratory 6
Findings
This section reports on the results of the survey and the findings of the interviews as they
relate to the research questions. Thus, the results and findings are reported through the distinct
lenses of knowledge, motivation, and organization influences identified in the conceptual
framework and the literature. The data collected from surveys and interviews either validated or
did not validate the assumed influences of the study. In the following sections, an influence that
is considered validated means that the assumed influence had substantive evidence from the
25%
12.5%
25%
12.5%
25%
88%
12%
0 10 20 30 40 50 60 70 80 90 100
Physics
Math
Computer Science
Nuclear Engineering
Metallurgical and Materials Engineering
Female
Male
Gender and terminal degree
63
survey and interview data collection to be identified as a gap for the stakeholder or organization.
An influence that was not validated means that there was not enough evidence from the survey
and interviews for the it to be identified as a gap in the stakeholders or organization. The chapter
concludes with a discussion of these results and findings.
Knowledge Findings
When an organization invests in resources to increase employees’ knowledge and skills,
there is a positive long-term impact on the organization’s productivity (Clark & Estes, 2008).
There are four knowledge types necessary for goal attainment: factual, conceptual, procedural,
and metacognitive (Clark & Estes, 2008; Krathwohl, 2002; Rueda, 2011). Table 7 reports the
complete list of assumed knowledge influences for this study. Procedural knowledge is the
understanding of methodologies and techniques related to how to do something, Mentors who
facilitate workshops at STEMaven need procedural knowledge, such as demonstration or
modeling, that show effective and evidence-based skills that employ culturally responsive
learning techniques. Culturally responsive learning techniques are methods used by teachers that
encourage students to relate course content to his or her cultural context (Wlodkowski &
Ginsberg, 1995). Mentors also need metacognitive knowledge, such as the awareness of one’s
cognition and having the ability to recognize when and why to undertake a task (Rueda, 2011).
Mentors can apply metacognitive knowledge when they become aware of a student struggling
with a concept and recognize the need to quickly provide a real-world example that fits the
student’s cultural context to help them relate the course material to a familiar viewpoint.
It is not about just finding the assets of a student, but, rather, being aware of when to
pivot during a workshop and having the ability to demonstrate optional teaching strategies so all
students can reach their full potential. This type of metacognition allows for mentors to build
64
enthusiasm, strengthen relationships, and in turn, increase content comprehension. As outlined in
Table 7, two of these knowledge types, procedural and metacognitive, were assumed influences
for STEMaven’s goal of mentors demonstrating improved knowledge and use of effective and
evidence-based instruction proficiencies with both knowledge types validated as gaps:
procedural and metacognitive.
Table 7
Knowledge Influences
Assumed Knowledge Influence Validated as a Gap?
Mentors do not know effective and evidence-based skills that
employ culturally responsive learning techniques. (Procedural)
Validated
Mentors do not understand how to demonstrate optional
strategies of teaching so all students can reach their full potential
when they encounter deficit thinking. (Metacognitive)
Validated
Procedural knowledge of effective and evidence-based skills that employ culturally
responsive learning techniques. With each additional workshop, mentors can exercise
procedural knowledge, which is crucial in the development to effectively facilitate a workshop
that utilizes culturally responsive learning techniques. Clark and Estes (2008) stated that people
need practice and corrective feedback to help them achieve specific work goals when it is
directly applied to a task to solve a performance problem. It consists of knowing how to do
something, the methods of inquiry, and criteria for using those skills, techniques, and methods.
Survey participants were asked to rate their knowledge and application of effective and
evidence-based teaching skills that employ culturally responsive teaching techniques while
facilitating a workshop. Over 60% of participants surveyed stated they strongly agreed or agreed
that they employ culturally responsive learning techniques and understand what skills are needed
65
to employ a culturally responsive lesson. Furthermore, 39% stated that they disagreed or strongly
disagreed. Figure 5 illustrates these results.
Figure 5: Ability to employ culturally responsive learning techniques and skills.
These findings suggest that there is still a need for increased knowledge related to
effective and evidence-based teaching skills that employ culturally responsive learning
techniques. Of the eight interview participants, three did not have this knowledge and could not
reflect on using this skill. For example, Debra supplied a response of “I am still unclear on how
to do this. [STEMaven] is trying to tailor the workshops to encompass strategies and lessons that
fit the student’s cultural context.” Similarly, Ryan replied,
That is a difficult one. I believe there are two aspects to answer this question. One is
socioeconomic background. We have a mix of foreign national [students] but I haven’t
really observed the biggest difference of shy vs. outgoing. I don’t think one behavior is
attached to one group being in there, unless it is obvious were they are from.
Having the prior knowledge of culturally responsive teaching techniques can be key to
reaching the students. Angela put it this way,
11%
50%
39%
0%
11%
50%
33%
6%
0 10 20 30 40 50 60 70 80 90 100
Strongly Agree
Agree
Disagree
Strongly Disagree
Ability to employ culturally responsive techniques
During my workshop, I employ culturally responsive learning techniques.
I understand what skills are needed to employ a culturally responsive lesson.
66
I actually think that is one of the biggest aspects of how we learn. The fact that these girls
are around other girls and a lot of mentors are female, that helps a lot and they seem more
open to ask questions and be engaged. I’ve seen in the past if students are not around
other girls, they shut down and don’t open up as much. How do you include this in your
lesson(s) to make them compatible with the social cultural contexts of middle school
girls?
On the other hand, Anna said “I can bring tricks on how I learned a topic or how someone taught
me different ways to learn a topic and how they were able to relate this back to a personal
experience.” Melissa mentioned, “When the students have an idea on how the workshop could be
more interactive, the mentors become true facilitators of that interaction.” The five participants
that could employ a method, felt that it contributed to an overall positive experience to the
students. Sarah declared, “I can relate and focus on the fun aspects of the workshop and
recognize that the students are also having fun.” She continued with, “You have to approach this
supplemental learning in a way that gears them toward interest.” A mentor who had only
facilitated two workshops stated,
Pop culture can influence learning, especially at this age and any cultural context. So, just
as pop culture can have a big impact on interest, I show up as a scientist that is young,
female, and extremely helpful to their needs. Hopefully, they see the correlation!
The comments from Anna and Sarah showed the importance of procedural knowledge;
mentors who facilitate more workshops gain experiential learning that supports effective and
evidence-based skills and techniques can be used to teach a culturally responsive workshop. This
mirrors research by Hynes (2012) who noted that teachers in after-school programs gain
knowledge by teaching lessons. They enhance their teaching strategies and content knowledge
67
by using examples or analogies students understand. While none of the interviewed participants
indicated that STEMaven had a structured professional development in this area, they did
recognize that this type of learning would be impactful to both the body of mentors and student’s
success. Jessica stated,
There is not a structured setting of support such as professional development, but I was
introduced to a process called Dynamic Programming. It is a way of understanding how
to explain things in the most simplistic fashion by breaking down problems into more
simpler terms.
The data from the survey and interviews in this study validated that this assumed
influence is a gap at STEMaven. The majority of participants questioned if there was a structured
method of modeling, they would be better prepared to relate the lesson to a student’s cultural
context. The next section discusses the metacognitive knowledge influences explored in the
participant interviews.
Metacognitive knowledge of mentors ’ skill strength in deficit thinking. Metacognitive
knowledge refers to the realization of mentors’ cognitive processes and allows them to reflect on
why they are to undertake a task and when to do it, which is an important factor of strategic
behavior in approaching problems (Rueda, 2011). This is directly related to deficient thinking,
which is an approach, often unconscious, that views those of different identities (often racial,
gender or other) as less able to succeed (Valencia, 1997). Survey participants were asked to rate
their ability to define deficit thinking. Only 50% of survey participants answered that they
strongly agreed or agreed that they could define deficit thinking. Figure 6 illustrates these results.
68
Figure 6: Survey and response: I can define deficit thinking.
The inability to define deficit thinking verbatim does not necessarily mean that one does
not have the notion of a student, due to their race, ethnic or socioeconomic background, will
struggle or fail with the course material. Moreover, they blame the student, not the structure of
the lesson, for thwarting the learning process. When mentors can reflect on strategic behavior
and techniques that increase the comprehension of a student, they are realizing their cognitive
processes and reflecting on why they are to undertake a task and when to do it.
In contrast to the survey participants, none of the interview participants could define
deficit thinking. However, once the definition was read to them, four could provide examples of
the cognitive process in relation to deficit thinking during their facilitation of a workshop. The
other four interviewees could not provide examples of demonstrating optional teaching strategies
due to facilitating a low number of just two or three workshops. Lisa, who has facilitated six
workshops, stated, “I quickly pivot my teaching to relate to the students and keep their interest.”
When probed to give a better understanding of how pivoting during her workshops refers to
deficit thinking, Lisa explained,
11%
39%
33%
17%
I can define deficit thinking
Strongly Agree Agree Disagree Strongly Disagree
69
I am aware when a student is not understanding concept and begins to losing interest in
the lesson. I go directly to that student, kneel down to their level or sit next to them, and
work with them one-on-one using different strategies that help us both focus on their
knowledge level and get them to a point of comprehension.
Lisa’s comments illustrate the importance of metacognitive knowledge to analyze one’s own
ability in relation to deficit thinking.
Jessica also noted the importance of understanding how to demonstrate optional
strategies of teaching so all students can reach their full potential when she encounters deficit
thinking. As a mentor who has facilitated just four workshops, she noted that she has made a
practice of reflection after each workshop so she can gain a better understanding of how to reach
the students that struggle the most. She noted,
I always learn and reflect from each workshop with the co-mentor. During the workshop
we try to relate modern everyday things on their level, like social media or things they
would do on an everyday basis. After the workshop, we reflect on additional real-life
examples that could have been utilized and be better able to relate at a deeper level.
This metacognitive knowledge helps mentors adapt to the ways they think and operate to be
more effective (Krathwohl, 2002).
The thought process of both Lisa and Jessica illustrated the type of metacognitive
knowledge necessary to achieving the realization of mentors’ cognitive processes that allows
them to reflect on why they are to undertake a task and when to do it. Perhaps Ryan, who has
facilitated well over 10 workshops throughout his tenure, summed up the critical importance of
metacognitive knowledge best when he said,
70
I don’t think about this carefully enough and I do not consciously believe that I have
deficit thinking. Usually, one of the elements I start with is where we have common
ground. I start [the workshop] where the majority can relate to the topic and get them to
talk about it with common knowledge and vocabulary and how this is related to real life.
Learning their perspective on things. Being as inclusive as can be. Makes it seem as if the
workshop if for everyone. Wonder if there are more ways to learn about this and if it
really helps?
Clearly, there is a development of metacognitive knowledge around understanding how
to demonstrate optional strategies of teaching so all students can reach their full potential when
they encounter deficit thinking. One becomes more effective at a task after reflecting on the way
one has performed it before (Di Stefano et al., 2014), which is directly related to a more thorough
understanding of the cognitive processes related to deficit thinking.
The data from the survey and interviews in this study validated the assumed influences as
noted in Table 7. The procedural and metacognitive knowledge needed to be successful in
capturing the interest and retention of middle school girls in computer science is multifaceted
and complex. Conversely, having just the comprehension of content is not enough, one must also
have the motivation of capturing middle school girls’ interest in computer science. The next
section discusses the validation of the motivational findings from the current study.
Motivation Findings
Motivation is the second dimension examined in this study. Clark and Estes (2008)
explained that motivation is centered on an employee’s personal beliefs about themselves and
their coworkers and that motivation encompasses active choice, persistence, and mental effort.
Self-efficacy, collective efficacy, interest, expectancy value, goal orientation, and attributions
71
influence motivation (Rueda, 2011). The motivation influences listed in Table 8 reflect the
complete list of assumed influences before data collection. Assumed motivational influences for
mentors at STEMaven are goal orientation and self-efficacy. By assessing and addressing
motivational challenges, mentors can begin the process of closing the gaps between their current
performance levels and their performance goals. When mentors’ performance gaps are closed,
the organization is closer to achieving its organizational goals (Bandura, 2000).
Table 8
Motivation Influences
Assumed Motivation Influence Validated as a Gap?
Mentors lack the satisfaction and motivation of the stakeholder goal
to facilitate a STEMaven workshop using improved knowledge and
use of effective and evidence-based instruction proficiencies. (Goal
Orientation)
Validated
Mentors lack efficacy while teaching a computer science related
workshop to middle school girls. (Self-efficacy)
Not Validated
Goal orientation of effective teaching skills. According to goal orientation theory, goal
commitment and implementation objectives are fundamental to achieving goal-directed behavior.
Setting goals and having reasons to achieve goals is one of the motives for trying to accomplish
and achieve tasks (Pintrich, 2003). If mentors are motivated and were willing to recognize their
lack of goal orientation toward the stakeholder goal, then identifying areas of weakness and
working toward improvement would be easier with the help of peers. Figure 7 reviews responses
to the survey question asking participants if they were satisfied with their facilitation of a
STEMaven workshop. Ninety-six percent of participants responded that they are very satisfied or
satisfied. These data represent mentors’ motivation to continually demonstrate knowledge and
the use of effective and evidence-based instruction proficiencies. In contrast, to the almost 100%
72
response rate in satisfaction of teaching a workshop, only 72% of participants reported that they
were very motivated to see their students succeed, represented in Figure 8.
Figure 7: Survey and response: Overall, how satisfied are you with facilitating a STEMaven
workshop?
Figure 8: Survey and response: How motivated are you to see students succeed?
50%
44%
6%
0%
0 10 20 30 40 50 60 70 80 90 100
Very satisfied
Satisfied
Neutral
Dissatisfied
Satisfaction with facilitating a STEMaven workshop
72%
28%
0%
0%
0 10 20 30 40 50 60 70 80 90 100
Very Motivated
Somewhat Motivated
Not Very Motivated
Not At All
Motivation to see students succeed
73
Interview data further expanded the survey data when asked about goal orientation in
regard to satisfaction and motivation of facilitating a workshop using improved knowledge and
use of effective and evidence-based instruction proficiencies. Anna replied,
It is very cool to see the students get excited about coding and see what they pick up on.
Watching the lightbulb go off and hearing the students shout sometimes that they got the
right answer, that is my favorite part and my motivation.
Lisa, reflecting on her experience replied, “I would like see them come back as mentors one day,
and, to me, I say I did a great job.” Jessica summed it up when she said,
I want kids to have the ability to see themselves in the future. If they can see a woman in
power and successful, they would then see themselves in this person. Girls are
discouraged to work hard and I want to see them be successful in their future. Most
people in power in the U.S. are white males. I want them to see that they can look like
me.
All of the interview participants cited fulfillment and motivation of teaching but could not
provide direct correlation to improved knowledge and use of effective and evidence-based
instruction proficiencies. However, interview participants did view effective teaching as vital to
the success of retention and interest of middle school girls in computer science. These data
provide a window into the lack of mentor’s goal orientation and validates the assumed influence.
If mentors are not continually motivated to see their students succeed, there would not be an
attrition of mentors facilitating workshops and thus a potential drop in students attending the
STEMaven program.
Individual self-efficacy for increased confidence of employing effective teaching
skills. Self-efficacy theory posits that confidence increases when belief in one’s own ability to
74
successfully accomplish a task, or efficacy, is high (Pajares, 2006). When mentors believe they
can be successful, their performance increases and students receive the benefits of achievement.
Richter et al. (2012) found when an employee can harness their self-efficacy, they would
proactively access resources such as team members’ knowledge, expertise, and insights.
Participant survey and interviews found evidence of self-efficacy in the STEMaven mentors’
while teaching a computer-science related workshop to middle school girls. Figure 9 summarizes
responses to the survey question asking participants if they are confident in their ability to teach
middle school girls in computer science. Ninety-six percent of participants responded that they
agree or strongly agree with this statement. Moreover, 89% of participants agreed or strongly
agreed that they could successfully navigate the aspects of a workshop with which they had little
previous experience (Figure 10).
Figure 9: Survey and response: I feel confident in my ability to teach middle school girl’s
computer science.
44%
50%
0%
6%
0 10 20 30 40 50 60 70 80 90 100
Strongly agree
Agree
Disagree
Strongly disagree
Confidence in teaching
75
Figure 10: Survey and response: I can navigate the aspects of the workshops with little/no
previous experience.
Similar to the survey questions, the interviews revealed the same responses. Angela, who
has facilitated seven workshops, said, “I like to learn. I want the kids to have the ability to see
themselves in the future. This is what builds my confidence each time.” Similarly, Melissa, who
has facilitated three workshops, stated,
It has been great. It has been fun and I have learned from it as well. It has been really cool
to see them get excited about coding and see what they pick up on. When students say,
“We got the right answer!” that is my favorite part!
Self-efficacy is a key motivational factor that influences mentors in successfully
facilitating each workshop. Not only does the data from the survey illustrate evidence of self-
efficacy, but the voices of each interview participant assisted in not validating the assumed
influence in regard to self-efficacy as noted in Table 8. Goal orientation, on the other hand, was
validated through both the survey and interview populations (Table 8). Goal orientation and self-
efficacy in relation to motivation are equally important to capture the interest and retention of
middle school girls in computer science. In addition to knowledge and motivation, organizations
50%
39%
11%
0%
0 10 20 30 40 50 60 70 80 90 100
Strongly Agree
Agree
Disagree
Strongly Disagree
Ability to navigate a workshop
76
influence the ability to achieve a goal. The next section looks at the organizational influences and
their validation.
Organizational Findings
Culture plays a significant role in organizations (Clark & Estes, 2008). For a change
effort to be successful, solutions must consider and adapt to the organizational culture (Clark &
Estes, 2008). Ultimately, for STEMaven mentors to achieve their goal and implement effective
teaching strategies that increase the interest and retention of middle school girls in computer
science, issues related to organizational culture must be addressed alongside the previously
outlined knowledge and motivation influences. The organization needs a culture that supports
strengthening teaching strategies, founded on collective engagement, shared purpose, and
collaboration to aid in motivating and retaining the interest of the participants. Therefore,
organizational influences are a vital aspect of gap analysis. The organizational influences listed
in Table 9 reflect the complete list of assumed influences and validation based upon data
collection. The list includes two assumed cultural model gaps and two assumed cultural setting
gaps.
77
Table 9
Organizational Influences
Assumed Organization Influence Validated as a
Gap?
Cultural Model Influence 1:
The organization needs a culture that supports change in existing teaching
strategies founded on collective engagement, shared purpose, and
collaboration to aid in motivating and retaining interest of the participants.
Validated
Cultural Model Influence 2:
Cultural Trust: There needs to be a culture of trust in the organization
between leadership and mentors.
Not Validated
Cultural Setting Influence 1:
Support systems: STEMaven needs professional development that is
centered around effective and evidence-based strategies.
Validated
Cultural Settings Influence 2:
Mentors and models: Mentors need effective role models who have
succeeded in increasing interest, confidence, and perceptions of computer
science.
Validated
In the survey, 67% of participants stated that STEMaven provides professional
development that is centered around new teaching strategies a great deal or somewhat as shown
in Figure 11. Figure 12 shows that 45% of surveyed participants believed a great deal or
somewhat that professional development is beneficial to both the mentor and the organization.
78
Figure 11: Survey question and response: The organization provides professional development
that is centered around new teaching strategies.
Figure 12: Survey question and result: The professional development is beneficial to both
yourself and the organization.
In tandem to the survey responses, interview participants did not believe that the
organization provided professional development and similar comments were heard from each in
regard to what they thought should be the focus of their teaching development. Debra stated, “I
would love to see consistent instructor professional development where all mentors are invited to
22%
17%
39%
22%
0 10 20 30 40 50 60 70 80 90 100
A great deal
Somewhat
A little
Not at all
Professional development centered around new teaching
strategies
28%
28%
28%
16%
0 10 20 30 40 50 60 70 80 90 100
A great deal
Somewhat
A little
Not at all
Professional development is beneficial
79
one location. It probably should be focused on training on their role, not really for behavior.”
Anna expressed,
I think the idea of a singular location to share ideas. I have done this with my current
research and it has been very beneficial. I especially like Dropbox and Google Drive to
collaborate. I think this would accelerate the mentoring process of ideas no matter the
distance.
Three interview participants similarly stated that they were not aware of any professional
development or observations that can provide feedback to strengthen teaching strategies. Angela
replied, “One thing I don’t have much experience in is working with middle school girls.” After
concluding her response, she asked, “Are there ways the program can improve its impact on my
development?” To further her thoughts, she stated,
One thing I know is that [STEMaven] hasn’t really provided the mentors additional
support or strategies that help with the [workshops] that they help out in over and over
again. We never have meetings or a call to learn from each other except the other mentor
you taught with. It would be very helpful if we had access to see what techniques work
and see how this can help.
Ryan, thinking about ways that could improve the organization responded with,
There needs to be more interaction with volunteers. There needs to be more scaling and
allowing more mentors to run more as instructors, not just a few people that take the lead.
It can be a little controlling. Professional development is needed to show the mentors a
checklist they can run with confidence and that they can effectively teach, not just be in a
mentor role. So yeah, mentors do need more training. Maybe as team of peers and
training.
80
As we place organizational culture in dialogue with knowledge and motivation, and
address them simultaneously, change efforts are more successful (Clark & Estes, 2008). The data
from the survey and interviews in this study validated all but one the assumed influences as
noted in Table 9. STEMaven should recognize the need to change the culture and support
mentors throughout their tenure. The organization needs to update the existing teaching
strategies and allow for collective engagement, shared purpose, and collaboration to aid in
motivating and retaining interest of the participants. The organization needs to provide
professional development that is centered around effective and evidence-based strategies. In
addition, the organization needs to provide mentors with effective role models who have
succeeded in increasing interest, confidence, and perceptions of computer science.
The next section explored additional themes found in the data.
Themes
STEMaven aims to increase the knowledge of its mentors to successfully utilize
effective and evidence-based strategies necessary to achieve the goal of instruction that
motivates and retains middle school girls’ interest in computer science. The prior section
examined the knowledge, motivation and organizational influences that affect how study
participants perceive their teaching strategies. This section explores themes identified in the data
that offer additional insight.
Among the important themes that emerged from this study is the importance of
professional development. Mentoring is different from effective teaching. Having the knowledge
of effective and evidence-based teaching strategies improves their teaching, which increases their
mentoring of middle school girls.
81
Theme 1: The importance of professional development. When surveyed, 17 of 18
participants agreed or strongly agreed that they feel confident about their ability to effectively
teach middle school girls computer science. Yet, only four out of eight interview participants
reported that they had very little prior knowledge of effective and evidence-based strategies to
utilize while facilitating a workshop. In addition, over 80% of surveyed participants noted that
STEMaven does not provide professional development, which is noted in the literature review as
key to developing the knowledge and skills needed to address learning challenges. Ryan, who
shared a list of professional development strategies he thought would be useful for mentors at
STEMaven said, “[would help foster] best practices on how to communicate and interact with
middle school-aged girls from all socioeconomic backgrounds and how to identify real-life age-
appropriate examples that can be used in the workshops.” Moreover, all interviewed participants
agreed that effective and evidence-based lessons are imbedded in each workshop and that
STEMaven wants to increase the retention and interest of computer science for students. Debra
said, “professional development is needed to show us and give us a checklist that we can run
with confidence. We need training. Maybe as a team of peers.” Moreover, Jessica added,
“Mentors do a great job of how the workshop will flow but as far as how to work one-on-one and
capture a deeper interest, we need additional training to make it even better.” As presented in the
literature, teachers in after-school programs gain knowledge by teaching lessons (Hynes, 2012).
They enhance their teaching strategies and content knowledge by using examples or analogies
students understand. After-school programs should provide an opportunity for teachers and
mentors to enhance their professional skills and content knowledge.
Theme 2: Mentors recognized the need to employ culturally responsive teaching
strategies. Survey data showed that 59% of participants understood what skills are needed to
82
have a culturally responsive lesson, and, supporting this finding, the majority of interview
participants were not able to describe what they do to incorporate these strategies during each
workshop. In contrast, 71% of survey participants responded that they take the time to learn
about the background of their students and learn why they are participating in a workshop. When
mentors in after-school programs have this frame of mind, their expectations of their students are
high and therefore impact student success more than a student’s own motivation (National
Center for Education Statistics, 2018). An overwhelming theme from the interviews provided a
nuanced understanding that mentors did seem confident that they promote positive learning
experiences.
Theme 3: Mentors acknowledged the need for self-efficacy for increased confidence
in employing effective teaching skills. Both survey and interview participants overwhelmingly
voiced they understood the importance of having confidence in their ability to facilitate a
successful workshop, and six out of eight interviewed participants agreed that the organization
should implement an ongoing and collaborative approach to reflection and feedback that occurs
after the completion of a workshop. Bandura (2000) asserts that, if there is perceived collective
efficacy, there is an impact on performance, commitment to the mission, and the ability to deal
with challenges. Angela stated, “mentors do not have a specific set of expectations for self-
growth communicated by and from the organization upon onboarding or during their
mentorship.” When mentors believe they can be successful, their performance increases and
students receive the benefits of achievement. Additionally, when asked if there are ways the
organization can improve its impact on mentor self-awareness, Sarah mentioned, “One thing I
know they haven’t really done for the mentors that teach the same class, is we never have
meetings or a call to learn from each other except from the other mentor you taught with.” Clark
83
and Estes (2008) stated that people need practice and corrective feedback to help them achieve
specific work goals.
Summary
This chapter has presented the results of the survey and the findings of the interviews as
they have related to the first three research questions. The discussion included both the assumed
influences presented in Chapter Three as well as the conceptual framework and the
corresponding literature. The findings offer a unique look into the potential the organization
seeks to provide increased opportunities through effective and evidence-based instruction and
mentorship for middle school girls to engage in workshops that motivate and retain their interest
in computer science.
It is not surprising that the data would reveal the mentors believe they are confident in
teaching a computer science skill, that they know how to be self-aware, but lack the skills of
effective and evidence-based teaching. Although there is a lack of professional development in
effective and evidence-based teaching strategies, this has not led to a decrease in disengagement
of students and frustration on the part of mentors.
One particularly unexpected finding in the study was how few participants in the survey
responded that they could not define deficit thinking. The same lack of knowledge occurred
during the interview. When given an example of deficit thinking in the classroom, all participants
answered confidently, not to the definition of deficit thinking but to examples of themselves
being able to identify assets in a student. They also provided examples of being aware of when to
pivot during a workshop and being able to demonstrate optional strategies of teaching so all
students can reach their full potential. While there were no major differences found in the mentor
responses, for the most part, one thing mentors shared was their ability to employ culturally
84
responsive learning techniques during their lesson(s), and overwhelmingly, mentioning that
STEMaven adds to their rating on this topic. For STEMaven to truly nurture actively engaged
learners, these markers must be defined. Though while the number of workshops facilitated on
background knowledge of a STEM discipline and amount of time mentoring at STEMaven
differ, engaged learning and the characteristics of the engaged mentor do not appear to
discriminate.
Chapter Five offers an outline of such a framework. The chapter makes recommendations
for the results and findings and provides an answer to the final research question, “What are the
recommendations for STEMaven’s practice in the areas of knowledge, motivation, and
organizational resources to increase the interest, confidence, and perceptions in computer
science?” The suggested interventions are offered based on the identified influences found
through both modes of inquiry. These recommendations offer a systems approach to how
STEMaven can support mentors as a collective force so they may, in turn, create the conditions
for middle school girls to increase their interest in computer science.
85
CHAPTER FIVE: DISCUSSION
This chapter discusses the assumed knowledge, motivation and organizational influences
related to achieving the organizational goal. Findings from this study showed there was gap in
knowledge on effective and evidence-based teaching skills to use while facilitating a computer
science workshop for middle school girls. Participants had similar motivational and
organizational influences that limited their proficiency and their goals for successfully
facilitating a computer science workshop. This chapter discusses several recommendations for
improving practices, organized by the categories of knowledge, motivation, and organization.
The new Kirkpatrick and Kirkpatrick (2016) methodology was used for an implementation and
evaluation plan that includes reaction, learning, behavior, and results. The next section offers a
concise review of the organizational goal, stakeholder groups and the purpose of the project.
Organizational Context and Mission
STEMaven is a 501(c)(3) nonprofit corporation founded in January 2017 to offer a
variety of workshops for middle school-aged girls to explore technology, coding, and science.
Their mission is to inspire middle school girls in East Tennessee to actively explore the fields of
technology, to close the gender gap in the technology profession, and to foster participants’
future careers. Additionally, STEMaven hosts 3-hour face-to-face workshops facilitated by
mentors who have background knowledge in computer science or a technology-related skill.
Since starting in 2018, STEMaven has continually grown in terms of number of participants
each year and hosted 500 to 600 middle school girl participants (STEMaven, 2019).
Organizational Performance Goal
STEMaven wants to inspire and encourage local middle school girls to pursue careers
related to computer science by providing free hands-on coding and technology-related
86
workshops. By December 2020, mentors at STEMaven will demonstrate improved effective and
evidence-based instruction proficiencies. The improvement of effective and evidence-based
instruction proficiencies will result in an increased ability to teach their workshops and
encourage the cultivation interest in computer science and aid in closing the gender gap in the
technology profession.
Description of Stakeholder Groups
Three stakeholder groups play a role in the goal of this study. The STEMaven program
director selects the workshops, recruits the mentors, and manages the program’s overall
performance. The mentors facilitate the learning of the curriculum, revise workshops to adapt to
the students’ learning level, and encourage the development of interest in computer science to
ensure progress and growth of attendance rates. The student participants partake in workshops
designed to embolden and retain an interest in computer science.
Goal of the Stakeholder Group for the Study
While the joint efforts of all stakeholders contributed to the achievement of the overall
organizational goal of providing increased opportunities for middle school girls to engage in
workshops that motivate and retain their interest in computer science, it is important to evaluate
the gaps in mentor knowledge of effective and evidence-based teaching strategies. Therefore,
the stakeholders of focus for this study were the mentors at STEMaven. The mentors
contributed to the achievement of the organization’s performance goal by facilitating workshops
that increase interest, confidence, and perceptions of computer science by the end of November
2019. The program plays a significant role in bridging the gender gap in science, technology,
engineering, and mathematics and improves the students’ problem-solving skills.
87
Purpose of the Project and Questions
The purpose of this study was to evaluate the influences that affect how STEMaven
mentors offer instruction that motivates and retains middle school girls’ interest in computer
science. The analysis focused on knowledge, motivation, and organizational influences related
to achieving this organizational goal. While a complete needs analysis would focus on all
STEMaven stakeholders, for practical purposes, the stakeholders in this analysis were all
STEMaven mentors.
1. To what extent is STEMaven contributing to the development of effective and evidence-
based teaching skills in the mentors?
2. What are the STEMaven mentors’ knowledge and motivation related to improving their
effective and evidence-based teaching skills?
3. What is the interaction between STEMaven’s organizational culture and context and the
mentors’ knowledge and motivation to improving effective and evidence-based teaching
skills?
4. What are the recommendations for STEMaven’s practice in the areas of knowledge,
motivation, and organizational resources?
Recommendations for Practice
The recommendations provide pathways for STEMaven to increase the knowledge and
motivation of mentors in employing effective and evidence-base instruction proficiencies.
Utilizing the recommendations, mentors will have an increased knowledge base to foster these
proficiencies, reflect on their self-awareness, and successfully facilitate their workshops to
increase interest in computer science for middle school girls. Moreover, with offering
professional development that allows mentors access to these skills, the organizational culture
88
will follow with a focused structure on development of these skills for mentors throughout their
time mentoring at STEMaven.
Recommendation One
The first recommendation is to increase the mentors’ knowledge about incorporating
effective and evidence-based strategies to teach middle school girls’ computer science. When
surveyed, mentors signaled that they were confident in teaching their workshops, though
interviews suggested that they need more in-depth procedural knowledge of how to incorporate
effective and evidence-based strategies to teach middle school girls. Allison and Rehm (2007)
found that teachers need to understand effective and evidence-based strategies to transfer skills
and knowledge into the curriculum they are teaching to enhance the learning of computer science
for middle school girls. This would suggest that providing mentors with access to digital and
print resources would help them increase their knowledge of current best practices. The
recommendation is to provide STEMaven mentors with access to these resources that explain
effective teaching strategies. The use of visual aids and hands-on learning materials would define
what practices are effective in retaining interest in computer science.
Shinn (1997) studied select principles of teaching and learning to determine the
relationship between teaching strategies and their effective use of instructional methods and
tools. Incorporating methods and tools during class time showed significant gains for teachers in
procedural knowledge achievement. The study supports the recommendation to provide
STEMaven mentors with access to these resources that explain effective teaching strategies.
Recommendation Two
The second recommendation is to increase the mentors’ knowledge on how to best reflect
on their skill strength in deficit thinking. Mentors need more in-depth metacognitive knowledge
89
on how to reflect on their effectiveness in teaching middle school girls in computer science with
regard to deficit thinking. A recommendation on the use of self-monitoring has been selected to
close this metacognitive knowledge gap. Akl et al. (2007) found that creating goals where
mentors can reflect on their workshops can have an impact on teaching and enhancement of
those skills. This would suggest that providing mentors effective strategies on how to apply the
knowledge and subsequently reflect on their effectiveness in utilizing deficit thinking would
increase the knowledge in this area. The recommendation is to have the mentors keep a journal
of their reflective thoughts and examination of methods they could have done differently during
the development and implementation of a workshop.
Di Stefano et al. (2014) reported one becomes more effective at a task after reflecting on
the way one has performed it before. The team conducted three studies based on the dual-process
theory of thought, which is when a decision is made using two different systems of thinking,
such as our intuition and conscious thought. The study found that when a group used reflection
and sharing through group talk, they performed, on average, 18% better at reflecting on the who,
what, when, where, and why. These reflective meetings can have a positive impact on group
morale and learning. Helping mentors reflect on their strengths and weaknesses creates
metacognition, which is a major factor of strategic behavior in approaching problems (Rueda,
2011) which is related to effective teaching skills.
Recommendation Three
The third recommendation is to expose mentors to best practices on how to evaluate an
effective workshop. Goal orientation theory focuses on mastery, individual improvement,
learning, and progress that promotes positive motivation. If mentors have the tools needed to
assess whether a workshop was successful, they have a better understanding of what skills are
90
needed to enhance their teaching. When a community of learners is created, motivation to
improve occurs (Yough & Anderman, 2006). When the organization employs strategies based on
strong goal orientation, productivity increases, with mentors reaching the goal along with
utilizing current resources and skills. In addition, strong goal orientation allows mentors to see
their personal contribution to the overall goal. The recommendation is to provide a job aid in the
form of a procedural map that outlines steps to evaluate a workshop to increase knowledge on
how to best evaluate a workshop and improve teaching practices. The process map must include
a flow diagram that allows the mentor to document every step of the evaluation process. This
allows for mentors to visualize the details of the evaluation process closely and in turn help
mentors identify strengths and weaknesses in their current teaching.
Mentors who are familiar with evaluation practices often equate the quality of their
workshop to the level of support, knowledge, and motivation of this process, and the capabilities
and proficiency of the coordinator (Lundh et al., 2013). Having the ability to dissect a workshop
is essential in ensuring the mentors understand the steps and techniques necessary to effectively
teach a workshop. Clark and Estes (2008) found that people need practice and corrective
feedback to help them achieve specific work goals. Understanding how to successfully evaluate a
workshop is a major factor in successfully facilitating after-school workshops that increase
interest and retainment of middle school girls in computer science. Having a structured
evaluation plan is one way mentors can learn effective strategies and techniques to increase the
engagement of students and encourage interactive learning.
Recommendation Four
The fourth recommendation is to increase the self-efficacy of mentors so they can
confidently facilitate a culturally responsive workshop. Self-efficacy theory focuses on personal
91
beliefs, expectations about one’s own capability to organize and implement. Motivation needs to
be supported by good evidence in line with one’s conceptual reasoning (Pintrich, 2003). When
the mentors are exposed to situations where they need to employ culturally responsive teaching
techniques, they become more aware of the skills needed and have increased confidence to
effectively apply this practice. The recommendation is to provide case studies describing
evidence-based practices that demonstrate the success of teaching middle school girls in similar
after-school computer science-based programs. Upon the implementation of evidence-based
practices to increase the mentors’ culturally responsive skills, mentors begin to grow their
toolbox of skills and successfully facilitate a workshop to teach to various cultural differences.
Borgogni et al. (2011) stated that, to strengthen ones’ self-confidence, one must work on
self-efficacy and perception. According to Rueda (2011), individuals with higher beliefs in their
competency and higher expectations of their role are more motivated to participate and retain
their tasks. With more applied practice in using culturally responsive teaching techniques,
mentors will be more motivated to improve their skills and be better prepared to effectively
facilitate a workshop.
Recommendation Five
The fifth recommendation is that the organization should encourage and foster a culture
that supports change in existing teaching strategies through professional development. The
organization focuses on the core content for each workshop, that often the effective and
evidence-based teaching strategies are not incorporated in the learning experience. If mentors
were able to implement new evidence-based teaching strategies in a workshop, it would make
them more effective teachers because they would be using best practices that fit their level of
skill. With the application of knowledge improvement of teaching skills will follow. This can
92
occur through the encouragement and fostering of a culture that supports new evidence-based
teaching practices. Productive support systems targeting the specific levels of individual teachers
increase peer support, roles, and job satisfaction (Kipps-Vaughan, 2013). For a change effort to
be successful, solutions must consider and adapt to the organizational culture (Clark & Estes,
2008). With an increased focus on the availability of new effective and evidence-based teaching
strategies, mentors would have more opportunities to learn about these best practices. The
organization should provide professional development that provides the resources mentors need
to enhance their knowledge and ability to demonstrate new evidence-based teaching strategies in
the workshops.
A learning organization is capable of encouraging employees to acquire and transfer
knowledge (Huffman et al., 2003). Although mentors may not facilitate a workshop on a weekly
or monthly basis, they should all have access to the latest evidence-based practices and, thus, the
ability to thrive in their workshop. This would suggest that, along with emphasizing long-term
active support through engagement, connections between teachers work and their own students
improved professional practice (Huffman et al., 2003).
Recommendation Six
The sixth recommendation is that the organization provide mentors with effective
role models in workshops demonstrating effective and evidence-based teaching skills. If the
mentors can witness other modeling effective and evidence-based teaching skills, they can reflect
on those skills and incorporate them into their workshops. Utilizing mentors is an intuitive
practice which provides teachers the support to solve problems and develop trustful relationships
(Fresko & Alhija, 2012). The recommendation is for the organization to provide consistent
interaction and observation opportunities of workshop facilitated by master teachers in various
93
teaching scenarios that support the conceptual understanding of their role and helps to develop
their practical skills as well as job satisfaction.
A key support system for improving the depth of knowledge for new and established
teachers are mentors and models. In educational environments, a mentor is a more experienced
coworker who supports teachers in the development and execution of instruction (Ulvik &
Sunde, 2013). Therefore, creating a culture that reflects this belief is essential within the program
to increase the interest and retention of middle school girls’ in computer science.
Implementation and Evaluation Framework
To implement recommendations and to effectively evaluate the suggested
recommendations, the plan utilizes the New World Kirkpatrick Model, which evolved from the
original Kirkpatrick Four Level Model of Evaluation (Kirkpatrick & Kirkpatrick, 2016). The
model includes four levels of training that revert in a backward design. This New World version
presents the levels in reverse order from the original. To provide effective and evidence-based
instruction and mentorship in workshops that motivate and retain their interest in computer
science, this model looks at the New World version beginning with the end in mind (Level Four).
Level Four (Results) refers to the degree to which participants achieve the stated outcomes from
professional development. Leading indicators, or observable measurements, are then defined.
Level Three (Behavior) then identifies the critical behaviors and required drivers to reinforce
mentor performance. Level Two (Learning) then determines the degree to which learning
occurred in the areas of knowledge and skills, attitude, confidence, and commitment. Finally,
Level One (Reaction) measures the impressions of the participants and the degree to which they
have found the experience relevant and engaging (Kirkpatrick & Kirkpatrick, 2016).
94
Implementation and Evaluation Plan
The New World Kirkpatrick Model is used to inform the implementation and evaluation
plan designed to address the recommendations (Kirkpatrick & Kirkpatrick, 2016). The model
proposes a backward design so that the starting point for any implementation and evaluation plan
be the organization’s goals. To implement effective curriculum with enhanced development of
interpersonal and managerial skills, this model looks at Level Four, (Results) which refers to the
degree to which participants achieve the stated outcome from the use of common language
regarding effective and evidence-based teaching and learning, increased understanding of deficit
thinking, and the accessibility to model teaching in computer science to middle-school-aged girls
becomes a constant. Leading indicators, or observable measurements, are defined. Level Three
(Behavior) identifies the critical behaviors and required drivers to reinforce effective and
evidence-based teaching skill development. Level Two (Learning) shows the degree to which
learning occurred in the areas of knowledge, skills, attitude, confidence, and commitment based
on their participation in the training. Lastly, Level One (Reaction) evaluates the degree to which
the learner finds the training positive, interesting, and relevant to their teaching. (Kirkpatrick &
Kirkpatrick, 2016).
Level 4: Results and Leading Indicators
As STEMaven mentors facilitate new workshops, it is important to develop a mechanism
by which to assess and measure the rate at which mentors successfully increase and retain middle
school girls’ interest in computer science. Observations and the gauging of proposed scenarios
that suggest progress is being made to create positive results are called leading indicators
(Kirkpatrick & Kirkpatrick, 2016). Leading indicators reassure stakeholders that the progress is
contributing to organizational outcomes (Kirkpatrick & Kirkpatrick, 2016). As such there are
95
specific leading indicators that act as flags throughout the implementation of the innovation,
which will either serve to pinpoint ongoing adjustments to the workshops or denote goal
attainment. These indicators are both internal and external short-term outcomes.
Internal outcomes include use of common language regarding effective and evidence-
based teaching and learning, increased understanding of deficit thinking, and the accessibility to
model teaching in computer science to middle-school-aged girls. As STEMaven achieves the
internal outcomes, it can then expect to see the external outcomes also realized. External
outcomes include increased retention rates of students and workshop facilitation satisfaction of
mentors, and mentors actively engaged in after-hours professional development. Table 10 below
outlines these internal and external outcomes and the related metrics and methods for measuring
them.
Table 10
Outcomes, Metrics, and Methods for External and Internal Outcomes
Outcome Metric(s) Method(s)
External Outcomes
Increased retention
rates of students.
Number of students returning for
additional workshops.
Number of students in each workshop.
Attendance rate data regarding
number of students in each
workshop.
Increased workshop
facilitation
satisfaction rates of
mentors.
Number of mentors who report satisfaction
while facilitating workshop(s)
Number of parents/guardians who report
satisfaction with their child’s STEM
workshop experience.
Mentor climate surveys.
Mentors actively
engaged in
professional
development.
The types and number of professional
development activities mentors are
engaged in, based on the site-based
definition of effective, evidence-based
learning.
Aggregate data from
observation forms, survey
data, and task analysis from
planning documentation.
96
Table 10, continued
Outcome Metric(s) Method(s)
Internal Outcomes
Common language utilized
across STEMaven regarding
effective and evidence-based
teaching and learning practices.
Number of mentors who speak
about effective and evidence-
based teaching and learning
practices in common ways across
workshops.
Data from interviews
regarding instructional and
professional development
practices at STEMaven.
Increased understanding of
deficit thinking and how culture
is a primary driver of
instruction.
Number of mentors who report
understanding of deficit thinking.
Mentor climate survey.
Mentors have models for best
practices and effective teaching.
Number of mentors engaged with
a model or mentor that has prior
success with teaching middle
school girls.
Data from interviews
regarding instructional and
professional development
practices at STEMaven.
Level 3: Behavior
Critical behaviors. After training, Level Three is the most important, but also the most
thought-provoking part of an implementation and evaluation plan because of the difficulty in
supporting and holding stakeholders accountable for applying their learning (Kirkpatrick &
Kirkpatrick, 2016). Therefore, it is imperative that mentor behaviors are supervised to safeguard
goal attainment. These behaviors include learning components of the framework for effective
and evidence-based teaching, document of effective and evidence-based teaching, identification
of personal learning needs in regard to evidence-based teaching, and attendance and engagement
in learning groups. Table 11 below outlines each of these critical behaviors, their related metrics,
methods, and timing.
97
Table 11
Critical Behaviors, Metrics, Methods, and Timing for Evaluation
Critical Behavior Metric(s) Method(s) Timing
1. Mentors will learn the
components of the
framework for effective
and evidence-based
teaching
The number of
effective and
evidence-based
learning activities
implemented.
Leadership will
fund and make
available training
related to
framework.
Data collected
during
observations.
During every other
workshop. Initially
shared prior to a
mentors first
workshop, then
repeated every six
months.
2. Mentors will document
effective and evidence-
based learning goals
The completion of
clear effective and
evidence-based
goals.
The degree to
which goals are
evidence-based.
Mentor portfolio
updated.
After each
workshop a mentor
facilitates
3. Mentors will identify
personal learning needs in
regard to evidence-based
learning.
The number of
identified needs
from each mentor.
Mentors will
report identified
needs to
leadership.
Mentor models
will review
identified needs
and discuss initial
needs with
leadership.
Before and after
every 3 workshops.
4. STEMaven will provide
learning opportunities
where mentors can share
resources, discuss issues,
engage in effective
instructional strategies and
reflect on best practices.
The number of
professional
learning meetings
attended by
mentors.
Leadership will
track attendance
and request
feedback on
engagement.
Monthly with each
mentor.
Required drivers. The critical behaviors above cannot occur on their own. The
organization must also cultivate the environment with specific support for the critical behaviors
to succeed. These supports are called required drivers. There are four types of drivers:
98
reinforcing, encouraging, rewarding, and monitoring. Reinforcing drivers are those that
emphasize the importance of the transfer of the new skills into daily activity (Kirkpatrick &
Kirkpatrick, 2016). These include the knowledge related solutions outlined previously in the
chapter, such as provide job-aides that support effective teaching and learning strategies and use
of effective teaching tools. Encouraging drivers are those systems, supports, and processes that
provide consistent inspiration for participants to continue the transfer of the skills (Kirkpatrick &
Kirkpatrick, 2016). The encouraging drivers include motivation related solutions such as
rationales that provide utility value for effective and evidence-based teaching, the modeling of
strategies, and targeted and useful feedback. Rewarding drivers are those which recognize the
appropriate implementation of the skills (Kirkpatrick & Kirkpatrick, 2016). These rewarding
drivers include recognition of mentors with best teaching practices awards and by providing a
knowledge base of pedagogy and a system of support including trust and collaboration. Table 12
below outlines the reinforcing, encouraging, and rewarding drivers necessary for mentors to
implement effective and evidence-based teaching in their workshops, and which critical
behaviors they support.
99
Table 12
Required Drivers to Support Critical Behaviors
Method(s) Timing
Critical
Behaviors
Supported
1, 2, 3, 4
Reinforcing
Job aid containing a glossary of key terms found in
handbooks that clearly outlines professional expectations.
Ongoing 1,2,3,
Job aid that includes a clear framework of STEMaven’s
definition and expectation of effective and evidence-
based teaching
Ongoing 1,2,3
Job aid that details effective teaching strategies for
meeting the needs of all students that align with the
STEMaven goal.
Ongoing 1,2,3
Job aid that compares details of specific strategies
associated with culturally based learning, including the
benefits of each strategy and when and why a mentor
should utilize the strategy.
Ongoing 1,2,3
Meetings for mentors to discuss strategies Bi-monthly 1,2,3,4
Encouraging
Rationale for effective and evidence-based teaching
(utility value)
Ongoing 1,2,3
Mentor observations Bi-monthly 1,2,3
Peer modeling during group discussions Bi-monthly 1,2,3,4
Mentors receive targeted and useful feedback from peers
and leadership
Workshop
based
1,2,3,4
Rewarding
Public recognition on STEMaven website and internal
communication newsboard/emails when mentors
successfully implement effective and evidence-based
teaching that retains interest
Workshop
based
1,2,3,4
Monitoring
Leadership with review each workshop to evaluate that
goals and outcomes are contained in content as well as
effective teaching strategies
Prior to each
workshop
1,2,3,4
Portfolios that contain key performance indicators along
with individual interviews.
Bi-annually 1,2,3
Organizational support. Support and accountability are key to how an organization
takes responsibility for what it offers to its community and must remain under a watchful eye to
stay relevant (Conner & Rabovsky, 2011; Darling-Hammond & Snyder, 2015; Hentschke &
100
Wohlstetter, 2004). Therefore, to have reliable organizational support, mentors and leadership
are both critical for the review and follow-up of the knowledge, motivation, and organizational
needs. The outcomes and behaviors will need action plans along with consistent review of
success or failure. Mentors will need access to systematic experiences both in and out of the
workshop environment to provide growth and the skills necessary to facilitate workshops that
increase interest, confidence, and perceptions of computer science. Additionally, dashboards that
support mentors in self- and peer-monitoring with highlighted key performance indicators and
bi-annual surveys and interviews also support accountability for the required drivers and critical
behaviors (Kirkpatrick & Kirkpatrick, 2016).
Level 2: Learning
Learning goals. Following completion of the recommended solutions offered above,
STEMaven mentors will be able to
1. Mentors need to know how to incorporate effective and evidence-based strategies of
teaching middle school girl’s computer science (Procedural Knowledge).
2. Mentors need to know what deficit thinking is so they can promote a positive learning
experience for middle school girls and employ culturally responsive teaching strategies
(Metacognitive).
3. Mentors need to see how effective their teaching is in increasing retention of interest in
computer science (Goal Orientation).
4. Mentors need increased confidence that they can teach computer science to middle school
girls (Self-Efficacy).
5. Actively engage with peers and leadership (Cultural Model).
101
6. Monitor progress toward culturally based learning goals and adjust where necessary
(Self-Efficacy and Procedural).
Program. The evidence-based professional development program is a comprehensive
plan that enhances effective teaching and learning strategies for mentors to facilitate workshops
that motivate and retain middle school girls’ interest in computer science. This program supports
mentors in achieving the above stated learning goals through structure that enhances the
comprehension of the elements to incorporate effective and evidence-based strategies of teaching
middle school girl’s computer science, applying a common language about the incorporation of
effective and evidence-based strategies used in the workshops, and actively engaging in the
execution of associated strategies and techniques. Additionally, the program will provide
mentors the ability to engage in constructive dialogue centered around successful techniques,
focused goal setting, and feedback that is aimed to encourage each mentor.
Throughout the process, the organization will provide job aids for effective and evidence-
based teaching strategies and models that compare details of specific strategies associated with
culturally based learning, including the benefits of each strategy and when and why a mentor
should utilize the strategy. The professional development program will occur in an ongoing
format before, during, and after the facilitation of a workshop. The program is designed to
incorporate peer to peer learning and feedback, in conjunction with handbooks outlining
definitions and expectations. To support the learning further, leadership, peer mentors, and
mentors alike, will monitor comprehension and growth through observations and individual
portfolios containing key performance indicators and feedback.
Evaluation of the components of learning. As mentors begin the process of
implementation of effective and evidence-based strategies into their workshops, they need a
102
sense that they are accomplishing applicable knowledge. Therefore, it is imperative to evaluate
the level to which the mentors have learned the procedural and metacognitive knowledge. Table
13 outlines these methods for evaluation of these components as well as timing.
Table 13
Evaluation of the Components of Learning for the Program
Method(s) or Activity(ies) Timing
Declarative Knowledge “I know it. ”
Knowledge checks through the use of check-lists during
professional development activities and peer discussions.
Periodically during in-person
and virtual workshops.
Knowledge checks through the use of individual
portfolios that contain goal setting to demonstrate
understanding of expectations.
Upon completion of a
workshop.
Procedural Skills “I can do it right now. ”
Demonstration of individual job aids to successfully
implement culturally responsive workshops.
Through observation notes from
peers and leadership.
Demonstration of knowledge and skills to successfully
engage in culturally responsive workshops.
Through observation notes from
peers and leadership.
Attitude “I believe this is worthwhile. ”
Pre- and post-surveys After facilitating 2 workshops or
quarterly-whichever comes first.
Discussions with peers and leadership on the value of
deficit thinking and how this can influence learning.
During feedback sessions with
peers and leadership.
Confidence “I think I can do it on the job. ”
Scaled survey After facilitating 2 workshops or
quarterly-whichever comes first.
Discussions with leadership following workshops and
feedback.
Following the facilitation of
every other workshop.
Commitment “I will do it on the job. ”
Creation of mentor goals related to individual action
plans.
During professional
development meetings.
Level 1: Reaction
Level One recommends utilizing three components to measure reactions to the program.
They consist of engagement, relevance, and satisfaction (Kirkpatrick & Kirkpatrick, 2016). To
ensure the desired outcomes, all three components are pivotal to the successful desired results.
103
As with the components of the professional development program above, Table 14 below
outlines the needs to measure reactions from the program.
Table 14
Components to Measure Reactions to the Program
Method(s) or Tool(s) Timing
Engagement
Attendance Required for each workshop
Completion of goals in individual portfolio Prior to facilitation of a new workshop
Self-Assessment Completion of each workshop
Observation During a facilitated workshop
Participation in peer review and feedback Throughout the year
Relevance
Online brief survey to monitor if skills and
knowledge are incorporated
After each facilitated workshop
Effectiveness of peer to peer learning,
modeling, and feedback
At the end of every other month
Customer Satisfaction
Online survey of mentors upon exit of
STEMaven program
End of mentorship
Check-in scaled survey (online) and
discussion (ongoing)
When new effective/evidence-based learning
strategies are introduced
Evaluation Tools
Immediately following the program implementation. The following sections
summarize the evaluation tools used during and immediately following the program
implementation and delayed evaluation tools based on the timeline Kirkpatrick and Kirkpatrick
(2016) suggest. To move the program forward and help guide the mentors obtain a wide range of
understanding of their experience and outcomes, multiple methods will be used to advise the
evaluation. Evaluation of the program through participant feedback helps to improve the
program, to maximize transfer of learning to behavior and subsequent organizational results, and
to demonstrate the value of training to the organization (Kirkpatrick & Kirkpatrick, 2016).
104
After the completion of a workshop, mentors will participate in a self-talk survey in their
individual portfolio that will assess their self-efficacy in applying, relevant utilization, and
obligation to the new skills, and general agreement with the content, delivery, and structure of
the professional development program. The selected facilitators of the professional development
program will request Level One feedback about application during the training through group
discussion and pulse-check self-talk that ask mentors to share their comments on one thing they
learned and one thing they are wanting to learn through a variety of outlets. The mentors can
communicate their self-talk through verbal face-to-face communication, digitally, or handwritten
on posters located throughout the room where all participants can add comments and
suggestions. These pulse-checks allow for trainers to gain a perspective of Level One and Level
Two comprehension or areas needing additional thought. Additionally, a final reflective question
will probe mentors to ascertain how they can employ the material learned to their next workshop.
Delayed for a period after the program implementation. A supplemental evaluation
will occur after a delayed period of time, customarily after the mentor has facilitated at least two
workshops to allow for marination of thought and implementation of the strategies. A blended
model, incorporating all the levels from reaction to results makes maximizing perspective on the
experience more likely to (Kirkpatrick & Kirkpatrick, 2016). The evaluation tool should include
survey items that measure engagement, relevance, and satisfaction (Level One), confidence in
and value of knowledge acquired (Level Two), the degree to which the mentor applied the
learning (Level Three), and the degree to which the training has impacted the successful
facilitation of a workshop (Level Four).
105
Assessment Analysis and Reporting
The Level Four goals, which examine the degree to which the training has impacted the
successful facilitation of mentors learning effective and evidence-based teaching strategies,
consists of skills evaluations, effective and evidence-based teaching strategies learned through
structured professional development, mentor and peer modeling, and time for reflection and
feedback. Through mentor roundtables, one-on-one reflective meetings between mentors and
director, mentors will be able to see if students are gaining interest in computer science. Mentors
will be asked for information on knowledge acquired, how training changed their approach to
teaching middle school girls and how it will impact their future workshop facilitation. Figure 13
shows the elements of professional development that supports the growth of effective and
evidence-based teaching skills.
Figure 13. Elements of professional development that supports the growth of effective and
evidence-based teaching skills.
Skills Evaluation
Effective and Evidence-
Based Teaching
Professional
Development
Mentor and Peer
Modeling
Reflection
106
Strengths and Weaknesses of the Approach
All methodological approaches have strengths and weaknesses. The Clark and Estes
(2008) framework was beneficial to review how knowledge, motivation and organizational
influences influenced mentor’s awareness of effective and evidence-based teaching strategies.
There are a variety of methods that may have exhibited diversified results. The framework was
effective in creating both the survey and interview questions, although holding focus groups and
reviewing historical documents may have shown additional themes. Nevertheless, the framework
was productive for this specific problem and stakeholder group. The framework assisted in
understanding mentor’s proficiencies and by employing best practices the organization can
impact mentors’ knowledge on effective skills that can aid in the interest in computer science.
Although the framework necessitated participant involvement, which took volunteering their
personal time, I considered it a successful approach to measure and study the stakeholder goal
and problem of practice.
Limitations and Delimitations
This study was limited in that survey results showed that participants felt they were
confident in facilitating their workshops. However, in interviews, the majority of participants
thought there was no practice of reflection or professional development to enhance and develop
these skills. From the interviews, the mentor did not believe that STEMaven focused on these
skills but, rather, concentrated on lessons pertaining to computer science. While the disparity was
not overwhelming in the survey results, interviews also had similar themes but were not
necessarily aligned with the survey results. Interviews also had limitations as some mentors had
not facilitated a workshop in months and, in some cases, had to be emailed and rescheduled due
to unforeseen circumstances. The survey response rate was high and the interviews were more
107
difficult to schedule due to the constraints to qualify. Even though the survey had a larger
number of participants than the interview portion, having a larger number of interviews could
have shed light on more knowledge, motivation and organizational influences.
The recommendations could be utilized by both established and new after-school
programs. Programs looking to increase mentors effective and evidence-based teaching strategies
can all benefit from the results and recommendations. Having a focused professional
development of effective and evidence-based strategies along with opportunities of self-
awareness and reflection can benefit all after-school programs providing mentors with the tools
necessary to increase the interest in a STEM field.
Future Research
Future research could study more innovative models of effective and evidence-based
teaching strategies in successful STEM-related after-school programs. Many programs offer
structured professional development and best practices on reflection and self-awareness. Some
after-school programs partner with community businesses to provide real-world problems that
can be utilized within the lessons to create and drive interest in STEM.
Research conducted with mentors in after-school STEM-related programs who have had
time to develop and utilize their skills in workshops would also be very interesting to study.
Having a chance to complete a longitudinal study of the student participants every five years
would bring a very interesting perspective on where the students are now and if they are
interested in having a career in STEM. Moreover, research conducted with the undergraduate
students at the local university and community college could prove useful to see which students
pursued their education in a STEM discipline due to the influence of a STEM-based after-school
108
program. In researching this, other STEM-based after-school programs can be created that
influence middle school girls to pursue a career in STEM.
Conclusion
To ensure that mentors at STEMaven will demonstrate improved effective and evidence-
based teaching strategies as a result of the professional development, it made sense to conduct
research with mentors who had facilitated a workshop(s) in the program and who had been a
mentor since January 2018. Through surveys and interviews, the Clark and Estes (2008)
framework allowed to study mentor’s knowledge, motivation and organizational influences in
regards to the importance of effective and evidence-based teaching skills. The study revealed
many gaps in mentors’ knowledge of what students need to create interest in computer science
and gaps in the organizational culture that contribute to the gap in contributing to development of
interest. The implementation and evaluation plan of recommendations was designed using the
New World Kirkpatrick Model. The model, backward in design, starts with the ultimate
organizational goal and then finds necessary learning and assessment components to slowly and
consistently bring goals to completion. Methodical data analysis will offer the tools to maximize
outcomes of all future initiatives (Kirkpatrick & Kirkpatrick, 2016). Through this model, STEM-
related after-school programs can address knowledge, motivation and organizational influences
identified and validated in this study, to ensure that mentors have a structured environment
where effective and evidence-based teaching strategies are the core of the program. If and when
this occurs, it will be embedded into the best practices mentors actively use and give them the
ability to prosper as mentors and facilitate workshops with a conscience of deficit thinking.
109
REFERENCES
Afterschool Alliance. (2017). Afterschool in communities of concentrated poverty. Washington,
DC: Author.
Akl, R., Keathly, D., & Garlick, R. (2007). Strategies for retention and recruitment of women
and minorities in computer science. Arlington, VA: International Network for
Engineering Education and Research.
Allison, B., & Rehm, M. (2007). Effective teaching strategies for middle school learners in
multicultural, multilingual classrooms. Middle School Journal, 39(2), 12–18.
https://doi.org/10.1080/00940771.2007.11461619
Alvarado, C., & Judson, E. (2014). Using targeted Conferences to recruit women into computer
science. Communications of the ACM, 57(3), 70–77. https://doi.org/10.1145/2500883
American Institute for Research. (2008). Afterschool programs make a difference: Findings from
the Harvard Family Research Project. Retrieved from http://www.sedl.org/pubs/sedl-
letter/v20n02/afterschool_findings.html
Bandura, A. (2000). An exercise of human agency through collective efficacy. Current
Directions in Psychological Science, 9(3), 75–78. https://doi.org/10.1111/1467-
8721.00064
Bevan, B., & Michalchik, V. (2013). Where it gets interesting: Competing models of STEM
learning after school. Afterschool Matters, 17, 1–8.
Blažev, M., Karabegović, M., Burušić, J., & Selimbegović, L. (2017). Predicting gender-STEM
stereotyped beliefs among boys and girls from prior school achievement and interest in
STEM school subjects. Social Psychology of Education, 20(4), 831–847.
https://doi.org/10.1007/s11218-017-9397-7
110
Bolman, L. G., & Deal, T. E. (2008). Reframing organizations: Artistry, choice, and leadership
(4th ed.). San Francisco, CA: Jossey-Bass.
Borgogni, L., Dello Russo, S., & Latham, G. (2011). The relationship of employee perceptions
of the immediate supervisor and top management with collective efficacy. Journal of
Leadership & Organizational Studies, 18(1), 5–13.
https://doi.org/10.1177/1548051810379799
Bystydzienski, J., Eisenhart, M., & Bruning, M. (2015). High school is not too late: Developing
girls’ interest and engagement in engineering careers. The Career Development
Quarterly, 63(1), 88–95. https://doi.org/10.1002/j.2161-0045.2015.00097.x
Campbell, M. (2011). Researchers look at ways to bridge the gender gap in STEM fields. The
Hispanic Outlook in Higher Education., 21(10), 26–27.
Carbonaro, M., Szafron, D., Cutumisu, M., & Schaeffer, J. (2010). Computer-game construction:
A gender-neutral attractor to computing science. Computers & Education, 55(3), 1098–
1111. https://doi.org/10.1016/j.compedu.2010.05.007
Cech, E., Rubineau, B., Silbey, S., & Seron, C. (2011). Professional role confidence and
gendered persistence in engineering. American Sociological Review, 76(5), 641–666.
https://doi.org/10.1177/0003122411420815
Ceci, S., Williams, W., & Barnett, S. (2009). Women’s underrepresentation in science:
Sociocultural and biological considerations. Psychological Bulletin, 135(2), 218–261.
https://doi.org/10.1037/a0014412
Clark, R., & Estes, F. (2008). Turning research into results: A guide to selecting the right
performance solutions. Charlotte, NC: Information Age.
111
Conner, T., & Rabovsky, T. (2011). Accountability, Affordability, Access: A review of the
recent trends in higher education policy research. Policy Studies Journal: the Journal of
the Policy Studies Organization, 39(1), 93–112. https://doi.org/10.1111/j.1541-
0072.2010.00389_7.x
Cooper, J. E., He, Y., & Levin, B. B. (2011). Developing critical cultural competence: A guide
for 21st-century educators. Thousand Oaks, CA: Corwin Press.
Cormas, P., & Barufaldi, J. (2011). The effective research-based characteristics of professional
development of the national science foundation’s GK-12 program. Journal of Science
Teacher Education, 22(3), 255–272. https://doi.org/10.1007/s10972-011-9228-1
Creswell, J. W., & Creswell, J. D. (2018). Research design: Qualitative, quantitative, and mixed
methods approaches (5th ed.). Thousand Oaks, CA: Sage Publications.
Curtis, D., & Lawson, M. (2002). Computer adventure games as problem-solving environments.
International Education Journal, 3(4), 43–56.
Darling-Hammond, L., & Snyder, J. (2015). Meaningful learning in a new paradigm for
educational accountability: An introduction. Education Policy Analysis Archives, 23(7).
https://doi.org/10.14507/epaa.v23.2005
Denner, J., Werner, L., Bean, S., & Campe, S. (2005). The girls creating games program:
Strategies for engaging middle-school girls in information technology. Frontiers, 26(1),
183–186. https://doi.org/10.1353/fro.2005.0008
Diamond, B. J., & Moore, M. A. (1995). Multicultural Literacy: Mirroring the Reality of the
Classroom. New York, NY: Longman.
112
Di Stefano, G., Gino, F., Pisano, G., & Staats, B. (2014). Learning by thinking: How reflection
improves performance (HBS Working Paper No 14-093). Boston, MA: Harvard Business
School.
Durlak, J., Mahoney, J., Bohnert, A., & Parente, M. (2010). Developing and improving after-
school program to enhance youth’s personal growth and adjustment: A special issue of
AJCP. American Journal of Community Psychology, 45(3-4), 285–293.
https://doi.org/10.1007/s10464-010-9298-9
Fashola, O. (1998). Review of extended-day and after-school programs and their effectiveness.
Baltimore, MD: Johns Hopkins University.
Fresko, B., & Alhija, F. (2012). Induction seminars as professional learning communities for
beginning teachers. Asia-Pacific Journal of Teacher Education, 43(1), 36–48.
https://doi.org/10.1080/1359866X.2014.928267
Gallimore, R., & Goldenberg, C. (2001). Analyzing cultural models and settings to connect
minority achievement and school improvement research. Education Psychologist, 36(1),
45–56. https://doi.org/10.1207/S15326985EP3601_5
Gardner, H. (1983). Frames of mind: The theory of multiple intelligences. New York, NY: Basic
Books.
Garmston, R. J., & Wellman, B. M. (1999). The adaptive school: A sourcebook for developing
collaborative groups. Norwood, Mass: Christopher-Gordon.
Gay, G. (2000). Culturally responsive teaching: Theory, practice and research. New York:
Teachers College Press.
Gay, G. (2012). STEM focus of after-school clubs. Washington, D.C.: Washington Informer.
113
Girod, M., Martineau, J., & Zhao, Y. (2004). After-school computer clubhouses and at-risk
teens. American Secondary Education, 32(3), 63–76.
Glesne, C. (2011). Becoming qualitative researchers: An introduction (4th ed.). Boston, MA:
Pearson.
Grogan, K., Henrich, C., & Malikina, M. (2014). Student engagement in after-school programs,
academic skills, and social competence among elementary school students. Child
Development Research, 2014, 1–9. https://doi.org/10.1155/2014/498506
Grolnick, W., Farkas, M., Sohmer, R., Michaels, S., & Valsiner, J. (2007). Facilitating
motivation in young adolescents: Effects of an after-school program. Journal of Applied
Developmental Psychology, 28(4), 332–344.
https://doi.org/10.1016/j.appdev.2007.04.004
Halim, L., Soh, T., & Arsad, N. (2018). The effectiveness of STEM mentoring program in
promoting interest towards STEM. Journal of Physics: Conference Series, 1088(1),
012001. https://doi.org/10.1088/1742-6596/1088/1/012001
Halpern, R. (2002). A different kind of child development institution: The history of after-school
programs for the low-income children. Teachers College Record, 104(2), 178–211.
https://doi.org/10.1111/1467-9620.00160
Hardin, E., & Longhurst, M. (2016). Understanding the gender gap: Social cognitive changes
during an introductory STEM course. Journal of Counseling Psychology, 63(2), 233–239.
https://doi.org/10.1037/cou0000119
Hentschke, G., & Wohlstetter, P. (2004). Cracking the code of accountability. The magazine of
the USC Rossier School of Education. Retrieved from http://hillkm.com/yahoo_site_
admin/assets/docs/unit_1_hentschke_wohlsetter.pdf
114
Hollister, R. (2003). The growth in after-school programs and their impact. Washington, DC:
Brookings Institution.
Huffman, D., Thomas, K., & Lawrenz, F. (2003). Relationship between professional
development, teachers’ instructional practices, and the achievement of students in science
and mathematics. School Science and Mathematics, 103(8), 378–387.
https://doi.org/10.1111/j.1949-8594.2003.tb18123.x
Hughes, R., Nzekwe, B., & Molyneaux, K. (2013). The Single Sex Debate for Girls in Science:
A Comparison between two informal science programs on middle school students’
STEM identity Formation. Research in Science Education, 43(5), 1979–2007.
https://doi.org/10.1007/s11165-012-9345-7
Hyllegard, K., Rambo-Hernandez, K., & Ogle, J. (2017). Fashion fundamentals: Building middle
school girls’ self-esteem and interest in STEM. Journal of Women and Minorities in
Science and Engineering, 23(1), 87–99.
https://doi.org/10.1615/JWomenMinorScienEng.2017018331
Hynes, M. M. (2012). Middle-school teachers’ understanding and teaching of the engineering
design process: A look at subject matter and pedagogical content knowledge.
International Journal of Technology and Design Education, 22(3), 345–360.
https://doi.org/10.1007/s10798-010-9142-4
Johnson, R. B., & Christensen, L. B. (2015). Educational research: Quantitative, qualitative,
and mixed approaches (5th ed.). Thousand Oaks: SAGE.
Junge, S. K. (2003). Building life skills through afterschool participation in experimental and
cooperative learning. Child Study Journal, 33(3), 165–179.
115
Kezar, A. (2001). Research-based principles of change. Understanding and facilitating
organizational change in the 21st century: Recent research and conceptualizations.
ASHE-ERIC Higher Education Report, 28(4), 113–123.
Kipps-Vaughan, D. (2013). Supporting teachers through stress management. Principal
Leadership. Education Digest, 79(1), 43–46.
Kirkpatrick, J. D., & Kirkpatrick, W. K. (2016). Kirkpatrick ’s four levels of training evaluation.
Alexandria, VA: ATD Press.
Krathwohl, D. R. (2002). A revision of Bloom’s taxonomy: An overview. Theory into Practice,
41(4), 212–218. https://doi.org/10.1207/s15430421tip4104_2
Krosgaard, M., Brodt, S., & Whitener, E. (2002). Trust in the face of conflict: The role of
managerial trustworthy behavior and organizational context. The Journal of Applied
Psychology, 87(2), 312–319. https://doi.org/10.1037/0021-9010.87.2.312
Krueger, R. A., & Casey, M. A. (2009). Focus groups: A practical guide for applied research.
Thousand Oaks, CA: SAGE Publications.
Ladson‐Billings, G. (1992). Reading between the lines and beyond the pages: A culturally
relevant approach to literacy teaching. Theory into Practice, 31(4), 312–320.
https://doi.org/10.1080/00405849209543558
Ladson‐Billings, G. (2001). Crossing over to Canaan: The journey of new teachers in diverse
classrooms. San Francisco: Jossey-Bass.
Lan, P., & Young, S. (1996). International technology transfer examined at technology
component level: A case study in China. Technovation, 16(6), 277–286.
https://doi.org/10.1016/0166-4972(96)00005-3
116
Leas, H. D., Nelson, K. L., Grandgenett, N., Tapprich, W. E., & Cutucache, C. E. (2017).
Fostering curiosity, inquiry, and scientific thinking in elementary school students: Impact
of the NE STEM 4U intervention. Journal of Youth Development, 12(2), 103–120.
https://doi.org/10.5195/JYD.2017.474
Liben, L., & Coyle, E. (2014). Developmental interventions to address the STEM gender gap:
Exploring intended and unintended consequences. Advances in Child Development and
Behavior, 47, 77–115. https://doi.org/10.1016/bs.acdb.2014.06.001
Little, P. M., Wilmer, C., & Weiss, H. B. (2008). After-school programs in the 21st century:
Their potential and what it takes to achieve it. Cambridge, MA: Harvard Family
Research Project.
Litzler, E., Samuelson, C., & Lorah, J. (2014). Breaking it down: Engineering student STEM
confidence at the intersection of race/ethnicity and gender. Research in Higher
Education, 55(8), 810–832. https://doi.org/10.1007/s11162-014-9333-z
Lundh, P., House, A., Means, B., & Harris, C. (2013). Learning from science: Case studies of
science offerings in afterschool programs. Afterschool Matters, 18, 33–48.
Ma, C., & Schapira, M. (2017). The bell curve: Intelligence and class structure in American life.
London, England: Taylor & Francis. https://doi.org/10.4324/9781912282470
Mahoney, J., Lord, H., & Carryl, E. (2005). An ecological analysis of after-school program
participation and the development of academic performance and motivational attributes
for disadvantaged children. Child Development, 76(4), 811–825.
https://doi.org/10.1111/j.1467-8624.2005.00879.x
Maxwell, J. A. (2013). Qualitative research design: An interactive approach (3rd ed.). Thousand
Oaks, CA: SAGE.
117
Mayer, R. E., Quilici, J. L., & Moreno, R. (1999). What is learned in an after-school computer
club? Journal of Educational Computing Research, 20(3), 223–235.
https://doi.org/10.2190/V1UG-2W6F-4RYH-5R9T
Mayer-Smith, J., Pedretti, E., & Woodrow, J. (2000). Closing of the gender gap in technology
enriched science education: A case study. Computers & Education, 35(1), 51–63.
https://doi.org/10.1016/S0360-1315(00)00018-X
McElvain, C., & Caplan, J. (2001). Creating effective after-school programs for middle and high
school students. NASSP Bulletin, 85(626), 35–44.
https://doi.org/10.1177/019263650108562604
McNally, T. (2012). Innovative teaching and technology in the service of science: Recruiting the
next generation of STEM students. The Journal of Scholarship of Teaching and
Learning, 12(1), 49.
Merriam, S. B., & Tisdell, E. J. (2016). Qualitative research: A guide to design and
implementation (4th ed.). San Francisco, CA: Jossey-Bass.
Mesiti, L. A., Parkes, A., Paneto, S. C., & Cahill, C. (2019). Building capacity for computational
thinking in youth through informal education. Journal of Museum Education, 44(1), 108–
121. https://doi.org/10.1080/10598650.2018.1558656
Migus, L. H. (2014). Broadening access to STEM learning through out-of-school learning
environments. Washington, DC: Committee on Successful Out-of-School STEM
Learning.
Morehouse, H. (2009). Making the most of the middle: A strategic model for middle school
afterschool programs. Afterschool Matters, 8, 1–10.
118
Mouza, C., Marzocchi, A., Pan, Y. C., & Pollock, L. (2016). Development, implementation, and
outcomes of an equitable computer science after-school program: Findings from middle-
school students. Journal of Research on Technology in Education, 48(2), 84–104.
https://doi.org/10.1080/15391523.2016.1146561
Naizer, G., Hawthorne, M., & Henley, T. (2014). Narrowing the gender gap: Enduring changes
in middle school students’ attitude toward math, science, and technology. The Journal of
STEM Education, 15(3), 29–34.
National Center for Education Statistics. (2018). The condition of education 2018. NCES 2018-
44. Retrieved from: http://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=2018144
Nugent, G., Barker, B., Grandgenett, N., & Adamchuk, V. (2010). Impact of robotics and
geospatial technology interventions on youth STEM learning and attitudes. Journal of
Research on Technology in Education, 42(4), 391–413.
https://doi.org/10.1080/15391523.2010.10782557
Outlay, C. N., Platt, A., & Conroy, K. (2017). Getting IT together: A longitudinal look at linking
girls’ interest in IT careers to lessons taught in middle school camps. Transactions on
Computing Education, 17(4), 1–20. https://doi.org/10.1145/3068838
Patton, M. Q. (2002). Qualitative research & evaluation methods (3rd ed.). Thousand Oaks, CA:
SAGE Publications.
Pajares, F. (2006). Self-efficacy theory. http://www.education.com/reference/ article/self-
efficacy-theory
Payton, F., White, A., & Mullins, T. (2017). STEM majors, art thinkers-Issue of duality, rigor,
and inclusion. Journal of STEM Education, 18(3).
119
Perera, V., Mead, C., Buxner, S., Lopatto, D., Horodyskyj, L., Semken, S., & Anbar, A. D.
(2017). Students in fully online programs report more positive attitudes toward science
than students in traditional, in-person programs. CBE Life Sciences Education, 16(4),
ar60. https://doi.org/10.1187/cbe.16-11-0316
Pinnell, M., Rowly, J., Preiss, S., Franco, S., Blust, R., & Beach, R. (2013). Bridging the gap
between engineering design and PK-12 curriculum development through the use the
STEM education quality framework. Journal of STEM Education. Innovations and
Research, 14(4), 28–35.
Pintrich, P. R. (2003). A motivational science perspective on the role of student motivation in
learning and teaching contexts. Journal of Educational Psychology, 95(4), 667–686.
https://doi.org/10.1037/0022-0663.95.4.667
Priyadarshini, R. (2009). The importance of role efficacy and self-efficacy in organizations and
its relationship with HR practices. Management and Labour Studies, 34(1), 57–72.
https://doi.org/10.1177/0258042X0903400104
Richter, A., Hirst, G., van Knippenberg, D., & Baer, M. (2012). Creative self-efficacy and
individual creativity in team contexts: Cross-level interactions with team informational
resources. The Journal of Applied Psychology, 97(6), 1282–1290.
https://doi.org/10.1037/a0029359
Robinson, S. B., & Firth Leonard, K. (2019). Designing quality survey questions. Los Angeles,
CA: SAGE.
Robnett, R. D. (2016). Gender bias in STEM fields: Variation in prevalence and links to STEM
self-concept. Psychology of Women Quarterly, 40(1), 65–79.
https://doi.org/10.1177/0361684315596162
120
Rorie, M., Gottfredson, D. C., Cross, A., Wilson, D., & Connell, N. M. (2011). Structure and
deviancy training in after-school programs. Journal of Adolescence, 34(1), 105–117.
https://doi.org/10.1016/j.adolescence.2010.01.007
Rubin, H. J., & Rubin, I. S. (2012). Qualitative interviewing: The art of hearing data (3rd ed.).
Thousand Oaks, CA: SAGE Publications.
Rueda, R. (2011). The 3 dimensions of improving student performance: Finding the right
solutions to the right problems. New York, NY: Teachers College Press.
Sahin, A. (2013). STEM clubs and science fair competitions: Effects on post-secondary
matriculation. Journal of STEM Education: Innovations and Research, 14(1), 5–11.
Sassler, S., Michelmore, K., & Smith, K. (2017). A tale of two majors: Explaining the gap in
stem employment among computer science and engineering degree holders. Social
Sciences, 6(3), 1–26.
Schein, E. H. (2004). Organizational culture and leadership (3rd ed.). San Francisco, CA:
Jossey-Bass.
Schein, E. H. (2017). Organizational culture and leadership (5th ed.). San Francisco, CA:
Jossey-Bass.
Schneider, B., Brief, A., & Guzzo, R. (1996). Creating a climate and culture for sustainable
organizational change. Organizational Dynamics, 24(4), 7–19.
https://doi.org/10.1016/S0090-2616(96)90010-8
Schuman, M. (2017). History of child labor in the United States-part 1: Little children working.
Monthly Labor Review, 140, 1.
Seaton, D., & Carr, D. (2005). The impact of participation in an ancillary science and
mathematics program (SEMAA) on engagement rates of middle school students in
121
regular mathematics classrooms. School Science and Mathematics, 105(8), 423–432.
https://doi.org/10.1111/j.1949-8594.2005.tb18062.x
Senge, P. (1990). The leader’s new work: Building learning organizations. Sloan Management
Review, 32(1), 7–23.
Shernoff, D. J. (2010). Engagement in after-school programs as a predictor of social competence
and academic performance. American Journal of Community Psychology, 45(3-4), 325–
337. https://doi.org/10.1007/s10464-010-9314-0
Shinn, Y. (1997). Teaching strategies, their use and effectiveness as perceived by teachers of
agriculture: A national study. Retrieved from https://lib.dr.iastate.edu/cgi/viewcontent.
cgi?article=13243&context=rtd https://doi.org/10.31274/rtd-180813-8037
Shortt, D. M. (1998). Gender and technology: Looking to the past. Canadian Women ’s Studies,
17(4), 89.
Shuen, J. A., Elia, A. R., Xu, K., Chen, C. F. J., Jiang, A., Litkowski, E., . . . Schwartz-Bloom, R.
D. (2011). Femmes: A one-day mentorship program to engage 4th-6th grade girls in
STEM activities. Journal of Women and Minorities in Science and Engineering, 17(4),
295–312. https://doi.org/10.1615/JWomenMinorScienEng.2011002292
Shute, V., Sun, C., & Asbell-Clarke, J. (2017). Demystifying computational thinking.
Educational Research Review, 22, 142–158. https://doi.org/10.1016/j.edurev.2017.09.003
STEMaven. (2019) About. Retrieved from https://www.orcsgirls.org/about
Tan, E., Barton, A., Kang, H., & O’Neill, T. (2013). Desiring a career in STEM-related fields:
How middle school girls articulate and negotiate identities-in practice in science. Journal
of Research in Science Teaching, 50(10), 1143–1179. https://doi.org/10.1002/tea.21123
122
Ulvik, M., & Sunde, E. (2013). The impact of mentor education: Does mentor education matter?
Professional Development in Education, 39(5), 754–770.
https://doi.org/10.1080/19415257.2012.754783
U.S. Department of Education. (2016). Advancing educational technology in teacher
preparation. Washington, DC: U.S. Department of Education.
U.S. Bureau Labor Statistics. (2017). Women in architecture and engineering occupations in
2016. Washington, DC: U.S. Bureau of Labor Statistics.
Valencia, R. (1997). The Evolution of Deficit Thinking: Educational Thought and Practice.
Washington, D.C.: The Falmer Press.
VanMeter-Adams, A., Frankenfeld, C. L., Bases, J., Espina, V., & Liotta, L. A. (2014). Students
who demonstrate strong talent and interest in STEM are initially attracted to STEM
through extracurricular experiences. CBE Life Sciences Education, 13(4), 687–697.
https://doi.org/10.1187/cbe.13-11-0213
Vekiri, I. (2010). Boys’ and girls’ ICT beliefs: Do teachers matter? Computers & Education,
55(1), 16–23. https://doi.org/10.1016/j.compedu.2009.11.013
Verma, A. K., Dickerson, D., & McKinney, S. (2011). Engaging students in STEM careers with
project-based learning—MarineTech project. Technology and Engineering Teacher,
71(1), 25–31.
Wade-Jaimes, K., Cohen, J. D., & Calandra, B. (2019). Mapping the evolution of an after-school
STEM club for African American girls using activity theory. Cultural Studies of Science
Education, 14(4), 1–30. https://doi.org/10.1007/s11422-018-9886-9
123
Wahl-Alexander, Z., Schwamberger, B., & Neels, D. (2017). In-depth analysis of a teacher’s
experience implementing sport education in an after-school context. Physical Educator,
74(4), 627–652. https://doi.org/10.18666/TPE-2017-V74-I4-7544
Weber, K. (2011). Role models and informal STEM-related activities positively impact female
interest in STEM. Technology and Engineering Teacher, 71(3), 18–21.
Werner, L., & Denning, J. (2009). Pair programming in middle school: What does it look like?
Journal of Research on Technology in Education, 42(1), 29–49.
https://doi.org/10.1080/15391523.2009.10782540
Williams, P. (2011). STEM education: Proceed with caution. Journal of Design and Technology
Education, 16(1), 26–35.
Wlodkowski, R., & Ginsberg, M. (1995). A framework for culturally responsive teaching.
Education Leadership, 53(1), 17-21.
Yough, M., & Anderman, E. (2006). Goal orientation theory. Retrieved from
http://www.education.com/reference/article/goal-orientation-theory/
Zeldin, L., Britner, S., & Pajares, F. (2006). A Comparative study of the self-efficacy beliefs of
successful men and women in mathematics, science, and technology careers. Journal of
Research in Science Teaching, 45(9), 1036–1058. https://doi.org/10.1002/tea.20195
Zhang, J. J., & Byrd, C. E. (2006). Successful after-school programs: The 21st century
community learning centers. [editorial]. Journal of Physical Education, Recreation &
Dance, 77(8), 3–12. https://doi.org/10.1080/07303084.2006.10597912
124
APPENDIX A
Survey Protocol
Confidentiality
All information that is collected in this study will be treated with confidentiality. While
aggregate results will be made available to STEMaven, you are guaranteed that neither
you, the program nor any of its personnel will be identified in any report of the results of
the study. Participation in this survey is voluntary and any individual may withdraw at
any time.
About the questionnaire
• This questionnaire asks for information about the program.
• This questionnaire should take approximately 5-10 minutes to complete.
• All questions can be answered by marking the one most appropriate answer.
• Please complete this questionnaire no later than October 25, 2019.
• When in doubt about how to answer a question, or if you would like more
information about it or the study, you can reach me by phone at the following
number 423-298-2142 or email at caseyjen@usc.edu.
Thank you for your cooperation!
Survey Items and Analysis
Key: Procedural Knowledge (P), Metacognitive (M) Self-Efficacy (SE), Goal orientation (GO),
Cultural Setting Influence (CS), Cultural Model Influence (CM)
Research
Question/
Data Type
KMO
Construct
Survey Item (question and
response)
Scale of
Measure-
ment
Potential
Analyses
Visual
Represent-
ation
Demo-
graphic:
sample
description
N/A
(Used for
summary
statistics)
1. Please identify your current
employment status *Mark only
one.
a. Full time
b. Part-time
c. Retired
d. Unemployed
Nominal Percentage,
Frequency
Table
Demo-
graphic:
sample
description
N/A
(Used for
summary
statistics)
2.What is your gender?
a. Female
b. Male
c. Decline to state
Nominal Percentage,
Frequency
Table
Demo-
graphic:
sample
description
N/A
(Used for
summary
statistics)
3. Do you currently volunteer as a
mentor at another after-school
STEM program?
a. Yes, if yes, how many
____
b. No
Nominal Percentage,
Frequency
Table
125
Demo-
graphic:
sample
description
N/A
(Used for
summary
statistics)
4.Have you facilitated more than 2
workshops since January 2018?
a. Yes. If more than 2, how
many ______.
b. No
Nominal Percentage,
Frequency
Table
Demo-
graphic:
sample
description
N/A
(Used for
summary
statistics)
5. What is the highest level of
formal education that you have
completed?
a. High School
b. Undergraduate
c. Master’s degree
d. Doctoral degree. Please
list your
concentration___________
Nominal Percentage,
Frequency
Table
Demo-
graphic:
sample
description
N/A
(Used for
summary
statistics)
6.When did you begin working in
your current position?
a. 2019
b. 2018
c. 2017
d. 2016
e. 2015
f. 2014
g. 2013 or earlier
Nominal Percentage,
Frequency
Table
Demo-
graphic:
sample
description
N/A
(Used for
summary
statistics)
7.What is your employment
outside of STEMaven?
a. Computer Scientist
b. Engineer (specify type)
________
c. Other
(specify)__________
Nominal Percentage,
Frequency
Table
Demo-
graphic:
sample
description
N/A
(Used for
summary
statistics)
8.What is the primary location that
you facilitate at STEMaven
workshop?
a. Local community college
b. Local middle school
c. Local high school
d. Local community center
e. Other (please list)
_______
Nominal Percentage,
Frequency
Table
Demo-
graphic:
sample
description
N/A
(Used for
summary
statistics)
9. How many months have you
been a mentor at STEMaven?
*Please write the total number of
months __________
Nominal Percentage,
Frequency
Table
What is the
stakeholder
’s
knowledge
related to
increasing
middle
Knowledge
(P) (Mentors
need
knowledge
of deficit
thinking so
they can
10. Please rate your knowledge of
deficit thinking and how this can
influence the learning environment
for each of the following. Please
also indicate how STEMaven
added to your rating.
126
school
girls’
interest in
computer
science?
promote a
positive
learning
experience
for middle
school girls
and employ
culturally
responsive
teaching
strategies.)
• I can define deficit thinking
and . (strongly agree, agree,
disagree, strongly disagree)
• To what extent does
STEMaven add to your rating
on this question. (A great deal,
somewhat, a little, not at all).
• I understand what skills are
needed to have a culturally
responsive lesson. ( strongly
agree, agree, disagree, strongly
disagree)
• To what extent does
STEMaven add to your rating
on this question. (A great deal,
somewhat, a little, not at all).
• During my lesson(s) at
STEMaven, I employ
culturally responsive learning
techniques. (strongly agree,
agree, disagree, strongly
disagree)
• To what extent does
STEMaven add to your rating
on this question. (A great deal,
somewhat, a little, not at all).
• I take the time to learn about
the background of my students
and reasons to why they are
participating in the program.
(strongly agree, agree,
disagree, strongly disagree)
• To what extent does
STEMaven add to your rating
on this question. (A great deal,
somewhat, a little, not at all).
What is the
stakeholder
motivation
related to
increasing
middle
school
girls’
interest in
computer
science?
Motivation:
SE
(Mentors
need
increased
confidence
that they can
teach
computer
science to
middle
11. Please rate your self-efficacy
(belief in yourself to be successful)
for each of the following. Please
also indicate how STEMaven
added to your rating.
• I feel confident about my
ability to teach middle school
girls in computer science.(
strongly agree, agree, disagree,
strongly disagree)
Ordinal Expected
to use:
Percentage,
Frequency,
Might use:
Mode,
Range
Table, bar
graph
127
school girls
in computer
science
discipline.)
• To what extent does
STEMaven add to your rating
on this question. (A great deal,
somewhat, a little, not at all).
• How do you feel about your
ability to teach middle school
girls in computer science?
(strongly agree, agree,
disagree, strongly disagree)
• To what extent does
STEMaven add to your rating
on this question. (A great deal,
somewhat, a little, not at all).
• Navigate the aspects of the
workshops with which I
have/had little previous
experience (strongly agree,
agree, disagree, strongly
disagree)
• To what extent does
STEMaven add to your rating
on this question. (A great deal,
somewhat, a little, not at all).
• The program director
encourages me to do my best.
(strongly agree, agree,
disagree, strongly disagree)
• To what extent does
STEMaven add to your rating
on this question. (A great deal,
somewhat, a little, not at all).
• Mentors share strategies for
successfully facilitating a
workshop.( strongly agree,
agree, disagree, strongly
disagree)
• To what extent does
STEMaven add to your rating
on this question. (A great deal,
somewhat, a little, not at all).
What is the
stakeholder
knowledge
and
motivation
Motivation:
GO
(Mentors
need to see
how
12. Please rate your goal
orientation (commitment and
implementation of objectives) for
each of the following.
Ordinal Expected
to use:
Percentage,
Frequency,
Table,
bar graph
128
related to
increasing
middle
school
girls’
interest in
computer
science?
effective
their
teaching is
in increasing
retention of
interest in
computer
science.)
• Overall how satisfied are you
with facilitating a STEMaven
lesson(s)?” (very satisfied,
satisfied, neutral, dissatisfied,
very dissatisfied)
• How motivated are you to see
students succeed?” (very
motivated, somewhat
motivated, not very motivated,
not at all motivated, not sure).
Might use:
Mode,
Range
To what
extent is
STEMaven
meeting its
goal of
increasing
middle
school
girls’
interest in
computer
science?
Organizatio
nal
Influences:
CM
(The
organization
needs a
culture that
supports
change in
existing
teaching
strategies
founded on
collective
engagement,
shared
purpose, and
collaboratio
n.)
13. Please rate the organizational
identity for each of the following.
• The organization provides
professional development that
is centered around new
teaching strategies. (A great
deal, somewhat, a little, not at
all).
• PD is offered that is centered
around new teaching
strategies.
(A great deal, somewhat, a
little, not at all).
• The PD is beneficial to both
yourself and the organization.
(very helpful, somewhat
helpful, somewhat unhelpful,
not helpful at all).
Ordinal Expected
to use:
Percentage,
Frequency
Might use:
Mode,
Range
Table,
bar graph
129
APPENDIX B
Interview Protocol
Welcome
Thank you again for taking the time out of your day to meet with me. This interview
should take approximately one hour to conduct. Does that still work for you? Please feel free
to enjoy some food and drink while we talk.
Before we get started with the interview questions, I would like to provide you with an
overview of my study and answer any questions you may have. I am currently enrolled as a
doctoral student at the University of Southern California (USC) in Los Angles, California. I
am studying Organizational Change and Leadership (OCL) and part of my requirements for
fulling a doctoral degree, I am conducting a study how the STEMaven Program increases the
number of middle school girls’ who attend the computer science program from year to year
and how teachers and mentors have the ability to increase the number of middle school girls
interested in attending the computer science program through their teaching strategies.
Before we begin, I want to assure you of my role and that I will not make judgements on
your performance or role in the program. Also, I commit to keep all answers collected in
strict confidence. You have received the information of the study; do you have any
additional questions before we get started?
Follow-up after introduction: As we discussed in our initial communication, I would like
to record the interview to accurately capture what you share with me today. May I please
collect your signed form for consent to record now? If there are no additional questions or
clarifications needed, may I have your permission to begin the interview?
130
Transition into Body of Interview
Now, as we begin the interview, I will give you a quick overview of the structure of our
time together. I am going to ask some questions, and I would like to encourage you to
answer honestly and opening. Feel free to ask me to clarify any question(s). There are no
wrong or right answers and the goal today is to hear your answers and points of view as best
as possible.
Body of the Interview
1. To begin, I am hoping we could start with you sharing a little about your professional
background. What brought you to the STEMaven program? What interested you about
becoming a mentor for this program?
2. Describe STEMaven, mentors and student demographics, and its’ mission and vision
statement.
3. How would you describe your experience as a mentor at STEMaven?
4. Explain the importance of how cultural can influence learning? How do you include this
in your lesson(s) to make them compatible with the social cultural contexts of middle
school girls?
5. Deficit thinking refers to mentors creating an environment that promotes a positive
learning experience and employs culturally responsive teaching strategies. What does
‘deficit thinking’ mean to you?
6. What strategies do you employ to address deficit thinking with other mentors and/or the
director? Are students included in these strategies?
7. Tell me your goals in becoming an effective mentor at STEMaven.
131
8. In what ways has STEMaven supported your development of becoming an effective
mentor? Are there ways the program can improve its impact on your development?
9. What would mentors say in regard to the director as being proactive in identifying
internal and external factors that may derail a workshop before it is facilitated?
10. Let’s talk about mentor training. In your opinion, do you believe that this training is
focused on how to teach the lesson(s) or more emphasis on how to capture the interest of
the students?
11. What supports or structures would you like to see the STEMaven program put in place
that could further support the goal of increasing the interest of middle school girls in
computer science?
12. How do you think STEMaven has an impact on the learning and development of middle
school girls in computer science?
Possible final wrap-up questions: You mentioned XXXXXX. Can you help me understand that
a little more? Could you please tell me what you meant by XXXXX?
Final thoughts: Please let me know if there is anything that you would add to our conversation
today that we did not discuss?
Closing: Thank you very much for sharing your thoughts with me today. What you have shared
with me today is very helpful in my study and I greatly appreciate your time and willingness to
be open with me.
132
APPENDIX C
Statement of Explanation About the Study to Accompany Email With Survey
Dear STEMaven mentor participant,
As a current STEMaven mentor, you are being invited to participate in a 5-minute survey.
This survey is part of a dissertation research project investigating the knowledge, motivation, and
organizational influences that affect middle school girls’ interest in computer science.
Your participation in the survey is completely voluntary and all of your responses will be
kept confidential. You may choose to skip any question. You may also choose to end your
participation at any time. No personally identifiable information will be associated with your
responses to any reports of these data.
If you meet specific study-related criteria, you may be asked to participate in a 60-minute
interview. Interview participation is completely voluntary. If you choose to volunteer, you will
be asked to provide contact information. Your contact information will be used to contact you to
schedule a video or audio conference or in-person interview. You may decline to participate at
any time. The identity of all interview participants will be kept confidential.
Thank you for your time and willingness to participate in this dissertation research
project.
133
APPENDIX D
Information Sheet/Facts for Exempt Non-Medical Research
University of Southern California
USC Rossier School of Education
Waite Phillips Hall
3470 Trousdale Parkway
Los Angeles, CA 90089
INFORMATION/FACTS SHEET FOR EXEMPT NON-MEDICAL RESEARCH
You are invited to participate in a research study. Research studies include only people who
voluntarily choose to take part. This document explains information about this study. You should
ask questions about anything that is unclear to you.
PURPOSE OF THE STUDY
This research study aims to understand the knowledge, motivation, and organizational influences
that affect middle school girls’ interest in computer science. The study is being conducted to
provide an understanding of the factors of how this program plays a significant role in bridging
the gender gap in science, technology, engineering, and mathematics and improves the students’
problem-solving skills. Furthermore, this study aims to shed light on how STEMaven has
benefitted or failed to benefit middle school girls’ interest in computer science. The knowledge
gained from this study can be used to implement improvements and create other programs that
equip middle school girls to successfully pursue and attain STEM-related careers.
PARTICIPANT INVOLVEMENT
If you agree to take part in this study, you will be asked to participate in a 5-minute survey. If
you meet specific study-related criteria, you may also be asked to participate in a 60-minute
audio-taped interview, conducted either in-person or via video-conferencing software. You do
not have to answer any survey questions you don’t want to. You do now have to participate in
the interview if you don’t want to. If you don’t want to be taped, you cannot participate in the
interview portion of this study.
PAYMENT/COMPENSATION FOR PARTICIPATION
You will not be compensated for participation in the survey. If you participate in the interview,
you will receive $10 Starbucks gift card as a thank you for your time. You do not have to answer
all of the questions in order to receive the card. The card will be emailed to you after completion
of the interview.
CONFIDENTIALITY
Audio recordings will be kept in a password protected computer file until completion of the
dissertation project, which is anticipated to be May 2020. At that time recordings will be
destroyed. Transcripts will also be stored in a password protected computer file. You will not
have access to recordings or transcripts. All participants will be assigned a pseudonym. The
134
school or schools which employ or have employed the participants will also be assigned a
pseudonym. An executive summary of the findings will be prepared and presented upon study
completion.
The members of the research team and the University of Southern California’s Human Subjects
Protection Program (HSPP) may access the data. The HSPP reviews and monitors research
studies to protect the rights and welfare of research subjects.
When the results of the research are published or discussed in conferences, no identifiable
information will be used.
INVESTIGATOR CONTACT INFORMATION
Principal Investigator Jennifer Casey via email at caseyjen@usc.edu or phone at (423) 298-2142
or Faculty Advisor Dr. Tracy Poon Tambascia at tpoon@rossier.usc.edu or 213-740-9747
IRB CONTACT INFORMATION
University of Southern California Institutional Review Board, 1640 Marengo Street, Suite 700,
Los Angeles, CA 90033-9269. Phone (323) 442-0114 or email irb@usc.edu.
Abstract (if available)
Linked assets
University of Southern California Dissertations and Theses
Conceptually similar
PDF
Girls in STEM: the underrepresented trajectory in Tennessee: an innovation study
PDF
Computer science education in California public high schools: an evaluation study
PDF
Evaluation study: building teacher efficacy in K8 computer science integration
PDF
Quality literacy instruction in juvenile court schools: an evaluation study
PDF
Developing a computer science education program: an innovation study
PDF
Perception of alternative education teachers readiness to instruct English language learners: an evaluation study
PDF
Affluent teens and school stress: an evaluation study
PDF
Access to standards-based curriculum for students with severe and multiple disabilities: an evaluation study
PDF
The challenges teachers face effectively implementing science, technology, engineering, and mathematics (STEM) curricula: an evaluation study
PDF
An examination of resource allocation strategies that promote student achievement: case studies of rural elementary schools in Hawaii
PDF
Aligning educational resources and strategies to improve student learning: effective practices using an evidence-based model
PDF
An evaluation of project based learning implementation in STEM
PDF
Leadership and the impact on organizational citizenship behaviors: an evaluation study
PDF
Effects of mentoring on public school administrators: an evaluation study
PDF
Attracting and retaining talent: improving the impact of workplace mentorship
PDF
The role of middle manager alignment in achieving effective strategy execution: an evaluation study
PDF
Influencing teacher retention: an evaluation study
PDF
Identification of barriers to mentoring and their impact on retention and advancement of underrepresented populations in the federal government
PDF
Using mastery learning to address gender inequities in the self-efficacy of high school students in math-intensive STEM subjects: an evaluation study
PDF
Barriers to gender equity in K-12 educational leadership: an evaluation study
Asset Metadata
Creator
Casey, Jennifer Nicole
(author)
Core Title
The mentorship of instructors and its impact on computer science interest among middle school girls: an evaluation study
School
Rossier School of Education
Degree
Doctor of Education
Degree Program
Organizational Change and Leadership (On Line)
Publication Date
04/21/2020
Defense Date
03/12/2020
Publisher
University of Southern California
(original),
University of Southern California. Libraries
(digital)
Tag
Computer Science,effective and evidence-based teaching strategies,mentors,middle school girls,Motivation,OAI-PMH Harvest,STEM
Language
English
Contributor
Electronically uploaded by the author
(provenance)
Advisor
Tambascia, Tracy (
committee chair
), MacCalla, Nicole (
committee member
), Maddox, Anthony (
committee member
)
Creator Email
caseyjen@usc.edu,jncasey530@gmail.com
Permanent Link (DOI)
https://doi.org/10.25549/usctheses-c89-285834
Unique identifier
UC11674109
Identifier
etd-CaseyJenni-8291.pdf (filename),usctheses-c89-285834 (legacy record id)
Legacy Identifier
etd-CaseyJenni-8291.pdf
Dmrecord
285834
Document Type
Dissertation
Rights
Casey, Jennifer Nicole
Type
texts
Source
University of Southern California
(contributing entity),
University of Southern California Dissertations and Theses
(collection)
Access Conditions
The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the a...
Repository Name
University of Southern California Digital Library
Repository Location
USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA
Tags
effective and evidence-based teaching strategies
mentors
middle school girls
STEM