Close
About
FAQ
Home
Collections
Login
USC Login
Register
0
Selected
Invert selection
Deselect all
Deselect all
Click here to refresh results
Click here to refresh results
USC
/
Digital Library
/
University of Southern California Dissertations and Theses
/
A needs assessment of Meadows Elementary School District STEM education after school program
(USC Thesis Other)
A needs assessment of Meadows Elementary School District STEM education after school program
PDF
Download
Share
Open document
Flip pages
Contact Us
Contact Us
Copy asset link
Request this asset
Transcript (if available)
Content
Running head: A NEEDS ASSESSMENT OF MEADOWS 1
A NEEDS ASSESSMENT OF MEADOWS ELEMENTARY SCHOOL DISTRICT STEM
EDUCATION AFTER SCHOOL PROGRAM
by
Nsoah Abu-Rasool
A Dissertation Presented to the
FACULTY OF THE USC ROSSIER SCHOOL OF EDUCATION
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulfillment of the
Requirements for the Degree
DOCTOR OF EDUCATION
May 2015
Copyright 2015 Nsoah Abu-Rasool
A NEEDS ASSESMENT OF MEADOWS 2
Dedication
I dedicate this work to everyone who has influenced me during this journey, my parents,
my family, and especially my wife Angela who is also my best friend.
A NEEDS ASSESMENT OF MEADOWS 3
Acknowledgements
I would like to thank God who gives me guidance, and all of the teachers and staff of the
educational systems who have prepared me for this day. A special thanks to the chairs, Dr.
Robert Rueda, Dr. Gale Sinatra, and Dr. Helena Seli for your feedback and guidance during the
dissertation process. Additionally, with much gratitude, the co-author of this document Linda
Moon who shared her expertise with me from the beginning of this project until its completion.
A NEEDS ASSESMENT OF MEADOWS 4
Table of Contents
List of Tables 7
List of Figures 8
Abstract 9
Chapter One: Overview of the Project 10
Background of the Problem 10
Statement of the Problem 11
Purpose of the Project 12
Organization of the Project 12
Chapter Two: Literature Review 14
Science, Technology, Engineering, and Math 15
Teaching STEM 15
Increased demands for STEM backgrounds 16
Issues concerning Meadows Elementary School District 17
Afterschool programs 18
Staff self-efficacy related to STEM 18
Professional Development of Staff 19
Curriculum Assessment 20
Needs Assessment 21
Summary 21
Chapter Three: Methodology 23
Site 24
District 24
RNH Program 25
Participating Stakeholders 26
Instrumentation 27
Survey 28
Instructor capacity questions 29
Professional development questions 30
Curriculum adequacy questions 30
Additional questions 31
Focus groups 31
Interviews 32
Observations 34
Procedures 34
Preassessment 35
Initial contact 35
Preliminary site visit 35
Scope of work 36
Subsequent meetings 36
IRB and district approval 37
Assessment 37
Survey distribution 38
Focus groups 38
Interviews 39
A NEEDS ASSESMENT OF MEADOWS 5
Observations 39
Document review 40
Postassessment 40
Data analysis 40
Executive summary 40
Chapter Four: Results 42
Participants 42
Question 1: Are the Instructors Knowledgeable and Motivated to Teach STEM 45
Survey Responses 45
Focus Groups with Program Leaders 48
Knowledge 49
Motivation 50
Interviews with Program Supervisors 51
Knowledge 52
Motivation 52
Observation 53
Question 2: Is Each Site Providing Sufficient Opportunities for
Professional Development 56
Survey Responses 56
Focus Groups with Program Leaders 58
Interviews with Program Supervisors 59
Observations 61
Question 3: Is the Current Curriculum Adequate for Promoting STEM Learning 62
Survey responses 62
Focus Group with Program Leaders 63
Interviews with Program Supervisors 64
Document Review 66
Summary 67
Chapter Five: Discussion 68
Discussion of findings 69
Instructor Capacity 69
Knowledge of program leaders 69
Motivation of program leaders 71
Professional Development 72
Curriculum Adequacy 74
Recommendations 75
Instructor capacity 76
Knowledge 76
Motivation 79
Other considerations 80
Professional Development 81
Gather input about perceived training needs 81
Increase frequency of trainings 82
Restructure STEM training workshops 83
Involve outside experts 83
Curriculum Adequacy 84
A NEEDS ASSESMENT OF MEADOWS 6
Prevent supply shortages 85
Realign project-based learning with STEM instruction 85
Familiarize RNH staff with standards 85
Future Considerations 86
Limitations of Project 88
References 92
Appendices:
Appendix A: Survey Questionnaire for RNH Program Leaders & Supervisors 99
Appendix B: Focus Group Protocol for RNH Program Leaders 102
Appendix C: Interview Protocol for RNH Program Supervisors 103
Appendix D: Observation Protocol 104
Appendix E: IRB Approval Letter 106
Appendix F: Data Collection Plan 107
A NEEDS ASSESMENT OF MEADOWS 7
List of Tables
Table 1: Methods of Data Collection for Each Needs Assessment focus 28
Table 2: Summary of Survey Responses 43
Table 3: Descriptive Statistics for RNH Staff 44
Table 4: Responses to Instructor Capacity Survey Questions by RNH Staff 46
A NEEDS ASSESMENT OF MEADOWS 8
List of Figures
Figure 1: Instructor Capacity 47
Figure 2: Professional Development 57
Figure 3: Curriculum Adequacy 63
A NEEDS ASSESMENT OF MEADOWS 9
Abstract
The purpose of this project was to conduct a needs assessment of the Reaching New
Heights (RNH) afterschool program in the Meadows Elementary School District (MESD). At
the request of the RNH program director three program areas were identified as targets for data
collection and analysis, in the science, technology, engineering, and mathematics (STEM)
program. Due to advances in technology and the nuances of 21
st
century learning and
curriculums, which include project based learning, the RNH program director wanted to know
the status of the staff in order to enhance or maintain program effectiveness through professional
development training. The methods used to collect data included a survey, observations, focus
groups, and individual interviews. The data were analyzed and recommendations from the
literature and findings within the project were made available to the RNH program director. The
three primary areas addressed in the recommendations included the STEM capacity of the
program leaders/instructors, the effectiveness of professional development, and curriculum
adequacy. Additionally, for the sake of protecting the anonymity of the participants, pseudonyms
were used to refer to the school district, school sites, and programs.
A NEEDS ASSESMENT OF MEADOWS 10
CHAPTER ONE: OVERVIEW OF THE PROJECT
This document reports the results of a capstone project that was completed as part of the
Ed.D culminating program requirements. It was designed as a dissertation of practice that targets
authentic problems of practice and that provides an opportunity for students to demonstrate skills
and competencies that will be required in future career activities. The present project was a needs
assessment/evaluation of a specific program, and was designed to address concerns of this
specific site rather than being designed as a generalizable research project. In line with the goal
of reflecting real world practice, this project was carried out collaboratively between Linda
Moon and Nsoah Abu-Rasool. To accurately reflect the division of work on this project, some of
this document is co-authored, and the specific chapters are labeled to reflect the collaborative
authorship where appropriate. In addition, in order to maintain confidentiality of participants,
this dissertation uses pseudonyms to refer to the school district, school sites, and programs.
A needs assessment of an afterschool program in Southern California is the focus of this
project. This chapter contains the background of the problem, which consists of declining
enrollments in the disciplines of science, technology, engineering, and mathematics (STEM),
followed by a statement of the problem concerning the issues within this project. Additionally,
the purpose of the project and the organizational framework of the dissertation are included in
this chapter as well.
Background of the Problem
For over half a century, innovations based on science and engineering have powered the
U.S economy, creating good jobs, a high standard of living, and international economic
leadership (Atkinson, 2012). A decline in students taking STEM courses is having an effect on
many companies and their ability to replace an aging workforce with those who have STEM
A NEEDS ASSESMENT OF MEADOWS 11
related skills. The U.S economy could be headed for a critical period of uncertainty if this trend
is allowed to continue, and future generations of students are not prepared for the modern
workforce. In education, global indicators such as the most recent Program in International
Student Assessment (PISA) scores show that of the 58 countries that took the PISA, the U.S.
ranked 27
th
in science, and 30
th
in mathematics (Baldi, Jin, Skemer, Green, & Herget, 2007).
The U.S. is looking to develop courses of action in order to become more competitive
and increase its world standing in the STEM centered fields. Basham, Israel, and Maynard
(2010) state that a consensus about the nation's need to improve the 21st century skills of
students has resulted in a growing convergence among government, corporate, foundation, and
education entities to support student learning in STEM. A collective effort with public and
private organizations may be needed to address the deficiencies in the STEM pipeline, especially
in the K-12 range. According to Duran and Sendag (2012) for societies to survive in this
competitive world, it is necessary to equip individuals with skills to conduct research, use and
transform information, think critically and reflectively, and make higher order decisions.
Statement of the Problem
During most of the 20th century, the U.S. possessed peerless mathematical prowess not
just as measured by the depth and number of the mathematical specialists who practiced here but
also by the scale and quality of its engineering, science, and financial leadership, and even by the
extent of mathematical education in its broad population (Subotnik, Tai, Rickoff, & Almarode,
2010). However, in the 21
st
century, the U.S. is facing a decline in STEM enrollment at all
levels of education. In addition, an aging STEM workforce and a low world standing in the
science and math fields is now the current status of the one-time leader and innovator in these
fields. Thompson and Bolin (2011) believe that in order for the U.S. to regain its economic
A NEEDS ASSESMENT OF MEADOWS 12
status as a world superpower, an adequate number of highly qualified and professionally trained
individuals must be capable of and willing to enter into various STEM careers each year.
Purpose of the Project
The purpose of this project was to assist Reaching New Heights (RNH), an afterschool
STEM program in the Meadows Elementary School District (MESD), evaluate their program for
the 2014-2015 school year by identifying strengths and weaknesses. Additionally, the aim was
to help the RNH program develop strategies to make it more organizationally efficient and more
beneficial to the students. This project was conducted by a team of doctoral students from the
University of Southern California’s Rosier School of Education, Linda Moon and Nsoah Abu-
Rasool, who worked jointly throughout the research process. A needs assessment was used for
this project in order to investigate what should occur in the RNH program compared to what
actually occurs. The RNH program director identified three specific areas that he wanted
examined in order to see if they were functioning as they should. The areas consisted of the
following:
1) The knowledge capacity of the instructors in STEM subjects
2) The effectiveness of professional development
3) A curriculum assessment
Organization of the Project
The remainder of this report contains four more chapters. Chapter two consists of a
literature review that provides relevant background information of STEM education.
Additionally, it examines the overall research concerning STEM and the impact it has on
education and the workplace. Finally, a specific focus on STEM issues that relate to the needs of
the RNH program are discussed. Chapter three provides a description of the methods used while
A NEEDS ASSESMENT OF MEADOWS 13
conducting the needs assessment and more detail regarding the instruments that were used to
capture data. Chapter four explains the results of the findings and lastly, Chapter five contains
the discussion of the results and recommendations for the RNH program.
A NEEDS ASSESMENT OF MEADOWS 14
CHAPTER TWO: LITERATURE REVIEW
The learning environment in the 21
st
century creates new challenges for both the student
and the instructor. The promotion of STEM centered instruction may be a significant addition to
these current challenges. A great opportunity exists to expand knowledge transfer with the aid of
new discoveries in technology and the delivery systems of information. Collins and Halverson
(2009) believe that it is the second educational revolution to occur in America, following almost
200 years after the revolution that took us from apprenticeship to universal schooling. The
United States is facing a shortage of STEM professionals in the workforce which is expected to
escalate as a result of baby boomers retiring in the coming years. Data from a 2009 study of
eighth graders conducted by the National Assessment of Educational Progress showed that 43
percent of Hispanics, 44 percent of American Indians, and 50 percent of African-Americans
scored "below basic" in math (Gomez & Albrecht, 2014).
The focus of this literature review is on research concerning STEM programs. The
information from the research is designed to help the reader gain a deeper knowledge of STEM,
elaborate on best practices for after school programs in urban school districts, and provide peer
reviewed information from studies similar to the current project. Given the specific objective of
developing an effective after school STEM program, the RNH program director has decided to
center on three areas: the instructors’ capacity related to STEM content, professional
development of the instructors, and curriculum assessment. Therefore, this review targets these
areas specifically to provide background for the overall project. In addition, this chapter contains
a review of relevant research and literature concerning STEM education, as well as, a specific
focus concerning the needs of the MESD.
A NEEDS ASSESMENT OF MEADOWS 15
Science, Technology, Engineering, and Math
The idea of teaching specialized subjects in the math and science disciplines is not a new
concept. According to Chute (2009) although the roots of the science, technology, engineering,
and mathematics movement date back to President Dwight D. Eisenhower, and the formation of
the National Aeronautics and Space Administration (NASA) and National Science Foundation
(NSF) in 1958, the acronym STEM was coined by Dr. Judith Ramaley, assistant director of the
Education and Human Resources Directorate, at NSF in 2001. As STEM education becomes a
greater focus for an increasing number of schools and teachers, it becomes clearer that there is a
need to better define what is meant by the term STEM education. The term STEM is often
defined only by having the terms science, technology, engineering, and mathematics follow in
parentheses, (ITEA, 2009). STEM education is an approach in which STEM subjects are
integrated through an instructional method that uses design-based, problem-solving, discovery,
and exploratory learning strategies (Fioriello, 2010). STEM pedagogy is rooted in
interdisciplinary applied application of knowledge designed around a cooperative effort to
provide students with a comprehensive, meaningful real-world learning experience (Gomez &
Albrecht, 2014).
Teaching STEM
STEM educators will be effective in the classroom if they understand their subject matter
deeply enough so that they can explain concepts and procedures from multiple perspectives
(Ejiwale, 2012). Grubbs (2013) states that, as STEM education demands greater integration
across all subject areas, technology teachers can showcase many of the cross-curricular projects
already occurring inside their classrooms that intrigue students and build their STEM skills.
A NEEDS ASSESMENT OF MEADOWS 16
A teacher’s effectiveness can lead to more content engagement by the student and a
decrease in the extraneous load when learning the more complex features associated with math
and science. Teachers, who prepare their students to be competitive in the global markets and
workforce and fill the highly technical jobs, are the ones who are needed in educational settings.
During a time when teaching has become more complex, some politicians are claiming that
anyone can teach that teachers just need to know a little content and a few tricks of the trade in
order to succeed (Darling-Hammond, 2006). Teaching strategies that develop students’
background in STEM so that they are able to enter that line of work are needed in all STEM
educational programs. According to Leamson (2000) when working through a lesson, the really
difficult part of teaching is not organizing and presenting the content, but rather doing something
that inspires students to focus on that content to become engaged.
This section examined the discussions concerning STEM and the teaching challenges that
may occur. An effective teacher is one who has a deep knowledge of STEM content and has the
ability to interact with students and encourage them to be more engaged in the learning process.
The next section will look at the demands for a STEM qualified workforce and students entering
the STEM pipeline.
Increased Demands for STEM backgrounds
Some would consider the technology age to be in its infancy, with new discoveries and
innovations happening daily. In order to meet the ever changing demands of this phenomenon,
and to keep pace with the technological changes, more people with STEM backgrounds will be
needed. As noted by National Academy of Sciences (2007), the United States is not producing
enough graduates in STEM related fields to meet the growing demands of an increasingly
competitive global economy. Despite the anticipated need for a highly skilled and technically
A NEEDS ASSESMENT OF MEADOWS 17
proficient STEM workforce, this gap will continue to grow as enrollments in the STEM
disciplines decline and a generation of baby boomers begins to retire over the next decade in the
U.S (NSF 2008).
Engaging the student in the learning process with hands-on activities and increased
communications is not a new concept. Even though technology and workforce demands may
change over time, a student’s success is more likely to be dependent upon his or her involvement
in learning. The financial crisis and state of the national economy has caused states, schools, and
public institutions to cut costs, increase efficiency, and limit the wasting of resources. According
to the National Governor's Association (NGA), many state governors view STEM education as
key to economic growth and development and are thus producing legislation and initiatives
intended to increase the production of STEM degrees at public institutions of higher education
(NGA, 2011).
High demand for a STEM qualified workforce is on the rise. Through a strong collective
effort of communication among businesses, government agencies, and educators, those demands
may possibly be met. The next section examines how an urban school district looks to enhance
its program in order to meet the rising demands of a STEM qualified workforce.
Issues Concerning Meadows Elementary School District
The RNH afterschool program at MESD is designed to help students expand their
knowledge in STEM related subjects. A knowledgeable staff of instructors with a high capacity
for teaching STEM would be an ideal acquisition for an afterschool program. In order to ensure
that instructors are effective, a well-designed professional development program should be
established by the RNH leadership. In addition, up to date and relevant curriculum should also
be taught so that students will have the best opportunity to learn. The review of existing data
A NEEDS ASSESMENT OF MEADOWS 18
from a previous evaluation of the RNH program could point out areas of concern, as well as,
positive details that are helping the program reach their organizational goals. A study was
conducted during the 2014 school year by an outside team of Fletcher and Cantrell who focused
on the MindWorks curriculum and an evaluation of RNH staff.
Afterschool program. The Afterschool Alliance, the National Afterschool Association,
and the National Summer Learning Association deemed 2011 as the, "Year of Science in After-
School" (Afterschool Alliance, 2011). The Afterschool Alliance’s collective effort to create
sustainable and high quality afterschool programs included strategies and planning. In addition,
the Afterschool Alliance took the initiative to bring awareness to STEM centered afterschool
programs by creating a movement to get people interested in STEM. These initiatives
emphasized creating quality curriculum and materials for afterschool programs in STEM fields
and suggested training afterschool program staff, and reviewing and revising the activities
(Sahin, 2013). The RNH program has been in operation since 1999. However, in order to make
sure that the program stays effective, the staff should embrace strategies to help with overall
sustainability and training. Robelen (2011) stated that the greatest threat to afterschool programs
was the lack of institutionalization and sustainability.
Staff self-efficacy related to STEM. Tschannen-Moran (1998) explains self-efficacy,
specifically teacher efficacy, as a motivational construct. More specifically, a teacher’s belief in
his/her ability may have an important impact on how he or she approaches their work and the
levels of engagement they display. Teachers with a greater sense of instructional efficacy spend
more time attending to student learning, provide more support to students with learning
difficulties, and give students more praise; in contrast, teachers with a lower sense of efficacy
spend less time on learning, offer less time to students with learning difficulties, and provide
A NEEDS ASSESMENT OF MEADOWS 19
more criticism (Gibson & Dembo, 1984). Teachers with low efficacy in the teaching of science
often lack motivation and effectiveness, display lower levels of content knowledge, and teach
less effectively in the classroom (Bhattacharyya, Volk, & Lumpe, 2009). Teachers with lower
levels of efficacy are less motivated to put forth effort during instruction and show lower levels
of persistence. This motivational construct is important to know because the instructors of the
MESD afterschool STEM program come from a variety of backgrounds and thus, may not be
prepared or confident in their abilities to teach.
The program leaders/instructors consist of retired personnel, high school graduates and
college students. They may not all have a STEM background but enjoy the teaching experience
and are inclined to help with the program. However, the diversity of knowledge may lead to
inconsistencies in teaching. The level of self-efficacy is currently unknown among the program
leaders and may lead to challenges during knowledge transfer of STEM content when teaching.
Professional Development of Staff
Professional development programs that focus on how students learn specific content in
science and mathematics produce greater effects on student learning than programs that focus on
teacher behavior (National Institute for Science Education, 1999). Professional development
utilizing investigated scientist-teacher partnerships increased understanding and use of inquiry-
based instruction by participating teachers (Caton, Brewer, & Brown, 2000; Dresner &
Moldenke, 2002). Teachers involved in these partnerships also develop deeper content
knowledge, greater knowledge about laboratory investigation, and better capabilities for
integrating experiments with the curriculum (Siegel, Mlynarczyk-Evans, Brenner, & Nielsen,
2005). Science teachers themselves report that they believe ongoing professional development
A NEEDS ASSESMENT OF MEADOWS 20
opportunities in the geosciences are essential for them to teach the subject effectively (Science
Education Resource Center, 2003).
The quality and quantity of K-12 STEM education is inextricably linked to the continued
professional development of K-12 teachers (Nadelson, Seifert, Moll, & Coats, 2012). An
effective professional development program should help RNH instructors increase their STEM
content capacity and aid in their pedagogical skills and curriculum delivery. Knowledge of the
curriculum and the ability to use spontaneity and effectively go off script to explain concepts is a
great asset to student learning outcomes. The next section expands into the area of curriculum
assessment and delivery.
Curriculum Assessment
In order to attract and retain a new generation of learners, engineering and technology
curricula need to be renovated to optimize the skills that are relevant today (Ejiwale, 2012). In
an elementary school setting STEM curriculum needs to match the knowledge level of the
students in order for the facilitation to be effective. According to Carr, Bennett, and Strobel
(2012) curriculum providers are now experimenting with various methods of bolstering science,
technology, engineering, and mathematics education from the elementary to high school levels.
The elementary school level is the primary focus of this project; however, some of the
recommendations and practices for higher levels of education may add value to the RNH
program. In addition, the RNH curriculum leans toward a project based learning (PBL)
framework so that the students engage in more hands on learning, in contrast to the more lecture
and theory driven style used in many schools during daytime classes.
PBL also offers numerous opportunities for group work; allowing students to practice
communication skills and preparing them for the teamwork that many careers require
A NEEDS ASSESMENT OF MEADOWS 21
(Livingstone and Lynch 2000). The PBL approach has been adopted for over four decades in
many educational settings and studies have demonstrated that students working in a PBL context
show improvement in performance and skills compared to students enrolled in traditional classes
(Lai, 2013). Although the RNH afterschool program uses the PBL framework, it is designed to
enhance the learning experience and not compete with the daytime courses. However, for this
project, the consistency and proficiency of PBL in the RNH program is currently unknown. PBL
is student directed, connected to the real world, research based, informed by multiple resources,
embedded with knowledge and skills, conducted over time, and concluded with an end product
(Diffily & Sassman, 2002).
Needs Assessment
According to Altschuld and Kumar (2010) informally, need implies a problem that
should be attended to or resolved…formally, need is the measurable gap between two
conditions ”the current state and the desired state.” The purpose of this project was to conduct a
needs assessment to address the concerns that were discussed by the RNH program director.
Summary
An effective STEM education pipeline is vital in the developmental process needed to
produce a new workforce equipped with the skills required to replace an aging STEM population
of workers who are retiring. The first contact with STEM education starts with the K-5 students
who are being introduced to science and math classes for the very first time. It is important that
they receive the base knowledge that will allow them to have success as they develop and enter
into more complex curriculum in more advanced academic levels.
The issues specific to the RNH program are similar to nationwide concerns, such as
instructor’s capacity for STEM related content and curriculum management. An investigation of
A NEEDS ASSESMENT OF MEADOWS 22
the RNH program, via a needs assessment, may reveal information that will be helpful to the
program director’s efforts to develop a more effective afterschool STEM program.
A NEEDS ASSESMENT OF MEADOWS 23
CHAPTER THREE: METHODOLOGY
Authors: Nsoah Abu-Rasool and Linda Moon
The research context for this project is the RNH program of MESD. According to their
website, their objective is “using a project based learning model as a framework; we promote an
inquiry based learning environment with an emphasis on 21st century learning skills.”
("Meadows elementary school district: Reaching New Heights,"). Based on these objectives,
dialogue with the RNH program director, and extant literature, the needs assessment of the RNH
programs will be structured around the following questions:
1. Are the instructors knowledgeable about and motivated in teaching STEM material?
2. Are instructors provided sufficient opportunities for professional development?
3. Is the current STEM curriculum adequate for promoting STEM learning?
In order to provide as thorough of an assessment as possible, this project has been
executed as a collaborative effort, where each of the co-evaluators have examined these issues—
instructor capacity, professional development, and curriculum adequacy—simultaneously from
differing lenses. Nsoah Abu-Rasool directed the focus groups with the program leaders (RNH
instructors) while Linda Moon conducted the interviews with the program supervisors (RNH site
coordinators). Differentiating the work in this way allowed us to cover more ground in the time
allotted for this project. Based on our findings, we have provided Meadows RNH with a more
holistic needs assessment that takes into account data from the juxtaposition of two different but
interdependent stakeholders. This chapter is organized by first providing an introduction to the
site and participants, followed by an overview of the instruments and procedures.
1
1
Linda Moon wrote this section.
A NEEDS ASSESMENT OF MEADOWS 24
Site
2
District
MESD began operation over 100 years ago in 1906. Many of the current schools first
opened in the 1950s to educate the expanding population after World War II. In the 1960s, the
district consisted of six elementary schools and one middle school up until September 2006, the
year in which the district was celebrating its 100
th
anniversary in education. That year they
decided to convert one of the elementary schools into a second middle school and move sixth
grade classes to be part of the middle schools. To effectively meet the demands of the
restructuring, new school boundaries were established to accommodate the opening of the new
schools, which were Science Middle School (formerly Biology Elementary) and Engineering
Elementary School. In the early 2000s, thanks to a local school bond supported by matching
funds from the State of California, all of Meadows’s schools were modernized and extensively
refurbished (www.Meadows.k12.ca.us).
Currently, MESD serves students from kindergarten to eighth grade, and consists of eight
campuses—six elementary schools and two middle schools, all of which have instigated RNH
programs. The elementary schools (K-5) are Recinos, Hammer North and South, Dos Santos,
Jack Benny, Carl Sagan, Carver, and Bush, and the middle schools (6-8) are Carter and Reagan.
Hammer was formerly two different schools—King and Roosevelt, but recently became one
campus, and are now referred to as Hammer North and FDR South, respectively. For the
purposes of this project, we will only collect data from the elementary school programs.
MESD is geographically located in the South Bay region of Los Angeles County.
According to the California Department of Education website, more than 50 percent of its
enrolled students are of Hispanic or Latino origin. Their Academic Performance Index (API)
2
Nsoah Abu-Rasool wrote this section. Linda Moon made contributions.
A NEEDS ASSESMENT OF MEADOWS 25
reports also show that a vast majority of the students come from socioeconomically
disadvantaged backgrounds.
RNH Program
It is within this context that the RNH program has been implemented. The program has
been around since 1999 and is primarily funded through the After School Education and Safety
(ASES) and 21
st
Century Community Learning Centers. Additionally, RNH maintains several
partnerships that help provide resources to the program and a connection to the community. The
Meadows RNH partnerships consist of Loyola Marymount University (LMU), Inside Arts, Press
friends, NASA’s Best, and Trash for Teachers. A few more contributors for the Meadows RNH
training resources include, Local Learning Community (LLC), Los Angeles County Office of
Education (LACOE), the LACOE After School Technical Assistance Unit (ASTAU),
MindWorks, LEGO Mindstorms, Pearson Science Curriculum, and Discovery Science.
RNH started mainly with homework help, arts and crafts, recreational activities and other
supplemental activities but has evolved into more intentional programming, which now includes
fitness, healthy behaviors, STEM, and 21
st
century learning skills. Moreover, they are trying
now to place kids in more leadership roles and have them facilitate as well. The key element
about RNH is that it is not an intervention or just an enrichment program but a program that
“extends the day” to provide students with more opportunities to learn outside of the classroom
without replicating what’s done in the classroom. The overarching mission of RNH is to provide
abundant opportunities and experiences for all, and their goals to that end are to provide holistic,
student-driven enrichment comprised of academics, sports and fitness, community partnership
and leadership opportunities. Being both a before and after school program, each RNH site is
open an hour before school officially starts, and from school dismissal until 6:00 p.m. daily,
A NEEDS ASSESMENT OF MEADOWS 26
except for holidays. For the scope of this project, we only assessed the programming that takes
place after school dismissal.
Participating Stakeholders
3
Together with the RNH director, we have identified the stakeholders of the RNH program
to be the program supervisors (site administrators), program leaders (instructors), parents,
students, schools, district specialists, union (classified), and school board. Among these
stakeholders, the most salient are the administrators, instructors and students. The program
supervisors have the largest influence as they deal directly with the policies and programming of
RNH. They have a vested interest in this needs assessment as they will need to justify to their
board both past and future funding to maintain the program. As for the program leaders, while
they may not have as pronounced an impact on the organization, they are the ones interacting
with the students on a regular basis and thus have a better understanding of how effective the
program is for the students themselves. Students are the most pertinent beneficiaries of an
effective STEM afterschool program, because the outcome would be an increase in their
academic achievement in STEM subjects, which in turn will provide educational and
professional opportunities for them in the future. RNH serves more than 1,800 students five
days a week at all eight MESD school sites.
For the purposes of this project, we are mostly interested in the stakeholders at the staff
level—program supervisors and program leaders—because they are both the recipients and
agents of change for implementing strategies for improving STEM education. There are 120
full-time and part-time staff members with varying levels of experience with STEM. The full-
time administrators, or program supervisors, oversee the program leaders. There are eight
program supervisors in MESD, one at each site. There are approximately ten part-time
3
Linda Moon wrote this section. Nsoah Abu-Rasool made contributions.
A NEEDS ASSESMENT OF MEADOWS 27
instructors, or program leaders, at each site who work approximately three and a half hours a day
after normal school hours. Their average age is about 20-25, ranging from recent high school
graduates to retired practitioners.
We have narrowed the scope of this project to include only the elementary school sites,
such that our participant pool consisted of 6 program supervisors and 50 program leaders. To
provide a more comprehensive needs assessment, we have divided up the qualitative work such
that one of us interviewed program supervisors while the other led the focus groups with
program leaders.
Instrumentation
4
We used a combination of interviews, focus groups, surveys, observations, and document
review to gather data. Table 1 shows which methods of data collection were applied to each of
the three needs assessment foci—instructor capacity, professional development, and curriculum
adequacy. Much of the instrumentation is the same for both program leaders and program
supervisors, both having participated in aggregated research (i.e., answering survey questions)
and disaggregated research (i.e., interviews or focus groups). However, the questions had slight
differentiation in wording such that it is tailored to the participants’ roles in the RNH program.
For example, if a question asked about program leaders’ motivation, program leaders would
supply their responses from their own perspective whereas program supervisors would discuss
their perceptions of the program leaders’ motivation.
4
Linda Moon and Nsoah Abu-Rasool wrote separate subsections of this section, as indicated by subsequent notes.
A NEEDS ASSESMENT OF MEADOWS 28
Table 1
5
Methods of Data Collection for Each Needs Assessment Focus
Focus Survey
Interviews/
Focus Groups Observations
Document
Review
Instructor capacity X X X
Professional development X X X
Curriculum adequacy X X
Survey
6
Two surveys, one for program leaders and one for program supervisors, were distributed
via Qualtrics. The survey link was distributed by the program director to each of the participants
and the survey was made available online for two weeks. Both groups received the same set of
questions, with a few revisions on wording to best fit each category of respondents (see
Appendix A). The survey was constructed by adapting from existing instruments such as the
Teacher Efficacy Scale, Teachers’ Attitudes About Professional Development Scale, Curriculum
Emphasis Survey, and the Power of Discovery STEM
2
’s Readiness and Needs Assessment Tool.
We also collaborated with the RNH program director in order to ensure that we were asking the
kinds of questions he was interesting in learning more about, in addition to revising some of the
wording so the questions can be more accessible to the population. The researchers also supplied
additional questions as deemed necessary such as demographic questions and other pertinent
questions specific to the site that was not covered by the existing instruments (e.g., “How often is
professional development such as staff training offered at your site?). The questions were
5
Linda Moon created this table.
6
Linda Moon wrote this section.
A NEEDS ASSESMENT OF MEADOWS 29
formatted for rating on a 6-point Likert scale (1 = “strongly disagree,” 2 = “disagree,” 3 =
“somewhat disagree,” 4 = “somewhat agree,” 5 = “agree,” and 6 = “strongly agree”). The entire
survey had a total of 63 items (41 questions with some of them involving multiple parts): 24
instructor capacity items (11 for knowledge and 13 for motivation), 9 professional development
items, and 8 curriculum adequacy items.
Instructor capacity questions. To measure instructor capacity, the questions were about
the extent of the program leaders’ knowledge and motivation. The 11 knowledge questions
consisted of both STEM content and pedagogical content knowledge where respondents had to
indicate to what extent they agree with statements such as, “I am aware of the basic principles of
STEM teaching and learning,” and “I know how to use effective teaching strategies to guide
student thinking and learning in STEM.” The same questions were posed to program supervisors
with the pronoun “I” replaced with “program leaders,” such that the question would be aimed at
gauging program supervisors’ perceptions of their program leaders’ knowledge. We ran
reliability analyses on these survey questions in order to determine the internal consistency of all
the items, where a reliability coefficient higher than .7 is favorable. After running a reliability
analysis on the knowledge questions, the Cronbach’s alpha (α) for program leaders was .91, and
the alpha for program supervisors was .94.
The 13 motivation questions were also the same for both program leaders and program
supervisors, but since the latter group was reporting their perceptions of program leaders’
motivation, the measurement scales were adjusted accordingly. Overall, the motivation scale
was highly reliable for program leaders (α=.95). The motivation questions were further divided
into the subscales of self-efficacy (3 items), teaching beliefs (7 items), value (2 items) and
interest (1 item). Cronbach’s alpha for the self-efficacy, teaching beliefs, and value subscales
A NEEDS ASSESMENT OF MEADOWS 30
were .88, .93, and .90, respectively. Reliability analysis was not conducted on the interest
segment, as it was only one question. For the program supervisors, the self-efficacy, value, and
interest questions were grouped together to measure overall perception of motivation (α=.85).
The teaching belief questions were converted to measure program supervisors’ perceptions of
their program leaders’ influence on students, creating an outcome expectancy subscale, which
yielded a reliability alpha of .95.
Professional development questions. The professional development questions were
aimed at determining whether or not program leaders were provided sufficient opportunities for
training and resources. There were nine items in total, four of which were dedicated to
ascertaining the availability of professional development, and five pertained to how important
program leaders and program supervisors believed professional development was to the program.
An example of an availability item was, “I (Program leaders) receive the training I (they) need to
be successful” and a sample importance item would be, “Professional development workshops
are vital to helping program leaders develop new teaching techniques.” Each of these statements
were rated by the respondents on the same 6-point Likert scale ranging from 1=”strongly
disagree” to 6=”strongly agree.” Reliability analysis for these questions yielded a Cronbach’s
alpha of .92 for availability and .87 for importance.
Curriculum adequacy questions. The final portion of the survey were questions about
the current STEM curriculum being utilized at RNH in order to determine its adequacy in
promoting STEM learning among students. Sample items include identifying competencies that
students can glean based on the curriculum (e.g. “use scientific vocabulary and principles in
everyday discussions”) and whether or not STEM-centered processes (i.e. inquiry skills, problem
solving, and critical thinking) are incorporated into the current STEM curriculum. There were 9
A NEEDS ASSESMENT OF MEADOWS 31
items in all for this section, all of which were also answered according to the 6-point Likert
scale. The Cronbach’s alpha for the curriculum adequacy questions was .94.
Additional questions. At the end of the survey, we included several demographic
questions for the purposes of descriptive statistics and accounting for external factors and
covariates. The demographic questions consisted of respondents’ gender, age, level of
education, number of years teaching/administering at RNH, intended or completed
undergraduate major. Additionally, we added a few questions that were not directly relevant to
the needs assessment but included anyway in order to obtain as rich a data set as we could. An
example of such a question would be, “Our program supports improvement in student academic
performance.”
Focus Groups
7
The focus group interviews with the program leaders were structured to identify their
knowledge of STEM principles as well as their motivation for teaching STEM classes. We
developed a focus group interview protocol to keep track of interview times, locations, and to
capture response data (see Appendix B). In addition, the prepared questions, in the protocols,
helped to establish discussions and to stay focused on the interview topic. The focus group
interviews provided the researcher a personal view of the program leader’s thought processes as
it related to STEM. The focus group protocol was designed with the expectation of generating a
candid discussion among the program leaders that would deliver valuable data related to their
STEM capacity and motivation. In addition, the focus group questions were also designed to
capture the information processing method of the program leaders by asking them to give
examples of how they used motivation and STEM concepts in the classroom. The focus group
interviews were conducted at each RNH site during the three month data collection period.
7
Nsoah Abu-Rasool wrote this section.
A NEEDS ASSESMENT OF MEADOWS 32
In order to thoroughly investigate the program leaders’ needs, in relation to professional
development, questions were asked during the focus group interview that targeted how they felt
about receiving adequate support and professional development training from the administrative
staff. The questions in the focus group interviews were framed so that the program leaders
would be encouraged to reveal his or her point of view about professional development and how
it has impacted their instruction directly. In addition, inquiries about the need for more resources
and other types of professional development support that could help the program leaders become
more efficient and prepared for teaching STEM were also asked and discussed.
The final needs assessment question, covered in the focus group interviews, centered on the
effectiveness of the RNH program’s curriculum. Questions concerning PBL and 21st century
learning were asked in order to find out how the program leaders felt about the current state of
the RNH program’s curriculum effectiveness. Additionally, the program leaders were asked if
they thought that the curriculum was in alignment with the RNH program’s goals. The focus
group questions were also designed to foster an environment, for the program leaders, that would
trigger open discussions about their experiences with the different curriculum vendors used
within the RNH program. Additionally, the response data from the program leaders could then
be analyzed and used by the RNH program director to ensure that the most efficient curriculum
would be used that offered the most value and benefit to the student’s STEM learning
experience.
Interviews
8
The program supervisors participated in one on one interview sessions with the USC
team during this study. An interview protocol was designed for the program supervisors that was
8
Nsoah Abu-Rasool wrote this section.
A NEEDS ASSESMENT OF MEADOWS 33
similar to the focus group protocol for the program leaders (see Appendix C). However, the
interview questions were created specifically for the program supervisors in the project.
The questions were first in regard to the program leader’s STEM capacity and the
program supervisor’s perceptions of how knowledgeable and efficacious the program leaders
seemed to be with respect to teaching STEM material. Since the program supervisors were
responsible for scheduling and implementing professional development, the next set of interview
questions were framed around what they thought professional development should look like and
what should be the desired result upon completing training. For example, some of the questions
covered during the individual interviews with the program supervisors asked, how well is the
professional development training attended, what is your perception of what a program leader
should know after they attend training, and if there are any observable changes in the program
leaders as a direct result of professional development, what should it look like.
Different curriculum vendors within the RNH program are used by the program leaders
to teach STEM courses. The program supervisors were asked, during the interviews, what was
the need for such a variety of curriculum vendors to teach STEM and which one seemed to be
the most beneficial to the RNH students? Again, questions were designed to see if there was an
alignment between the program leaders and the program supervisors in their understanding of the
curriculum vendors that provided the most benefit to the RNH students.
The interviews with the program supervisors were conducted individually to avoid
conflict on sensitivity issues or anything unforeseen that could have occurred during a combined
focus group interview with the program leaders. In addition, we wanted to create a nonintrusive
or threatening environment in order to allow the most freedom, for all participants within the
study, with the intent of getting the most candid response. Additionally, the interviews were
A NEEDS ASSESMENT OF MEADOWS 34
designed to see the point of view of the program supervisors in relation to the actions or
perceptions of the program leaders as they navigated through the same three needs assessment
questions.
Observations
9
Observations were conducted at each RNH site in order to view program leaders in the
classroom setting as they taught STEM content to their students. An observation protocol was
developed to capture descriptive data of the RNH sites and actions of the program
leaders. Sections in the protocol included adding details about the physical space of the site,
demographics of participants, and the sequential actions of the program leaders (see Appendix
D). The aim of these observations was to further illuminate findings from the focus group
sessions and in the Qualtrics survey regarding their views on teaching STEM. Access to the
various school sites, where the observations were conducted, was arranged and authorized
through prior coordination with the MESD RNH program director.
A total of seven observations occurred, six of which were mainly of the program leaders,
at each one of the participating elementary schools in the RNH program. The program leaders’
interaction with students, their teaching of STEM subjects, use of the STEM curriculum, along
with classroom environment was documented during the course of the observations. The final
observation was of a professional development session at which the same protocol was used to
assess the STEM workshops offered.
Procedures
10
The needs assessment for this project follows the framework of the three-phase model
designed by Altschuld and Kumar (2010). This model was used because it offers the opportunity
9
Nsoah Abu-Rasool wrote this section.
10
Linda Moon wrote this section. Nsoah Abu-Rasool made contributions.
A NEEDS ASSESMENT OF MEADOWS 35
to do a preassessment, assessment, and postassessment of the sites pertaining to this project.
Following this model, we were able to determine the most pressing needs in RNH during the
preassessment phase, conduct further research on these needs during the assessment phase, and
formulate recommendations based on those results in the postassessment phase.
Presassessment
To carry out this preassessment phase, we held meetings with a liaison at STEM
2
(a
consortium of STEM afterschool programs in California), a project coordinator from Los
Angeles County of Education’s (LACOE), the RNH program director at Meadows, and our
faculty committee chairs. From these conversations, we were able to determine and prioritize
RNH’s needs that have driven this project.
Initial contact. The representatives from STEM
2
and LACOE were our primary contacts
that helped set up many of the meetings to ensure collaboration between the University of
Southern California (USC) and MESD. It was through their networks that the USC team was
able to get in contact with MESD to make RNH the focus of this project. They were both
instrumental in helping us get the project going by initiating phone meetings and site visits with
the RNH director. Our first meeting was held via phone conference on March 20, 2014, with the
RNH director and our committee chairs in attendance. During this meeting, we were given a
general overview of the RNH program, including its goals. In addition, the history of the RNH
program within MESD and STEM-related concerns were also discussed during the meeting. The
RNH director felt at this time that a needs assessment would be most beneficial to understand
how to further their STEM-focused structures.
Preliminary site visit. Our second point of contact was a site visit on May 9, 2014,
attended by my co-evaluator and committee chairs. In addition to observing the site facilities,
A NEEDS ASSESMENT OF MEADOWS 36
the program director gave the group an overview of their curriculum, activities, and the different
partnerships they have developed over the years. Afterward, the group discussed the next steps
for the needs assessment by talking through what RNH perceived their needs were and what our
research team could assist with that was within our scope. Upon listing these items, the group
formulated a timeline for actions, roles and responsibilities to be upheld by both parties.
Scope of work. Based on these conversations, our liaison from LACOE’s Technical
Assistance Unit drafted a Scope of Work that detailed the nature of our relationship with the
Meadows RNH site and distributed it to the USC team on June 3, 2014. Upon receipt, the USC
team clarified the capacity to which we were able to participate in the project, since the RNH
program director had originally identified eight needs to investigate. It was at this point that we
narrowed our focus to providing recommendations for instructor capacity, professional
development, and curriculum adequacy, along with assistance with analyzing existing data from
an evaluation conducted the year before by an outside group (Fletcher & Cantrell, 2014). The
resulting Scope of Work document outlined RNH’s objectives, needs, stakeholders, and
resources as well as the support capacity that could be provided by our USC research team.
Subsequent meetings. Another phone meeting on July 25, 2014 was held at which we
confirmed the contents of the Scope of Work document. We reviewed the proposed timeline of
events and responsibilities of both USC’s research team and Meadows RNH. A follow-up on
site meeting was scheduled for August 12, 2014 where we planned to further discuss how to
proceed in detail regarding the needs assessment process, as well as receive hard copies of the
2014 RNH evaluation for analysis.
A NEEDS ASSESMENT OF MEADOWS 37
IRB and district approval. We completed our Collaborative Institutional Training
Initiative (CITI) certification through the CITI program website in February 2014. The online
training involved modules that explained various facets of research involving human subjects.
We submitted an application for approval of our project from the USC Institutional Review
Board (IRB) on September 22, 2014. We submitted for approval as an exempt study because our
research will only involve adults over the age of 18. The USC IRB committee approved our
project on September 27, 2014 (see Appendix E).
As for working with the MESD, we were not required to fill out any additional contracts
other than submit the Scope of Work to the RNH program director. Following the necessary
protocol set by the MESD Department of Human Resources, we each completed TB testing and
background checks and were cleared for the project by the district on October 9, 2014.
Assessment
In the assessment phase, the research team collected data using various methods in hopes
of identifying root causes and trends of the needs. We utilized similar instruments but will be
addressing administrators (program supervisors) and instructors (program leaders) separately.
We employed more than one method of data collection for each of the three needs to be assessed
in order to triangulate the data and provide more accurate feedback in the postassessment phase.
We created a data collection plan that included dates and times, which the RNH program director
and the assistant director then distributed to all the schools involved in this project (see Appendix
F). The rationale for conducting our research only on Thursdays was because all of Meadows
had early dismissal on Thursdays (1:00pm), so the RNH program is longer and there is more
flexibility. Also, Thursdays were the days each site usually held their weekly meetings so we
A NEEDS ASSESMENT OF MEADOWS 38
were able to conduct interviews and focus groups during their regular meetings time as not to
take away from program time.
Survey distribution. After creating a questionnaire using Qualtrics, we made the survey
available through an online URL that we sent to the RNH program director and the assistant
director. From there, these RNH administrators took the prerogative to forward the link to the
six elementary school RNH sites. The link remained active for two weeks during which time
program leaders and program supervisors were required to complete it. While the program
leaders and program supervisors were strongly encouraged to participate, participation was
completely voluntary and there was no compensation for completing the survey.
Focus groups. Similarly, we conducted in-person focus groups with program leaders to
garner their opinions and views regarding their experiences within RNH, professional
development opportunities, and the STEM curriculum in use. The program director asked each
site to assemble three to four program leaders to participate in the focus group. The resulting
group was a well-balanced mixture of veteran as well as newer program leaders, including at
least one activity specialist—a former program leader and assistant to the program supervisor at
each site. The duration of each focus group lasted approximately 40 minutes, and Nsoah Abu-
Rasool was the main interviewer during these sessions, although Linda Moon was also in
attendance. We met with the group at Hammer Elementary on October 23, 2014, Jack Benny on
October 30, 2014, Carver on November 13, 2014; Recinos on December 4, 2014, Dos Santos on
December 11, 2014, and Bush on December 18, 2014. For transcription and note-taking
purposes, we asked each of the participants for permission to record the audio of the focus group.
For confidentially purposes, our results indicate only the content of the focus groups, and the
identity behind specific statements is kept anonymous.
A NEEDS ASSESMENT OF MEADOWS 39
Interviews. We conducted in-person interviews with each of the RNH program
supervisors in MESD, or administrators, to garner their opinions and views regarding their
program leaders, professional development, and the STEM curriculum in use. Each interview
was 30 to 40 minutes long, and Linda Moon was the main interviewer during these sessions,
although Nsoah Abu-Rasool was also in attendance. We met with the program supervisor from
Hammer on October 23, 2014; the supervisors from Carver and Jack Benny on November 13,
2014; the supervisor from Recinos on December 4, 2014; and the supervisors from Dos Santos
and Bush on December 11, 2014. For transcription and note-taking purposes, we asked each of
the interviewees for permission to record the audio of the interview. For confidentially purposes,
our results indicate only the content of the interviews, and the identity behind specific statements
is kept anonymous.
Observations. We also spent 90-120 minutes on observations at each site on the same
days we held interviews with the program supervisors. We visited two to three RNH classrooms
during the time we had allotted for observations. Each of the sites accommodated our schedules
by rearranging their schedules so that the classrooms we visited would be focused on STEM. All
of the sections we observed used the MindWorks curriculum, and following our makeshift
observation protocol, we jotted notes on the classroom environment and the events that
transpired during that time. We did not interact with the students or intervene with the integrity
of the class in any way. The main purpose of these observations was to validate or contradict the
information gleaned from the surveys and interviews or focus groups.
We also attended a professional development session which took place at Hi-Tech
Middle School on January 31, 2015. This was a series of workshops hosted by the Local
Learning Community (LLC) which is a consortium of five different South Bay school districts:
A NEEDS ASSESMENT OF MEADOWS 40
Meadows, Hi-Tech, Santos, Sunnyside, and Happy Valley. We sat in on the two STEM
workshops that were offered, both of which were about how to apply scientific thinking.
Document review. At the request of the RNH director, we reviewed two major
documents for the purposes of this study—the 2014 evaluation of RNH (Fletcher & Cantrell,
2014) and samples of the MindWorks curriculum. The purpose of reviewing these documents
was predominantly to guide the current project and make recommendations in Chapter 5.
Postassessment
In this third and final phase, all of our findings have culminated in an action plan for the
RNH program. We have coalesced in Chapter 4 the qualitative and quantitative data about
program leaders’ and program supervisors’ views regarding the three foci of this study and
provide detailed feedback and recommendations in Chapter 5.
Data analysis. We have transcribed and coded qualitative data, such as interviews, focus
groups, and observations, to identify transcending themes and patterns. All quantitative data,
such as survey responses, were inputted and analyzed using SPSS software. We have focused
predominantly on descriptive analyses–means and standard deviations–and simple comparative
analyses where appropriate.
Executive summary. We have summarized our findings and have reported back to the
district and RNH program by constructing a five-page executive summary in the form of a needs
assessment report, as outlined by Altschuld and Kumar (2010), that includes,
1) Scope of the report,
2) Front material, authorship and acknowledgements,
3) The three phases and what was done in each,
4) Results for phase I,
A NEEDS ASSESMENT OF MEADOWS 41
5) Results for phase II,
6) Results for phase III, and
7) Appendices (p. 145).
By addressing instructor capacity, professional development, and curriculum adequacy from the
lenses of both, the program leaders and program supervisors, our hope was to have provided a
more comprehensive needs assessment from which RNH can enact necessary changes and
maintain the strong components of the program.
A NEEDS ASSESMENT OF MEADOWS 42
CHAPTER FOUR: RESULTS
Authors: Nsoah Abu-Rasool and Linda Moon
We collected data from each of the six RNH sites of the MESD using surveys, focus
groups, interviews, and observations in order to answer the following three questions for this
needs assessment:
1) Are the instructors knowledgeable and motivated in teaching STEM material?
2) Is each site providing sufficient opportunities for professional development?
3) Is the current curriculum adequate for promoting STEM learning?
The use of qualitative and quantitative methods enabled us to gather rich and meaningful
data from both the program leaders and the program supervisors. In addition, we were able to
create a more holistic understanding of the various needs of the STEM portions within the RNH
program at each site.
11
Participants
12
Table 2 shows a summary of survey response rates, based on the initial distribution to 50
program leaders, and six program supervisors. We initially had a 96% response rate (N=48) for
program leaders, but three of the response sets were eliminated because the survey was not
completed. There was no indication or known reason why three respondents did not complete
the survey. Thus, the Qualtrics survey analysis is based on 45 program leaders, with ages
ranging from 18 to 60, with an average age of 25 (SD=7.667). Female participants (N=29)
outnumbered the male participants (N=15), while one respondent did not indicate a gender. All
six program supervisors completed the survey, with their ages ranging from 28 to 42, and
average age of 32 (SD=5.62). There were more female participants (N=5) than male participants
11
Linda Moon wrote this section. Nsoah Abu-Rasool made contributions.
12
Linda Moon wrote this section.
A NEEDS ASSESMENT OF MEADOWS 43
Table 2
13
Summary of survey responses.
Participant Sent Returned Valid Response rate
Program leaders N=50 N=48 N=45 90%
Program supervisors N=6 N=6 N=6 100%
13
Linda Moon created this table.
A NEEDS ASSESMENT OF MEADOWS 44
Table 3
14
Descriptive statistics for RNH program leaders (N =45) and program supervisors (N=6).
N M SD
Program Leaders
Gender 44* 1.66 .48
1 = Male 15
2 = Female 29
Age 45 2.23 .70
1 = Ages 18-21 11
2 = Ages 22-24 19
3 = Ages 25+ 15
Previous teaching experience 45 2.49 1.60
1 = None 22
2 = Less than 1 year 8
3 = 1 to 3 years 7
4 = More than 3 years 8
Years at RNH 45 2.44 1.10
1 = Less than 1 year 11
2 = 1 to 3 years 13
3 = 3 to 5 years 11
4 = More than 5 years 10
Program Supervisors
Gender 6
Male 1
Female 5
Age 6 32 5.62
Years in current position 6 6.83 3.06
Note. *1 missing value in this category
14
Linda Moon created this table.
A NEEDS ASSESMENT OF MEADOWS 45
(N=1), and the average amount of time they have been in their current positions is almost seven
years (M=6.83, SD=3.06). Table 3 provides a summary of these statistics.
In addition to partaking in the online survey, only the program leaders participated in the
focus groups and only the program supervisors participated in the individual interviews. Three
to four program leaders at each site participated in the 30-minute focus group session, for a total
of 24 program leaders. One or two activity specialists, who were all formerly program leaders,
also sat in on the focus group sessions but they did not partake in the online survey. We also
interviewed all six program supervisors in 30- to 40-minute sessions.
Question 1:
Are the Instructors Knowledgeable and Motivated to Teach STEM?
Survey Responses
15
To measure program leaders’ knowledge and motivation to teach STEM, both program
leaders and program supervisors were given survey questions comprised of Likert scale items
(1=strongly disagree; 6=strongly agree). The motivation portions for the program leaders were
further broken down to assess program leaders’ self-efficacy, teaching beliefs, value, and
interest. The program supervisors, in addition to being asked about their perceptions of their
program leaders’ knowledge and motivation, were also asked about outcome expectancies—
whether or not they felt that the program leaders were directly influencing STEM learning
among the RNH students.
The average values for the two components of instructor capacity, knowledge and
motivation, as reported by program leaders and program supervisors are summarized in Table 4.
The overall mean for program leaders’ knowledge as reported by program leaders was 4.75
15
Linda Moon wrote this section.
A NEEDS ASSESMENT OF MEADOWS 46
Table 4
16
Responses to instructor capacity survey questions by RNH program leaders (N =45) and
program supervisors (N=6).
N M SD
Program leaders
Knowledge 45 4.75 .77
Motivation 45 5.03 .86
Self-efficacy 45 5.01 1.02
Teaching beliefs 45 4.93 .85
Value 45 5.24 .93
Interest 45 5.05 1.07
Program supervisors
Knowledge 6 3.82 .56
Motivation 6 4.24 .65
Outcome expectancy 6 4.36 .78
16
Linda Moon created this table.
A NEEDS ASSESMENT OF MEADOWS 47
(SD=.77), while the mean for program supervisors’ perceptions of program leaders’ knowledge
was slightly lower at a value of 3.82 (SD=.56). For motivation, the overall means as reported by
program leaders and program supervisors were 5.03 (SD=.86) and 4.24 (SD=.65), respectively.
For the program leaders, the motivation questions were further analyzed by separating into four
components: self-efficacy (M=5.01; SD=1.02), teaching beliefs (M=4.93; SD=.85), value
(M=5.24; SD=.93), and interest (M=5.05; SD=1.07). Program supervisors’ perceptions of their
program leaders’ influence on students was measured as the outcome expectancy scale, which
yielded a mean of 4.36 (SD=.78).
Figure 1. Mean instructor capacity responses. This figure illustrates the differences in responses
to knowledge and motivation questions by 45 RNH program leaders and 6 program supervisors.
The mean scores along with the standard deviations are shown.
17
In this way, we were able to assess both program leaders’ and program supervisors’
views on the two critical components of instructor capacity—knowledge and motivation—using
the same survey questions. Program leaders answered questions pertaining to their own
17
Linda Moon created this figure.
A NEEDS ASSESMENT OF MEADOWS 48
knowledge and motivation, whereas program supervisors provided their perceptions of their
program leaders’ knowledge and motivation. Program leaders reported somewhat higher means
for both knowledge and motivation than did program supervisors (see Figure 1). The
significance of these reported values will be further discussed in Chapter 5.
Focus Groups with Program Leaders
18
Focus group interviews were conducted at all six of the RNH sites in order to determine
the program leader’s knowledge and motivation of STEM centered content. The aim was to ask
questions that would allow the program leaders to elaborate on the interview topic and respond
as a participant in a casual discussion. For example, when the program leaders were asked about
their understanding of STEM education there was no universal response that would show
communication or a common philosophy of a STEM concept within the organization. However,
the program leaders did respond with their own individual account of what STEM education is
and what it means from their perspective.
The focus group sessions occurred about one hour prior to the program leaders start time
for their STEM classes, most took place in the RNH program supervisor’s office, except for two,
which were held in classrooms used by the program leaders prior to the start of class. The focus
group usually consisted of three to five program leaders and one activities specialist. The
activities specialist at each site managed the different educational clubs within the RNH
program, which are designed to enhance student interest and give them a sense of autonomy in
their learning. The educational clubs are chosen by the students, this dynamic gives them the
chance to learn more or participate in a subject which they have interest and they have requested.
The design of the focus group interviews consisted of a combination of structured and
semi-structured questions which allowed the researcher to stay on target with questioning as well
18
Nsoah Abu-Rasool wrote this section.
A NEEDS ASSESMENT OF MEADOWS 49
as provide the flexibility consistent with an open dialogue, which encouraged the participants to
express their point of view, and understanding of STEM and motivation related topics. The
environment at all sites appeared to be relaxed and the program leaders seemed to be candid in
their responses without any noticeable uncooperative vibes or tensions. Descriptive data, such as
the amount of years they have worked in the RNH program and educational levels, was collected
from the program leader’s responses to the focus group interviews. Additionally, the accounts of
the program leaders motivation and STEM knowledge capacity provided valuable details for data
collection.
Knowledge. To get a sense of the program leaders’ STEM knowledge a series of
questions were developed for the focus group interview. The intent of the questioning was to
find out the STEM related background of the program leader, and their overall knowledge of
STEM. For instance, when asked, “What is STEM education?” one program leader laughed and
simply recited what the letters in the acronym stood for, while another replied, “STEM education
leans toward innovation so that kids can have an experience that will inspire them to someday
become scientist.” At all six sites, where the program leaders participated in the focus group
interviews, there appeared to be no common theme or consistent response of the definition or
perception of STEM education. Overall, the answers were quite different and it seemed that
each program leader had their own vision of what STEM actually meant to them and how much
they actually knew about the subject.
With respect to STEM content knowledge, the program leaders generally felt that they
did not have sufficient background knowledge of the material, but felt they knew enough to teach
the material to their students. When asked about specific STEM-related skills, such as project-
based learning or inquiry skills, the responses varied significantly. For instance, one program
A NEEDS ASSESMENT OF MEADOWS 50
leader stated, “ Problem based learning is a base but sometimes because of lack of materials we
have to improvise when we are low on materials.” Another program leader, when asked about
project based learning, responded, “Let me try to remember, it’s not lecturing it’s more like
hands on, so you spend like five minutes giving instructions and teaching the rest of the time.”
Especially with respect to project-based learning, because this is the name for the form they fill
out for their clubs, the discussions about PBL were mostly about their processes about how they
fill out the form, rather than the broader concept of PBL as related to STEM.
In terms of prior STEM knowledge, all of the program leaders revealed they did not have
extensive background knowledge or previous experience with STEM. One program leader
stated that, “I didn’t know anything about STEM until about two or three years ago when I
started working here in the RNH program.” Another program leader revealed that they had taken
a robotics class in high school and that was their only experience to STEM prior to working in
the RNH program.
Motivation. Although the program leaders collectively admitted to having a limited
knowledge of STEM content, this did not consequently translate to any ineptitude or lack of
confidence and motivation in their ability to teach STEM subjects effectively. Moreover, even
though the program leaders admitted, in the focus group interviews, that their knowledge outside
of the given curriculum was limited, they still seemed motivated to teach because they felt that
they were having a positive effect on the student’s academic growth.
The motivation section of the focus group interview included questions aimed at finding
out why the program leaders taught STEM and what was the motivation behind wanting to teach
STEM. Together the program leaders felt that they were achieving some sort of goal by teaching
STEM to the students. For example, one of the program leaders stated,
A NEEDS ASSESMENT OF MEADOWS 51
The excitement in the kids, I feel that is the biggest motivation for me, to see them
excited about learning, the smiles on their faces when they are building something, and
knowing I had a part, that is my motivation.”
During another focus group interview, one of the program leaders revealed that their
daily goal is to enhance the lives of their student’s in the RNH program.
The program leaders responses, to the motivation questions, seemed to follow a constant
theme of wanting to teach and enhance the academic lives of their students. In addition, they
seem to be very motivated to teach their students and help them achieve academic growth. For
instance, a program leader stated, “the reality is that I’m supposed to be teaching them something
but in actuality they are teaching me something and they don’t even realize it, that is really
motivating to me and it can’t get any better than that.” Another program leader said, “I think it is
just interesting to see how they take everything in, it makes me think back to when I was
younger and how I enjoyed taking in information.” Although the program leaders expressed
excitement about their motivation to teach and to see the progress of their students, there did not
seem to be any mention of them having motivation to engage or learn more about subjects
related to STEM education on their own during the focus group interviews.
Interviews with Program Supervisors
19
Each of the program supervisors participated in an interview with the researchers that
lasted for thirty to forty minutes. One portion of the interview focused on the program
supervisors’ perceptions and assessments of their program leaders’ knowledge and motivation,
both overall and specifically regarding STEM content. Some general expectations that were
mentioned of the program leaders included characteristics such as punctuality, energy,
passionate, responsibility, flexibility, creativity, and communication. Most of the program
19
Linda Moon wrote this section.
A NEEDS ASSESMENT OF MEADOWS 52
supervisors emphasized that their program leaders must be a “kid-magnet,” in addition to being
willing to make mistakes and learn from them.
Knowledge. When asked to assess program leaders’ knowledge when it comes to
teaching STEM and project-based learning, program supervisors stated that the level of
knowledge undoubtedly varied from individual to individual. Program supervisors were
confident that the program leaders had sufficient knowledge to lead the clubs to which they were
assigned, but not many actually have any more background knowledge than what is provided in
the STEM curriculum used in RNH. In addition to content knowledge, one factor that was
discussed is that program leaders may not have a lot of pedagogical knowledge, since more than
half of the program leaders have little to no experience with teaching, let alone teaching STEM
material (see Table 3). All of the program supervisors stated that their staff could definitely
benefit from additional professional development, and more focused training for STEM.
Motivation. When asked if they thought that program leaders were generally motivated
when they taught STEM, program supervisors again said that level of motivation varied among
individuals. Some program leaders are naturally more motivated and energetic in everything that
they do, but others’ motivation could easily be affected by several factors. One critical factor is
the program leaders’ personal interest in a particular topic that day or week. If it was a topic that
is inherently interesting to them, then program leaders were much more motivated to teach that
topic, and also do additional research, such as finding online videos, to present to their
students. Another factor, as also reported by program leaders, was the motivation level of the
students themselves. They feed off the excitement and engagement of the students, especially if
the students are retaining information or actively participating. As one program supervisor
observed,
A NEEDS ASSESMENT OF MEADOWS 53
The best motivation is self-motivation. Most staff members are motivated when they see
how their work is impacting their kids. One program leader reported that his instruction
regarding healthy behaviors seemed to be working because one of his students was
starting to noticeably slim down. He was very excited that he seemed to be making a
difference in that student’s life.
A third factor, which program supervisors expressed helps program leaders’ motivation is
feedback and affirmation. One program supervisor conveyed that she found it effective when
she provided feedback to her program leaders, and stressed the fact that she was there to support
them in whatever they needed. Along the same vein, another program supervisor attested to
working hard in order to collect program leaders’ buy-in to their collective visions and goals, that
by taking such ownership, they will be more motivated to work hard. Yet another program
supervisor said she usually gives little gifts for her program leaders, not purely as incentives but
to show her appreciation for their hard work. By maintaining open lines of communication and
providing constructive feedback and support, program leaders are more motivated in their jobs.
Observations
20
An observation of the program leaders, at all six of the RNH program sites, provided
valuable information in regards to their STEM capacity in the classroom, and teaching
styles. The goal of the observations was to observe the program leaders in a classroom setting
and collect data with the aid of an observation protocol. However, upon completion of the
observations, the relationship between what was observed and what was stated during the focus
group interviews and Qualtrics survey will be further discussed in Chapter 5.
The physical makeup of each classroom in the MESD was very similar. The rooms all
had adequate natural and artificial lighting, which allowed for unobstructed views of the teaching
20
Nsoah Abu-Rasool wrote this section.
A NEEDS ASSESMENT OF MEADOWS 54
and reading needs of the students. The walls were covered with educational content that
matched the elementary school setting and various grade levels, such as the whole number scale,
world maps, and the alphabet. The participants in the classroom consisted of the program leader
and a student population that ranged from 20-30 in class size. The grade level of the students
ranged from kindergarten to 5
th
grade, and the instructors seemed well-equipped at making the
adjustment to the different grade levels by their approach and engagement techniques during
instruction.
A consensus of what learning looked like, from the teaching styles observed, consisted of
four primary components. The first component consisted of a STEM-focused lecture taught by
the program leader. Moreover, the program leaders usually positioned themselves in the front of
the class and the students sat at either their desk or in a semi-circle on the floor. The program
leaders maintained eye contact with the students during the lectures to see if they were engaged
in the lessons or had questions. Second, the program leaders provided a demonstration, either
through an actual display of the upcoming activity or through a video on the classroom television
or on a tablet (i.e., iPad). The third component was an activity that consisted of some sort of
participation or practical application of the lesson by the students. The observed student activity
usually had components of the lesson that had been explained during the lecture and
demonstration component of the learning session. The fourth and final component was centered
on a reflection of what had just occurred from the lecture, demonstration, and the student
activity. The program leaders would give feedback and ask the students questions pertaining to
the, “how, what, and why,” of the learning experience to see what they had retained. The
reflective questions, asked by the program leaders, seemed to trigger within the students an
ability to recall what they had learned from the previous three components of the program
A NEEDS ASSESMENT OF MEADOWS 55
leaders teaching style. In addition, the program leaders were viewed providing guidance,
facilitation, and student engagement techniques, which seemed to foster an environment that
allowed the students to be more involved in the learning process.
The program leader’s feedback, in particular, seemed to be very helpful to the
students. During the observations, the facial expression of the students was a big giveaway of
how they seemed to feel about the feedback from the program leaders. For example, when a
smile appeared the student was usually receiving positive feedback for working correctly on an
activity and when the, “ah-ha I understand,” expression was visible they were usually receiving
corrective feedback for guidance and gaining a deeper understanding of the
subject. Additionally, the interaction between the program leaders and the students showed that
the program leaders had a grasp of the content outlined in the curriculum. However, they did not
seem to provide more in-depth answers or feedback beyond what was explicitly scripted in the
curriculum.
Within the four components of teaching displayed by the program leaders, the sequence
of events usually followed the same pattern at all sites observed. For example, during an
observation, two components of STEM were noticed that featured science and engineering. The
observation protocol contained a section with the sequence of events separated into three frames
of reference, the beginning, middle, and the end. The initial sequence of events, the beginning,
program leader gave a lecture on the science component, severe weather and the damages it can
cause to man-made structures. In addition, the program leader explained to the students that they
would be participating in an activity that would allow them to build a house, the engineering
component, with materials provided to withstand severe weather. The second sequence of
events, the middle, consisted of a demonstration followed by an activity of the students building
A NEEDS ASSESMENT OF MEADOWS 56
an actual severe weather resistant home. As the students worked together in groups the program
leaders walked around the classroom to monitor their progress and facilitate questions or concur
with students who were achieving success within the guidelines of the project. Finally, in the last
sequence of events, the end, the program leader asked reflective questions in an effort to gain a
deeper understanding of the students work and to see what the student had learned during the
period of instruction. Some of the paraphrased questions asked by the program leader were,
“why did you chose to build in a given location, why did you chose a certain material, and what
severe weather were you trying to defend against in your design?” Overall, the students seemed
happy, engaged in their activities, interacted respectfully to one another, and appeared to have a
close relationship with the program leaders.
Question 2:
Is Each Site Providing Sufficient Opportunities for Professional Development?
Survey Responses
21
Both program leaders and supervisors were asked questions regarding both the
availability and access to professional development and their views on its importance (see Table
5). The responses were based on a 6-point Likert scale (1=strongly disagree, 6=strongly
agree). Program leaders reported a mean of 4.78 (SD=1.1) for professional development
availability, as in trainings and resources they have been and are currently supplied with, and a
mean of 5.42 (SD=.74) for professional development importance. For the same questions,
program supervisors reported somewhat lower but not statistically significant means for both
sections: for availability, the mean was 4.58 (SD=.75) and for importance, the mean was 4.13
(SD=.12). Two of the original questions, “I have been enriched by the training events I have
attended,” and “Without the training I receive in professional development, I would not be able
21
Linda Moon wrote this section.
A NEEDS ASSESMENT OF MEADOWS 57
to teach STEM effectively,” were discarded because there were errors in the construction of the
response choices in Qualtrics.
Figure 2. Mean professional development responses. This figure depicts the responses from
program leaders (N=44 for availability, and N=43 for importance) and program supervisors
(N=6) regarding the availability and importance of professional development at RNH. The
means and standard deviation for each categorization are shown.
22
In addition to the Likert scale items, both program leaders and program supervisors were
asked two open-ended questions: 1) In the past year, how many of the following STEM-related
support services have your received—training and coaching, and 2) Please rank the following to
what you think are the most pressing needs related to implementing STEM activities in your
program—training, information and resources, coaching, site visits, and consult with outside
experts. Program leaders reported a mean of 1.3 (SD=1.11) training events and 1.24 (SD=1.3)
coaching events, whereas program supervisors reported means of 1.83 (SD=1.72) and 2.33
22
Linda Moon created this figure.
A NEEDS ASSESMENT OF MEADOWS 58
(SD=2.5), respectively. Only 12 program leaders answered the question regarding pressing
needs, and all of them listed training, information, and resources, as the most pressing need.
Four out of six program supervisors supported this as the most pressing need, while the
remaining two indicated that coaching and consult with outside experts were the most pressing
needs.
Focus Groups with Program Leaders
23
The objective for this section of the focus group interview was to find out the frequency
of professional development trainings offered and attended by the program leaders, and if they
were perceived to be addressing their needs. According to the program leaders, professional
development is offered in the RNH program to the program leaders and it is often taught by the
program director and program supervisors. Moreover, the program leaders indicated that there
are usually three major professional development sessions for the RNH program within the
school year. The first professional development is held in January, which is called Local
Learning Community (LLC); it consists of a two-day forum of rotating subjects, including
STEM, which last about an hour per subject. It is held off-site with trainers from various
education centers in the region and includes all of the personnel in five neighboring districts’
afterschool programs. The second professional development is held in the summer at the end of
the school year in June, and the third is held prior to the beginning of the next school year in
August. The second and third professional development sessions are held at on-site locations
within the MESD.
The specific topics covered in the professional development sessions ranged from
organization and management of the classroom to behavioral management of students who may
have challenges in those areas. One of the program leaders stated,
23
Nsoah Abu-Rasool wrote this section.
A NEEDS ASSESMENT OF MEADOWS 59
We received professional development training on the STEM curriculum from the
NASA’s Best vendor, which consisted of training on how to use the materials provided in
the curriculum box set, as well as, how it should be maintained.
In most cases the program leaders felt that the professional development training was
helpful but thought that it should include more emphasis on STEM-related subjects and
organizational goals. In fact, one of the program leaders stated,
I would like to see someone show us exactly how to do a PBL for consistency, because
sometimes when we turn them in, some things are not right, or we need to make
corrections. Once, at a training session, they gave us a PBL already filled out but they
did not show us how to do it so it was unclear to us how to fill it out correctly.
Interviews with Program Supervisors
24
As stated previously, program supervisors agreed that additional training would prove
beneficial for their staff. The main modes of training for all of the Meadows RNH staff are
through the mass training events held once before the school year begins and once during winter
break. These sessions are typically led by the program director and all the program
supervisors. Program leaders also participate in a professional development workshop held by
the Local Learning Community (LLC) that is comprised of afterschool programs throughout
Meadows, Santos, Hi-Tech, Sunnyside, and Happy Valley. In addition to training in STEM, the
training sessions cover various topics such as behavior management, curriculum training, and
project-based learning. A few program supervisors stated that they would like to see STEM
workshops that were more experiential in nature, as opposed to a more didactic model, and also
workshops that focused less on how to make STEM better but even more fundamentally, what
STEM actually is and why it is important.
24
Linda Moon wrote this section.
A NEEDS ASSESMENT OF MEADOWS 60
Starting last year, the training sessions were organized by tiers, where the program
supervisors categorized each of the program leaders into one of four tiers—beginning,
intermediate, advanced, and applying—based on years of experience and general performance
evaluations. Program supervisors found that this method was beneficial because for a while the
program leaders had been saying that there was a lot of repetition at training sessions. By
organizing them into tiers, they were able to sort information based on its suitability for veteran
or newer staff members.
In addition to these large-scale events, each of the program supervisors initiates their own
training and coaching sessions at their site. As of late, however, program supervisors express
that they find it difficult to squeeze in time for much site-specific professional development.
One of the main reasons is that the RNH staff is now unionized, so there are specific standards
regarding breaks and overtime schedules that need to be complied. All of the program
supervisors asserted that they try to hold meetings at least once a week, but a lot of the time, the
meeting time gets devoted to lesson planning and preparations. Due to this phenomenon, a few
of the program supervisors suggested that the program leaders need continual training, not just at
the district-wide RNH sessions, but perhaps through visiting another site apart from their
own. One supervisor suggested,
We need a system of coaching and staff development at the site level. If program
supervisors would form subcommittees on a particular subject…master it, and offer
training to program leaders throughout RNH, that would be ideal.
Additionally, program supervisors feel that program leaders would most likely want to
receive training from experts outside of their own site supervisors. The reason for this, as
inferred from the interviews, is that they sense program leaders are viewing the training sessions
A NEEDS ASSESMENT OF MEADOWS 61
by the same people to be redundant, thus devaluing professional development altogether.
Moreover, program supervisors indicate that they need the time to recharge and be trained also.
Because they conduct their own versions of coaching at their respective sites, they feel much of
the large-scale professional development sessions should be relegated to outside experts.
Observations
25
We were able to attend one of the large-scale Local Learning Community (LLC) training
sessions held on January 31, 2015 at Hi-Tech Middle School. This professional development
event, as mentioned previously, is collaboration between five school districts: Meadows, Hi-
Tech, Santos, Sunnyside, and Happy Valley. They offered two back-to-back workshops
regarding STEM instruction, which was attended by three program leaders in the first session
and eight program leaders in the second, with no attendees from Meadows at either
session. Both workshops were led by the same speaker, a retired K-12 educator who specialized
in afterschool programs.
The focus of both workshops was about how to engage students into the process of
scientific thinking. The first workshop focused on inductive versus deductive reasoning while
the second workshop focused on using elements of Dimensions of Success (DoS) and developing
learning goals. Both sessions started out with an icebreaker where participants had to try to
remove all but one card from a deck of cards balanced on top of a paper cup. The purpose of this
exercise was to integrate the scientific process into an activity seemingly unrelated to
science. During this exercise, participant used skills such as formulating a question, predicting,
planning, executing, reflecting and repeating, in order to achieve the task at hand. Afterwards,
program leaders in the first session practiced inductive and deductive reasons by categorizing
fruits and vegetables. The use of both pictures on cards and handling actual fruits and vegetables
25
Linda Moon wrote this section.
A NEEDS ASSESMENT OF MEADOWS 62
that were brought in by the speaker made the learning experience more engaging. The second
session was centered around the question, “What degree of slope for a ramp allows a car to go
the furthest before stopping,” so participants took the materials provided (e.g. tracks, racecars,
protractors, measuring tape, baking sheets) to experiment with this question.
Throughout both sessions, the speaker was focused more on the process and the
conversations surrounding the process rather than whether or not the task was achieved. The
participants were asked to articulate what they did, what they learned, and how could they use
what they had learned in the future. Additionally, they were urged to encourage similar
discussions with their own students in each of their contexts.
Question 3:
Is the Current Curriculum Adequate for Promoting STEM Learning?
Survey Responses
26
Both program leaders and program supervisors were asked to evaluate the curriculum
they utilize to teach STEM in each of their RNH sites. The survey asked respondents to assess
how they believed the students were responding to the STEM curriculum, how STEM
components were being integrated into their programs, and whether or not their STEM
programming was compliant with general expectations of project-based learning. While the
surveys did not specifically asked to rate each of the curricula, from our interactions with the
program leaders and supervisors, the main STEM curricula in use at RNH are MindWorks,
KidzScience and NASA’s Best. As depicted in Figure 3, program leaders rated curriculum
adequacy at a mean of 5.01 (SD=.71), and program supervisors at a mean of 4.62 (SD=.53).
26
Linda Moon wrote this section.
A NEEDS ASSESMENT OF MEADOWS 63
Figure 3. Mean curriculum adequacy responses. This figure depicts the responses from program
leaders (N=42) and program supervisors (N=6) regarding adequacy of the current STEM
curriculum to promote STEM learning among students. The means and standard deviation for
each categorization are shown.
27
Focus Groups with Program Leaders
28
The aim for this section of the focus group was to identify the program leaders’
perception of the STEM curriculum used in the RNH program and to gather their opinions to
what extent they felt the curriculum was effective. The STEM curriculum used by the RNH
program consists of several vendors ranging from, MindWorks, KidzScience, Trash for
Teachers, and NASA’s Best. The program leaders revealed that the curriculum comes in box
sets and are stored in the program supervisor’s office or in a storage area on the school
site. Each curriculum vendor has a particular set of materials, components, and lesson plans used
for curriculum delivery. Some are equipped with materials and laminated instructions that can
be reused multiple times; however, there are also some that have materials designed for a one
27
Linda Moon created this figure.
28
Nsoah Abu-Rasool wrote this section.
A NEEDS ASSESMENT OF MEADOWS 64
time use, which leaves shortages for other program leaders who share the curriculum box
sets. Instructions on how to use the curriculum are included from each vendor in all of the box
sets.
The RNH program also incorporates project-based learning (PBL) as part of their
curricular strategy. When asked about PBL, program leaders revealed that they were unsure
about what it is and how it should be used. One stated that, “I once heard of PBL but needed
more clarification on what it actually meant,” and another stated that, “I had seen a presentation
about PBL during a professional development session but I was still unclear about how it should
be used and on what occasion.” A contributing factor to this confusion may be that PBL is used
throughout RNH in a specific context, as it is the name of the lesson plan worksheet they fill out
for their clubs. Upon hearing PBL, they are focused more on the logistics of filling out the form,
and not so much on the broader construct of project-based learning. It appears that PBL has been
integrated into the programming but it has become more of colloquial way to refer to their lesson
plans, and evidence of a deeper understanding of the construct of PBL was lacking.
Interviews with Program Supervisors
29
While each of the RNH sites use MindWorks, KidzScience, and NASA’s Best as their
STEM curriculum, the interview was mostly focused on each site’s experience with MindWorks,
as it was the most widely and frequently used curriculum. The program supervisors, for the most
part, are content with MindWorks, mainly due to its convenience. Since everything—supplies
and scripted lesson plan—comes in a box, the program leaders have little to prepare, which is
expedient since they do not have sufficient time to prepare as it is. As one program supervisor
described,
29
Linda Moon wrote this section.
A NEEDS ASSESMENT OF MEADOWS 65
Because of the lack of time, [MindWorks] is convenient. It comes in a box with all the
supplies, all the copies are ready, there’s a script for newbies but there is room for
creative [program leaders] to make adjustments. Prior to this, we had to make up our
own STEM curriculum and the outcome was hit or miss… and MindWorks has already
been tested.
Regardless, all of the program supervisors state that they encourage their program leaders
to “think outside the box,” by searching for additional resources to supplement the boxed
curriculum, and to alter the suggested activities based on classroom size and amount of time they
have.
Despite its convenience, program supervisors do pinpoint some of the challenges in
utilizing the MindWorks curriculum. Each MindWorks box represents a 10-week unit, which
gets rotated throughout each site, among seven to eight different clubs. Because of this rotation,
and even though the box is theoretically designed for 125 students, a lot of the supplies run out
along the way, which was a chief complaint of the program leaders as well. In addition, while
the scripted lesson plans are helpful, they can also cause some program leaders to not prepare as
much, and end up reading the material for the first time in the classroom.
Regardless of the curriculum’s strengths and weaknesses, program supervisors all state
the importance of capturing student engagement by whatever means. A few of them used the
phrase, “disguised learning” as a vital component in the classroom; they believe in kids learning
subconsciously through something fun that seems like play. As one program supervisor stated,
“We are still moving forward and still learning and trying to make sure we are doing what we
can to be knowledgeable and have enough resources for the students.”
A NEEDS ASSESMENT OF MEADOWS 66
Document Review
30
The main STEM curriculum is called MindWorks Resources. According to the
MindWorks website, the curriculum is designed in compliance with common core standards, pre-
and post- tests, project-based learning components, and hands-on learning (“MindWorks
resources,” 2013). There is one overarching theme per school year, which is divided into seven
different units. For the current school year, the theme is “Mission Possible” and the seven
different units are Myth Buster Kids, Whodunit, A Week in the Life, Sticky Situations, Storm
Chasers, Entrepreneurs and Inventors, and Future Builders. Every week in each of these units,
students explore a different subject matter, for a total of six subjects, or clubs, in six weeks: Math
Matters (math), Around the World (social studies), Art in Action (art), Exploration (science),
Bring on the Books (reading), and Spreading the Word (language). Each of these clubs has their
own box, complete with a step-by-step lesson guide and most of the materials needed for all the
activities.
Each lesson is formatted in the following manner: overview of the lesson, lesson
objectives, materials list, pre-test, pre-activity discussion, activity, extension activities, post-
activity discussion, and post-test, which contain the same questions given in the pre-test. The
lesson guide is scripted so that the instructors not only have instructions to follow but it includes
the dialogue they can read aloud to the students. The lessons are also differentiated into two age
groups: kindergarten to second grade, and third to fifth grade.
Depending on the RNH site, MindWorks is taught once or twice a week, lasting
anywhere between 45 to 90 minutes. While each unit is supposed to last for six weeks, RNH
typically uses each box for 10 weeks before it is rotated to a different group. Because of this,
some of the lessons are stretched out over two weeks to accommodate the schedule. A further
30
Linda Moon wrote this section.
A NEEDS ASSESMENT OF MEADOWS 67
discussion of the MindWorks curriculum in light of interviews with program leaders and
program supervisors is presented in Chapter 5.
Summary
With regards to the three needs assessment concerns for the RNH program—instructor
capacity, professional development, and curriculum adequacy, both program leaders and
program supervisors see room for improvement. The program leaders believe that they lack a
strong foundation in STEM knowledge, but are confident in their ability to teach and their
motivation to teach. Additionally, they indicated that the enlightened look displayed on their
student’s faces, when they learn something new, is a motivational event that they look forward to
on a daily basis when reporting for work. Program supervisors feel that the program leaders
have sufficient knowledge for the task at hand, but further training would be useful. They also
feel that program leaders’ motivation is largely contingent on their personal interest in a
particular topic and the level of the students’ engagement.
In the area of professional development, the program leaders felt that overall, they are
receiving training that will help them succeed as STEM instructor; however, they would like to
learn more about STEM and other teaching methods that is in alignment with the RNH mission
and vision, such as PBL and 21
st
Century learning strategies. Program supervisors would like to
see continual, hands-on training for staff and if possible, conducted by outside experts.
In terms of curriculum adequacy, both program leaders and program supervisors attest to
the convenience of MindWorks, as everything comes in a box. However, program leaders are
encouraged to enhance the learning experience by “thinking outside the box,” and supplementing
the scripted lesson plans with additional resources.
A NEEDS ASSESMENT OF MEADOWS 68
CHAPTER FIVE: DISCUSSION
Authors: Linda Moon and Nsoah Abu-Rasool
Engaging students in STEM during out-of-school time (OST) programs is a difficult but
necessary prerogative, as STEM proficiency becomes increasingly important for students and for
the economy at large. Even so, the reality is that OST programs cannot hope to provide
extensive academic direction in STEM as there are definite limitations in instructor capacity,
professional development, and adequate curricula. However, OST programs, such as
afterschool, can provide outlets for students to develop self-efficacy and interest, which in turn
may assist students’ overall performance in STEM (Dierkling, 2007).
In the RNH program, there is a concerted effort to promote STEM learning in their K-5
students. The purpose of this needs assessment was to use multiple data collection methods and
two interest groups to understand the current state of RNH instruction, professional development,
and curriculum in an effort to determine what areas require further attention. The three central
questions guiding this assessment were:
1. Are instructors knowledgeable and motivated in teaching STEM?
2. To what extent are instructors provided opportunities for professional development?
3. Is the current curriculum sufficient for promoting STEM learning?
The previous chapter presented the results of the data analysis. In this chapter, these
results are discussed and implications and recommendations based on these findings are
presented. The discussion is organized as follows: 1) interpretation of the findings, 2) limitations
of the project, and 3) recommendations for RNH and possibly future research.
31
31
Linda Moon wrote this section. Nsoah Abu-Rasool made contributions.
A NEEDS ASSESMENT OF MEADOWS 69
Discussion of Findings
Instructor Capacity
32
From our research, the RNH program exhibits strengths in the following areas when it
comes to instructor capacity. For one, the program leaders indicated, during the focus group
sessions and in their survey responses, that they were highly motivated and enthusiastic about
teaching STEM subjects to their students. They enjoyed being a part of the RNH program and
valued their contributions to STEM enrichment and learning as a whole. Although the program
leaders revealed that their background knowledge of STEM-related subjects was limited, they
indicated they had sufficient knowledge to teach the material at hand. In addition, the program
leaders’ passion to engage students in STEM learning was a positive theme that was noticed
throughout the focus group interviews and the classroom observations.
Beyond instruction in the classroom, the program leaders have built relationships with
their students, which is a vital component of out-of-school time instruction (Miller, 2010).
Program leaders seemed to have a genuine concern for their students. As one program leader
stated, “I really enjoy the kids, to watch them learn something that they have never seen before
and to see the smile on their face, that’s my motivation, that’s what keeps me going.” This was a
sentiment that was echoed with all the RNH staff across all the sites we researched. Overall, the
program leaders seemed to be willing to learn and they believe that more STEM-related training
will help them become even more effective instructors within the RNH program.
Knowledge of program leaders.
33
According to the survey results, program leaders’
self-report of their knowledge (M=4.75) is noticeably higher than program supervisors’
perceptions of their knowledge (M=3.82). This could be because the program leaders know
32
Nsoah Abu-Rasool wrote this section.
33
Linda Moon wrote this section.
A NEEDS ASSESMENT OF MEADOWS 70
more than the supervisors are aware of or they have a higher sense of self-efficacy or confidence
in their teaching knowledge. The latter notion is supported by the data, as the program leaders’
self-efficacy score was high (M=5.01). Another possible explanation for this discrepancy is that
because pedagogical knowledge and content knowledge were combined in the knowledge score,
the score could reflect more on what’s visible in the program leaders’ teaching, rather than
assessment of content knowledge. This is supported by program supervisors’ comments
regarding program leaders not having sufficient teacher training in addition to observations made
by the researchers wherein some of the program leaders tended to read off of the scripted lesson
guide and appeared to have not prepared enough in advance.
Even though program leaders reported a higher knowledge score, the score was not a
strongly positive one, which was reflected in the focus groups and observations. With respect to
knowledge of specific STEM-related terminology and concepts, there was some confusion
among program leaders. While all of them knew what the acronym stood for, their
understanding of each of the components was fixated on the more traditional sense of science
instruction, for example, rather than the principles of science that can be applied across
disciplines. The same was true when asked about how their STEM instruction related to 21
st
century learning skills. Beers (2012) mentions that, 21
st
century skills encompass various
competencies ranging from creativity and innovation to critical thinking and collaboration, but
program leaders only focused on the technology as 21
st
century learning skills. Furthermore,
when discussing project-based learning (PBL), program leaders responded mostly about their
lesson plans, also entitled PBL, rather than the underlying principles.
A NEEDS ASSESMENT OF MEADOWS 71
Motivation of program leaders.
34
Similar to the knowledge survey results, there was
also a divergence in perceptions of motivation, as program leaders reported an average level of
motivation as 5.03, whereas program supervisors perceived their motivation to be 4.24. This is
surprising because, as discussed in Chapter 2, afterschool programs and the like are inherently
low stakes environments that generally yield lower levels of motivation (Bevan et al., 2010;
Freeman et al., 2009). However, according to the data, program leaders are highly motivated in
teaching STEM. This is further supported because all of the sub-categories of motivation that we
surveyed had relatively high scores: self-efficacy (M=5.01), teaching beliefs (M=4.93), value
(M=5.24), and interest (M=5.05). A possible explanation for why program supervisors’
perceptions of program leaders’ motivation was lower could be that they have not directly asked
their program leaders about their confidence in teaching STEM, their teaching beliefs, values, or
interest. They answered the survey questions purely on their observations, which is not entirely
reliable because a lot of these motivational factors are not outwardly apparent.
Moreover, the overall mean for outcome expectancies was 4.36, which shows that
program supervisors only somewhat agree that program leaders directly influence students’
learning outcomes pertaining to STEM. This reflects program supervisors’ composite
perceptions regarding their program leaders’ knowledge and motivation. From their
understanding of their program leaders’ knowledge and motivation, it would seem program
supervisors believe that other factors in addition to program leaders’ influence have profound
impacts on students’ STEM learning.
34
Linda Moon wrote this section.
A NEEDS ASSESMENT OF MEADOWS 72
Professional Development
35
There are also definite strengths in professional development within RNH. Professional
development is a very useful tool for organizations looking to keep their personnel sharp and up
to date on the latest policies and procedures in a given industry. It is important to note that the
leadership within the RNH program understands the importance of professional development and
how it can be used to help train their staff become more efficient in all aspects of educational
growth including STEM-related subjects. Based on our surveys, interviews, and observation, the
RNH program has three types of professional development events that occur at different times
during the school year: 1) the Local Learning Community (LLC), a large-scale training event
involving five neighboring districts, 2) district-wide training event dedicated to all the RNH
sites, and 3) training and coaching occurring at the individual site level. The program leaders
conveyed their enjoyment in the professional development trainings and expressed their value for
such trainings to help them become effective instructors. They were generally pleased that the
RNH program is committed to making professional development a major part of their teaching
experience. Additionally, program leaders and program supervisors alike have responded
positively to the practice of offering “tiered” workshops, where program leaders are organized
into four different tiers (i.e., beginning, intermediate, advanced, applying) based on their
experience. Innovations such as these show promise for continued growth in the area of
professional development at RNH.
Based on our data, more occurrences of professional development may be a start for
improving this area. Program leaders and program supervisors show similar views about the
availability of professional development, which encompasses resources, training, and
coaching. In terms of whether or not they think professional development is readily available,
35
Nsoah Abu-Rasool wrote the first part and Linda Moon wrote the second part of this section.
A NEEDS ASSESMENT OF MEADOWS 73
program leaders and program supervisors reported average scores of 4.78 and 4.58,
respectively. Although on the higher end of the spectrum, these scores do not reflect very high
positive responses. This is aligned with responses garnered during focus groups and interviews,
where both program leaders and program supervisors expressed that there needs to be more
training and resources. Specifically, more focused training on STEM would be beneficial. The
researchers were only able to observe one STEM training session, but based on that experience,
we would agree that STEM training did not involve a heavy emphasis on the importance of
STEM and did not have a lot of applicable elements to take back to the classroom. A possible
reason for this is that the workshop instructor neither knew nor gauged the amount of prior
knowledge and experience that the attendees had regarding STEM, so the instructor tended to
assume program leaders already knew a lot of the information.
Perhaps due to fewer training events than desired, program staff, and program leaders in
particular, believe them to be highly important. The overall mean score for the importance of
professional development was remarkably higher for program leaders (M=5.42) than it was for
program supervisors (M=4.13). One possible explanation for this phenomenon is that program
supervisors find that a lot of the information disseminated at training events is not readily
implemented into the classroom, thereby construing the idea that program leaders do not deem
professional development to be important. Another reason for this could be that program
supervisors are more knowledgeable about the resources and training that are needed, as they
were all program leaders at one point. Even though the survey questions specifically asked them
about current professional development efforts, some of their answers may reflect what they
once thought about professional development when they were program leaders. For example, in
question 21 of the survey questionnaire, “Professional development workshops are vital to
A NEEDS ASSESMENT OF MEADOWS 74
helping program leaders develop new teaching techniques,” they might have answered from their
own perspectives on how they were not dependent on such things, possibly. These are just
conjecture, but a variety of different factors are at play that could account for these discrepancies
in scores.
A surprising result, however, is that even though program leaders indicated through the
survey and during focus groups that they regarded professional development to be highly
important and necessary, there were no representatives from Meadows present at either STEM
workshop at the LLC training observed by the USC researchers in January 2015. This may go to
show that the results regarding professional development may not have been specific to or even
related to STEM at all, but rather they found training in other areas (e.g. behavior management)
to be helpful. After all, literature shows that effective professional development is concerned
also with building long-term career and life skills (Freeman et al., 2009; Sheldon et al.,
2010). Regardless, given that they expressed the need for additional STEM training, the
program leaders might have sought out these workshops to supplement their STEM instructional
knowledge.
Curriculum Adequacy
36
Project-based learning is an essential tenet of STEM and of the RNH program, which is
reflected in their main STEM curriculum, MindWorks. The alignment of the MindWorks
curriculum with the RNH program teaching goals helps maintain academic continuity within the
program. In addition, the MindWorks curriculum is designed for elementary and middle school
students enrolled in afterschool educational programs, including programs with a STEM
focus. RNH staff commented on the benefits of having the user-friendly MindWorks curriculum
to teach STEM, during the focus groups and interviews, when they explained how the step-by-
36
Nsoah Abu-Rasool wrote this section. Linda Moon made contributions.
A NEEDS ASSESMENT OF MEADOWS 75
step instructions, pre- and post-test, and hands-on activities are all included as part of the
curriculum in each box set. The program leaders also indicated that the step-by-step instructions,
albeit a bit tedious, were very helpful for teaching STEM especially when they had limited
background knowledge or preparation time.
Regarding whether or not the STEM curriculum is sufficient for promoting STEM
learning among students, program leaders were slightly more affirmative (M=5.01) than were
program supervisors (M=4.62). This was an unexpected result because many of the program
leaders found MindWorks to be tedious in our focus group discussions. However, because the
survey only referred to the STEM curriculum more generically and did not specifically ask about
MindWorks, these responses could reflect their overall opinions about all the STEM-related
curricula used throughout RNH combined or about a single curriculum that was not MindWorks
(e.g. KidzScience, NASA’s Best).
Even though most of the program leaders mentioned that they found the MindWorks
curriculum to be tedious because of its step-by-step format, it appears that this type of scaffold
curriculum is necessary in this context. All of the program supervisors stated that there was not
enough time for program leaders to prepare their lessons each day. One of the main reasons for
the lack of preparation time is that the RNH staff are now unionized, which make it challenging
for program supervisors to ask program leaders to invest any additional time for which they will
not be compensated. In such a context, MindWorks seems to be a convenient curriculum to use
in that it requires minimum preparation. However, the program supervisors, based on their
response, do not seem overly enthusiastic about MindWorks’ prospects for promoting STEM
learning in their afterschool students.
A NEEDS ASSESMENT OF MEADOWS 76
Recommendations
37
Despite the limitations of this project that will be discussed later, this needs assessment
still identified major areas in which the MESD RNH program could enhance their STEM
activities and instruction at each of their sites. As evidenced in our interactions with the RNH
staff, program leaders and program supervisors are tasked with a myriad of roles and
responsibilities outside of STEM instruction. But given the rising importance of STEM at both
the individual and global level, we feel attention to STEM programming should be a top
priority. Hence, the following recommendations should be taken into account by the RNH staff,
for the purposes of preparing their students to develop proficiency and motivation in STEM-
related subjects, careers, and thinking processes.
Instructor Capacity
Knowledge. To enhance program leaders’ knowledge, it seems appropriate that program
supervisors take inventory of their program leaders’ background knowledge and also
pedagogical knowledge. While program leaders and program supervisors contend that program
leaders have sufficient knowledge for the task of teaching the MindWorks curriculum, it appears
that they may be lacking in sufficient metacognitive knowledge that would allow them to go
beyond scripted lesson plans. Developing metacognitive knowledge in program leaders would
be heuristic in nature, enabling program leaders to become self-aware of their knowledge and to
apply appropriate strategies to enhance their knowledge and consequent instruction (Anderson &
Krathwohl, 2001). Instructors with sufficient metacognitive knowledge about applying STEM
content will be able to assess the task at hand, evaluate their own knowledge and skills, plan an
appropriate teaching approach, apply and monitor various teaching strategies, and reflect and
adjust if necessary (Ambrose, Bridges, DiPietro, Lovett, & Norman, 2010). Based on these
37
Linda Moon wrote this section. Nsoah Abu-Rasool made contributions.
A NEEDS ASSESMENT OF MEADOWS 77
criteria, we suggest that program leaders and program supervisors take the following actions to
achieve higher levels of knowledge in RNH program leaders.
Assess the task at hand. Program supervisors should review each MindWorks unit with
all the program leaders prior to the start of said unit. Even though each program leader will
implement the lessons at different times, having reviewed it collectively will allow the program
leaders to know what to expect, rather than possibly diving into each unit without any
background. During this overview session, program supervisors can entertain questions, allow
collaboration and group brainstorming for ideas, and emphasize the desired outcomes of each
lesson.
Evaluate knowledge and skills. While using the MindWorks curriculum to teach STEM
in the RNH program does not require program leaders to be fluent in STEM content, assessing
knowledge is important for situations where one may possess incorrect prior knowledge that
subsequently impedes student learning. Moreover, assessing prior knowledge would also help
program leaders unlock relevant background information that can enhance their lessons.
A relatively simple way to conduct a STEM knowledge assessment in RNH would be to
distribute the list of key terms supplied by MindWorks that students are expected to know at the
end of a MindWorks lesson. Program leaders can indicate the depth of their knowledge of each
term with statements such as “I have never heard of this term,” “I have heard of this term,” “I
could define this term,” “I could explain it to someone else,” and “I can use it to solve problems”
(Ambrose et al, 2010, p. 29). The rationale for this assessment is that if the instructors
themselves only have cursory understandings of key terms, students cannot be expected to
develop deeper understandings, which is ultimately the goal of effective STEM
instruction. Following this assessment, program leaders should work to address the areas in
A NEEDS ASSESMENT OF MEADOWS 78
which they need more knowledge, and program supervisors should assist program leaders in
identifying and addressing these knowledge gaps accordingly.
Plan an appropriate approach. The lesson plans are already provided in the STEM
curriculum, but program leaders can benefit from planning how to execute the lesson plans. Due
to time constraints, this step could be as simple as asking program leaders to orally explain how
they intend to implement a particular lesson. Being cognizant of all the components of the task
in this manner may yield more intentional and well-prepared instruction.
Apply strategies and monitor progress. At the end of each lesson, program supervisors
should require program leaders to do a brief self-assessment of how successful it was. The
assessment could range from program leader conduct to perceptions of student learning. A wide
range of self-assessment checklists can be found at http://teacher.scholastic.com/professional/
self-assessment/checklist/. In addition, program supervisors could sit in on a lesson periodically,
using the same checklists, and provide substantial and timely feedback to help program leaders
be more effective.
Reflect and adjust. An exercise that may prove beneficial in the reflection process is to
have program leaders think about what they could have done differently by brainstorming
multiple strategies (Ambrose et al., 2010). For example, a lesson might have required students to
form groups of four in order to create buildings out of scrap materials that could withstand the air
from a hair dryer to simulate hurricanes. Program leaders can reflect on various elements that
could have been changed, such as the number of students per group, the use of a hair dryer, the
use of different materials, the time given to complete the task, etc. While this may not help the
lesson that has already passed, it helps program leaders practice evaluating their strategies, and
A NEEDS ASSESMENT OF MEADOWS 79
moreover, these notes can be passed on to another program leader who may have to replicate this
lesson at a later time.
Motivation. Because program leaders’ personal interest and value seem to be important
factors in determining motivation, there should be a concerted effort in increasing these factors
for the program leaders such that it similarly impacts their motivation. We suggest the following
strategies to increase interest and value for STEM instruction.
Increase situational interest. Hidi and Renninger’s (2006) research demonstrates that
individual interest can be triggered by situational interest, especially in a context like RNH
where there is little or no personal interest in STEM. Program supervisors at each site could
develop situational interest for the program leaders by offering freedom with instruction and
allowing them to make meaningful choices. Program leaders are already given free reign with
the material and are encouraged to improvise as needed, but the scope of this freedom is so vast
that it can be overwhelming. Providing meaningful choices will help program leaders develop
interest and engagement with the task at hand (Schraw, Flowerday & Lehman, 2001). Program
supervisors can enact this by infusing decision-making into the culture of RNH where program
leaders are routinely encouraged to make meaningful choices.
One way to promote this kind of organizational culture is for program leaders to choose
which subject they would like to teach. The current format obligates program leaders to rotate
through all of the subjects with the same group of students, but it might be more beneficial for
program leaders to select the subject they would like to teach and have the students rotate
instead. This would allow for each program leader to become somewhat of an expert in the
chosen subject which, in turn, may spur interest.
A NEEDS ASSESMENT OF MEADOWS 80
Add intrinsic value. Increasing intrinsic value for the task of STEM instruction is the
next critical action step, which is closely tied to interest. One way to add intrinsic value is by
catering to the program leaders’ interests, and allowing them to develop lessons with such vested
interests in mind. For example, if a program leader is very skillful in music, that program leader
should be encouraged to draw from experiences and anecdotes related to music when teaching
the lesson. Program supervisors could scaffold this process by creating a simple worksheet for
each lesson plan that asks, “what are your areas of expertise or interest” and “what are some
ways you could relate this week’s lesson to those areas.”
Add utility value. Program leaders may also need increased utility value for their work
with RNH. Perhaps they feel they do not need higher order (i.e., metacognitive) knowledge for
the purposes of their jobs, but guiding them to see how instrumental their current experiences can
be for the future could have positive effects on their motivation. We suggest that RNH
administrators facilitate seminars with former program leaders that have now gone on to
different career paths. A panel consisting of such individuals can share with current program
leaders about the kinds of skills and knowledge that they have gleaned from RNH that have been
influential in their subsequent careers. While attendance cannot be compulsory, program
supervisors can offer incentives (e.g., entering attendees into a raffle for a prize, certificates, etc.)
to increase participation.
Other considerations. For future employment, it may be prudent to hire program
leaders with majors or career aspirations in the STEM field to spearhead STEM teaching. This
does not mean that RNH should only hire program leaders who have STEM interests or
affiliations, but to perhaps delegate STEM instruction to such a person. While the current STEM
curriculum does not require one to be an expert in any of the STEM subjects, having an
A NEEDS ASSESMENT OF MEADOWS 81
instructor who has an inherent interest and relation to the field may generate similar attitudes
among the students. While this reasoning may not be applicable in traditional classrooms
settings, in an OST setting such as RNH, the program leaders serve as positive adult role models
with whom the students can build mentoring relationships that they may otherwise not have
opportunities for at school or home (Lauer et al., 2006; Mahoney et al., 2007; Gottfredson et al.,
2010). One way to gauge a program leader’s interest in the STEM field would be to utilize the
STEM Semantics Survey that measures interest in STEM and in STEM careers, which is
available at http://stelar.edc.org/ instruments/stem-semantics-survey.
Professional Development
38
Professional development training is supported by the RNH program and training session
are designated throughout the school year. The program leaders believe they were proffered
opportunities to receive professional development, but not overwhelmingly so. But even if they
had been exposed to enough training events, the challenge does not lie simply in
attendance. Effective professional development should allow for ample and ongoing practice, in
order for transfer of any skill learned in a workshop or other training event (Darling-Hammond,
Wei, Andree, Richardsons, & Orphanos, 2009). In coalescing literature such as these with our
findings, we propose the following solutions to enhance professional development at RNH.
Gather input about perceived training needs. In order to ensure the effectiveness of
professional development training, program supervisors should request input from program
leaders on specific areas of training which they would like to receive, especially regarding
STEM. Some of these topics might be: what STEM is, the core principles of STEM learning,
practical ways to implement STEM, skills learned in STEM, integrating STEM concepts into
non-STEM contexts, additional resources for STEM instruction. Based on feedback from the
38
Nsoah Abu-Rasool wrote this section. Linda Moon made contributions.
A NEEDS ASSESMENT OF MEADOWS 82
program leaders, RNH administrators (i.e., program director and supervisors) should construct
training tailored to the more urgent and prevalent needs. Since the program leaders are
requesting specific trainings they would have a vested interest in them and would be all the more
likely to attend. It would also empower program leaders if they knew that their suggestions
were being considered and necessary changes were being made as a result of their feedback and
requests. Following this training event, program leaders should be given an assessment form that
clearly identifies that this training was offered in response to their requests. Essentially, this
form is asking if this particular training event satisfied their needs. An exemplar of an
assessment form can be found at http://www.usi.edu/science.
Increase frequency of trainings. When professional development training is given, it is
often met with failure because it is usually offered without a chance to continue practicing the
newly learned skill. According to Darling-Hammond et al. (2009), without support during this
phase, it is highly unlikely that teachers will persevere with the newly learned
strategy. Professional development should be ongoing, and scheduling at designated sites with
organized and structured training should be the objective. Remedial training with feedback
should also be included to insure that the participants are engaged and learning.
The frequency of professional development will ultimately be balanced according to the
managerial needs of the RNH program leadership; however, retaining the knowledge gained
during professional development must be continually practiced. We suggest that RNH
professional development sessions occur once every two months at minimum, in the months that
the inter-district LLC workshops are not offered. And instead of targeting a slew of various
topics, each professional development session should hone in on one or two topics, allowing
ample time for practice of an introduced skill or concept. Weekly training would be ideal but
A NEEDS ASSESMENT OF MEADOWS 83
staffing needs may result in scheduling conflict. However, program supervisors should
continually coach program leaders at each of their sites by scheduling one-on-one meetings with
a different program leader each day, lasting as short as 15-20 minutes, where program leaders
can voice areas in which they need additional support. This would also be a time for program
supervisors to ask whether or not information from training sessions has been helpful. Bridging
the information disseminated in trainings and the actual workplace will help engender an
environment of learning and practice (Thoonen et al., 2011).
Restructure STEM training workshops. Knapp (2003) reports that real teacher
learning occurs in professional development training when it: 1) focuses on instruction with a
strong emphasis on high learning standards, 2) develops teachers' content and pedagogical
knowledge, and 3) models effective instructional practices and activities in the training itself and
for future classroom use. We propose that every STEM workshop include both a pedagogical
and a preparation component. A pedagogical focus in professional development will prepare the
program leaders to be more efficient in the classroom with class management skills that will
enhance student learning. Examples of this would be program leaders learning how to develop a
lesson plan or learning how to facilitate a learning activity more effectively. Preparation
strategies would entail activities such as doing a trial run of the lesson prior to class in order to
get an idea of the amount of time it will take to teach or perform the activity and troubleshooting
the lesson to help better manage things that could go wrong.
A tiered training model, which is already in use, would maximize effectiveness by
targeting the appropriate audience for each training session by identifying beginning,
intermediate, advanced, and applying program leaders. This will help mitigate redundancy for
A NEEDS ASSESMENT OF MEADOWS 84
veteran program leaders, and ensure novice program leaders are not overwhelmed with
unfamiliar concepts.
Involve outside experts. A few of the program leaders and supervisors expressed that
they would like to see more outside experts, particularly in STEM. This may be a costly venture
depending on how much such experts need to be compensated for their time and travel. RNH is
unique in their employment of full-time afterschool staff (i.e., program supervisors), and
therefore should maximize the potentials of their existing staff. Moreover, relegating training to
outside experts could be disadvantageous especially because said expert is most likely going to
be unfamiliar with all the academic, social, and political dynamics at RNH. Training conducted
by internal staff would most likely generate more specific and relevant information, rather than
generic best practices.
However, in order to achieve superior training by program supervisors and other related
RNH staff; they in turn need to be well-trained, which can more easily be executed with the
assistance of outside experts. One way this could be achieved is to invite masters or doctoral
students to host workshops during their district-wide training events, and offer a small
honorarium to that effect. Otherwise, perhaps Meadows could partner with another institution,
such as USC, where students working on a teaching credential or even a doctoral degree could
obtain credits by participating in these events.
Curriculum Adequacy
39
While our data did not provide a clear understanding of the MindWorks curriculum’s
capacity for promoting STEM learning, we found that there are some issues regarding
curriculum that do need to be addressed. We found that RNH stakeholders should be concerned
with the following areas: 1) having sufficient supplies, 2) reinforcing project-based learning
39
Linda Moon wrote this section. Nsoah Abu-Rasool made contributions.
A NEEDS ASSESMENT OF MEADOWS 85
model, and 3) compliance with school instruction and state standards (i.e., Common Core State
Standards and Next Generation Science Standards). Accordingly, we provide the following
recommendations for RNH’s consideration.
Prevent supply shortages. Program leaders and program supervisors alike stated that
supplies in the MindWorks kits tended to run out because of the box being rotated from class to
class. To prevent supply shortages, program supervisor could have the disposable supplies for
each class removed and placed in separate bags or containers, such that whenever a class
receives their box, their supplies will remain intact. Some of the curriculum supplies are
reusable and do not need to be removed from the kits, but any material that can only be used
once, should be evenly divided among the program leaders for future use.
Realign project-based learning with STEM instruction. A deeper understanding of
project-based learning is an identified need of the program leaders. One way to ensure that all
involved stakeholders are aware of the principles of PBL, is to implement a systemic change
with respect to the protocols filled out by program leaders also called PBL. Even though the
principles of PBL are embedded in this form, it seems to get overlooked and is just seen as
obligatory sections of a lesson plan worksheet. The first step would be to change the name of the
worksheet/protocol and work to remove using “PBL” as the shorthand name for the worksheet
out of the organizational culture. The second would be to add a question that would have the
program leader indicate that all the components of a project-based learning model is being
adhered to: elements such as having the problem as a starting point, small group collaboration,
minimal direct instruction, and time for self-study (Schmidt et al., 2007; Bybee, 2010). This
way, PBL could then possibly become a standardized way for the program leaders to teach PBL,
which is a core component of STEM, consistently throughout the RNH program.
A NEEDS ASSESMENT OF MEADOWS 86
Familiarize RNH staff with standards. While the MindWorks curriculum claims to be
in compliance with Common Core State Standards (CCSS) and Next Generation Science
(NGSS) standards, program staff members do not seem to be all too familiar with what they
actually are and how they impact instruction. First of all, both program supervisors and program
leaders need to be aware of what they are by utilizing online resources (see
http://www.commoncore. pearsoned.com for CCSS and http://www.weareteachers.com for
NGSS). To account for there being insufficient time for preparation and training, these modules
can be broken down into shorter segments as needed.
Moreover, the RNH administrators should designate a time when they can reassess the
organizational goals and objectives in reference to CCSS and NGSS. According to their website
(http://www.Meadows.k12.ca.us/RNH), their program goals are the following: 1) learning that is
active, 2) learning that is collaborative, 3) learning that is meaningful, 4) learning that supports
mastery, and 4) learning that expands horizons. In order to enrich the STEM components,
program staff should consider creating actionable objectives under each of these umbrella goals
that are geared towards STEM learning and in compliance with CCSS/NGSS. Additionally,
each of these goals should be C
3
goals--concrete, challenging, and current (Clark & Estes,
2008). For example, one of the Common Core mathematics competencies is: “construct viable
arguments and critique the reasoning of others” (http://www.cde.ca.gov/re/cc). This competency
could easily translate into a C
3
goal with a STEM focus under the larger goal of “learning that
supports mastery.” By reworking state standards into the program goals and enforcing these
goals, RNH can not only be informed about the standards being implemented currently, but the
program itself will already be in alignment with these competencies.
Further Considerations
A NEEDS ASSESMENT OF MEADOWS 87
While the current project was a needs assessment of instructor capacity, professional
development, and curriculum adequacy in promoting STEM learning among RNH students, a lot
of program leaders and program supervisors conveyed that they are most excited when they see
that they are impacting the kids directly. One possible study that can be conducted in the future
would be the efficacy of the STEM segment of RNH in predicting student involvement with
STEM in the future. This would most likely be in the form of a longitudinal study that would
follow the same group of students participating in the STEM program for a number of years,
possibly until senior year of high school. At that point, they might ask these students about their
career plans and their views on STEM-related careers. A simplified version of this study could
be for the RNH program to track down past students who had once been a part of the STEM
programming at RNH and ask them similar questions about how they felt the RNH program,
with regards to STEM instruction, had or had not shaped their views on STEM.
Another possible study could be to gather information regarding students from both the
daytime faculty (i.e., credentialed teachers) at each school site and the RNH after school
staff. While previous evaluations of RNH attempted to catalogue this relationship, more
descriptive and complete data could be compiled to determine a causal relationship, if any,
between student involvement in RNH and their academic performance. The current project
relied mostly on reflection and feedback given by the RNH program staff, but further research
could show, for example, if students’ grades have improved since enrolling in RNH or if students
in RNH are achieving higher than those who are not.
Another point of departure would be to assess how STEM is being integrated across
disciplines, or other clubs at RNH. While this needs assessment was focused on the MindWorks
clubs, true STEM learning would be for scientific or engineering thought processes to be applied
A NEEDS ASSESMENT OF MEADOWS 88
in all activities. This would require assessing how various aspects of a STEM-centered mindset,
such as critical thinking, problem solving, and inquiry skills, are being employed in a non-STEM
discipline, such as dance or journalism.
Furthermore, given that the RNH staff have built and maintained relationships with the
students enrolled in RNH, serving as mentors and positive adult role models, they are in a unique
position to influence students out of the scope of teachers and parents. As informal educators,
students enter into voluntary relationships with the program staff so there is a lot of inherent
trust. Learning how to effectively utilize these relationships in promoting STEM learning
outcomes could be another source for further research. This kind of study would involve
interviewing both program leaders and students, and even parents, in order to identify attributes
of trustworthy and relational program leaders, along with the extent to which students adhere to
advice and instruction set for by such persons.
A final aspect for consideration is that there was no way of knowing how much of the
issues are related to budgeting. A more comprehensive assessment may require reevaluating
how funds are allocated and may be redistributed to bring about necessary changes that may on
the whole benefit students’ learning of STEM.
Limitations of Project
40
As comprehensive as we strived to be, this needs assessment had some limitations,
mainly in the areas of data collection. The first major aspect in which we faced limitations was
in that of survey design. For one, the survey was designed prior to meeting with any of the
program leaders or supervisors, so the survey questions were much more vague than
necessary. The instructor capacity questions could have separated knowledge into content and
pedagogical content knowledge so it could be more clear in which area of knowledge the
40
Nsoah Abu-Rasool wrote this section. Linda Moon made contributions.
A NEEDS ASSESMENT OF MEADOWS 89
program leaders are stronger. For the motivation component of instructor capacity, we could
have added more items under interest and value for a more complete picture of how those two
components actually are relevant. Moreover, there could have been more questions that asked
about the excitement and energy, which is a factor of motivation that is most visible to the
supervisors, regardless of what their internal beliefs about motivation might be. The professional
development questions could have been separated to ask about the different training events
offered by RNH and by LLC. And as mentioned above, the curriculum questions could have
been more explicit in naming MindWorks as the STEM curriculum for the purposes of the
project. Additionally, we did not collect demographic data regarding race and ethnicity, which
we did not need for the purposes of this study but may have been instrumental for other
considerations.
Similarly, focus groups and interviews could have been more streamlined, with concise
and unambiguous questions. According to Merriam (2009), the way in which questions are
worded is a crucial consideration in extracting the type of information desired. An obvious place
to begin is by making certain that what is being asked is clear to the person being
interviewed. For instance, during focus group sessions, when participants were asked questions
about project based learning (PBL), they responded by giving descriptions of how they set up
their lesson plans, which coincidentally is also called PBL. While their lesson plan forms were
probably modeled after the broader construct of PBL, this underlying theory was not mentioned,
which was, in fact, the root of the question being asked. Sometimes, these kinds of unexpected
responses would lead to straying off topic and would consume a larger portion of the interview
sessions than we anticipated. This became a perceptible challenge, as we already had a very
limited amount of time allotted for these interviews, based on the schedules of the RNH staff.
A NEEDS ASSESMENT OF MEADOWS 90
For observations, it might have been beneficial to observe more classrooms and more
frequently, especially on different days of the week. Due to difficulties in scheduling, we were
only able to meet with the program leaders on one Thursday for each site. Perhaps having the
ability to conduct more observations throughout the week at each site, would have provided a
more holistic picture of what takes place during the course of a school week.
In terms of data analysis, we only focused on descriptive data, but we could have also ran
some correlational analyses. For instance, we could have ran a correlation to see if there was a
relationship between program leader knowledge and motivation. We could have also compared
the results by gender, age, or years of experience to see if there were any significant findings
between the groups.
Another limitation is in sampling. We only had 45 survey respondents, which was a
significant number with respect to the 50 program leaders total, but still a relatively small
sampling size in terms of gleaning reliable data. Additionally, more descriptive data could have
been collected if we had also included students and parents in this project. Program leaders and
program supervisors are both on the administrative side of STEM teaching, but students and
parents are on the receiving end, thus their input would have provided an even more
comprehensive analysis, especially with regard to the extent to which STEM learning is actually
taking place in RNH.
Finally, there is the issue of bias. Most of the information was collected through self-
reporting, which yields inherent bias and sometimes erroneous responses. The Qualtrics survey
contained 58 questions, including demographic questions and all the embedded questions (e.g.,
4-a, 4-b, 4-c, 4d). Because of the lengthiness of the survey and no incentive to finish it,
respondents possibly did not think through each question thoroughly, or just picked a number all
A NEEDS ASSESMENT OF MEADOWS 91
the way down. In addition, a lot of the respondents skipped the open-ended questions. In the
focus group and interviews, there is the possibility that the respondents were not entirely
forthcoming about their opinions due to the sensitivity of the topics and wariness of being
identified, even with assurances of anonymity.
A NEEDS ASSESMENT OF MEADOWS 92
References
Afterschool Alliance. (2011). After school and summer programs: Committed partners in STEM
education. Retrieved fromhttp://www.afterschoolalliance.
org/STEM_JointPositionPaper.pdf
Altschuld, J. W., & Kumar, D. D. (2010). Needs assessment an overview.
Thousand Oaks, California: Sage
Ambrose, S. A., Bridges, M. W., DiPietro, M., Lovett, M.C., & Norman, M. K. (2010).
How learning works: 7 research-based principles for smart teaching.
San Francisco: Jossey-Bass.
Anderson, L. W., & Krathwohl, D. R. (2001). A taxonomy for learning, teaching, and assessing:
A revision of Bloom’s taxonomy of educational objectives.
New York: Addison Wesley Longman, Inc.
Atkinson, R. D. (2012). Why the current education reform strategy won't work.
Issues in Science and Technology, 28(3), 29-36. Retrieved from
http://search.proquest.com/docview/1035325038?accountid=14749
Baldi, S., Jin, Y., Skimmer, M., Green, P. J., and Herget, D. (2007). Highlights from PISA
2006: Performance of U.S. 15-Year-Old Students in Science and Mathematics Literacy in
an International Context (NCES 2008-016). Washington, DC: U. S. Department of
Education, National Center for Education Statistics, Institute of Education Sciences
Basham, J. D., Israel, M., & Maynard, K. (2010). An ecological model of STEM
education: Operationalizing STEM for all. Journal of Special Education Technology,
A NEEDS ASSESMENT OF MEADOWS 93
25(3), 9-19. Retrieved from
http://search.proquest.com/docview/813326914?accountid=14749
Beers, S. Z. (2012). 21st century skills: Preparing students for their future.
Bevan, B., Michalchik, V., Bhanot, R., Rauch, N., Remold, J., Semper, R., & Shields, P. (2010).
Out-of-school time STEM: Building experience, building bridges.
Bhattacharyya, S., Volk, T., & Lumpe, A. (2009). The influence of an extensive inquiry-based
field experience on preservice elementary student teachers' science teaching beliefs.
Journal of Science Teacher Education, 20(3), 199-218.
Bybee, R. W. (2010). Advancing STEM education: A2020vision. Technology and Engineering
Teacher, 30-35.
California Department of Education. (2015). Environment education/ science definitions.
Retrieved from http://www.cde.ca.gov/pd/ca/sc/oeedefinitions.asp
Carr, R. L., Bennett, L. D. IV., & Strobel, J. (2012). Engineering in the K-12 STEM
standards of the 50 U.S. states: An analysis of presence and extent.
Journal of Engineering Education, 101(3), 539-564. Retrieved from
http://search.proquest.com/docview/1024810519?accountid=14749
Caton, E., Brewer, C., & Brown, F. (2000). Building teacher-scientist partnerships: Teaching
about energy through inquiry. School Science and Mathematics, 100(1), 7-15.
Chute, E. (2009). STEM education is branching out: Focus shifts from making science, math
accessible to more than just brightest. Pittsburg Post-Gazette. Retrieved October 14,
2011, from http://www.post-gazette.com/ pg/09041/947944-298.stm
Clark, R. E. & Estes, F. (2008). Turning research into results: A guide to selecting the right
performance solutions. Atlanta GA: CEP Press
A NEEDS ASSESMENT OF MEADOWS 94
Collins, A., & Halverson, R. (2009). Rethinking education in the age of technology.
New York, NY: Teachers College Press
Darling-Hammond, L. (2006). Constructing 21st century teacher education.
Journal of Teacher Education, 57, 300-314.
Darling-Hammond, L., Wei, R. C., Andree, A., Richardson, N., & Orphanos, S. (2009).
Professional learning in the learning profession: A status report on teacher development
in the United States and abroad. Washington, DC: National Staff Development Council.
Dierking, L. D. (2007). Linking after-school programs and STEM learning: A view from another
window: Oregon Sea Grant.
Diffily, D., & Sassman, C. (2002). Project-based learning with young children.
Portsmouth, NH: Heinemann.
Dresner, M., & Moldenke, A. (2002). Authentic field ecology experiences for teachers.
The American Biology Teacher, 64(9), 659-663. Retrieved from
http://search.proquest.com/docview/218992245?accountid=14749
Ejiwale, J. A. (2012). Facilitating teaching and learning across STEM fields.
Journal of STEM Education : Innovations and Research, 13(3), 87-94.
Retrieved from http://search.proquest.com/docview/1015211575?accountid=14749
Fioriello, P. (2010). Understanding the basics of STEM education.
Retrieved from http://drpfconsults.com/understandingthe-basics-of-stem-education/
Freeman, J., Dorph, R., & Chi, B. (2009). Strengthening after school STEM staff development.
Lawrence Hall of Science, University of California Berkeley.
Gibson, S., & Dembo, M. H. (1984). Teacher efficacy: A construct validation.
Journal of Educational Psychology, 76(4), 569-82.
A NEEDS ASSESMENT OF MEADOWS 95
Gottfredson, D., Cross, A. B., Wilson, D., Rorie, M., & Connell, N. (2010). Effects of
participation in after-school programs for middle school students: A randomized trial.
Journal of Research on Educational Effectiveness, 3(3), 282-313. doi: 10.1080/
19345741003686659
Gomez, A., & Albrecht, B. (2014). True STEM education.
Technology and Engineering Teacher, 73(4), 8-16.
Retrieved from http://search.proquest.com/docview/1468440583?accountid=14749
Grubbs, M. (2013). Robotics intrigue middle school students and build STEM skills.
Technology and Engineering Teacher, 72(6), 12-16.
Retrieved from http://search.proquest.com/docview/1318545436?accountid=14749
International Technology Education Association (ITEA/ ITEEA). (2009). The overlooked STEM
imperatives: Technology and engineering. Reston, VA: Author
Knapp, M. S. (2003). Professional development as a policy pathway.
Review of research in education, 27, 109-157.
Meadows elementary school district: Reaching new heights. Retrieved May 8, 2014, from
http://www.Meadows.k12.ca.us/RNH
Lai, C. (2013). A study of computer project-based learning on electric circuits for 4th graders.
The Journal of Human Resource and Adult Learning, 9(1), 55-61.
Retrieved from http://search.proquest.com/docview/1433043389?accountid=14749
Lauer, P. A., Akiba, M., Wilkerson, S. B., Apthorp, H. S., Snow, D., & Martin-Glenn, M. L.
(2006). Out-of-school-time programs: A meta-analysis of effects for at-risk students.
Review of educational research, 76(2), 275-313.
Leamson, R. (2000). Learning as biological brain change. Change, 32(6). 34-40.
A NEEDS ASSESMENT OF MEADOWS 96
Livingstone, D., and Lynch, K., 2000, Group project work and student-centered active learning:
two different experiences: Studies in Higher Education, v. 25, p. 325-345.
Mahoney, J. L., Parente, M. E., & Lord, H. (2007). Afterschool program engagement: Links to
child competence and program quality and content. The Elementary School Journal,
107(4), 385-404.
Merriam, S.B. (2009). Qualitative Research: A guide to design and implementation. San
Francisco, CA: Jossey-Bass
Miller, P. M. (2010). Community-based education and social capital in an urban after-school
program. Education and Urban Society, 44(1), 35-60.
Nadelson, L. S., Seifert, A., Moll, A. J., & Coats, B. (2012). i-STEM summer institute:
An integrated approach to teacher professional development in STEM.
Journal of STEM Education : Innovations and Research, 13(2), 69-83.
Retrieved from http://search.proquest.com/docview/1009901964?accountid=14749
National Academy of Sciences (2007). Rising above the gathering Storm:
Energizing and employing America for a brighter economic future. Washington, DC:
National Academies Press
National Governors Association (NGA). (2011). The governors' speak:
A summary of the 2011 state of the state addresses.
Retrieved from www.nga.org/files/live/sites/ NGA/files/pdf/GOVSPE AKl 104.PDF
National Institute for Science Education. (1999). Form and substance in mathematics and
science professional development. (Issue Brief Vol. 3 No. 2).
Madison, WI: M. M. Kennedy.
National Science Foundation. (2008). Chapter 3, Figure 3-5. Arlington, VA
A NEEDS ASSESMENT OF MEADOWS 97
Robelen, E. W. (2011). Awareness grows of importance of learning science beyond school.
Education Week, 30(27), 2-5.
Sahin, A. (2013). STEM clubs and science fair competitions: Effects on post-secondary
matriculation. Journal of STEM Education : Innovations and Research, 14(1), 5-11.
Retrieved from http://search.proquest.com/docview/1349180552?accountid=14749
Schmidt, H. G., Loyens, S. M., Van Gog, T., & Paas, F. (2007). Problem-based learning is
compatible with human cognitive architecture: Commentary on kirschner, sweller, and
clark (2006). Educational Psychologist, 42(2), 91-97.
Schraw, G., Flowerday, T., & Lehman, S. (2001). Promoting situational interest in the
classroom. Educational Psychology Review, (13), 211–224.
Science Education Resource Center. 2003. Developing the Earth science teacher workforce: The
role of geoscience departments and introductory courses.
Available at http://serc.carleton.edu/ teacherprep/index.html (accessed 23 June 2012).
Siegel, M. A., Mlynarczyk-Evans, S., Brenner, T. J., & Nielsen, K. M. (2005). A natural
selection: Partnering teachers and scientists in the classroom laboratory creates a dynamic
learning community. Science Teacher, 72(1), 42-45.
Sheldon, J., Arbreton, A., Hopkins, L., & Grossman, J. B. (2010). Investing in success: Key
strategies for building quality in after-school programs.
American journal of community psychology, 45(3-4), 394-404.
Subotnik, R. T., Tai, R. H., Rickoff, R., & Almarode, J. (2010). Specialized public high schools
of science, mathematics, and technology and the STEM pipeline:
What do we know now and what will we know in 5 years? Roeper Review, 32(1), 7-16.
Retrieved from http://search.proquest.com/docview/206685690?accountid=14749
A NEEDS ASSESMENT OF MEADOWS 98
Thompson, R., & Bolin, G. (2011). Indicators of success in STEM majors:
A cohort study. Journal of College Admission, (212), 18-24.
Thoonen, E. E., Sleegers, P. J., Oort, F. J., Peetsma, T. T., & Geijsel, F. P. (2011). How to
improve teaching practices the role of teacher motivation, organizational factors, and
leadership practices. Educational Administration Quarterly, 47(3), 496-536.
Tschannen-Moran, M., Hoy, A.W., & Hoy, W.K. (1998). Teacher efficacy:
Its meaning and measure. Review of Educational Research, 68(2), 202 - 248. doi:
10.2307/1170754
Tsupros, N., R. Kohler, and J. Hallinen, 2009. STEM education: A project to identify the missing
components, Intermediate Unit 1 and Carnegie Mellon, Pennsylvania
A NEEDS ASSESMENT OF MEADOWS 99
Appendix A
Survey Questionnaire for RNH Program Leaders and Program Supervisors
Each of the following questions will be answered on a 6-point Likert scale (1-strongly disagree, 2-
disagree, 3-somewhat disagree, 4-somewhat agree, 5-agree, 6-strongly agree), unless otherwise
specified. [45 questions]
Construct measured Question Source
Instructor
Capacity
Knowledge
My (Program leaders’) academic background and/or work
experience has given me (them) the necessary skills to be an
effective instructor.
Knowledge I (Program leaders) are aware of the basic principles of STEM
teaching and learning.
Knowledge I (Program leaders) am/are aware of what inquiry-based learning is.
Knowledge
I (Program leaders) have sufficient knowledge of the science
curriculum used in the program.
Adapted from
the Baseline
IFSS Coach
Survey
Knowledge
I (Program leaders) have sufficient knowledge of the different ways
technology is content used in the program.
Knowledge
I (Program leaders) have sufficient knowledge of the how
engineering is content used in the program.
Knowledge
I (Program leaders) have sufficient knowledge of the how math
content is used in the program.
Knowledge
I (Program leaders) know how to use effective teaching strategies to
guide student thinking and learning in science.
Knowledge
I (Program leaders) know how to use effective teaching strategies to
guide student thinking and learning in technology.
Knowledge
I (Program leaders) know how to use effective teaching strategies to
guide student thinking and learning in engineering.
Knowledge
I (Program leaders) know how to use effective teaching strategies to
guide student thinking and learning in math.
Motivation
(self-efficacy)
I (program leaders) am/are very motivated to get new ideas and
strategies to better teach STEM.
Motivation
(self-efficacy)
I (Program leaders) have the confidence to teach STEM related
curriculum and activities well.
Motivation
(self-efficacy)
I (Program leaders) have the skills to teach STEM related curriculum
and activities well.
Motivation
(teaching
beliefs;
outcome
expectancy)
Program leaders are a powerful influence on student achievement
in STEM.
Adapted from
Teacher
Efficacy Scale
Motivation
(teaching
beliefs;
outcome
expectancy)
If a student is able to master a STEM concept, it is because I knew
implemented the necessary steps in teaching the curriculum.
Adapted from
Teacher
Efficacy Scale
Motivation
(value)
I/Program leaders believe that it is important for students excel in
STEM.
Motivation I think it is very important to incorporate STEM into all program
A NEEDS ASSESMENT OF MEADOWS 100
(value) activities.
Motivation
(interest)
My/Program leaders’ personal interest in STEM directly affects how
well I/they teach STEM related curriculum.
Motivation
(teaching
beliefs;
outcome
expectancy)
I believe that facilitating related activities at my site has a direct
impact on...
- my/program leaders’ own knowledge of STEM related content.
- my/program leaders’ own interest in STEM related content
- students’ knowledge of STEM related content
- students’ interest in STEM related content
- students’ awareness of an interest in pursuing STEM related
careers.
Adapted from
the STEM
Survey for Site
Coordinators
Professional
Development
I (Program leaders) have the resources and supplies to do my (their) job well. Adapted from
the Baseline
IFSS Coach
Survey
and the STEM
Survey for Site
Coordinators
I (Program leaders) receive the training I (they) need to be successful.
I (Program leaders) receive the coaching that I (they) need to be successful.
I (Program leaders) get the support that I (they) need from the program
supervisor.
In the past year, how many of the following STEM related support services have
you received?
__________ Training on how to integrate elements of STEM into program activities
__________ Coaching on how to implement STEM training content into program
activities
I think that the following factors have greatly helped the implementation of STEM
related activities within my program:
adequacy of space/facilities
availability of trained staff
access to resources, curricula, and/or materials access to internet-
connected computers
Please rank the following according to what you think are the most pressing
needs related to implementing STEM activities at your site (1=MOST pressing,
5=LEAST pressing).
training (e.g., integrating STEM into program activities, evaluating STEM
activity outcomes)
information and resources (e.g., existing STEM curricula, developing
partnerships with other agencies, funding opportunities, evaluation
tools) that I might be able to use in my program
coaching (e.g., how to integrate and/or improve STEM components in
program activities)
consultation outside experts (e.g., how to integrate and/or improve
STEM components in program activities)
site visits to other sites implementing effective STEM activities
Professional development workshops are vital to helping program leaders
develop new teaching techniques.
Adapted from
Teacher
Opinions
About
Professional
Development
Questionnaire
I (Program leaders) have been enriched by the training events I (they) have
attended.
Without the training I (they)receive in professional development, I(they) would
not be able to teach STEM effectively.
Curriculum
Our program provides exciting and engaging enrichment opportunities for
students.
Adapted from
the Baseline
IFSS Coach
Survey and
STEM Survey
for Site
Our program supports improvement in student academic performance.
The curriculum includes activities and approaches aimed at improving the
leadership and character development of students.
Please indicate how often you facilitate each of the activities occur at your site (1-
A NEEDS ASSESMENT OF MEADOWS 101
2 times a month; 1-2 times a week; 3-4 times a week; every day; not at all)
technology-based activities (e.g., using iPads, multimedia projects)
science-based activities (e.g., robotics, environmental science projects)
math-related activities (e.g., creating bar gRNHhs to display data)
engineering-related activities (e.g. robotics, building devices)
Coordinators
Using the current STEM curriculum…Using the current curricula…
students are able to use scientific vocabulary and principles in everyday
discussions.
students are able to use and recognize that science and technology are
developed to meet societal needs and expand human capability
students are able to voice their own opinions about STEM
students are able to show interest in STEM-related questions and issues
students are able to and confidently pursue personal interests and
natural curiosities about STEM
students are able to consider and career possibilities within STEM-related fields.
Adapted from
Curriculum
Emphasis
Survey
The following skills are almost always built into our program activities:
inquiry skills -based learning (posing a question and investigating an
answer)
problem solving (identifying and employing strategies to resolve
problems)
critical thinking (identifying and considering multiple aspects of an
issue/situation to make an informed conclusion)
Adapted from
STEM Survey
for Program
Directors/
Grant
Managers
I think our current STEM curriculum is headed in the right direction to help
students to excel in STEM.
I believe the students enjoy the current STEM curriculum.
Demographic
information
Please indicate your position in RNH:
[] program supervisor
[] program leader
How many years have you been in your current position?
Please indicate your gender.
[] Female
[] Male
What is your age?
What is the last degree you have obtained or are currently working on?
[] High school graduate, diploma or the equivalent (for example: GED)
[] Some college credit, no degree
[] Trade/technical/vocational training
[] Associate degree
[] Bachelor’s degree
[] Master’s degree
[] Professional degree
[] Doctorate degree
What is your intended or completed field of study or major of expertise, if
applicable?
How many years of teacher training outside of RNH, if any, do you have?
(PL only) Please list each club you facilitate.
Indicate your site.
[] Anderson
[] FDR
[] Green
[] Mitchell
[] Smith
[] Twain
A NEEDS ASSESMENT OF MEADOWS 102
Appendix B
Focus Group Protocol for RNH Program Leaders
Interviewer: Date:
Interviewees: Titles:
Contact info:
Start time: End time:
Introduction:
Hello, my name is XX, and I am a doctoral
student at the University of Southern California’s
Rossier School of Education and I am conducting a
needs assessment for the RNH after school STEM
education program. During this conversation, we are
hoping to learn more about your role and experiences
with RNH and the STEM program. I want to assure
you that your comments will be strictly confidential.
We will not identify you, or your school site, by
name. I would like to tape record this interview in
order to have an accurate record of our conversation.
Would that be okay? The focus group session should
take approximately 30 minutes. Do you have any
questions before we begin?
A. Background
Before I ask you specific questions about the RNH
Program I would like to start by asking you some
background questions about:
1) How long have you been in your position?
2) What is your prior experience and training?
a) What is your STEM experience and
training?
b) What is your leadership experience?
3) Talk me through your role as a program
supervisor.
B. Instructor Capacity
1) What is your understanding of STEM education?
2) How do you typically prepare to teach STEM
content?
3) What do you think would be the best way for
students to learn STEM content?
a) Why do you feel that this method is the best
way to teach STEM?
b) How do you know students are learning?
4) What is your understanding of 21
st
century
learning skills?
5) What is your understanding of project based
learning?
6) What motivates you to teach STEM?
C. Professional Development
1) How often do you attend professional
development sessions?
a. How long do they usually last?
b. Who conducts them?
2) Describe a typical PD session.
a. What do you like most about the
professional development session?
b. What do you least like?
3) Talk me through how you apply what you have
learned from a professional development session
in the classroom.
a. How do you think it is working for the
students?
4) Do you have an opportunity to make suggestions
from your classroom experiences during
professional development sessions?
a. If so how are the received?
b. If not why?
5) What would you like to see changed in how PD
is conducted at your site?
a. Why would you choose to do it
different?
b. What would you keep the same?
6) Describe the resources available to you.
a. What additional resources would you
like?
D. Curriculum Assessment
1) What is latest STEM curriculum being used to
help teach students?
2) How does your STEM curriculum address 21
st
century learning skills?
3) How is project based learning being used in the
STEM after school program?
4) How is the curriculum generally received by…
a. Students?
b. Instructors?
5) Why do you think the RNH STEM curriculum
works or doesn’t work?
Is there anything else that I didn’t ask that you
would like to mention?
Thank you for taking the time to answer my
questions.
A NEEDS ASSESMENT OF MEADOWS 103
Appendix C
Interview Protocol for RNH Program Supervisors
Interviewer: Date:
Interviewee: Contact info:
Start time: End time:
Introduction:
Hello, my name is XX, and I am a doctoral
student at the University of Southern California’s
Rossier School of Education and I am conducting a
needs assessment for the RNH after school STEM
education program. During this conversation, we
are hoping to learn more about your role and
experiences with RNH and the STEM program. I
want to assure you that your comments will be
strictly confidential. We will not identify you, or
your school site, by name. I would like to tape
record this interview in order to have an accurate
record of our conversation. Would that be okay?
The interview should take approximately 30
minutes. Do you have any questions before we
begin?
A. Background
1) How long have you been in your position?
2) What is your prior experience and training?
a) What is your STEM experience and training?
b) What is your leadership experience?
3) Talk me through your role as a program
supervisor.
B. Instructor Capacity
1) What do you look for in a program leader’s
understanding of STEM education?
2) What expectations do you have of program
leaders in how they prepare and teach STEM?
3) How do you evaluate program leaders’
performance?
4) How do you address program leaders’
motivation? (e.g. incentives)
C. Professional Development
1) How often are professional development
sessions offered at your site?
a. How long do they usually last?
b. Who conducts them?
2) Describe a typical PD session.
3) Talk me through how you prepare for a
professional development session.
a. How do you think it is working for the
program leaders?
4) Is there anything about your PD sessions you
would like to change?
a. If so, why would you choose to do it
differently?
b. What would you keep the same?
5) What was your best professional development
experience and why?
a. Your worst and why?
6) Where do program leaders get resources
needed to teach STEM classes?
D. Curriculum Assessment
1) What is latest STEM curriculum being used to
help teach students?
2) How does your STEM curriculum address 21
st
century learning skills?
3) How is project based learning being used in the
STEM after school program?
4) How is the curriculum generally received by…
a. Students?
b. Instructors?
5) Why do you think the RNH STEM curriculum
works or doesn’t work?
Is there anything else that I didn’t ask that you
would like to mention? Thank you for taking the
time to answer my questions.
A NEEDS ASSESMENT OF MEADOWS 104
Appendix D
Observations Protocol
Name of Observer Date Time
Location Study
Brief Summary of Observation
Physical Space
Define the physical space.
Geographical
Temporal
Physical
Political
Utility: What is the purpose
of event/setting?
Participant reactions to
physical setting
Other
People/Participants
Who are the participants
taking place in
observation/event?
How many participated?
Demographical information
(e.g. racial, ethnic, gender,
class):
What does learning look like?
How are activities organized- structure, content etc.?
What materials are being used and by whom?
Does the learning context appear to be teacher or student centered?
o When and in what ways?
What are students doing?
Types of activities
Ways of engaging in activities
Ways of interacting with peers and staff
How do they appear to be making choices about what they are doing- both activity type and tasks within an
activity?
What are the staff members doing?
How do the staff members interact with kids?
o Do they ask questions? If so, what kind and in what circumstances?
How do the staff members facilitate the activities?
How are rules enforced?
A NEEDS ASSESMENT OF MEADOWS 105
Do staff members demonstrate particular skill sets that appear effective for working with this age group?
Other observations about student behavior:
Do students seem happy?
Do students seem engaged in their activities?
Do students seem to have close relationships with program staff?
Do students interact with each other respectfully?
Purpose of Events/Observation
Why is the event taking
place? Are there any
political contexts to be
discussed?
Who was invited to event?
Who was not?
Was there any discussion
of educational policy?
Why? How so?
What are the positions of
the various participants
involved?
Power dynamics
Roles
What is being discussed?
Sequence of Events
Beginning
Middle
End
Observer Role
What am I doing? What is
my role throughout the
observation?
Describe some of my
interactions with other
participants throughout the
observation.
How did my
interaction/presence affect
the observation
participants?
Other
Pictures
A NEEDS ASSESMENT OF MEADOWS 106
Appendix E
IRB Approval Letter
UNIVERSITY OF SOUTHERN CALIFORNIA
UNIVERSITY PARK INSTITUTIONAL REVIEW BOARD
FWA 00007099
Exempt Review
Date: Sep 27, 2014, 08:46am
Principal Investigator: Robert Rueda
ROSSIER SCHOOL OF EDUCATION
Faculty Advisor:
Co-Investigators:
Gale Sinatra
ROSSIER SCHOOL OF EDUCATION
Nsoah Abu-Rasool
ROSSIER SCHOOL OF EDUCATION
Linda Moon
ROSSIER SCHOOL OF EDUCATION
Project Title: RNH STEM Needs Assessment
USC UPIRB # UP-14-00532
The iStar application and attachments were reviewed by UPIRB staff on 9/27/2014
The project was APPROVED.
Based on the information provided for review, this study meets the requirements outlined in 45 CFR 46.101(b)(1),
(2) & (4) and qualifies for exemption from IRB review. The study is not subject to
further IRB review. IRB exemption of this study was granted on 9/27/2014.
To access IRB-approved documents, click on the “Approved Documents” link in the study workspace. These
are also available under the “Documents” tab.
Researchers are reminded that some schools require permission to conduct research even if the research is exempt
from IRB review.
Sincerely,
Rose Ann Fleming, CIP
Approved Documents: view
Funding Source(s):
No Funding Sources
A NEEDS ASSESMENT OF MEADOWS 107
Appendix F
Data Collection Plan
DATE SITE TIME DATA PARTICIPANTS
Thursday,
October 23
Hammer
12:15-12:45pm Focus group Program leaders
1:00-3:00pm Observations
3:00-3:30pm Interview Program supervisor
Thursday,
October 30
Benny 12:15-12:45pm Focus group Program leaders
Thursday,
November 13
Carver
12:15-12:45pm Focus group Program leaders
1:00-3:00pm Observations
3:00-3:30 Interview Program supervisor
Benny
4:00-5:30pm Observations
5:30-6:00pm Interview Program supervisor
Thursday,
December 4
Recinos
12:15-12:45pm Focus group Program leaders
1:00-3:00pm Observations
3:00-3:30pm Interview Program supervisor
Thursday,
December 11
Dos Santos
12:15-12:45pm Focus group Program leaders
1:00-3:00pm Observations
3:00-3:30pm Interview Program supervisor
Bush
4:00-5:30pm Observations
5:30-6:00pm Interview Program supervisor
Thursday,
December 18
Bush 12:15-12:45pm Focus group Program leaders
Abstract (if available)
Abstract
The purpose of this project was to conduct a needs assessment of the Reaching New Heights (RNH) afterschool program in the Meadows Elementary School District (MESD). At the request of the RNH program director three program areas were identified as targets for data collection and analysis, in the science, technology, engineering, and mathematics (STEM) program. Due to advances in technology and the nuances of 21st century learning and curriculums, which include project based learning, the RNH program director wanted to know the status of the staff in order to enhance or maintain program effectiveness through professional development training. The methods used to collect data included a survey, observations, focus groups, and individual interviews. The data were analyzed and recommendations from the literature and findings within the project were made available to the RNH program director. The three primary areas addressed in the recommendations included the STEM capacity of the program leaders/instructors, the effectiveness of professional development, and curriculum adequacy. Additionally, for the sake of protecting the anonymity of the participants, pseudonyms were used to refer to the school district, school sites, and programs.
Linked assets
University of Southern California Dissertations and Theses
Conceptually similar
PDF
A needs assessment and evaluation of an elementary afterschool STEM program in a small urban school district
PDF
A needs assessment of an urban elementary school district's afterschool program
PDF
A needs assessment and program evaluation of an urban school district's after-school STEM program
PDF
A needs assessment of an after school science, technology, engineering, and mathematics program
PDF
A needs assessment of an after school, science, technology, engineering, and mathematics program
PDF
A needs assessment for Hillview Middle School's STEM education program
PDF
STEM integration: a case study of a 21st century skills professional development program for math and science teachers in an urban school
PDF
Capacity building for STEM faculty and leaders: supporting university students with ADHD in earning STEM degrees
PDF
STEM identity development: examining the experiences of transfer students
PDF
Assessing the effectiveness of an inquiry-based science education professional development
PDF
Analysis of STEM programs in Oregon public high schools
PDF
Impact of technology on teaching and learning practices at high‐technology use K-12 schools: a case study
PDF
Promoting STEM integration, interest and identity among elementary school students
PDF
Community college students in STEM: a quantitative study investigating the academic beliefs of students enrolled in online and on campus information technology courses
PDF
A study of California public school district superintendents and their implementation of 21st -century skills
PDF
Transformational technology practices in K-12 schools: a case study
PDF
A capstone project: closing the achievement gap of English learners in literacy at Sunshine Elementary School using the gap analysis model
PDF
An examination of K-12 STEM integration by combining science inquiry with engineering design
PDF
Educational resource allocation at the elementary level: a case study of one elementary school district in California
PDF
Fostering competent professionals: instructional systems specialists at the instructional systems technology program
Asset Metadata
Creator
Abu-Rasool, Nsoah
(author)
Core Title
A needs assessment of Meadows Elementary School District STEM education after school program
School
Rossier School of Education
Degree
Doctor of Education
Degree Program
Education (Leadership)
Publication Date
04/24/2015
Defense Date
03/13/2015
Publisher
University of Southern California
(original),
University of Southern California. Libraries
(digital)
Tag
capacity,curriculum,Knowledge,Motivation,OAI-PMH Harvest,professional development,STEM
Format
application/pdf
(imt)
Language
English
Contributor
Electronically uploaded by the author
(provenance)
Advisor
Rueda, Robert (
committee chair
), Seli, Helena (
committee member
), Sinatra, Gale M. (
committee member
)
Creator Email
nsoah.aburasool@att.net,nsoah.aburasool@usc.edu
Permanent Link (DOI)
https://doi.org/10.25549/usctheses-c3-560237
Unique identifier
UC11301531
Identifier
etd-AbuRasoolN-3391.pdf (filename),usctheses-c3-560237 (legacy record id)
Legacy Identifier
etd-AbuRasoolN-3391.pdf
Dmrecord
560237
Document Type
Dissertation
Format
application/pdf (imt)
Rights
Abu-Rasool, Nsoah
Type
texts
Source
University of Southern California
(contributing entity),
University of Southern California Dissertations and Theses
(collection)
Access Conditions
The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the a...
Repository Name
University of Southern California Digital Library
Repository Location
USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA
Tags
capacity
professional development
STEM