University of Southern California Dissertations and Theses

STEM + design thinking training: investigation of perceived changes in self‐efficacy, pedagogy, and conceptual development at the K-5 level

(USC Thesis Other)

STEM + design thinking training: investigation of perceived changes in self‐efficacy, pedagogy, and conceptual development at the K-5 level

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content Running head: STEM + DESIGN THINKING

STEM + DESIGN THINKING TRAINING:
INVESTIGATION OF PERCEIVED CHANGES IN SELF-EFFICACY, PEDAGOGY,
AND CONCEPTUAL DEVELOPMENT AT THE K-5 LEVEL

by

Trisha Callella

A Dissertation Presented to the
FACULTY OF THE USC ROSSIER SCHOOL OF EDUCATION
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulfillment of the
Requirements for the Degree
DOCTOR OF EDUCATION

May 2015

Copyright 2015 Trisha Callella

STEM + DESIGN THINKING 2

DEDICATION
This dissertation and the doctoral degree are dedicated to my mom for the many sacrifices
she made in order to provide me with the strong educational foundation that led to this. Most
importantly, for being my first teacher, instilling within me a love of learning, and always
believing in me. She continues to be a role model of sheer will, determination, initiative,
perseverance, and the value of self-motivation. Those characteristics are what got me this far.
Actually, all of my achievements can be attributed to those motivational factors she demonstrates
day in and day out. She has always supported my ideas and offered endless support. I am deeply
grateful that she took care of Shyllah every weekend enabling me to reach this final level of
education making her dream come true. Thanks to her, our family tree now has someone who has
not only graduated from college but has now earned a doctoral degree which was her dream for
me since childhood. She modeled and instilled strong moral values, compassion, and empathy
for people and animals. This would not have been possible without her continuous support and
love.

STEM + DESIGN THINKING 3
ACKNOWLEDGEMENTS
I would like to take this opportunity to thank the members of my committee who have
supported me throughout this process with their patience, guidance, and questions. At times, they
believed in me more than I did. I originally chose Dr. Freking, Dr. Maddox, and Dr. Rueda as my
committee due to their incredibly high level of rigor, high expectations, and passion paired with
expertise in the domains of STEM to hold me accountable for the highest quality within the
science, engineering, and education (learning) domains. What I didn’t realize at the time were
the life lessons and self-awareness to be generated from them along the way. Dr. Freking, Dr.
Maddox, and Dr. Rueda provided valuable feedback in the way that I learn best through
analogies, higher level questioning, and philosophical thought. Their belief in the process of
questioning rather than telling forced me to continue the self-guided path toward deeper learning
and led to increased self-reflection. New learning emerged throughout the process that
connected to other aspects of life beyond education. Their continuous patience, positivity, and
open minds were critical to my success and will forever be appreciated.
Words cannot express my sincere gratitude and appreciation for my Educational
Psychology classmates. Together we laughed, discussed ideas, and created some memories that
will last a lifetime. It was a wild ride. There is nobody I would have rather been with along the
way. They were a USC bonus gift. Thank you!
Who knows if I would have even arrived at USC if it hadn’t been for people in the
educational field who supported me. Superintendent Dr. Greg Franklin and my principal, Erin
Kominsky, wrote letters of recommendation that made it all possible in the first place. Erin
listened to my ideas and discoveries and checked on me during each Operation Dissertation
Lockup. With the full support of the President and members of the Board, Superintendent Dr.
STEM + DESIGN THINKING 4
Sherry Kropp, and (then) Assistant Superintendent, Dr. Mark Johnson, gave me the freedom to
learn in far off places and test new ideas within the Los Alamitos Unified School District. Their
belief in me and granting of wide open spaces to explore, learn, design, test, and redesign
parallels the engineering design process which is the thread tying STEM together. Dr. Bill
McDonald came to my rescue when I ran into a barrier to conducting my research. To him and
his staff of volunteers I am eternally grateful. What a fun time we had! To all of my students and
families who believed in me, my ideas, and the philosophy of being lifelong beta testers, I am
sincerely grateful. They will always be such a huge part of my life.
Most of all, heartfelt appreciation goes out to my husband, daughter, and friends who
made great sacrifices along the way to make it all possible. Between work and school, I became
nearly nonexistent but they understood and supported me the entire time. Tremendous thanks to
everyone who always believed in me, supported me, understood my absence, and patiently
waited until I could once again have an additive role in their lives. I am very fortunate and
grateful.

STEM + DESIGN THINKING 5
TABLE OF CONTENTS

DEDICATION 2

ACKNOWLEDGMENTS 3

LIST OF TABLES 9

LIST OF FIGURES 10

ABSTRACT 11

CHAPTER 1: OVERVIEW OF THE STUDY 12
Overview of the Study 12
Purpose of the Study 12
Value of the Study 13
Background of the Problem 14
Call for Action 14
Change Over Time 14
STEM Integration 16
Developing Conceptual Understanding 17
Professional Development Implementation 18
Developing Self-Efficacy 21
Summary 23
Statement of the Problem 23
Purpose of the Study 25
Theories and Conceptual Frameworks 26
Social Cognitive Theory 26
Participatory Design 26
Importance of the Study 27
The National Challenge 27
Foundational Research 28
Efficacy Research 28
Constraint Minimizing Research Design 29
Potential Impact 29
Teachers as the Change Agents 30
Alignment with Previous Research 31
Limitations and Delimitations 31
Self-Efficacy Development: Related Limitations 31
Delimitations Related to the Context 32
Delimitations of Designing Under Constraint 33
Impact Outweighs Limitations 34
Definition of Terms 34
Organization of the Dissertation 36

STEM + DESIGN THINKING 6
CHAPTER 2: REVIEW OF THE LITERATURE 37
Introduction 37
What is STEM? 37
An Emerging Concept 37
Integration in Practice 38
Teacher Self-Efficacy 42
Developing Self-Efficacy 44
Innovation 45
Engineering Design 46
Terminology 46
A Developmental Process 47
Divergent Motives 48
Teacher Professional Development Models 49
The Workshop Model Approach 50
Combining Professional Development Workshops with Ongoing
Training 51
Results of Innovative Professional Development Designs 52
Results of Research on Professional Learning Communities (PLCs) 53
Implications and Conclusions of Professional Development Models 55
STEM Through Participatory Design 56
Participatory Innovation 57
Maker Movement 57
Perspective of this Study 58
Mutual Learning 60
Innovation Ecosystem 60
Design of this Study 61
Theme of Change 62
Conceptual Development and Conceptual Change 62
Multiple Perspectives: Terminology Differences 62
Perspective of this Study 63
Cognitive Conflict 64
Conceptual Change Within the Engineering Design Cycle 64
Synthesis 66
Learning 66
Convergence Model of Innovation 67
Theoretical and Conceptual Frameworks 70
Social Cognitive Theory 70
History 70
Relevance to this Study 71
Perspective of this Study 72
Research on the Theory 72
Organizational Learning Theory 73
History 73
Relevance to this Study 74
Perspective of this Study 75
Research on the Theory 75
STEM + DESIGN THINKING 7
Conceptual Change 76
History 76
Relevance to this Study 76
Perspective of this Study 77
Research on the Theory 79
Conclusion 79

CHAPTER 3: METHODOLOGY 81
Introduction 81
Purpose of the Study 81
Research Questions 82
Methods of the Study 82
Research Design 82
Sample and Population 83
Sampling Procedures 83
School Selection 84
Research Setting 84
Participant Selection 85
Selection Process 86
Instrumentation 87
Rationale 87
Data Collection 89
Surveys 89
Semi-structured Interviews 91
Engineering Notebooks and Document Analysis 92
Theoretical and Conceptual Frameworks 93
Research Question to Instrument Alignment 94
Data Analysis 95
Research Question 1 96
Research Question 2 97
Research Question 3 99

CHAPTER 4: RESULTS 102
Purpose of the Study 102
Logic Model Approach to Evaluation 103
Reporting of Results 106
Research Question 1 106
Research Question 2 118
Research Question 3 125
Summary 126

CHAPTER 5: DISCUSSION OF FINDINGS, CONCLUSIONS, AND
IMPLICATIONS 129
Discussion of Findings and Results 131
Implications for Practice 137
Future Research 138
STEM + DESIGN THINKING 8
Summary 142
REFERENCES 145

APPENDICES
Appendix A: Self-Efficacy for Teaching STEM Survey 158
Appendix B: Teacher Criteria for Participation 162
Appendix C: Interview Protocol 163
Appendix D: Importance of STEM Subjects Survey 165
Appendix E: School Criteria for Participation 167
Appendix F: Attitudes Toward Collaboration Survey 168
Appendix G: Importance of STEM Subjects Survey Data 169
Appendix H: Attitudes Toward Collaboration Survey Data 170
Appendix I: Models Used in STEM Team Training 171
Appendix J: Engineering Notebooks 173
Appendix K: Interview and Engineering Notebook Data 178
Appendix L: STEM Training Organizational Big Ideas and Activities 182
Appendix M: Model of Innovation 184

STEM + DESIGN THINKING 9
LIST OF TABLES
Table 1: Instrumentation Usage 88
Table 2: Research Question Alignment 94
Table 3: Participant Data 103
Table 4: Logic Model 104
Table 5: Self-Efficacy for Teaching STEM Survey: Pre-Post Mean Data 107
Table 6: Sample of Evidence from Interviews and Engineering Notebooks for RQ1 114
Table 7: Sample of Evidence from Interviews and Engineering Notebooks for RQ2 120

STEM + DESIGN THINKING 10
LIST OF FIGURES
Figure 1: Convergence Model of Innovation 68
Figure 2: Question Analysis Graphs: Pre-, Post-, and Change in STEM Ratings on
Scale of 1-6 108

Figure 3: Mean Change Item Analysis 109
Figure 4: Self-Efficacy for Teaching STEM: Item Analysis Per Teacher Participant –
Pre/Post 110
Figure 5: Participant x Content x Research Question Analysis of Mentions in Qualitative
Data 126

STEM + DESIGN THINKING 11
Abstract
The purpose of this study was to research and identify which elements of a collaborative, hands-
on series of STEM training sessions may have related to perceived changes in teachers’ self-
efficacy, classroom lesson design, and conceptual understanding of engineering within STEM at
the elementary level. The study involved eleven teachers of grades 1-5 from a suburban public
school in Southern California who volunteered to participate in the research in May 2014. The
results provide specific activities and processes that led to increased self-efficacy for teaching
STEM, conceptual development for STEM, and conceptual change related to engineering within
STEM at the elementary levels. Prior to this study, a tool to determine changes to teacher self-
efficacy for teaching STEM through the engineering design process was not available. This
research utilized a modified version of existing tools which could be used in further research
investigating teacher self-efficacy for teaching STEM. Specific tools, resources, and activities
that contributed to the growth as identified through the data analysis were also identified. A new
model converging disciplines related to promoting innovative change within organizations also
emerged which could be analyzed in future research for application to STEM integration at the
K-5 level.

STEM + DESIGN THINKING 12
CHAPTER 1
OVERVIEW OF THE STUDY
Purpose of the Study
The purpose of this study was to research and identify the elements of a STEM focused
experiential training series that may have related to perceived changes in teachers’ self-efficacy,
classroom lesson design, and conceptual understanding of engineering within STEM at the
elementary level. For the purposes of this study, STEM (science, technology, engineering, math)
was defined as the combination of two or more of the STEM disciplines with connected
relevance explored through an engineering design approach not merely joining two or more
content areas. The rationale behind the design of the STEM team training was to create a self-
sustained system of ongoing learning within the school itself. The activities designed for the
teacher participants were based on learning theories and grounded in social cognitive theory
within the participatory design framework. While there was a six-part structure and plan based
on learning theories for each training session, the discussions, discoveries, and next steps for the
teachers as learners were unique to the context of the school and driven by the learners
themselves. Through design thinking, the teachers became the learners, problem solvers, and
innovators prepared to solve the real world challenge of designing integrated STEM lessons for
classroom instruction and an aspirational STEM exploratory space within their school. The
teachers learned about the engineering design process by literally applying design thinking, the
engineering design cycle, and a problem solving approach to fill a genuine need at their school.
The elements of the study were designed to prepare the teachers at the school to think like
engineers, develop a concept of integrated STEM instruction, frame their concepts of
engineering at the elementary level, and build their self-efficacy as STEM integrationists.
STEM + DESIGN THINKING 13
Value of the Study
While the value of the study relates to classroom implementation, building teacher self-
efficacy in STEM, and providing foundational concepts related to STEM, it’s primary purpose
was to serve as the first step in the actual participatory design. In a sense, it was the launching
pad. The teachers would be developing their concepts of engineering by becoming engineers,
thinking like engineers, and collaborating across grades and disciplines. The study sought to
investigate factors that could lead the teachers to develop higher levels of self-efficacy and
readiness for full implementation and transfer of the learned concepts. The transfer would take
place as a follow up to the study discussed herein. The ultimate goal was for the teachers to
apply what they learned through the collaborative learning experience to engage all stakeholders
in the participatory design of a STEM space on their school campus. In other words, the training
would scale out to all stakeholders at the school site thereby building a system-wide STEM
learning community for continued practitioner-researcher learning.
The design thinking and engineering cycle learned while engaged in the study could be
implemented as the future STEM space is co-constructed and iterated by the stakeholders
themselves. The process for the expansion is beyond the scope of this study but important to
explain to fully understand the value and contribution made beyond the time frame of this study.
The teachers would apply and extend the knowledge, concepts, and experiences to solve a
genuine challenge-based, problem-based, service learning problem at their school. First, they
would learn how to transfer their learning into their classroom lesson design. The study builds
the foundation for the concepts, experiences, thinking, and collaboration that would be required
to make the STEM space participatory design possible. To change the integration of STEM, it is
necessary to first change the way teachers think about STEM and engineering in particular.
STEM + DESIGN THINKING 14
Background of the Problem
Call for Action.
In order to meet the growing demands of corporations and to ensure economic viability
and competitiveness of the nation within today’s high-tech information age, President Obama
and other leaders have called for an integration of science, technology, engineering, and math
(STEM) into classroom instruction beginning in elementary school
(http://www.whitehouse.gov/issues/education/reform). Research has been emerging in support of
the integration of STEM at the elementary level as well (Brophy, Klein, Portsmore, & Rogers,
2008; Cotabish, Dailey, Robinson, & Hughes, 2013; Drew, 2011; Macalalag & Tirthali, 2010;
Mann, Mann, Strutz, Duncan, & Yoon, 2011; Nadelson, Seifert, Moll, & Coats, 2012). This
necessitates educational change to which Sellars (2012) and Brophy et al. (2008) identified
teachers as the change agents not policies, legislators, curriculum developers, or administrators.
The Iterative Model of Educational Change as proposed in the preliminary report for the
National Academy of Sciences (Honey, Pearson, & Schweingruber, 2014., pg. 150) released in
February of 2014 provided the structure for initiating this change by teachers for teachers. This
study mirrored the iterative model and provides foundational support for its effectiveness at the
school level.
Change Over Time.
Historically, our national public education system has segregated the disciplines of science
and math while technology and engineering have been mostly nonexistent at the K-5 level. The
long-term negative impact of this systemic sorting of disciplines is a current paucity of skilled
workers within the STEM fields (Drew, 2011; Epstein & Miller, 2011). In this digital age of
innovation, jobs no longer fall into discrete categories mirroring school subjects. Instead, the
STEM + DESIGN THINKING 15
emerging careers rely upon knowledge and thinking that spans the disciplines interweaving
concepts to solve the real world problems of today. To fill this need, there is an ever-expanding
call for teachers to better prepare students for their future by integrating the STEM content areas
and providing problem based challenges that mirror real world issues (National Academy of
Sciences, 2014). This requires an engineering design approach to thinking and problem solving
(Honey et al., 2014). The Next Generation Science Standards (NGSS) explicitly integrate
engineering education throughout the K-12 grades and emphasize the importance of concept
development, exploration, and inquiry (Pellegrino, Wilson, Koenig, & Beatty, 2014). Teachers
are now expected to teach engineering concepts within an integrated STEM approach. In order to
teach engineering, the teachers must first learn to think like engineers themselves, explore
alternative ways to solve problems, and iterate based on a comparison of results to expected
outcomes or goals. Those three aspects of developing a concept of engineering were integral to
the innovative design and implementation of this study.
Teaching is contextually dependent. In the past, teachers taught isolated subjects in
isolated classrooms within structures bound by time, location, and information constraints.
Digitalization has changed the learning opportunities available to learners of all ages. While the
systemic educational barriers of structure and power may still reside within the public education
system, opportunities to trigger situational interest have never been greater. The paradox is that
while STEM related resources are becoming more readily available, teachers do not know how
to access, approach, or utilize the resources. More importantly, the pedagogical content
knowledge (Appleton, 2008; Brophy et al., 2008; Buczynski & Hansen, 2009; Koh, Chai, &
Tsai, 2013; Goldschmidt & Phelps, 2010; Makgato, 2012; Mishra & Koehler, 2006; Nadelson et
al., 2012), background experience (Cotabish et al., 2013), and self-efficacy (Annetta, Frazier,
STEM + DESIGN THINKING 16
Folta, Holmes, Lamb & Cheng, 2012; Appleton, 2008; Yoon, Diefes-Duz, & Strobel, 2013) in
the separate STEM fields is low for most elementary school teachers preventing them from being
effective STEM instructors. To change the course of STEM education, there is a need to first
establish a solid, research-based pathway of STEM concept development, integration, and
implementation.
STEM Integration.
Integrated STEM has been most frequently found in modules presented within informal
learning environments or high school classrooms (NAS, 2014). Research (Becker & Park, 2011;
NAS, 2014) has found that the goals of informal learning environments and school-based
programs were divergent. School-based programs were more achievement oriented while
informal learning environments such as after school sessions were more mastery oriented thereby
promoting stronger learning outcomes (NAS, 2014). However, when approached as problem
solving in the real world, it mirrors problem or project-based learning challenges presented
within elementary classrooms. These usually require applying the engineering process to solve a
problem under constraints of materials, time, space, money, and information. Through cycles of
iterative design, students apply math and science knowledge using tools of technology. At the
elementary level, this may be the easiest path to integrative STEM implementation (Dow &
Klemmer, 2009; NAS, 2014; Roehrig, Moore, Wang, & Park, 2012). Perhaps the most relevant
problem-based challenge to be solved hovers around a mutually agreed upon definition of
integrative STEM implementation within the research literature and pathways to implementation
within elementary schools.

STEM + DESIGN THINKING 17
Developing Conceptual Understanding.
How teachers learn the concept of STEM integration is another unanswered question due
to the innovative nature of STEM as a concept itself (Honey et al., 2014). First, teachers need to
conceptualize what STEM means and implies for learners and teachers. Then, they need to
develop a schema for organizing their ideas of the separate subjects into a synthesized view of
overlapping concepts. One of the strongest predictors of concept development or conceptual
change is the level and extent of actual experiences that allow teachers to face misconceptions
and engage in cognitive conflict (Heddy & Sinatra, 2013; Mayer, 2011). The STEM team
training was designed to provide those opportunities embedded within a hands-on experiential
collaboratively designed structure. Many teachers have had exposure and experience with math
and science. However, elementary teachers have had minimal exposure and experience with
what is arguably the most important STEM element – engineering (Brophy et al., 2008;
Carberry, Lee, & Ohland, 2010; Macalalag & Tirthali, 2010; Yoon & Evans, 2012; Yoon et al.,
2013). At the time of the study, there was still no definitive way of identifying what engineering
knowledge and associated skills were or should be at the K-5 level (Honey et al., 2014).
However, there was growing consensus that supported a focus on the engineering design process
(Becker & Park, 2011; Brophy et al., 2008; Macalalag & Tirthali, 2010; Roehrig et al., 2012) that
could be both defined and illustrated to build conceptual understanding. It is from within this
frame that the study was devised.
In addition to the conceptual understanding of both engineering design and integrated
STEM, there was the historical issue of elementary school teachers lacking the background
experience in the separate STEM fields to teach with the depth and complexity expected of them
within the Common Core State Standards (CCSS) and the Next Generation Science Standards
STEM + DESIGN THINKING 18
(NGSS). How could teachers best develop the necessary background experiences within the
separate fields? Was the additive effect enough to constitute STEM integration within
classrooms? How could teachers build conceptual understanding about integrative STEM
instruction and engineering design pedagogy? There was ample evidence within the research on
professional development to address the first question but the second was left unanswered by the
research available at the time of the study. The final question was embedded within the research
questions of the study.
Professional Development Implementation
To date, research has focused on creating professional development or training for teachers
on the STEM disciplines separately as well as emerging STEM training. Professional
development research has centered on teaching the disciplines separately to develop content
knowledge first (Buczynski & Hansen, 2009; Macalalag & Tirthali, 2010; Nadelson, Seifert,
Moll, & Coats, 2012; Yu, Luo, Sun, & Strobel, 2012), addressing self-efficacy issues (Annetta,
Frazier, Folta, Holmes, Lamb & Cheng, 2013; Nadelson, Callahan, Pyke, Hay, Dance, &
Pfiester, 2013), and a combination of training in pedagogy and content knowledge simplified as
PCK (Appleton, 2008; Brophy et al., 2008; Buczynski & Hansen, 2009; Cotabish et al., 2013;
Ejiwale, 2012; Makgato, 2012; Mishra & Koehler, 2006; Sellars, 2012; Shulman, 1986; Yoon et
al., 2013; Yu et al., 2012). Overall, there is a high correlation between the duration, intensity, and
support embedded in the professional development and the student learning outcomes especially
when related to the STEM fields.
Professional development opportunities are strategically organized efforts to facilitate
change in teachers’ beliefs, attitudes, content knowledge, classroom instruction, and ultimately
improve student learning. Many of the studies of interventions reference the earlier work of
STEM + DESIGN THINKING 19
Guskey (1986) in relation to fostering teacher change. Guskey (1986) presented a model of
teacher change in which a sequence of events leads to a change in teachers’ classroom practices
that improve student learning outcomes. As a result of improved student learning outcomes, the
teacher’s beliefs and attitudes are changed and the cycle of continuous growth carries on.
However, many researchers have found contradictions with Guskey’s model of teacher change
through staff development when it relates to innovation. In contrast, research conducted after
Guskey (1986) has overwhelmingly supported exactly the opposite direction in procuring teacher
change suggesting that improved student learning via professional development (PD) actually
begins with changes in teachers’ perceptions, beliefs, and self-efficacy (Makgato, 2012;
Nadelson et al., 2012; Sargianis, Yang, & Cunningham, 2012; Sellars, 2012; Yoon et al., 2013).
Perhaps the incongruence between Guskey’s (1986) model of teacher change and STEM models
of teacher change (Buczynski & Hansen, 2009; Goldschmidt & Phelps, 2009; Yoon et al., 2013)
is related to the fact that nearly 30 years has past and STEM education was not a concept of
education at the time of Guskey’s original research. Since then, teaching STEM beginning in
elementary school has emerged as a national priority in education. While attempting to state
support for his model, Guskey was actually prophetic of STEM professional development (PD)
when he stated, “Professional development programs based on the assumption that change in
attitudes and beliefs comes first are typically designed to gain acceptance, commitment, and
enthusiasm for teachers and school administrators before the implementation of new practices
and strategies.” (pg. 383). Indeed, that is the goal of the innovative practice of teaching STEM
across the disciplines in elementary school. There is accumulating support obtained from the
research of the past five years (Buczynski & Hansen, 2009; Yoon et al., 2013; Yu et al., 2012)
suggesting that STEM integration at the elementary level is so innovative that changing self-
STEM + DESIGN THINKING 20
efficacy must take precedence to changing practice. That contradicts Guskey’s (1986) assertion
that “evidence of improvement . . . precedes . . . significant change in attitudes and beliefs of
most teachers” (pg. 384) but supports the changing implementation of STEM PD in education
today. Before teachers can teach STEM successfully to achieve improved student learning
outcomes, they must develop high self-efficacy (Annetta et al., 2013; Buczynski & Hansen,
2009; Cotabish et al., 2013; Hutchison, 2012; Mushayikwa, E., 2008; Nadelson et al., 2013;
Nadelson et al., 2012; Yoon et al., 2013; Yu et al., 2012).
This is particularly relevant and pertinent in the STEM fields since aggregated research has
found that most teachers have low self-efficacy, little content knowledge (CK), and questionable
pedagogical knowledge of how to teach STEM particularly in elementary classrooms (Cotabish
et al., 2013; Goldschmidt & Phelps, 2009; Makgato, 2012; Yoon et al., 2013; Yu et al., 2012). If
teachers do not have the pedagogical content knowledge (PCK) to teach STEM and have
corresponding low self-efficacy rates, then they will not be able to change their teaching practice
without first developing the content knowledge that helps them integrate the disciplines
efficiently and effectively for students.
Building upon professional development research for elementary educators, Buczynski
and Hansen (2009), Darling-Hammond (1995), Mishra and Koehler (2006), and Makgato (2012),
concur that the traditional drive-by, one-day workshop model is insufficient for improving
student learning or deep understanding by teachers. For that reason, the training involved a
series of five half day once a week sessions. Mishra and Koehler (2006) discovered that context-
neutral staff development (ignoring the school site, classrooms, and students themselves) leads to
generic solutions that will fail since they overemphasize content but often ignore pedagogy
(basic CK only). Instead, scaffolded experiences that consider the context in which the teacher
STEM + DESIGN THINKING 21
will be teaching are most effective (Sargianis et al., 2012; Yoon et al., 2013). This study added
the participatory design and exploratory approach to maximize the research and context. Specific
to STEM PD, researchers (Buczynski and Hansen, 2009; Annetta et al., 2013; Makgato, 2012;
Sargianis et at., 2012; Yoon et al., 2013; and Yu et al., 2012) derived the same conclusion that
professional development opportunities designed to begin with developing pedagogical content
knowledge to build self-efficacy should be the primary goal of PD in elementary education. For
that reason, the first research question of this study investigated the perceived changes related to
the training on teacher self-efficacy for teaching STEM at the elementary level.
Developing Self-efficacy.
In order for teachers to reach the goals set forth by the executive report presented to
President Obama, Prepare and Inspire: K-12 Education in Science, Technology, Engineering,
and Math (STEM) Education for America’s Future (President’s Council of Advisors on Science
and Technology, 2010), the Next Generation Science Standards
(http://www.nextgenscience.org), and the National Academy of Sciences (2014), teachers must
have positive self-efficacious feelings and attitudes about their ability to teach STEM in an
integrated fashion. One hypothesis in this study was that teachers who felt self-efficacious about
teaching STEM would be more likely to try elements of the training in their own classrooms.
Therefore, the history of research related to self-efficacy is essential.
Efficacy comes in variant forms. Two forms are relevant to this study and have been
researched across the disciplines: teacher self-efficacy and collective efficacy. Teacher self-
efficacy itself can be segmented into two realms: efficacy for teaching and professional efficacy.
Efficacy for teaching involves the self-determined, future-oriented, contextually dependent belief
that one has the ability to teach. Professional efficacy relates to a teacher’s beliefs in the ability
STEM + DESIGN THINKING 22
to work with the professional community. One is not dependent upon the other. A teacher can
feel highly efficacious within the classroom teaching math (teaching), have low self-efficacy for
teaching science (teaching), and have low self-efficacy in his or her ability to collaborate with
members on the grade level team (professional). This study focused on both aspects of teacher
self-efficacy. Self-efficacy for teaching STEM was addressed by the first research question:
What factors influence teacher self-efficacy in teaching STEM at the elementary level?
Professional self-efficacy was addressed within the format of the STEM training. Collective
efficacy involves the belief of a group of people to be able to join together to meet a goal.
Collective efficacy is embedded within the training that was designed as the foundation for the
later construction of a school wide STEM exploration space. The collaborative planning, design,
and iterations of the STEM learning space would require a high level of collective efficacy
across the stakeholders in the community.
The problem specific to STEM is that a vast amount of research provides evidence that
elementary teachers have reported low self-efficacy in each of the separate STEM fields
(Appleton, 2008; Buczynski & Hansen, 2010; Cunningham, 2004; Hechter, 2011; Nadelson et
al., 2012; National Academy of Sciences Report, 2014; Yoon, Pedretti, Bencze, Hewitt, Perris, &
Oostveen, 2006; Yoon, Evans, & Strobel, 2012). This has a potentially compounded effect when
integration is considered. Teachers will not be as successful in their implementation unless they
feel efficacious (Honey et al., 2014). This in turn could create a negative spiral of declining
success due to the reciprocal relationship between self-efficacy and quality of the
implementation. Researchers agree that the impact of self-efficacy on teaching and student
learning is significant (Chan, 2008; Chang, 2009; Cheung, 2008; Gökçek, Günes, & Gençtürk,
2013; Nie, Tan, Liau, Lau, & Chua, 2013; Schechter & Tschannen-Moran, 2006). Therefore,
STEM + DESIGN THINKING 23
efforts to increase teacher self-efficacy are paramount to building successful experiences within
classrooms and were embedded within the activities and design of this study.
Summary
This study was built upon the research of self-efficacy, concept development,
participatory design, and STEM integration to address many of the research needs and
recommendations cited in the report from the National Academy of Sciences (2014). It was
seeking to fill the many gaps of knowledge related to K-5 STEM integration through an
innovative, participatory approach that parallels engineering design thinking grounded within
social cognitive learning theory. The creative design, methodology, and approach to devising the
study and training elements embedded within have the potential for making a positive impact in
the fields of motivation, learning, design thinking, creativity and innovation, teacher education,
professional development, and STEM itself. The potential value of the long term effects of this
foundational study will be examined in a follow up study when the actual construction of the
collaboratively designed and created STEM space begins, is explored, and encounters iterations.
Statement of the Problem
As stated in the National Academy of Sciences report (2014), innovation requires
creativity and experimentation. Little was known about STEM integration, the long-term effects,
and the ways in which it could or should be implemented (Honey et al., 2014). How to motivate
teachers to explore, try, and learn STEM concepts was also unknown. What was known was that
teachers with higher self-efficacy are more willing to take risks, persevere longer when faced
with challenges, set higher goals for themselves, and have more successful outcomes (Bandura,
2006; Gibson & Dembo, 1984; Guskey, 1988; Nie et al., 2013; Rittmayer & Beier, 2009;
Vardaman, Amis, Dyson, Wright, Van de Graaff, & Randolph, 2012; Zimmerman, 2000). Like
STEM + DESIGN THINKING 24
self-efficacy, learning evolves, ebbs, and flows depending on the contextual elements. Building
from the known to the unknown is critical to innovative research design. What is known is that if
we want teachers to feel more willing to explore and experiment within the realm of STEM, then
research on how to build teacher self-efficacy is vital.
Every aspect of STEM within elementary classrooms is wrought with problems. There
are no clear definitions of terms, concepts, or models of how successful STEM integration
should look (Honey et al., 2014). Research is just emerging, empirically based correlational data
is sparse, and testing and measurement tools specific to STEM have yet to become available
(Honey et al., 2014). These issues pose elevated challenges to researchers as they must
individually define their terms. Synthesizing emerging research beyond common
recommendations will be difficult until the experts in STEM establish clear guidelines,
terminology, and validated tools for researchers.
The multidimensionality of STEM provokes many problems. If the factors that contribute
to increasing teacher self-efficacy in teaching STEM can be identified, then successful
professional development opportunities can be created for existing teachers undergoing the
paradigm shift from teaching content knowledge separately to designing learning opportunities
emphasizing the engineering design cycle within STEM. Progress in the self-efficacy of
teaching across the disciplines is needed in order for cross-pollination to positively affect student
learning. If the specific elements, experiences, or activities explored via an innovative,
participatory design approach to a STEM training series emphasizing design thinking that would
lead to increased use of STEM in lesson design could be identified, then specific
recommendations for the design and implementation of future training that would provide long
STEM + DESIGN THINKING 25
term site based collaboration and continued learning could be offered. Those were the goals as
built into the research questions driving the study.
Purpose of the Study
The purpose of this study was to research and identify the elements of a STEM focused
experiential training series that may have related to perceived changes in teachers’ self-efficacy,
classroom lesson design, and conceptual understanding of engineering within STEM at the
elementary level. On a larger scale of impact potential, the overarching goal of the study was to
apply innovative design thinking and the engineering cycle to solve a real world need to discover
effective methods for helping elementary teachers integrate STEM through engineering design
within their classrooms. This study gathered needed research to establish paths that could help
teachers learn the foundational concepts and thinking approaches to successfully implement
integrated STEM experiences within their classrooms. The teachers became learners,
participants, co-constructors, and designers as part of the participatory design approach within
the framework of social cognitive theory. By participating in the creation and iteration of their
own ideal STEM space, they thought like engineers to help develop concepts and self-efficacy
for teaching engineering design within their classrooms. In essence, this study was in itself
engineering design as it was designed under constraints drawing on scientific and mathematical
knowledge in search of solutions to a relevant, real world problem within education today.
An integral part of this study was exploring specific ways of building both self-efficacy
in teaching engineering and experiences that would lead to developing elementary teachers’
concepts of engineering. The specific research questions being investigated included:
1. What factors influence teacher self-efficacy in teaching STEM at the elementary level?
2. What elements, experiences, or activities within a STEM focused training transfer to
STEM + DESIGN THINKING 26
lesson design?
3. What experiences or activities contribute to teachers’ development of the concept of
engineering within STEM?
Theories and Conceptual Frameworks
Social Cognitive Theory
While constructivism is the basis for learning by doing and encompassed at least half of
every training session, this research study was predominately based upon Bandura’s work (1977)
with the Social Cognitive Theory. The difference could be found within the framework of
participatory design – the social factor. Social Cognitive Theory is the theoretical framework on
which this study was constructed. It is a theory of learning and cognition based on the initial
work of Albert Bandura (1977) within the field of psychology that explains how learning is
achieved through the interactions and interrelationships among personal, behavioral, and
environmental factors. The National Academy of Sciences report (2014) supported the design
and emphasis on Social Cognitive Theory as it identified the social psychological processes as
key design features in integrated STEM learning environments (pg. 87). While the message was
intended for classroom instructional design, the researcher applied it to teachers participating in
this study as they were likely to be novice learners within the STEM fields. The study was
designed to incorporate the following key features of the theoretical framework: observation,
active engagement, peer collaboration, building self-efficacy, environments most conducive to
learning, social aspects of learning, and goal setting.
Participatory Design
Through the lens of the participatory design approach to learning, teachers learn best
when given the task of co-constructing their own learning thus also incorporating constructivist
STEM + DESIGN THINKING 27
learning theories. Ownership of the learning is a key construct and involves allowing each
person’s voice to contribute to the group learning goals. Participatory design is designing under
constraint that essentially operationalizes the engineering process. It exemplifies reflection in
action. By engaging in this process themselves, it is hypothesized that the teachers would better
conceptualize what the engineering design process as integrated through STEM disciplines really
involves. Through experiential learning and collaboration, the teachers would co-construct an
aspirational ideal STEM space. This process mirrored what the teachers would need to do to their
lesson design in order to integrate STEM effectively into instruction.
Participatory design is “a process driven by social interaction and engagement” and
involves co-designing the “artifacts, processes, and environments that shape their lives”
(Robertson & Simonsen, 2014). It short, it involves the process of mutual learning and shared
concept construction and parallels social learning theories. The process involves increasing
participants’ feelings of personal agency that has been found to increase self-efficacy (Bandura,
1982). From the research compiled by Robertson & Simonsen (2014), one can infer that key
tenets of participatory design include: design accountability (pg. 5) of what is created and to
those who will use it, user-participant design (pg. 5), and active engagement.
Importance of the Study
The National Challenge
In a 2014 report intended for educational researchers, the National Academy of Sciences
(NAS) proposed a consolidated series of challenges related to the nationwide challenge posed by
the President’s Council of Advisors on Science and Technology (2010) of providing STEM
education for all students beginning in kindergarten. The challenges included recommendations
for research, design, and implementation of studies related to STEM education. The report
STEM + DESIGN THINKING 28
emphasized the importance of engineering design being interwoven among science, math, and
technology. Self-efficacy, motivation, and interest related to STEM fields were also addressed
and emphasized due to the current discrepancy between the number of needed and the actual
number of workers within the STEM fields today (Honey et al., 2014). All of these
recommendations were embedded within the research questions to be investigated in this study.
The report boldly advanced a research agenda targeted at innovative researchers willing
to take risks and try new methodologies, approaches, and designs within their studies. This study
met those challenges then extended them by layering creative techniques within creative
activities and approaches not addressed within the research literature available at the time of the
study. This in turn created a complex design that combined three of the six research design
challenges set forth by the research agenda: foundational research, efficacy research, and scale-
up research (pgs. 137-138).
Foundational Research
As foundational research, this study set out to test, develop, and iterate innovative
methodologies, approaches, activities, and design features hypothesized to factor into increased
teacher learning and promote connections across disciplines and contexts. At the cognitive level,
the study was seeking to build pedagogical content knowledge, develop concepts, deconstruct
misconceptions for reconstruction into new mental models, and promote metacognitive thought.
On a tangible level, the study provided the foundation for ongoing collaborative work to design
and construct the real STEM learning space to be used by all stakeholders.
Efficacy Research
As efficacy research, this study was built upon the original goal of the researcher – to
build a STEM space within a school based on empirical research and collaboratively constructed
STEM + DESIGN THINKING 29
through the lens of engineering design. Anything aspirational requires high self-efficacy in order
to maintain the necessary motivation to persevere when faced with challenges and constraints.
The training was designed to promote that thinking, confront and overcome constraints, build
collective efficacy within the school, promote collaborative discussion, and explore the iterative
design process with purpose and relevance. The participatory design approach appeared to be a
solid match for efficacy research as defined by the National Academy of Sciences (2014, pg.
140).
Constraint Minimizing Research Design
To advance the collective understanding of STEM integration, the National Academy of
Sciences report on STEM integration (Pellegrino et al., 2014) suggested removing constraints to
allow the necessary creativity and experimentation within research design that would be needed
to solve the many problems involved with the innovative nature of STEM integration. By not
recommending any specific approaches or strategies, the researchers on the committee reinforced
the need for unique and untested approaches to increasing STEM integration in K-12 classrooms.
This research study carved a previously unexplored and thus untested path to possible STEM
integration via the participatory design approach guided by social cognitive theory and designed
under constraint. The contributions to the fields of education and STEM integration involve the
introduction of novel methods of increasing both cognitive (pedagogical content knowledge,
concepts, metacognition) and affective (self-efficacy, interest) outcomes (Kumar, 2013).
Potential Impact
This study has the potential of becoming a foundational study within STEM efforts to
help teachers integrate STEM into their lesson design. The 2014 report by the National Academy
STEM + DESIGN THINKING 30
of Sciences would describe the training in this study as maximized complexity, since it was
ambitiously seeking to design an integrated deep learning experience for and with the teachers of
a school (pg. 43).
While most descriptions of importance of studies center around outcomes, this study by
design added another layer of importance. The report by the National Academy of Sciences
(2014) discussed how research studies call upon a range of theoretical frameworks with one
commonality: the social element. In fact, the committee concluded that based on years of
research on cognition and learning, STEM instruction should involve active learning and should
be “deeply social” (pg. 78). This research study extended that one step further to propose the idea
that with innovations involving both content and pedagogical content knowledge shifts within
education, theoretical convergence applies. A theory should draw upon the plethora of learning
and cognition research. Social cognitive theory was the driving force throughout the study. The
design was innovative to parallel the level of innovation being studied incorporating
participatory design sometimes referred to as participatory innovation.
Teachers as the Change Agents
In focusing on teacher outcomes first, the overall goal as with any learning was transfer.
This study investigated specific ways of enhancing transfer from a specific location into a
classroom thereby changing the context, resources, and delivery of the learning. This aided in the
teachers’ development of representational fluency as they took what was learned and applied it to
lesson design within their classrooms with emerging automaticity. Evidence of ways to build
pedagogical content knowledge then connecting that new learning to background knowledge in
intellectually efficient ways has the potential to move the STEM integration conversations within
research teams toward a focus on transfer across contexts.
STEM + DESIGN THINKING 31
Alignment with Previous Research
The final recommendation by the NAS report in February of 2014 exemplified the last
but possibly most impactful element of this study. It ended with a call for action to researchers
stating that designers and those implementing integrated STEM initiatives should “explicitly
ground their efforts in an iterative model of educational improvement”. One could infer that this
also prompts an iterative model of self-efficacy. The diagram on page 150 of the NAS report
(2014) shows a flow map starting with goals and objectives leading to the design of the
instruction that is then implemented and studied through data analysis. The results analyzed lead
to the stage within the flow to revise and the cycle continues with movement within and among
the steps in a recursive manner thus generating continuous, incremental growth and educational
improvement. This study mirrored that iterative model of educational change answering the call
to action and providing foundational research on which future research can be expanded and
revised. Thus, the engineering design process and iterative model of change can carry on from
this study forward just as the STEM team training will evolve into a school wide STEM space
for exploration, experimentation, iteration, and innovation.
Limitations and Delimitations

Self-Efficacy Development: Related Limitations

Self-efficacy is specific to each individual, affected by context, domain specific, and task

specific. Tools for measuring self-efficacy are numerous, but at the time of the study there were

no tools specific to STEM. That posed two limitations. First, self-efficacy was measured via

self-report scales that can have questionable validity as teachers have been found to both over

and underestimate their ratings on self-efficacy scales for a variety of reasons (Bandura, 2006).

STEM + DESIGN THINKING 32
Since there was no other instrument available to gather self-efficacy data and none available
specifically for STEM measurements, one needed to be constructed synthesizing and
extrapolating elements from the most widely used domain specific self-rating scales available.
Although found to have high content validity, the scale has been limited in use to the population
studied. This is consistent with Bandura’s recommendation for measures to be constructed for
each context. Further research using the new Self-Efficacy for Teaching STEM instrument (see
Appendix A) needs to be conducted to establish reliability beyond the participants in this study.
Delimitations Related to the Context
Delimitations of the study hover around the contextual factors embedded within the
study. Participatory design is always context bound. The social cognitive approach emphasizes
the social aspect of the study. Therefore, context was layered throughout the study. While the
context shifted from a collaborative space to a classroom, it was highly dependent upon the
participants themselves. Since it was action research involving training, the results are bound by
context. However, all instrumentation, methods, and resources could be used in any context.
Although the collaboration would differ, the processes, approaches, and methodology used to
craft this design still lends itself to beneficial application across similar K-5 elementary schools
who show interest in STEM integrated teaching. Evidence supporting this assertion is currently
being analyzed since the same researcher repeated the process at a similar school three months
later.
Another delimitation related to the selection of the teacher participants. The researcher
chose to include specific criteria. Fortunately, all teachers who volunteered did meet the criteria,
so no teachers had to be excluded from the study. Due to the school budgetary constraints and
the number of substitutes allowed at the school site on a single day, only half of the teachers
STEM + DESIGN THINKING 33
could participate. To ensure that the findings would not be affected, the researcher left all
materials for the teachers to share with the entire staff making the experience as inclusive as
possible across the school setting.
It is possible that the researcher’s personality may have been an additional context related
delimitation. The high level of optimism and positivity that is shared while facilitating
professional development sessions may have affected the perceived changes in self-efficacy for
the teachers participating in the study. Since key goals were to inspire, motivate, and spark
interest in teachers throughout the session, there was no effort to hold back the high level of
enthusiasm.
While originally titled a STEM PLC, this was changed prior to commencing the study with
the teachers due to the overuse of the term PLC in education that has led to immense variation of
the construct. To eliminate any previous misconceptions impacting the study, the term “design
team” was used instead when referring to the group as a whole.
Designing within the context itself was a constraint. Being context bound, the findings are
not generalizable beyond the school site. To account for this and maximize testing and
replication opportunities for further researchers to investigate the effectiveness of similar
instrumentation, resources, and activities, the researcher was careful to include clear, specific
details and examples throughout the study. Efforts to use open source, freely accessible materials
and include details of the study with specificity should maximize the reliability of the findings
and make the study inclusive within a broad range of school contexts.
Delimitations of Designing Under Constraint
Design under constraint is a key factor of every aspect of this study. This imposes
limitations based on the constraints within the context. The research design and iterations were
STEM + DESIGN THINKING 34
subjected to time, location, and cost constraints as with any hands-on learning experiences.
Efforts to minimize the design delimitations were made by conducting the research within the
schools and classrooms of the teachers themselves, providing teacher choice within the context
of the school, and including as many free or low cost alternatives as possible.
Impact Outweighs Limitations
Due to the scarcity of empirical research related to STEM integration, any study
involving innovation adds challenge resulting in the potential for unforeseen limitations and
delimitations. The intersections of different theoretical and conceptual frameworks to creatively
design methodology that has been previously untested and nonexistent posed challenges to the
generalizability of the study. Synthesizing successful methods of increasing teacher pedagogical
content knowledge then applying it to the emerging multidisciplinary field of STEM may have
uncovered the stated delimitations. Despite these issues, the potential impact for successfully
facilitating teacher learning and transfer, concept development, and increasing teacher self-
efficacy far outweighed any limitations or delimitations posed by the study.
Definition of Terms
To minimize the chance of leaving terms open to interpretation and align to research
recommendations (Murphy & Alexander, 2000; Honey et al., 2014), key language used in this
study is defined below.
Concepts – established, accepted meanings (Pugh, 2011, pg. 110)
Design Thinking – posing questions and exploring constraints in creative ways that proceed in
entirely new directions; often involves collaboration and experimentalism (Van Wulfen, 2014)
Engineering – applied or practical aspect of several processes used in devising a system,
component, or protocol to meet an identified need (Carberry et al., 2010, pg. 71); iterative
STEM + DESIGN THINKING 35
design and the optimization of materials and technologies to meet needs as defined by criteria
under given constraints (Carr, Bennett, & Strobel, 2012, pg. 9)
Engineers use – systematic processes, mathematical tools, and scientific knowledge to develop,
model, analyze, and improve solutions to problems (Carr, Bennett, & Strobel, 2012, pg. 9)
Engineering design – processes that are dynamic and include phases of problem definition,
problem solving, testing, and iteration (Carr, Bennett, & Strobel, 2012, pg. 9)
Learning - a change in what the learner knows caused by the learner’s experience (Mayer, 2011,
pg. 14)
Mathematics – study of patterns and relationships (Honey et al., 2014)
Participatory design – co-construction of the details embedded within the activity achieved
through collaboration and active participation
Participation – to investigate, reflect upon, understand, establish, develop, and support mutual
learning processes as they unfold between participants in collective “reflection in action” during
the design process (Robertson & Simonsen, 2012)
Participatory innovation – the participatory design framework as applied to an innovation
Professional Learning Community (PLC) – educators collaboratively working together in an
ongoing process of learning with the end goal of helping students achieve better results
Representational Fluency – the automaticity involved in making connections between
representations across the disciplines; the ability to make connections by transposing within and
translating between representations across the disciplines; a measure of understanding of
engineering concepts (Johri, 2011)
Scale-up – process of moving from innovation to broad-based adoption as a result of reshaping
and adaptation from the original design; 5 dimensions of scale (Dede, 2007): depth,
STEM + DESIGN THINKING 36
sustainability, spread, shift, and evolution
Science –the body of knowledge about the natural world as investigated through the process of
inquiry to uncover new knowledge (Honey et al., 2014)
Self-efficacy – a person’s judgment of the capability to organize and execute courses of action
required to attain designated types of performance (Bandura, 1986, pg. 391)
Social Cognitive Theory- theory of learning and cognition based on the initial work within the
field of psychology by Albert Bandura (1977); the learning achieved through the interactions and
interrelationships among the person, the behaviors, and the environment; cognitive + affective +
biological events all factor into the learning process
STEM- combination of two or more disciplines with connected relevance explored through an
engineering design approach not merely joining two or more content areas (Honey et al., 2014)
Technology – tools used to solve problems (Honey et al., 2014)
Transformative experience – when a person actively uses curricular concepts in everyday life to
see and experience the world in a new, meaningful way (Pugh, 2011, pg. 107)
Organization of the Dissertation
This dissertation is organized into five chapters. Chapter two provides a synthesis of the
literature related to STEM and STEM integration, teacher self-efficacy, engineering design,
teacher professional development, and participatory design. Chapter three explains the
methodology used in the study. The fourth chapter reports the results and includes the analysis of
the data and original insights. The final chapter includes a discussion of the findings,
implications for practice, recommendations for future research, and conclusions.
STEM + DESIGN THINKING 37
CHAPTER 2
REVIEW OF THE LITERATURE
Introduction
What exactly is STEM and what was the status of STEM in education at the time of the
study? The answer to both questions parallels the engineering design cycle itself as the
terminology and status are still evolving. Just as there are numerous models of the engineering
design cycle, there are also numerous definitions and concepts related to STEM. The importance
lies in the clarification related to the expected implementation more than within the definitions
themselves. It is through this lens that the research was synthesized.
The crafting of this study was based on the synthesis of available literature across the
disciplines of the cognitive sciences, psychology, sociology, philosophy, science, technology,
engineering, mathematics, educational leadership, educational psychology, and organizational
learning. It is from this synthesized research that the following elements originated: status of
STEM, STEM integration, teacher self-efficacy, engineering design and design thinking, teacher
professional development, and innovative participatory design.
What is STEM?
An Emerging Concept
Due to the innovative nature and emerging emphasis of STEM integration within K-12
classrooms, perhaps beginning with what STEM is not as highlighted by the most recent report
from the National Academy of Sciences (2014) makes the most sense. STEM is not reserved for
afterschool programs or special academies nor is it constrained to formal or informal programs
(pg. 26). STEM in education is not simply teaching science, technology, engineering, and math.
STEM in educational settings is not equivalent or isolated to robotics nor is robotics necessarily
STEM + DESIGN THINKING 38
STEM. Then what exactly is STEM integration within K-12 classrooms? The committee of
experts from across the disciplines who collaborated on the document which was most influential
and inferentially supportive of this study took bold steps toward the advancement of conceptual
understanding of STEM (Honey et al., 2014). The committee highlighted the impact of
integrated approaches requiring the intersection and overlapping of the disciplines on motivation,
interest, and achievement (pg. vii). Simplified, STEM is the combination of at least two of the
following disciplines: science, technology, engineering, and math. In this study, it was
engineering design that wove science together with math and/or technology. The trend in
research is to identify STEM as the integration of the content areas as taught through the
engineering design approach (Becker & Park, 2011).
The STEM disciplines can be defined individually, as a set of interactions, or as an
integrated pedagogical approach. Math is the study of patterns and relationships, many of which
are used in science, technology, and engineering. Science is the study of the natural world
including both knowledge and the inquiry process. Scientific investigations lead to new
knowledge which combines with math to inform engineering. Engineering is the application of
science, math, and technology to design human-made solutions to problems while under
constraints such as materials, time, space, money, or information which spirals back to science
and math. Technology consists of the artifacts used to help solve problems. The integrative
approach frame of mind views the STEM disciplines as interconnected and intertwined.
Integration in Practice
The definition of STEM integration seems to be an ongoing educational challenge
(Nadelson et al., 2013). The teachers, leaders, researchers, students, and parents are receiving
information from various sources of media attempting to define STEM integration. Few
STEM + DESIGN THINKING 39
classrooms are available as exemplary models (Nadelson et al., 2013). Even more rare is STEM
integration in the elementary classroom which is most often led by a teacher who has a multiple
subject teaching credential rather than a credential specializing in one of the STEM fields
(Nadelson et al., 2013).
First, consider the development of the concept of STEM as an integrated approach to
teaching. Embedded within this construct is the expectation that teachers have the content
knowledge within each discipline, understand the definition of integration, and have a successful
pedagogical approach to how they teach. Each aspect of this first step is problematic from a
historical point of view. Blank (2012) observed a disturbing trend of decreasing amounts of
science instruction in classrooms. Technology in the hands of students in classrooms is another
trend as a direct result of innovative global development (Arnone, Small, Chauncey, &
McKenna, 2011; Baum & Walter, 2011; Li & Pow, 2011; Liu, Olmanson, & Toprac, 2011;
Norris & Soloway, 2011; Spires, Lee, & Turner, 2008). The emerging trend of including
engineering design within K-5 classrooms is so new that few studies of effective models have
been found (Becker & Park, 2011). To the contrary, most elementary teachers lack the content
knowledge, conceptual understanding, and efficacy for teaching STEM (Cotabish et al., 2013;
Goldschmidt & Phelps, 2009; Makgato, 2012; Yoon et al., 2013; Yu et al., 2012) or engineering
(Carberry et al., 2010; Yoon & Evans, 2012). Math is the one subject within STEM that is
consistent (Becker & Park, 2011; Kim & Chang, 2009). Indeed, research still supports the
instruction of math alone even when STEM integration is a goal (Becker & Park, 2011; Honey et
al., 2014). Yet integrative approaches to teaching STEM have been the emerging emphasis of
research (Becker & Park, 2011; Honey et al., 2014). This initiates a concern and revolving
problem within emerging studies of STEM that agree there has yet to be a definitive explanation
STEM + DESIGN THINKING 40
of “integration” (Honey et al., 2014). Without a clear definition of terms, validity and reliability
of research studies could be suspect (Salkind, 2013). Teachers cannot understand how to
integrate instruction until the terms can be clearly explained and modeled.
Even within the research, the criteria for the identification of integration varies widely. A
meta-analysis conducted by Becker & Park (2011) coded STEM integration by simply
combining content areas to identify seven forms of integration within the synthesized research.
For example, E-M-S represented the integration of engineering, math, and science. However,
there was no indication of whether these subjects were equivalently emphasized, if one subject
area was the catalyst for others, or if one subject led into the next. Using a different approach to
define integration, Hurley (2001) identified integration through categorization within five levels:
sequenced, parallel, partial, enhanced, and total.
Applying this segmentation to STEM, sequenced integration would involve two or more
STEM disciplines planned and taught back to back. Parallel STEM integration would involve
teachers choosing parallel concepts in math and science to plan and teach. Partial integration
involves part of each discipline being taught together and part separately. The fourth level of
integration, enhanced, involves a primary and secondary focus within the same lesson yet
simultaneously. One subject area would be prioritized over the other during a lesson. Finally,
total integration would involve two or more STEM disciplines taught with equal importance at
the same time. For the purpose of this study, the researcher proposes that the interactions among
the disciplines are important to identify since learning falls along a continuum not within discrete
subjects. The disciplines cannot simply be combined without attached qualitative value to the
content.
STEM + DESIGN THINKING 41
Since there was no established expectation in the STEM literature (Honey et al., 2014), this
study initially planned to maximize the value of Hurley’s (2001) meta-analysis by categorizing
the data into one of the five categories described knowing that outcomes approaching total
integration would be most desirable within the context of the study while the middle level,
partial, could arguably be the goal in many K-5 classrooms depending on the level of the learners
(Becker & Park, 2011). However, that would have limited the specificity of the terminology and
would not have had an additive effect within STEM research. The levels are also general to any
integration rather than STEM specific due to the identification prior (2001) to STEM as an
integrative practice. This called for reflective analysis and consideration in light of the innovative
nature of the study within a contextually dependent learning space – the classroom.
The researcher viewed the qualitatively different ways of defining integration within the
literature as a possible contributing factor to the lack of empirical research on STEM integration
(Becker & Park, 2011), so a systematic and analytical tool for defining integration was designed
to build upon all available research. It was important to consider how a lesson changes over time
within itself and may shift among levels and content delivery within a complex lesson design.
The researcher was seeking to combine the coding approach of Becker and Park (2011) with the
five levels of integration identified by Hurley (2001): sequenced, parallel, partial, enhanced, and
total. In each of the five week sessions, a higher level of integration was explored, discussed,
analyzed and given time for individual reflection in terms of how it could transfer into lesson
design. Shifts from one level to another were the subject of the analysis. This study focused on
progress and incremental growth affected by context rather than achieving the highest level of
integration in and of itself. It took a mastery approach to learning and set forth to provide
experiences aligned to mastery learning.
STEM + DESIGN THINKING 42
Teacher Self-efficacy
Since the seminal work on self-efficacy by Albert Bandura in 1977, researchers around
the world have consistently validated the impact of self-efficacy on teaching and student learning
(Chan, 2008; Chang, 2009; Cheung, 2008; Gokcek et al., 2013; Nie et al., 2013; Schechter &
Tschannen-Moran, 2006). Self-efficacy is a person’s perception of the ability to perform a
specific task in a particular domain within a specific context in the future (Bandura, 1977). It is
based upon one’s beliefs reflecting a judgment of the ability to attain a specific goal within a
specific context. These judgments indicate a person’s confidence in the ability to reach
performance expectations that are goal-oriented and task specific. It involves intention, goal-
setting, and self-regulation while performing the task. The importance of goals is key to
conceptualizing self-efficacy. People with higher levels of self-efficacy related to a specific task
will choose more challenging goals, persist when faced with barriers or obstacles, and will
persevere for longer periods of time (Gibson & Dembo, 1984; Guskey, 1988). This level of
motivation is a key element of self-efficacy as the more a person achieves the goals, the higher
the level of self-efficacy reported.
Science, technology, engineering, and math (STEM) integration within elementary
classrooms is both innovative and complex (Sikma & Osborne, 2014), so researchers are just
beginning to focus their attention on how self-efficacy impacts STEM integration within
classrooms. As many teachers lack academic background experiences or pedagogical content
knowledge within specific STEM fields (Appleton, 2008; Buczynski & Hansen, 2010;
Cunningham, 2004; Hechter, 2011; Nadelson et al., 2012; National Academy of Sciences Report,
2014; Yoon et al., 2012: Yoon et al., 2006), it would be postulated that they would have low self-
efficacy in teaching integrated STEM content within their classrooms. This is problematic in
STEM + DESIGN THINKING 43
light of the call from companies, researchers, and government leaders to teach STEM beginning
in kindergarten (Cotabish et al., 2013; Drew, 2011; Nadelson et al., 2012; National Academy of
Sciences, 2014). How to best help elementary teachers shift pedagogical approaches from direct
instruction to more constructivist approaches is one challenge (Nie et al., 2013; Sikma &
Osborne, 2014). How to help them develop the content knowledge in science, technology,
engineering, and math to successfully become STEM integrationists is yet another. The first
research question (RQ1) was seeking to determine the level of teachers’ self-efficacy toward
teaching STEM and methods of increasing their self-efficacy toward integrated content delivery
grounded in constructivism as recommended by the National Academy of Sciences (2014).
The reciprocal relationship between increased levels of self-efficacy and academic
performance has been well documented (Bandura, 1997; Goddard, Hoy, & Woolfolk, 2000;
Pintrich, 2003; Tschannen-Moran, Woolfolk Hoy, & Hoy, 1998). STEM integration involves a
new approach to teaching as well as learning new content outside of the range of background
knowledge of most elementary teachers (Appleton, 2008; Buczynski & Hansen, 2010;
Cunningham, 2004; Nadelson et al., 2012; National Academy of Sciences Report, 2014). To
positively impact student performance, researchers would be wise to focus sustained attention on
teacher self-efficacy. As teachers begin to feel more efficacious about teaching STEM, they will
take on more challenging roles and set higher expectations for themselves (Bandura, 2006;
Guskey, 1988; Nie et al., 2013; Rittmayer & Beier, 2009; Vardaman et al., 2012; Zimmerman,
2000). This translates to more challenges and higher expectations for students. As with any
innovation within education, the methodology employed in trying to increase the teacher self-
efficacy will be critical. Knowing how teachers’ self-efficacy develops is essential to any
planning for transformational learning opportunities.
STEM + DESIGN THINKING 44
Developing Self-Efficacy
Research converges upon four key sources of information that develop self-efficacy
beliefs: mastery experiences, vicarious experiences, social persuasion, and physiological
reactions (Bandura, 1997; Chan, 2008; Mintzes, Marcum, Messerschmidt-Yates, & Mark, 2013;
Pajares, 2005; Putney & Broughton, 2011; Rittmayer & Beier, 2009; Zimmerman, 2000).
Mastery experiences are practice opportunities to build successes. Specific to STEM, this could
involve hands-on activities that engage teachers in the discovery approach to learning with
incrementally more challenging activities. Vicarious experiences involve watching role models.
To develop STEM self-efficacy teachers could visit classrooms to observe peers who are
successfully implementing an integrated STEM model within their lessons. Social persuasion
involves the feedback, support, and judgments received from others. This is where the staff,
parents, administration, and students themselves impact the level of teacher self-efficacy.
Finally, the physiological reaction a teacher feels when engaging in the preparation or
participation of the task must be considered. It is important to lower the affective filter and
provide opportunities for success. Integral to all information sources is the context: who, where,
when, and what was happening when planning and attempting to reach the goals.
Self-efficacy is specific to each individual, affected by context, domain specific, and task
specific. Tools for measuring self-efficacy are numerous, but at the time of the study there were
no tools specific to STEM. That posed two limitations. First, self-efficacy was measured via self-
report scales that can have questionable validity as teachers have been found to both over and
underestimate their ratings on self-efficacy scales for a variety of reasons (Bandura, 2006). Since
there was no other instrument, one had to be constructed by modifying previous tools.

STEM + DESIGN THINKING 45
Goal setting is another critical factor related to a teacher’s self-efficacy (Cox & Beier,
2009; Pajares, 2002). The successive opportunities for successful achievements at an incremental
rate are most important. Research supports personal construction of proximal goals rather than
distal goals (Rittmayer & Beier, 2009; Zimmerman, 2000). A distal goal can be so far from
where a teacher is starting that sustained perseverance may be over challenged. Instead, the
pacing of smaller goals that can be achieved separately will build increasing levels of self-
efficacy that in turn prepares the teacher cognitively and affectively for the next challenge. It is
these incremental challenges that are approached and achieved as smaller goals leading to larger
goals that lead to self-sustained growth in efficaciousness. Since STEM is still an uncharted land
for most teachers, that level of sustained self-motivated perseverance and tenacity will be
necessary to reach the end goal of teaching STEM in an integrated fashion while feeling highly
self-efficacious doing so.
Innovation
As an instructional innovation, teaching STEM in the elementary classrooms has been
found to be related to both teaching style and self-efficacy. Consistent with social cognitive
theory (Bandura, 1977; Bandura, 1997), teachers with a more constructivist approach versus
didactic approach to teaching were more efficacious about teaching (Nie et al., 2013).
Researchers have found that the level of teacher efficacy was positively related to the openness
to new pedagogies and innovative instruction (Cheung, 2008; Gokcek et al., 2013; Nie et al.,
2013). The ability to subsequently achieve successful performance and having multiple
experiences focused on mastery were found to be powerful indicators of increased self-efficacy
(Carberry et al., 2010; Chang, 2009). While causal claims cannot be formed, it can be inferred
that offering professional development for elementary teachers emerging into the innovative
STEM + DESIGN THINKING 46
territory of STEM integration that sets them up for successful enactive experiences would benefit
from dual immersion in both instructional and self-efficacy goal oriented activities.
Engineering Design
Terminology
Incorporating engineering into elementary education is an innovation (Sun & Strobel,
2013). The National Academy of Science report (2014) explicitly stated that there is growing
emphasis on the engineering design process within K-12 engineering education that supports the
approach and design of this study. Engineers, like educators, “design inventions and solutions”
(pg. 20) to solve real world problems (Roehrig et al., 2012). It is in the creative thinking and
design approach that engineering can be distinguished from other STEM disciplines. Evidence
can be found in etymology. The word “engineer” can be traced back to the medieval Latin word
“ingeniare” meaning to design and the Latin word “ingenium” which means clever invention
(Mann et al., 2011). While engineers often do design clever inventions to satisfy the needs and
wants of humans while under design constraints, this study used a more general definition for
engineering design.
For this study, the definition for engineering design used by Carberry, Lee, and Ohland
(2010) was utilized: “applied or practical component of engineering . . . which consists of
several processes used in devising a system, component, or protocol to meet an identified need”
(pg. 71). There are five specific reasons why their definition was selected to drive this study.
First, it emphasizes that it is a process and that it should be iterative. Second, it states that what is
created must fit an identified need. Their definition centers on thinking before, during, and after
the process to design, prototype, test, evaluate and redesign. By focusing on iteration to create a
solution for an identified need, it should also align to the research questions on self-efficacy
STEM + DESIGN THINKING 47
which are related to motivational learning theories as well as social cognitive theories. The third
reason is directly related to validity. The study validated a self-efficacy in engineering design
instrument that was used. The fourth reason for using the definition of Carberry et al. (2010) was
that it was one of the few studies on engineering design in education which the researcher
located that identified a specific rationale for selecting the iterative engineering model on which
it was based. With so many models claiming to be “the engineering design cycle”, it was
important to select one with clear purpose and alignment to research. This study validated the use
of the model created by the Massachusetts Department of Education (2006). The model and self-
efficacy scale were found to have content, criterion, and construct validity (Salkind, 2013) which
improves the validity of this study. Finally, the researchers (Carberry et al., 2010) built their
study on Bandura’s social cognitive theory that aligns to the theoretical framework of this study.
They cited the importance of experiences upon building both self-efficacy and engineering
design understanding both of which are aligned to the research questions of this study.
Maintaining theoretical alignment and consistency on so many levels increased the validity of
this research.
A Developmental Process
Within the engineering design focus of this study and the research cited, the overarching
theme was the developmental process. Consider the connection: young children, ideas,
constructs, thinking, understanding, concepts, self-efficacy, and learning are all developmental.
Development involves a process, trial and error, testing and revising, then implementing or
redesigning. The social cognitive theoretical framework supporting and contributing to the
design process of this study involved identification of a need, researching, developing a plan,
creating a prototype, testing the prototype, then redesigning for the actual study. It is the
STEM + DESIGN THINKING 48
engineering design process applied to learning in the same way that the STEM exploration space
is a redesign and application of the engineering design process to be learned through the STEM
training.
Divergent Motives
What exactly is engineering design within an elementary educational framework? In many
ways, this remains the most pivotal unanswered question. The motives and goals of the
engineering in K-12 initiatives are being scrutinized. Kelley (2012) warned of confusions formed
by dual missions within the literature. The most recent publication from the National Academy
of Engineering titled “The Importance of Engineering Talent to the Prosperity and Security of
the Nation” suggests that future focused goals of filling the engineering pipeline with talent are
the driving force. However, teachers rarely focus on the future goals of student careers at the
elementary level. Instead, they focus on the learning that is supported and encouraged by the
report only one month earlier from the same source: National Academies Press. Clearly there are
mixed messages related to motive as foreseen by Kelley (2012). Is the goal to increase the
number of students pursuing STEM careers or to provide K-12 students with a better STEM
education? The concern can be highlighted when viewed through the lens of research and
motivational theories. An increase in the number of students pursuing STEM careers is a goal-
oriented approach with quantitative data analysis focused on outcomes. Providing better STEM
learning opportunities for all K-12 students is a mastery approach with qualitative data analysis
focused on the learning process. Most initiatives align themselves to the goals of increasing
America’s global competitiveness and future economic growth not student learning (Fulton &
Britton, 2011). From the perspective of a teacher and researcher, this is problematic.

STEM + DESIGN THINKING 49
Kelley’s warning to identify the actual goal and clarify the mission seems warranted since
innovation is involved. Rogers’ (2003) Diffusions of Innovations theory applied to engineering
in education as an innovation would consider the time allotted to learn engineering design
concepts, how it will be communicated or shared out, and the social context in which it is to
spread. This generates a subset of questions related to Kelley’s concern of motives and goals.
How is the decision to infuse engineering into lesson design made now and will that change over
time? Is it optional, collective, or based on a top down authoritative model? The self-efficacy and
motivational theories suggest different outcomes based on the level of perceived control over the
decision. What support system is in place for successful pedagogical experimentation, feedback,
and sustainability? In future research, those unanswered questions may explain the variation in
levels of successful implementation across schools.
Teacher Professional Development Models
If teachers do not experience integrative STEM content and pedagogical approaches, then
it is unlikely that they will be able to teach integrative STEM within their classrooms (NAS,
2014). Teachers who had those experiences as a result of participation in a STEM program
professional development in-service identified three specific factors that contributed to their
learning: collaboration, hands-on learning, and credibility of the instructor. Collaboration was
the factor in selecting the design framework. Hands-on learning goals led to the social cognitive
theoretical framework.
Offering clear support and evidence of the importance, relevance, and potential impact of
this study are the three factors related to educator expertise cited in the NAS report (2014):
content knowledge, self-efficacy, and opportunities for collaboration. The strategic construction
of this study addressed all three critical elements. Content knowledge was the focus point from
STEM + DESIGN THINKING 50
which each of the STEM training sessions was collaboratively constructed. Self-efficacy was
addressed through the specific, targeted, positive feedback from the researcher, social persuasion
within the teachers who formed the training team, and positive physiological responses elicited
through the hands-on exploration opportunities.
The Workshop Model Approach
Building upon professional development (PD) research for elementary educators,
Buczynski and Hansen (2009), Darling-Hammond (1995), Makgato (2012), and Mishra and
Koehler (2006) concur that the traditional one-day workshop delivery model is insufficient for
improving student learning or deep understanding by teachers. Mishra and Koehler (2006)
discovered that context-neutral staff development (ignoring the school site, classrooms, and
students themselves) leads to generic solutions that will fail since they overemphasize content
but often ignore pedagogy (basic CK only). Instead, scaffolded experiences that consider the
context in which the teacher will be teaching are most effective (Sargianis et al., 2012; Yoon et
al., 2013). Specific to STEM PD, researchers (Annetta et al., 2013; Buczynski and Hansen,
2009; Makgato, 2012; Sargianis et at., 2012; Sun & Strobel, 2013; Yoon et al., 2013; and Yu et
al., 2012) concur that PD designed to begin with developing pedagogical content knowledge to
build self-efficacy should be the primary goal. The implementation, duration, and mediation of
that professional development have varied results and impact upon student learning. This
research design was purposefully incremental, ongoing, and as long term as possible given that it
was the end of the school year.
The studies investigating a professional development workshop approach only
(Buczynski & Hansen, 2009; Sargianis et al., 2012; Yoon, Diefus-Dux, & Strobel, 2013) offer
conclusive results. The workshop-only approach is not sufficient to develop teacher’s content
STEM + DESIGN THINKING 51
knowledge that lasts over one week according to pre and post tests. Those studies all recommend
ongoing mentoring, coaching, or professional development opportunities which continued
beyond the conclusion of this study but were not a part of this research.
Combining Professional Development Workshops with Ongoing Training
Researchers building upon successful professional development models in an effort to
achieve more long-term results of teacher change in both self-efficacy and content knowledge
have added ongoing training modules or additional follow up workshops during the school year
in an effort to sustain growth beyond the PD day (Annetta et al., 2013; Buczynski & Hansen,
2009; Goldschmidt & Phelps, 2009; Macalalag & Tirthali, 2010; Nadelson et al., 2012; Yoon et
al., 2013). The duration of the staff development had a direct impact on the transfer of concepts
and content into classroom instruction and student learning (Darling-Hammond, 1995). Those
PD workshops that lasted fewer than 2 days or 14 hours had no statistically significant effects on
student learning (Darling-Hammond, 1995). This study lasted five weeks and would later involve
ongoing PD and a blended learning model including online modules. Later, a customized website
created for the school that curated STEM content for classroom use would be created for
sustainability beyond the scope and time constraints of the study.
Based on the case studies of Makgato (2012), one can infer that another reason why
ongoing professional development has been found to be qualitatively and quantitatively more
effective than a one day workshop model is the need for developing effective pedagogical
approaches to teaching STEM. His work on developing teachers’ technological pedagogical
content knowledge (TPCK) unveiled that “teachers don’t know how to use the appropriate
constructivist methods and principles in the teaching and learning of technology” (pg. 1398). His
work cited four goals of a constructivist classroom: 1) teach within the zone of proximal
STEM + DESIGN THINKING 52
development 2) make connections to real life and background experiences, 3) use various
sources and materials to allow discovery learning rather than telling answers, and 4) small group
interactions and dialogues are key to constructing knowledge. While Makgato’s results concur
with other researchers that content knowledge development is critical for STEM infusion into
classrooms (Buczynski & Hansen, 2009; Macalalag & Tirthali, 2010; Mishra & Koehler, 2006;
Yoon et al., 2013), it goes further by emphasizing the role of teaching methodology.
Results of Innovative Professional Development Designs
In agreement with the vital role of reflection in teachers’ knowledge growth (Guskey,
1986; Makgato, 2012; Mishra & Koehler, 2006; Shulman, 1986) is the research by Yu, Luo, Sun,
and Strobel (2012) which suggested a teacher competency model specific to STEM integration in
K-6 classrooms. Their research concluded that content knowledge (CK) is not sufficient to
promote teacher change in STEM integration. Instead, their research equalizes the roles of
content knowledge, attitudes, and skills. From this, one can infer that a teacher’s self-efficacy,
perceptions, pedagogical methodology knowledge, and content knowledge all play key roles in
learning innovative content and transfer to teaching. This study also involved a multifaceted
approach to gathering metacognitive thinking. In blank composition books labeled Engineering
Notebooks, each teacher reflected upon the activities and interactions of the training throughout
the duration of the study synthesizing the content knowledge, experiences, and discussions.
Utilizing the power of 24/7 technology access, the study by Nadelson et al. (2013) found
that a three-day summer institute was enough to evoke significant gains in confidence and self-
efficacy among the first through fifth grade teachers. The study involving five half days over the
course of five weeks was expected to result in similar findings related to gains in self-efficacy. It
did. It also revealed the impact of ongoing access to support when needed. This research study
STEM + DESIGN THINKING 53
added a flipped professional development model maximizing the 24/7 technology access by
creating an online portal that could be maintained after the conclusion of the study providing
ongoing learning opportunities.
Results of Research on Professional Learning Communities (PLCs)
This study incorporates the recommendations of all previously cited research within a
voluntary professional learning community (PLC). According to the National Academy of
Sciences preliminary report (2014) and a joint study by the National Commission on Teaching
and America’s Future and WestEd, collective intelligence and collaboration within professional
learning communities (PLCs) lead to more effective implementation of STEM integration.
Professional learning communities are successful arenas for infusing STEM pedagogical content
knowledge within a school setting. The report found that the learning team model could be an
effective mode for all teachers with all students. This inclusive finding justifies the rationale of
establishing a STEM Design Thinking Training Team for the participants in the study to build
collective efficacy, collective responsibility, and collective intelligence. The most impactful
finding was that even poorly designed PLCs led to reported gains in efficacy (Fulton & Britton,
2011). Well-designed PLCs were found to improve pedagogy, include content knowledge, and
encourage reflection. This research study is aligned to the recommendations and successful
models of PLCs to reach all of these goals while also increasing teacher self-efficacy for
teaching STEM and engineering design. While there is an abundance of research on factors
leading to success in PLCs in general, few researchers have studied STEM specific PLCs (Fulton
& Britton, 2011). This study adds to the available research.
As could be expected, the credibility of the facilitator, design, and implementation of any
shared learning team model are critical factors for success. Given that STEM is an innovation,
STEM + DESIGN THINKING 54
these elements may arguably be even more important as the social context may be more
influential than in a regular student work focused PLC. Building collaborative professional
capacity within the school setting has also been associated with immediate gains in teacher self-
efficacy and pedagogical content knowledge (Fulton & Britton, 2011). Involving all interested
teachers along with the school principals has been found to be effective in providing support
structures across school systems (Fulton & Britton, 2011). This study incorporated all of those
elements.
From an organizational learning perspective, collaborative problem solving through
shared learning teams has proven to be effective in promoting change, increased knowledge, and
increased group efficacy (Guskey, 2002; Tschannen-Moran, Uline, Woolfolk Hoy, & Mackley,
2000). From the participatory design perspective, teachers are designers of classroom instruction
and must be comfortable working in learning teams (Blair-Early, 2010). Blair-Early (2010)
describes participatory design research as having four key features: collaboration, reflection,
participatory problem solving, and self-evaluation comparing it to action research. All of these
elements have been included within the design of this study.
Across the literature on professional learning communities, research supports models
built upon collaboration, inclusion across stakeholders including principals (Tschannen-Moran et
al., 2000), formation of small learning teams, maximizing online resources (Fulton & Britton,
2011), incorporating lesson study (Lawrence & Chong, 2010; Sibbold, 2010 ), hands-on active
learning (Ejiwale, 2012), promoting reflection (Ejiwale, 2012), and building teacher’s sense of
ownership (Fulton & Britton, 2011). All of these facets were woven into the design of this study.

STEM + DESIGN THINKING 55
Implications and Conclusions of Professional Development Models
The most effective professional development workshops involve hands-on constructivist
pedagogy in which teachers plan or design lessons for immediate use in their classrooms. To
enhance the content knowledge of technology, Mishra and Koehler (2006) found evidence that
hands-on activities and projects such as making movies, redesigning websites, and developing
online courses were successful. Research suggests that when incorporating the design element,
teachers show more PCK versus merely growth in CK without the ongoing plan for effective
classroom implementation (Makgato, 2012; Mishra & Koehler, 2006; Yu et al., 2012). This lends
support for the alignment of this study within the participatory design framework. Specifically,
the approaches used in the PD that were expected to be most effective involved project based
learning (Cotabish et al., 2013; Mishra & Koehler, 2006), inquiry (Buczynski & Hansen, 2009;
Cotabish et al., 2013; Macalalag & Tirthali, 2010), or the design cycle (Annetta et al., 2013;
Macalalag & Tirthali, 2010; Mishra & Koehler, 2006; Yu et al., 2012). The constructivist-
oriented approach led to increased self-efficacy (Darling-Hammond, 1995; Ejiwale, 2012;
Makgato, 2012; Mishra & Koehler, 2006; Yu et al., 2012). Time for reflection involving note
taking or journaling was also a key element in successful PD interventions (Annetta et al., 2013;
Darling-Hammond, 1995; Ejiwale, 2012; Mishra & Koehler, 2006; Nadelson et al., 2012;
Sellars, 2012; Yu et al., 2012). This study has been designed around that research to optimize
the knowledge already gathered then apply it to the innovative design of the study from a social
cognitive perspective.
Continuous professional development should involve peer observations of practice,
analysis of student work, collaborative discussions, empowering teachers, deepening content
knowledge, and should be sustained over time (Darling-Hammond, 1995). Teachers should
STEM + DESIGN THINKING 56
become facilitators across the STEM fields and classrooms can become labs of creativity and
innovation (Ejiwale, 2012). The STEM Design Thinking Training Team functioned as a small
collaborative lab that could later grow and evolve to include all stakeholders as the inclusive
school could become the lab of innovation to design the STEM exploration space.
STEM Through Participatory Design
According to the National Academies Press report (Honey et al., 2014), extensive
collaboration is an integral part of the design needed for integrated STEM education among
teachers to predict the success (pg. 86). The report also asserts that the way the groups are
formed, the activities they encounter, and active engagement are critical factors of success. The
participatory design is inclusive by nature and the activities the STEM team experienced were
strategically and purposefully selected and aligned to the research and learning goals. According
to Simon (2010), participation thrives on constraints and involves contribution, collaboration,
and co-creation by the users. In the words of the President of the National Academy of
Engineering, William Wulf, engineering is “design under constraint” (2001). Merging all of the
ideas together leads to the justifiable evidence and support for the design of this study.
The thread that ties all of the ideas together conclusively is the “user”. Research concurs
that the teachers are the key to STEM integration success (NAS, 2014; Simonsen & Robertson,
2013). The teachers are the designers, models, integrators, and evaluators of the nation’s
children. When every teacher has ownership and drives his or her own learning, then the overall
effect of the group is elevated especially when recognizing that it promotes diversity and
inclusivity. Through participatory design, every teacher has a chance to share ideas and inspire
learners through experiential user driven discoveries. It begins with the teachers, so this study
STEM + DESIGN THINKING 57
was crafted for the teachers to give them a voice and sense of empowerment as they learn how to
integrate STEM into their lesson design and develop conceptual understanding of STEM
integration.
Participatory Innovation
Participatory design as applied to innovation has been used interchangeably with the term
participatory innovation (Simonsen & Robertson, 2014). It is “user-driven innovation”
(Simonsen & Robertson, 2014) in which the users are the stakeholders who cooperatively and
collaboratively make or modify things to bring them into alignment with their needs. It is the
design element – the how – not the what – which is created or modified therefore it focuses on
the process as applied to solve problems.
The key word is action as it involves informal action research involving active engagement,
retooling, rethinking, and redesigning. Participatory design is experiential, action oriented, and
not bound to a set of rules, procedures, or outcomes leaving it open to interpretation based on the
contextual features. This study paralleled the study by Bjorgvinsson, Ehn, and Hillgren (2010)
as it was originally designed to foster and support long-term relationships within the school
context allowing the stakeholders to continue being co-creators as what they designed after the
study entered their real life context as the STEM exploration space for innovation.
Maker Movement
The theme across all design labs or maker spaces is exploration, creation, and discovery
(Doorley & Witthoft, 2012; Drew, 2011; Honey & Kanter, 2013; Martinez & Stager, 2013).
There is a growing trend to create exploration spaces where people of all ages, called “makers”,
can engage in discovery learning through experiential play. This movement has been called the
Maker Movement due to the association with Make magazine and MakerFaires. The mission of
STEM + DESIGN THINKING 58
the Maker Movement is to encourage pedagogical approaches to learning that involve hands-on,
experiential learning. The Maker Movement is directly aligned to the Social Cognitive Theory
and constructivist approaches to learning. Dale Dougherty, the founder, was heralded by
President Obama in his call to action to make “every child a Maker” (Honey & Kanter, 2013;
Martinez & Stager, 2013). What is at the core of the president’s challenge is that learning can
and should be fun, provide intrinsic motivation, and involve active exploration and design.
President Obama has also challenged America to incorporate STEM into all classrooms K-12.
Research is beginning to converge around the possibilities within the educational settings to
combine these two goals of making and learning integrated STEM concepts and content (Honey
& Kanter, 2013; Martinez & Stager, 2013). This study incorporated the fundamental principles
of the Maker Movement in every shared learning training session as it maximized learning
through hands-on exploration based learning.
Perspective of this Study
Making, tinkering, discovery-based learning, free exploration, constructivism, project-
based learning, and engineering are being used synonymously with increasing frequency
(Martinez & Stager, 2013). This study assumes there are differential effects of using the various
terms and that engineering is not equivalent to tinkering or “making” in three distinct ways. First,
while both involve a level of trial and error, discovery, and creation, engineering has a cycle of
iteration that the researcher viewed as critical to metacognitive thought. The process of thinking
about ways to redesign something that was created for incremental improvement is not an
underlying condition of “making” or “tinkering” in which something could be created then put
aside to create something new. Second, based on years of classroom teaching experience, the
researcher held the perspective that “making” and “tinkering” did not necessarily involve solving
STEM + DESIGN THINKING 59
a problem nor do they need to relate to real life. Engineering by definition in this study needed to
have a purpose, solve a problem, and/or have a real world application. Third, engineering
involves thinking or planning, creating prototypes, and redesigning for improvements. Making
and tinkering in practice and by definition do not always require planning or thinking ahead nor
do they necessarily involve making improvements or redesigning which are critical aspects of
the engineering design process terminology used within this study (Martinez & Stager, 2013).
What the researcher did agree with was the element of discovery, creation, and hands-on
exploration implied within all terms. However, to qualify as engineering from a learning
perspective within a K-5 context of education, this study extended the making and tinkering to
real life problem solving driven by purpose in which there was cognitive thought involved
throughout the process to reconstruct, change, or adapt along the way (Yu et al., 2012).
Since clarification of terminology is one of the strongest criticisms involved with STEM
research (Honey et al., 2014), this study attempted to further distinguish between subtleties in
words. Based on social cognitive theory applied to learning in the K-5 classroom, the terms
making and tinkering are not synonymous with the participatory design in this study. While both
have roots in constructivism and hands-on discovery, making and tinkering do not by definition
require a social context. In fact, they can be done alone. That contradicts the purpose, structure,
and philosophy of participatory design and the social cognitive theory in which the social context
is arguably the critical factor related to outcomes. Making and tinkering can indeed occur within
a participatory design. However, a social interaction or collaboration element rather than parallel
activities among people in the same space would need to be evident to constitute participatory
design.

STEM + DESIGN THINKING 60
Mutual Learning
A core feature of participatory design is mutual learning (Simon, 2010; Simonsen &
Robertson, 2014). Mutual learning is contextually bound. Context is a core element of
participatory design as the people, behaviors, and environment interact differently in each
example of participatory design. The authenticity of the context and the purpose of the activities
have a direct impact upon success (Honey & Kanter, 2013; Mann et al., 2011; Simon, 2010).
Mutual implies a social context. Learning implies cognition. Together they form the outcome of
applied social cognitive theory. Mutual learning can impact collective efficacy which again
draws evidence for merging social cognitive theory with participatory design within an
innovation (Dow & Klemmer, 2009). The format that promotes this mutual learning is within
workshop style design sessions providing feedback, eliciting ideas, and thinking creatively
(Yamauchi, 2012) which mirrors this study design.
Innovation Ecosystem
Conceptually, participatory design experiences can be thought of as design labs in which
brainstorming, ideation, creation, redesign, and use are common cycles. However, at any point in
the process there could be a shift depending on actions or events that occur within the context or
the design itself. It is this constant state of flux that exemplifies participatory design as no two
places in which the process is applied will necessarily look the same. Indeed, they will also
rarely be replicated step by step as procedural replication is by design in opposition to
participatory innovation. It is this constant state of change that supports what Bannon and Ehn
have called an innovation ecosystem (Simonsen & Robertson, 2014).

STEM + DESIGN THINKING 61
Design of this Study
This study was designed with the social cognitive theory as the backbone supported
through participatory design. The participatory design does not necessarily share the same
learning approaches as social cognitive theory as it could appear by name to be more open ended
and designed by the participants. This study merges the two with the learning as the focus. The
structure for each training session was designed based on the four influences of self-efficacy:
enactive, vicarious, social persuasion, and physiological reactions (Bandura, 1977). The enactive
phase involved hands-on discovery. Vicarious learning took place through observing both live
and video-based models engaged in engineering related tasks or discussions. Social persuasion
was one of the closest factors to pure participatory design as the context was the driving factor
and feedback was extensive. The collaborative design and redesign of the ideal STEM
exploration space was aligned with the participatory design approach as it was user driven.
Worked examples have been found to promote learning (Mayer, 2011) and were available
for each activity. The key word here is “available”. They were incorporated using augmented
reality and accessible by scanning an image with an iPad. This was purposeful by design to avoid
what Kulkarni, Dow, and Klemmer (2014) cited as problematic within participatory design –
worked examples increased conformity. By having them available yet invisible (augmented
reality) and requiring an extra step to access, the rationale was that the conformity drawbacks
could be minimized or obsolete for those who did not scan the worked example. The worked
example (Mayer, 2011) would also reduce the demands on working memory knowing the
teachers would qualify as novices within the fields of engineering design and integrated STEM
(see Appendix B for criteria for participation). According to research on cognitive load, worked
examples support novices and lead to more positive learning gains than discovery learning alone
STEM + DESIGN THINKING 62
(Juhani & Sweller, 1999). This is still within the bounds of participatory design as key
researchers (Dow & Klemmer, 2009) also found that worked examples increased levels of
creativity (Kulkarni et al., 2014). Reflections and the Engineering Notebook notes were
intentionally included to promote metacognitive thought and self-efficacy although not part of
pure participatory design. Therefore, this study incorporated what could be considered an
enhanced participatory design structure grounded in social cognitive theory.
Theme of Change
Upon close inspection of participation design and participatory innovation research, it is
clear that the consistent theme is change. The organization, collaboration, participants,
experiences, and reflective practices are constantly changing. The concept of STEM and
engineering within elementary schools is also in a deep process of change that will be context
dependent. Theories and concepts undergoing change are clearly aligned to a participatory
design approach. They also align in their goals of promoting change that is both sustainable and
transformative while inspiring and fostering motivation, curiosity, and learning.
Conceptual Development and Conceptual Change
As with participatory design, concept development and conceptual change are
progressive with incremental alterations and adaptations along the way that can result in a
transformative experience. As with innovation, concepts need to be cultivated and open to
modification when faced with a nonconforming factor.
Multiple Perspectives: Terminology Differences
Whether STEM integration falls within the learning frames of conceptual development or
conceptual change is debatable from multiple perspectives. For many elementary teachers,
engineering and STEM have simply never been considered. Many do not have any prior
STEM + DESIGN THINKING 63
experiences, educational background, or concepts of STEM thus lending support toward the
perspective that there was no change involved, merely the development of new schema. Other
teachers have misconceptions related to concepts of engineering and STEM which require a
change or shift in schematic organization prompting conceptual change. Martin and Schwartz
(Vosniadou, 2013, ch. 23) state that conceptual change involves building new knowledge
suggesting that they do not distinguish between new concepts and existing concepts. What their
research adds to the discussion is the unique focus on innovation. Supportive of the researcher’s
perspective, Martin and Schwartz distinguish conceptual change from innovation by identifying
innovation as having new ideas and new material structures (pg. 447).
Perspective of this Study
The researcher of this study was faced with cognitive conflict while trying to select a
necessary perspective from which to view this study. Close analysis of the actual concepts of
STEM and engineering specific to grades K-5 helped delineate a reference point. Since
integrated STEM at the elementary level is an innovation, “integrated STEM” did not exist
before as a concept, and is recent within the research literature of elementary education, the
researcher felt that two research questions (RQ1, RQ2) center on concept development.
However, the third research question (RQ3) is clearly related to conceptual change as
engineering did exist and most elementary teachers have at least heard of it to have constructed
an idea of what it involved. A counterpoint could be posed for RQ3 that since engineering is an
innovation within elementary education (Sun & Strobel, 2013) and this study is specifically
within the elementary school setting, pedagogical thinking reflects more upon concept
development than conceptual change. Most elementary teachers have never considered
engineering at the elementary level (Brophy et al., 2008). Anticipating a range within conceptual
STEM + DESIGN THINKING 64
understanding at the school, some of the experiences and innovative activities created for the
training were built upon the research promoting conceptual change (Crismond, 2001; Heddy &
Sinatra, 2013; Mayer, 2008; Pugh, 2011; Thagard, 2012; Vosniadou, 2013).
Cognitive Conflict
Since learners have natural methods of avoiding any cognitive conflict that is necessary to
promote conceptual change (Mayer, 2008; Thagard, 2012), the researcher purposefully
constructed activities that strongly encouraged the teachers in the training to recognize,
acknowledge, discuss, and reflect upon these challenges. As adult learners, the teachers may
have had misconceptions far longer than students making them more challenging to unravel and
reconstruct. It is in the reconstruction of mental models that transfer is promoted (Mayer, 2008).
According to Mayer (2008), there are three steps to conceptual change. First, learners must
recognize the anomaly or misconception. Second, they must construct new mental models. Third,
they need to use the new model. In order for these three steps to occur, the learner must be
motivated to want to understand a concept which Sinatra and Pintrich (2002) identified as
intentional conceptual change. It is goal-directed, within the learner’s conscious control, and
involves regulation of cognition, metacognition, and motivational processes.
Conceptual Change Within the Engineering Design Cycle
Much of learning is simply the process of revising mental models. This includes first
recognizing that we have mental models then understanding what they are and allowing for
adaptation in the presentation of new information. This mirrors the engineering design process as
problems (cognitive conflicts) are encountered, ideas are generated, solutions are tested (new
mental models constructed), iterations take place (revisions to mental models), and solutions are
successfully used that solve the problem (new mental model transferred to use). The process
STEM + DESIGN THINKING 65
does not begin until a problem (cognitive conflict) arises, so the first step in provoking
conceptual change is facilitating the recognition of misconceptions. The training was designed to
initiate this first step toward changing teachers’ misconceptions surrounding engineering by
incorporating the following effective instructional strategies assembled from available research
(Chi, 2013; Heddy & Sinatra, 2013; Mayer, 2011): use of non-examples, compare/contrast
analysis opportunities, the inquiry approach, carefully guided discussions, confrontational and
refutation text, repeated trials to avoid theory based bias, concrete hands-on models and
experiences, and sets of statements to evaluate for validity.
The idea for posing a set of statements of which the teachers would have to assess the
validity and justify their thinking with evidence was drawn from a case study shared by Mayer
(2008, pg. 228) in which small groups of students examined a set of statements related to
scientific laws. Through this process, the students formulated their concepts and discussed points
and counterpoints. Mayer (2008) described this stage as the formulation stage that was followed
by the transfer stage. Learning is the ability to transfer knowledge from one context to another, to
make connections, and to form links among ideas for long-term storage and retrieval. The
stronger the level of transfer, the easier it is to build future connections and to retrieve that
knowledge with automaticity. Therefore, the transfer stage is the target of any training. For the
students in Mayer’s case (2008), transfer was demonstrated when they explained the rule or law
they selected as the most applicable to the real world problem they were to solve. In this study,
transfer was hypothesized to appear through vocabulary use, modifications to the ideal STEM
space models, changes in discussion and reflection content, and ultimately changes to lesson
design within classrooms.
STEM + DESIGN THINKING 66
Synthesis
So how can STEM, self-efficacy, collective efficacy, engineering design, professional
development, participatory design, and conceptual development or change all be unequivocally
tied together and justified as a consolidation into one unified research study? The answer is
organizational learning applied to the educational context as described by the work of Carlile
(2002, 2004) and implemented into the research of Yamauchi (2012). This research study is
grounded in social cognitive theory within the participatory design framework to focus on both
affective and cognitive factors of learning. Learning involves knowledge. Both Carlile and
Yamauchi agree that knowledge can be both a constraint and a platform for innovation.
As was noted in the description of conceptual change, people have concepts, knowledge,
and schema ingrained within them. They take it for granted that those constructs are reality even
when first faced with contradictory evidence. They must face the cognitive conflict and mental
challenges to take on varied perspectives or reconstruct schema. Often, being given new
information is not enough to promote learning. People often need to dispute and then be
convinced that an alternate view is actually correct. It is this cognitive work that promotes
conceptual change. STEM and engineering design are subject to concept development and/or
change within most elementary settings. In this research study, teachers faced the cognitive
struggles that were hypothesized to increase self-efficacy, collective efficacy, and conceptual
change. This was a process just as learning and the construction of new knowledge are processes
that develop and change over time.
Learning
Carlile (2002, 2004) made a unique distinction related to knowledge construction that was
validated by the work of Yamauchi (2012) and relates to every research question and concept
STEM + DESIGN THINKING 67
within this study. His work in distributed collaboration and innovation is a close match for this
study when examined through the lens of learning – the ultimate goal for all stakeholders. In his
pragmatic look at knowledge, he identified three distinct types of learning and extended the
generalized idea of “learning is transfer” within educational psychology literature. Specifically,
he described learning as three different conditions: transfer, translation, and transformation.
Within the discipline of educational technology and related to technological pedagogical content
knowledge, this mirrors the SAMR model of substitution, augmentation, modification, and
redefinition (Puentedura, 2013). The SAMR model is used to describe the life cycle of
technology adoption and mirrors the innovation ecosystem as described within Simonsen &
Robertson’s (2014) collected research on participatory design. Thus, the connections across
participatory design, STEM, innovation, and learning are present.
Convergence Model of Innovation
Complexity is embedded within any innovative study. Depth and complexity are embedded
in this study through the coalescing of the design, construction, factors, and frameworks across
the disciplines. Complexity involves interwoven concepts, models, and theories. Innovation
involves a new and unique contribution inspired by something that already existed. The
researcher proposes that since STEM is an integrated approach involving conceptual change,
pedagogical shifts, and innovation within education, an innovative, integrative model of learning
is applicable. The researcher proposes a congruence among the following approaches: Carlile’s
(2002, 2004) model of organizational learning, Puentedura’s (2013) SAMR model of
technological pedagogical content learning and Hurley’s (2001) levels of innovation integration
as applied to STEM integration. Note that all approaches were designed prior to the emergence
of the concept of STEM integration, focus on different fields, and were designed for varied
STEM + DESIGN THINKING 68
approaches. However, they all converge within the realm of learning and when viewed through
the lens of STEM innovation have a potentially impactful collaborative framework on which to
build learning within schools. Of note is the consistency in the term “transformational”.
Transformational experience within conceptual change theories also involves use in new,
meaningful ways. The Convergence Model of Innovation in Figure 1 provides a visual
representation of this synthesis.
Figure 1.

Comparing Carlile’s (2002, 2004) knowledge sharing model within design with
Puentedura’s SAMR model of technological pedagogical content learning and Hurley’s (2001)
levels of innovation integration within the context of STEM presents a unique and potentially
transformative perspective on the importance of research that intentionally crosses and merges
disciplines.

STEM + DESIGN THINKING 69
Transfer as defined by Carlile (2002) uses fixed knowledge to exchange information.
Within the SAMR model, this is mere substitution. It involves moving information from one
place to another without any change in the schema or level of application. Within STEM
integration as an innovation it could be viewed as sequenced or parallel integration. All are the
lowest levels of learning. A deeper level of learning occurs with translation (Carlile),
augmentation (Puentedura), and partial integration (Hurley). According to Carlile, translation
involves changes in design of both the features and work practices. This is analogous to
Puentedura’s augmentation (functional improvement) and modification (significant task
redesign) levels and parallels Hurley’s partial and enhanced levels of integration. The deepest
learning occurs at the transformational level that is supported by research within conceptual
change theories. At the transformational level of design (Carlile), features, work practices, AND
goals are redefined by the users within participatory design. This is similar to the highest level
of technology integration within Puentedura’s SAMR model, redefinition, in which new tasks are
created that didn’t seem possible before without the technology. Within STEM, that would
involve the integration across the disciplines of science, math, and technology through
engineering design. Applying Hurley’s levels of integration to STEM would identify this as the
total integration level. Transformative experiences are the foundation of conceptual change
(Heddy & Sinatra, 2013; Pugh, 2011; Pugh, Linnenbrink-Garcia, Koskey, Stewart, & Manzey,
2009) and organizational learning (Yamauchi, 2012) when faced with innovation such as STEM
and engineering at the elementary level. It’s all connected. Seemingly, participatory design,
STEM, technological pedagogical content knowledge, organizational learning, and innovation
assemble within this joint model (see Figure 1) and within this study to provide innovative and
potentially transformative applications to future research.
STEM + DESIGN THINKING 70
Analysis of the three types of learning within organizations was critical to the synthesis of
this research study, the design, and the innovative nature of STEM and engineering in
elementary schools. The primary purpose of this study was to lay a solid distributed foundation
within the elementary school being studied in preparation for the participatory design of the
STEM exploration space that would be available to teachers and students.
Theoretical and Conceptual Frameworks
Movements toward STEM integration schools, user engagement in design, and increased
creativity outlets are representative of the alignment of the theoretical models and frameworks
combined for this study. They reflect social cognitive theory, participatory design, organizational
learning, and theories of conceptual change.
Social Cognitive Theory
History
Social Cognitive Theory emerged from the 1960’s research of Albert Bandura on
modeling and vicarious learning. His demonstration of the impact of observational learning on
young children was originally called the Observational Learning Theory or Social Learning
Theory. He identified four key processes in learning: attention, motivation, retention, and
reproduction. The cognitive factors of attention, retention, and reproduction were integral to his
research and led to a more expansive theory involving human behavior not just learning. It has
since been known as the Social Cognitive Theory as it emphasizes both the social context and the
cognitive factors associated with learning. Key to the theory is the triadic responsibility model
that demonstrates how the environment, personal factors, and behaviors are all interconnected
and important elements in learning.

STEM + DESIGN THINKING 71
Relevance to this Study
This study was designed around the Social Cognitive Theory as the first dimension of a
multidimensional research plan to have the teachers apply what they learned while participating
in the study to real life applications in the context of their school (Dede, 2012). The environment
of the school and training, behaviors of the participating teachers, and the personal factors such
as prior knowledge, experience, and levels of efficacy were all critical elements of the study.
They form the triadic model of learning within Social Cognitive Theory. The participatory
design added an important user-constructed dimension to the social piece while the design
thinking activities of the training added depth to the cognitive growth. All of the activities, tools,
construction of new self-efficacy scales, and the purposeful design have been driven by the work
of Bandura (1977, 1997, 2006). The four approaches to learning information provided the
structure of the training session design: enactive, vicarious, social persuasion, and physiological
reactions. The importance of the internal mental states highlighted by Social Cognitive Theory
were directly associated with all three research questions showing tight alignment between the
purpose of the study and the approach taken.
Self-efficacy is a key component of Social Cognitive Theory, was a primary
recommendation for study within STEM integration by researchers cited previously, and was the
foundation of the first research question (RQ1). Both teacher self-efficacy and collective efficacy
were key components of this study as the teachers needed to develop self-efficacy for teaching
STEM through engineering design thinking. Collective efficacy was needed to promote the
scalablity and sustained learning environment necessary to successfully design, construct, and
iterate a collaborative STEM exploration space within the school environment at the conclusion
of the study.
STEM + DESIGN THINKING 72
Perspective of this Study
Social Cognitive Theory was the thread that wove the study together as well as the
structure that inspired all elements of the design. To add depth, complexity, and integrate
motivational theories, other complementary frameworks have been layered upon Social
Cognitive Theory highlighting the innovation involved in this study. Prior to this study, that
layering had not been conducted but was necessary to reach the desired outcomes of the training
and need for those outcomes within the most recent STEM reports (Honey et al., 2014;
Pellegrino, 2014). Since this study was based on the premise that learning is a continuous
process that could be promoted through collaborative structures, specific feedback, and
observational learning, it was in direct alignment with Bandura’s work. People learn from one
another and the STEM team training was designed with this in mind. All research questions were
built upon the Social Cognitive Theory.
Research on the Theory
Researchers have found that teachers’ self-efficacy is directly linked to their ability to
learn and reach specific goals (Bandura, 1997; Pajares, 1996; Pajares, 2005; Rittmayer & Beier,
2008; Woolfork & Hoy, 1990; Zimmerman, 2000). Learning involves “a change in what the
learner knows caused by the learner’s experience” (Mayer, 2011). Since all outcomes involve
learning and cognition, the close links between self-efficacy and learning were important in the
planning of the training. Since the social context is critical to the Social Cognitive Theory,
maximizing the collaboration and social aspects while applying the learning concepts was vital.
Expanding the work on teacher self-efficacy to include groups of teachers working
together is the body of research on collective efficacy. Collective teacher efficacy involves the
teachers’ beliefs that by working together for the benefit of the group it will lead to positive
STEM + DESIGN THINKING 73
outcomes (Chan, 2008; Cheung, 2008; Goddard et al., 2000; Schechter & Tschannen-Moran,
2006). This important link between teacher efficacy, learning, and transfer to positive impacts
upon student achievement transcends time and location to include learners around the globe
(Chan, 2008; Cheung, 2008). According to Cheung (2008), “teacher efficacy is a very important
factor for the improvement of education in every part of the world”. Since the world has never
been more globally connected than now, this worldwide finding across research that teacher self-
efficacy and collective efficacy positively impact learning of both teachers and their students is
important to note. Since the triadic responsibility model of the Social Cognitive Theory shows
how the environment, personal factors, and behaviors are all interconnected and important
elements in learning, the global perspective adds breadth to the future importance of the
interactions within learning environments given current and future virtual and blended learning
options.
Organizational Learning Theory
History
Chris Argyris was influential in the field of organizational learning and is an example of
how important it is to continually learn and adapt to new information that changes over time. His
original work surrounded the construction of what he viewed as mental maps for behaviors that
helped people plan, implement, then evaluate their actions (Argyris, 2010). This mirrors design
thinking. Donald Schön’s work on reflection in action and learning processes within
organizations (1993) has been used as the foundation of professional development for educators
and this series of STEM training sessions. Capturing how teachers think “on the go” and make
decisions is critical within a participatory design framework and innovation such as STEM
integration and engineering at the K-5 level. Argyris applied design thinking as his theories
STEM + DESIGN THINKING 74
evolved through work with Donald Schön to focus on reflection, individual inquiry, and the
learning cycle a person undergoes through the learning process. The learning cycle initiated
through trial and error paired with an emphasis on experiential learning combine to draw
similarities to engineering design. It is upon the key ideas of a learning cycle (learn – conflict –
revise), active learning, and how it affects an organization that this study drew theoretical
implications into the design.
Relevance to this Study
The history of organizational learning is echoed throughout the study. The key concept of
a learning cycle being continuous aligns to the engineering design cycle and an emphasis on
incremental growth and progress within and among people in the school. Structurally, this study
applied the concept by beginning with the teacher as the experiential learner then expanded it out
to a group within a school later to be expanded to the entire school organization. From a content
perspective, the design included all four stages of the learning cycle: concrete hands-on
experiences, reflection, abstract conceptualization, and active experimentation.
This study involved a subset of an entire school population that had not had any
professional development on STEM or engineering design, so it was considered to be an
organization that was emerging within the learning cycle. As Bannon and Ehn (in Simonsen &
Robertson, 2014) so aptly described the STEM integration movement as an innovation
ecosystem, the members living in the environment would need to successfully adapt, learn, and
grow as a school both individually and collectively.
This study focused on building a collaborative shared learning team structure that could
evolve, change over time, and be dynamically redesigned in response to contextual changes such
as changes in staff and across locations. However, this study focused primarily on the initial
STEM + DESIGN THINKING 75
design which would provide the foundation on which further collaboration, reflection, and
redesign could take place. Building collaborative learning experiences within the participatory
design framework aligns with organizational learning theory as it is the incremental growth of
each learner that adds to the collective growth of the school.
Perspective of this Study
Adapting to the paradigm shift currently underway in STEM initiatives within elementary
schools requires shifts in concept knowledge, pedagogical approaches, and content knowledge
across disciplines in which most elementary school teachers have not received domain specific
training. It was assumed and confirmed by the study that the teachers would have low self-
efficacy for teaching STEM and little prior knowledge of the concepts of integrated STEM or
how engineering fit into their elementary school.
This study was based on the perspective and model of organizational learning posited by
Carlile (2002, 2004) yet rooted in the work of Chris Argyris and Donald Schön. It has been
applied to the educational context in this study and is supported by the work of Yamauchi (2012)
within participatory design.
Research on the Theory
Empirical research exists which shows collaborative problem solving through shared
learning teams has proven to be effective in promoting change, increased knowledge, and
increased group efficacy (Guskey, 2002; Tschannen-Moran et al., 2000). The emphasis was on
mutual learning.

STEM + DESIGN THINKING 76
Conceptual Change
History
Thagard (2012) states that the history of conceptual change dates back only to the 1960’s
with the work of Thomas Kuhn and Paul Feyerabend on paradigm shifts in science. According
to Thagard (2012), Carey (1986) is one of the leading researchers within educational psychology
on concept development and conceptual change.
The researcher did not find any empirical studies on conceptual change specific to STEM
integration which lends evidence to the perspective that STEM integration at the elementary
level is an innovation that necessitates concept development not change. Most conceptual change
research is related to science (Chi, 2013; Heddy & Sinatra, 2013; Mayer, 2011; Thagard, 2012)
due to the many misconceptions often held by both children and adults and the historical origin.
Interestingly, Thagard’s collection of essays and research on conceptual change is called the
“Cognitive Science of Science” (2012).
Relevance to this Study
As with Organizational Learning Theory, mental model construction, deconstruction, and
reconstruction underlies conceptual change. STEM integration and engineering design within
elementary classrooms requires both pedagogical and content knowledge shifts. Misconceptions
may revolve around either based on media exposure and prior experiences.
Before concepts can change, the learner must be aware of the current concepts and
experience cognitive conflict challenging the concept (Heddy & Sinatra, 2013; Mayer, 2011;
Thagard, 2012). This study was designed to bring common misconceptions into conscious
awareness, maintain the necessary attention to unravel them, and help teachers structure or
restructure their schemas associated with STEM integration and engineering.
STEM + DESIGN THINKING 77
Distinguishing between transfer, translation, and transformation, Carlile (2002, 2004)
stated that transfer was simply substitution of knowledge. Martin and Schwartz (Vosniadou,
2013, ch. 23), used a parallel frame of reference when describing transfer within conceptual
change stating it is simply adding new knowledge on top of old knowledge. In their research of
conceptual change within innovation, they broaden the term transfer to include the importance of
context. This is consistent with the theoretical and conceptual frameworks of this study.
Creating a STEM exploration space within a school would be a transformative experience
on many dimensions. Heddy and Sinatra (2013) suggested that transformative experience could
be a pedagogical approach to stimulate conceptual change. Earlier research (Dole & Sinatra,
1998) found evidence that relevance and engagement foster conceptual change. Together, it
could be inferred that the research supports the participatory design approach (engagement) of
the authentic STEM exploration space (relevant) as a pedagogical approach to STEM and
engineering design concept development or change. The training was designed to provide such
experiences as it incorporated these three characteristics: expanded perceptions, experiential
value, and motivated use (Pugh, 2011).
Perspective of this Study
While future research may emerge that proves STEM integration is conceptual change once
STEM integration is mainstream, this study holds the perspective that it is currently an
innovation, was previously not a concept within elementary education, and is a recent
phenomenological movement. Engineering at the elementary school is more complex to identify
within the framework of conceptual change as engineering design is an innovation but most
teachers do have pedagogical concepts of engineering. In this study, the perspective was that
engineering at the elementary level is both conceptual development and conceptual change. The
STEM + DESIGN THINKING 78
reason for the distinction was for precision in both language and perspective to increase validity
and reliability.
The perspective taken and assumption made within this study was that the strategies,
approaches, and tools for generating cognitive conflict that can lead to concept formation or
reorganization are the same or similar. Aligned to the work of Thagard (2012), the researcher
also viewed the “a-ha” moment as a critical factor in both concept development and conceptual
change. It is through that journey of self-discovery that meaningful connections are made,
reconnected, or disconnected. The connection must be made by the learner him/herself to
disconfirm previously held beliefs necessary for new mental model construction. The teachers
are the learners.
Within STEM, there are an infinite number of potential misconceptions in any of the
separate disciplines. Combined, those misconceptions could be exponentially disruptive to the
learning process. Through strategic alignment to the goals, accurate content knowledge, and the
varied use of strategies that promote conceptual change, misconceptions can be unraveled,
analyzed, and reorganized. Being grounded in Social Cognitive Theory, this study provided
modeling and contextual support for cognitive clarity and connection construction. Within the
participatory design structure, the teachers were able to explore misconceptions to provoke those
“a-ha” moments for themselves which were expected to lead to accurate conceptual
understanding.
If learning truly does take place within an innovation ecosystem (Yamauchi, 2012), then as
life evolves so will some concepts and ideas. It is possible that the STEM integration concept
that has been researched, is being recommended (Honey et al., 2014), and underlies this study
STEM + DESIGN THINKING 79
could change over time with new knowledge. For now, it is an educational innovation with
strong research support and a call for action.
Research on the Theory
Researchers disagree on whether conceptual change is in fact different from learning
(Vosniadou, 2013). Vosniadou (2013) cites the distinct difference lying in the fact that the
“change” involves reorganizing content or organization of existing knowledge. Chi (Vosniadou,
2013, ch.3) explained it clearly as incomplete knowledge (learning) versus conflicting
knowledge (conceptual change). Further evidence that conceptual change is distinct from
learning reflects back to Mayer’s definition of learning being utilized in this study (2011). The
learning is defined as a change caused by the learner’s experience. It does not specify whether
learning was accurate or the formation of misconceptions. Conceptual change theory research
specifically involves the restructuring of schema and unraveling misconceptions to form new
accurate schema (Heddy & Sinatra, 2013; Sinatra & Pintrich, 2002; Thagard, 2012; Vosniadou,
2013). This distinction is critical to research involving conceptual change including this study
itself.
Conclusion
The universal theme across all of the research literature within this chapter is adaptation.
Teachers need to be responsive and adaptive to their students, themselves, environments,
educational systems, and worldwide trends. They need to be adaptive in terms of their
pedagogical approaches, when confronting misconceptions to restructure conceptual
understanding, and in their collaborative interactions within participatory design learning
opportunities. Designing lessons that are adaptive to learners and adapting to new ideas to be
integrated into classroom instruction are key abilities of effective teachers. Teachers are learners.
STEM + DESIGN THINKING 80
As learners, they need opportunities to practice adaptation in a safe judgment free environment
purposefully crafted to trigger curiosity, intrinsic motivation, and inquiry. This study was
intentionally designed to provide those conditions to optimize learning and promote adaptation in
thinking and actions.
The alignment of participatory design and engineering design thinking with the iterative
process as the thread tying it all together is clear. Since learning is a cognitive process that
develops and progresses along a continuum rather than among discrete levels, the thinking was
the foundational starting point of the research. This study began with the focus on developing,
changing, or reorganizing teachers’ thinking across all research questions. The cognitive thought
involved with engineering design and participatory design are parallel ideas that merge to
promote change through iterative processes. Engineering design provides the process and
participatory design provides the organization. The levels of learning involved across the
disciplines add depth and complexity to the study that would not be imaginable without such an
extensive literature review beyond the scope of educational research. By thinking about learning
from multiple perspectives, a new model of integrative innovation levels which literally
integrates models from different learning theories emerged and could be instrumental in
designing future tools and instruments to strengthen the empirical research within and across the
STEM disciplines and with any future innovation related to learning.

STEM + DESIGN THINKING 81
CHAPTER 3
METHODOLOGY
Introduction
While the study was designed prior to the release of the National Academy of Sciences
(NAS) report in February 2014, the report supports every aspect of the study including the
methodology, participatory design approach, theoretical models, framework, goals, outcomes,
implementation, and the nature and scope of the integration (pg. 32-35).
All competencies were embedded within the training design. The cognitive competencies
were addressed through the participatory design approach, Engineering Notebook reflections,
and carefully crafted experiences that involve critical analysis and creativity. The interpersonal
competencies were integral elements of participatory design and the method by which the STEM
training team functioned. There was a high level of responsibility to oneself, the members of the
shared learning team, and the stakeholders who would later explore within the STEM space they
would begin to design at the conclusion of the study. Communication and collaboration are
hallmarks of participatory design. Flexibility and initiative drive the engineering design process
that interweaves all activities. Metacognition is the key purpose of the reflective portions of the
training sessions. Research and this study were in tight alignment with relation to goals,
approach, and innovative thinking required to solve the nationwide problem related to STEM
integration.
Purpose of the Study
The purpose of this study was to research and identify the elements of a STEM focused
experiential training series that may have related to perceived changes in teachers’ self-efficacy,
classroom lesson design, and conceptual understanding of engineering within STEM at the
STEM + DESIGN THINKING 82
elementary level. On a larger scale of potential future value, the overarching goal of the study
was to combine design thinking with the engineering cycle to solve a real world need to discover
effective methods for helping elementary teachers integrate STEM within their classrooms. By
participating in the creation and iteration of their own ideal STEM space, the teachers were
applying engineering design thinking to help develop concepts and self-efficacy for teaching
engineering design within their classrooms. An integral part of this study was exploring specific
ways of building both self-efficacy in teaching engineering and experiences that would lead to
developing elementary teachers’ concepts of engineering.
Research Questions
The specific research questions being investigated included:
1. What factors influence teacher self-efficacy in teaching STEM at the elementary level?
2. What elements, experiences, or activities within a STEM focused training transfer into
lesson design?
3. What experiences or activities contribute to the development of the conceptual
understanding of engineering within STEM?
Methods of the Study
Research Design.
Data on self-efficacy, lesson design, and conceptual understanding, was gathered. The data
was gathered from self-report rating scales, surveys, reflective journals, and interviews. By using
such varied sources of information, the data triangulation increased the validity of the study.
Validity involves providing evidence of accuracy in results. When varied sources lead to a
similar conclusion, then the research usually has increased validity.

STEM + DESIGN THINKING 83
Each data source required extensive use of deductive coding from general big ideas down
to specific details and inductive coding to collapse the details into meaningful and relevant data
addressing the research questions. The qualitative data analysis process mirrored the engineering
design process in many ways as the data needed to be explored then analyzed, tested for coding
that would help themes emerge, redesigned as new information was gathered, then organized
into a final set of clear themes to be discussed and triangulated with the quantitative data
(Creswell, 2009; McEwan & McEwan, 2003).
The research design was related to integrative STEM teaching that is open to subjective
interpretation on a continuum rather than rigid data therefore calling for qualitative research.
Those tools included the Engineering Notebooks and the Semi-structured Interviews (see
Appendix C). However, the Self-Efficacy for Teaching STEM (see Appendix A) and the STEM
Subjects Surveys (see Appendix D) generated quantitative data.
Sample and Population
Sampling Procedures
The ultimate goal of this research design was to prepare teachers to transfer the learning
into their classroom practices and the collaborative design of a real STEM space on the school
campus the following school year. The researcher worked backward from the initial STEM space
goal that inspired this study. The process of designing the study mirrored design thinking and the
engineering cycle. The identification of a school that had little or no STEM training
opportunities was important. However, the decision making factor was whether or not the school
had the interest, space, and funds to truly build a STEM space on the campus upon completion of
the study. It was a true real world, problem-based challenge.

STEM + DESIGN THINKING 84
According to Nadelson (2013), most elementary schools were not teaching STEM due to
the innovative nature of STEM instruction. Therefore, the sample population needed to be
representative of most common practices in order for the results to be most impactful on
educational research. The goal was to identify a purposeful unique sampling (Merriam, 2009)
involving elementary teachers who had an interest in learning about STEM. Since the
participatory approach would be utilized as the design framework, it was important to allow all
interested teachers to participate.
School Selection
Although the researcher was working in a school district and had access to schools that
met the criteria, the decision was made to conduct the training in an unknown school district to
increase reliability of the results and findings. The researcher identified school districts within a
one hour drive that were similar in context to the one in which she had worked for 21 years.
From that list, the goal was to identify school districts who had not already started STEM
training or conversations. The researcher identified a school within a school district that was
virtually identical to the school where she taught for 17 years in size, demographics, teacher
experience, and location. To participate in the study, the principal needed to assure the researcher
that the school met all of the criteria listed in Appendix B. The principal was approached and
upon verifying the twelve items needed to meet the criteria, the forms were signed and the study
began.
Research Setting
The decision to select a school similar to prior experience in composition, demographics,
location, and API was made in an effort to increase ecological validity. The elementary school
that was selected was located in a suburban public unified school district in Southern California.
STEM + DESIGN THINKING 85
As of 2013, the student population was 424 students in grades K-5 (ed-data.k12.ca.us). The 2013
API score was 943 which is high and comparable to the researcher’s prior school. The high API
may relate to the level of openness to STEM integration and could be studied in future research
within schools of various API scores. With a statewide rank of 10 and a similar schools rank of 6
it is clear that it was a high performing school that was not performing as high as its peers.
According to the online database maintained by the state of California (ed-data.k12.ca.us),
17% of the students reporting a language other than English as the first language per self-
reported data cards. While the school was not as ethnically diverse as some schools in Southern
California, it was more diverse than many in the United States and was reflective of the
community it served. The design of this study placed high importance upon context. Therefore,
the closer the match between the researcher’s background experience and the school being
studied the stronger the ecological validity. Locating a school that best matched the researcher’s
background, had interest in STEM integration, and had not implemented any professional
development in STEM or engineering reduced the population sample selection available in light
of the time, cost, and distance constraints involved.
Participant Selection
The design under constraint extended to the participant selection and necessitated a
selection process. Certain criteria were established for selection eligibility to ensure maximized
validity. Since the research questions centered around teacher conceptual development and
teacher self-efficacy, the participants had to be teachers. This is contrary to true participatory
design in which all stakeholders would be involved yet the criteria are vital to the study. The true
participatory design features would be implemented upon completion of the data collection when
STEM + DESIGN THINKING 86
the STEM team training expanded full scale across the school. In order to participate in the
study, the volunteers needed to meet all 12 selection criteria listed in Appendix B.
Selection Process
Due to the time and research demands, it was not anticipated that there would be more
than 5 teachers willing to participate in the initial STEM team training. In case the researcher
misjudged the level of interest and self-sacrifice the teachers were willing to give, a plan for
narrowing down from a larger pool was created. Initially, the plan was to invite all interested
teachers to attend the STEM team training gatherings even if they had to miss a few. It was an
all-inclusive opportunity for every teacher on the campus. The researcher ideally wanted to study
all teachers, but the distance, funding, and time did not permit so it was redesigned within the
constraints. From the teachers who met the twelve criteria for inclusion of the study, the
researcher was seeking to have at least one teacher from every grade level if possible. The
alternative would be to focus on grade level teams who already had a history of collaboration on
some level. The target number was between 8-10 teachers. If more than ten teachers volunteered
and met the criteria, then the plan was to sort the teachers into grade level piles then randomly
select to ensure grade level representation across the school and adding the only level of
randomness possible within the constraints.
Ultimately, there were eleven teachers who chose to participate in the study. All eleven
met the qualifying criteria, were present and participating in every session of the STEM team
training, and all filled out all materials needed for data collection. The principal also attended all
sessions as an active participant to both learn and support his teachers as they would be
implementing what they learned on his school site of which he was accountable. However, he
did not participate in the data collection process.
STEM + DESIGN THINKING 87
Instrumentation
Rationale
As with any complex study, the number of tools and variety of instruments used to gather
empirical evidence were numerous and varied. Some tools did not yet exist and needed to be
created. This resulted in an extensive data collection and analysis process. To aid in the ability
for future researchers to replicate the study, the researcher collected as much data as possible
given the end of school year time constraints. With innovative design incorporated into a study
that had never been done before, it was far more important to have too much data rather than to
miss a critical piece. The rich data set that was collected during the study allowed for
triangulation that increased the validity and reliability of the results. To decrease the cognitive
load, Table 1 was created to collapse the information.
STEM + DESIGN THINKING 88

STEM + DESIGN THINKING 89
Data Collection
Surveys
On the first day, consent forms were provided indicating the teachers knew they could
terminate the survey and/or study at any time. For those volunteering, the Self-Efficacy for
Teaching STEM Survey (see Appendix A), Attitudes Toward Collaboration Survey (see
Appendix F), and STEM Subjects Survey (see Appendix D) were administered at the start of the
first of five half day collaborative training sessions. All three instruments were again
administered at the end of the final day of the STEM team training.
An instrument for measuring self-efficacy in STEM through engineering design was not
available at the time of the study, so the researcher designed the Self-Efficacy for Teaching
STEM Survey by adapting and modifying questions from various self-efficacy scales and
domain specific efficacy scales (Fink, 2013). Since the study was based upon Bandura’s Social
Cognitive Theory, his recommendations for creating self-efficacy rating scales (1996) were
implemented. To measure collective self-efficacy, questions related to efficacy in decision
making, influence of school resources, enlisting parental and community involvement, and
creating a positive school climate were strategically selected from Bandura’s Teacher Self-
Efficacy Scale. Due to the wide use of Riggs and Enochs (1990) Science Teaching Efficacy
Belief Instrument (STEBI), some statements were included with the modification of the words
“science” to “STEM integration”. Self-efficacy in engineering was measured with the
statements selected from the Teaching Engineering Self-efficacy Scale (TESS) developed by
Yoon & Griffin (2012). Finally, since this study took place within the K-5 environment which
had recommendations from the K-12 Engineering Standards (Carr, Bennett, & Strobel, 2012) to
focus most heavily on engineering design, portions of the self-efficacy scales needed to
STEM + DESIGN THINKING 90
specifically address engineering design. To accomplish this, the scale used in the work by
Carberry, Lee, and Ohland (2010) which defined engineering design within this study was
incorporated into the Self-Efficacy for Teaching STEM Survey. By drawing from the most
highly used and rated efficacy scales currently available, the researcher was striving to maximize
the validity of the new tool which was necessary due to the innovative nature of STEM
integration and engineering design at the K-5 level and corresponding lack of measurement tools.
In total, the Self-Efficacy for Teaching STEM Survey (see Appendix A) included 13 Likert scale
items rated on a self-reported scale of 1-6.
The survey component of the Self-Efficacy for Teaching STEM facilitated the gathering of
additional data including background experience, training, interest, and prior knowledge. This
was important information to modify any STEM team training plans to best match the needs of
the teachers and to stay within their zone of proximal development to scaffold their learning
experiences. The items included were derived from existing measurement tools. Specific to
STEM integration, items were included from the STEM Semantics Survey (Knezek &
Christensen, 2008). Those items also served as a triangulation source of data related to self-
efficacy.
Two other short surveys were included as pre and post training measures: Importance of
STEM Subjects Survey (see Appendix D) and Attitudes Toward Collaboration Survey (see
Appendix F). The Importance of STEM Subjects Survey was comprised of six Likert-scale
questions with a forced choice rating scale of 1-6. It was a modified version of the STEM
Semantics Survey (Knezek & Christensen, 2008) with an additional last question including a
rank order of STEM subjects expanded to include art and music. The Attitudes Toward
Collaboration Survey was researcher generated to account for any effects due to the participatory
STEM + DESIGN THINKING 91
design of the study. It included five Likert-scale rating questions related to teacher ability to
make decisions and collaborate at the school site. The information was needed to add depth to
the analyzing process across the tools. Summatively, the Self-Efficacy Rating Scale, Importance
of STEM Subjects Survey, and Attitudes Toward Collaboration Survey had three key
components being researched: self-efficacy for teaching STEM through engineering design,
importance of STEM subjects at the teacher’s grade level, and attitudes toward collaboration.
Additional information needed to best prepare the STEM team training activities
maximizing Social Cognitive Theoretical approaches within the participatory design framework
was solicited through open response format. As with any survey or questionnaire relying upon
self-reported measures, the validity of the instrument would be subject to validity questions.
The Self-efficacy Rating Scale and Survey, Attitudes Toward Collaboration Survey, and
Importance of STEM Subjects Survey were merged into one online survey that would
automatically generate data while allowing the teachers to use their iPads if desired. Paper
versions were also available. It was expected that the online version would save time and lead to
higher completion rates which would increase reliability. All participating teachers preferred to
complete the online version both before and at the conclusion of the study. In addition to
automatically generating organized data spreadsheets, the data collection process could be
verified as pure and unaltered due to date/time stamps and a script extension that would have
tabulated any modifications had there been any.
Semi-structured Interviews
Semi-structured interviews utilizing the Interview Protocol (see Appendix C) were
conducted to gather the teachers’ ideas, thoughts, opinions, and reflections after the twenty hours
of collaborative learning during the five half day team training sessions. The interview data was
STEM + DESIGN THINKING 92
used to gather evidence of transfer in learning and metacognitive thought. Teachers gave their
verbal consent to be audiotaped using Audionote on the iPhone before answering the questions
on the Interview Protocol. They were informed of their right to terminate the interview at any
time with no consequences. The interviews were conducted in real time, one-on-one, with the
primary researcher consistently interviewing across all interviews to establish reliability and
validity. The Interview Protocol underwent various iterations through a revision process
including peer-review, sample interviewing, and question revisions based on feedback prior to
being added to the instrumentation. The interview questions were aligned to the research
questions. Each interview was estimated to take 30 minutes depending on the length of
participant responses. The actual interviews ranged from 24 minutes to 44 minutes in length. The
interviews were uninterrupted, continuous conversations with probes to dive deeper into the
participant’s thinking whenever possible.
Emphasis was placed upon ensuring that the protocol design was tightly connected to the
research questions and purpose of the study (Merriam, 2009). The revisions to each iteration of
the interview protocol and the observation protocol were reviewed prior to use to ensure that
each element had an additive effect on the information that was gathered and aligned to the
purpose of the study.
Engineering Notebooks and Document Analysis
To create a rich data set for each teacher, document analysis was added to the
instrumentation. Document analysis consisted of qualitative coding of each participating
teacher’s Engineering Notebook created throughout the STEM team training. The Engineering
Notebooks were blank college ruled composition books that allowed teachers to take notes,
record reflections, record data during hands-on experiences, and draw diagrams of an ideal
STEM + DESIGN THINKING 93
STEM space that would be taken through the iterative engineering cycle. The idea of an
Engineering Notebook was a last minute addition to the design of the study inspired by a high
school scenario shared in the 2014 report by the National Academy of Sciences (pg. 116) that
was released two months prior to initiating the study. It was a key element of the study and
served as a strong model that teachers could implement into the classroom lesson design
throughout the year. Giving the teachers the experience of using the Engineering Notebook
themselves was expected to increase the likelihood that it would transfer into classroom practice.
Images of the Engineering Notebook can be found in Appendix J. This was an invented tool
which had not been tested for reliability and validity but was expected to and did yield invaluable
qualitative data for the research study.
Theoretical and Conceptual Frameworks
According to Maxwell (2013), the rarity of STEM integration in elementary classrooms
would qualify this study as a complex social phenomena possibly lending itself to a
phenomenological study. However, since the overarching purpose of the research was to
identify factors that might increase teacher self-efficacy and concept development, the theoretical
foundation was closer to research related to promising practices, self-efficacy, and grounded
theory. To do this, the study involved cross case analysis with grounded theory methodology.
The constant comparative method of data analysis (Merriam, 2009) in which the factors of each
teacher are constantly compared complements the inductive methods of gathering the factors and
watching them evolve into conceptual links across the teachers. The conceptual links that
identify common factors across teachers would lead to the formulation of a substantive theory
(Merriam, 2009) of factors leading to effective ideas for increasing STEM implementation in
classrooms at the elementary level.
STEM + DESIGN THINKING 94
Research Question to Instrument Alignment
Alignment of the research questions to the instruments selected to measure was critical to
ensuring validity and reliability. The instruments needed to measure what they were designed to
measure while addressing the specific research questions of the study to be purposeful and lend
evidence of reaching the expected outcomes. To alleviate the possibility of redundancy in
explaining the alignment or confusion in light of the numerous tools of data collection, Table 2
was constructed illustrating the alignment. The table was constructed as an accountability
measure for the researcher to ensure that each instrument was both necessary and additive prior
to initiating the study.

STEM + DESIGN THINKING 95
Data Analysis
The data analysis involved both quantitative and qualitative processes making it a mixed
methods research study. The Self-Efficacy for Teaching STEM Survey, Attitudes About
Collaboration Survey, and Importance of STEM Subjects Survey generated quantitative data
directly addressing the research questions. The Self-Efficacy for Teaching STEM Survey was
analyzed using SPSS statistics software to establish validity as a teacher created instrument
critical to the outcomes of the study. The open ended questions on the survey were qualitatively
coded, cross analyzed for emerging themes, then listed in a spreadsheet format for triangulation
across all data instruments to determine findings of the study.
The Attitudes Toward Collaboration Survey underwent an item by item as well as cross
item analysis. What mattered most in this data was the change in the pre and post ratings on both
an individual and collective basis. The net change per participant as well as the net average
change in attitudes toward collaboration while participating in the STEM team training were also
calculated to determine changes as a result of the shared learning team experience specifically
related to leadership potential, ability to share views openly, and collaborative efficacy among
the participants.
The Importance of STEM Subjects Survey was organized for analysis within a spreadsheet
with pre and post ratings within a single cell for comparison on a participant basis. The averages
were then calculated and the resultant change in average STEM score ratings were compiled.
The individual rankings and collective average ranking of science, technology, engineering,
math, art, and music (STEMAM) were then calculated to determine which subjects had
identifiable shifts of importance as a result of the shared learning team training and to what
degree comparatively with the others. Finally, the average STEM score was calculated for the
STEM + DESIGN THINKING 96
group to determine if the importance of the STEM subjects separately and together had any
noticeable shifts between the beginning and end of the research study.
The semi-structured interviews and Engineering Notebook analyses had an additive effect
on addressing all three research questions and were subjected to extensive qualitative coding
processes. Each was analyzed for overlapping ideas, recurring thoughts, and repetition of key
concepts or words. The researcher color-coded the responses on a participant basis then analyzed
the individual results for synthesis across participants in search of emerging themes and
outcomes addressing the research questions. A spreadsheet was then created with key
determinants for each research question along with self-reflective statements gathered from both
instruments (see Appendices I-P). This spreadsheet was then photocopied, color-coded, and
analyzed for emerging themes and ideas related to the purpose and intended outcomes of the
study.
Research Question 1: What factors influence teacher self-efficacy in teaching STEM at the
elementary level?
In order to determine the factors that influenced self-efficacy, the pre and post ratings data
from the Self-Efficacy for Teaching STEM needed to be collected. A carefully constructed set of
items rated on a scale of 1-6 involving forced choice was created by the researcher and titled the
Self-Efficacy for Teaching STEM Survey (see Appendix A). Sample items were pilot tested on
22 teachers in a school similar to the school being studied. The purpose of the instrument was to
quantifiably measure whether or not the participants in a twenty hour customized professional
development series had a change in their reported levels of self-efficacy for teaching STEM
through the engineering design process after the training. Any difference would then be analyzed
by degree of change.
STEM + DESIGN THINKING 97
The instrument was used as the pre and post assessment of self-efficacy with all eleven
participating teachers who were similar in experience and background knowledge to those on
which the instrument was piloted. All items were the same and presented in the same order and
format during both the pre and post assessments.
In order to cross analyze and add validity, the interviews and Engineering Notebook
records were qualitatively coded in search of emerging themes related to self-efficacy in teaching
STEM at the elementary level. Every item on each tool was categorized by research question,
analyzed for similar responses, and synthesized to be combined with the quantitative data of the
Self-Efficacy for Teaching STEM Survey for triangulation.
Research Question 2: What elements, experiences, or activities within a STEM team training
transfer into lesson design?
To determine the specific facets of the STEM team training that facilitated transfer to shifts
and/or inclusion in classroom instruction, the two components addressing attitudinal ratings of
the importance of STEM subjects and attitudes toward collaboration were analyzed for any shifts
or changes between and among the pre and post responses. From an inferential perspective, any
shifts could be indicative of transfer from the training setting of the old science lab room to the
individual teacher classrooms thereby impacting instructional decision making and student
exposure to the content and pedagogy of the training.
The open-response background information data from the semi-structured interviews was
analyzed for any possible variation due to number of years teaching and grade level taught. The
number of years teaching was analyzed for two specific reasons. Teachers teaching more than
twelve years had background experience in the pedagogical approaches similar to the
pedagogical approach needed for STEM integration since they had taught before the focus on
STEM + DESIGN THINKING 98
California’s CAT-6 state test scores in 2002. Prior to twelve years ago, teachers had more
freedom to design instruction based on their students rather than focusing on a state mandated
curriculum and preparing for state testing.
The grade level taught was analyzed for three specific reasons. For the past ten years, state
testing at the elementary level was only in grades 2-5. That meant teachers of kindergarten and
first grade students were not using class time to prepare for a test nor were they focusing their
instruction with the knowledge of an end of the year state test on the content. The direct result
was more freedom to make classroom instructional decisions than those tied to content delivery
being assessed at the end of the year. Since the school was a high performing school as partially
determined by state test scores, this reality could have been a factor in the results of the study. It
was also the end of the year and teachers had just completed their state testing pilot for the new
Smarter Balanced Assessment Consortium (SBAC) testing. The second reason why the grade
level taught was analyzed was the distinct difference in state adopted materials between the
foundation building years (K-2) and the content focused years (3-5). Finally, the instructional
style of teachers shifts with the grade levels from a more student centered to teacher centered
style as grade levels increase which could have affected pedagogical shifts and changes. The
number of years teaching and the grade level taught were two key variables neglected in the
analysis and discussion of all research studies reviewed by the researcher during the literature
review process.
The semi-structured interviews were then analyzed through qualitative coding in search
of emerging themes and elements specifically mentioned by the participating teachers as
impacting transfer to lesson design within their classrooms. The analysis was added to the
column titled RQ2 in the Analysis Spreadsheet found in Appendix K.
STEM + DESIGN THINKING 99
Research Question 3: What experiences or activities contribute to the development of the
conceptual understanding of engineering within STEM?
The key instruments in determining the specific experiences or activities that contributed to
the conceptual understanding of engineering within STEM were the semi-structured interview
responses and the Engineering Design Notebook notes. Secondary tools then cross-analyzed for
possible hidden variables were the Importance of STEM Subjects Survey and the Attitudes
Toward Collaboration Survey.
The Semi-Structured Interviews were transcribed from the Audionote recordings and typed
by the researcher. They were then qualitatively coded by the researcher watching for patterns of
ideas, thoughts, comments, and specific data that would emerge into themes. The process began
with inductively color coding then led to deductive coding as themes emerged. The process is
much like creating a word cloud to determine word relevance and frequency. Specific examples
gathered from each participant were typed into a spreadsheet as direct evidence addressing the
research question. The specific examples then underwent cross-analysis to determine the specific
experiences or activities that contributed to the development of the conceptual understanding of
engineering within STEM. Through this process, the researcher also uncovered specific evidence
of conceptual change in the understanding of STEM itself. This was an unexpected result but
adds depth and purpose to the results of the study. The Interviews and Engineering Notebooks
Analyses (see Appendix K) cites direct quotes and data from the interviews and Engineering
Notebooks.
In order to maximize the increased validity from triangulating the qualitative data, the
researcher used the same color-coding process with the Engineering Notebook analysis. First, all
pages of the notebooks were photocopied to enable the participants to keep their self-created
STEM + DESIGN THINKING 100
records of individualized learning. The photocopies were then color coded at the participant level
with specific evidence being added to the spreadsheet. The tabulated spreadsheet data underwent
further analyzation to determine patterns and trends across the participants that generated
inferential data as discussed in the results section of the study.
The individual STEM Subjects Survey items were analyzed independently to determine
which subject ratings may have shifted in valuation among the 1- 6 Likert-scale forced choice
options (see Appendix D) between pre and post training results of each participating teacher.
First, the ratings of the overall importance of the individual subjects were tabulated on the
spreadsheet (see Appendix G). Next, the average for pre and post ratings were calculated and
added to an additional column of the spreadsheet. Finally, the overall average change across
participants was calculated to deepen the perspective. The purpose of this was to determine
whether or not inferences could be made between quantitative participant valuation of STEM
ratings changes and qualitative evidence from the inferences and Engineering Notebook
notations.
The last question on the Importance of STEM Subjects Survey was analyzed to determine
if there were any changes within and among the following individual subjects: science,
technology, engineering, math, art, and music (STEMAM). To do this, the researcher added a
column to the same spreadsheet listing the pre and post rankings of STEMAM by participant.
Totals across the participants’ changes were calculated and analyzed independently and in
conjunction with changes in the rating scales of the subject areas.
Finally, the Attitudes Toward Collaboration Survey results were analyzed to see if there
were any possible links open to interpretive inferencing among conceptual change for STEM,
conceptual change for STEM through engineering design, and attitudes toward collaboration.
STEM + DESIGN THINKING 101
The training was a collaboratively constructed participatory experience. Any reported variation
in conceptual change related to any teachers who did not report the same or increased ratings on
the Attitudes Toward Collaboration Survey may have been collaboration related not training
related. To rule out this possibility, another spreadsheet of tabulated data was constructed by the
researcher. The five Likert-scale of 1-6 rated items on the Attitudes Toward Collaboration
Survey related to perceived decision making ability and teacher ability to work with other
teachers at the school site. Each question was analyzed in four ways: participant change in rating,
pre training mean, post training mean, and the collective change in the rating post training. Next,
the net change per participant was calculated to see the overall effect of collaborating with the
teachers across grade levels at the same school site after the twenty hours of participatory
training. Finally, the net average of the change per participant was calculated to determine the
overall effect of combining teachers across grade levels within the same school setting to
collaboratively learn in the training. All tabulated and calculated data collected can be found in
the Attitudes Toward Collaboration Survey Data in Appendix H.

STEM + DESIGN THINKING 102
CHAPTER 4
RESULTS
Purpose of the Study
The purpose of this study was to research and identify the elements of a STEM focused
experiential training series that may have been related to teachers’ self-efficacy, classroom
lesson design, and conceptual understanding of engineering within STEM at the elementary
level. Due to the participatory design of the study that included many hands-on collaborative
activities, the participants themselves were the most important element of the study. For this
reason, all instrumentation, methodology, questioning, analysis, and discussion of results had to
begin with the characteristics of the teachers themselves.
The eleven teachers participating as the training team had no prior STEM training and had
never learned or worked across grade levels before. Collectively, they were teaching grades 1-5
and RSP in May and June of 2014 when the study was conducted. Their teaching experience
ranged from 4 - 27 years with a mean of 14.9 years (see Table 3). Their school had a new
principal that year who had transferred in from being principal for two years in a different school
district. They were prompt on arrival at every team training session, participated with increased
enthusiasm, and actively engaged in all elements of the study.

STEM + DESIGN THINKING 103
Table 3
Participant Data

Logic Model Approach to Evaluation
Since the goal was to train the STEM team of teachers who had never had any training in
STEM before the study commenced and the outcomes involved changes in pedagogy and
conceptual understanding, the study was evaluated in terms of a logic model. The logic model
has been proven effective in evaluating teacher training effectiveness (Bellini, Henry, & Pratt,
2011; He, Rohr, Miller, Levin, & Mercier, 2010; Newton, Poon, Nunes, & Stone, 2013). The
goal of a logic model approach to evaluation is to explicitly identify the processes that impact
desired outcomes. This study essentially merged the two areas of evaluation research and social
science research by incorporating the logic model to aid in future STEM related teacher training
plans. As evaluation research, the variables such as the hands-on activities and other outputs are
part of a complex interaction with the outcomes. As social science research, the variables are
usually more isolated. Both evaluation and social science research employ the same
STEM + DESIGN THINKING 104
methodology while social science is typically more theoretical and evaluation more practical in
nature. This study merged the two due to the complexity of the training and content into what
Newton, Poon, Nunes, and Stone (2013) identify as applied social science research.
The purpose of the training was to apply the process of the STEM team training to
positively impact teacher self-efficacy, conceptual understanding, and conceptual change, and
classroom lesson design. According to the applied logic model, the goal was to explicitly link the
outputs with the outcomes that addressed the research questions forming the basis of this study
(Newton et al., 2013). The resources used were the inputs with the actions and activities being
the outputs. The missing element of the logic model involved the outcomes to determine the
effectiveness of the training with relation to the research questions. Tools for measuring the
outcomes specific to this study included the Self-Efficacy in STEM Survey, Importance of
STEM Subjects Survey, Attitudes for Collaboration Survey, semi-structured interviews, and
document analysis of the Engineering Notebooks. The outcomes specific to this research study
that were measured are related to self-efficacy (RQ1), changes in classroom lesson design
(RQ2), conceptual development of STEM (RQ3), and conceptual change of engineering in
STEM (RQ3). The logic model constructed for this study can be viewed in Table 4.
Table 4
Logic Model

Inputs Outputs Outcomes
Resources used:
· time
· money
· substitutes
· hands-on materials
Activities and Actions:
· explored example
and non-example
photos
· explored books and
Short, medium, and long-
term change expected:
Short-term:
· gain exposure to
STEM resources
STEM + DESIGN THINKING 105
· books
· STEM related
materials
· Technology (iPads,
projector, iPhone,
apps)
· Legos
· Robots
· Magneatos
· Kickstarter.com
· Shark Tank clips
· Project Runway
video clips
· Super Awesome
Sylvia clips
· Real classroom clips
· Cardboard
· Misc. building
materials
· Cars
· Grid paper
· Pretzels
· Graham crackers
· Ritz crackers
· Chart paper
· Markers
· Snap Circuits
· Little Bits
· ATOMS
· Led lights
· AA Battery packs
· GoldieBlox
· Photos
· Video clips
· 5 E’s model,
engineering design
process diagram, 7
cross cutting
concepts diagram
(see Appendix I)
resources
· built cars from
discarded items
· built homes for the
Three Little Pigs
· designed and
redesigned an ideal
STEM learning
space in the old
science lab
· analyzed workflows
and planning for
lesson design
· constructed puzzles
to analyze our
learning preferences
and pedagogical
approaches
· analyzed short video
clips (3-5 min) of
STEM in K-5
classrooms
· applied design
thinking to lesson
design
· used the engineering
design process to
collaboratively solve
problem based
challenges
· gain exposure to
STEM pedagogy
· develop interest in
STEM
· learn about and
develop value in
teaching STEM
through engineering
design
· learn vocabulary
needed to effectively
plan, teach, and
reinforce STEM
concepts
· work collaboratively
across grade levels
in each training
session
· build higher levels
of self-efficacy for
teaching integrated
STEM
· develop clear
conceptual
understanding of
STEM at K-5 level
· begin process of
conceptual change
for engineering in
STEM at K-5 level
· awareness and
understanding of the
engineering design
process
· awareness and
understanding of
design thinking
Medium-term outcomes:
· application of
STEM concepts to
lesson design
· application of
engineering design
concepts to lesson
design
· natural use of STEM
STEM + DESIGN THINKING 106
vocabulary modeled
and used
· understanding and
use of the
engineering design
process in lesson
design

Reporting of Results
Research Question 1 (RQ1): What factors influence teacher self-efficacy in teaching STEM at
the elementary level?
The tools that helped identify factors of the STEM team training that influenced self-
efficacy in teaching STEM for the participating teachers included the Self-Efficacy Rating Scale
and Survey, the semi-structured interviews, and the notes written in their Engineering
Notebooks. Together, the data was synthesized into specific elements of the training.
Before looking at the actual elements of the training, it was important to identify whether
there was a change in teacher self-efficacy in teaching STEM at all. The Self-Efficacy for
Teaching STEM Survey did indicate that there was an increase in self-efficacy among all
participants (see Table 5). The mean change on the self-reported rating scale of 1-6 was +31
meaning that overall the teachers reported an increase to their self-efficacy for teaching STEM
(mean = 54.36) greater than twice the value of their mean ratings before the training (mean=
16.9). There was a 37.46 point average change between the collective Pre- (16.9) and Post
(54.36) mean ratings. That equates to nearly 321% increase in average self-efficacy ratings with
the lowest increase in self-efficacy reported by Participant 2 (P2=24 point change) and the
highest increase in self-efficacy from Participant 3 (P3=50). While data illustrates the
effectiveness of the training, a reminder of the small sample size must be noted.
STEM + DESIGN THINKING 107
Table 5.

The results suggest that the teachers did report higher levels of self-efficacy as a result of
the twenty hour customized professional development meaning that the short-term outcome of
increasing teacher self-efficacy for teaching STEM in elementary classrooms was achieved. The
Wilcoxon Signed-Rank test calculated on the small sample size (n=11) indicates the critical
value of W at p≤ 0.05 was 10. Therefore, the result is significant at p ≤ 0.05. Item by item bar
graphs of collective data from the survey ratings with question level analysis depicting pre, post,
and change values were also generated. Figure 2 includes static screenshots of the data.
STEM + DESIGN THINKING 108

Figure 2. Question Analysis Graphs: Pre-, Post-, and Change in STEM Ratings on Scale of
1-6. Graphs illustrating mean self-reported ratings on 12 questions (Q1-Q12) on the Self-
Efficacy for Teaching STEM Survey before and after the training along with the change per
item.

STEM + DESIGN THINKING 109
Although the small sample size may affect the meaningfulness of the data from a

replication standpoint, differences on an item by item basis were observable. On a more

analytical level of item analysis illustrating change in self-efficacy after the training, it suggests

that the items that most influenced their increased self-efficacy were Q5 (mean change = 4.0),

Q6 (mean change = 3.64), and Q9 (mean change = 3.55). The item garnering the greatest change

(Q5) was “I can identify the 5 E’s learning cycle.”. Q6 was “I can give examples of each stage of

the 5 E’s learning cycle.” The third largest change in reported ratings was from Q9: “I can

discuss how constraints affect the outcome of an engineering design project.” The two most

change inducing items relate to the 5 E’s learning cycle which is the learning cycle impacting

pedagogy (see Appendix I). Figure 3 provides a screenshot of that mean change item analysis.

Figure 3. Mean Change Item Analysis. The graph illustrates the average change in self-
reported ratings on 12 questions (Q1-A12) on the Self-Efficacy for Teaching STEM Survey
before and after the training along with the change per item. Scale 1-6.

STEM + DESIGN THINKING 110
Digging deeper into the data, item analysis on question by participant basis comparing

pre, post, and change in self-efficacy ratings helps identify individual growth. Static images

bound to the confines of 2D tangibility can be viewed in Figure 4.

STEM + DESIGN THINKING 111
STEM + DESIGN THINKING 112
STEM + DESIGN THINKING 113

Figure 4. Self-Efficacy for Teaching STEM: Item Analysis Per Teacher Participant –
Pre/Post. Graphs illustrating the ratings pre (top) and post (bottom) training per teacher
participant (P1-P11) from the Self-Efficacy for Teaching STEM Survey. Scale 1-6. The
interactive version can be accessed via https://infogr.am/self-efficacy-for-teaching-stem-item-
analysis-prepost.

From the data in Figure 4, it can also be inferred that participant 4 (P4) made the most

growth (mean change=5) followed by participant 9 (P9 = mean change of 4.67) and participants

1 and 3 (P1 and P3) with self-reported ratings growth of 4.33 on a scale of 1-6 on the three most

influential items (Q5, Q6, and Q9). This quantitative data can be further analyzed and

triangulated with the qualitative data from these participants as gathered from the Engineer’s

Notebook and Interview data.

Since increased self-efficacy was established quantitatively, it was important to increase

the validity of that outcome through triangulation using the data collected and analyzed from the

semi-structured interviews and Engineering Notebooks as cross-analyzed in Appendices J and K.

STEM + DESIGN THINKING 114
Table 6 provides a sample of the evidence from the interviews and Engineering Notebooks

directly addressing RQ1.

Table 6
Sample of Evidence from Interviews and Engineering Notebooks for RQ1

Participants Quotes from Engineering
Notebooks:
Self-reflective statements
Quotes from Interviews:
Self-efficacy for Teaching
STEM
Coded words
and phrases
from
Engineering
Notebooks

P1 “Ah-ha! Starting every lesson
with a problem!”

“Now I will start lessons by
posing problems with
constraints.”
“Now I know more about the E
in STEM.”

How do you feel about the way
you teach now compared to how
you taught before we started?
“better but stressed (drew a
happy face) sooo much to
learn.”

Rube
Goldberg
P2 “My Ah-Ha! I will have
more hands-on. Like the grid
paper idea.”

“After seeing myself use it, I
could then picture what would
work in the classroom.”

Do you think your teaching has
changed? “Yes. I needed to see
what is involved. I needed to
participate in modeling.”

“Science always intimidated
me. I wish I had more
opportunity to show my creative
side in all subjects - not just art.

Do you feel that your
conceptual understanding of
engineering design has changed
since the day we first started?
“Yes. I didnt’ think I could do
it.”
Creating
things
puzzles

STEM + DESIGN THINKING 115

“I’m never really sure and then
give things a try and surprise
myself.”
P3 Has your teaching changed?
“more hands-on and more
open-ended”

“This will enhance my
teaching. It will help me
engage students.”
“I have alot to learn but feel I
can start and now have great
resources.”

“I learned so much about how
the engineering system works.”
5 E’s
hands-on
go explore
P4 “I knew nothing about the
engineering component of
STEM. Now I feel willing to
take risks and try projects.”

“Looking forward to starting
and incorporating engineering.”

“exciting”

“I’m more excited now.”

P5 “loved everything you
explained and all of your
energy. the learning cycle,
the difference between
scientists and engineers”

“I would like to read some of
the novels, try circuits - art
circuit was cool, snap
circuits”
“I don’t feel very smart - totally
out of my realm. I need to see
ideas and then morph or change
them”

“got me excited”

P6 “I feel I am constantly trying
to keep up with new ideas -
technology and teaching
approaches. It is nice to
know I can pull in experts as
resources in the parent
community.
(As a result of our training . . .)
“I am more open to experiment
with STEM ideas/experiments
that I don’t know how to do
myself.”

“I’m more open to different
approaches to teaching now. I
feel like I’m constantly
changing.”

“I’m more open to new ideas.”
Hands-on
STEM + DESIGN THINKING 116

“I feel I have much more
knowledge of engineering
design and how it can be
implemented in the lower
grades.”
P7 “Now I’m enthused about
science lab revamping” (after
STEM space planning)

How do you feel about the way
you teach now compared to how
you taught before we started?
“more empowered”

How confident are you that you
could help design an real STEM
exploration space at RV that
would emphasize and foster
deep learning? “very confident”

“Thank you! I’m excited.”
Learning
through
analogies
Hands-on
P8 “I loved day 3 - building 3
piggies house. And circuits
were fun too!”

“I enjoy trying new things so
that will help me to dig into
STEM. I’m not scared.”

“I’m beginning to understand
more about STEM.”

My concept of STEM has
changed “by becoming more
approachable”.

“My teaching style can be
adapted to this new style. I’m
becoming ok with chaos!”

How confident are you that you
could help design a real STEM
exploration space at RV school
that emphasizes and fosters
deep learning?
“very confident”
“I’m going to
do more
hands-on
exploration.”

“use cool
videos to
engage”

P9 “I need to use the correct “I showed the Super Awesome “Rube
STEM + DESIGN THINKING 117
vocabulary with students.”

“Some of your videos were
very inspiring. Like I want to
do this!”
Sylvia video already! My
students were glued to the
video!”

“I am a STEM teacher,
naturally! This is me!”

Goldberg
mazes for
next year!”
Chindogu
design a car
“designing a
house to hold
5 pounds”
“I’m going to
the Trash for
Teachers
store.”
P10 “I need to allow kids to
explore more and take risks
more. Think outside the
box.”

“I’m definitely more open for
exploratory learning”

“I feel more comfortable with
exploratory learning.”

“I’m excited to try engineering
projects with real world
problems

“I feel that I can actually teach
my second graders while posing
real world problems.”

P11 “I want to do more eng in my
t”

“It was very hard for me but
so fun in the end” (imp. Sq.
Cir)
“I am excited to do this.”

Cross-analyzing the item analysis by participant quantitative data of Figure 4 with the
qualitative data of Table 6 provides evidence of the perceived value of the hands-on engineering
design problem-based activities collaboratively constructed and approached through the 5 E’s
model. For example, participant 4 (P4) reported the most quantitative growth (mean change = 5)
while also stating, “I knew nothing about the engineering component of STEM. Now I feel
willing to take risks and try projects.” Participant 9 (mean change = 4.67) cited that she applied
STEM + DESIGN THINKING 118
something from the session into her classroom instruction: “I showed the Super Awesome
Sylvia video already! My students were glued to the video!” That participant also had the self-
discovery statement: “I am a STEM teacher, naturally! This is me!” which is clear evidence of
increased self-efficacy.
Interest was sparked and maintained throughout the sessions. Qualitative data from the
Engineering Notebooks analyzed as a factor of change over time indicates that interest,
vocabulary use, confidence in the use of STEM resources and the engineering design cycle,
increased use of design thinking, and self-motivation increased from the first through the last
training sessions for every participating teacher in spite of the fact that it was the end of their
school year when they were also completing report cards and preparing to wrap up their year.
Research Question 2: What elements, experiences, or activities within a STEM team training
transfer into lesson design?
The transfer of what was modeled, practiced, and learned within the STEM team training
of only 20 hours was a medium-term outcome when applied to the logic model approach to
evaluation. Therefore, the results showing that there was already direct evidence of this transfer
into lesson design were important and validated the effectiveness of the training. Training
sessions 3-5 did include modeling, analysis, and collaborative construction of lessons for
classroom implementation that required integration across all STEM disciplines.
Before the teachers could design integrated STEM with engineering design lessons
within their classrooms, they needed to have increased interest and value in all STEM subjects
including engineering. The change in STEM ratings as indicated on the STEM Subjects Survey
Data (see Appendix G) clearly illustrates that all eleven participating teachers ranked the
individual STEM subjects higher at the end of the training than they had on the first day (mean
STEM + DESIGN THINKING 119
change = +.86). Individually, every teacher showed a positive increase in the Importance of
STEM Subjects Survey with a range of .25-1.25. Collectively, the average STEM score change
was .86 of a point higher on the 6 point Likert scale. Both technology and engineering had 9/11
teachers increase their ranking as compared to the other STEMAM (STEM+art+music) subjects.
Many lesson design activities created for the training intentionally expanded STEM to
include the arts integration which the group defined to include literature, spoken word (hip hop
style poetry), music, art, media, and design. This was important to mention in light of the results
of the Importance of STEM Subjects Survey Data which indicate that 6/11 teachers ranked art
and 5/11 ranked music as less important between the first and last training session. The data
shows that the two subjects that increased in value the most were technology and engineering.
Notably, these are the two newest subjects at the K-5 levels. As they were increased in value,
another subject had to decrease to meet the structural constraint of the question type. Cross
inspection leads to the following results: 7/11 teachers who increased technology or engineering
also decreased art and music. Another distinctive fact is that the training emphasized the arts not
art alone. It is possible that the focus on STEM may have skewed the results away from the “art”
option. Perhaps that led to change in the ordinal ratings reported for art as found in the survey
data. However, it was necessary to include only the word art not “arts” in the pre and post survey
to ensure that definitional differences of “the arts” did not affect the data gathered knowing the
pre-survey data would be collected prior to any shared learning or collaborative schema
construction.

The elements, experiences, and activities within the training involved ongoing
collaboration. Therefore, it was important to assess any change in their ability to collaborate
STEM + DESIGN THINKING 120
together to eliminate that variable from the outcomes related to RQ2. Appendix H shows the five
questions and corresponding pre/post changes in ratings on an individual and collective basis.
Eight out of the eleven teachers showed an increase across all questions related to collaboration,
one showed no net change, and two showed decreases in their collaboration ratings. Upon closer
inspection of the three showing no net change or decreases, the information is more indicative of
the specific area perceived to have decreased over the timespan of the training. The teacher with
the overall net zero change reported a top score of 6 across all rating scales both pre and post
training. The teacher with net -2 rating recorded a decrease of one point on the two leadership
questions: ability to influence decisions at school and express views freely at school. The teacher
with net -4 rating recorded a decrease in all questions directly addressing the ability to
collaborate with other teachers at the school. These results may have had an impact on the ability
for the teachers to transfer the learning into lesson designing for classroom implementation.
The data most useful in identifying the elements, experiences, or activities within a
training that transfer into lesson design came from the semi-structured interviews. The teachers
were eager to share the many specific parts of the training that impacted their classroom lesson
design. For ease of reading and to reduce the cognitive load of dissecting Appendix K, Table 7
was organized to identify the specific quotes of evidence related to RQ2.
Table 7.
Sample of Evidence from Interviews and Engineering Notebooks for RQ2

Participating
Teacher
Direct items transcribed and collected from the interview data

P1 4 C’s of 21st Century Learning
Super Awesome Sylvia
STEM + DESIGN THINKING 121
crowdsourcing
kickstarter
talking about the engineering design cycle
tried the ATOMS
group activities
problem solving with constraints
5 E’s of engineering design cycle
hands-on
lots of new ideas
visuals on the board
design notebook
P2 outsource parents
kickstarter
5 E’s learning cycle
books
actual building - use of materials
creating cube connecting bridges
challenging the class to come up w/ wheels for my model car
squishy circuits
building
graphic organizers to organize ideas
hands-on
use of materials
thought provoking
“letting the children be more involved (building, touching, collaborating -
tie in w/ CGI math)
P3 5 E’s
surveying the parents about jobs and interests
“hands-on manipulatives were great How to go about teaching STEM”
squishy circuits
building w/ certain materials w/ constraints
the process for design
resources and materials and where to get them
“hands-on items make it very interesting”
P4 5 E’s
Super Awesome Sylvia
seeing it from a child’s perspective
5 R’s
kickstarter
parents for content knowledge
making the car
structures within structure
hands-on projects - using the manipulatives, making houses, cars, robots
STEM + DESIGN THINKING 122
seeing videos of your kids and class
seeing photos of your kids and class
seeing so many photos
doing the puzzle
P5 crowdsourcing
5 E’s learning cycle
vocabulary
kickstarter
Shark Tank
P6 kickstarter
7 crosscutting concepts used for planning
earthquake analogy
Tesla
read aloud chapter book
hands-on activities
snap circuits
“seeing activities from student perspective helped me to understand how
to teach it”
new vocabulary
“new approaches - more exploration with students”
more activities and PBL projects
recycling video
5 E’s learning cycle
P7 5 E’s
manipulatives
engineering books for each grade level
so many detailed examples and resources
depth and complexity prompts
using Thinking Maps to brainstorm
vocabulary - constraints
collective intelligence
global market
resources
5 E’s learning cycle
crosscutting practices NGSS CCSS
diff betw scientists and engineers
relevance/purpose - users
iteration cycle
design under constraints
crowdsourcing
so many samples and resources
kickstarter
7 crosscutting concepts
videos of your kids in classroom really doing it
STEM + DESIGN THINKING 123
seeing photos of your STEM classroom
hands-on experiments
“everything and more “
P8 crowdsourcing parents
kickstarter
abandoned power plant
5 E’s learning cycle
7 crosscutting concepts of NGSS and CCSS
read aloud books and chapter books for each grade level
circuits
“love kickstarter and will integrate it too”
so many ideas
P9 4 C’s
how to better use parent vol
5 E’s learning cycle
7 crosscutting principles NGSS
resources, books, tools
sample STEM challenges broken down into each lesson
STEM/eng read aloud chapter books
examples of ways to use eng in the classroom
def. of eng at elem level
Super Awesome Sylvia
“I introduced squishy circuits.”
the boxcar challenge - “I’m going to do that!”
real examples
P10 Super Awesome Sylvia
squishy circuits
5 E’s learning cycle
NGSS- practices, content, context
kickstarter
123D Make
building cars out of recycled materials
resources
videos
building projects
links
design notebook
puzzle activity
brainstorming systems
earthquake analogy
“breaking down the activities and showing how the transdisciplinary
approach can work at any grade”
7 cross-cutting principles NGSS

STEM + DESIGN THINKING 124
P11 squishy circuits
Super Awesome Sylvia
5 E’s learning cycle
collaborative spaces
hands-on activities
structure & sequence
vocabulary of eng design & STEM
kickstarter
shark tank
coor grid art sample shared
crowdsourcing concept for content
lego scale models
earthquake analogy
puzzle activity
7 cross-cutting principles NGSS

Photos taken on teachers’ cell phones providing evidence of lesson design already
incorporated into classroom implementation were shared with the researcher by the participating
teachers during the last two weeks. Based on these photos, it was clear that the teachers were in
fact already implementing the following elements, experiences, and activities by the 4th and 5th
training session: the 5 E’s Learning Cycle, the Engineering Design Cycle, the hands-on STEM
design challenges that they actually did, Squishy Circuits (loaned to a teacher then passed among
teachers), Snap Circuits (loaned to the school), the puzzle activity, using Legos for scale
modeling, using the iPad apps shared, reading the chapter books shared, students recording their
thinking in Engineering Notebooks, inviting parent experts in to teach STEM content, STEM and
engineering design vocabulary, and Super Awesome Sylvia as a child modeling STEM interest
and creativity.

STEM + DESIGN THINKING 125
Research Question 3: What experiences or activities contribute to the development of the
conceptual understanding of engineering within STEM?

As previously discussed, the Importance of STEM Subjects Data factors in when

discussing experiences or activities related to STEM integration and implementation.

Engineering within STEM adds complexity to the issue since STEM is embedded within the

context. The experiences or activities that contribute to the development of the conceptual

understanding of engineering within STEM could have been impacted by the individual

participant’s overall ratings and ranking shifts of STEM subject importance. Overall, every

teacher showed a change in their view of engineering both as a concept and in how it could be

woven into classroom instructional design.

In addition to the evidence cited in Appendix K, a list of quotes were collected from the

Engineering Notebooks and interviews (see Appendix J) lending further support that teachers did

reach the outcome of conceptual change for engineering in STEM.

In order to triangulate the data and offer specific ideas for implementation into future

professional development training sessions related to the three research questions addressed in

this study, all qualitative sources were also analyzed to extract additional data. Specifically,

Tables 4, 5, and 6 were analyzed by how they were qualitatively coded to extract quantitative

data related to participants’ self-initiated open-ended responses. This data can be viewed in

Figure 5 that categorizes each of the most heavily coded items shared in the training by

participant and by research question. Essentially it generated a participant x content x research

question analysis that offered even richer information for future studies and training. The most

often mentioned item was the 5 E’s Learning Model (n=28), followed by hands-on activities

(n=22), and Kickstarter (n=19). The use of the seven cross-cutting principles chart, Squishy

STEM + DESIGN THINKING 126
Circuits, and Super Awesome Sylvia videos tied (n=10) for the next most often specific elements

of the training offered by the teachers via qualitative data collection. The totals represent the total

number of unsolicited mentions and have no comparative value beyond ranking.

Figure 5. Participant x Content x Research Question Analysis of Mentions in Qualitative
Data. Table depicts the number of times each participant (P1-P11) mentioned each of the most
coded items through the qualitative data collection tools used for RQ1, RQ2, and RQ3. Totals are
the total number of mentions across the participants and can be ranked to show frequency.

Summary
The results suggest that both short-term and medium-term outcomes were achieved and
valuable evidence was gathered to address all three research questions. What surprised the
researcher the most was the richness and sheer volume of specific details offered by the
STEM + DESIGN THINKING 127
participants when analyzing and synthesizing the data. The participants were eager to share how
their interest, understanding, and perceived ability to teach STEM through the engineering
design process increased. What was most surprising was that there really were no outliers among
the data. Apart from the one teacher (P10) who cited decreased perception of the ability to
collaborate with fellow teachers at the school (see Appendix H), the other teachers appear to
have collectively increased in their levels of self-efficacy, understanding of STEM integration,
transfer of STEM integration into lesson design, and understanding of the role of engineering
within integrated STEM implementation at the K-5 level.
Teachers at the school site did work collaboratively to learn new concepts and
approaches to teaching STEM and engineering design using design thinking. Overall, the
collaboration quality and reported attitude toward collaboration improved over time. The net
change among all participants was +1.27 points on a scale of 1-6 which is a significant collective
jump after only 20 hours of training. If time permitted, the researcher would have preferred to
have this study as a longitudinal study adding classroom observations and additional sessions
throughout the year to monitor future growth.
The collective growth cuts across all data gathered. Each survey, rating scale, interview,
and Engineering Notebook provided clear and direct evidence of individual growth. The research
questions were aligned to this participant based growth knowing the teachers return after the
session to their own individual classrooms. Beginning with the focus on self-efficacy in STEM
and the increase reported by every teacher, it appears that they believe they can and will teach
with design thinking applied to STEM integration.
The results support getting teachers out of their classrooms and into spaces where they
work across grade levels to share ideas and literally crowdsource their knowledge. Most
STEM + DESIGN THINKING 128
importantly, the results provide evidence that all short-term outcomes were achieved through this
training model. A few of the medium term outcomes were even emerging at the conclusion of
the study.
This study provides specific, valuable examples of what to actually do (outputs) in a
STEM training to reach similar outcomes. Providing hands-on problem based challenges with
constraints to be addressed through and across the STEM disciplines clearly makes a difference
in the learning of the participating teachers. Using easily accessible, low cost, open source
materials rather than a set program or packaged STEM kit makes the study easily replicable in
any school regardless of budgetary constraints. Providing teachers with the opportunity to be
explorative learners appears to be key to promoting STEM integrative lesson design and
changing teacher pedagogy at the K-5 level.

STEM + DESIGN THINKING 129
CHAPTER 5
DISCUSSION OF FINDINGS, CONCLUSIONS AND IMPLICATIONS
As stated in the National Academy of Sciences report (2014), innovation requires creativity
and experimentation. This study required creativity, design thinking, and the ability to
collaboratively redesign when a problem was discovered. At the time of the study in May of
2014, little was known about STEM integration, the long-term effects, or the ways in which it
could or should be implemented (Honey et al., 2014). What was known was that teachers with
higher self-efficacy are more willing to take risks, persevere longer when faced with challenges,
set higher goals for themselves, and have more successful outcomes (Bandura, 2006; Gibson &
Dembo, 1984; Guskey, 1988; Nie et al., 2013; Rittmayer & Beier, 2009; Vardaman et al., 2012;
Zimmerman, 2000). What was known was that in order for teachers to feel more willing to
explore and experiment within the realm of STEM, research on how to build teacher self-
efficacy was vital.
Specific elements of the training that led to increased teacher self-efficacy were identified.
Since the results showed increased self-efficacy for every teacher who participated, many
inferences about the design, implementation, and instrumentation for data collection can be
made. Most importantly, leaders in school systems have ideas for building teacher self-efficacy
for teaching STEM in K-5 classrooms.
The implementation of integrated STEM within elementary classrooms was problematic
for many reasons at the time the study commenced. There were no clear definitions of terms,
concepts, or models of how successful STEM integration should look (Honey et al., 2014).
Research was just emerging, empirically based correlational data was sparse, and testing and
measurement tools specific to STEM had yet to become available (Honey et al., 2014). These
STEM + DESIGN THINKING 130
issues posed elevated challenges to researchers as they were left to individually define their
terms. Synthesizing emerging research beyond common recommendations would be difficult
until the experts in STEM established clear guidelines, terminology, and validated tools for
researchers. This study attempted to clarify the terminology and gave specific examples of how
to help teachers take on the vocabulary of STEM integration.
The multidimensionality of STEM provoked many problems as well. If the factors that
contribute to increasing teacher self-efficacy in teaching STEM could be identified, then
successful professional development opportunities could be created for existing teachers
undergoing the paradigm shift from teaching content knowledge separately to designing learning
opportunities emphasizing the engineering design cycle within STEM. This required a change
in pedagogy as well as planning lesson design. Progress in the self-efficacy of teaching across
the disciplines was needed in order for cross-pollination to positively affect student learning. If
the specific elements, experiences, or activities explored via an innovative, participatory design
approach to a STEM focused training series that lead to increased use of STEM in lesson design
could be identified, then specific recommendations for the design and implementation of training
that would provide long term site based collaboration and continued learning could be offered.
The study aimed to have an additive effect upon the previous base of knowledge related to how
and what would need to be included to reach those goals.
The purpose of this study was to research and identify the elements of a STEM focused
experiential training series that related to perceived changes in teachers’ self-efficacy, classroom
lesson design, and conceptual understanding of engineering within STEM at the elementary
level. The overarching goal of the study was to apply design thinking and the engineering cycle
to solve a real world need to discover ways to help elementary teachers integrate STEM through
STEM + DESIGN THINKING 131
engineering design within their classrooms. This study identified specific activities, tools, and
approaches that were reportedly helpful in building teacher self-efficacy (RQ1), conceptual
development for STEM (RQ3), conceptual change of engineering within STEM (RQ3), and
integrated STEM lesson design for classroom implementation (RQ2).
Discussion of Findings and Results
Research within organizational learning has found that collaborative problem solving
through shared learning teams has proven to be effective in promoting change, increased
knowledge, and increased group efficacy (Guskey, 2002; Tschannen-Moraneta et al., 2000).
From the participatory design perspective, teachers are designers of classroom instruction and
must be comfortable working in learning teams (Blair-Early, 2010). The increased collective net
average on the Attitudes Toward Collaboration Survey (+1.27) is evidence that the organization
of the team training was indeed effective at reaching desired outcomes. The individual ratings on
the survey providing evidence that nine of the eleven participants felt they could work more
collaboratively with others at their school sites regardless of grade level (net change -4 - +4) also
substantiates the value of creating learning opportunities for shared learning teams. Since the
teams had not worked collaboratively before, the two teachers who decreased their ratings of the
ability for themselves and the staff to work across grade levels may have been impacted by the
new experience. Future researchers might consider adding in some team bonding activities in the
beginning. The time constraints and time of year for the school calendar did not permit any in
this study.
The first research question of this study investigated the factors that influence teacher self-
efficacy in teaching STEM at the elementary level. Once the specific outcome of increased
teacher self-efficacy was established through the data collection and analysis process, specific
STEM + DESIGN THINKING 132
details emerged that suggest ways other researchers and professional development trainers may
be able to positively impact teacher self-efficacy for teaching STEM at the K-5 level.
Specifically, the tools, models, and activities that the participating teachers reported as helpful in
building their self-efficacy were identified.
Since goal setting was found to be a critical factor related to a teacher’s self-efficacy
(Pajares, 2002; Rittmayer & Beier, 2009), the proximal goals of recording metacognitive thought
throughout each session in Engineering Notebooks, taking risks to explore new activities and
ideas, and solving design challenges were embedded within each session (see Appendix L). The
successive opportunities for successful achievements at an incremental rate were found to be
most important in the research of Rittmayer and Beier (2009) and Pajares (2002) and were
corroborated by this research. By pacing smaller goals that could be achieved separately and
through strategic scaffolding, increased levels of self-efficacy did in fact result in helping
teachers better prepare cognitively and affectively for the next challenge. It was these
incremental challenges that were approached and achieved as smaller goals leading to larger
goals that reportedly led to self-sustained growth in efficaciousness. The teachers’ ability to
subsequently achieve successful performance and having multiple experiences focused on
mastery were aligned to the results of powerful indicators of increased self-efficacy (Carberry et
al., 2010; Chang, 2009). Therefore, scaffolded experiences that escalate in depth and complexity
over time within a shared learning team may positively affect the outcomes of a STEM teacher
training at the elementary level.
Consistent with social cognitive theory (Bandura, 1977; Bandura, 1997), applying the
constructivist approach was found to lead to more efficaciousness about teaching STEM which
also aligns to previous research by (Nie et al., 2013). Researchers have found that the level of
STEM + DESIGN THINKING 133
teacher efficacy was positively related to the openness to new pedagogies and innovative
instruction (Cheung, 2008; Gokcek et al., 2013; Nie et al., 2013). The ability to subsequently
achieve successful performance and having multiple experiences focused on mastery were found
to be powerful indicators of increased self-efficacy (Carberry et al., 2010; Chang, 2009). While
causal claims cannot be formed, it can be inferred that offering professional development for
elementary teachers emerging into the innovative territory of STEM integration that sets them up
for successful enactive experiences would benefit from dual immersion in both instructional and
self-efficacy building activities.
What was surprising was the level of detail the teachers recounted in their Engineering
Notebooks and interviews. Most surprising was the speed at which the teachers took the hands-
on activities straight to their classrooms. After each session, the trainer offered to let anyone
borrow anything that was used, shared, or left over. Teachers did indeed borrow materials each
time to test them out in their classrooms. At first, they just mentioned trying them. At week 3, a
teacher shared a photo with the principal that was later forwarded to the researcher. The photo
showed the students engaging in the same Three Little Pigs home construction challenge that the
teachers had done collaboratively in week 2. When that photo was later shared with the group,
others began to share photos they had taken in their classrooms. The conversations, willingness
to share photos, and pitch in their voices was additional qualitative evidence of increased teacher
self-efficacy.
In addition to including more hands-on activities and providing the materials for the
teachers to test the shared experiences with the children, many other specific elements of the
training series were cited by the teachers as positively impacting their belief in themselves when
teaching STEM in their classrooms. Time to explore, experiential learning, and specific models
STEM + DESIGN THINKING 134
of the learning cycle and engineering design cycle (see Appendix I) were mentioned most by
teachers. Solving design challenges, analyzing them with the seven cross-cutting principles chart,
and analyzing photos and short video clips of STEM in elementary classrooms were also
mentioned by many of the participating teachers. Reportedly, the most useful factor in helping
teachers understanding the concept of STEM and engineering design within STEM was the use
of the website kickstarter.com. It was used extensively as evidence and analogous learning
within each session due to the format. Every item on the site has to be listed citing the stages of
the engineering design process with evidence and a video pitch. Those connected to the
extensive use of clips from the television show “Shark Tank” that were used in the training. The
results in Figure 5 show that it was the second most influential factor of the training (n=19
mentions) and was mentioned by 10/11 participants. It would be interesting to see future
research design and professional development implementation that includes some of these
features as the researcher was unable to find either used in any previous studies during the
literature review process.
The second research question investigated the elements, experiences, or activities
(outputs) that affected lesson design (outcome). The researcher was investigating prior research
and assertions (Honey et al., 2014) that in order for teachers to teach integrative STEM within
their classrooms, they would first need to experience integrative STEM content and pedagogical
approaches themselves. Since the teachers’ use of integrative STEM within classroom lesson
design was null and void for all teachers prior to the training and no training was conducted
concurrently with this research study, the inference can be made that the training supported and
extended that research which was released just two months prior to the first training session.
Transfer into lesson design of vocabulary, concepts, ideas, tools, resources, and pedagogical
STEM + DESIGN THINKING 135
approaches began around week three and continued well after the conclusion of the study when
the researcher was still receiving photos from the principal as he observed the teaching himself
during walkthroughs and observations.
This second research question did prove to be a bit more challenging as it embedded a time
variable. With only five half day sessions across five weeks, the researcher was not sure there
would be enough time to actually see transfer into lesson design. Surprisingly, it was at week 3
that it started to become evident that teachers were not only talking about redesigning their
approach to lesson design and pedagogy for implementing their ideas but actually doing it. In
addition to the previously mentioned photo sharing the teachers spontaneously began doing,
there was also an increased use of vocabulary in their natural language that was evident over the
period of five weeks.
In alignment and supportive of the research by Yu, Luo, Sun, and Strobel (2012),
the roles of content knowledge, attitudes, and skills were all important in this study. The
teachers’ individual and collective self-efficacy, perceptions and conceptual understanding,
pedagogical knowledge, and content knowledge all played key roles in their learning of the
STEM concepts, content, and approaches to reach a level of transfer into classroom practice.
Of the specific features shared by the teachers, there were six that stood out week after
week and in the data collected. These included: 7 crosscutting concepts (see Appendix I), use
and understanding of the 5 E’s learning cycle (see Appendix I), Super Awesome Sylvia
YouTube videos, making and borrowing materials to use Squishy Circuits, Engineering
Notebooks, and the design thinking process as compared to the engineering design process (see
Appendix I). By the last week, every teacher had mentioned incorporating the design thinking
process, engineering design process, and the 5 E’s learning cycle into their classrooms. A few
STEM + DESIGN THINKING 136
teachers had already started student Engineering Notebooks. Some teachers had shown Super
Awesome Sylvia YouTube videos and others had borrowed and facilitated conversations around
students discovering how to make Squishy Circuits in which LED lights light up with only
playdough and a battery pack. Those Squishy Circuits were the most commonly shared photos
by the teachers within the five week training series.
Since the ultimate goal of any professional development for teachers is to see the transfer
into classroom teaching to positively impact student learning, the specific findings as shared by
the teachers themselves may be valuable in future action research and program planning for
teachers. Including some of the most cited experiences may lead to positive transfer within other
school sites since the activities are not context dependent. Important to note is the fact that all
resources cited are open source and freely available on the internet. The only one of the six that
involves any cost is the Squishy Circuits through the discovery approach. A simple and
inexpensive solution is to ask parents to send in AA batteries and playdough. A simple
conductive playdough recipe to cook it for under $3.00 is also freely available on the internet.
The only cost would be the LED lights which were purchased as a pack of 50 on a popular quick
ship delivery site for $5.00. For little or no cost at all, incorporating all six elements that
positively transferred to lesson design with these eleven teachers may be simple and cost
effective additions in future STEM professional development plans.
The third research question investigated the experiences or activities (outputs) that
contributed to teachers’ development of the concept of engineering within STEM (outcome).
This addressed conceptual understanding of STEM and conceptual change of what engineering
was and could be within STEM and in elementary classrooms. This study provided modeling
and contextual support for cognitive clarity and connection construction. In agreement with the
STEM + DESIGN THINKING 137
research of Thagard (2012), the participating teachers were able to explore misconceptions that
provoked those moments of self-solicited discovery that led to conceptual understanding.
The unanswered question was which of the hands-on activities was most beneficial. The
researcher did not explore comparatively so future research could add subcategorized questions
on surveys that could rank and delineate which are more beneficial than others.
Thinking about thinking is reflection. The Engineering Notebooks which gathered reflective
thought were cited as being one of the most influential tools in helping the teachers learn and
prepare to apply the concepts, content, and approaches to STEM integration through the
engineering design process. This was not surprising as Fulton & Britton (2011) found that well-
designed PLCs encouraged reflection and garnered positive results.
The teachers reported clear, specific examples that led to shifts in thinking. As the
Interviews and Engineering Notebooks Combined Data (see Appendix K) indicates, all teachers
had misconceptions, generalizations, and stereotypical responses about engineering prior to
participating in the study. The post survey responses provide evidence that the teachers had
different views of engineering by the end of the research study. The three most commonly
occurring words were design, process, and cycle. The teachers reorganized their schematic
representations of engineering to focus on the engineering design process often citing the 5 E’s
learning cycle and the engineering design cycle analyzed, discussed, and applied in the weekly
sessions.
Implications for Practice
Analysis and synthesis across the data and questions being studied identify five
potentially key elements: hands-on exploration based experiences, concrete design thinking and
engineering design visuals tied to the experiences, analysis of photos and video clips from K-5
STEM + DESIGN THINKING 138
STEM classrooms, shared open source free online resources, and design thinking challenges that
incorporated STEM concepts, ideas, and problems to be solved across the disciplines. Since all
of those items mentioned are free and available to any teacher anywhere in the world, STEM
learning is not bound to any specific location, demographic, or socioeconomic level. Every
teacher can be exposed to and have experience with building STEM integration capacity. This
study offers specifically detailed ways to make this practical for the future of STEM education.
Future Research
While the data appeared to provide evidence of growth for all teachers in self-efficacy for
teaching STEM, conceptual development of STEM, conceptual change of engineering within
STEM, and transfer to lesson design, there are a few recommendations for future research to
determine if the results of the small study appear in other contexts. The study placed a high
emphasis on the engineering design process of which a key step is redesign. Future research has
the opportunity to redesign this study thereby applying the exact process the study was
researching in the first place.
Due to the collaborative nature of the training, the context was a critical factor that
affected the results and replicability. While the teachers had not worked collaboratively across
grade levels before in a training, their years of working together at the same school could have
been a hidden variable to their increased collective self-efficacy for collaboration (+1.27). The
researcher did not account for this in the collected data. Future research could ascertain the
number of years the teachers had been at the particular school site. This study only asked the
total number of years teaching. The difference in that information could be a variable in their
ability to work collaboratively, learn together within the same space, freely take risks, and share
within conversations. By simply adding the additional question to the survey, the variable could
STEM + DESIGN THINKING 139
be minimized or eliminated. Since building collaborative professional capacity within the school
setting has been associated with immediate gains in teacher self-efficacy and pedagogical content
knowledge (Fulton & Britton, 2011), the collaborative element could be explored with more
depth in future research.
The teachers had no previous STEM training when the study commenced. Schools where
teachers have had STEM training of any kind may encounter different results for two important
reasons. First, if the training was poor then the teachers may need conceptual change rather than
conceptual development. This requires first unraveling misconceptions then reconstructing
schema – a two-part process. If the training was strong, then an additional instrument gathering
data on the individual and collective background knowledge of the participants would be
important to ensure the training was within the zone of proximal development for the teachers
themselves to learn.
Another factor that future research could investigate is the self-reported changes in self-
efficacy across schools that were and were not part of the strong standards push focused on year
over year increased state test scores. Schools that were heavily standards driven required their
teachers to state objectives and follow curriculum and pacing guides. Other schools allowed
more teacher driven decision making and did not require stating objectives before lessons,
completing workbooks, or following a day by day calendar of lessons. The varied contextual
elements related to years of standards based, curriculum driven, textbook centered instruction
could affect the outcomes. That didactic approach to instruction could be a barrier or perceived
barrier to transformative STEM integration.
Due to the timing in the school year, the only option for implementing the training
sessions was in a half-day format requiring substitute teachers. Bringing in substitutes requires
STEM + DESIGN THINKING 140
teacher construction of sub plans that adds to the time input within the logic model evaluative
approach. Fortunately, it did not appear to have a negative effect upon this study. However,
future researchers may want to time the study in accordance with minimizing additional teacher
time use. The end of the school year was the ideal time to conduct the study according to the
principal, teachers, and researcher who had taught K-5 for 21 years at the time of the study. State
testing was over, teachers were less stressed, teachers felt like they had a bit more time to try
new ideas at the end of the school year, and they would have the summer to extend their learning
in preparation for anticipated transfer into lesson design the following year. All of these were
the case for this school except the teacher stress factor. Teachers were just beginning their report
card period that required additional outside of school time to complete. Future researchers might
benefit from starting the study in March or April as the researcher originally intended.
The five half-day sessions were not the original intent of the design but a consequence of
the delayed start due to IRB approval and the school year ending at the end of May at the school
participating in the study. Originally and ideally, the study would have taken place in 2-hour
sessions each week for 12 consecutive weeks. This was not possible due to the school calendar
and IRB approval dates, so the engineering design process was applied to redesign the
implementation to make it work within the calendar constraints. The result was the five 4-hour
sessions. There was no net loss of content between the 24 to 20 hour modifications due to the
lack of stopping/restarting. In fact, a bonus of the redesign was more time for the design thinking
challenges that require time for exploration and risk taking as part of the engineering design
process. Future research could investigate different time structures.
Across the literature on shared learning teams, research supports models built upon
collaboration, inclusion across stakeholders including principals (Tschannen-Moran et al., 2000),
STEM + DESIGN THINKING 141
formation of small learning teams, maximizing online resources (Fulton & Britton, 2011),
incorporating lesson study (Lawrence & Chong, 2010; Sibbold, 2010 ), hands-on active learning
(Ejiwale, 2012), promoting reflection (Ejiwale, 2012), and building teacher’s sense of ownership
(Fulton & Britton, 2011). All of these facets were woven into the design of this study and can be
implemented in any future elementary school teacher training with similar outcomes in mind.
The principal was present and participating in all sessions with the teachers. Future research
could account for any impact a leader’s presence might have upon group engagement and
learning. Online resources were shared and maximized, however the time constraints did not
allow for deep exploration or curation within the study period. This could be explored further in
subsequent research. Lesson study was incorporated, hands-on active learning accounted for a
chunk of each session, and reflection was promoted through the use of the Engineering
Notebooks with stopping points throughout to provide time for such metacognitive thought.
Giving teachers this time for reflection involving note taking or journaling had strong previous
research support (Annetta et al., 2013; Darling-Hammond, 1995; Ejiwale, 2012; Mishra &
Koehler, 2006; Nadelson, Seifert, Moll, & Coats, 2012; Sellars, 2012; Yu et al., 2012). This
study expands the theoretical implications of those studies to add specific details of what and
how to include reflective thinking time.
The outputs of the training that were expected to be most effective involved project based
learning (Cotabish et al., 2013; Mishra & Koehler, 2006), inquiry (Buczynski & Hansen, 2009;
Cotabish et al., 2013; Macalalag & Tirthali, 2010), and the design cycle (Annetta et al., 2013;
Macalalag & Tirthali, 2010; Mishra & Koehler, 2006; Yu et al., 2012). This was in fact the case
for this study. The constructivist-oriented approach was also expected to lead to increased self-
STEM + DESIGN THINKING 142
efficacy (Darling-Hammond, 1995; Ejiwale, 2012; Makgato, 2012; Mishra & Koehler, 2006; Yu
et al., 2012) which was reportedly one of the key factors of the training design.
Summary
Research has emerged in support of the integration of STEM requiring teaching across the
disciplines at the elementary level (Brophy et al., 2008; Cotabish et al., 2013; Drew, 2011;
Macalalag & Tirthali, 2010; Mann, Mann, Strutz, Duncan, & Yoon, 2011; Nadelson et al., 2012).
Teachers are now expected to teach engineering concepts within an integrated STEM approach.
In order to teach engineering, the teachers must first learn to think like engineers themselves,
explore alternative ways to solve problems, and iterate based on a comparison of results to
expected outcomes or goals. Those three aspects of developing a concept of engineering were
integral to the innovative design and implementation of this study. According to Sellars (2012)
and Brophy et al. (2008), teachers are the critical change agents which explains why this study
centered around and was designed to promote change in teacher self-efficacy, conceptual
understanding, and pedagogy. This study attempted to do that within the constraints of time and
context.
The key elements according to this study included the constructive experiences, access to
free and open source materials, and the use of clear models, photos, and video clips of actual K-5
STEM classrooms. When teachers began to see themselves as successful STEM learners within
the collaborative space of the STEM team training, they began to test the ideas in their own
classrooms. When they tested the ideas, they found success which elevated their reported levels
of self-efficacy.
Teachers who originally had stereotypical responses and impressions of engineering did
redesign their own thinking to align with the engineering design process and experiences they
STEM + DESIGN THINKING 143
encountered while participating in the study. Key factors that may have promoted their
conceptual change included the heavy use of the engineering design cycle, 5 E’s learning model,
and design thinking in each training session.
Although the length of the study was only five weeks, there was evidence suggesting that
teachers were already transferring learned vocabulary, concepts, activities, and lesson design
approaches into their classroom planning and instruction. This is significant and suggests that
future researchers can explore some of the same activities and approaches for longer time frames
to determine whether increased time spent learning together leads to different levels of lesson
design transfer. The factors that the participating teachers indicated as facilitative in reaching the
outcome of transferring the new learning into lesson design included: analysis and application of
the 7 crosscutting concepts, 5 E’s learning cycle, Super Awesome Sylvia YouTube videos,
making and borrowing materials to use Squishy Circuits, Engineering Notebooks, and the design
thinking process as compared to the engineering design process.
The Self-Efficacy for Teaching STEM Survey that helped in the evaluation process of the
study may be helpful in future studies of teacher self-efficacy for teaching STEM at the
elementary level. Other tools that were found to be effective in achieving the outcomes of the
study and could be easily implemented in any future study regardless of the context include the
Engineering Notebook for reflection and the Semi-Structured Interview Protocol which gathered
rich data from the participants.
The sample population was a subset of the teaching population at a highly ranked school.
Future research could investigate the impact on an entire teaching staff , a low performing school
and/or a school in program improvement. Another option would be training in grade level teams
to maximize the potential for peer support.
STEM + DESIGN THINKING 144
The study was designed on the foundation of Bandura’s Social Cognitive Theory that
identifies four key processes in learning: attention, motivation, retention, and reproduction. The
training was built upon and collected evidence aligned to all four processes. The outcomes
indicate increased attention, motivation, transfer from working memory to longer term memory,
and transfer from practice to classroom implementation.
Successful STEM integration within K-5 classrooms will require creativity within training
(see Appendix L for this training design) and classroom application to reach levels of transfer
success within STEM. The Convergence Model of Innovation (Appendix M) could be used to
identify the observed levels of STEM integration through engineering design within classrooms
(see Appendix M). Teachers truly are the change agents who will engage in, extend, and create
the innovations for learners today and in the future.

STEM + DESIGN THINKING 145
REFERENCES
Annetta, L. A., Frazier, W. M., Folta, E., Holmes, S., Lamb, R., & Cheng, M. (2013). Science
teacher efficacy and extrinsic factors toward professional development using video games
in a design-based research model: The next generation of STEM learning. Journal of
Science Education and Technology, 22, 47-61.
Appleton, K. (2008). Developing science pedagogical content knowledge through mentoring
elementary teachers. Journal of Science Teacher Education, 19, 523-545.
Argyris, C. (2010). Organizational traps: Leadership, culture, organizational design. Oxford
University Press.
Arnone, M. P., Small, R. V., Chauncey, S. A., & McKenna, H. P. (2011). Curiosity, interest and
engagement in technology-pervasive learning environments: a new research agenda.
Educational Technology Research and Development, 59(2), 181-198.
Bandura, A. (1977). Self-efficacy: Toward a unifying theory of behavioral change.
Psychological Review, 84(2), 191.
Bandura, A. (1994). Selfefficacy. John Wiley & Sons, Inc..
Bandura, A. (1997) Self-efficacy: The exercise of control. Freeman: New York.
Bandura, A. (2006). Guide for constructing self-efficacy scales. Self-efficacy Beliefs of
Adolescents, 5(307-337).
Baum, M., & Walter, E. A. (2011). Will the iPad revolutionize education? Learning &
Leading with Technology, 38(7).
Becker, K. H., & Park, K. (2011). Integrative approaches among science, technology,
engineering, and mathematics (STEM) subjects on students' learning: A meta-analysis.
Journal of STEM Education: Innovations and Research, 12, 23-37.
STEM + DESIGN THINKING 146
Bellini, S., Henry, D., & Pratt, C. (2011). From intuition to data: Using logic models to measure
professional development outcomes for educators working with students on the autism
spectrum. Teacher Education and Special Education: The Journal of the Teacher
Education Division of the Council for Exceptional Children, 34(1), 37-51.
Bill & Melinda Gates Foundation (2009). MET Project: Working with Teachers to
Develop Fair and Reliable Measures of Effective Teaching, Seattle, WA: Bill &
Melinda Gates Foundation.
Blank, R. K (2012). What is the impact of decline in science instructional time in elementary
school? A paper prepared for the Noyce Foundation.
Björgvinsson, E., Ehn, P., & Hillgren, P. A. (2010). Participatory design and democratizing
innovation. In Proceedings of the 11th Biennial Participatory Design Conference (pp.
41-50). ACM.
Blair-Early, A. (2010). Beyond Borders: Participatory Design Research and the Changing Role
Brophy, S., Klein, S., Portsmore, M., & Rogers, C. (2008). Advancing engineering education in
P12 classrooms. Journal of Engineering Education, 97(3), 369-387.
Buczynski, S. & Hansen C. B. (2009). Impact of professional development on teacher practice:
Uncovering connections. Teaching and Teacher Education, 26(3), 599-607.
Carberry, A. R., Lee, H. S., & Ohland, M. W. (2010). Measuring Engineering Design Self
Efficacy. Journal of Engineering Education, 99(1), 71-79.
Carey, S. (1986). Cognitive science and science education. American Psychologist, 41(10),
1123-1130.
Carey, S. (2011). Concept innateness, concept continuity, and bootstrapping. Behavioral and
Brain Sciences, 34, pp 152-162
STEM + DESIGN THINKING 147
Carlile, P. R. (2002). A pragmatic view of knowledge and boundaries: Boundary objects in new
product development. Organization Science, 13(4), 442-445.
Carlile, P. R. (2004). Transferring, translating, and transformation: An integrative framework
for managing knowledge across boundaries. Organization Science, 15(5), 555-568.
Carr, R. L., Bennett, L. D., & Strobel, J. (2012). Engineering in the K12 STEM Standards of
the 50 US States: An Analysis of Presence and Extent. Journal of Engineering
Education, 101(3), 539-564.
Chan, D. W. (2008). General, collective, and domain-specific teacher self-efficacy among
Chinese prospective and in-service teachers in Hong Kong. Teaching and Teacher
Education, 24(4), 1057-1069.
Chang, Y. L. A. (2010). A case study of elementary beginning mathematics teachers’ efficacy
development. International Journal of Science and Mathematics Education, 8(2), 271-
297.
Cheung, H. Y. (2008). Teacher efficacy: A comparative study of Hong Kong and
Shanghai primary in-service teachers. The Australian Educational Researcher, 35(1),
103-123.
Chi, M. T. H. (2013). Two kinds and four sub-types of misconceived knowledge, ways to change
it, and the learning outcomes. International Handbook of Research on Conceptual Change.
Christensen, R., & Knezek, G. (2008). Self-report measures and findings for information
technology attitudes and competencies. In International Handbook of Information
Technology in Primary and Secondary Education (pp. 349-365). Springer US.
Cotabish, A., Dailey, D., Robinson, A., & Hughes, G. (2013). The effects of STEM intervention
on elementary students’ science knowledge and skills. School Science and Mathematics,
STEM + DESIGN THINKING 148
113(5), 215-226.
Cox, Cody B., and Margaret E. Beier. "The moderating effect of individual differences on the
relationship between the framing of training and interest in training." International
Journal of Training and Development 13.4 (2009): 247-261.
Creswell, J. (2009). Research design: Qualitative, quantitative, and mixed methods approaches.
SAGE Publications, Incorporated.
Crismond, D. (2001). Learning and using science ideas when doing investigateandredesign
tasks: A study of naive, novice, and expert designers doing constrained and scaffolded
design work. Journal of Research in Science Teaching, 38(7), 791-820.
Cunningham, C. M. (2004). Engineering is elementary, Boston Museum of Science. Retrieved
from http://www.mos.org/eie.
Darling-Hammond, L. & McLaughlin, M. W. (1995). Policies that support professional
development in an era of reform. Phi Delta Kappan, 76(8), 597-604.
Dede, C. (2007). Reinventing the role of information and communications technologies in
education. The Yearbook of the National Society for the Study of Education (0077-
5762), p. 11.
Doorley, S., & Witthoft, S. (2012). Make space: How to set the stage for creative collaboration.
Hoboken, N.J: John Wiley & Sons.
Dow, S. P., Glassco, A., Kass, J., Schwarz, M., Schwartz, D. L., & Klemmer, S. R. (2010).
Parallel prototyping leads to better design results, more divergence, and increased self-
efficacy. ACM Transactions on Computer-Human Interaction (TOCHI), 17(4), 18.
Dow, S. P., & Klemmer, S. R. (2011). The efficacy of prototyping under time constraints. In
Design Thinking (pp. 111-128). Springer Berlin Heidelberg.
STEM + DESIGN THINKING 149
Drew, D. E. (2011). STEM the tide: Reforming science, technology, engineering, and math
education in America. Baltimore: Johns Hopkins University Press.
Ejiwale, J. A. (2012). Facilitating teaching and learning across STEM fields. Journal of STEM
Education, 13(3), 87-94
Epstein, D., & Miller, R. T. (2011). Slow off the mark: Elementary school teachers and the
crisis in science, technology, engineering, and math education. Center for American
Progress.
Fink, A. G. (2012). How to conduct surveys: A step-by-step guide. Sage Publications.
Fulton, K., & Britton, T. (2011). STEM teachers in professional learning communities: From
good teachers to great teaching. National Commission on Teaching and America's
Future.
Gibson, S., & Dembo, M. H. (1984). Teacher efficacy: A construct validation. Journal of
Educational Psychology, 76(4), 569.
Goddard, R. D., Hoy, W. K., & Hoy, A. W. (2004). Collective efficacy beliefs: Theoretical
developments, empirical evidence, and future directions. Educational Researcher, 33(3),
3-13.
Gökçek, T., Günes, G., & Gençtürk, E. (2013). Evaluation of primary school teachers'
technological self-efficacy. International Online Journal of Educational Sciences, 5(1).
Goldschmidt, P. & Phelps, G. (2009). Does teacher professional development affect content and
pedagogical knowledge: How much and for how long? Economics of Education Review,
29, 432-439.
Guskey, T. R. (1986). Staff development and the process of teacher change. Educational
Researcher, 15(5), 5-12.
STEM + DESIGN THINKING 150
Guskey, T. R. (1988). Teacher efficacy, self-concept, and attitudes toward the implementation of
instructional innovation. Teaching and Teacher Education, 4(1), 63-69.
Guskey, T. R. (2002). Professional development and teacher change. Teachers and teaching:
Theory and Practice, 8(3/4), 381-390.
He, Y., Rohr, J., Miller, S. D., Levin, B. B., & Mercier, S. (2010). Toward continuous program
improvement: Using a logic model for professional development school program
evaluation. School-University Partnerships, 4(1), 15-28.
Hechter, R. P. (2011). Changes in preservice elementary teachers’ personal science teaching
efficacy and science teaching outcome expectancies: The influence of context. Journal of
Science Teacher Education, 22(2), 187-202.
Heddy, B. C., & Sinatra, G. M. (2013). Transforming misconceptions: Using transformative
experience to promote positive affect and conceptual change in students learning about
biological evolution. Science Education, 97(5), 723-744.
Honey, M., & Kanter, D. (2013). Design, make, play: Growing the next generation of STEM
innovators. New York, NY: Routledge
Honey, M., Pearson, G., & Schweingruber, H. (Eds.). (2014). STEM Integration in K-12
Education:: Status, Prospects, and an Agenda for Research. National Academies Press.
Hurley, M. M. (2001). Reviewing integrated science and mathematics: The search for evidence and
definitions from new perspectives [Electronic version]. School Science & Mathematics, 101(5),
259-270.
Hutchison, L. F. (2012). Addressing the STEM teacher shortage in American schools: Ways to
recruit and retain effective STEM teachers. Action in Teacher Education, 34, 541-550.
International Technology and Engineering Educators Association. (2002). Kids inventing
technology series. Reston, VA.
STEM + DESIGN THINKING 151
Johri, Aditya (2011). The socio-materiality of learning practices and implications for the field of
learning technology. Research in Learning Technology, 19 (3), 2156-7069.
Kelley, T. R. (2009). Using engineering cases in technology education. The Technology Teacher,
68(7).
Kim, S., & Chang, M. (2010). Computer Games for the Math Achievement of Diverse Students.
Educational Technology & Society, 13(3), 224-232.
Koh, J. H. L., Chai, C. S., & Tsai, C. C. (2013). Examining practicing teachers’ perceptions of
technological pedagogical content knowledge (TPACK) pathways: a structural equation
modeling approach. Instructional Science, 41(4), 793-809.
Kulkarni, C., Dow, S. P., & Klemmer, S. R. (2014). Early and repeated exposure to examples
improves creative work. In Design Thinking Research (pp. 49-62). Springer International
Publishing.
Kumar, V. (2012). 101 design methods: A structured approach for driving innovation in your
organization. John Wiley & Sons.
Lawrence, C. A., & Chong, W. H. (2010). Teacher collaborative learning through the lesson
study: identifying pathways for instructional success in a Singapore high school. Asia
Pacific Education Review, 11(4), 565-572.
Li, S. C., & Pow, J. W. C. (2011). Affordance of deep infusion of one-to-one Tablet-PCs into
and beyond classroom. International Journal of Instructional Media, 38(4)
Liu, M., Horton, L., Olmanson, J., & Toprac, P. (2011). A study of learning and motivation in a
new media enriched environment for middle school science. Educational Technology
Research and Development, 59(2), 249-265.
Macalalag, A. & Tirthali, D. (2010). Teacher professional development in elementary schools:
STEM + DESIGN THINKING 152
Improving student achievement through science and engineering. Paper presented at the
P-12 Engineering and Design Education Research Summit, Seaside, OR, August 2010.
Makgato, M. (2012). Identifying constructivist methodologies and pedagogic content knowledge
in the teaching and learning of technology. Social and Behavioral Sciences, 47, 1398-
1402.
Mann, E. L., Mann, R. L., Strutz, M. L., Duncan, D., & Yoon, S. Y. (2011). Integrating
engineering into K-6 curriculum developing talent in the STEM disciplines. Journal of
Advanced Academics, 22(4), 639-658.
Martinez, S. L. (2013). Invent to learn: Making, tinkering, and engineering in the classroom
Gary S. Stager Publishing.
Massachusetts Department of Education. (2006). Massachusetts science and
technology/engineering curriculum framework. Malden, MA.
Maxwell, J. A. (2012). Qualitative Research Design. Sage Publications.
Mayer, R. E. (2008). Applying the science of learning: evidence-based principles for the design
of multimedia instruction. American Psychologist, 63(8), 760.
Mayer, R. E. (2011). Applying the science of learning. Boston: Pearson/Allyn & Bacon.
McEwan, E. K., & McEwan, P. J. (2003). Making sense of research: What's good, what's not,
and how to tell the difference. Corwin Press.
Merriam, S. B. (2014). Qualitative research: A guide to design and implementation. San
Francisco: John Wiley & Sons.
Mintzes, J. J., Marcum, B., Messerschmidt-Yates, C., & Mark, A. (2013). Enhancing Self-
Efficacy in Elementary Science Teaching With Professional Learning Communities.
Journal of Science Teacher Education, 24(7), 1201-1218.
STEM + DESIGN THINKING 153
Mishra, P. & Koehler, M. J. (2006). Technological pedagogical content knowledge: A
framework for teacher knowledge. Teachers College Record, 108(6), 1017-1054.
Murphy, P. K., & Alexander, P. A. (2000). A motivated exploration of motivation terminology.
Contemporary Educational Psychology, 25(1), 3-53.
Mushayikwa, E. & Lubben, F. (2008). Self-directed professional development – Hope for
teachers working in deprived environments? Teaching and Teacher Education, 25, 375-
382.
Nadelson, L. S., Callahan, J., Pyke, P., Hay, A., Dance, M., Pfiester, J. (2013). Teacher STEM
perception and preparation: Inquiry-based STEM professional development for elementary
teachers. The Journal of Educational Research, 106, 157-168.
Nadelson, L. S., Seifert, A., Moll, A. J., & Coats, B. (2012). i-STEM summer institute: An
integrated approach to teacher professional development in STEM. Journal of STEM
Education, 13(2), 69-83.
Newton, X. A., Poon, R. C., Nunes, N. L., & Stone, E. M. (2013). Research on teacher
education programs: Logic model approach. Evaluation and Program Planning, 36(1),
88-96.
Nie, Y., Tan, G. H., Liau, A. K., Lau, S., & Chua, B. L. (2013). The roles of teacher efficacy in
instructional innovation: its predictive relations to constructivist and didactic instruction.
Educational Research for Policy and Practice, 12(1), 67-77.
Norris, C., & Soloway, E. (2011, November-December). The 10 barriers to technology adoption:
technology will absolutely change K12 learning. District Administration, 47(10), 92.
Pajares, F. (2002). Overview of social cognitive theory and of self-efficacy.
Pajares, F. (1996). Self-efficacy beliefs in academic settings. Review of Educational Research,
STEM + DESIGN THINKING 154
66(4), 543-578.
Pellegrino, J. W., Wilson, M. R., Koenig, J. A., & Beatty, A. S. (Eds.). (2014). Developing
assessments for the next generation science standards. National Academies Press.
Pintrich, P. R. (2003). A motivational science perspective on the role of student motivation in
learning and teaching contexts. Journal of Educational Psychology, 95(4), 667.
Puentedura, R. R. (2013). SAMR: Getting to transformation.
http://www.hippasus.com/rrpweblog/archives/2013/04/16/SAMRGettingToTransformati
on.pdf Retrieved December 29, 2014.
Pugh, K. J., LinnenbrinkGarcia, L., Koskey, K. L., Stewart, V. C., & Manzey, C. (2010).
Motivation, learning, and transformative experience: A study of deep engagement in
science. Science Education, 94(1), 1-28.
Pugh, K. J. (2011). Transformative experience: An integrative construct in the spirit of Deweyan
pragmatism. International Handbook of Research on Conceptual Change, 1101-1137.
Putney, L. G., & Broughton, S. H. (2011). Developing collective classroom efficacy: The
teacher’s role as community organizer. Journal of Teacher Education, 62(1), 93-105.
Riggs, I. M., & Enochs, L. G. (1990). Toward the development of an elementary teacher's
science teaching efficacy belief instrument. Science Education, 74(6), 625-637.
Rittmayer, A. D. & Beier, M. E. (2009). Self-Efficacy in STEM. In B. Bogue & E. Cady (Eds.).
Applying Research to Practice (ARP) Resources.
Roehrig, G.H., Moore, T.J., Wang, H.-H., & Park, M.S. (2012). Is adding the E enough? Journal
of Pre-College Engineering Education Research, 2(1), 28-34.
Rogers, E. M. (2003). Diffusion of innovation. Free Press.
Salkind, N. J. (2012). Tests & measurement for people who (think they) hate tests &
STEM + DESIGN THINKING 155
measurement. Sage.
Sargianis, K., Yang, S., & Cunningham, C. M. (2012). Effective engineering professional
development for elementary educators. Presented at the American Society for Engineering
Education Annual Conference, San Antonio, TX.
Schechter, C., & Tschannen-Moran, M. (2006). Teachers' sense of collective efficacy: An
international view. International Journal of Educational Management, 20(6), 480-489.
Schön, D. A. (1983). The reflective practitioner: How professionals think in action (Vol. 5126).
Sellars, M. (2012). Teachers and change: The role of reflective practice. Social and Behavioral
Sciences, 55, 461-469.
Shulman, L. S. (1986). Those who understand: Knowledge growth in teaching. Educational
Researcher, 15(2), 4-14.
Sikma, L., & Osborne, M. (2014). Conflicts in Developing an Elementary STEM Magnet
School. Theory Into Practice, 53(1), 4-10.
Simon, N. (2010). The participatory museum. Santa Cruz, Calif: Museum 2.0
Simonsen, J., & Robertson, T. (2013). Routledge international handbook of participatory design.
New York: Routledge.
Sinatra, G. M., & Pintrich, P. R. (Eds.). (2002). Intentional conceptual change. Routledge.
Spires, H. A., Lee, J. K., Turner, K. A., & Johnson, J. (2008). Having our say: Middle grade
student perspectives on school, technologies, and academic engagement. Journal of
Research on Technology in Education, 40(4), 497-515.
Sun, Y., & Strobel, J. (2013). Elementary Engineering Education (EEE) Adoption and Expertise
Development Framework: An Inductive and Deductive Study. Journal of Pre-College
Engineering Education Research, 3(1), 32-52.
STEM + DESIGN THINKING 156
Thagard, P. (2012). The cognitive science of science: Explanation, discovery, and conceptual
change. Mit Press.
Tschannen-Moran, M., Hoy, A. W., & Hoy, W. K. (1998). Teacher efficacy: Its meaning and
measure. Review of educational research, 68(2), 202-248.
Tschannen-Moran, M., Uline, C., Hoy, A. W., & Mackley, T. (2000). Creating smarter schools
through collaboration. Journal of Educational Administration, 38(3), 247-272.
Tuovinen, J. & Sweller, J. (1999). A comparison of cognitive load associated with discovery
learning and worked examples. Journal of Educational Psychology, 91(2), 334-341.
Van Wulfen, G. (2014). The innovation expedition: A visual toolkit to start innovation.
Amsterdam: BIS Publishers.
Vardaman, J. M., Amis, J. M., Dyson, B. P., Wright, P. M., & Van de Graaff Randolph, R.
(2012). Interpreting change as controllable: The role of network centrality and self-
efficacy. Human Relations, 65(7), 835-859.
Vosniadou, S. (2013). Conceptual change in learning and instruction: The framework theory
approach. International handbook of research on conceptual change, 11-30.
Wicklein, R. C. (2006). Five good reasons for engineering design as a focus for technology
education. The Technology Teacher, 65(7), 25-29.
Wulf, William (2001). http://www.brynmawr.edu/womeninscience/keynoteaddress.html
Yamauchi, Y. (2012). Participatory Design. Field Informatics, 123-138. Springer Heidelberg:
New York.
Yoon, S., Evans, M. G., & Strobel, J. (2014). Validation of the teaching engineering self
efficacy scale for K12 teachers: A structural equation modeling approach. Journal
of Engineering Education, 103(3), 463-485.
STEM + DESIGN THINKING 157
Yoon, S., Pedretti, E., Pedretti, L., Hewitt, J., Perris, K., & Van Oostveen, R. (2006). Exploring
the use of cases and case methods in influencing elementary preservice science teachers'
self-efficacy beliefs. Journal of Science Teacher Education, 17(1), 15-35.
Yoon, S. Y., Diefes-Dux, H., & Strobel, J. (2013). First-year effects of an engineering
professional development program on elementary teachers. American Journal of
Engineering Education, 4(1), 67-84.
Yoon, Y. S., & Griffin, M. G. (2012). Development of the teaching engineering self-efficacy
scale (TESS) for K-12 teachers. In Proceedings of the 2012 American Society for
Engineering Educators Annual Conference & Exposition, San Antonio, TX.
Yu, J. H., Luo, Y., Sun, Y., & Strobel, J. (2012). A conceptual K-6 teacher competency model
for teaching engineering. Social and Behavioral Sciences, 56, 243-252.
Zimmerman, B. J. (2000). Self-efficacy: An essential motive to learn. Contemporary educational
psychology, 25(1), 82-91

STEM + DESIGN THINKING 158
APPENDIX A
STEM + DESIGN THINKING 159
STEM + DESIGN THINKING 160
STEM + DESIGN THINKING 161

STEM + DESIGN THINKING 162
APPENDIX B
TEACHER CRITERIA FOR PARTICIPATION

1 currently teaching full time at a public elementary school with a diverse population
2 no prior training in STEM, engineering, or engineering design
3 willing to agree to NOT attend any training or outside collaboration on STEM,
engineering, or engineering design while participating in the study
4 available to meet on the specified dates for 1.5 hours each
5 willing to collaborate with colleagues and participate through active engagement in the
STEM team training
6 willing to have the researcher observe and videotape a lesson at the beginning and end of
the study
7 willing to fill out surveys, rating scales, and participate in interviews throughout the study
8 willing to allow the researcher to take notes, photos, and collect documents
9 willing to collaborate with colleagues and participate through active engagement in the
STEM training
10 at least somewhat interested in learning about STEM (cannot assume by volunteering)
11 agree that they are volunteering and will not be compensated or otherwise rewarded in
any way for their participation or time

STEM + DESIGN THINKING 163
APPENDIX C
INTERVIEW PROTOCOL
Interview Protocol
Teacher ______________ Grade taught _____ Date ________________

Which activities or experiences were most beneficial to understanding STEM integration and
engineering design (please be specific if possible)?
Please describe how your ability to design lessons and learning opportunities that incorporate
engineering design and/or integrate STEM has changed as a result of participating in this group.
Have you already tried anything new (vocabulary, activities, approaches?)?
If so, what?
What new ideas will you try next year?
What effect do you think the Design Notebook had on your thinking or learning?
What effect do you think the redesign of the aspirational STEM exploration space had on your
thinking or learning?
Do you think your teaching has changed?
Why or why not?
How?
What did we do that will directly impact your teaching or planning next year?
What were the most interesting or exciting new ideas you now have?

What is your concept of STEM integration today?
Do you feel that your conceptual understanding of STEM has changed since the day we first
STEM + DESIGN THINKING 164
started?
Why or why not?
How?
Do you feel that your conceptual understanding of engineering design has changed since the day
we first started? Why or why not?
How?
Final thoughts
What 3 words pop into your head first to describe what we’ve done together?
How do you feel about the way you teach now compared to how you taught before we started?
How confident are you that you could help design a real STEM exploration space at your school
that would emphasize and foster deep learning?
What would you change about what we did together?
What was your favorite part? Why?
Conclusion
Those were all of the questions that we wanted to ask.
Do you have any final thoughts that you would like to share?

Thank you so much for your time.

STEM + DESIGN THINKING 165
APPENDIX D
IMPORTANCE OF STEM SUBJECTS SURVEY

STEM + DESIGN THINKING 166

STEM + DESIGN THINKING 167
APPENDIX E
SCHOOL CRITERIA FOR PARTICIPATION
1 public elementary school with a diverse population
2 in session throughout the duration of the study (before summer break)
3 agreement to collaboratively construct a STEM space with all stakeholders implementing
the concepts and pedagogical approaches learned in the STEM team training at the conclusion of
the study
4 no prior STEM or engineering training
5 no STEM or engineering training, additional outside collaboration, or school wide
learning opportunities throughout the duration of the study
6 personal interest in STEM
7 willingness to support teachers trying new ideas, taking risks, and exploring new
pedagogical approaches
8 willingness to support teachers after the study with continued professional development
in STEM
9 willingness to allow the researcher to come back monthly for ongoing STEM team
training opportunities if the teachers were interested
10 a plan for where a large space could be converted into a STEM exploration space for all
stakeholders
11 agreement to allow the researcher to collect all necessary data
12 provide a space for the STEM team training gatherings

STEM + DESIGN THINKING 168

APPENDIX F
ATTITUDES TOWARD COLLABORATION SURVEY

STEM + DESIGN THINKING 169
APPENDIX G
IMPORTANCE OF STEM SUBJECTS SURVEY DATA

STEM + DESIGN THINKING 170

APPENDIX H
ATTITUDES TOWARD COLLABORATION SURVEY DATA

STEM + DESIGN THINKING 171

APPENDIX I
MODELS USED IN STEM TEAM TRAINING

7 Cross-Cutting Concepts of the Next Generation Science Standards (NGSS)

5 E’s Learning Cycle

STEM + DESIGN THINKING 172

Engineering Design Process

STEM + DESIGN THINKING 173

APPENDIX J
ENGINEERING NOTEBOOKS

Images from two different Engineer’s Notebooks from Sessions 1 and 2.

STEM + DESIGN THINKING 174
Images from different Engineer’s Notebooks from Sessions 4 and 5.

STEM + DESIGN THINKING 175

STEM + DESIGN THINKING 176

LIST OF DIRECT QUOTES FROM ENGINEERING NOTEBOOKS
“ah-ha! starting every lesson with a problem!!”
“Now I will start lessons by posing problems with constraints”
“Now I know more about the E in STEM.”
“I really like the engineering design cycle. I’m trying to use it with writing and math.”
“My Ah-Ha! I will have more hands-on. Like the grid paper idea.”
Do you feel that your conceptual understanding of engineering design has changed since the day
we first started?
“Yes. I didn’t think I could do it.”
“I learned so much about how the engineering system works.”
“I knew nothing about the engineering component of STEM. Now I feel willing to take risks and
try projects.”
“Looking forward to starting and incorporating engineering.”
“It changed totally. I see the importance of putting engineering into the curriculum.”
STEM + DESIGN THINKING 177
“Absolutely. Looking forward to starting and incorporating engineering.”
“ask problems, imagine, plan, create, improve if your first plan doesn’t work”
“loved everything you explained and all of your energy. the learning cycle, the difference
between scientists and engineers”
“I feel I have much more knowledge of engineering design and how it can be implemented in the
lower grades.”
“I loved day 3 - building 3 piggies house. And circuits were fun too!”
“My teaching style can be adapted to this new style. I’m becoming ok with chaos!”
“I’m definitely more open for exploratory learning”
“I’m excited to try engineering projects with real world problems
“I need to allow kids to explore more and take risks more. Think outside the box.”

STEM + DESIGN THINKING 178
APPENDIX K
INTERVIEW AND ENGINEERING NOTEBOOK DATA

STEM + DESIGN THINKING 179
STEM + DESIGN THINKING 180
STEM + DESIGN THINKING 181

STEM + DESIGN THINKING 182
APPENDIX L
STEM TRAINING ORGANIZATIONAL BIG IDEAS AND ACTIVITIES

STEM + DESIGN THINKING 183

STEM + DESIGN THINKING 184
APPENDIX M
MODEL OF INNOVATION

Abstract (if available)

Abstract The purpose of this study was to research and identify which elements of a collaborative, hands‐on series of STEM training sessions may have related to perceived changes in teachers’ self‐efficacy, classroom lesson design, and conceptual understanding of engineering within STEM at the elementary level. The study involved eleven teachers of grades 1-5 from a suburban public school in Southern California who volunteered to participate in the research in May 2014. The results provide specific activities and processes that led to increased self‐efficacy for teaching STEM, conceptual development for STEM, and conceptual change related to engineering within STEM at the elementary levels. Prior to this study, a tool to determine changes to teacher self‐efficacy for teaching STEM through the engineering design process was not available. This research utilized a modified version of existing tools which could be used in further research investigating teacher self‐efficacy for teaching STEM. Specific tools, resources, and activities that contributed to the growth as identified through the data analysis were also identified. A new model converging disciplines related to promoting innovative change within organizations also emerged which could be analyzed in future research for application to STEM integration at the K-5 level.

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

Preservice teacher preparation for engineering integration in K-5

PDF

The effects of math anxiety and low self-efficacy on students’ attitudes and interest in STEM

PDF

An examination of K-12 STEM integration by combining science inquiry with engineering design

PDF

One Hawai’i K-12 complex public school teachers’ level of computer self-efficacy and their acceptance of and integration of technology in the classroom

PDF

Impact of participation in STEM organizations and authentic learning experiences on women of color engineering students

PDF

The relationship of students' self-regulation and self-efficacy in an online learning environment

PDF

The relationship between levels of expertise, task difficulty, perceived self-efficacy, and mental effort investment in task performance

PDF

Exploring primary teachers' self-efficacy and technology integration in early reading instruction

PDF

The relationship between culturally responsive self-efficacy and attitudes of trauma-informed practices among K-12 educators

PDF

Technology integration and self-efficacy of in-service secondary teachers in an international school

PDF

Teacher self-efficacy and instructional coaching in California public K-12 schools: effective instructional coaching programs across elementary, middle, and high schools and the impact on teacher...

PDF

Professional learning communities: the role of school principals, district directors, assistant superintendents, and superintendents in developing collective efficacy in public secondary school...

PDF

Embracing the challenge of growing the “T” in STEM and its role in teaching and learning: a case study

PDF

Indonesian teachers' adoption of technology in the K-12 classroom: a TAM-based quantitative study

PDF

An investigation on the integration of science and literacy for English language learners

PDF

An exploration of the effect of the development of spatial awareness as a prerequisite science, technology, engineering, and math (STEM) skill on the STEM gender achievement gap: an innovation study

PDF

How does the evidence-based method of training impact learning transfer, motivation, self-efficacy, and mastery goal orientation compared to the traditional method of training in Brazilian Jiu-Jitsu?

PDF

The impact of globalization, economics and educational policy on the development of 21st century skills and STEM education in Costa Rica

PDF

Keeping principals at the top of the class: an innovation study of principal leadership, self-efficacy, and the pursuit of leadership development

PDF

21st century teaching and learning with technology integration at an innovative high school: a case study

Asset Metadata

Creator Callella, Trisha (author)

Core Title STEM + design thinking training: investigation of perceived changes in self‐efficacy, pedagogy, and conceptual development at the K-5 level

School Rossier School of Education

Degree Doctor of Education

Degree Program Education (Leadership)

Publication Date 04/21/2015

Defense Date 03/02/2015

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag 5 E's,Callella,conceptual change,conceptual development,design thinking,elementary,Engineering is Elementary,Integration,K-5,kickstarter,OAI-PMH Harvest,pedagogy,Primary,Project Lead the Way,Project Runway,self‐efficacy,Shark Tank,Steam,STEM,Super Awesome Sylvia,Trisha Callella

Format application/pdf (imt)

Language English

Contributor Electronically uploaded by the author (provenance)

Advisor Freking, Frederick W. (committee chair), Maddox, Anthony B. (committee member), Rueda, Robert (committee member)

Creator Email trishacallella@gmail.com

Permanent Link (DOI) https://doi.org/10.25549/usctheses-c3-555213

Unique identifier UC11301524

Identifier etd-CallellaTr-3348.pdf (filename),usctheses-c3-555213 (legacy record id)

Legacy Identifier etd-CallellaTr-3348.pdf

Dmrecord 555213

Document Type Dissertation

Format application/pdf (imt)

Rights Callella, Trisha

Type texts

Source University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Access Conditions The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the a...

Repository Name University of Southern California Digital Library

Repository Location USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA