University of Southern California Dissertations and Theses

Application of professional learning outcomes into the classroom: an evaluation study

(USC Thesis Other)

Application of professional learning outcomes into the classroom: an evaluation study

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content Running head: APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

1

APPLICATION OF PROFESSIONAL LEARNING OUTCOMES INTO THE CLASSROOM:
AN EVALUATION STUDY

by

Annie Rinaldi

________________________________________________________________________

A Dissertation Presented to the
FACULTY OF THE USC ROSSIER SCHOOL OF EDUCATION
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulfillment of the
Requirements for the Degree
DOCTOR OF EDUCATION

May 2018

Copyright 2018 Annie Rinaldi

APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

2
ACKNOWLEDGEMENTS
“I must finish what I’ve started, even if, inevitably, what I finish turns out not to be what I
began.” — Salman Rushdie
In 2015, I was presented with an opportunity by my dear friend and colleague, Rosanne
Lampariello Cameron, to join her in pursuing an Ed.D. in Organizational Change and Leadership
at the University of Southern California, and I politely turned her down. However, her
persistence to persuade me to become a practitioner-scholar finally wore me down, and I applied
to the program after Rosanne lovingly dragged me out of my house to her office one morning.
When I did get my letter of acceptance, I was guardedly happy about it and at the same time
overwhelmed by what I knew would be an arduous road ahead of me. How does one balance
being a full-time educator and a full-time student? There were many times when I just wanted to
quit, but my comadre Rosanne was there sitting beside me during countless sleepless nights
saying, “We got this!” I learned so much during this study, and I am indebted to her.
I am also incredibly grateful to my parents, Nino and Angela, who sacrificed so much to
come to America and instill in me a love of learning as I grew up. My parents are over the moon
that I took my learning to this level.
While working on my study, I was supported by friends and family who motivated me to
continue even when my study veered at one time in so many different directions. Glenn Kenny
was always there to make sure that my writing had flow, and I not only improved my writing, but
I made a dear friend. A big thank you to Donnie Salamanca for making sure that I was “in the
zone” and to stay there even when I wanted an excuse to stop. I also appreciate from the bottom
of my heart “mi gente,” Don Crean, Becky Davis, John Halpin, Ruth Hensley, Katherine Nelson,
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

3
Lisa Okamoto, Margery Weller, and Jeanne Woo for keeping me sane through the whole
process.
My team at work was also so incredible these past three years as I shared with them my
learning, and they encouraged me every step of the way. My boss, Lois Klein, was also so
supportive and cheered me on days where I could not see the light. I would also like to thank my
mentor and coach, Lou Stewart, for pushing me to stretch and grow as a practitioner-scholar.
I also raise my glass of appreciation to the Tuesday Trojans who welcomed me into their
group during the second semester. Andy Surber, Arvind Ramakrishnan, Chris Riddick, Erika
Maldonado, Melissa Singh, Rosanne Lampariello Cameron, and Stephanie George all made the
task of the KMOs an enjoyable exercise of much-needed levity. Cheers!
Finally, I am grateful to my Chairperson, Dr. Kathy Stowe, who imparted “tough love,”
invaluable knowledge and feedback, which only made the work in my study more valuable
throughout the process. My committee members, Dr. Kathy Hanson and Dr. Monique Datta,
also gave me amazing additional perspectives on my writing. I have so much respect for my
committee and the many ways that they helped me through this study.
I am so glad that I made this journey at the University of Southern California. The
program was excellent and it has given me a foundation to continue my work in education in
working with teachers to continuously provide excellent learning opportunities for all students.
Fight On!

APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

4
TABLE OF CONTENTS
Acknowledgements 2
List of Tables 6
List of Figures 7
Abstract 8
Chapter 1: Introduction 9
Introduction of the Problem of Practice 9
Organizational Context and Mission 11
Organizational Performance Status/Need 12
Importance of the Organizational Evaluation 13
Organizational Performance Goal 14
Goal of the Stakeholder Group for the Study 15
Related Literature Review 16
Purpose of the Project and Questions 17
Conceptual and Methodological Framework 18
Definitions 18
Organization of the Dissertation 19
Chapter 2: Literature Review 20
Introduction 20
Professional Development and Professional Learning 20
Teacher Influence on Student Learning 21
Professional Learning and Change of Teaching Practice and Barriers 22
Professional Learning Conditions to Mitigate the Barriers to Creating a 22
Professional Learning Community
The Clark and Estes (2008) Gap Analysis Framework 24
Stakeholder Knowledge, Motivation, and Organizational Influences 25
Conceptual Framework: The Interaction of Stakeholders’ Knowledge and 37
Motivation and the Organizational Context
Conclusion 40
Chapter 3: Methodology 41
Introduction 41
Data Collection and Instrumentation 43
Data Analysis 48
Credibility and Trustworthiness 49
Validity and Reliability 51
Ethics 52
Limitations and Delimitations 54

APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

5
Chapter 4: Results and Findings 56
Participating Stakeholders 57
Results and Findings 58
Extent of Teacher Professional Learning Transference into the Classrooms 58
KMO Influences on Professional Learning Transference 67
Summary 78
Chapter 5: Discussion and Recommendations 80
Recommendations for Practice to Address KMO Influences 80
Integrated Implementation and Evaluation Plan 90
Summary 101
Limitations and Delimitations 102
Future Research 103
Conclusion 104
References 105
Appendices 115
Appendix A: Survey Protocol 115
Appendix B: Focus Group Interview Protocol 117
Appendix C: Documents and Artifacts Analysis 120
Appendix D: Informed Consent/Information Sheet 121
Appendix E: Assessment and Grading Initial Workshop Evaluation 123
Appendix F: Assessment and Grading Program Evaluation 124
Appendix G: Student Data Wall 125

APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

6
LIST OF TABLES
Table 1. Organizational Mission, Organizational Goal, and Stakeholder Performance 16
Goals
Table 2. Knowledge Influences, Knowledge Types, and Knowledge Assessments 29
Related to RMS Goals
Table 3. Motivation Influences and Motivational Influence Assessments Related to 33
RMS Goals
Table 4. Summary of Assumed Needs for Knowledge, Motivation, and Organizational 36
Issues
Table 5. Survey Results for Teachers’ Professional Learning Needs 69
Table 6. Survey Results for Teachers’ Interests of Future Professional Learning 71
Activities
Table 7. Survey Results of Teachers’ Value of Instructional Practice 73
Table 8. Survey Results on the Teachers’ Value of the Professional Learning Outcomes 74
Table 9. Survey Results of the School Providing Enough Support for Professional 76
Learning
Table 10. Summary of Knowledge Influences and Recommendations 82
Table 11. Summary of Motivation Influences and Recommendations 85
Table 12. Summary of Organization Influences and Recommendations 88
Table 13. Outcomes, Metrics, and Methods for External and Internal Outcomes 93
Table 14. Critical Behaviors, Metrics, Methods, and Timing for Evaluation 94
Table 15. Required Drivers to Support Critical Behaviors 96
Table 16. Components of Learning for the Program 98
Table 17. Components to Measure Reactions to the Program 99
Table 18. RMS Organization Success Data Dashboard 101

APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

7
LIST OF FIGURES
Figure 1. The transfer of adult learning from professional development to the 39
classroom
Figure 2. Aggregated student data of the 2016-17 RMS district STAR reading 63
benchmarks
Figure 3. Kirkpatrick New World Model 91

APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

8
ABSTRACT
This study employed a mixed methods approach to evaluate the application of professional
learning outcomes into the classroom. Professional learning in a school setting were examined
in terms of the outcomes and the motivation to use the learning into the classroom and how if at
all did it benefit students. Teachers from a small rural middle school in the Northeast part of
California responded to an electronic survey and participated in a focus group regarding learning
a specific research-based instructional practice in professional learning and applying it into their
classroom instruction. The Clark and Estes Gap Analysis Framework was utilized to assess
knowledge, motivational, and organizational influences on the application of professional
learning outcomes into the classroom. The results and findings of this study demonstrated that
teachers possessed the knowledge about the instructional practice from professional learning and
there were enough organizational support of the professional learning. The motivation needed to
apply the professional learning outcomes into the classroom was dependent on the level of
collaboration in the professional learning events. The results and findings of this study
emphasized the importance of understanding adult learning and establishing a viable professional
learning community of adult learners. A plan that identifies how to enable organization supports,
collegial trust, academic goals for students, and collective teacher efficacy to develop and build
capacity of a professional learning community.

APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

9
CHAPTER 1
INTRODUCTION
Introduction of the Problem of Practice
The U.S. Department of Education has spent nearly $3 billion each year on professional
development since 2002 under Title II-A, its teacher-quality program (National Center for
Education Statistics [NCES], 2017). The intent behind this funding was the belief that
improving teachers’ content knowledge and their understanding of effective instructional
strategies would transfer to student achievement (USDOE, 2010). However, in 2015, only 36%
of fourth graders and 34% of eighth graders in the United States read at or above proficient
levels (NCES, 2011). This shortfall has led educators to reevaluate and redefine the standards
for effective professional development and move away from how it was previously defined in the
No Child Left Behind Act (NCLB). In an era of accountability reforms and the goal to prepare
all students to be college and career ready, it is critical for schools to provide quality effective
professional development that improves teachers’ practice that leads to increases in student
learning. In 2015, the Every Student Succeeds Act (ESSA) shifted professional development to
a process of on-going continuous improvement throughout the school year that is “intensive,
collaborative, job-embedded, data-driven, and classroom-focused” (Every Student Succeeds Act
[ESSA], 2015-2016, p. 295).
Though the NCLB had its merits, it fell short of being effective because it concentrated
on preparing teachers with content through “stand-alone, 1-day, or short-term workshops” rather
than sustained and focused professional learning, and did not evaluate if the knowledge learned
transferred into the classrooms (ESSA, 2015-2016). An earlier study of teacher professional
development found that only 10% of teachers transferred the knowledge of traditional workshop
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

10
models into their practice (Bush, 1984). The body of the research illustrates the challenge of
professional development is not about learning a new instructional strategy but the successful
implementation of the new learning into the classroom (Fullan, 2001; Guskey, 2002; Joyce &
Showers, 2002; Ermeling, 2010). In fact, a survey of teachers who participated in traditional
teacher professional development showed that more than 70% stated that what they learned in
the workshops had no relevancy to their work with students (Bill and Melinda Gates Foundation,
2014; Wei, Darling-Hammond, Andree, Richardson, & Orphanos, 2009).
An examination by the Regional Educational Laboratory of 1,300 studies conducted from
1986 to 2003 to measure the effectiveness of professional development intended to improve
student academic outcomes found that only nine of those studies met the What Works
Clearinghouse standards (Yoon, Duncan, Lee, Scarloss, & Shapley, 2007). This seminal work
indicates the lack of empirical research that has been done to address effective professional
development that changes teaching practices and impacts student learning. Additionally, these
nine studies showed that transference of professional learning to the classroom improved where
teachers received more than 14 hours of professional development on a focused area of topic
regarding instructional strategies (Yoon et al., 2007). Moreover, the overall student academic
achievement improved by 21 percentile points when professional development was sustained
over a period of time (Yoon et al., 2007).
The researchers found that in order for professional learning activities to be effective in
improving classroom practice, teachers’ needs as adult learners have to be addressed. According
to Zemke and Zemke (1984), adult learners are self-directed and have been found to want to
learn collaboratively, to learn skills that apply to their work, and to have measurable goals to
assess their learning. These qualities of the adult learner lend themselves to the establishment of
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

11
professional learning (Borko, 2004; Wei et al., 2009; Darling-Hammond & Richardson, 2009;
Desimone, 2009; Garet, Porter, Desimone, Birman, & Yoon, 2001; Gulamhussein, 2013;
Guskey, 2002; Stewart, 2014; Timperley, 2008). This dissertation evaluated the elements of
professional learning as to their direct or indirect impact on student outcomes in the classroom at
the site of study.
Organizational Context and Mission
Mountain View School District (pseudonym) is a small public school in Sierra County
serving approximately 1,200 students in four schools: a comprehensive high school, a
continuation high school, a middle school, and an elementary school. The district is located in
the northern part of California, in a rural town with a total population of 7,500 full-time
residents. The student demographics of Mountain View School District (MVSD) are 65% Latino
or Hispanic, 1% African-American, 3% mixed-race, Asian American, Filipino or Native
American, and 31% White (California Department of Education [CDE], 2015). Sixty-two
percent of the students receive free or reduced lunch based on the National Lunch Food program
(CDE, 2015). Regent Middle School (pseudonym [RMS]) is the middle school in MVSD and is
the focus of this evaluation study.
RMS serves approximately 300 students of diverse backgrounds in grades six through
eight. The student to teacher ratio is 30:1 (CDE, 2015). The student demographics at RMS are
consistent with the student demographics of MVSD. According to the California Assessment of
Student Performance and Progress, only 51% of RMS students in 2016 met or exceeded on
literacy achievement level descriptors, and only 43% met or exceeded on mathematics
achievement level descriptors (CDE, 2017). These results show that more than half of RMS
students did not meet literacy and mathematics achievement even though MVSD provided
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

12
professional development for teachers and administrators to promote educator effectiveness,
spending approximately $107,000 in the 2015-16 and 2016-17 school years.
RMS currently employs 31 staff members, including 15 full-time teachers and three part-
time, one library aide, two paraprofessionals and an additional six classified personnel who work
in clerical, custodial, and food service positions (CDE, 2015). All teachers are considered to be
“highly qualified” under the No Child Left Behind Act, and paraprofessionals pass a test before
being eligible for employment in the district. There is one principal at RMS, who also acts as the
district’s Title III coordinator for K-12.
The mission statement of RMS hangs on the wall of the school entrance, and reads, “…to
inspire, educate, and empower our community’s future leaders by providing a healthy, safe, and
respectful environment, which fosters the students’ academic, social, and emotional growth”
(CDE, 2015). The schoolwide instructional focus is that all students will read actively, think
critically, and strive to grow, and this message is evident on banners and posters throughout the
school. The RMS Instructional Leadership Team (ILT) is made up of the principal and one
teacher from each grade level. RMS has been working with an educational consulting firm since
August 2015 to improve student literacy.
Organizational Performance Status/Need
In order to improve the literacy achievement of all students at RMS, the teachers need to
take the knowledge of their professional learning and apply that knowledge to change teaching
practices in the classroom while closely monitoring student learning. RMS teachers and the ILT
chose student-generated questioning as the area of focus for their professional learning to
improve student literacy. If RMS staff focuses on one best practice during professional learning
and are successful in the implementation of the strategy schoolwide, then the professional
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

13
learning practices could be used to implement future best practices schoolwide. Thus, the
organizational goal is that by December 2017, all the teachers at RMS will have implemented the
instructional strategy of student-questioning schoolwide that will be learned during professional
learning activities offered during the 2017-2018 school year. The goal of this change in teaching
practice will be that at least 60% of the students will perform at or above benchmark in reading
comprehension, as demonstrated by the results of the district’s STAR reading assessment (STAR
Reading). Therefore, there is a need for RMS teachers to participate in professional learning that
is focused, on-going, schoolwide, and data-driven. Previous professional development
workshops at RMS lacked a shared vision and had no continuity from one workshop to another.
This resulted in RMS not having a schoolwide approach to impact change and improve student
literacy. This study evaluated the professional learning practices at RMS and its ability to
change teacher practices, as measured by the implementation of the student-generated
questioning technique in classrooms schoolwide.
Importance of the Organizational Evaluation
To improve the application of teacher professional learning outcomes into the classroom,
the focus of this evaluation study examined the professional learning strategies of RMS, and
their effectiveness on changing teachers’ practice in the classrooms. One particular strategy that
RMS teachers chose as their primary focus in professional learning is student-generated
questioning. The reason behind their choosing to focus on only one instructional strategy is to
better assess and measure the effectiveness of that strategy as it relates to student literacy. If the
particular professional learning practices implemented at RMS are successful with this one
strategy, then those practices could be further used with the schoolwide implementation of future
instructional practices to affect student achievement positively. The professional learning
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

14
effectiveness will be evaluated using the New Kirkpatrick Model, which focuses on the reaction,
learning, behavior, and results of the training. The New Kirkpatrick Model will be further
discussed in Chapter Five of this study (Kirkpatrick & Kirkpatrick, 2016). The goal of
implementing student-generated questioning schoolwide is to improve overall reading
achievement at RMS as measured by the district benchmark, Renaissance Place STAR Reading,
which is administered at least four times per school year. Currently, only 50% of RMS students
are at or above reading level on the district and state assessments. This lack of student reading
achievement will have a profound, long-term effect on the students as they progress from middle
school through high school, for these students will not be able to succeed in rigorous high school
courses. Research has consistently shown that students taking more rigorous coursework in high
school affects their outcomes after high school and improves their chances at social mobility
(Lucas, 2001). The demand for a more highly skilled workforce requires a higher level of
education, and thus students increase their opportunities to social mobility when participating in
higher-track courses (Royster, Gross, & Hochbein, 2015). The goal is that by implementing
student-generated questioning combined with the teachers’ participation in professional learning
activities around this practice, student achievement in literacy will increase by 10%.
Organizational Performance Goal
By December 2017, after RMS teachers have participated in professional learning
provided by the RMS principal and RMS teachers involved in the instructional leadership team
of the school, all RMS teachers will have fully implemented the instructional strategy of student-
generated questioning in their classrooms to increase student reading achievement as measured
by the district benchmark, Renaissance Star Reading Assessment. The student-generated
questioning strategy is defined as a research-based instructional strategy to develop students’
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

15
divergent (brainstorming via prior knowledge), convergent (categorizing and prioritizing), and
metacognitive (reflective) thinking abilities within the classroom (Rothstein & Santana, 2011).
To achieve this goal, the teachers will use the student-generated questioning strategy
across all content areas and will continue to participate in professional learning to reflect on their
practice and become more adept at the implementation of student-generated questioning which
targets literacy. The principal and the teachers of RMS established this goal after seeing data
that only 51% of the students met standards on the Smarter Balanced Assessment, a state
accountability assessment, in 2016. The goal will be evaluated by examining RMS teachers’
responses to survey items, data acquired from a focus group interview, an analysis of
professional learning documents like agendas and minutes from those meetings, classroom
artifacts demonstrating implementation of student-generated questioning, and an analysis of
aggregate student data from the Renaissance Learning STAR Reading assessment.
Goal of the Stakeholder Group for the Study
Due to the lack of overall student reading achievement at RMS, there is a need to develop
a plan to improve the classroom practices that address student literacy across all content areas.
Both students and teachers will benefit from having a goal that addresses the need to improve
literacy during middle school in order to offer more rigorous learning opportunities to students in
high school. Table 1 outlines the RMS mission, a global organizational goal, and two teacher
performance goals. RMS teachers will work toward achieving the organizational goal, to have
60% of all students reading at or above grade level as measured by the Renaissance Learning
STAR reading district benchmark. They will use student-generated questioning to promote
student literacy and will participate in regular professional learning activities to reflect on the
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

16
strategy and improve their practice. Therefore, the stakeholders of focus for this study were all
RMS teachers.

Table 1
Organizational Mission, Organizational Goal, and Stakeholder Performance Goals
Organizational Mission
It is the mission of Regent Middle School to inspire, educate, and empower our community’s
future leaders by providing a healthy, safe, and respectful environment, which fosters the
students’ academic, social, and emotional growth.
Organizational Performance Goal
By December 2017, all the teachers at Regent Middle School will have implemented across
all content areas the instructional strategy of student-questioning as studied during
professional learning so that at least 60% of the students are performing at or above
benchmark in reading comprehension, as demonstrated by the results of the district’s STAR
reading assessment (STAR Reading).
Regent Middle School Teacher Goal
By December 2017, 100% of RMS teachers will participate in year-long professional
learning activities to focus and implement the strategy of student-generated questioning to
promote the literacy achievement of all RMS students.

Both the organizational performance goal and the teacher goals directly address the RMS
mission of educating all students to achieve their individual potential.
Related Literature Review
Professional development for teachers is used as an educational reform to improve
student learning and student outcomes (Fishman, Marx, Best & Tal, 2003; Garet et al., 2001;
Guskey, 2002). However, the research shows that even though teachers participated in
professional development, the majority reported that it was not useful in their practice (Wei et
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

17
al., 2009). The real challenge is not learning new content or a new skill, but it is to transfer the
new knowledge into the classroom from the professional development activities (Camburn &
Han, 2015; Fullan, 2001; Joyce & Showers, 2002). In a quasi-experimental study that examined
the types of professional development that increased teachers’ self-efficacy and teachers’
successful implementation of a new instructional strategy in their classroom, it was found that
professional development that included information, practice, modeling, and coaching improved
the conditions of implementation of the new strategy (Tschannen-Moran & McMaster, 2009).
There is a need to build research that links effective professional learning to a community of
teachers who change their practice to improve student learning and student outcomes.
Purpose of the Project and Questions
The purpose of this project was to conduct a needs analysis in the areas of knowledge and
skill, motivation, and organizational resources necessary to reach the organizational performance
goal. The organizational goal for RMS is that all the teachers will have implemented across all
content areas the instructional strategy of student-questioning as studied during professional
learning so that at least 60% of the students are performing at or above benchmark in reading
comprehension, as demonstrated by the results of the district’s STAR reading assessment (STAR
Reading). The analysis includes a list of possible needs for improving professional learning
practices at RMS to better impact student learning. A complete needs analysis will focus on all
RMS stakeholders, and for practical purposes the stakeholders in this study are the RMS
teachers.
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

18
As such, the questions that guide this study are the following:
1. To what extent are RMS teachers transferring what they learned in professional
learning into the classroom, as measured by the implementation of student-generated
questioning in the classroom?
2. What are the knowledge, motivation, and organizational influences that hinder or
support the transference of professional learning outcomes into the classroom?
3. What are the recommendations for organizational practice in the areas of knowledge,
motivation, and organizational resources?
Conceptual and Methodological Framework
This study used Clark and Estes’ (2008) gap analysis, which is a systematic, analytical
method that helps to clarify organizational goals and identify the gap between the actual
performance level and the preferred performance level within RMS. This study also used a
needs analysis as the conceptual framework. The methodological framework that was utilized is
a mixed method study. Assumed knowledge, motivation, and organizational influences were
generated based on student performance data, personal knowledge, and related literature. These
influences were assessed by using a survey, focus groups, literature review, and content analysis.
Research-based solutions are recommended and evaluated in a comprehensive manner.
Definitions
For the purpose of establishing clarity and consistency, the following section defines the
terms that will be used throughout this study.
Adult Learning Theory or Andragogy. According to Knowles (1970), adult learning
theory suggests that the best learning environment for adults include activities that are
collaborative and use a problem-based approach.
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

19
Literacy. Literacy is competency in reading and writing, which includes the ability to
decode the written word, construct the meaning of the text, and produce writing.
Professional learning. Professional learning includes opportunities that allow educators
to engage with other educators and grow professionally.
Student-generated questioning. Allows students to develop a practice of inquiry while
learning (King, 1992).
Organization of the Dissertation
This dissertation is organized into five chapters. This chapter provided the reader with
the key concepts and terminology commonly found in a discussion about teacher professional
development and professional learning to change teacher practice and improve student learning.
This chapter also provided RMS’s mission, goals, and stakeholders and asserted the initial
concepts of gap analysis adapted to the needs analysis. Chapter 2 offers a review of the current
literature surrounding the scope of the study. Topics of adult learner theory, teacher professional
development, and the implementation of professional learning communities will be addressed.
Chapter 3 details the assumed needs for this study as well as the methodology underlying the
choice of participants, data collection, and analysis. In Chapter 4, the data and results are
assessed and analyzed. Chapter 5 provides solutions, based on data and literature, for closing the
perceived gaps as well as recommendations for the implementation of an evaluation plan for the
proposed solutions.

APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

20
CHAPTER 2
LITERATURE REVIEW
Introduction
Chapter 2 provides an outline of teacher professional learning literature, the emergence of
professional learning as the model of professional development, and the connection to the
optimal conditions of creating a professional learning community as a way to impact student
learning and student outcomes. The first section focuses on a historical overview of factors that
lead to professional learning and student learning and student outcomes. The second segment
addresses the features of professional learning strategies that influence teachers and change their
practice. The chapter ends with a gap analysis to examine the barriers that teachers face in
implementing instructional strategies learned during professional learning to positively influence
students’ literacy, through the dimensions of knowledge, motivation, and organization.
Professional Development and Professional Learning
With the Every Student Succeeds Act (ESSA) that passed in 2015 and governs the United
States K–12 public education policy, there is currently a shift of how teacher professional
development is delivered to teachers, transitioning from stand-alone workshops to a cohesive,
collaborative program that is school site based that takes into consideration the teacher as an
adult learner in a professional learning community. The belief is that effective teacher
professional development will change teacher quality and subsequently impact student learning
for improved student achievement (McLaughlin & Talbert, 2006). However, there is little
empirical evidence that professional development or professional learning communities achieves
these goals (Borko, 2004; Desimone, 2009). In order to measure the effect of professional
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

21
learning on both teacher and student learning and outcomes, there needs to be a consensus on
what constitutes professional learning.
According to Joyce and Showers (2002), effective professional learning includes the
provision of new information, modeling, practice opportunities, performance feedback, and
coaching. Other features include the duration of support (Bill and Melinda Gates Foundation,
2014; Guskey & Yoon, 2009); supporting teacher beliefs of the instructional strategy being
learned to implement in the classroom (McLeskey & Waldron, 2004); and treating teachers as
adult learners, who are part of a professional community (Slayton & Mathis, 2010). Creating the
professional learning community for teachers also includes its set of attributes, such as
establishing collective efficacy, trust, a belief that teacher and student learning are linked, and a
commitment to continuous improvement (Grossman, Wineburg, & Woolworth, 2001). It is
important to evaluate and improve professional learning for its impact on student learning and
outcomes.
Teacher Influence on Student Learning
Students in K-12 educational settings spend the majority of their day with teachers, and
the influence of what teachers do and know is a significant influence on student learning (Hattie,
2009; Rivkin, Hanushek, & Kain, 2005; Rockoff, 2004). Therefore, teacher preparation,
retention, and continuing professional development are key strategies for improving student
learning (Hattie, Masters, & Birch, 2016).
When effective professional development does not prepare a teacher, the results are that
students’ learning and outcomes are impacted negatively. Aaronson, Barrow, and Sander (2007)
examined teachers and student achievement in the Chicago public high schools in a value-added
model, which estimated a teacher’s effectiveness by determining a student’s future achievement
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

22
on standardized tests with their performance on past tests. A student’s teacher then adds value or
decreases value to the student’s performance. The findings of the study showed teachers
positively or negatively impact students’ achievement in ninth grade math by one standard
deviation. However, though the study proved that teachers do affect student achievement, this
study did not observe the teacher characteristics that positively contributed to student
achievement.
Professional Learning and Change of Teaching Practice and Barriers
Specific conditions or features have proven effective in professional learning to change
teaching practices that promote student achievement. The most important condition is to foster a
professional community within the school, taking into consideration that teachers are adult
learners. It is not easy to create a professional community around change because teachers do
not want to take risks for fear of the change not working (Le Fevre, 2013), there is a general lack
of trust with the process (Grossman et al., 2001), and there is traditionally not enough time set by
the organization to engage in a cycle of inquiry with other teacher professionals (Grossman et al.,
2001; Horn & Little, 2010).
Professional Learning Conditions to Mitigate the Barriers to Creating
a Professional Learning Community
Effective Professional Learning Conditions
Specific conditions have proven effective in establishing professional learning
communities. Because teachers spend most of their time at their school sites, it is important to
establish the community at the school and that leadership sets aside specific time for the
professional learning community to meet.
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

23
Adult learner. Knowles (1970) describes the adult learner under andragogy theory as
someone who is involved in her own learning, comes with prior experience, has learning needs
that are relevant to the learner’s work, and is problem-centered or goal-oriented rather than
content-centered. From these principles, professional learning community needs can be
addressed when creating the most effective conditions for adult learners to transfer their work in
professional learning to the classroom.
Duration. Professional learning that is longer and ongoing and allows teachers time to
learn a new strategy and build capacity for the implementation of that strategy has a greater
impact on student learning (Wei et al., 2009).
Support. Increasing the time of professional learning is not enough, if it is not sustained
with support from the professional community. Support in professional learning includes
modeling, coaching and feedback (Desimone, Porter, Garet, Yoon, & Birman, 2002; Garet et al.,
2001; Joyce & Showers, 2002), and also creating an environment of trust where dialogue and
reflection about practice can take place (Camburn & Han, 2015; Slayton & Mathis, 2010).
Active learning. Professional learning should include meaningful activities where
teachers deepen their knowledge and improve their instructional practices (Borko, 2004). Active
learning activities can include hands-on activities, reflection, and collaboration.
Data-Driven Decision-Making. The Data-Driven Decision-Making (DDDM) model
includes a continuous cycle of collecting different types of data to then organize and synthesize
knowledge that can be used to make decisions, such as setting goals, addressing needs, and
evaluating the effectiveness of teaching practices to improve student outcomes (Marsh, Pane, &
Hamilton, 2006).
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

24
Content focused. Research shows that content focused forms of professional learning
that are job-embedded can support and change teacher practice in the classroom (Camburn &
Han, 2015). When the professional learning connects with teachers’ work, there is an increase in
collaboration among the teachers (Darling-Hammond & Jaquith, 2012), and it improves the
success of transferring what is learned in professional learning to be implemented in the
classroom (Wei et al., 2009). Furthermore, this connection and collaboration create a long-term
commitment to an instructional focus that not only focuses on individual teacher improvement
but also on overall schoolwide improvement (Elmore, 2002).
Collective participation. There is a correlation between professional learning activities
and group competency. Goddard, Hoy and Hoy (2000) surveyed over 400 teachers who worked
in 47 elementary schools within one urban school district and they found that collective teacher
efficacy showed a positive impact on student achievement.
The Clark and Estes (2008) Gap Analysis Framework
This mixed-methods evaluation study used Clark and Estes’ (2008) gap analysis, a
systematic, analytical method that helps to clarify organizational goals and identify the gap
between the actual performance level and the preferred performance level within the
organizational teacher stakeholder group at RMS. The gap analysis conceptual framework will
be used to address improving the application of professional learning outcomes into the
classroom to then improve overall middle school students’ reading comprehension levels. The
methodological framework will employ a mixed-methods approach to collect data. Assumed
knowledge, motivation, and organizational influences that interfere with organizational goal
achievement will be generated based on personal knowledge and related literature. These
influences were by using survey, focus groups and interviews, literature review, and content
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

25
analysis. Research-based solutions were recommended and evaluated in a comprehensive
manner.
Stakeholder Knowledge, Motivation, and Organizational Influences
The stakeholder goal of RMS is for all the teachers to implement the instructional
strategy of student-questioning learned during professional learning activities to influence the
literacy achievement of all students positively. To successfully transfer the strategy of student
questioning from professional learning to change classroom practices, teachers will need to
enhance their knowledge and skills. According to Clark and Estes (2008), it is important for an
organization to provide the knowledge and expertise to its stakeholders to achieve the
organization’s goals. To provide the new knowledge and expertise, RMS will need to provide its
teachers the appropriate training. Effective training includes coaching and feedback to
successfully implement a new teaching strategy in their classrooms (Clark & Estes, 2008). This
section provides a review of the relevant literature that deals with the knowledge necessary to
accomplish the stakeholder goal of implementing the student questioning strategy to influence
student literacy achievement positively.
The teachers, the stakeholder group for this study, must understand the strategy and its
use in improving student learning in the area of literacy. Then the teachers need to use the
strategy in their classrooms. Finally, teachers must reflect on their teaching and the use of the
instructional strategy to not only improve their instruction but also to develop a collective
organizational efficacy with the schoolwide use of student questioning.
The analysis of literature will focus on the conceptual topic of effective professional
learning to change teacher practice, the use of student questioning as part of the professional
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

26
learning, and the metacognitive topic of reflection to build collective efficacy among all the
teachers.
Knowledge Influences
According to Krathwohl (2002), the four different types of knowledge are factual,
conceptual, procedural, and metacognitive. He explains factual is the knowledge required for
certain subjects; conceptual is the ability to classify, understand principles, make generalizations,
and develop theories; procedural is the knowledge used to perform specific skills within the
subject, and metacognitive is the awareness of one’s cognitive strengths and weaknesses when
problem-solving (Krathwohl, 2002). Furthermore, in metacognition, the learners possess an
awareness and control of their cognitive processing, referred to as self-regulatory learning
(Baker, 2008; Mayer, 2011; Pintrich, 2003; Zimmerman, 1989). The four knowledge types are
useful to design specific learning outcomes and then evaluate the proficiency performance.
Importance and benefits of using student-generated questioning. Conceptual
knowledge influences a teacher’s ability to apply the strategy of student-generated questioning.
Though there is not a great deal of research on teacher practice of using student-generated
questioning in the classroom, the importance of student-generated questioning can be observed
through the framework of promoting self-regulation and metacognition in learning (Wong,
1985). The importance of this instructional strategy is evident in the study that Cano García,
García, García-Berbén, Pichardo Martínez, and Justicia Justicia (2014) conducted which
examined 449 ninth graders enrolled in science classes to show a correlation between the
students’ learning approaches and reading comprehension with levels of prior knowledge and the
practice of student-generated questions. The results suggested that improving students’ quality
of self-generated questions improved students’ self-regulation in reading comprehension.
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

27
Understanding the practice of student-generated questioning for self-regulated learning in
reading comprehension is necessary for teachers when establishing student learning outcomes
(Cano García et al., 2014). King (1992) found that student-generated questioning resulted in
better student performance as compared to summarizing and note-taking. The process of
student-generated questioning is important for teachers to value because it allows their students
to become critical thinkers.
Understanding how to explicitly use student-generated questioning in the classroom.
Procedural knowledge refers to the necessary information that is needed to accomplish certain
tasks and participate in certain activities to reach a specific outcome (Krathwohl, 2002).
Teachers completely understanding the process of teaching students to ask and generate
questions is important because it helps the students become aware of important points in the text
(Wong, 1985). There has been research examining the benefits of teachers using the higher-
order questioning during instruction on student comprehension. However, few studies have
examined the benefits of student-generated questions and higher-order questioning on learning.
In a literature review article on teachers’ roles in self-regulated learning, many of the reviewed
studies indicate a strong correlation between teachers’ explicit instruction of self-regulation
learning and the resulting positive effects on student performance (Moos & Ringdal, 2012).
In a qualitative study of 105 graduate students who were K-12 teachers, the results of the
study found a close relationship between the teachers’ knowledge of metacognition, an integral
part of self-regulation, and the teachers’ understanding of teaching metacognition to students
(Wilson & Bai, 2010). Therefore, the more knowledge the teachers have about self-regulation,
then the more likely that it will be taught to the students. Middle school teachers need to have
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

28
the knowledge and skills to apply and to facilitate student-generated questioning as a strategy of
self-regulation to improve overall reading comprehension.
Teacher reflection on classroom practice of student-generated questioning.
Metacognitive knowledge refers to the awareness and understanding of one’s state, evaluating
one’s strengths and challenges, and self-awareness (Krathwohl, 2002). When teachers are
implementing the practice of student-generated questioning in the classroom, it is important that
teachers self-reflect and self-regulate on the practice as they become better at it. In a study using
video analysis of 17 elementary school teachers as a means of self-reflection, the results showed
marked improvement in teachers’ use of self-regulated learning practices in the classroom
(Dignath-van Ewijk & van der Werf, 2012).
When teachers take the time to examine and reflect on classroom practices and the impact
of those practices on students, they can make informed decisions about what they need to do or
change in their practice (Elmore, 2002). Rueda (2011) also states that teachers must use a
metacognitive process to examine their practice and make the necessary adjustments to improve
their teaching.
Table 2 summarizes the knowledge influences and assessments for the stakeholder goal at
RMS. The literature supports the importance of knowledge influences on teachers’ goals. In
summary, when RMS teachers understand the effectiveness of student-generated questioning on
promoting student literacy, learn how to implement student-generated questioning in the
classroom, and reflect on their teaching of the strategy with the students, the implementation of
the instructional strategy will be successful and there will be more of a likelihood that the
stakeholder goal will be met.

APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

29
Table 2
Knowledge Influences, Knowledge Types, and Knowledge Assessments Related to RMS Goals
Organizational Mission
The mission of Regent Middle School is to inspire, educate, and empower our community’s
future leaders by providing a healthy, safe, and respectful environment, which fosters the
students’ academic, social, and emotional growth.
Organizational Global Goal
By December 2017, all the teachers at Regent Middle School will have implemented across
all content areas the instructional strategy of student-questioning as studied during
professional learning so that at least 60% of the students are performing at or above
benchmark in reading comprehension, as demonstrated by the results of the district’s STAR
reading assessment (STAR Reading).
Stakeholder Goal
By December 2017, 100% of RMS teachers will participate in year-long professional
learning activities to focus and implement the strategy of student-generated questioning to
promote the literacy achievement of all RMS students.
Knowledge Influence
Knowledge Type (i.e.,
declarative (factual or
conceptual), procedural,
or metacognitive)
Knowledge Influence
Assessment
Teachers need to know the
importance of using student-
generated questioning to
improve literacy instruction.
Declarative
(Conceptual)
Teachers review student work
using student-generated
questioning to assess students’
improvement in literacy.
Teachers need to know how to
explicitly teach the process of
student-generated questioning
to promote student literacy.
Procedural Teachers observe
implementation of student-
generated questioning in the
classroom by other teachers and
administrator.
Teachers need to know how to
reflect on their effectiveness
using the student-generated
questioning strategy to
promote student literacy.
Metacognitive Teachers will self-reflect after
professional learning activities
on their use of the instructional
strategy in their classrooms and
document adaptations of their
practice of the strategy.

APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

30
Motivational Influences
Motivation affects organizational performance by activating and directing behavior
towards desired outcomes (Mayer, 2011; Pintrich, 2003). Choice, persistence and mental effort
are three indicators for motivation (Clark & Estes, 2008). Self-efficacy and expectancy-value
are the two motivational theories that will be discussed in this section as it relates to the
stakeholder goal.
This section will review the relevant literature on motivational influences that are
essential to the attainment of the RMS teacher stakeholder goal of implementing the instructional
strategy of student-generated questioning to promote the literacy achievement of all RMS
students. The two motivational influences related to the RMS teacher stakeholder goal are
related to task value, which falls under the domain of expectancy-value theory, and self-efficacy.
Expectancy-value theory explores whether individuals can do a task, as well as whether
individuals have the desire to do a task (Eccles & Wigfield, 2002). Self-efficacy theory asserts
that beliefs about one’s ability to learn and accomplish tasks form the basis of human motivation
(Pajares, 2006). For RMS teachers to be successful in implementing student-generated
questioning in their classrooms, they must find value in the use of that strategy.
Utility value. The utility value theory states that stakeholders must see that there is an
importance of working towards a particular task or performance goal. Eccles and Wigfield
(2002) define utility value theory as for how well the individual’s plans satisfy her overall goals.
If there is no value of following through with the individual’s plans, then the individual may be
less motivated to continue in her effort of working towards her task or goal (Eccles & Wigfield,
2002).
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

31
Teacher values. When teachers are provided with the importance and value of using the
strategy to promote self-regulated learning, then teachers will be more enthusiastic about using
the strategy (Eccles & Wigfield, 2002; Pajares, 2006). Lombaerts, Engels and Van Braak (2009)
found in a study with a sample of 399 elementary school teachers that teachers’ efficacy beliefs
about self-regulated learning and their satisfaction of the strategy contributed to a stronger
influence of practice of self-regulated learning in the classroom.
Self-efficacy theory. According to Bandura (2000), self-efficacy is the belief in one’s
ability to be successful to accomplish a task. Self-efficacy can influence teachers in how they
approach goals, tasks, and challenges to teaching and learning. However, Gibson and Dembo
(1984) went further to define teachers’ self-efficacy as their personal belief that they possess the
abilities to affect student learning positively. People must believe that their actions can result in
the desired outcomes so that they will persevere in the face of adversity (Guskey, 1989; Pajares,
2006).
Teacher self-efficacy. Teachers’ self-efficacies regarding motivation may influence the
educational learning environment for students. A study conducted by Spruce and Bol (2015)
found a misalignment of teachers’ beliefs, knowledge, and practice of self-regulated learning.
Teachers having too much self-efficacy or too little self-efficacy were found to lack knowledge
of student-regulated learning, therefore reducing the practice of the strategy in the classroom. In
another study of 47 primary school teachers where surveys were given with open-ended
questions about student-generated learning, the results revealed that the teachers held a strong
belief in the strategy but not a clear knowledge of it (Dignath-van Ewijk & van der Werf, 2012).
These results make it necessary to not only examine teachers’ self-efficacy about using the
strategy of student-generated questioning but also the necessity of observing teachers’ behaviors
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

32
to assure there is an alignment of beliefs and knowledge. With the development of the teachers’
self-efficacy as it relates to the goal of implementing and changing instructional practice, it is
important to look at the efficacy of all the RMS teachers as a group.
Pajares (2006) defines collective efficacy as the shared set of beliefs and attitudes, held
by a group in an organization, regarding their ability to perform the tasks needed to achieve their
goals. RMS teachers can increase the organizational goal of improving student literacy by
developing their sense of collective efficacy on the implementation of student-generated
questioning schoolwide.
Table 3 further describes the two motivational influences and corresponding assessments
for the RMS stakeholder goal. When RMS teachers demonstrate self-efficacy and work together
to create an environment that has high collective efficacy, the likelihood of achieving the
stakeholder goal of implementing instructional strategies will be more likely.

APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

33

Table 3
Motivation Influences and Motivational Influence Assessments Related to RMS Goals
Organizational Mission
The mission of Regent Middle School is to inspire, educate, and empower our community’s
future leaders by providing a healthy, safe, and respectful environment, which fosters the
students’ academic, social, and emotional growth.
Organizational Global Goal
By December 2017, all the teachers at Regent Middle School will have implemented across
all content areas the instructional strategy of student-questioning as studied during
professional learning so that at least 60% of the students are performing at or above
benchmark in reading comprehension, as demonstrated by the results of the district’s STAR
reading assessment (STAR Reading).
Stakeholder Goal
By December 2017, 100% of RMS teachers will participate in year-long professional
learning activities to focus and implement the strategy of student-generated questioning to
promote the literacy achievement of all RMS students.
Assumed Motivation Influences Motivational Influence Assessment
Utility Value: Teachers need to understand the
value of using effectively student-generated
questioning as a strategy to positively influence
student literacy.
Teachers will answer survey questions
that will ascertain the value they have in
their implementation of student-generated
questioning in their classrooms to improve
student literacy.
Self-efficacy: Teachers should demonstrate
confidence in their abilities to implement
student-generated questioning in the classroom
practice.
Teachers will answer interview questions
that pertain to the teachers’ confidence in
their ability to use student-generated
questioning.

APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

34
Organizational Influences
In K-12 organizations, it is important to examine the organizational barriers that might
impede teachers from being able to implement high-quality instructional strategies in the
classroom, like student-generated questioning, to improve literacy for all students. According to
Clark and Estes (2008), improving an employee’s performance is one of the most significant
value-added activities an organization can undertake. To support and improve teachers’
performance in the K-12 organization, Clark and Estes (2008) identified that useful and
productive organizational work processes and resources need to be in place for an organization to
achieve its performance goals.
Work processes. Organizational work processes and procedures define how the work in
any organization is performed and ensures employees how to be as productive as possible to
achieve the organizational goals (Clark & Estes, 2008). The work processes and procedures are
defined by an organizational vision and measurable goals. In a K-12 organization, teachers need
to know how analyzing student performance data and implementing a schoolwide instructional
strategy directly relates to and impacts student achievement.
Building cycles of inquiry around student performance data in school-based professional
communities allows teachers to improve teaching and learning (Birenbaum, Kimron, Shilton, &
Shahaf-Barzilay, 2009; Copland, 2003; DuFour, 2015). According to Malloy (2011), the steps of
an effective practitioner-led inquiry includes having the “right people” tackling the problem of
practice, stating the goals of the investigation, using the existing research, framing the inquiry
with meaningful questions, using the data effectively, and evaluating the inquiry process. One
example of a framework to use during the inquiry process is Data-Driven Decision-Making
(DDDM). Based on the research of Marsh et al. (2006), the DDDM model includes a continuous
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

35
cycle of collecting data to then be able to organize and synthesize the new knowledge from the
existing knowledge that can later be used to make decisions, such as evaluating effectiveness of
policies and practices, that will eventually improve student outcomes (Marsh et al., 2006).
If a school is to achieve its performance goals, the organization must ensure that teachers
can have the time to practice a model like DDDM. Allowing teachers to engage in ongoing
analysis of student performance data to determine the impact of instructional strategies will
support them meeting their organizational goals of providing educational excellence to all
students.
Alignment of organizational culture with organizational behavior. It is necessary to
align organizational work processes to the organizational culture. According to Clark and Estes
(2008), a culture of continuous improvement will align the organizational work process and the
organizational culture. Total Quality Management (TQM) is a process that supports a culture of
continuous improvement (Clark & Estes, 2008). Zmuda, Kuklis, and Kline (2004) adapted the
TQM to school organizations and applied the following principles: to identify the organization’s
core beliefs, create a shared vision; use data to determine gaps between the current status of the
performance goals and the shared vision, identify ideas and possible solutions, develop and
implement an action plan, and establish collective accountability. Thus, the teachers will
positively impact student achievement because the school culture supports teacher collaboration
and decision-making on the implementation of effective instructional strategies that improves
reading comprehension for all students (Moran & Brightman, 2000; Thoonen, Sleegers, Oort,
Peetsma, & Geijsel, 2011).

APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

36

Table 4
Summary of Assumed Needs for Knowledge, Motivation, and Organizational Issues
Sources Knowledge Motivation Organization
Learning and
Motivation Theory
• Need understanding of
how to use student-
generated instructional
strategy.
• Need understanding of
when to use student-
generated instructional
strategy to promote
student literacy
development.
• Need understanding of
why to reflect on their
implementation of the
student-generated
questioning strategy.
• Need the desire to
understand the
value of using the
student-generated
questioning
strategy to
promote the
literacy
development.
• Need the desire to
build collective
efficacy with all
teachers.
• Need a structure
for teachers to
analyze student
interim reading
assessment data.
• Need a structure
to provide enough
time in the school
organization for
teachers to reflect
on their
implementation
of student-
generated
questioning.
Related Literature • Need examples of how
student-generated
questioning is effective.
• Need literature that
shows how making
changes to teaching
practices benefits
student learning.
• Need examples of
the value of
teachers
persevering with
changing their
instructional
practice to improve
student literacy.
• Need examples to
value collective
efficacy.
• Need literature
that supports the
use of student
assessment data to
plan instruction.
• Need examples of
how a school can
support teachers
to continue to
implement
professional
development
strategies.

APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

37
Conceptual Framework: The Interaction of Stakeholders’ Knowledge and Motivation
and the Organizational Context
In this section, the conceptual framework is presented and forms the basis of this study.
Maxwell (2013) defines the conceptual framework as the beliefs and ideas that the researcher
maintains about the topic of study. A conceptual framework establishes the researcher’s
worldview and provides the approach as to how the phenomena will be explored and offers
possible answers to the study’s research questions (Maxwell, 2013). Thus, a conceptual
framework contains what the researcher is going to study, the concepts, assumptions,
expectations, beliefs, and theories that influence the study, the relationships between these
different entities, and how these relationships contribute to the topic of study (Maxwell, 2013;
Merriam & Tisdell, 2016).
This study will use Clark and Estes’ (2008) gap analysis, a systematic, analytical method
that will help to clarify organizational goals and identify the gap between the actual performance
level and the preferred performance level within the organizational teacher stakeholder group at
RMS. The gap analysis includes the analysis of knowledge, motivation, and organizational
influences (Clark & Estes, 2008). It will be used to evaluate the transfer of adult learning from
professional development activities to the instructional practice in the classroom that will, by
proxy, influence overall middle school students’ reading comprehension achievement. Based on
the related literature of professional development theory to improve student achievement,
assumed knowledge, motivation, and organizational influences that interfere with RMS’s
organizational goal achievement will be analyzed. These influences will be assessed by using
survey, focus group interview, and document data collection methods. Research-based solutions
to these barriers will be recommended and evaluated in a comprehensive manner.
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

38
Also, the conceptual framework will develop the structure and relationship between the
concepts of transferring adult learning by drawing upon the theories of professional learning
communities. Professional learning communities and implementing student-generated
questioning strategies in the classroom will be used in this study as agents to achieve the RMS’s
organizational goal of improving student reading achievement.
This study’s conceptual framework has three components (see Figure 1). The RMS
teachers are the stakeholder group of focus. The stakeholder goal is for teachers to implement
student-generated questioning learned from professional learning into the classroom in order to
improve student learning and student outcomes. Figure 1 illustrates what conditions are needed
between the professional learning activities and the classroom in order to change instructional
practices.
The first element designated by the large blue squares are the knowledge, motivational,
and organizational influences that might interfere with the transfer of RMS teacher learning from
professional development activities to the classroom practices, specifically the implementation of
the student-generated questioning strategy. The knowledge influencers had to do with the
professional learning participation and the instructional strategy implementation. Then the area
of motivation has a relationship with the knowledge influencers in that it addresses the teachers’
beliefs and attitudes of professional learning and changing instruction. The research shows that
the development of a professional collaborative team improves the teaching and learning of all
students (Elmore, 2002). Finally, organizational influencers are found in the last blue square of
the diagram. When the organization provides the structure and support needed for professional
learning and implementation to occur, the implementation gap is eliminated. When these

APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

39

Figure 1. The transfer of adult learning from professional development to the classroom

APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

40
influencers are addressed, the flow between professional learning and classrooms is a back and
forth process to show a commitment by the teachers for the improvement of student learning and
outcomes.
Conclusion
Though there is a lack of empirical studies on the effect of professional learning on
student achievement, the chapter provided an overview of the knowledge, motivation, and
organizational influences that support or impede RMS teachers’ implementation of the student-
generated questioning strategy to improve student literacy. This study is to improve professional
learning delivery to change teachers’ practices in the classrooms. The influences as described in
the research literature will be further evaluated with the data collected from a teacher survey, a
focus group interview, and a review of the organization’s documents and artifacts.

APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

41
CHAPTER 3
METHODOLOGY
Introduction
This chapter describes the conceptual framework and the methodology of the study,
which are gap analysis and the multi-methods approach. The research was conducted as an
evaluation study, with the purpose of determining the needs associated with the implementation
of a student-generated questioning strategy learned during professional learning to promote the
literacy improvement of all RMS students. Developing an understanding of the RMS teachers’
implementation of a student-generated questioning instructional strategy learned during
professional learning showed a slight impact upon student literacy achievement as measured by
the district benchmark.
The following research questions guided this mixed methods case study:
1. To what extent are RMS teachers transferring what they learned in professional
learning into the classroom, as measured by the implementation of student-generated
questioning in the classroom?
2. What are the knowledge, motivation, and organizational influences that hinder or
support the transference of professional learning outcomes into the classroom?
3. What are the recommendations for organizational practice in the areas of knowledge,
motivation, and organizational resources?
Participating Stakeholders
This study used non-probability, purposeful sampling to select participants, a typical
method for qualitative studies (Maxwell, 2013; Merriam & Tisdell, 2016). The focus of this
study was to evaluate the influence of teacher professional development on the implementation
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

42
of student-generated questioning in RMS classrooms. All the teachers at RMS were at this time
receiving professional development to implement student-generated questioning. Selecting all
sixteen RMS teachers for the study ensured a diversity of teachers as to age, gender, and number
of years of experience in teaching. The maximum variation sampling (Maxwell, 2013; Merriam
& Tisdell, 2016) allowed the study to reflect the typical heterogeneity of the teacher population
at RMS.
Survey Sampling Criterion and Rationale
Criterion 1. All RMS teachers participating in teacher professional development to
implement the student-generated questioning strategy made up the survey sample. This survey
used purposeful sampling due to the small size of the teacher population at this school, and the
purpose of the study was to evaluate the teacher professional learning conditions to successfully
implement a student-generated questioning strategy school-wide. The rationale for using the
survey was to allow the teachers the opportunity for the most honest responses if they were more
guarded in their answers during the interviews.
Survey Sampling (Recruitment) Strategy and Rationale
The purpose of this study was to evaluate the impact of teacher professional development
on the teacher implementation of student-generated questioning in RMS classrooms. All RMS
teachers were given the opportunity to participate in the survey. The survey for this study used a
non-random sample that was purposeful because all the participants were available to complete
the survey (Fink, 2013).
The RMS teachers completed the survey through a link that I provided to them via email.
The email included a detailed explanation of how I would use the survey data as part of the
research for this study. The response rate expected for the survey was 90%, but in actuality was
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

43
60%. All participants were informed that they could ask questions at any time before, during,
and after the survey, or they could choose not to participate.
Focus Group Interview Sampling Criterion and Rationale
Criterion 1. All RMS teachers participating in teacher professional development to
implement the student-generated questioning strategy made up the interview sample. Merriam
and Tisdell (2016) suggest using purposive sampling so that the respondents selected will
contribute the data necessary to answer the researcher’s questions.
Focus Group Interview (Recruitment) Strategy and Rationale
Fink (2013) describes the ideal size of a focus group as 10-12 participants. The actual
sample size resulted in 10 RMS teachers. The focus group interview occurred after the survey
data collection. Data from the surveys were used to revise focus group interview questions.
Data Collection and Instrumentation
This section describes the approach that I used to collect data for my study. Though the
survey is a data collection method consistent with quantitative research and provides frequency
data to track changes, show trends, and monitor the use of a treatment in a study (Creswell, 2014;
Fink, 2013), I analyzed the participants’ responses as they related to the transfer of teachers’
learning from professional learning communities to practices into the classrooms. Meanwhile,
the focus group interviews and document and artifact analysis are data collection methods
consistent with qualitative research that draw on participants’ perspectives and experiences of the
phenomenon being studied (Creswell, 2014; Maxwell, 2013; Merriam & Tisdell, 2016;). These
three methods of data collection were appropriate to answer my research questions, and also
increased the credibility and dependability of my study via triangulation. Triangulation is the
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

44
strategy to analyze and compare findings in multiple methods of data collection (Creswell,
2014).
Surveys
The purpose of this study was to understand how teachers are experiencing a transfer of
learning from professional learning to their classrooms at RMS and if it influences student
outcomes positively. Surveys are used primarily in quantitative research to study variables and
frequencies of data (Creswell, 2014). However, in this study, “qualitative survey analysis” was
used to explore the perspectives and experiences of the participants (Fink, 2013). Because of the
study’s small sample size, I did not establish and analyze frequencies of responses, but instead
analyzed the participants’ responses to determine the themes that developed within my given
population of RMS teachers and then used these responses to guide the focus group interview.
Therefore, I established the trustworthiness and dependability of the survey items by
triangulation, using focus group interviews and document analysis and using the conceptual
framework of this study to measure the influences of motivation, knowledge, and organization.
The conceptual framework is outlined in Appendix C.
The surveys were self-administered online using Survey Monkey to be most efficient and
cost-effective. The sample size included 16 teacher participants. It was administered to the
teachers who have participated in professional learning communities since August 2016, and it
took approximately 20 minutes to complete. Parameters, such as time in the intervention, were
important to examine the effectiveness of the program (Fink, 2013). The survey was based on
teachers’ perceptions of their participation in professional learning. There were 15 survey items,
and they were grouped by the knowledge, motivational, and organizational influences that aid in
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

45
the transfer of teachers’ learning from professional learning communities to classroom practices
that affect student learning. The survey items for this study are in Appendix A.
In the conceptual framework, I proposed that student outcomes were influenced by the
knowledge, motivational, and organizational influences of RMS teachers implementing the
student-generated questioning strategy after their participation in professional learning.
Andragogy, professional learning community, and professional development theories were the
external factors that influenced teachers’ transfer of learning.
Focus Group Interview
The use of interview allows the researcher to “go beyond silent observation” and collect
rich data of the research participants’ perspectives (Patton, 2002). Merriam and Tisdell (2016)
state that interviews are necessary when the behavior is not able to be observable in a particular
setting. As the qualitative researcher, the researcher is the primary instrument of data collection
in the study, but in this study I did not conduct the focus group interview because I was the
primary supervisor of the participants and my position of power could have promoted reactivity,
thus being a threat to validity (Merriam & Tisdell, 2016). Therefore, I asked a trusted colleague,
who did not hold a supervisorial role over the participants, to facilitate the interview. My
colleague was trained by me to conduct the interview and record the participants with audio.
The interview protocol for this study is listed in Appendix B. A research memo was created by
the interview facilitator immediately after the interview. A research memo enables the
researcher to describe impressions, reactions, and thoughts that will be used to analyze data
(Maxwell, 2013; Merriam & Tisdell, 2016).
The study’s conceptual framework guided the concepts covered in the focus group
interview. I also explored in depth the themes that emerged during the focus group interview.
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

46
The questions were open-ended and allowed the perspective of the participants to be collected
and avoid researcher’s bias (Krueger & Casey, 2009). An open-ended interview guide facilitates
the researcher understanding the worldview of a particular group of people (Patton, 2002). The
nature of these questions yielded data that Weiss (1994) describes as a “holistic description” by
understanding the participants’ relationship to the organization, which in this study was the
middle school, RMS.
The focus group interview had a sample size of eight RMS teacher participants. The
interviews followed a semi-structured interview format where teachers’ interactions with the
professional learning community, teachers’ implementation of the student-generated questioning
strategy in the classrooms, and the teachers’ implementation of that instructional strategy on
student outcomes guided the types of questions asked during the interview. The focus group
interview took place over the course of two hours in the RMS library after school to allow time
for a rich discussion as well as sufficient time to explore the various influences and themes
included in the conceptual framework for this study. The focus interview group recording was
transcribed by a third party online service within 24 hours of the focus group interview meeting.
I used the transcription to code and analyze the data.
Documents and Artifacts
In a qualitative study, the collection of documents and artifacts is an unobtrusive method
of collecting data in a research setting (Creswell, 2014; Merriam & Tisdell, 2016). Documents
include both written and visual images relevant to the study, and artifacts are objects from the
research setting (Merriam & Tisdell, 2016). The documents and artifacts that I collected for this
study helped me construct the narrative analysis of the findings of the phenomenon (Maxwell,
2013; Merriam & Tisdell, 2016). This method of data collection was also part of the
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

47
triangulation strategy to improve the trustworthiness of the study (Miles, Huberman, & Saldana,
2014).
The document collection for the study included aggregated student data on district
benchmarks measuring reading achievement, professional learning meeting agendas and
minutes, and classroom artifacts demonstrating the use of the student-generated questioning
strategy. I had access to all these documents and artifacts because I work at RMS and they are
already collected as part of my job as the instructional leader of my school. As a researcher,
though, I asked and received permission of my supervisor, the superintendent, and my leadership
team at my school to use them in my study.
The collection of these documents and artifacts are linked to the conceptual framework of
this study to reflect the constructs at work (Miles et al., 2014). The aggregated student district
benchmark data in reading was analyzed before and after the RMS teachers had participated in
professional learning to see if there were an influence on student literacy. The professional
learning meeting agendas and minutes were used to do a “content analysis.” A content analysis
in qualitative research is a form of “. . . measuring the frequency and variety of messages and
confirming hypotheses” (Merriam & Tisdell, 2016, p. 179). In my study, I analyzed the agendas
and minutes to find out how professional learning communities addressed the transfer of learning
from the professional learning communities to the classrooms and how student data analysis in
the PLCs affected the teachers’ practices in the classroom. The artifacts of student work or
teacher posters displaying the use of the student-generated questioning strategy were utilized for
analysis as to the use of the strategy in the classrooms. The documents and artifacts provided
more narrative analysis centered on the conceptual framework of this study to understand what
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

48
influenced the RMS teachers to transfer their learning from professional learning communities to
classroom instruction and thus impact student outcomes.
Data Analysis
Collecting Survey Data
I used Pazzaglia, Stafford, and Rodriguez’s (2016) five-step survey analysis and
reporting process for educators. First, I developed a survey analysis plan to map out the survey
items as they related to the research questions, and suggested types of analysis and presentation.
The surveys were administered via Survey Monkey to keep the participants anonymous, as I was
their direct supervisor. Then I prepared and checked data files to check for errors and clean the
data. Afterward, I calculated the response rates and summary statistics. Descriptive statistical
analysis was conducted once all survey results were submitted and I had summarized my data.
Because my stakeholder group of RMS teachers was fewer than 20, the percentage of
stakeholders who agreed or strongly agreed are presented along with the percentages of those
who disagreed or strongly disagreed.
Collecting Focus Group Interview Data
The survey data guided the development of questions asked during the focus group
interview. Since I was the direct supervisor of the participants, I trained an interviewer, who
conducted the interview and then submitted the recordings to a third party transcript service to
anonymize the data. Then I developed a codebook and looked for word patterns. I then probed
for theoretical comparisons of the words and phrases after open and axial coding of the text
(Strauss & Corbin, 1990). Data analysis began after data collection. I documented my thoughts,
concerns, and initial conclusions of the data about my conceptual framework and research
questions.
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

49
In the first phase of analysis, I used open coding, looking for empirical codes and
applying a priori codes from the conceptual framework. The second phase of analysis was
conducted where empirical and a priori codes were aggregated into analytic/axial codes. In the
third phase of data analysis, I identified pattern codes and themes that emerged about the
conceptual framework and study questions. I then analyzed documents and artifacts for evidence
consistent with the concepts in the conceptual framework.
I used the theoretical comparisons to validate the knowledge causes, and to summarize
and synthesize the findings. Analyzing and reporting of qualitative data in gap analysis involved
using a priori categories consisting of the four knowledge types, which are factual, conceptual,
procedural, and metacognitive (Anderson & Krathwohl, 2001). Therefore, while reviewing
interview transcripts, the analysis revolved around the participants’ responses when each type of
knowledge was assessed. However, the interviews also revealed a cause that had not been
considered previously in Chapter 2. New causes were validated by document and artifact
analysis that followed the interview data collection and analysis.
Credibility and Trustworthiness
The purpose of this qualitative study was to describe the process by which the teachers of
RMS transferred their learning of an instructional strategy called the student-generated
questioning instructional strategy, a form of student-generated questioning, from their
participation in professional learning communities and into their classroom instruction. This was
evidenced by the degree that the strategy was fully implemented with fidelity in the classrooms
by RMS teachers, and gauged as to whether it had any effect on student achievement in reading
comprehension.
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

50
As the primary researcher in this qualitative part of this study, I was also the primary
instrument for collecting and analyzing data (Maxwell, 2013; Merriam & Tisdell, 2016).
Trustworthiness requires that the researcher report findings and events accurately and truthfully
(Merriam & Tisdell, 2016). Qualitative researchers must be aware of how their experiences and
beliefs can shape the interpretation of the data they collect (Maxwell, 2013). Therefore, as the
principal of RMS, I was mindful of the assumptions and biases that I brought to this research.
The first bias that I addressed was that I had actively participated in the professional learning
community activities with the RMS teachers and this might conflict with my answering the
research questions in an unbiased manner. The second bias was that I was the direct supervisor
of the RMS teachers, which produced researcher reactivity, and the teachers could have not been
forthcoming about their feelings in the survey and focus group interview. To counteract
researcher reactivity based on my position and improve the credibility of the study, I trained a
colleague to facilitate the focus group interview. The protocols for the focus group interview
emphasized confidentiality. Descriptors of participants were removed from the transcript before
I analyzed the data as a researcher. Participants were also told how the focus group interview
would maintain the confidentiality of all the participants to increase accurate reflections of the
participants’ experiences.
Furthermore, I addressed possible reactivity of teachers participating in the focus group
interview with their co-workers, as it may increase socially “correct” peer responses and not
convey an accurate opinion of each participant. Strategies to mitigate the social influences
included using open-ended questions, a facilitator aware of the possible reactivity, and a
comparison of the survey data to the focus group interview responses (Maxwell, 2013).
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

51
Other strategies to improve trustworthiness of the study, recommended by Merriam and
Tisdell (2016), were:
• Triangulation of data: this study used focus group interview and document collection
methods to triangulate themes in the data;
• Member checks: a subset of the participants (key informants) reviewed the findings
and provided their input;
• Rich data: the study allowed a focus group interview protocol that was semi-
structured with open-ended questions;
• Numbers of sample: the sample size of the focus group interview was around 6-8
participants, which represented about 50% of the total teachers at RMS, and the
sample was self-selected and voluntary;
• Audit trails: the researcher kept research memos, detailed descriptions of the methods
of data collection, and maintained the interview and document collection protocols.
Validity and Reliability
Merriam and Tisdell (2016) suggest that validity and reliability should be directly related
to the purpose of the study itself. In this study, this research was conducted as an evaluation of
the implementation of the student-generated questioning strategy to improve student literacy at
RMS.
Surveys are used to measure program outcomes or assess needs and must be both reliable
and valid (Fink, 2013). The survey used in this study followed Creswell’s (2014)
recommendations to ensure the validity and reliability. Specifically, I compared RMS teachers’
survey responses to the focus group interview responses to check for validity. The sample size
was small with 10 participants; therefore, the external validity threat was that the findings could
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

52
not be generalized across a large population. However, the reader of the study will be able to
transfer the findings to a similar population like RMS. Since this was a mixed-methods study,
the use of rich and detailed description of the participants and setting will allow the readers of
this study to determine if the findings will be reliable and transferable to other contexts based
upon similar characteristics of this study (Creswell, 2014; Maxwell, 2013).
The sample of the survey was a non-probability, purposeful sampling where the
participants volunteered to take the survey (Fink, 2013). There was a potential for bias with both
the participants who chose to take the survey and those who did not. The way that I mitigated
this bias was to compare the two groups of participants to ensure that the group that did take the
survey was representative of the overall population of RMS teachers. The knowledge,
motivational, and organizational influences were used to generate the survey questions. I used a
version of TALIS survey from OECD to generate my questions. The responses were analyzed
not by the variables of the responses but as a comparison to the findings of the focus group
interview.
Ethics
Being the principal of RMS, I had access to aggregated student data, professional
development documents, and teachers for surveys and interviews. Before starting the study, I
asked permission from the district superintendent to conduct this research at RMS, and I
informed all teachers that if they volunteered to participate in the study that the survey would be
anonymous and that the focus group interview would be carried out by an outside interviewer
trained by me. All processes and proceedings related to this study were kept confidential and
professional at all times. Careful attention was made during the study to position myself as a
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

53
doctoral student and not as an administrator, and I only used my University of Southern
California contact information when I engaged with the research participants.
As the principal of RMS, I made it clear that the data collected in this study would not be
used in any way as evidence in any teacher’s annual evaluation. Because I am a supervisor at the
school, it was important that I share with the participants my role as a student researcher from the
University of Southern California. I informed them that the data collected from this study may
be used as the basis for presentations to administration, teachers, and staff at RMS, as well as
others, to recommend changes in professional development at RMS. However, the information
in these reports would not name the individual teachers and would not use any information about
them that would identify who they are. I also provided them with a copy of the completed
dissertation if they were interested. Participation in this study was voluntary, and teachers were
given the choice to not answer any questions they did not feel like answering or withdraw from
the interview at any time.
Before beginning the research study, I gained approval to conduct this research project
from University of Southern California’s Institutional Review Board (IRB) and The Office for
the Protection of Research Subjects (OPRS). The OPRS reviews and monitors research studies
to protect the rights and welfare of research subjects. Federal statutes require a review of the
procedures a potential researcher will employ with human subjects before any research may
begin. The IRB at the University of Southern California performs these reviews. Therefore, I
completed an application for approval for the use of human participants in non-medical research.
Additionally, I ensured the protection of human subjects by protecting the identity of the
research participants by using pseudonyms and removing any identifiers in the interview
transcript.
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

54
The purpose of the study was fully explained to participants, and participants were
allowed to withdraw from the study at any time. All interview questions were reviewed and
approved by USC IRB to ensure that the questions did not pose a threat to the psychological
well-being of the participants. Before the beginning of the study, participants were provided an
information consent form. The informed consent form indicated an agreement to participate in
the study, have their quotes published in the final report without identifying them, and have their
rights protected. Informed consent was an ongoing process to ensure that the participants had a
full understanding of the research and their role within it if they chose to participate.
Limitations and Delimitations
As the principal of RMS and the primary researcher of this study, teachers’ responses
may not have been truthful even with the promise of anonymity and confidentiality and with the
use of a third party to facilitate the focus group interviews and anonymize the data in the
interviews and surveys. Another limitation to the study was that only one strategy was chosen
schoolwide and not implemented on a daily basis, and the teachers only had to implement the
student-generated questioning strategy a minimum of four times a year.
The other limitation to consider is that the sample size of the study was small due to the
relatively small size of the organization. While examining the study findings, it was important to
consider these limitations, but it was necessary in order to accomplish the purpose of the study,
which was to examine the influence of professional learning on teachers to implement the
student-generated questioning instructional strategy in their classrooms to improve overall
student literacy at RMS.
Delimitations of this research included the questions asked of the study participants and
the participants’ time commitment regarding the focus group interview, and on their responding
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

55
to the survey questions. Despite the researcher’s best efforts at constructing survey and
interview items, allowed me, the researcher, to create a narrative as to the implementation of the
student-generated questioning strategy in the classroom to improve student literacy.

APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

56
CHAPTER 4
RESULTS AND FINDINGS
The purpose of this study was to evaluate the factors that promote or inhibit Regent
Middle School (RMS) teachers’ ability to transfer knowledge from professional development
into their classrooms to positively affect student learning. The examination of the professional
learning practices focused on the knowledge, motivation, and organizational influences that
impacted the transference of the teachers’ professional learning outcomes into the classroom at
RMS.
The following research questions were used to guide the study:
1. To what extent are RMS teachers transferring what they learned in professional
learning into the classroom, as measured by the implementation of student-generated
questioning in the classroom?
2. What are the knowledge, motivation, and organizational influences that hinder or
support the transference of professional learning outcomes into the classroom?
3. What are the recommendations for organizational practice in the areas of knowledge,
motivation, and organizational resources?
To address the research questions, the data collection and analysis included a survey, a
focus group interview, and a review of the agendas of professional learning and disaggregated
student reading data using the third district benchmark of the 2016-17 school year. This chapter
presents the results and findings based on data analysis relative to the data collection approaches.
The third research question regarding the research-based recommendations and solutions are
outlined in Chapter 5.
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

57
Participating Stakeholders
The participants in this study were nine of the 18 teachers at RMS. Teachers represented
all subject matters and grade levels. To be eligible to participate in the research survey,
participants had to be a teacher who attended all of the professional learning days at the school
during the 2016-17 school year. Professional learning for the 2016-17 school year comprised 14
minimum school days negotiated by the teacher union and the district for professional
development. The minimum school day ended at 12:15 PM and the professional learning took
place from 12:45 PM to 2:35 PM.
At RMS, a total of 13 out of the 18 RMS teachers met the criteria above. The other five
teachers did not qualify due to various reasons, such as extended medical leave, shared teachers
with the high school, and new teachers who came halfway through the year. The survey created
on Survey Monkey was emailed to the 13 teachers. This resulted in nine teachers completing the
entire survey for a response rate of 69%. Specific demographic data of survey participants was
not collected because it was a small sample and could jeopardize the confidentiality of the
participants. Also, the teachers who were eligible to participate in the survey were from all
academic departments and possessed a range of teaching experience.
A week after the survey administration, the 13 eligible RMS teachers received in their
school mailboxes an information sheet that informed them of the focus group interview portion
of this study, where they were invited to participate by filling out a volunteer form and returning
it to the secondary investigator to maintain confidentiality. The secondary investigator contacted
the teachers to confirm participation and gave teachers two choices for interview meeting times
that would be most convenient to them. In all, nine out of the 13 RMS teachers confirmed
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

58
participation and were interviewed in the school library by the secondary investigator after
school for a period of two hours in September 2017.
Results and Findings
This section presents the results and findings of the study from the data collected from a
survey, focus group interview responses, and the document collection of aggregated student
achievement data and classroom materials used for the schoolwide strategy of student-generated
questioning. It is organized by the first two research questions:
1. To what extent are RMS teachers transferring what they learned in professional
learning into the classroom, as measured by the implementation of student-generated
questioning in the classroom?
2. What are the knowledge, motivation, and organizational influences that hinder or
support the transference of professional learning outcomes into the classroom?
When addressing the first research question, the results and findings are organized by themes
that emerged from the data collection and analysis. For the second research question, the results
and findings are organized by the Clark and Estes (2008) model of identifying the barriers,
needs, or assets that impact the organizational goals from the lens of knowledge, motivation, and
organizational influences. At the end of Chapter 4, the results and findings are summarized and
the significance of the research is explained. In Chapter 5, the third research question of
suggested recommendations for organizational practice will be addressed.
Extent of Teacher Professional Learning Transference into the Classrooms
The first research question posed in the study asked to what extent RMS teachers
transferred their training in professional learning into the classroom, as measured by the
implementation of the student-generated questioning strategy in the classroom. Answering this
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

59
question allows the organization to identify the barriers, needs, or assets of professional learning
at the school to help improve student learning outcomes in the classroom. The following three
themes emerged after data collection and analysis:
• Teachers transferred professional learning outcomes of implementing the student-
generated questioning strategy into the classroom, but were unsure how to effectively
use it to improve student learning.
• Teachers understood the value of data analysis to adjust their practices to improve
student learning, but needed more time to collaborate and establish direction on how
to create action plans during professional learning.
• Teachers valued being part of the professional learning community where they
implemented new strategies through collaboration and analyzing data, but wanted
even more time to collaborate and reflect on their practice and its effect on student
learning.
This section concludes with a synthesis of the results and findings for the first research question.
Using Professional Learning Outcomes to Improve Instructional Practice
The RMS organizational goal of implementing the student-generated questioning strategy
into all the classrooms was based on teachers first learning the strategy in professional learning
and being able to transfer that learning into an application of the strategy into the classroom.
From the survey responses, focus group interview, and the analysis of professional learning
agendas, it is apparent that there was declarative knowledge of the student-generated questioning
strategy and that it was being utilized in the classrooms. However, the teachers struggled with
how the strategy could be used effectively to improve student outcomes. This section explores
two sub-themes of teachers’ knowledge: being able to transfer the instructional strategy from
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

60
professional learning activities into the classroom and examining how the teachers used the
strategy to improve student learning.
Teachers have declarative knowledge about the use of the student-generated
questioning strategy. In the survey, 78% (7/9) of teachers reported using the schoolwide
strategy of student-generated questioning in the classroom more than four times in the 2016-17
school year. There were no respondents that reported not using the strategy at all. In the focus
group interview, teachers discussed their use of the strategy and the importance of it to stimulate
students’ critical thinking and problem-solving skills. One teacher stated: “During our
professional learning, we have been focusing on the question formulation technique, and we’ve
been trained in that process. Within the classroom, I have been using that as well to help my
students think critically.”
One of the posters in a science class presented a teacher prompt containing a picture of a
plant cell and of an animal cell, with questions written down solicited by the students that
included: Why does one cell wall have a repetitive shape and the other does not? Does the
nucleus in each cell do the same thing? Why are there less structures in one cell than the other
cell? Is one a plant cell and the other an animal cell? and so forth. This poster suggests that the
teacher used this activity to introduce the difference of plant and animal cells due to the
unspecific nature of the questions. One teacher explaining the strategy used by the teacher
described the strategy as:
We start out with focused statement, not a question, but a statement. We explained to the
students that they have some rules that they need to follow. They need to not judge the
other person, any questions. They could attempt to write as many questions as possible.
From there, they have to . . . then go through their questions and decide if they open-
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

61
ended questions. They then switch one open question to a closed question, and one
closed question to an open question. Then, they prioritized their questions based on
importance and sometimes we guide them in that process for what we’re aiming to get
out of the lesson.
Teachers saw that the strategy specifically gave students autonomy in their learning with their
peers. Another teacher noticed using the strategy that the students “. . . come up with better
questions than I could as teacher-directed questioning.”
The survey also showed that 78% (7/9) of teachers used the student-generated
questioning strategy more than four times throughout the 2016-17 school year. This was also
supported by instructional materials in and outside the classrooms that showed the use of the
instructional strategy. In posters collected from a seventh grade English Language Arts
classroom, collaborative student groups responded to a prompt that read: “Inside the Khushal
School, we flew on wings of knowledge.” Underneath the prompt, students wrote questions,
such as: Why are the Taliban scared of educated girls? Who are the teachers at the school? Why
did Malala feel free only at school? and so forth. The questions ranged in complexity and there
were different colored circles around the questions. It could be surmised from the circles that the
students were grouping questions by similarity.
These examples of teachers using the student-generated questioning strategy show that
teachers transferred the professional learning outcomes into the classrooms. The challenge for
the teachers was how to effectively use the strategy and not just go perfunctorily through the
steps of implementing the strategy to improve student learning.
Teachers lack a focus on the student-generated questioning strategy in their
instruction. While the student-generated questioning strategy promotes students’ critical
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

62
thinking skills by increasing a sense of autonomy through collaboration among students, teachers
need to know how to implement the questions into their instruction. This calls to mind the
dominant model of lesson planning, which has the objectives of the lesson planned out in
advance of the actual lesson (Tyler, 1949).
In the focus group interview, there was discussion around how to fit the strategy into the
content. One teacher stated, “My biggest thing was the frustration of what do I do with these
wonderful questions now and not knowing where to go.” Other teachers felt the same challenge
and did not know what to do with the questions. This lack of having lesson objectives for
student-generated questioning activities did point out the need of increasing teacher reflection on
enhancing their metacognitive knowledge on the practice. Another teacher added:
I agree . . . one of the most significant challenges was and especially learning and still
today, okay, we have all of this great collaboration and everything going on, but what am
I looking to get from this? It’s always been kind of a challenge and that’s when
backward mapping started where I was like, what is my objective? What do I want the
kids to get out of this? How do I know that they’ve had it and how their QFT [question
formulation technique] going to help me get there. That still is a little bit a challenge for
me.
This teacher understood the importance of having an end in mind when utilizing the student-
generated questioning strategy to improve student learning. When analyzing reading student
outcomes on 2016-17 interim district benchmarks, as shown in Figure 2, only 46% of all students
from grades six through eight were at or above grade level standards throughout the year.
Therefore, the schoolwide practice of student-generated questioning in classrooms did not
significantly improve reading achievement. The organizational goal of implementing the
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

63
student-generated questioning strategy to improve reading achievement so that at least 60% of
students performed at or above benchmark in reading comprehension was not met.

Figure 2. Aggregated student data of the 2016-17 RMS district STAR reading benchmarks

Using Interim Assessments in Instructional Practices
The organizational goal of implementing the instructional strategy of student-generated
questioning across all content areas was met, but 60% of students did not perform at or above
benchmark in reading comprehension as demonstrated by the results of the district’s STAR
reading assessment. In the survey, teachers responded that they used data analysis to guide their
instructional practices. However, during the focus group interview, teachers stated that they
were novices at using data to impact student achievement. This section investigates the sub-
themes of teachers’ knowledge about data-analysis and their use of data in their instruction.
138
86
39
32
0
20
40
60
80
100
120
140
160
At/Above On Watch Intervention Urgent Intervention
NUMBER OF STUDENTS
LEVELS ON ASSESSMENT
STAR Reading Benchmark Results June 2017
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

64
Teachers have declarative knowledge about interim assessments. In the survey, 78%
(7/9) of teachers reported using interim assessments, specifically the district’s STAR reading
benchmark assessments, to guide their instructional practice. In the focus group interview, it was
apparent that the use of the data to make instructional practice decisions was something new.
During the 2016-17 school year, RMS teachers created a data wall using colored pocket
charts and cards with students’ first and last names and their scores on the district’s quarterly
STAR reading benchmark assessments as shown in Figure 2. The teachers then changed the
cards if the students moved during each quarter. This year-long activity has had an impact on the
teachers to begin using data more effectively. During the focus group interview, one teacher
remarked on the use of the data wall:
I think when it comes to . . . I want to focus on when we’ve actually looked at data, and
made our cards of our kids’ reading levels, and where they’re at. That work felt good
because I looked at those scores on our program every day . . . When I actually saw my
kids were and where they were moving, it really did influence my teaching practices. It
made me more aware of those students and what I needed to do with them . . . .”
Another teacher explained that before the use of the data wall she did not know how to make
data-driven decision making. She stated: “I’ve always loved data. I’ve never known what to do
with it. I’ve always thought it was a judgment about my teaching or to look at my students.”
The new paradigm of using the data for decision making was also a revelation for another
teacher, who added:
I think a lot of times we’re just supersaturated with the whole data thing. There’s so
many numbers out there. What I like personally was during minimum days that the
professional learning where we actually laid out the boards and actually, physically
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

65
moved the kids around. I found that useful because it gave us a chance to actually
identify a kid and talk to him in a collaborative fashion to where you could really see that
student’s progress.
This teacher saw how interim assessments could be used as a tool to have goal setting and
feedback with students. This part of the interview illustrated that teachers were novices in data-
driven decision making because they were in the middle of a paradigm shift. Teachers are at the
beginning stages of seeing the importance of interim data analysis as a tool to reflect upon their
teaching practices and student learning.
Paradigm shift from “just data” to the data-driven decision-making process. While
interim assessments provide opportunities for teachers to participate in an inquiry process to
improve student learning, researchers note that there are multiple factors that influence the use of
data, like the data quality, school organizational supports (leadership), professional learning
community attributes, teachers’ pedagogical content knowledge, and teachers’ self- and
collective efficacy (Schildkamp & Poortman, 2015). In the conceptual map for this study, these
factors are represented as the teacher navigates between the classroom, professional learning, and
interacting with students in the classroom (see Figure 1 for the relationship between these factors
of data-driven decision-making).
The results and findings of this study point out how RMS teachers are in a
transformational shift on their use of data in decision making. In the survey, only 11% (1/9) of
teachers want more time on data analysis during professional learning, and only 33% (3/9) of
teachers felt that participating in the inquiry process of data analysis impacted their classroom
instruction. This is contradictory to the teachers’ discussion during the focus group interview
that was largely positive in the use of the data wall to track students as they were assessed with
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

66
quarterly interim assessments. One of the teachers explained their frustration in how to use the
data:
The fact is, and we can know the scores of every single one of our students, and we can
look at who’s proficient, who’s not, who’s reading, who’s not. The fact is, what are we
going to do with it? We need to figure out what to do with it, not just analyze it because
if you analyze it that’s great. Let’s put that in the basket but what are we going to do
with it? Yeah, we need time to analyze but we also need time to make some type of plan.
Another teacher concurred with this teacher that there was a need to work on the part of the data-
driven decision-making process which Marsh et al. (2006) describes as setting goals, addressing
needs, and evaluating the effectiveness of teaching practices to improve student outcomes. The
teacher similarly stated:
With time gets in the way, and I think if we had more time, then we could take those
further steps. But we hadn’t taken any further steps. I do believe that we’re going to use
the data to help come up with goals. What do we want? How are we going to attain
those goals? What are we going to do? What are the actual things that we’re going to do
that we can touch, see, and feel, and look at that are going to help us attain the goals that
are based on data.
This teacher understands that the school is at the beginning stages of analyzing interim
assessment and knows the factors that are now needed to make the paradigm shift to data-driven
decision making to improve student outcomes. Researchers recommend clear plans for data use
in professional learning. As evidenced in the interview, teachers valued being part of the
professional learning community at RMS where they implemented new strategies like the
student-generated questioning strategy into the classroom and analyzed data, but they wanted
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

67
more time for peer collaboration. This third theme of teachers needing more time to develop
work processes and procedures around data-driven decision making will be addressed in the next
section where the results and findings are delineated by the Clark and Estes (2008) KMO model.
Synthesis of the Extent to Which Teachers Implemented Professional Learning Outcomes
From the data analysis of the study, three themes emerged as to the extent to which
teachers implemented professional learning outcomes into the classroom. The first is that
teachers understood and valued their time during professional learning and transferred the
school-wide instructional practice of student-generated questioning into the classroom. The
second theme was that teachers were unclear about reflecting on the practice and reflecting on
the data of student literacy to adjust or improve student learning. Teachers needed more
guidance on how to effectively set learning objectives for the instructional practice and how to
create action plans based on interim assessments. Lastly, the third theme was that teachers
needed organizational support to provide them with the time to collaborate on instructional
practices and data.
KMO Influences on Professional Learning Transference
The second research question in this study examined the knowledge, motivation, and
organizational influences on the teachers’ application of RMS’s professional learning outcomes
into the classroom. While all of the knowledge, motivation, and organizational influences were
validated by the study, there were additional influences that emerged from the data as discussed
when addressing the first research question, and these influences stemmed from professional
learning community engagement and guidance as the teachers focused on improving student
learning. The lens of the Clark and Estes (2008) gap analysis model identified the knowledge,
motivation, and organizational influences that contributed to the successful application of the
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

68
professional learning outcomes into the classroom, and then identified additional influences
regarding how to maintain a professional learning community.
Knowledge Influences on Professional Learning Outcomes
As previously discussed in the research, results and findings of the first research question
found that teachers had the declarative knowledge of implementing the instructional strategy of
the student-generated questioning strategy into the classroom learned during professional
learning, but lacked the procedural and metacognitive knowledge of using the instructional
practice. First, teachers were unclear on how to connect the instructional strategy with
instruction. Second, teachers did not have enough time to reflect on their use of the instructional
strategy and its effects on student learning. In this section, the procedural and metacognitive
knowledge influences will be discussed.
Teachers unclear as to how to connect the school-wide instructional strategy with
instruction. The practice of using student-generated questioning is to improve students’ self-
regulation in reading comprehension (Cano García et al., 2014). It is teacher facilitated and then
the teacher helps draw connections for the students from their questions to the content being
taught. The findings and results showed that the teachers understood the concept behind the
student-generated questioning strategy and how it helps to develop critical thinking but were
confused on how to reflect upon the practice to promote student literacy. Teachers in the survey
did not feel as if they needed to participate in more training on how to implement the strategy,
but desired more time in using data and how to connect the instructional strategy with learning
objectives. As shown in Table 5, 78% (7/9) of teachers expressed a low level or no need for
additional training on student-generated questioning, whereas 78% of teachers expressed a
moderate to high level of need for more training on the inquiry process of data analysis.
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

69
Table 5
Survey Results for Teachers’ Professional Learning Needs
High level
(n)
Moderate
level (n)
Low level
(n)
No need
(n)
Collaborative cycle for data
analysis of interim assessments
11% (n=1) 67% (n=6) 22% (n=2) 0% (n=0)
Collaborative cycle for data
analysis of CAASPP assessments
11% (n=1) 67% (n=6) 22% (n=2) 0% (n=0)
Training on the student-generated
questioning strategy as the
school-wide practice
11% (n=1) 22% (n=2) 56% (n=5) 22% (n=2)

Teachers were engaged in professional learning and used peer observation and peer
feedback to ensure that the instructional practice of student-generated questioning was done
correctly. In the professional learning agendas from the 2016-17 school year, the training on the
student-generated questioning strategy appeared six days out of the 14, the setting up of times for
peer observations on the strategy with subsequent debriefs of the observations was listed twice,
and the remainder of the days were focused on growth mindset and data analysis. One teacher
during the focus group interview credited the peer observation and peer feedback as to how she
learned to use student-generated questioning in the classroom. The teacher stated:
I did enjoy that collaboration with the other teachers and having the ability to watch Susie
[pseudonym used during the interview] teach her science QFT [sic student-generated
questioning] lesson because her lesson was a lot different than anything I’ve ever done
with QFT. I thought it was amazing.
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

70
This collaboration among teachers helped teachers to build confidence in implementing the
practice in all classrooms. Another teacher agreed with the teacher about the use of collaboration
in helping with the implementation of student-generated questioning:
I agree with Gladys [pseudonym] and that I have found working with other teachers has
made me so much less stagnant, and so much more excited about my teaching, and what
I’m doing with my kids. One of my most favorable, at the time it didn’t feel like it, was
experiencing with some collaboration that we did. I had a colleague come in and asked if
she could just a QFT . . . I really looked closely at my lesson; and I appreciated having
that collaboration and definitely want to do more of the peer observation and peer
collaboration with my colleagues.
All the teachers in the professional learning community, as evidenced by the meeting agendas,
the surveys, and the focus interview, knew how to implement the instructional practice. Where it
becomes unclear, as discussed in the previous section, is the teachers’ reflection on the
effectiveness of the practice itself as it relates to improving student literacy. Schön’s (1987)
reflection-on-action model stated that the practitioner reflects about a past event in conjunction
with new information based on theoretical perspectives to process the experience and future
actions. In the next section, the lack of collaboration time on the self-reflection process which
would increase teachers’ metacognitive knowledge, as to the use and effectiveness of the
practice, is further discussed.
Teacher reflection on classroom practice of student-generated questioning. RMS
teachers had a favorable experience working with one another when learning to implement the
new instructional practice; however, there was a need for more planning and guidance on how to
evaluate the effectiveness of the practice. In the survey, as shown in Table 6, 89% (8/9) of
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

71
teachers responded that the inquiry process for data analysis and collaborative time in
departments was of most interest to them in regards to future professional learning activities.

Table 6
Survey Results for Teachers’ Interests of Future Professional Learning Activities
Data Collected Most Important (n) Least Important (n)
Collaborative cycle for data analysis of interim
assessments
89% (n=8) 11% (n=1)
Collaborative time in departments 89% (n=8) 11% (n=1)

During the focus group interview, a teacher made the point about connecting and synthesizing
the existing knowledge to create new insights to then be used to problem-solve and evaluate new
practice, which is part of the data-driven decision-making model (Marsh et al., 2006). She
stated:
When we actually get to collaborate, we don’t get a lot of time to work with each other at
all, because I actually don’t feel that we’re going to work with grade level based material
or department. I feel like professional learning is all geared towards something else.
While there is no overall consensus of what this collaboration time looks like, based on this
teacher’s assertion, it is clear that there is a general lack of time and guidance to allow teachers
to develop metacognitive knowledge of their professional learning outcomes. This finding is
significant because it identifies factors influencing professional development interventions to
ensure not only effective implementation but also sustainability of the teachers’ use of the
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

72
instructional strategy in their classrooms and reflection on adaptation of their practice of the
strategy.
Motivation Influences on Using Student-Generated Questioning
There were two assumed motivation influences that led the research for this study. The
first was that teachers understood the value using the student-generated questioning learned from
professional learning and the second one was that teachers understood that their own efficacy
and their collective efficacy with the rest of the staff using the student-generated questioning
technique was important to improve student outcomes. The results and findings demonstrated
that teachers valued learning and implementing the instructional strategy of student-generated
questioning schoolwide, but also valued their participation in the professional learning
community.
Teachers valued the use of student-generated questioning to improve student
literacy outcomes. Teachers in the focus group interview demonstrated that they valued using
the strategy and the interim assessments, but needed guidance in the next steps of developing
learning objectives and action plans. This was confirmed by a specific question in the survey
that asked if they valued using student-generated questioning in the classroom, with 100% (9/9)
of the teachers agreeing and strongly agreeing that the practice was of value to them as shown in
Table 7.

APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

73
Table 7
Survey Results of Teachers’ Value of Instructional Practice
Strongly
Agree (n) Agree (n)
Disagree
(n)
Strongly
Disagree (n)
Student-generated questioning
strategy
11% (n=1) 89% (n=8) 0% (n=0) 0% (n=0)

One teacher during the interview saw that the implementation of the instructional practice
developed student autonomy. The teacher described the value of the instructional practice:
One huge goal I think often kind of gets set aside is student ownership. It really
encourages student ownership in the questioning technique but also in the ultimate
answers. That’s a new thing. I found it very powerful. The kids bought in and had
ownership.
All the teachers saw that student-generated questioning was beneficial to students. Almost all
the teachers also reported that interim assessments were instrumental to gauge students’ learning
outcomes. While the teachers valued the instructional strategy and the use of interim data that
they learned during professional learning, they also acknowledged the challenges of developing
goals and objectives for the strategy and data, rather than just implementing the strategy and
performing data analysis as a routine.
Teachers found challenges in meeting the organizational goals. The results and
findings of the study revealed that teachers saw the value of student-generated questioning, data
analysis, and collaboration, but still did not know how their actions could affect student
outcomes. As shown in Table 8, the majority of teachers (8/9) reported that the activities in
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

74
professional learning that they participated in had a moderate to large impact on them
professionally.

Table 8
Survey Results on the Teachers’ Value of the Professional Learning Outcomes
In the past year that you
participated in professional
learning, how did you feel about
the impact of the…
No impact
(n)
Small
impact level
(n)
Moderate
impact (n)
Large
impact (n)
Collaborative cycle for data
analysis of interim assessments
0% (n=0) 11% (n=1) 78% (n=7) 11% (n=1)
Training on the student-generated
questioning strategy
0% (n=0) 11% (n=1) 67% (n=6) 22% (n=2)
Peer coaching/feedback on the
student-generated questioning
strategy
0% (n=0) 11% (n=1) 78% (n=7) 11% (n=1)

The barrier to high self-efficacy and high collective efficacy remains on setting the
appropriate goals and objectives to implement these strategies. Guskey (1989) explained that
teachers must believe that their actions can produce the desired results. The results in this case
indicated that the teachers believed that the student-generated questioning strategy would
increase student achievement in reading comprehension. A teacher in the focus group interview
stated that they knew how to execute the strategy but found it challenging to understand the
objective:
Okay, we have all of this great collaboration and everything going on, but what am I
looking to get from this? It’s always been kind of a challenge and that’s when the
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

75
backward mapping started for me. Like what is my objective? What do I want the kids
to get out of this? How do I know that they’ve got it and how is their QFT going to help
me get there? That still is all a little bit of a challenge for me.
As discussed in the results and findings of the first section addressing the extent that RMS
teachers implemented the student-generated questioning strategy, the themes of teachers being
unsure how to effectively use it to improve student learning with a learning objective and their
needing more time to collaborate to create action plans after data analysis were central themes in
the study. Mayer (2011) defines instructional objectives as involving being able to distinguish
between learning (a change in knowledge) and performance (a task). In the next section, the
organizational influences that affect teachers being able to create instructional objectives and
then use the data to make decisions on the instruction will be discussed.
Organizational Influences on Using Student-Generated Questioning
There were two assumed organizational influences investigated and validated in this
study, and they concerned the alignment of work process and procedures with the organizational
culture. The first influence was developing an inquiry process using student data to determine
the impact of instructional strategies to support the organizational goal of improving student
literacy. The second influence was using Total Quality Management (TQM) as a process to
support a culture of continuous improvement. In this section, each of these influences is
discussed.
Teachers used data but were not fully implementing an inquiry process. The DDDM
model is an inquiry process that includes a continuous cycle of collecting data to then be able to
organize new knowledge from the existing knowledge, which then can later be used to make
decisions, such as evaluating effectiveness of practices used to improve student outcomes (Marsh
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

76
et al., 2006). As discussed in the results and findings of the first research question, RMS
teachers need time and guidance to create action plans based on their data analysis for the
quarterly district benchmarks on reading. Allowing teachers to engage in ongoing analysis of
student learning and performance data will allow teachers to assess the impact of instructional
strategies and whether adjustments need to be made. In the survey, the school leadership was
reported as supporting the professional learning process (9/9), as shown in Table 9, but the
meeting agendas for professional learning showed that the majority of the professional learning
time was spent on declarative knowledge of the instructional practice and the data analysis
collected quarterly throughout the year. Not found in the professional learning meeting agendas
were agenda items to discuss the learning objectives of the instructional practice or “reflection-
on-action” and the time given to create action plans after reviewing data analysis, as evidenced
by the time required to move student information cards onto the data wall.

Table 9
Survey Results of the School Providing Enough Support for Professional Learning
In the past year,
Strongly
Disagree (n)
Disagree
(n) Agree (n)
Strongly
Agree (n)
Enough time for professional
learning
33% (n=3) 67% (n=6) 0% (n=0) 0% (n=0)
School principal supported the
professional learning process
0% (n=0) 0% (n=0) 78% (n=7) 22% (n=2)

APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

77
A teacher in the focus group interview was looking for professional learning to
“. . . guide us in coming up with strategies to work with our students.” Another teacher
highlighted that there was a need to “. . . get more people to come in as long as it’s all guided and
structured with what we’re trying to achieve. I’m still working on this . . . .”
Teachers have not yet established collective accountability on student outcomes.
TQM uses data to determine gaps between the current status of the performance goals and the
shared vision, identify ideas and possible solutions, develop and implement an action plan, and
establish collective accountability (Clark & Estes, 2008). RMS teachers determined there was an
academic achievement gap in reading comprehension as measured by the district’s benchmark
assessments between students meeting or exceeding standards and those that did not. Teachers
then researched instructional strategies that targeted critical thinking and reading comprehension,
and decided to implement schoolwide the strategy of student-generated questioning. They set an
organizational goal to have 60% of the students perform at or exceeded standards in reading
comprehension by the end of the 2016-17 school year. At the end of the year, the organizational
goal was not yet met and only 47% of the students were at or exceeded standards. Slayton and
Mathis (2010) suggested that educational leaders need to provide the staff support to engage in
deeper learning of how to better serve students’ academic needs. Increased time during
professional learning to identify ideas and possible solutions and also develop and implement
action plans should lead to more effective use of instructional strategies and the use of interim
data analysis.
Synthesis of the Results and Findings of KMO Influences
The knowledge, motivation, and organizational influences were validated in this study
and presented in the sections above. The key themes that emerged from this study were:
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

78
• Teachers transferred professional learning outcomes of implementing the student-
generated questioning strategy into the classroom, but were unsure how to effectively
use it to improve student learning.
• Teachers understood the value of data analysis to adjust their practices to improve
student learning, but needed more time to collaborate and receive guidance on how to
create action plans during professional learning.
• Teachers valued being part of the professional learning community where they could
implement new strategies like the student-generated questioning strategy into the
classroom and analyze data through peer collaboration, but wanted more time to
collaborate.
All the themes had a prevailing organizational barrier, that there was not enough time for
teachers to fully participate in the inquiry process and to create a culture of continuous
improvement. This organizational barrier obstructed the knowledge and motivation assets that
the teachers possessed. However, Clark and Estes (2008) illustrated that even when knowledge
and motivation are not the gaps to the performance goal, the work process and procedures in the
organization still need to be supported and adapted.
Summary
The results and findings from the survey, focus group interview, and the document
analysis indicate that the teachers understand the importance and value of participating in Regent
Middle School’s professional learning community and implementing their newly learned
knowledge, like instructional strategies and analyzing interim assessment data. However, they
are unclear how to evaluate the effectiveness of their implementation of research-based
strategies, like the student-generated questioning. Part of evaluating the effectiveness of their
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

79
instruction is to analyze interim data and then devise action plans. Teachers reported needing
more time and guidance to create learning objectives and to develop action plans within the
professional learning community. In Chapter 5, recommendation to address these organizational
barriers will be discussed.

APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

80
CHAPTER 5
DISCUSSION AND RECOMMENDATIONS
In Chapter 4, the assumed influences in the study were analyzed using quantitative and
qualitative research and organized in Clark and Estes’ (2008) model of knowledge, motivation
and organizational assets and barriers to the organizational goal. In Chapter 5, the significance
of the results and findings will be addressed with regard to theoretical principles and provide
recommendations for improvement and further study. This chapter is organized into categories
of validated and not validated knowledge, motivation, and organizational influences. The New
World Kirkpatrick Evaluation Model (Kirkpatrick & Kirkpatrick, 2016) was used to plan,
implement, and evaluate the solutions and recommendations outlined in this chapter in reference
to the teachers’ application of professional learning outcomes into the classrooms to improve
student achievement. The model has four levels: results, behavior, learning, and reaction. The
Kirkpatrick model will provide observable, measurable success outcomes to achieve the
organizational goal.
Recommendations for Practice to Address KMO Influences
Knowledge Recommendations
Introduction. The knowledge influences in Table 10 represent the complete list of
assumed knowledge influences and their probability of being validated the survey, the group
interview, and the documents analysis. The second influence was not validated by the data as
teachers displayed knowledge of how to teach the student-generated questioning strategy. The
other two influences listed in Table 10 were a priority and are supported by recommendations
that address these knowledge influences.
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

81
There are the four knowledge types of factual, conceptual, procedural, and metacognitive
knowledge described by Krathwohl (2002) that were utilized to analyze adult learning
transference from professional learning into the classroom. Clark and Estes (2008) suggested
that the increase of knowledge is necessary before applying that knowledge procedurally. In the
case of RMS teachers, the declarative knowledge that teachers had of the student-generated
questioning strategy learned during professional learning and increased their procedural
knowledge of it in the classroom. However, RMS teachers struggled with reflecting on the
effectiveness of their practice of using the student-generated questioning strategy, which falls
under the influence of metacognitive knowledge. Table 10 provides a list of the knowledge and
procedural influences that were first identified in the literature review and then having a gap
through data collection and analysis. Table 10 also shows the context-specific recommendations
for these highly influences based on theoretical principles.

APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

82
Table 10
Summary of Knowledge Influences and Recommendations
Assumed Knowledge
Influence: Cause, Need,
or Asset*
Validated:Y
es, High
Probability,
or No
(V, HP, N)
Priority:
Yes, No
(Y, N) Principle and Citation
Context-Specific
Recommendation
Teachers need to know
the importance of using
student-generated
questioning to improve
literacy instruction. (D)
N N Procedural knowledge
increases when
declarative knowledge
required to perform the
skill is available or
known (Clark & Estes,
2008).
Information is
provided during
professional
learning events
with the use of
research to validate
the strategy.
Teachers need to know
how to explicitly teach
the process of student-
generated questioning to
promote student literacy.
(P)
N N Acquiring skills for
expertise frequently
begins with learning
declarative knowledge
about individual
procedural steps (Clark
& Estes, 2008).
Use of infographic
strategy card for
teachers to
reference in their
classroom.
Teachers reflect on their
effectiveness using the
student-generated
questioning strategy to
promote student
literacy. (M)
V Y The use of
metacognitive
strategies facilitates
learning (Baker, 2008).
Through a
reflection exercise,
using a Google
survey after
professional
learning events.
* Knowledge type for each influence: (D)eclarative; (P)rocedural; (M)etacognitive

Declarative knowledge solutions, or description of needs or assets. Conceptual
knowledge influences the procedural knowledge of RMS teachers’ ability to apply the strategy of
student-generated questioning (Clark & Estes, 2008). Though there is not a great deal of
research on teacher practice of using student-generated questioning in the classroom, the
importance of student-generated questioning can be observed through the framework of
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

83
promoting self-regulation and metacognition in learning (Wong, 1985). Understanding the
practice of student-generated questioning for self-regulated learning in reading comprehension is
necessary for teachers when establishing student learning outcomes (Cano García et al., 2014).
King (1992) found that student-generated questioning resulted in better student performance as
compared to summarizing and note-taking. The process of student-generated questioning is also
important for teachers to value because it allows their students to become critical thinkers. The
attributes of the student-generated questioning strategy that promotes critical thinking in students
are determined by how the teachers use the prompts to lead the students to generate questions on
their own accord within the parameters of the strategy (King, 1992).
Organizations need to possess the procedural knowledge of how to organize and follow
processes in order to accomplish their organizational goals (Krathwohl, 2002). The RMS
teachers’ lack of procedural knowledge of student-generated questioning strategy will be
addressed in both the professional learning that they will receive during the school year, and
through the development of job aids in the form of infographic strategy cards given during
professional learning. The use of reflection will facilitate learning (Baker, 2008). At the end of
professional learning, RMS teachers will be given time to self-reflect on their teaching practices.
Procedural knowledge solutions, or description of needs or assets. Procedural
knowledge refers to the necessary information that is needed to accomplish certain tasks and
participate in certain activities to reach a specific outcome (Krathwohl, 2002). Teachers
completely understanding the process of teaching students to ask and generate questions is
important because it helps the students become aware of important points in the text (Wong,
1985). Middle school teachers need to have the knowledge and skills to apply and to facilitate
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

84
student-generated questioning as a strategy of self-regulation to improve overall reading
comprehension.
Metacognitive knowledge solutions, or description of needs or assets. Metacognitive
knowledge refers to the awareness and understanding of one’s state, evaluating one’s strengths
and challenges, and self-awareness (Krathwohl, 2002). When teachers are implementing the
practice of student-generated questioning in the classroom, it is important that teachers self-
reflect and self-regulate on the practice as they become better at it. Teachers who take their time
to examine and reflect on classroom practices and the impact of those practices on students, can
make informed decisions about what they need to do or change in their practice (Elmore, 2002).
Rueda (2011) also states that teachers must use a metacognitive process to examine their practice
and make the necessary adjustments to improve their teaching.
Motivation Recommendations
Introduction. The motivation influences in Table 11 represent the complete list of
assumed motivation influences and their probability of being validated based on the most
frequently mentioned motivation influences to achieving the stakeholders’ goal during informal
interviews and supported by the literature review and the review of motivation theory. Clark and
Estes (2008) suggest that the three indicators of motivation in task performance are choice,
persistence and mental effort. Individuals who are motivated to go beyond intention, pursue a
goal with focus, and apply new knowledge to solve or perform a new task are able to achieve
their goals. As illustrated in Table 11, the motivational influences of utility value, self-efficacy,
and collective efficacy that have a high probability of being validated and have a high priority for
achieving the stakeholders’ goal are listed. Table 11 also shows specific recommendations for
these influences based on learning theory.
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

85
Table 11
Summary of Motivation Influences and Recommendations
Assumed Motivation
Influence: Cause, Need,
or Asset*
Validated:
Yes, High
Probability,
No
(V, HP, N)
Priority:
Yes, No
(Y, N) Principle and Citation
Context-Specific
Recommendation
Utility Value: Teachers
need to understand the
value of effectively
using student-generated
questioning as a
strategy to positively
influence student
literacy.
HP Y Individuals are more
likely to engage in an
activity when it
provides value to
them (Eccles &
Wigfield, 2002).
Provide relevant
activities during
professional learning
for staff to analyze
student work that
resulted from student-
generated questioning
lessons in the
classrooms.
Self-efficacy: Teachers
should demonstrate
confidence in their
abilities to implement
student-generated
questioning in their
classroom practice.
Y Y Effective
observational learning
is achieved by first
organizing and
rehearsing modeled
behaviors, then
enacting it overtly
(Ambrose, Bridges,
DiPietro, Lovett, &
Norman, 2010).
Self-efficacy is
increased as
individuals succeed in
a task (Bandura,
2000).
Provide training
during professional
learning in which the
instructional strategy
is modeled, and then
provide coaching by
principal, and peer
observation support.
Collective efficacy:
Teachers need to work
with other teachers in
the school as a
professional learning
community to improve
their teaching practice
collectively.
Y Y According to Bandura
(2000), collective
efficacy is the
performance of a
group of individuals
in an organization.
For schools, collective
efficacy is teachers’
belief that their
practices can affect
students’
performance.
Maintain a
professional learning
community where the
adult learner is
valued.

APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

86
Utility value. The utility value theory states that stakeholders must see that there is an
importance of working towards a particular task or performance goal. Eccles and Wigfield
(2002) define utility value theory as to how well the individual’s plans satisfy her overall goals.
If there is no value of following through with the individual’s plans, then the individual may be
less motivated to continue in her effort of working towards her task or goal (Eccles & Wigfield,
2002).
There is a correlation between shared values, the teaching practice derivatization, a focus
on student learning, collaboration, and reflection on practice in a professional learning
community (Louis, Marks, & Kruse, 1996). This increases the motivation of teachers to learn
student-generated questioning during professional learning, apply the strategy in the classroom,
and then reflect on the effect of the practice on student learning back in the professional learning
community.
In order to increase the teachers’ self-efficacy, the effective observational learning is
achieved by first organizing and rehearsing modeled behaviors, then enacting it overtly
(Ambrose et al., 2010). In order to do this, the instructional strategy will be modeled during
professional learning, followed by coaching and peer observation support.
In order to enhance collective efficacy, RMS will maintain a professional learning
community where the adult learner is valued. In a professional learning community that has
collective efficacy, it must improve professional practice as a collective whole and continue all
teachers’ intellectual development (Grossman et al., 2001).
Organization Recommendations
Introduction. The organization influences in Table 12 represent the complete list of
assumed organization influences and their probability of being validated based on the most
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

87
frequently mentioned organization influences to examine the organizational barriers that might
impede teachers from being able to implement high-quality instructional strategies in the
classroom, like student-generated questioning, to improve literacy for all students. Clark and
Estes (2008) suggest that the organization and stakeholder goals are often not achieved due to a
lack of resources, most often time and money, and when stakeholder goals are not aligned with
the organization’s mission and goals. Gallimore and Goldenberg (2001) propose two constructs
about culture – cultural models or the observable beliefs and values shared by individuals in
groups, and cultural models, or the settings and activities in which performance occurs. Thus,
both resources and processes, and cultural models and settings must align throughout the
organization’s structure to achieve the mission and goals. As such, as indicated in Table 12,
some organizational influences have a high probability of being validated and have a high
priority for achieving the stakeholders’ goal. Table 12 also shows the recommendations for
these influences based on learning theory.
Cultural setting influence: Lack of using work processes. Organizational work
processes and procedures define how the work in any organization is performed and ensures
employees are as productive as possible to achieve the organizational goals (Clark & Estes,
2008). The work processes and procedures are defined by an organizational vision and
measurable goals. In a K-12 organization, teachers need to know how analyzing student
performance data and implementing a schoolwide instructional strategy directly relates to and
impacts student achievement.

APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

88
Table 12
Summary of Organization Influences and Recommendations
Assumed Organization
Influence: Cause, Need,
or Asset*
Validated:
Yes, High
Probability,
No
(V, HP, N)
Priority:
Yes, No
(Y, N) Principle and Citation
Context-Specific
Recommendation
Teachers need a
structure for teachers to
analyze student interim
reading assessment data
and student work.
HP Y Organizational
performance increases
when individuals
communicate constantly
and candidly to others
about plans and
processes (Clark &
Estes, 2008).
Organizational
performance increases
when processes and
resources are aligned
with goals established
collaboratively (Clark &
Estes, 2008).
Conduct
professional
development
events with
established
protocols and time
to analyze data and
student work.
The school organization
needs to provide enough
time for teachers in the
school organization to
reflect on their
implementation of
student-generated
questioning.
V Y Organizational
performance increases
when individuals
communicate constantly
and candidly to others
about plans and
processes (Clark &
Estes, 2008)
Conduct whole
organization
meetings to
communicate the
vision, mission and
goals, and
individual and team
accomplishments.
Cultivate a culture
of professional
community with all
stakeholders in
achieving
organization goals
by encouraging
feedback and
communication by
teacher
stakeholders.

APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

89
Building cycles of inquiry around student performance data in school-based professional
communities allows teachers to improve teaching and learning (Birenbaum et al., 2009; Copland,
2003; DuFour, 2015). According to Malloy (2011), the steps of an effective practitioner-led
inquiry includes having the “right people” tackling the problem of practice, stating the goals of
the investigation, using the existing research, framing the inquiry with meaningful questions,
using the data effectively, and evaluating the inquiry process. One example of a framework to
use during the inquiry process is Data-Driven Decision-Making (DDDM). Based on the research
of Marsh et al. (2006), the DDDM model includes a continuous cycle of collecting data to then
be able to organize and synthesize the new knowledge from the existing knowledge, which can
later be used to make decisions, such as evaluating effectiveness of policies and practices, that
will eventually improve student outcomes (Marsh et al., 2006).
If a school is to achieve its performance goals, the organization must ensure that teachers
can have the time to practice a model like DDDM. Allowing teachers to engage in ongoing
analysis of student performance data to determine the impact of instructional strategies will
support them in meeting their organizational goals of providing educational excellence to all
students.
Alignment of organizational culture with organizational behavior. It is necessary to
align organizational work processes to the organizational culture. According to Clark and Estes
(2008), a culture of continuous improvement will align the organizational work process and the
organizational culture. Total Quality Management (TQM) is a process that supports a culture of
continuous improvement (Clark & Estes, 2008). Zmuda et al. (2004) adopted the TQM to school
organizations and applied the following principles: to identify the organization’s core beliefs,
create a shared vision, use data to determine gaps between the current status of the performance
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

90
goals and the shared vision, identify ideas and possible solutions, develop and implement an
action plan, and establish collective accountability. Thus, the teachers will positively impact
student achievement because the school culture supports teacher collaboration and decision-
making on the implementation of effective instructional strategies that improve reading
comprehension for all students (Moran & Brightman, 2000; Thoonen et al., 2011).
Integrated Implementation and Evaluation Plan
Implementation and Evaluation Framework
The New World Kirkpatrick Model is the framework for the implementation and
evaluation plan of this study (Kirkpatrick & Kirkpatrick, 2016), which was originally the
Kirkpatrick Four Level Model of Evaluation (Kirkpatrick & Kirkpatrick, 2016). The original
Kirkpatrick’s four levels, were reaction, learning, behavior, and results. The new model reverses
the order of the levels and suggests that the evaluation plan needs to begin with a clear
understanding of the intended results and uses “leading indicators” to bridge the gap between the
organization’s initiatives and goals as Level Four. Leading indicators are defined as short-term
observations and measurements that suggest that critical behaviors are being achieved in order to
arrive at the organizational goal (Kirkpatrick & Kirkpatrick, 2016). Level 3 examines the
behaviors of participants in the organization’s training and the degree to which participants
transfer and apply what they have learned to their work. Critical behaviors, required drivers, and
on-the-job learning are part of the new model framework. Critical behaviors are specific actions
needed to achieve the desired results; required drivers are systems to reinforce and monitor job
performance; and on-the-job learning provides an opportunity for all the stakeholders in the
organization to share the responsibility for positive results in the organization. Level 2 focuses
on learning and to what degree the participants acquire new knowledge and skills, and Level 1
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

91
evaluates the reaction or level of engagement participants have to the actual training (Kirkpatrick
& Kirkpatrick, 2016). Evaluating the results of initiatives as they are being implemented allows
for continuous improvement to be made and contributes to the overall success of the initiative.
Figure 3 provides a visual presentation of the Kirkpatrick New World Model (Kirkpatrick &
Kirkpatrick, 2016).

Figure 3. Kirkpatrick New World Model

Organizational Purpose, Need and Expectations
As an implementation and evaluation plan is developed, the purpose of the organization
must be considered to ensure value alignment for training participants and facilitators. The
stakeholder group identified for this study is teachers at RMS and a stakeholder goal was
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

92
identified by the teachers and the administrator, who is I, to use a best teaching practice in order
to improve student reading outcomes. Teachers’ impact on student learning is correlated with
the success of the organizational mission of RMS, which is to prepare students for the future.
As stated in the beginning of this chapter, all the teachers at RMS will have implemented
across all content areas the instructional strategy of student-questioning as studied during
professional learning so that at least 60% of the students are performing at or above benchmark
in reading comprehension, as demonstrated by the results of the district’s STAR reading
assessment (STAR Reading). Further, by December 2017, 100% of RMS teachers will
participate in year-long professional learning activities to focus and implement the strategy of
student-generated questioning to promote the literacy achievement of all RMS students. This
evaluation will establish a baseline for the learning transference of best teaching practices from
professional learning activities into the classroom to improve student reading outcomes.
Level 4: Results and Leading Indicators
Table 13 shows the proposed Level 4: Results and Leading Indicators in the form of
outcomes, metrics and methods for both external and internal outcomes for RMS. If the internal
outcomes are met as expected as a result of the training and organizational support for
implementing best teaching practice in the classrooms, then the external outcomes should be
actualized.

APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

93
Table 13
Outcomes, Metrics, and Methods for External and Internal Outcomes
Outcome Metric(s) Method(s)
External Outcomes
Increased community
perception of the school.
Increased number of positive
responses versus negative
responses on a community survey.
Twice a year community survey
offered on the school website.
Increased number of
students using student-
generated questioning on
their own.
100% of students use student-
generated questioning in the
classroom.
Track use via principal
observation and teacher peer
observation of the technique in
the classrooms on a regular basis.
Increased number of
students setting their own
learning goals with
guidance from their
teachers.
100% of students know their
current grades and know how to
improve them.
Student feedback of their use of
the district grade app to know
their grades and set goals with it.
Internal Outcomes
Teachers have increased
satisfaction in the
professional learning
community process.
Increase of teachers’ satisfaction
of professional learning as
measured by monthly surveys.
Surveys will measure teachers’
feedback of professional learning
community by surveys that
measure satisfaction or
dissatisfaction, and then compare
quarterly.
Increased number of
coaching cycles completed
by the principal.
The numbers of days are
calendared in Google calendar.
Coaching cycles are every two
weeks.
Increased quarterly peer
observation cycles
completed by the teachers.
The days are calendared in
Google calendar and shared with
all.
Data is reviewed every nine
weeks after interim assessments
are administered.
Increased use of
schoolwide best
instructional practices by
teachers in the classroom.
Feedback will be collected by
principal.
Data is reviewed weekly.

APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

94
Level 3: Behavior
Critical behaviors. The stakeholders of focus are the RMS teachers. The first critical
behavior is that the principal must monitor implementation of the schoolwide best practice of
student-generated questioning in the classrooms. The second critical behavior is that the RMS
Instructional Leadership Team must seek staff input for professional learning community
agendas. The third critical behavior is that RMS teachers and the principal must analyze student
data to correctly identify gaps in the reading program. The specific metrics, methods, and timing
for each of these outcome behaviors appears in Table 14.

Table 14
Critical Behaviors, Metrics, Methods, and Timing for Evaluation
Critical Behavior Metric(s) Method(s) Timing
1. Teachers will create
learning objectives when
using the student-
generated questioning
strategy to improve
student reading.
100% of the teachers will
demonstrate an
understanding of the
value of using the
student-generated
questioning strategy to
improve student reading.
Teachers will answer
questions related to
the value of student-
generated
questioning on a
teacher survey.
A baseline survey
will be conducted at
the start of the
school year, then
subsequently at the
end of each quarter.
2. All teachers will
participate in
instructional planning
sessions during
professional learning to
improve student literacy.
There are 16 professional
development days
calendared for 2017-18
school year. Therefore,
100% of the teachers will
have more time to
collaborate and plan.
Teachers will give
input on professional
learning agendas to
include planning and
collaboration time.
16 times throughout
the year.
3. Teachers and
principal will analyze
student data to correctly
identify gaps and create
action plans.
100% of the teachers will
participate in scheduled
data analysis professional
learning and will create
action plans.
Teachers will
collaborate to
analyze interim data
reading assessment
and create action
plans.
Every quarter.

APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

95
Required drivers. RMS teachers need the support of their principal, the RMS
Instructional Leadership Team, and the organization to support and reinforce what teachers learn
in training during professional learning events in the organization to encourage them to apply
what they learned in the training to the classroom to positively affect student learning. Rewards
are based on respecting the teachers as adult learners and recognizing student achievement as
part of their effort in the training. Table 15 shows the recommended drivers to support critical
behaviors of RMS teachers.
Organizational support. Regent Middle School will provide the support and structure
needed to achieve the critical behaviors that will have the most impact on student learning. The
support will include using an inquiry process when examining student data, tailoring professional
learning opportunities to the adult learner, structuring time for teachers to reflect on their
teaching practice, providing peer and administrator coaching and feedback, and creating a culture
of collective accountability around student academic outcomes.

APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

96
Table 15
Required Drivers to Support Critical Behaviors
Method(s) Timing
Critical
Behaviors
Supported
(1, 2, 3)
Reinforcing
Information is provided during professional learning events with
research support to validate instructional strategies and how to
create learning objectives to support those strategies.
Two times per
month.
1, 3
Provide training during professional learning in which the
instructional strategy is modeled, and then provide coaching by
principal, and peer observation support.
Two-three times
per month.
1
Provide time for collaboration and planning to create action plans
after interim data analysis.
Every ten weeks. 2
Encouraging
Provide relevant activities during professional learning for staff
to analyze student work that resulted from student-generated
questioning lessons in the classrooms.
Two times per
month.
1, 2, 3
Coaching feedback from principal to teachers and teachers to
teachers.
Weekly 1, 2, 3
Rewarding
Maintain a professional learning community where the adult
learner is valued by seeking all staff input on professional
learning agendas.
Ongoing 2
Reporting student achievement increases to school stakeholders
via a multiple of ways.
Quarterly 2, 3
Monitoring
Through a reflection exercise, using a Google survey after
professional learning events.
Monthly 1, 2, 3
Principal does classroom walkthroughs to monitor teacher
implementation of best practices.
Weekly 1, 2

APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

97
Level 2: Learning
Learning goals. When the recommended solutions are actualized, RMS teachers will be
able to:
1. Know the importance of using student-generated questioning to improve literacy
instruction. (D)
2. Know how to explicitly teach the process of student-generated questioning to
promote student literacy. (P)
3. Reflect on their effectiveness using the student-generated questioning strategy to
promote student literacy. (M)
4. Understand the value of effectively using student-generated questioning as a strategy
to positively influence student literacy. (Value)
5. Demonstrate confidence in their abilities to implement student-generated questioning
in the classroom practice. (Self-efficacy)
6. Work with other teachers in the school as a professional learning community to
improve their teaching practice collectively. (Collective Efficacy)
Program. The learning goals in the previous section will be achieved by establishing a
professional learning community among the teachers and administrator to focus on student
literacy and transferring that knowledge into the classroom. The organization will set aside 20
days where students will be released early so that the professional learning community can meet
to allow time to build capacity of best practice implementation, data-driven decision making,
active learning, support, and collective participation in the process of continuous improvement.
Support in the program will include modeling, coaching, and feedback.

APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

98
Table 16
Components of Learning for the Program
Method(s) or Activity(ies) Timing
Declarative Knowledge “I know it.”
Knowledge checks through discussion during professional
learning events.
After professional learning.
Knowledge checks during PLC collaboration by department. After each collaboration session.
Procedural Skills “I can do it right now.”
Demonstration in groups and individually in using the
student-generated questioning to successfully implement it
in the classroom.
During professional learning.
Quality of the feedback from teachers during share outs of
their future use of the instructional strategy.
After professional learning.
Attitude “I believe this is worthwhile.”
Teacher analysis of student data and student work during
professional learning.
During professional learning.
Quality of the teacher feedback from post professional
learning surveys.
After professional learning,
Confidence “I think I can do it on the job.”
Discussions following practice and feedback. Two times per month throughout the
year.
Survey items related to teachers’ confidence in delivering
instructional literacy strategies to improve student learning.
Survey to be completed at the
beginning, middle, and end of the
school year.
Commitment “I will do it on the job.”
Discussions following practice and feedback. Two times per month throughout the
year.
Discussions following principal coaching. Weekly.
Discussions following teacher peer observations. Quarterly.

APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

99
Level 1: Reaction
Kirkpatrick and Kirkpatrick (2016) state that it is important to measure the reaction of
participants in the training to measure their satisfaction, level of engagement, and whether or not
participants felt the training was relevant to their job. Table 17 lists the methods that will be
used to determine the reactions and level of engagement of the teachers to the training that will
be provided.

Table 17
Components to Measure Reactions to the Program
Method(s) or Tool(s) Timing
Engagement
Teachers’ reflection-on-practice During professional learning
Teacher collaboration Twice monthly
Principal and teacher coaching sessions Weekly
Student data analysis After every benchmark
Relevance
Brief pulse-check with teachers at the beginning of each
professional learning session.
Ongoing, beginning of professional
learning sessions
Professional learning evaluation surveys. After professional learning
Teacher Satisfaction
Brief pulse-check with teachers at the beginning of each
professional learning session.
Ongoing
Professional learning evaluation feedback survey regarding
the professional learning activities so that the input could be
used for future professional learning sessions.
After professional learning
Individual teacher check-ins with the principal. At least three times per quarter

APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

100
Evaluation Tools
Immediately following the program implementation. Brief Google surveys will be
administered to teachers after each professional learning event. RMS teachers will be asked to
indicate the relevance of the material to their job performance and their overall satisfaction with
the content and delivery of the training during the school year.
For Level 1, during the in-person workshop, the instructor will conduct periodic brief
pulse-checks by asking the participants about the relevance of the content to their work and the
organization, delivery, and learning environment. Level 2 will include checks for understanding
using games or competition among groups in responding to questions and scenarios drawn from
the content.
Delayed for a period after the program implementation. Approximately six weeks
after the implementation of the training, and then again at 15 weeks, leadership will administer a
survey containing open and scaled items using the Blended Evaluation approach to measure,
from the participant’s perspective, satisfaction and relevance of the training (Level 1),
confidence and value of applying their training (Level 2), application of the training to the
classroom implementation of best teaching practice (Level 3), and the extent to which their
implementation of the teaching practice has made a difference in student achievement in literacy.
Data Analysis and Reporting
The Level 4 (Results) goal of this implementation plan is measured by the time that the
organization gives to teachers to support their collaboration on learning objectives for
instructional strategies, collaboration to develop action plans from interim assessment data, the
percentage of students reading at or above standards, and community perception of the school.
The dashboard below in Table 18 will report the data on those areas as a monitoring and
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

101
accountability tool. Furthermore, the dashboard will represent when to correct problems during
the implementation of the student-generated questioning strategy. The success of RMS will be
to increase overall student achievement in reading.

Table 18
RMS Organization Success Data Dashboard
Key Measurement Target Actual Rating
Student Achievement 60% Proficient 47% Proficient

Collaboration Time on Learning Objectives Once a month Not yet
Collaboration Time on Data Action Plans Every ten weeks Not yet
School Perception 75% approval 76% approval
Notes: Positive change Same No change

Summary
The New World Kirkpatrick Model (Kirkpatrick & Kirkpatrick, 2016) was used as the
framework for planning, implementing, and evaluating the recommendations for Regent Middle
School to achieve its goal to apply and use a research-based instructional strategy schoolwide
and increase student reading achievement by 60%. The model is useful to gauge the
effectiveness of training by measuring it on four levels of results, behavior, learning, and
reaction. The model allows to assess progress while training is taking place in order to make the
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

102
necessary changes to be successful at achieving the training goals. Key data analysis questions
in the Kirkpatrick model are: Does the training meet expectations? If not, why not? If so, why?
The achievement of the stakeholder goal of using the student-generated question strategy and
measure student outcomes will be an leading indicator of the organization achieving its goal of
improving student reading achievement schoolwide.
Limitations and Delimitations
This study used a mixed method approach that included a survey, a focus group
interview, and document analysis. As noted in Chapter 3, I was the principal researcher in the
study and I was also the administrator of the school where I conducted my study. To avoid any
conflicts of interest, I used a CITI certified administrator from another school in my district to
conduct the focus group interview. The data from the focus group interview was pseudonymous
and transcribed by a transcription service. The use of an outside researcher impacted the
collection of the data in that some of the follow-up questions that the researcher posed and the
data collected from some of those follow-up questions were not pertinent to the study. For
example, the researcher asked the participants if they wanted more time during minimum days
that the district had collectively bargained with the teachers’ union. Adding more time to a
collectively bargained item was not useful to the study. The second impact to the study was that
the focus group interview responses were anonymous. There was no ability to disaggregate the
data by departments or grade levels. Also, there was no way to member check the data from the
focus group interview. One significant limitation of the instructional strategy of student-
generated questioning was that it was not implemented in the classrooms on a daily basis.
Therefore, the impact of the strategy upon student learning was limited in its possible
effectiveness.
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

103
Data collection was completed in a one and half month period and limited to one school
stakeholder, the teachers. These delimitations of the study were necessary in order to complete
the study. Other school stakeholders, including students, would have made the study results and
findings more salient since instructional practices affect them. Moreover, the study was limited
to a rural middle school; therefore, the recommendations put forth by this study could only be
used by schools with similar demographics of RMS.
Future Research
The establishment and efficacy of a professional learning community of teachers who
collectively address student learning outcomes is an important problem of practice that this study
began to address. For the purpose of this study, one instructional strategy was examined which
had been the focal point of the professional learning at RMS for the 2016-17 school year. The
study specifically measured teachers’ knowledge transference from professional learning into the
classroom, and analyzed if this would have any effect on student outcomes. The study showed
that the implementation of the instructional strategy was not enough to positively affect student
learning outcomes in the area of literacy. The limitation of the timeframe of the study and the
delimitation of only using teachers as stakeholders did not allow further investigation on students
and their value and understanding of the instructional strategy being used in the classroom. A
longitudinal study of students learning in the classroom in similar schools and districts would
add to the conversation of how implementation of teacher strategies in the classroom affects their
learning.
Though this study focused on the transference of one instructional strategy from
professional learning into the classroom, a possible future study could include studying the use
of multiple schoolwide strategies implemented on a daily basis. Additionally, the teachers in the
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

104
study repeatedly stated that they needed more time to reflect on their own implementation of the
practice with their colleagues. In future research, using the same organization, RMS, a
qualitative study on teacher interaction, specifically using the research of Grossman et al.’s
(2001) work on teacher community, could provide valuable information on how to better support
a professional learning community of teachers within the organization.
Conclusion
At Regent Middle School, and schools across the United States, the effective
establishment of a professional learning community is imperative in allowing teachers to learn
with their peers at work in order to improve student achievement. This study evaluated the
application of teachers’ professional learning of the research-based instructional strategy,
student-generated questioning, into the classroom and then measured its effectiveness on student
learning. Teachers were selected as the stakeholders of the study as they directly work with
students, and thus have the largest impact on student learning. The Clark and Estes (2008) gap
analysis model was used to examine the knowledge, motivation, and organizational influences on
RMS teachers. The framework revealed that teachers implemented professional learning
outcomes in the classroom, but had not yet developed the reflection-on-action to connect the
implementation to the student learning and the student outcomes. However, studying only one
strategy before adding more schoolwide strategies was beneficial to the teachers in having the
time to build collective efficacy. This reflection-on-action requires more time for teachers to
collaborate on learning objectives and data action plans during professional learning. The
organization has to support this need by providing more structured time for these activities.
When the organization provides more time for professional reflection, teachers improve their
ability to connect the art and science of teaching, which benefits all students.
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

105
REFERENCES
Aaronson, D., Barrow, L., & Sander, W. (2007). Teachers and student achievement in the
Chicago public high schools. Journal of Labor Economics, 25(1), 95–135.
Ambrose, S. A., Bridges, M. W., DiPietro, M., Lovett, M. C., & Norman, M. K. (2010). How
learning works: Seven research-based principles for smart teaching. San Francisco, CA:
John Wiley & Sons.
Anderson, L. W., & Krathwohl, D. R. (2001). A taxonomy for learning, teaching, and assessing:
A revision of Bloom’s taxonomy of educational objectives. New York, NY: Longman.
Baker, L. (2008). Metacognition in comprehension instructions. In C. C. Block & S. R. Parris
(Eds.), Comprehension instruction: Research-based best practices (pp. 65–79). New
York, NY: The Guilford Press.
Bandura, A. (2000). Exercise of human agency through collective efficacy. Current Directions in
Psychological Science, 9(3), 75–78.
Bill and Melinda Gates Foundation. (2014). Teachers know best: Teachers’ views on
professional development. Seattle, WA: Author. Retrieved
from http://k12education.gatesfoundation.org/wp-content/uploads/2015/04/Gates-
PDMarketResearch-Dec5.pdf
Birenbaum, M., Kimron, H., Shilton, H., & Shahaf-Barzilay, R. (2009). Cycles of inquiry:
Formative assessment in service of learning in classrooms and in school-based
professional communities. Studies in Educational Evaluation, 35(4), 130–149.
Borko, H. (2004). Professional development and teacher learning: Mapping the
terrain. Educational Researcher, 33(8), 3–15.
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

106
Bush, R. N. (1984). Effective staff development. In Making our schools more effective:
Proceedings of three state conferences. San Francisco, CA: Far West Laboratory.
California Department of Education. (2015). School accountability report card reported using
data from the 2013-14 school year. Sacramento, CA: Author.
California Department of Education. (2017). California Assessment of Student Performance and
Progress (CAASPP) results. Sacramento, CA: Author.
Camburn, E. M., & Han, S. W. (2015). Infrastructure for teacher reflection and instructional
change: An exploratory study. Journal of Educational Change, 16(4), 511–533.
doi:10.1007/s10833-015-9252-6
Cano García, F., García, Á., García-Berbén, A. B., Pichardo Martínez, M. D., & Justicia Justicia,
F. (2014). The effects of question-generation training on metacognitive knowledge, self
regulation and learning approaches in Science. Psicothema, 26(3), 385–390.
Clark, R., & Estes, F. (2008). Turning research into results: A guide to selecting the right
performance solutions. Charlotte, NC: Information Age.
Copland, M. A. (2003). Leadership of inquiry: Building and sustaining capacity for school
improvement. Educational Evaluation and Policy Analysis, 25(4), 375–395.
Creswell, J. W. (2014). Research design: Qualitative, quantitative, and mixed methods
approaches (4th ed.). Thousand Oaks, CA: Sage.
Darling-Hammond, L., & Jaquith, A. (2012). Creating a comprehensive system for evaluating
and supporting effective teaching. Stanford, CA: Stanford Center for Opportunity Policy
in Education (SCOPE).
Darling-Hammond, L., & Richardson, N. (2009). Research review/teacher learning: What
matters. Educational Leadership, 66(5), 46–53.
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

107
Desimone, L. M. (2009). Improving impact studies of teachers’ professional development:
Toward better conceptualizations and measures. Educational Researcher, 38(3), 181-199.
Desimone, L. M., Porter, A. C., Garet, M. S., Yoon, K. S., & Birman, B. F. (2002). Effects of
professional development on teachers’ instruction: Results from a three-year longitudinal
study. Educational Evaluation and Policy Analysis, 24(2), 81–112.
Dignath-van Ewijk, C., & van der Werf, G. (2012). What teachers think about self-regulated
learning: Investigating teacher beliefs and teacher behavior of enhancing students’ self-
regulation. Education Research International, 2012, 1–10.
DuFour, R. (2015). How PLCs do data right. Educational Leadership, 73(3), 22–26.
Eccles, J. S., & Wigfield, A. (2002). Motivational beliefs, values, and goals. Annual Review of
Psychology, 53(1), 109–132.
Elmore, R. F. (2002). Bridging the gap between standards and achievement. Washington, D.C.:
Albert Shanker Institute. Retrieved from http://www.shankerinstitute.org/resource/
bridging-gap-between-standards-and-achievement
Ermeling, B. A. (2010). Tracing the effects of teacher inquiry on classroom practice. Teaching
and Teacher Education, 26(3), 377–388.
ESSA (2015). Every Student Succeeds Act of 2015, Pub. L. No. 114-95 § 295 Stat. 1177 (2015-
2016).
Fink, A. (2013). How to conduct surveys: A step-by-step guide (5th ed.). Thousand Oaks, CA:
Sage.
Fishman, B. J., Marx, R. W., Best, S., & Tal, R. T. (2003). Linking teacher and student learning
to improve professional development in systemic reform. Teaching and Teacher
Education, 19(6), 643–658.
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

108
Fullan, M. (2001). The new meaning of educational change. New York, NY, and London, UK:
Teachers College Press and Routledge Falmer.
Gallimore, R., & Goldenberg, C. (2001). Analyzing cultural models and settings to connect
minority achievement and school improvement research. Educational Psychologist,
36(1), 45–56.
Garet, M. S., Porter, A. C., Desimone, L., Birman, B. F., & Yoon, K. S. (2001). What makes
professional development effective? Results from a national sample of
teachers. American Educational Research Journal, 38(4), 915–945.
Gibson, S., & Dembo, M. H. (1984). Teacher efficacy: A construct validation. Journal of
Educational Psychology, 76(4), 569–582.
Goddard, R. D., Hoy, W. K., & Hoy, A. W. (2000). Collective teacher efficacy: Its meaning,
measure, and impact on student achievement. American Educational Research
Journal, 37(2), 479–507.
Grossman, P., Wineburg, S., & Woolworth, S. (2001). Toward a theory of teacher
community. The Teachers College Record, 103, 942–1012.
Gulamhussein, A. (2013). Teaching the teachers: Effective professional development in an era of
high stakes accountability. Alexandria, VA: Center for Public Education.
Guskey, T. R. (1989). Attitude and perceptual change in teachers. International Journal of
Educational Research, 13(4), 439–453.
Guskey, T. R. (2002). Professional development and teacher change. Teachers and
Teaching, 8(3), 381–391.
Guskey, T. R., & Yoon, K. S. (2009). What works in professional development? Phi Delta
Kappan, 90(7), 495–500.
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

109
Hattie, J. A. C. (2009). Visible learning: A synthesis of 800+ meta-analyses on achievement.
Abingdon, UK: Routledge.
Hattie, J. A. C., Masters, D., & Birch, K. (2016). Visible learning into action: International case
studies of impact. Abingdon, UK: Routledge.
Horn, I. S., & Little, J. W. (2010). Attending to problems of practice: Routines and resources for
professional learning in teachers’ workplace interactions. American Educational
Research Journal, 47(1), 181–217.
Joyce, B. R., & Showers, B. (2002). Student achievement through staff development.
Nottingham, UK: National College for School Leadership.
King, A. (1992). Facilitating elaborative learning through guided student-generated
questioning. Educational Psychologist, 27(1), 111–126.
Kirkpatrick, J. D., & Kirkpatrick, W. K. (2016). Four levels of training evaluation. Alexandria,
VA: ATD Press.
Knowles, M. S. (1970). The modern practice of adult education (Vol. 41). New York, NY: New
York Association Press.
Krathwohl, D. R. (2002). A revision of Bloom’s Taxonomy: An overview. Theory into Practice,
41(4), 212–218.
Krueger, R. A., & Casey, J. (2009). Successful focus groups: Practical guidelines for research.
Thousand Oaks, CA: Sage.
Le Fevre, D. M. (2014). Barriers to implementing pedagogical change: The role of teachers’
perceptions of risk. Teaching and Teacher Education, 38, 56-64.
doi:10.1016/j.tate.2013.11.007
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

110
Lombaerts, K., Engels, N., & Van Braak, J. (2009). Determinants of teachers’ recognitions of
self-regulated learning practices in elementary education. Journal of Educational
Research, 102(3), 163–174.
Louis, K. S., Marks, H. M., & Kruse, S. (1996). Teachers’ professional community in
restructuring schools. American Educational Research Journal, 33(4), 757–798.
Lucas, S. R. (2001). Effectively maintained inequality: Education transitions, track mobility, and
social background effects. American Journal of Sociology, 106(6), 1642–1690.
Malloy, C. L. (2011). Ten steps to effective practitioner-led inquiry. Edge, 6(4), 6–16.
Marsh, J. A., Pane, J. F., & Hamilton, L. S. (2006). Making sense of data-driven decision making
in education (RAND Education occasional paper). Santa Monica, CA: RAND
Corporation. Retrieved from http://www.rand.org/pubs/occasional_papers/OP170.html
Maxwell, J. A. (2013). Qualitative research design: An interactive approach (3rd ed.). Thousand
Oaks, CA: Sage.
Mayer, R. E. (2011). Applying the science of learning. Boston, MA: Pearson Education.
McLaughlin, M. W., & Talbert, J. E. (2006). Building school-based teacher learning
communities: Professional strategies to improve student achievement. New York, NY:
Teachers College Press.
McLeskey, J., & Waldron, N. L. (2004). Three conceptions of teacher learning: Exploring the
relationship between knowledge and the practice of teaching. Teacher Education and
Special Education, 27(1), 3-14.
Merriam, S. B., & Tisdell, E. J. (2016). Qualitative research: A guide to design and
implementation (4th ed.). San Francisco, CA: Jossey-Bass.
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

111
Miles, M. B., Huberman, A. M., & Saldana, J. (2014). Qualitative data analysis: A methods
sourcebook. Thousand Oaks, CA: Sage.
Moos, D. C., & Ringdal, A. (2012). Self-regulated learning in the classroom: A literature review
on the teacher’s role. Education Research International, 2012, 1–15.
Moran, J. W., & Brightman, B. K. (2000). Leading organizational change. Journal of Workplace
Learning, 12(2), 66–74.
National Center for Education Statistics. (2011). The Nation’s Report Card: Reading 2011
(NCES 2012–457). Washington, D.C.: Institute of Education Sciences, U.S. Department
of Education.
National Center for Education Statistics. (2017). Education Department budget history table: FY
1980–FY 2018 President’s budget. Washington, D.C.: Institute of Education Sciences,
U.S. Department of Education. Retrieved from
https://www2.ed.gov/about/overview/budget/history/index.html
Pajares, F. (1996). Self-efficacy beliefs in academic settings. Review of Educational Research,
66(4), 543–578.
Patton, M. Q. (2002). Qualitative interviewing. Qualitative Research and Evaluation Methods, 3,
344–347.
Pazzaglia, A. M., Stafford, E. T., & Rodriguez, S. M. (2016). Survey methods for educators:
Analysis and reporting of survey data (part 3 of 3) (REL 2015-164). Washington, D.C.:
U.S. Department of Education, Institute of Education Sciences, National Center for
Education Evaluation and Regional Assistance, Regional Educational Laboratory
Northeast & Islands.
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

112
Pintrich, P. (2003). A motivational science perspective on the role of student motivation in
learning and teaching contexts. Journal of Educational Psychology, 95(4), 667–686.
Rivkin, S. G., Hanushek, E. A., & Kain, J. F. (2005). Teachers, schools, and academic
achievement. Econometrica, 73(2), 417–458.
Rockoff, J. E. (2004). The impact of individual teachers on student achievement: Evidence from
panel data. The American Economic Review, 94(2), 247-252.
Rothstein, D., & Santana, L. (2011). Make just one change: Teach children to ask their own
questions. Cambridge, MA: Harvard Education Press.
Royster, P., Gross, J., & Hochbein, C. (2015). Timing is everything: Getting students back on
track to college readiness in high school. The High School Journal, 98(3), 208–225.
Rueda, R. (2011). The 3 dimensions of improving student performance. New York, NY:
Teachers College Press.
Schildkamp, K., & Poortman, C. (2015). Factors influencing the functioning of data teams.
Teachers College Record, 117(4), 1-42.
Schön, D. A. (1987). Educating the reflective practitioner: Toward a new design for teaching
and learning in the professions. San Francisco, CA: Jossey-Bass.
Slayton, J., & Mathis, J. (2010). Building the leaders we need: The role of presence, learning
conditions, and andragogy in developing leaders who can change the face of public pre-K
through 12 education. In A. H. Normore (Ed.), Global perspectives on educational
leadership reform: The development and preparation of leaders of learning and learners
of leadership (pp. 23–45). Bingley, UK: Emerald Group.
Spruce, R., & Bol, L. (2015). Teacher beliefs, knowledge, and practice of self-regulated learning.
Metacognition and Learning, 10(2), 245–277.
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

113
Stewart, C. (2014). Transforming professional development to professional learning. Journal of
Adult Education, 43(1), 28–33.
Strauss, A., & Corbin, J. (1990). Basics of qualitative research: Grounded theory procedures
and techniques. Newbury Park, CA: Sage.
Thoonen, E. E. J., Sleegers, P. J. C., Oort, F. J., Peetsma, T. T. D., & Geijsel, F. P. (2011). How
to improve teaching practices: The role of teacher motivation, organizational factors, and
leadership practices. Educational Administration Quarterly, 47(3), 496–536.
Timperley, H. (2008). Teacher professional learning and development (Educational Practices
Series No. 18). Brussels, Belgium: International Academy of Education.
Tschannen-Moran, M., & McMaster, P. (2009). Sources of self-efficacy: Four professional
development formats and their relationship to self-efficacy and implementation of a new
teaching strategy. The Elementary School Journal, 110(2), 228–245. doi:10.1086/605771
Tyler, R. W. (1949). Basic principles of curriculum and instruction. Chicago, IL: University of
Chicago Press.
U.S. Department of Education. (2010). State and local implementation of the No Child Left
Behind Act: Volume VIII — Teacher quality under NCLB: Final report. Washington,
D.C.: Author. Retrieved from
https://www2.ed.gov/about/offices/list/opepd/ppss/reports.html
Wei, R. C., Darling-Hammond, L., Andree, A., & Richardson, N., & Orphanos, S. (2009).
Professional learning in the learning profession: A status report on teacher development
in the United States and abroad. Dallas, TX: National Staff Development Council.
Weiss, R. S. (1994). Learning from strangers. New York, NY: Free Press.
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

114
Wilson, N. S., & Bai, H. (2010). The relationships and impact of teachers’ metacognitive
knowledge and pedagogical understandings of metacognition. Metacognition and
Learning, 5(3), 269–288.
Wong, B. Y. (1985). Self-questioning instructional research: A review. Review of Educational
Research, 55(2), 227–268.
Yoon, K. S., Duncan, T., Lee, S. W. Y., Scarloss, B., & Shapley, K. L. (2007). Reviewing the
evidence on how teacher professional development affects student achievement (REL
2007-No. 033). Washington, D.C.: U.S. Department of Education, Institute of Education
Sciences, National Center for Education Evaluation and Regional Assistance, Regional
Educational Laboratory Southwest.
Zemke, R., & Zemke, S. (1984). 30 things we know for sure about adult learning. Innovation
Abstracts (Vol. VI, No. 8). Austin: National Institute for Staff and Organizational
Development, University of Texas at Austin.
Zimmerman, B. J. (1989). Models of self-regulated learning and academic achievement. In B. J.
Zimmerman & D. H. Schunk (Eds.), Self-regulated learning and academic achievement
(pp. 1-25). New York, NY: Springer.
Zmuda, A., Kuklis, R., & Kline, E. (2004). Transforming schools: Creating a culture of
continuous improvement. Alexandria, VA: ASCD.

APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

115
APPENDIX A
SURVEY PROTOCOL
In this survey, professional learning (PL) is defined as activities that develop an educator’s skills,
knowledge, expertise and other characteristics to better serve students in a teacher professional
development setting. Please only consider PL that you have participated in the 2016-17 academic
school year.

This study is for Annie Rinaldi’s doctoral dissertation. The purpose of the study is to understand
how teachers’ learning of the student-generated questioning strategy during professional
development at Regent Middle School (RMS) has transferred to classroom instruction and to
determine if this had any effect on student achievement in reading comprehension. Another goal
of the study is to understand how to better support teachers’ practice with the use of professional
development.

All data gathered from the results of this survey will remain confidential through anonymity
generated by Survey Monkey. Your participation in this survey is greatly appreciated.

Knowledge Influences on Professional Learning (PL)

1. In the last six months that you participated in PL in your school, please indicate the extent to
which you have such needs in each of the areas listed:
a. Collaborative cycle for data analysis of interim assessments.
□ No Need □ Low Level of Need □ Moderate Level of Need □ High Level of Need
b. Collaborative cycle for data analysis of CAASPP assessments
□ No Need □ Low Level of Need □ Moderate Level of Need □ High Level of Need
c. Training on the student-generated questioning strategy
□ No Need □ Low Level of Need □ Moderate Level of Need □ High Level of Need

2. I consider myself an expert in data analysis to guide my classroom instruction.
□ Strongly disagree □ Disagree □ Agree □ Strongly agree

3. I consider myself an expert in the use of the student-generated questioning strategy
□ Strongly disagree □ Disagree □ Agree □ Strongly agree

4. In the last six months, I have used data analysis to guide my classroom instruction…
0 times 1-2 times 3-4 times more than 4 times

5. In the last six months, I have used student-generated questioning strategy in my classroom
instruction…
0 times 1-2 times 3-4 times more than 4 times

6. In the last six months that you participated in PL in your school, did you want to participate
in more time in PL:
□ Yes □ No (if No, please skip to question number 5)
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

116
7. If you marked yes in question number 3, please indicate which of the following reasons best
explains which activities you would have liked to participate if you had more time in PL.
Please rank the activities in order of importance to you (1 being the most important and 5
being the least important).

Collaborative cycle for data analysis of interim assessments.____
Training on student-generated questioning.____
Coaching/feedback on student-generated questioning.____
Planning lessons on student-generated questioning.____

Motivation Influences on Professional Learning (PL)

8. I value the use of student-generated questioning to develop students’ literacy.
□ Strongly disagree □ Disagree □ Agree □ Strongly agree

9. In the last six months that you participated in PL in your school, how did the following
activities impact your classroom instruction:
a. Collaborative cycle for data analysis of interim assessments.
□ No Impact □ A small impact □ A moderate impact □ A large impact
b. Coaching/feedback on student-generated questioning
□ No Impact □ A small impact □ A moderate impact □ A large impact
c. Training on student-generated questioning
□ No Impact □ A small impact □ A moderate impact □ A large impact

10. Please rate the value in participating in PL for the last sixth months.
□ No Value □ Little Value □ Moderate Value □ High Value

11. Please rate the impact of your participation in PL on student learning.
□ No Impact □ A small impact □ A moderate impact □ A large impact

Organizational Influences on Professional Learning (PL)

12. There is enough time for PL.
□ Strongly disagree □ Disagree □ Agree □ Strongly agree

13. The school principal supports the PL process.
□ Strongly disagree □ Disagree □ Agree □ Strongly agree

14. The principal’s feedback is helpful in my teaching practice.
□ Strongly disagree □ Disagree □ Agree □ Strongly agree

15. The peer coaching is helpful in my teaching practice.
□ Strongly disagree □ Disagree □ Agree □ Strongly agree

APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

117
APPENDIX B
FOCUS GROUP INTERVIEW PROTOCOL
Name of Facilitator:___________________________________________________________
Names of Participants:_________________________________________________________
___________________________________________________________________________
Time Started:___________________ Time Ended:_____________________ Total time:____
Hello, my name is XXXXXXXX and I am a doctoral student at the University of Southern
California and the colleague of the principal of RMS, who is also a doctoral student at the
University of Southern California and the primary researcher for this study.

Thank you for agreeing to participate in this study to evaluate professional development at RMS.
I really appreciate your time to share your thoughts with me. The focus group interview should
not last longer than two hours.

Before we get started, I would like to give you an overview of the study and address any
questions you may have about being a participant in this study.

This study is for Annie Rinaldi’s doctoral dissertation. The purpose of the study is to understand
how teachers’ learning of student-generated questioning during professional development at
RMS is transferred to classroom instruction, and to determine if this had any effect on student
achievement in reading comprehension. Another goal of the study is to understand how to better
support teachers’ practice with the use of professional development.

I want to assure you that I am not here as a supervisor. I am here as a researcher for this study
and the data I collect will be used for that purpose. This means that the questions are non-
evaluative and your answers will not be used in teacher evaluations. All data gathered during this
interview will remain confidential through the use of pseudonyms that you will choose, and
interview data will be restricted solely for dissertation research. Also, the audio will be
transcribed by a third party before given to Annie Rinaldi to ensure anonymity of all the
participants.

The results of the study may be used as the basis for presentations to administration, teachers,
and staff at RMS, as well as others, to inform changes in professional development and the use
of student-generated questioning to support student reading achievement at RMS. However, the
information in these reports will not name you and will not use any information about you that
will identify you. I am happy to provide you with a copy of the completed dissertation if you are
interested. Participation in this study is voluntary and you may choose to not answer any
question you do not feel like answering or withdraw from the interview at any time.

APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

118
Do you have any questions about the study before we begin? If you do not have any more
questions, I would like everyone’s permission to begin. May I also have your permission to
audio record our interview to better capture the conversation? Again, no one other than I will
have access to these recordings.

Opening

1. Since it is a small group, we will refrain from asking who you are to protect your anonymity
and ask a general question: What are your goals for your students at RMS?

Knowledge Influences on Professional Learning (PL)

2. Please describe your experiences in professional learning (PL) during minimum day Fridays
and how did those experiences impact your teaching practices in the classroom.

3. Please tell me about the goals of the PL at RMS.
a. How does the PL support student learning, if at all.

4. Please describe your work in the PL as it relates to collaborating with other teachers.
a. How do you work with other teachers?
b. How did you learn to collaborate with other teachers?
c. What kind training have you had in learning how to collaborate?
d. What has been your most favorable experience working with other teachers?
e. What has been your least favorable experience working with other teachers?

5. Please explain to me what the student-generated questioning strategy is.

6. Please walk me through some of the important steps you on the PL work with the student-
generated questioning strategy implementation in the classroom.
a. What does it look like during the lesson with the students?
b. What would I see if I were in the classroom?
c. What would I hear if I walked around the classroom?
d. Tell me what students would be doing.
e. What else?

7. What is the most significant factor that challenged you in implementing student-generated
questioning?

8. What is the most significant factor that assisted you in implementing student-generated
questioning?

9. Please describe the resources that are working and should stay in regards to the use of PL in
RMS.

APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

119
Motivation Influences on Professional Learning (PL)

10. Please describe the value you see in participating in PL, if any.
a. If you don’t see any value, please tell more about that.

11. Please tell me how you feel about your work in the PL.
a. Do you think PL influences your teaching practice? If so, how?

12. Please tell me about a time you used student-generated questioning in the classroom and
brought the information back to the PL.

Organizational Influences on Professional Learning (PL)

13. Please describe your experiences with the use of student data at RMS.
a. How do you feel about those experiences?

14. Please tell me how you analyze student data.

15. Please tell me about the influence of data analysis on the work of PL if any.

16. What are some constraints on the work of the PL?” How do you feel about that influence?

17. How is the principal involved in the PL? What is her role?
a. Please tell about the influence of the principal on the work of PL.
b. How do you feel about that influence?

Closing

18. Please tell me how you think that PL may be improved at RMS?

19. Please tell me about anything else you would like to share.

APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

120
APPENDIX C
DOCUMENTS AND ARTIFACTS ANALYSIS

Artifacts Related to
Professional Learning
Outcomes Observations Date Collected and Location

APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

121
APPENDIX D
INFORMED CONSENT/INFORMATION SHEET
University of Southern California
USC Rossier School of Education
3470 Trousdale Pkwy, Los Angeles, CA 90089

INFORMATION/FACTS SHEET FOR EXEMPT NON-MEDICAL RESEARCH

The Impact of Professional Development at Mammoth Middle School on the School-wide
Implementation of Student-Generated Questioning Strategy to Improve Students’ Critical
Thinking Skills

You are invited to participate in a research study. Research studies include only people who
voluntarily choose to take part. This document explains information about this study. You should
ask questions about anything that is unclear to you.

PURPOSE OF THE STUDY
The purpose of this study is to understand the impact that Mammoth Middle School’s
professional development has had with the school-wide implementation of the student-generated
questioning strategy and its potential benefits to student educational outcomes in reading
achievement.

PARTICIPANT INVOLVEMENT
If you agree to participate in the study, you will be asked to fill out a survey, which is anticipated
to take about 5 minutes. You do not have to answer any questions you don’t want to answer. You
will also participate in two semi-structured focus group interviews lasting approximately 45
minutes each. One interview will be at the beginning of the study and the other interview will
occur towards the end of the study. Guiding questions will be asked but the interview will be
conversational and follow-up questions may be asked as well. The focus group conversations
will be audio recorded. You do not have to answer any questions you don’t want to answer. If
you do not want to be recorded, you will not be able to participate in the study.

ALTERNATIVES TO PARTICIPATION
Your alternative is to not participate. Your relationship with Mammoth Unified School District
will not be affected whether you participate or not in this study.

CONFIDENTIALITY
There will be no identifiable information obtained in connection with this study. Your name,
address or other identifying information will not be collected. Any identifiable information
obtained in connection with this study will remain confidential. Your responses will be coded
with a false name (pseudonym) and maintained separately. The audiotapes will be destroyed
once they have been transcribed. The data will be stored on a password-protected computer in
the researcher’s office for three years after the study has been completed, and then destroyed.
APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

122
The members of the research team and the University of Southern California’s Human Subjects
Protection Program (HSPP) may access the data. The HSPP reviews and monitors research
studies to protect the rights and welfare of research subjects.

INVESTIGATOR CONTACT INFORMATION
Principal Investigator Annie Rinaldi via email at arinaldi@usc.edu or phone at (213) 280-9763.

IRB CONTACT INFORMATION
University Park Institutional Review Board (UPIRB), 3720 South Flower Street #301, Los
Angeles, CA 90089-0702, (213) 821-5272 or upirb@usc.edu

APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

123
APPENDIX E
ASSESSMENT AND GRADING INITIAL WORKSHOP EVALUATION
The purpose of this survey is to evaluate the training provided regarding the effectiveness of the
student-generated questioning and/or creating action plans using interim data assessments. Your
feedback is important to assess the quality of this training, as well as, providing the appropriate
supports that are needed to reinforce your learning. Future training will include your input from
this survey. Thank you.

Scale 1-4 (Strongly disagree to strongly agree)

1. I know what student-generated questioning strategy is and how to use it (L2: Declarative)
2. I believe the student-generated questioning strategy is valuable to student learning (L2:
Attitude)
3. I feel confident about applying what I learned today into the classroom (L2: Confident)
4. The professional learning today held my interest (L1: Engagement)
5. I am clear of what is expected of me when I apply student-generated questioning into the
classroom (L1: Relevance)

Open-ended questions

6. How do you plan to apply what you learned today to your job (L2: Procedural)
7. Please describe the process of student-generated questioning (L2: Commitment)
8. Please explain the likelihood that you are to share your reflection on the use of student-
generated questioning in the next professional learning meeting (L1: Reaction)

Scale 1-4 (Strongly disagree to strongly agree)

1. What I learned in the workshop has been very valuable to improve student learning
outcomes (L1: Relevance)
2. I was able to apply student-generated questioning fully after professional learning than
before it. (L2: Learning)
3. My peers (colleagues) and I use the infographic strategy card to implement every step of
student-generated questioning strategy (L3: Behavior)
4. I am able to analyze student work and data to adapt my use of the instructional strategy
(L4: Results)

APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

124
APPENDIX F
ASSESSMENT AND GRADING PROGRAM EVALUATION
The purpose of this survey is to evaluate the ongoing training and professional development you
have participated in regarding the implementation of student-generated questioning strategy into
the classroom. Your responses will assist us in understanding your level of satisfaction, your
level of learning, how you have implemented the training in your position, and how successful
the program has been in helping you transfer your knowledge from professional learning into the
classroom.

Scale 1-4 (Strongly disagree to strongly agree)

L1: Reaction

1. What I learned in professional learning has been useful in improving reading
comprehension.
2. Professional learning this year has been a good use of my time.

L2: Learning

3. I find that the strategy that I learned during professional learning is valuable to student
learning.

L3: Behavior

4. In the professional learning community, I discuss with my department team ways to use
the instructional strategy to best support my students’ learning.

L4: Results

5. The schoolwide practices that I have learned is helping my students in reading
comprehension.

APPLICATION OF PROFESSIONAL LEARNING OUTCOMES

125
APPENDIX G
STUDENT DATA WALL

Abstract (if available)

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

Overcoming the cultural teaching gap: an evaluative study of urban teachers’ implementation of culturally relevant instruction

PDF

Collaborative instructional practice for student achievement: an evaluation study

PDF

Narrowing the English learner achievement gap through teacher professional learning and cultural proficiency: an evaluation study

PDF

Closing the achievement gap for students with disabilities: a focus on instructional differentiation - an evaluation study

PDF

Nourish to flourish: strengthening social emotional wellness of teachers to mitigate stress, enrich engagement, and increase efficacy: an evaluation study

PDF

Equitable schooling for African American students: an evaluation study

PDF

Approaches to teaching for twenty-first century learners in South Korea: An evaluation study of GSS

PDF

Assessing the meaning and value of traditional grading systems: teacher practices and perspectives

PDF

Improving instructor skills (IIS): a Needs analysis

PDF

Student engagement in online education: an evaluation study

PDF

An evaluation of project based learning implementation in STEM

PDF

Digital learning in K12: putting teacher professional identity on the line

PDF

College and career readiness through independent study: an innovation study

PDF

Educator professional development for technology in the classroom: an evaluation study

PDF

Creating conditions for Teacher Flow: supporting student-centered learning through design of optimal P-12 professional development

PDF

Employment rates upon MBA graduation: An evaluation study

PDF

Teacher retention influences: an evaluation study

PDF

Curriculum mapping initiative: an evaluation study

PDF

The knowledge, motivation, and organization influences affecting the frequency of empathetic teaching practice used in the classroom: an evaluation study

PDF

Sustained mentoring of early childhood education teachers: an innovation study

Asset Metadata

Creator Rinaldi, Ann Elizabeth (author)

Core Title Application of professional learning outcomes into the classroom: an evaluation study

School Rossier School of Education

Degree Doctor of Education

Degree Program Organizational Change and Leadership (On Line)

Publication Date 02/15/2018

Defense Date 12/20/2018

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag adult learning,adult learning theory,adult learning transfer,andragogy,Clark and Estes gap analysis,collective efficacy,middle school,OAI-PMH Harvest,professional learning,Professional Learning Community,schoolwide practice,student learning outcomes

Language English

Contributor Electronically uploaded by the author (provenance)

Advisor Stowe, Kathy (committee chair), Datta, Monique (committee member), Hanson, Kathy (committee member)

Creator Email arinaldi@usc.edu,rinaldi.annie@gmail.com

Permanent Link (DOI) https://doi.org/10.25549/usctheses-c40-474444

Unique identifier UC11267449

Identifier etd-RinaldiAnn-6042.pdf (filename),usctheses-c40-474444 (legacy record id)

Legacy Identifier etd-RinaldiAnn-6042.pdf

Dmrecord 474444

Document Type Dissertation

Rights Rinaldi, Ann Elizabeth

Type texts

Source University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Access Conditions The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the a...

Repository Name University of Southern California Digital Library

Repository Location USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA