Close
About
FAQ
Home
Collections
Login
USC Login
Register
0
Selected
Invert selection
Deselect all
Deselect all
Click here to refresh results
Click here to refresh results
USC
/
Digital Library
/
University of Southern California Dissertations and Theses
/
Efficacy of non-formal education programs in educational outcomes of marginalized Filipino children: an evaluation study
(USC Thesis Other)
Efficacy of non-formal education programs in educational outcomes of marginalized Filipino children: an evaluation study
PDF
Download
Share
Open document
Flip pages
Contact Us
Contact Us
Copy asset link
Request this asset
Transcript (if available)
Content
Efficacy of Non-Formal Education Programs in Educational Outcomes of Marginalized
Filipino Children: An Evaluation Study
by
Kevin E. Cross
A Dissertation Presented to
FACULTY OF THE USC ROSSIER SCHOOL OF EDUCATION
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulfillment of the
Requirements for the Degree
DOCTOR OF EDUCATION
December 2020
Copyright 2020 Kevin E. Cross
ii
Dedication
To my wife Jennifer and my daughter Natalie
To the children of the Philippines
iii
Acknowledgments
A dissertation is never truly the product of one individual and mine was no exception. I
would like to express my heartfelt thanks to my dissertation chair, Dr. Emmy Min, for her
patience, responsiveness, thoughtfulness, and guidance. Thank you for dispelling doubt, building
confidence, bringing understanding, and shepherding me through this process. I am eternally
grateful.
To my committee members, Dr. Monique Datta and Dr. Cathy Krop, thank you for your
feedback on my imperfect drafts, your thought-provoking questions, and your insightful
comments. Dr. Datta, I was truly blessed to start out this journey with you as my professor and
complete it with you as my friend. I will never forget your kindness and the many times you said
the exact words I needed to hear to pursue this journey to its end. To Dr. Eric Canny, my writing
advisor, thank you for looking for the rhythm in my writing and making it sing. I could not have
asked for a more incredible committee.
To Cohort 11 of the Organizational Change and Leadership program, thank you for your
intelligence, humor, and friendship over the past 2 years. Among the many incredible individuals
I met on this journey, I want to acknowledge the friendship of Sarah Stashkiw and Amy
Stoplestad, whose intelligence, humor, and support were a constant source of strength and
inspiration. Special thanks also to Rev. Dr. Christopher Dreisbach, my ethics professor, mentor,
colleague, and friend of 30 years, who was an early and ardent believer in my dream to complete
my doctorate.
To my family, thank you for your enduring patience, love, and support. My parents, Dr.
Louis L. Cross, Jr. and Alice Severn Cross, may they rest in peace, could not share in this
achievement, but nonetheless made it possible through their love and guidance. Thank you to my
iv
sister Paula, for her support, encouragement, and for teaching me not to take myself quite so
seriously. To Jeeves, who shared the long road with me, but did not see its end.
To my wife, Jennifer, to whom this dissertation is dedicated, words cannot adequately
express my gratitude for the many ways in which you supported me over the past 2 years. Thank
you for every extra chore you finished, for every extra errand you ran, and for every moment of
sleep you sacrificed while I labored over my dissertation. To my daughter Natalie, who inspired
my topic and inspired me, thank you for being completely and utterly confident in me even on
days when I did not feel too confident in myself. Without the love, encouragement, and support
of these two women, this journey would not have been possible.
v
Table of Contents
Dedication ....................................................................................................................................... ii
Acknowledgments.......................................................................................................................... iii
List of Tables ................................................................................................................................ vii
List of Figures .............................................................................................................................. viii
Abstract .......................................................................................................................................... ix
Chapter One: Introduction to the Problem of Practice.................................................................... 1
Organizational Context and Mission ........................................................................................... 1
Organizational Goal .................................................................................................................... 3
Related Literature ........................................................................................................................ 3
Importance of Evaluation ............................................................................................................ 5
Description of the Stakeholder Groups ....................................................................................... 6
Stakeholder Groups’ Performance Goals .................................................................................... 7
Stakeholder Group for the Study ................................................................................................. 7
Purpose of the Project and Questions .......................................................................................... 8
Methodological Framework ........................................................................................................ 8
Definitions ................................................................................................................................... 9
Organization of the Project ......................................................................................................... 9
Chapter Two: Review of the Literature ........................................................................................ 11
Education as a Means of Realizing Human Potential ............................................................... 11
Non-formal Education ............................................................................................................... 13
Non-formal Education in the Philippines .................................................................................. 20
Challenges in Assessing Efficacy of Philippine Non-formal Education Initiatives .................. 24
The Clark and Estes (2008) Gap Analytic Conceptual Framework .......................................... 26
Stakeholder Knowledge, Motivation, and Organizational Influences ..................................... 28
Conclusion ................................................................................................................................. 52
Chapter Three: Methods ............................................................................................................... 54
Participating Stakeholders ......................................................................................................... 55
Qualitative Data Collection and Instrumentation ...................................................................... 58
Credibility and Trustworthiness ................................................................................................ 65
Ethics ......................................................................................................................................... 66
Limitations and Delimitations ................................................................................................... 69
Chapter Four: Results and Findings .............................................................................................. 70
Stakeholders .............................................................................................................................. 71
Research Question 1: What are LEAP education team members’ knowledge and motivation
related to achieving this organizational goal? ........................................................................... 73
vi
Research Question 2: What is the interaction between organizational culture and context and
LEAP team educator knowledge and motivation? .................................................................... 91
Summary ................................................................................................................................... 95
Chapter Five: Solutions and Recommendations ........................................................................... 97
Introduction and Overview........................................................................................................ 97
Recommendations for Practice to Address KMO Influences ................................................... 98
Integrated Implementation and Evaluation Plan ..................................................................... 120
Strengths and Weaknesses of the Approach ........................................................................... 137
Limitations and Delimitations ................................................................................................. 139
Recommendations for Future Research .................................................................................. 140
Conclusion ............................................................................................................................... 141
References ................................................................................................................................... 143
Appendix A: Interview Protocol ................................................................................................. 155
Appendix B: Focus Group Interview Protocol ........................................................................... 164
Appendix C: Central Visayas Children’s Center Student Survey – Q3 2020............................. 169
Appendix D: Survey to Administer Immediately Following the Training Session .................... 173
Appendix E: Survey to Administer Immediately Following Each Quarterly PLC Session ....... 176
vii
List of Tables
Table 1: Organizational Mission, Global Goals, and Stakeholder Goals ....................................... 7
Table 2: Knowledge Influences, Types, and Assessments for Analysis ....................................... 36
Table 3: Motivation Influences and Assessments for Analysis .................................................... 40
Table 4: Organizational Influences and Assessments for Analysis .............................................. 47
Table 5: LEAP Team Composition and Education-Related Responsibilities .............................. 55
Table 6: Composition of LEAP Team Sample and Education-Related Responsibilities ............. 72
Table 7: Validation of Conceptual, Procedural, and Metacognitive Knowledge Findings .......... 75
Table 8: Summary of LEAP Team Member Proficiencies in Knowledge Influences by Role .... 75
Table 9: Validation of Motivation Findings ................................................................................. 86
Table 10: Summary of LEAP Team Member Proficiencies in Motivation Influences by Role ... 86
Table 11: Validation of Organizational Findings ......................................................................... 91
Table 12: Summary of CVCC’s Performance in Addressing Organization Influences ............... 92
Table 13: Summary of Knowledge Influences and Recommendations ........................................ 99
Table 14: Summary of Motivation Influences and Recommendations ...................................... 107
Table 15: Summary of Organization Influences and Recommendations ................................... 112
Table 16: Outcomes, Metrics, and Methods for External and Internal Outcomes ..................... 122
Table 17: Critical Behaviors, Metrics, Methods, and Timing for Evaluation ............................ 124
Table 18: Required Drivers to Support Critical Behaviors ......................................................... 126
Table 19: Evaluation of the Components of Learning for the Program ..................................... 130
Table 20: Components to Measure Reactions to the Program .................................................... 132
Table 21: Sample Report Extract detailing LEAP Team Members’ Perceptions of Program
Efficacy versus Metric-based Performance Outcomes ............................................................... 135
viii
List of Figures
Figure 1: Conceptual Framework of Stakeholder Knowledge, Motivation, Organizational, and
Systemic Influences. ..................................................................................................................... 50
Figure 2: Applications of Selected Methods to Conceptual Framework ...................................... 60
Figure 3: Sample Report Extract Depicting Participant Self-perception of Level 2 Outcomes
Prior to and Immediately Following Initial Training .................................................................. 135
ix
Abstract
This study evaluated the efficacy of the Central Visayas Children’s Center’s (CVCC) Learning
and Educational Assistance (LEAP) program, a supplemental non-formal education (NFE)
program designed to improve the performance of marginalized Filipino children in formal
schools. Using the Clark and Estes (2008) gap analytic conceptual framework, the study
leveraged qualitative interviews and document analysis to analyze knowledge, motivation, and
organizational influences in relation to CVCC’s innovative NFE approach. Findings from the
study indicated that CVCC’s LEAP team needs to address conceptual and procedural knowledge
gaps related to identifying student performance challenges and developing corresponding NFE
programs. As part of this effort, CVCC’s LEAP team needs to close knowledge gaps related to
benchmarking and measuring student performance as a basis for assessing NFE program
efficacy. From an organizational perspective, the findings indicated that CVCC also needs to
provide opportunities to foster LEAP team educators’ mastery level goal orientation in
developing NFE instruction. Specific recommendations for addressing these gaps are provided,
as well as an accompanying implementation and evaluation framework based on Kirkpatrick and
Kirkpatrick’s (2016) New World Model. This study provides context for the complexities
involved in developing supplemental NFE capacities and specific recommendations for
addressing challenges in measuring the efficacy of an NFE implementation in its nascent stages.
Keywords: non-formal education; alternative education, education for all, educational
access, equity and quality, out-of-school children and youth, human rights, sustainable
development goals
1
Chapter One: Introduction to the Problem of Practice
The large number of underserved and out-of-school children (OOSC) around the globe
highlights the need to examine the efficacy of non-formal education (NFE) programs designed to
provide marginalized children with an education that is available, accessible, acceptable, and
adaptable (Tomaševski, 2001). Despite focused international attention on underserved and
OOSC and an exponential increase in alternative education programs designed to address their
needs, significant questions remain about the goals of these programs, quality of education
provided, and measurement of program outcomes (Ganimian & Murnane, 2016; Shephard,
2014). In his systematic review of 9,721 studies related to the efficacy of NFE programs
worldwide, Shephard (2014) did not identify a single evidence-based study that demonstrated
that non-formal education programs delivered on targeted educational outcomes. This problem is
important to address because under the United Nations Convention on the Rights of the Child, all
children have the right to a basic education (United Nations, 1989).
Organizational Context and Mission
Central Visayas Children’s Center (CVCC)
1
is a nonprofit organization providing
holistic, residential support services for up to 132 abandoned, abused, and neglected children in
an urban area in the central Philippines. CVCC’s overarching goal is to provide for the safety,
security, well-being, and development of disadvantaged Filipino children through the provision
of residential care services, education, and community outreach. A central component of
CVCC’s mission is formal school reintegration and the provision of supplemental NFE services
to its residential children.
1
All references to organizations, programs, and individuals in this research study are pseudonyms.
2
CVCC’s “Learning and Education Assistance Program” (LEAP) was formally launched
in 2019 to provide supplemental NFE to CVCC students with the goal of improving academic
achievement in formal schools. Broadly defined, non-formal education entails any systematic
and organized educational activity pursued outside of the context of formal schooling (Coombs
& Ahmed, 1973). While non-formal education has been historically characterized as an
alternative to formal education, the UNESCO Institute for Statistics (2012) broadened NFE’s
definition to include supplemental programs that improve educational access, equity, and quality
for marginalized populations. CVCC’s approach focuses on providing NFE support outside of
the context of schooling to support academic outcomes in formal schools, as opposed to
supplanting formal education with a standalone NFE approach. To achieve this goal, LEAP
emphasizes the identification of individual learning achievement gaps, creation of individual
development plans, and utilization of CVCC non-formal educational supports to increase student
self-efficacy and academic achievement in formal schools.
In order to attain these goals, CVCC’s LEAP team maintains close coordination with the
local public and private schools at which CVCC children attend, including Juan Luna Charter
School (JLCS), Bonifacio Science High School (BSHS), Roxas National Vocational School
(RNVS), Leyte National High School (LNHS), and Lorenzo Ruiz Elementary School (LRES).
In order to promote reintegration and improved academic achievement, CVCC employs a full-
time education coordinator, part-time English-as-a-Second Language (ESL) instructor, part-time
evening tutor, 5 full-time social workers, and 8 full-time house parents as members of its LEAP
education team. The level of individual staff engagement in LEAP educational activities varies
by the team member’s role. While the Education Coordinator is fully devoted to the program,
CVCC’s house parents provide informal tutoring and homework support on as-needed basis.
3
CVCC’s residential program participants range in age from 5- to 18-years old and include
children previously categorized as abandoned, abused, or neglected.
Organizational Goal
The goal of CVCC’s LEAP team is to improve the academic achievement of 100% of
CVCC’s residential children through non-formal education by 5% as measured by academic
performance in formal schools by June 2021. CVCC’s Executive Director and LEAP education
team established this goal as an initial target for LEAP’s inaugural year.
Related Literature
Numerous studies have indicated that governmental and non-governmental organizations’
efforts to provide free and compulsory primary education to marginalized children have not
succeeded in many developing nations. According to the United Nations Educational, Scientific,
and Cultural Organization (UNESCO; 2015), despite the rapid proliferation of NFE programs
designed to meet the needs of marginalized children, high-quality primary education is
inaccessible to many such children in developing nations. In their EFA (Education for All)
Global Monitoring Report (2015), UNESCO reported that universal primary education, a key
2015 program goal of the EFA initiative, was not achieved despite funding, political support, and
monitoring. While enrollment rates in primary education are increasing at the global and regional
level, there remains a disparity in educational opportunities in developing nations (UNESCO-
Bangkok, 2015). According to UNESCO-Bangkok (2015), regional progress has been significant
in Asia and the Pacific, but 17 million children remain out of school in the region and the
progress made by many developing nations still presents challenges related to completion rates
and the overall quality of education.
Current statistics illustrate the extent of the challenge in the Philippines. According to the
United Nations' Children's Rights & Emergency Relief Organization, there are an estimated 1.8
4
million abandoned and neglected children in the Philippines (Kaiman & De Leon, 2017) whose
primary avenue for an education resides in NFE opportunities. Despite universal, free K-12
education, Filipino children from low-income households do not have equal access to
educational opportunities (Philippine Statistics Authority, 2017; Tan, 2017). At the national
level, the Philippine Statistics Authority ([PSA]; 2017) reported that children from the lowest-
income households are almost seven times more likely to be out of school than those from the
wealthiest households. The agency also reported that female students from low-income
households were twice as likely to be out of school as their male peers. Tan (2017) noted that
only 18% of students from the lowest income decile graduated from high school in 2011, while
over 77% of students from the highest income decile completed high school.
The factors underlying these disparities are complex, but intimately connected to
children’s experiences when faced with conflicting priorities from what Stanton-Salazar (2011)
refers to as simultaneously existing social worlds. Stanton-Salazar (2011) and Boykin (1986)
identified the conflicts experienced by children operating in sociocultural environments designed
to benefit dominant cultural groups, while disadvantaging marginalized populations and creating
systemic inequities in education. As a result of these systemic inequities, marginalized children
have limited access to essential resources readily available to children from dominant groups,
which disadvantages marginalized children in terms of the accessibility, equity, and quality of
formal educational opportunities available to them.
Non-formal education programs are a response to these inequities and, by definition, are
typically viewed as alternatives to formal schooling (Yasunaga, 2014). While no universally
accepted definition of formal education currently exists, NFE is often distinguished based on it
use of learning strategies that address the barriers marginalized children encounter in accessing
formal educational opportunities (Morpheth & Creed, 2012; Thompson, 2001). However, the
5
dichotomous distinctions between formal and non-formal education have diminished over time,
with NFE increasingly viewed as complement to formal education, as opposed to an alternative
(Yasuganga, 2014).
In response to the high number of out-of-school children and youth in the Republic of the
Philippines, the Philippine Department of Education (DepEd) developed the Alternative
Learning System (ALS) which affords Filipino learners of all ages the opportunity to obtain a
high school equivalency degree (World Bank, 2018). However, ALS credentials face significant
challenges due to their lack of transferability to and acceptance by formal academic and
employment settings (UNESCO Institute for Statistics, 2012). Funding for the ALS program
represents less than 1% of the Philippines’ public education budget and enrollments represent
only 12% of those Filipinos qualified to participate (World Bank, 2018).
A critical challenge facing NFE programs both in the Philippines and other contexts is the
paucity of research demonstrating their efficacy in delivering on targeted educational outcomes
(Shephard, 2014). As a result, while many NFE programs are accessible, affordable, and
adaptable, little data exists to support their degree of acceptability. Non-formal education
programs struggle to demonstrate results, leading to questions about their credibility and
efficacy. A key priority in implementing NFE programs is the development and measurement of
evidence-based interventions that ensure that targeted educational outcomes are achieved.
Importance of Evaluation
The Central Visayas Children’s Center (CVCC) launched LEAP in 2019 to improve
student self-efficacy and academic achievement in formal schools. As opposed to supplanting
formal education as a means of providing basic, quality education to marginalized children,
LEAP focuses on supplementing formal education with non-formal approaches that address the
distinct needs of their residential children. This blended approach stems from CVCC’s earlier
6
experiences with street-based education, which the organization believed addressed issues of
availability, accessibility, and adaptability, but failed to meet standards of acceptability.
According to its founder, CVCC’s street education program struggled to demonstrate acceptable
academic outcomes, resulting in the program’s closure in 2016 (personal communication,
February 18, 2019). As a result, CVCC redirected its resources towards meeting the needs of its
residential children by ensuring that they either continue to attend or are reintegrated into formal
schools at a grade level appropriate to their development. In order to promote the children’s
academic success, CVCC is focusing on removing non-academic barriers to academic
achievement, promoting the children’s sense of self-efficacy, and providing supplemental NFE
to improve academic achievement. CVCC wants to evaluate the LEAP’s contribution towards
achieving these goals. At a higher level, the evaluation also responds to questions related to
NFE’s efficacy and its appropriate role in providing marginalized children with education that is
available, accessible, accessible, and adaptable.
Description of the Stakeholder Groups
CVCC’s stakeholder groups include its residential children, education team, and
administrators. As the primary beneficiaries of CVCC’s education initiative, the center’s
residential children play an integral role in learning outcomes. Rueda (2011) noted that a core
goal of education is developing self-regulating learners, as opposed to viewing students as
passive recipients of education. CVCC’s LEAP team is also a key stakeholder group due to their
instrumental role in transforming global organizational objectives into ongoing educational and
instructional goals that directly influence student self-efficacy and achievement. LEAP team
members include CVCC’s Education Coordinator, English-as-a-Second Language Instructor,
evening tutor, social workers, and house parents, all of whom fulfill varying roles in addressing
the children’s academic development. The LEAP team’s accountability is maintained through
7
extrinsic performance incentives tied to student success. In a similar fashion, administrators are
evaluated on student achievement in a tangible way as organizational funding is directly tied to
their leadership skills and resultant organizational performance.
Stakeholder Groups’ Performance Goals
Table 1
Organizational Mission, Global Goal, and Stakeholder Goals
Organizational Mission
CVCC’s mission is to provide for the safety, security, well-being, and development of disadvantaged
Filipino children through the provision of residential care services, education, and community
outreach.
Organizational Performance Goal
The goal of CVCC’s LEAP team is to improve the academic achievement of 100% of CVCC’s
residential children through non-formal education by 5% as measured by academic performance in
formal schools by June 2021.
Stakeholder Goals
Administrators
By October 2020, the
Program Director
/administrators will finalize
the LEAP program’s
objectives and allocate
program resources.
LEAP Team
By December 2020, the
Education Coordinator and
team members will develop
individual learning plans for
all students enrolled in LEAP,
identifying specific
improvement areas and goals
for academic performance.
Students
By June 2021, 100% of LEAP
participants will demonstrate
increased levels of self-efficacy and
improved academic achievement in
formal schools resulting from the
provision of supplemental, non-
formal education. as measured by
self-assessment surveys and
academic performance.
Stakeholder Group for the Study
While the roles played by CVCC’s founder, administrators, social workers, educators,
house parents, and students are essential to the realization of student achievement, CVCC’s
8
LEAP team members play an instrumental role in transforming the organization’s global
objectives into daily NFE programs and instructional approaches, such as assessing student
performance, designing NFE curriculum, and preparing lessons. LEAP team members represent
a cross-section of administrators, social workers, educators, and house parents, each of whom
fulfill specific roles within the LEAP team. Therefore, the stakeholders of focus for this study
will be CVCC’s LEAP team members.
Purpose of the Project and Questions
The purpose of this project is to evaluate the degree to which LEAP team members are
supporting the improvement of CVCC’s residential children’s academic performance in formal
schools through the implementation of NFE programs. The analysis will focus on knowledge,
motivation and organizational influences related to achieving this organizational goal. While a
complete evaluation project would focus on all CVCC stakeholders, for practical purposes the
stakeholders to be focused on in this analysis are members of the LEAP education team. As such,
the questions that guide this study are the following:
1. What are LEAP education team members’ knowledge and motivation related to
achieving the organizational goal of ensuring that all CVCC students demonstrate
improved academic achievement in formal schools through the provision of non-formal
education?
2. What is the interaction between organizational culture and context and LEAP team
educator knowledge and motivation related to the provision of non-formal education?
Methodological Framework
Clark and Estes (2008) gap analytic framework was used to assess the gap between the
LEAP team’s targeted and actual performance in delivering NFE programs that improved CVCC
children’s academic achievement in formal schools. The study employed an inductive,
9
qualitative approach to data collection and analysis. CVCC’s LEAP team members’ current
knowledge, skills, and motivation in relation to improving children’s self-efficacy and academic
achievement were assessed using literature review, document analysis, interviews, and a focus
group. Research-based solutions were recommended and evaluated in a comprehensive manner.
Definitions
Basic Education is defined as “… the education intended to meet basic learning needs which lays
the foundation on which subsequent learning can be based. It encompasses early childhood,
elementary and high school education as well as alternative learning systems for children, out-of-
school youth (OSY) and adult learners and for those with special needs” (Guerrero, 2007, p. 2).
Non-formal Education (NFE) is defined as “… any organized, structured and systematic learning
service delivered outside the framework of the formal school system to a specific segment, group
or sub-group of the population for a specific objective, at low cost in terms of both time and
resources” (Thompson, 2001, p. 8).
Out-of-school children (OOSC) are defined as children in the official primary school age range
who are not enrolled in pre-primary, primary, or secondary schools (UNESCO Institute for
Statistics, n.d.).
Pantawid Pamilyang Pilipino Program (4P) is a conditional cash transfer (CCT) program
administered by the Republic of the Philippines’ Department of Social Welfare and Development
(DSWD) that incentivizes impoverished families to participate in health, educational, and
nutritional programs benefitting children from birth to 18-years of age.
Organization of the Project
This project is presented in five chapters. This chapter introduced the problem of practice
within the broader context of the universal right of a child to an education and challenges in
achieving that goal. It provided the reader a description of CVCC’s mission, goal, and
10
stakeholder groups, as well as introducing its LEAP program and framework for addressing the
educational challenges faced by marginalized children. Chapter Two provides a review of the
literature related to the provision of available, accessible, acceptable, and adaptable education to
marginalized children, including a description of NFE’s evolution and role; an overview of NFE
models; a survey of local implementation strategies in the Philippines; and a discussion of
ongoing challenges to educational access, quality, and equity. Chapter Three provides details of
the knowledge, motivation, and organizational aspects being studied, as well as a description of
the qualitative methods used to evaluate CVCC’s performance. In Chapter Four, the results of
the qualitative research are synthesized, analyzed, assessed, and presented. Chapter Five
proposes solutions based on an analysis of the data collected and extant literature, as well as
recommendations for addressing the gaps between targeted and actual program outcomes.
Specific recommendations are provided for implementing and evaluating the proposed solution.
11
Chapter Two: Review of the Literature
This literature review will examine the factors influencing the implementation of non-
formal education (NFE) initiatives for marginalized and OOSC in the Philippines. The review
begins with general research on the literature related to the importance of basic education in the
development of a child’s human potential. The review will next describe the literature related to
NFE responses to marginalized and OOSC, emphasizing issues of educational access, quality,
and equity in the Philippines. The review will then present an in-depth description and
assessment of major government and non-profit NFE initiatives in the Philippines based on the
literature. This section will focus on the approaches used and challenges encountered by NFE
initiatives designed to promote educational availability, accessibility, acceptability, and
adaptability. Finally, the review will apply Clark and Estes’ (2008) Gap Analytic Conceptual
Framework to describe the impact of knowledge, motivation, and organizational influences on
the CVCC’s LEAP program to improve the self-efficacy and academic achievement of
disadvantaged Filipino children.
Education as a Means of Realizing Human Potential
Basic education is foundational to the development of a child’s full human potential. The
importance of education’s role in developing human potential is emphasized in the development
of global and national agendas guaranteeing children’s right to universal basic education
(National Economic and Development Authority, 2017; Read & Atinc, 2018; UNESCO, 2007;
United Nations, 1989; World Bank, 2018). Numerous research studies have demonstrated that
access to basic education plays an integral role in children’s long-term development and success
(Heckman et al., 2018; Kayani et al. 2017; UNESCO, 2018). In their research on the returns on
education in environments of imperfect information and diverse student abilities, Heckman et al.
(2018) demonstrated a positive causal relationship between schooling and market and non-
12
market outcomes. A key finding of the study indicated that, while higher education does not
always drive ex post marginal returns, a high school education consistently demonstrated
positive return for students of all ability levels (Heckman et al., 2018). In their research on the
socioeconomic impact of education in Pakistan, Kayani et al. (2017), noted that education plays
not only a role in the long-term socioeconomic advancement of children, but also improves
social cohesion and equity within a nation. UNESCO (2018) underscored the important role that
education plays in breaking the cycle of poverty and reintegrating excluded children into society.
The organization estimated that the number of poor people globally could be reduced by more
than half if all adults completed their secondary education (UNESCO, 2018). Despite evidence
of the importance of education in developing children’s long-term potential, lack of access to a
quality education based on socioeconomic inequalities is pervasive in developing nations in Asia
(Son, 2013).
Despite heightened awareness, increased funding, and the proliferation of NFE programs
targeted at ensuring the provision of basic education to out-of-school children and youth
(OSCY), evidence of NFE program efficacy is limited (Rose, 2009; Shephard, 2015; Steer et al.,
2015; UNESCO, 2015; UNESCO-Bangkok, 2015). Globally, UNESCO (2015) reported
challenges with limited data on the short- or long-term efficacy of NFE initiatives. Of the
research data available on NFE efficacy, the organization indicated the data’s limited utility for
planning and policy purposes due to the fact sponsoring agencies frequently conducted the
program evaluations and assessments (UNESCO, 2015). In their 2015 Asia-Pacific Regional
Education for All Report, UNESCO-Bangkok (2015) indicated that, while the number of NFE
programs significantly increased in the Asia-Pacific region, it was difficult to measure their
impact due to insufficient data collection and reporting mechanisms. Steer et al. (2015)
highlighted the complexity of comparing the efficacy of formal versus non-formal approaches to
13
addressing the needs of marginalized children due to the lack of rigorous quantitative studies.
Research on Philippine government and non-profit solutions to OSCY reveal inconsistent
assessments of NFE program efficacy, including challenges related to access, quality, and equity
(Philippines Statistics Authority, 2017; Rose, 2009). According to Rose (2009), an ongoing
concern remains that, while NFE initiatives are providing educational access to a greater number
of marginalized children, the poorest and most vulnerable children continue to be excluded. The
Philippine Statistics Authority (PSA) reported that about 9% of individuals aged 6 through 24
were out of school in 2017, with more than half of OSCY falling within the bottom 30% of per
capita income (2017). According to Son (2013), per capita household expenditure as
representative of the household’s overall standard of living is the most important influence in
determining a Filipino child’s fair access to education. These findings indicate the demand for
additional research related to the efficacy non-formal education, as well as its appropriate role, in
providing quality educational opportunities to marginalized children.
Non-formal Education
Non-formal education as a means of addressing the educational needs of marginalized
children arose out of growing perceptions of education as a fundamental human right and the
inability of formal schools to provide universal basic education. The following section
synthesizes the extant literature on non-formal education, focusing on the educational rights of
children; the inability of formal education to fully meet international pressures towards education
for all; the evolution of NFE as a response; and the inherent challenges involved in defining the
most appropriate NFE roles and models to provide marginalized children with an education that
meets the standards of accessibility, quality, equity, and affordability.
A key finding that emerges from a review of the literature is that, while education is
essential to realizing human potential, little information is being collected about the short- or
14
long-term efficacy of NFE programs designed to provide a basic, quality education. In its role as
a supplemental NFE program, CVCC’s LEAP initiative is designed to respond to the educational
needs and enhance academic achievement of previously abandoned, abused, and neglected
children attending formal schools. As an integral component of this initiative, CVCC is
interested in evaluating the efficacy of the LEAP program in achieving the goals of enhanced
self-efficacy and improved academic achievement of students from marginalized or vulnerable
backgrounds.
Basic Education as a Human Right
The principle of basic education as a human right emerged after World War II, but the
challenges posed by 21
st
century globalization, urbanization, migration, and socioeconomic
inequality have generated renewed international interest in delivering on the mandate of
universal primary education. Basic education was formally established as a human right by the
Universal Declaration of Human Rights (1948) and later reaffirmed in legal instruments such as
the International Covenant on Economic, Social, and Cultural Rights (1966) and the UN
Convention on the Rights of the Child (1989) (McCowan, 2011). “Education for All” (EFA)
operationalized the international mandate for a human-rights based approach to universal
education. EFA, a rights-based education agenda developed at world conferences held in Jomtien
(1990) and Dakar (2000), established the goal of universal primary schooling by 2015
(McCowan, 2011). During this timeframe, international focus on education shifted from a
framework that emphasized the development of human capital and economic return-on-
investment to a framework that emphasized the full social and cultural development of the
individual as a legal and moral right (Robeyns, 2006). According to Robeyns (2006), the human
rights framework, while accepting the importance of the human capital framework, disassociated
free, universal primary education from individual economic outcomes. Despite the international
15
commitment to universal basic education, political, economic, and socio-cultural factors have
influenced the ability of many nations to comply with the EFA’s guidelines on achieving
universal basic education as defined by Goal No. 6 of the Dakar Framework for Action
(Robeyns, 2006) or Millennium Development Goal No. 2 (UNESCO, 2015).
The shift in perspective from basic education as a means of developing human capital to
a means of fulfilling human rights has transformed the global and national educational and
developmental landscape and challenged national governments’ capacity to achieve the
aspirational goals of universal basic education. The U.N. estimated that 262 million children and
adolescents remained out of school in 2017 (United Nations, 2017). According to the UNESCO
Institute of Statistics, an estimated 61 million primary school children and 71 million lower
secondary school children remained out of school in 2010 (Chinapah et al., 2013). Although
access to education improved globally since the establishment of the EFA agenda in 2000,
serious challenges remain in terms of the quality of education provided (Chinapah et al, 2013).
Nations with limited resources encounter difficulty in achieving universal, basic education,
particularly when faced with other socioeconomic restraints that influence educational access
and participation (Charvon & Chase, 2016). Chinapah et al. (2013) reported that factors such as
economic crises, globalization, military conflict, socioeconomic inequality, and urbanization
exacerbate existing challenges in the provision of universal basic education. According to the
Morpeth and Creed (2012), poverty, natural disasters, displacement, mobility, geography,
language, and marginalization serve as additional barriers to education, transcending traditional
interpretations of school accessibility. Morpeth and Creed (2012) also associated non-enrolment,
low attendance, high-drop-out rates, and low achievement to challenges with educational access,
quality, and relevance. In order to overcome these barriers to access and achieve EFA goals,
16
many nations have turned to NFE as a means of addressing challenges related to providing
universal primary education.
Definition of Non-formal Education
Non-formal education is largely defined in the ways in which it contrasts with formal
education and responds to the specific educational needs of marginalized populations in the
community. However, no universally accepted definition of formal education currently exists
(Yasunaga, 2014). In their seminal work on non-formal education as a means of reducing
poverty, Coombs and Ahmed (1973) defined non-formal education as “… any organized,
systematic educational activity carried on outside the framework of the formal system to provide
selected types of learning to particular subgroups of the population, adults as well as children”
(p. 8). Definitions of NFE emphasize learner-oriented approaches that differentiate NFE
delivery, facilitation, approaches, and techniques from formal education (Thompson, 2001).
According to Yasunaga (2014), while non-formal education is largely defined in terms of its
contrast with formal education, evolving definitions de-emphasize the distinction in recognition
of the formal and informal nature of learning in all settings and contexts.
In contrast to definitions that solely emphasizes the distinctions between formal and NFE,
the UNESCO Institute for Statistics (UIS; 2012) defines NFE as a structured alternative or
supplement to formal education, emphasizing its frequent use in guaranteeing educational access
to marginalized populations. Thompson (2001) characterized NFE by its learner-orientation,
contextual relevance, flexible timing, low cost, and participatory management. UIS (2012)
echoed several of these characterizations, acknowledging NFE’s flexibility in its timing,
intensity, and curriculum. However, according to UIS (2012), the distinctions made between
formal and NFE contribute to challenges related to NFE’s acceptance by formal educational
systems.
17
NFE qualifications are not typically recognized by national authorities, resulting in
challenges in terms of the transferability of NFE credentials to formal educational frameworks
(UIS, 2012). This lack of integration between formal and NFE frameworks also raises concerns
about the overall quality of NFE initiatives (Thompson, 2001). The complexities of defining
NFE reflect its evolution as a response to formal education’s inability to address the challenges
of marginalized populations, as well as NFE’s adaptive nature in meeting evolving societal
challenges to educational provision.
Evolution of Non-formal Education
Non-formal education (NFE) evolved as a response to the inability of formal education
systems to adapt to the needs of populations facing social, cultural, economic, and logistical
constraints to educational access. Coombs and Ahmed (1973) attributed the emergence of NFE
as a response to the educational needs of marginalized populations, particularly in meeting the
needs of children who had not attended or completed primary or secondary school. Yasunaga
(2014) also traced the origins of NFE to the 1960s as a response to poorly performing formal
education models. According to Yasunaga (2014), interest in NFE temporarily waned in the
1980s as a result of public policy shifts emphasizing reform in formal education, but re-emerged
in parallel with the emergence of a human-rights framework on education. NFE initiatives have
evolved as response to the inadequacy of formal education to meet the needs of marginalized
populations and, as a result, NFE must address the barriers and gaps inherent in the formal
education system to be successful.
Role of Non-formal Education
The primary role of non-formal education is to provide quality, basic education to the
educationally marginalized. Yasunaga (2014) indicated that NFE is recognized as a legitimate
approach to addressing the educational needs of marginalized children in international normative
18
frameworks and demonstrates the potential to reach underserved children facing educational
marginalization. Additionally, NFE approaches address many of the key barriers to access,
availability, acceptability, and adaptability encountered by marginalized children (Yasunaga,
2014). Based on the barriers to access inherent in formal schools, Brennan (1997) emphasized
the importance of not replicating the shortcomings of formal education in NFE responses to the
educational needs of marginalized populations. According to Brennan (1997), NFE may be
categorized according to its role as a complement, alternative, or supplement to the formal
education system. Defining which of these roles is most appropriate is intimately tied to
understanding the limitations of formal education in meeting the specific needs of marginalized
populations within a given context (Brennan, 1997). However, the diverse roles that NFE
initiatives are designed to fulfill frequently lack clarity. In a systematic review of 33 studies on
children and adolescents in street situations (CASS) interventions, Berckman et al.’s (2012)
experienced difficulty identifying the program goals of NFE interventions and a corresponding
high degree of complexity in measuring program efficacy. Furthermore, it is difficult to isolate
NFE’s role as a distinct entity isolated from broader systemic influences. As one example of the
importance of a system’s perspective in defining NFE’s role, Morpeth and Creed (2012)
emphasized the importance of NFE’s integration into a diversified education system to provide
structured avenues for educationally marginalized children to reintegrate into formal school
settings and provide legitimacy to NFE certifications (Morpeth & Creed, 2012). Non-formal
education evolved in response to barriers to availability, access, acceptability, and adaptability,
resulting in a range of approaches designed to address the diverse set of barriers to quality, basic
education. However, the models that developed in response to these needs are diverse and define
education at levels ranging from the attainment of basic self-sufficiency to serving as substitutes
for formal education.
19
Non-formal Educational Models
Despite the diverse nature of barriers facing educationally marginalized populations and
resultant diversity in NFE responses, NFE approaches may be categorized according to
objective-based strategies reflecting the model’s purpose or intent. The U.N. Convention on the
Rights of the Child defined “protection” and “participation” as the overarching principles for
interventions related to children in street situations (CISS), a distinct category of marginalized
children (Berckmans, 2012). These principles, according to Berckmans (2012), are realized
through the provision of education, development, and self-sufficiency. Yasunaga (2014)
identified four basic NFE models, including 1) remedial and supplemental NFE, 2) vocational
and skills development, 3) experimental and innovative non-formal design, and 4) other NFE
types (e.g., indigenous and traditional education). The majority of NFE initiatives serving
marginalized children focus on remedial and supplemental NFE and vocational and skills
development (Yasunaga, 2014). UNESCO-Bangkok (2015) identified four types of NFE most
common to the Asia-Pacific region, including literacy and post-literacy; equivalency; vocational
training; and life skills and livelihood development programs. An analysis of each of these
categorizations reveals elements of both a human-rights based and human capital development
perspective towards education. Despite the evolution of diverse models for addressing
educational marginalization, NFE initiatives have not successfully demonstrated their efficacy in
delivering on the goal of quality, universal primary education in developing nations.
NFE Access, Quality, Equity, and Cost-Effectiveness
Evaluating the efficacy of both formal and NFE requires consideration of several core
challenges related to educational provision to marginalized children. Steer et al., (2015)
identified four key principles used to assess the provision of universal basic education, including
access and scale, quality, equity, and cost-effectiveness. The principles of access, equity, and
20
quality are also reflected in Tomaševski’s (2001) “4-A scheme” which defines the role of
government in the provision of a human-rights based education as ensuring that education is
available, accessible, acceptable, and adaptable. UNESCO estimated that universal primary
education will cost developing nations an average of $10.6 billion between 2015 and 2030,
underscoring the importance of ensuring that educational investments demonstrate returns-on-
investment (Steer et al., 2015). While variations of the “access-quality-equity-cost” assessment
standard exist, a review of the literature underscores the importance of this framework in
international and national approaches to evaluating formal and non-formal education and
assessing the strengths and weaknesses of each.
Assessments of NFE’s ability to meet the standards of access, quality, and equity vary
significantly. According to the Steer et al. (2015), questions remain about NFE’s role in the
provision of universal basic education, its consistency with a human-rights based approach to
education, and empirical concerns about its quality and equity. Steer et al. (2015) indicated that
issues of quality are a key concern for critics of NFE approaches. Rose (2009) noted that NFE
initiatives are generally associated with high-quality, cost-effective educational provision, yet
simultaneously viewed as lesser alternatives to formal education. Debates related to the cost-
effectiveness and quality of NFE vis-à-vis formal education are compounded by definitional
issues related to what qualifies as NFE and a resultant challenge to measuring participation, cost,
and efficacy (Steer et al., 2015). A key challenge in measuring NFE efficacy is clarifying what
qualifies as NFE, the impetus for its development, and its appropriate role in providing a basic,
quality education to marginalized children.
Non-formal Education in the Philippines
The Philippine government’s response to educational provision to marginalized children
stemmed from significant challenges related to the accessibility, quality, equity, and affordability
21
of formal education. Based on the literature, the formal response to these challenges emerged as
three core strategies, including DepEd’s “Alternative Learning System” (ALS), the Department
of Social Welfare and Development’s (DSWD) Pantawid Pamilyang Pilipino Program (4P), and
the accreditation of NFE initiatives. The following narrative describes the scope of the challenge
facing marginalized children in the Philippines, the root causes of those challenges, and the
programmatic Philippine government’s responses to incentivizing marginalized children’s
participation in both formal schools and NFE initiatives.
Educational Challenges in the Philippines
While the Philippines has made progress in improving the availability of and access to
universal primary education, challenges remain in assuring the acceptability and adaptability of
educational opportunities for marginalized populations. In their research on the applications of
data to improve learning, Read and Atinc (2017) described the long-term educational reforms
undertaken by the Philippines since 2000. Philippine reforms have focused on school-based
management initiatives and significant increases in educational funding (Read & Atinc, 2017).
As a result of efforts to improve access, equity, and quality in education, the Philippines has
reduced the number of OOSC, decreased enrollment gaps between the rich and the poor,
improved its infrastructure, and eased teacher shortages (Read & Atinc, 2017). Despite these
improvements, the Philippines still experiences low completion rates, weak student performance,
low teacher quality, and high dropout rates (Read & Atinc, 2017). The Philippines’ commitment
to universal primary education has led to the development of policies that purport to promote
access, equity, and quality, but that fail to adequately address the challenges of educationally
marginalized populations (Baum, 2012). Lack of improvement is tied to overemphasis on inputs
as opposed to addressing root causes of underperformance, as well as failure to fully decentralize
and implement school-based management initiatives (Read & Atinc, 2017). According to Baum
22
(2012), the Philippines’ ongoing challenge in delivering on a human-rights based educational
agenda relate to the government’s failure to acknowledge the nation’s economic, infrastructure,
systemic, and political realities. The Philippine Institute for Development (2009a) attributes the
poor performance of the formal education system to an overemphasis on the construction of new
schools, hiring of new teachers, and acquisition of textbooks, as opposed to focusing on essential
changes in governance. Tan (2017) stressed that the expenditure of limited public resources on
expansive growth in education sector negatively impacts marginalized populations by diluting
resources and creating unintended access barriers. While education reform, increased investment,
and infrastructure development have increased the availability of formal education programs and
reduced access barriers to primary education in the Philippines, the nation has yet to recognize its
goal of universal access, equity, and quality in the provision of education.
Out-of-School-Children-and-Youth in the Philippines
Philippine formal education underserves children from marginalized families, specifically
those living in poverty. According to the Philippine Statistics Authority (2017), 9% of Filipino
children aged 6 to 24 do not attend school. The results of the PSA’s 2017 Annual Poverty
Indicators Survey indicated that students from low-income households, particularly females,
represent a disproportionate share of OOSC. The primary reasons identified by children for not
attending primary and secondary school included lack of personal interest, illness or disability,
caring for a family member, and financial matters (PSA, 2017). According to the Philippine
Institute for Developments Studies ([PIDS], 2009b), elementary enrollment rates have increased,
but drop-out rates remain high, particularly among students from the poorest families.
Only 18% of Filipino students from the lowest income decile graduate from high school (Tan,
2017). The barriers to access, equity, and quality inherent in the Philippines’ formal education
23
system has created demand for non-formal education initiatives designed to bridge the gap
between the nation’s aspirational educational goals and educational realities.
Current Non-formal Education Initiatives in the Philippines
Both state and non-state actors have developed non-formal education programs designed
to provide basic education to marginalized populations, but the programs lack funding, cohesion,
and alignment with formal education systems. The Philippine Department of Education (DepEd)
operates a non-formal education program known as the “Alternative Learning System (ALS)”
(Read & Atinc, 2017). The ALS focuses on human capital development and economic self-
sufficiency (World Bank, 2018). However, the ALS struggles with transferability and funding
issues. Completion of the ALS is not generally accepted by employers as a high school
equivalent and lacks transferability to both academic and employment contexts (World Bank,
2018). Despite the expansive population it is targeted to serve, the ALS program also receives
less than 1% of the public education budget (World Bank, 2018). According to the World Bank
(2018), enrollment in ALS increased to over a half a million participants in 2017, but 3.7 million
youth between the ages of 15 and 24 still do not attend formal or non-formal educational
programs. Non-profit organizations also play a significant role in NFE designed to address major
gaps in state-sponsored services to the educationally marginalized (Bano, 2008). However, while
DepEd accredits non-profit education programs, it does not actively monitor their performance
or efficacy (World Bank, 2018).
The Philippine Department of Social Welfare and Development (DSWD) oversees the
Pantawid Pamilyang Pilipino Program, (4P), a conditional cash transfer (CCT) program used to
incentivize families to send their children to formal schools, but oversight of the program is
insufficient (Read & Atinc, 2017). Catubig and Villano (2017) indicated that the variation in
estimates related to 4P’s efficacy at the national and provincial levels stem from factors common
24
to all social programs, such as inconsistent implementation, supply-side differences and socio-
environmental factors. However, analysis of existing CCT programs demonstrated that CCT
programs require sophisticated oversight and are not considered cost-effective in terms of
achieving learning outcomes or improving educational attainment (EFA, 2015). Although the
Philippines provides alternative pathways to a basic education, the quality of those alternatives
are unproven while a substantial number of OOSC in the Philippines continue to face academic
and non-academic barriers to basic education.
Challenges in Assessing Efficacy of Philippine Non-formal Education Initiatives
Assessing NFE efficacy requires an understanding of the spectrum of academic and non-
academic challenges facing both NFE providers and the children they serve. Based on the extant
literature, as discussed below, marginalized children are both a product of formal education’s
approach to educational provision, as well as socioeconomic factors such as poverty that lie
outside of the scope of formal education’s control.
Academic Challenges
A core challenge to attaining EFA lies in addressing the academic barriers that prevent
marginalized children from accessing a basic, quality education in formal schools. Morpeth and
Creed (2012) stated that the very design of formal school models may serve as a barrier to access
for marginalized children. Similarly, Grover (2007) identified the disparity which exists between
a marginalized child’s reality and the formal education system’s expectations as a direct
contributor to the large number of OOSC. Amongst the academic barriers hindering the
achievement of EFA goals by formal schools in the Philippines are resource constraints,
insufficient governance, shortages of qualified staff, geographical and cultural barriers, natural
disasters, conflicts, low internal efficiency and underdeveloped monitoring and evaluation
25
systems (UNESCO, 2015). However, infrastructural and resource challenges are only one aspect
of the challenge.
Quality is also an issue. A 2014 assessment of primary and secondary teachers’
knowledge of subject matter content revealed that the average teacher was unable to correctly
answer over 50% of assessment questions in their subject areas, raising concerns about their
mastery of the curriculum (PIDS, 2009a). According to the institute, the lack of high-quality
teachers is an underlying cause of low student retention (PIDS, 2009a). Only 58 out of 100
children who enroll in the first grade in the Philippines will go on to high school, with almost
25% of Filipino children leaving school by Grade 4 (PIDS, 2009a). The most commonly cited
reason listed for children leaving primary and secondary school is a “lack of personal interest”
(PSA, 2017), raising questions regarding the correlation between teachers’ capabilities and
students’ motivation. Measuring the efficacy of both formal and NFE programs requires an
understanding of both the academic and non-academic factors that serve as barriers to
educational access, equity, and quality.
Non-Academic Challenges
Measuring the efficacy of formal and NFE programs is complex due to confounding
variables such as child agency, the relevance of non-formal education to a child’s context, and
alignment with the child’s self-interest. Researchers have identified an array of probable causes
underlying the relatively poor K-12 enrollment, retention, and completion rates of low-income
Filipino children in comparison to children from higher income families. According to data from
the PSA’s 2017 Annual Poverty Indicator Survey, the primary reasons provided by children age
6 to 11 for being out of school include lack of personal interest (30.1%), illness or disability
(28.8%), and financial matters (11.2%). For students aged 12 to 15, lack of personal interest
(65.5%), financial matters (15.8%), and illness or disability (11.8%) are the primary reasons
26
listed for not attending school (PSA, 2017). Son (2013), based on an analysis using the World
Bank’s Human Opportunity Index (HOI), attributed the disparity between primary and secondary
school completion rates to the higher opportunity costs of sending children to secondary school.
Rufino (2015) reinforced this concept, noting that almost 16% of 16-year olds and 23.06% of 17-
year olds were exclusively working in 2011. While analysis points to a variety of factors that
may explain the poor K-12 completion rates of low-income Filipino children, poverty is a
common theme in explaining this phenomenon.
Current research and statistics demonstrate a clear correlation between poverty and K-12
educational outcomes in the Philippines. Son (2013) reported that household poverty, as
measured by per capita household expenditures, plays a significant role in determining a child’s
level of access to educational opportunities. Rufino (2015) noted poverty’s negative impact on
educational outcomes, as well as the role it plays in forcing Filipino children to engage in
dehumanizing employment and in sustaining poverty. Statistics from the PSA’s 2017 Annual
Poverty Indicator Survey also illustrated the ways in which increasing levels of poverty, as
measured by annual family income, correlate with higher percentages of OSCY. However,
poverty as a predictor and justification of educational outcomes does not address the systemic
challenges experienced by low-income families in accessing educational opportunities. It is
essential to examine structural barriers preventing impoverished Filipino children from accessing
educational opportunities to understand their high level of disengagement from traditional K-12
educational programs. Addressing the challenges of non-formal education of marginalized
children in the Philippines requires an examination of the knowledge, motivation, and
organizational capacity of the organizations created to address these challenges and the gaps that
exist between their aspirations and outcomes.
The Clark and Estes (2008) Gap Analytic Conceptual Framework
27
Clark and Estes’ (2008) gap analytic conceptual framework provides a systematic
approach to identifying organizational and stakeholder performance gaps between targeted and
actual performance goals. Performance gaps are associated with knowledge, motivation, and
organizational (KMO) assumed influences on stakeholder or organizational performance (Clark
& Estes, 2008). Krathwohl’s (2002) revised taxonomy categorizes knowledge into four types,
including factual, conceptual, procedural, and metacognitive. The knowledge component of the
framework emphasizes the importance of stakeholders knowing not only “what to do,” but “how
to do it” (Clark & Estes, 2008; Eccles, 2006; Rueda, 2011). In essence, factual and conceptual
knowledge comprise understanding “what to do,” while procedural and metacognitive
knowledge comprise “how to do it.” The choices stakeholders make, their level of commitment
to those choices, and the mental effort devoted to pursuing those choices comprise the element of
motivation influences reflected in the framework (Clark & Estes, 2008; Eccles, 2006; Rueda,
2011). Organizational processes, resources, and culture may also serve as influences to increased
performance (Clark & Estes, 2008). Collectively, the KMO influences effectively encompass the
array of influences that may explain performance gaps and inform performance solutions.
CVCC’s LEAP team’s goal is to improve the academic achievement of 100% of CVCC’s
residential children by 5% as measured by academic performance in formal schools by June
2021. The following sections will analyze the needs of LEAP team members in attaining this
goal through the lens of Clark and Estes’ (2008) KMO influences. The first section will focus on
the knowledge and skills needed. The second section will address LEAP team members’
motivational influences on performance, including choice, persistence, and mental effort. The
final section will examine the impact of organizational processes, resources, and culture. The
KMO framework will also serve as the basis for assessing stakeholders’ impact on overall
organizational performance as part of the methodological discussion in Chapter Three.
28
Stakeholder Knowledge, Motivation, and Organizational Influences
This review focuses on evidence-based research on knowledge, motivation, and
organizational influences and their relationship to the performance outcomes of the CVCC’s
LEAP program. Specifically, the review emphasizes the ways in which knowledge, motivation,
and organizational influences impact the performance of LEAP team members as key
stakeholders in improving CVCC children’s academic achievement in formal schools. LEAP
team members serve an integral role in the children’s academic performance through the
provision of NFE ranging from one-on-one tutoring to the identification of academic support
services within formal schools. Additionally, their roles reside at the intersection of the LEAP
program, local schools, and the children. As a result, understanding the ways in which
knowledge, motivational, and organizational influences impact LEAP team members’
performance is essential for developing a deeper understanding of LEAP’s overall performance
Knowledge Influences
Clark and Estes (2008) indicated that knowledge is a possible cause of performance gaps
in organizations, is essential to effective work performance, and serves as a prerequisite for
enhanced human performance and achievement. An emphasis on both “what to do” and “how to
do it” is necessary to learning and the acquisition of the knowledge needed to achieve
performance goals (Clark & Estes, 2008; Eccles, 2006; Rueda, 2011). However, Rueda (2011)
illustrated the complexity inherent in defining the set of interrelated tasks that comprise a role
within an organization and identifying the knowledge and skills required to perform them. In
order to clarify the knowledge required by a CVCC LEAP team member, it is important to define
the discrete knowledge influences that impact their performance outcomes and associated
assessment approaches for identifying current knowledge gaps. LEAP team members require an
understanding of the requisite knowledge and skills necessary to enhance the academic
29
achievement of CVCC’s marginalized children. A step in understanding the knowledge required
to perform the role of a LEAP team member and identifying knowledge gaps is defining the
types of knowledge needed to achieve the stakeholder’s goal.
Krathwohl (2002) utilized Bloom’s taxonomy to segregate knowledge into distinct
categories, including factual, conceptual, procedural, and metacognitive knowledge. According
to Krathwohl (2002), factual knowledge is composed of the basic foundational knowledge
associated with a specific discipline, such as terminology or symbols. Conceptual knowledge
identifies knowledge associated with domain-specific principles, concepts, or structures
(Krathwohl, 2002). Procedural knowledge includes the methodological knowledge or processes
associated with a specific discipline. Finally, metacognitive knowledge includes an individual’s
awareness of their own cognition, cognitive processes, and strategic problem-solving behaviors
(Krathwohl, 2002).
The revised version of Bloom’s Taxonomy provides a framework for categorizing the
requisite knowledge needed by LEAP team members by knowledge type, which also aids in the
identification and design of assessments capable of measuring a LEAP team member’s and
organization’s progress towards its goals. While this review will highlight four knowledge
influences on stakeholder performance, additional applications of the taxonomy are possible.
Knowledge of the Relationship between Student Performance Data and Student Achievement
LEAP team members need the knowledge to differentiate learning outcomes. The
measurement of student achievement responds to internal and external demands to demonstrate
student achievement and organizational accountability. According to Berckmans et al. (2012),
interventions are based predominantly on theory, as opposed to empirical research on the
efficacy of interventions. LEAP team members require knowledge about how to measure
marginalized children’s current achievement levels in terms of educational status and
30
performance (Shephard, 2014). Developing a baseline for student achievement is essential to
measuring outcomes of NFE initiatives and demonstrating program efficacy. However, NFE
initiatives have historically encountered challenges collecting, analyzing, and applying data to
improve performance and measure efficacy. There exists a paucity of empirical research on the
efficacy of interventions designed to serve marginalized children (Berckmans et al., 2012). Rose
(2009) also reported that the prevailing characterization of NFE programs delivering better
educational outcomes and greater cost-effectiveness is unsubstantiated based on data indicative
of longer-term outcomes such as access to higher education, employment opportunities, or
economic self-sufficiency. This knowledge influence merits attention, particularly in light of the
importance of responding to CVCC’s internal and external accountability requirements to its
primary stakeholders.
LEAP team members also need an understanding of approaches to using benchmarking
and data-driven decision making to measure program efficacy. In a review of data-driven
decision-making initiatives in education, Dowd (2005) highlighted the potential merits of
implementing effective benchmarking strategies in educational settings. The author noted that a
key benefit of using appropriate benchmarking strategies is to assist educators in purposefully
analyzing data related to student learning and progress (Dowd, 2005). Dowd (2005) and Baker
(2006) identified three benchmarking approaches for measuring progress against targeted
outcomes, including performance, diagnostic, and process-oriented benchmarking. Performance
benchmarking, due to its relative simplicity and cost-effectiveness, responds to the demand for
data that demonstrates program efficacy (Dowd, 2005). In contrast, Dowd (2005) noted that
diagnostic benchmarking provides a measure of the organization’s performance and focuses on
program improvement. Dowd (2005) defined process benchmarking, the final approach, as a
rigorous program evaluation focused on organizational learning. Each approach varies in terms
31
of its resource requirements, simplicity of use, and the quality of data generated for decision-
making. Additionally, each process draws on varying levels of knowledge, ranging from
declarative-factual knowledge of the different approaches benchmarking to metacognitive
knowledge using those measures to drive organizational learning. Marsh (2012) cautioned that a
primary challenge in measuring performance includes sustaining the technical and cultural
supports for these initiatives within the organization. As a result, LEAP team members need to
carefully select the measures used based on their organization’s capacity to implement and
maintain them.
LEAP team members also need to develop valid, reliable, objective, and referenced
assessment instruments to measure student learning outcomes, processes, and capabilities
(Mayer, 2011). Rueda (2011) defines achievement as mastery of the content knowledge related
to a specific domain. In addition, Rueda (2011) underscored the importance of developing
assessments that measure learning outcomes from a multidimensional perspective, as opposed to
focusing on isolated measures. According to Mayer (2011), effective measures of learning
include assessment of retention and transfer, including near and far transfer. Learning outcomes
can be categorized as “no learning,” “rote learning,” and meaningful learning” (Mayer, 2011).
The author noted that these measures provide educators the tools necessary to understand how
well students have integrated the knowledge and gauge the depth of their cognitive processing.
In order to demonstrate improvements in CVCC students’ performance, LEAP team members
need to acquire the knowledge necessary to develop and implement benchmarking and
assessment strategies that effectively measure student learning outcomes.
Knowledge of How to Identify Gaps in Students’ Ability to Succeed in Formal Schools
In addition to a conceptual knowledge of the relationship between student performance
data and achievement, LEAP team members need procedural knowledge related to using student
32
performance data to identify gaps in students’ ability to succeed in formal schools. This
knowledge influence is procedural in nature as it reflects domain-specific principles, concepts, or
structures that influence LEAP team members’ ability to assess compare targeted learning
outcomes with marginalized children’s academic achievement. Dowd (2012) emphasizes the
importance of using qualitative and quantitative measures to monitor student outcomes and
adjust programmatic approaches accordingly.
In order to meet their performance goals, LEAP team members need to understand that
non-formal education is necessitated by the inability of formal school systems to fully address
the needs of marginalized children (Brennan, 1977; Coombs & Ahmed, 1973; Morpeth & Creed,
2012; UIS, 2012; Yasunaga, 2014). As a result, NFE cannot solve the challenges presented by
marginalized children using the same structures and approaches used by formal education (Bano,
2008; Brennan, 1997; Thompson, 2001). Over 60% of CVCC children are one or more years
behind grade level and average class sizes in local schools range from 35 to 65 students (personal
communication, August 18, 2020). As a result, LEAP team educators need knowledge of how to
identify academic and non-academic gaps in CVCC students’ ability to succeed in formal
schools as a precursor to designing and developing programs capable of bridging those gaps.
LEAP team members also need to understand how marginalized children respond to NFE
initiatives after coming from environments that present conflicting priorities in what Stanton-
Salazar (1997) refers to as simultaneously existing social worlds. This entails understanding the
competing demands faced by marginalized children and the ways in which those experiences
influence their perception and experience of education. According to CVCC’s founder, virtually
all of the center’s residential children have experienced some form of abandonment, abuse, or
neglect (personal communication, February 18, 2019). This factor, compounded with the
33
experience of endemic poverty, leads to attributions that the founder refers to as a “legacy of
failure” (personal communication, February 18, 2019).
Rana and Chaudry (2012) also described the negative psychological impacts that
marginalized children experience as a result of living in communities that simultaneously
exclude them while still evaluating them based on societal norms. Nasir et al. (2014), using
Maslow’s hierarchy of human development as a framework, reported that Pakistani street
children cannot advance to higher levels of self-actualization because their basic physiological
needs have not been met. Perhaps nowhere is this better reflected than the expectation that
children should attend formal education programs, while at the same time failing to provide them
with the socioeconomic support needed to meet their basic physiological needs. While CVCC’s
holistic approach is designed to support the social, emotional, and physical needs of previously
abandoned, neglected, abused children, significant academic and non-academic barriers to
learning persist in formal schools. LEAP team members need knowledge of how to identify
learning challenges resulting from academic and non-academic barriers as a basis for designing
and developing effective NFE programs.
Knowledge of Effective Learning Theories for Use with Marginalized Children
The third knowledge influence LEAP team members need is understanding theories of
learning that are effective for educating marginalized children. This knowledge influence is
declarative-conceptual in nature as it reflects domain-specific principles, concepts, or structures
that influence marginalized children’s learning outcomes and academic achievement. Based on
Shephard’s (2014) systematic review on NFE efficacy, Kolb’s Experiential Learning Theory
(ELT) and Vygotsky and Feuerstein’s Mediated Learning Experience (MLE) feature prominently
in NFE programs. In ELT, Kolb advocated a constructivist theory of learning in which
knowledge is created through the transformation of experience (Kolb & Kolb, 2012). ELT rejects
34
the theory of educators as transmitters of knowledge and students as receptacles of knowledge
(Kolb & Kolb, 2012). ELT more closely aligns with Mayer’s (2011) description of knowledge
construction, which characterizes educators as “cognitive guides” and learners as “active sense
makers.” In a similar fashion, Vygotsky and Feuerstein’s MLE theory focuses on the interaction
between a mediator, who manipulates stimuli in an effort to develop a learner’s cognitive
functions, and a learner, who integrates and internalizes the MLE process to enhance future
learning (Tzuriel, 2012). Both theories emphasize the central importance of the learner’s role in
building cognitive representations based on experience and interaction, as opposed to being
passive recipients of knowledge (Mayer, 2011).
LEAP team members need knowledge related to learning theories that advance student
achievement and, as a result, their own performance goals. Selected theoretical constructs must
recognize the importance of knowledge construction, generative cognitive load, and
metacognition in achieving learning outcomes (Mayer, 2011; Rueda, 2011). While not mutually
exclusive, LEAP team members also need to balance ELT and MLE’s process-oriented focus
with the need to measure performance outcomes for the purpose of internal and external
accountability.
Knowledge of How to Improve Instructional Approaches to Promote Higher Academic
Achievement
The final knowledge influence LEAP team members need is understanding how to
improve instructional approaches to improve the academic outcomes of marginalized children.
This knowledge is metacognitive in nature as it requires educators to evaluate their existing
approaches to identifying and leveraging the most effective learning processes to use in a given
situation (Mayer, 2011). Metacognition is composed of two core cognitive processing functions,
including metacognitive awareness and metacognitive control (Mayer, 2011). Metacognitive
35
awareness reflects a learner’s understanding of effective learning strategies, while metacognitive
control dictates the appropriate application of these strategies (Flavell, 1979).
The identification of metacognitive strategies and self-regulated learning play an
important role in fostering improved performance in both teachers and students (Jiang, 2016).
Prytula (2012) underscored the importance of self-regulated learning in teacher’s long-term
professional development, as well as in promoting students’ self-regulated learning. Veenman et
al. (2006) suggested that the use of metacognitive instruction by teachers promotes the use of
metacognitive strategies among students. According to the authors, students learn metacognitive
strategies through modeling the behaviors of parents and teachers and that the development of
these skills are essential to improving students’ academic performance, particularly amongst
low-performing students (Veenman et al., 2006). Kramarksi and Michalsky (2009) suggested
that the use of metacognitive strategies by educators is a precondition of developing self-
regulated learning in students.
LEAP team members need knowledge related to improving their own pedagogy as a
condition of promoting better student achievement outcomes. Bandura (2005) stated that the
mastery of conceptual and procedural knowledge is insufficient to solve complex problems in the
absence of metacognitive strategies. As a result, LEAP team members need to demonstrate the
conceptual knowledge required to identify student performance challenges, the procedural
knowledge to develop NFE solutions to address those challenges, and the metacognitive
knowledge needed to continuous evaluate and improve those solutions.
Table 2 presents CVCC’s organizational mission, the global goal for its LEAP program,
and the stakeholder goal for the organization’s LEAP team members. In addition, it summarizes
the key knowledge influences, knowledge types, and sample assessment approaches for the three
knowledge influences described in the previous section from the perspective of the stakeholders.
36
Table 2
Knowledge Influences, Types, and Assessments for Analysis
Organizational Mission
CVCC’s mission is to provide for the safety, security, well-being, and development of
disadvantaged Filipino children through the provision of residential care services, education,
and community outreach.
Stakeholder Goal
The goal of CVCC’s LEAP team is to improve the academic achievement of 100% of
CVCC’s residential children through non-formal education by 5% as measured by academic
performance in formal schools by June 2021.
Assumed Knowledge
Influences
Knowledge Type (i.e.,
declarative, procedural, or
metacognitive)
Knowledge Influence
Assessment
LEAP team members need
knowledge of the
relationship between
student performance data
and student achievement
Declarative (conceptual)
LEAP team members will
be asked to describe use
of student performance
data to identify barriers to
academic achievement
LEAP team members need
knowledge of how to
identify gaps in students’
ability to succeed in formal
schools
Declarative (procedural)
LEAP team members will
be asked to describe two
approaches to
benchmarking student
achievement to inform
decision-making.
LEAP team members need
knowledge of how to apply
learning theories to
promote higher academic
achievement
Declarative (conceptual) LEAP team members will
be asked to describe
application of one or more
learning theories designed
to improve academic
achievement
LEAP team members need
knowledge of how to
improve NFE instructional
approaches to promote
higher academic
achievement
Metacognitive LEAP team members will
be asked to describe the
use of reflective practices
and evidence-based
outcomes to enhance
instructional approaches
37
Motivation Influences
Motivation, according to Clark and Estes (2008) is one of three causes of performance
gaps. In order to attain performance goals, individuals need both the knowledge of how to
complete a task and the desire to do so (Clark & Estes, 2008; Eccles, 2006; Mayer, 2011; Rueda,
2011). Motivation, as defined by Mayer (2011), originates within the individual and is goal
oriented. Clark and Estes (2008) identified three ways in which motivation influences goal-
directed behaviors, including choice, persistence, and mental effort. In order to achieve a goal an
individual must actively choose to pursue a goal, persist in working toward that goal in an
environment of competing priorities, and dedicate an appropriate level of mental effort towards
achieving the goal (Clark & Estes, 2008). A comprehensive assessment of organizational
performance requires consideration of motivation influences based on evidence-based theoretical
approaches.
Motivation theory is composed of six major theoretical perspectives, including self-
efficacy and competence; attributions and control beliefs; value; interest; goals and goal
orientation; and emotions (Pekrun, 2011; Rueda, 2011). While each of these theories has
relevance to LEAP team members’ performance and achievement, this section will focus on two
relevant motivation influences, including self-efficacy and goal orientation. These two theories
could provide LEAP team members with a framework for identifying performance challenges
that can inhibit achievement.
LEAP Team Members Need to Believe in Their Own Abilities
In order to achieve their performance goals, LEAP team members need a sense of self-
efficacy. According to Pajares (2006), self-efficacy is an individual’s perceptions about their
ability to perform a given action at a specific level of performance. According to the author, self-
efficacy beliefs are a prerequisite to action, perseverance, and self-regulation. These
38
characteristics closely align to Clark and Estes (2008) use of choice, persistence, and mental
effort as indicators of goal-directed behavior. Research has demonstrated that self-efficacy
beliefs can positively or negatively influence individuals’ choices, regardless of whether their
beliefs are accurate or objective (Clark & Estes, 2008; Rueda, 2011). As a result, understanding
the root causes of low self-efficacy is critical to ensuring that individuals choose to pursue a task,
persist in the face of obstacles, and devote the effort necessary to achieve performance goals.
The factors that influence an individual’s self-efficacy beliefs are complex. Rueda (2011)
listed prior knowledge, feedback, and past success and failure as significant contributors in the
development of self-efficacy. Pajares (2006) stated that self-efficacy beliefs result from four
primary sources that incorporate and expand on Rueda’s list, including mastery experience,
vicarious experience, social persuasion, and physiological reactions. Pajares’ framework serves
as a foundation for diagnosing self-efficacy issues and suggests approaches to promoting higher
self-efficacy, including scaffolding work to appropriately challenge learners, modeling
behaviors, and providing collaborative learning opportunities (Pajares, 2006).
LEAP team members require self-efficacy beliefs in order to achieve targeted
performance goals. In order to ensure higher self-efficacy among LEAP team members, CVCC
administrators need to promote self-efficacy by establishing challenging, achievable goals;
providing accurate feedback focused on tasks; and increasing opportunities for collaboration
between LEAP team members. According to Pajares (2006), self-efficacy promotes the
increased use of self-regulation and metacognition, which leads to reduced stress, increased
persistence, and higher achievement outcomes. LEAP team members, faced with limited
resources and the demands of working with a marginalized population, need to believe they are
capable of developing effective curriculum and instructional approaches that address the diverse
educational needs of marginalized children.
39
LEAP Team Members Need a Mastery Level Goal Orientation towards NFE provision
Goal orientation theory focuses on the underlying reasons that motivate individuals to
engage in a task (Pintrich, 2003; Rueda, 2011). Rueda (2011) noted that goal orientation theory
acknowledges two broad categories of goals, including mastery goals and performance goals.
Individuals who embrace mastery goals, according to Yough and Anderman (2006), strive to
fully comprehend the task, focus on self-improvement, and compare their achievement to their
own prior achievement. In contrast, the authors stated that individuals who embrace performance
goals focus on publicly demonstrating their abilities, competing with others, and comparing their
achievement with the achievement of others (Yough & Anderman, 2006).
In order to achieve their performance goals, LEAP team members need not only believe
that they are capable of performing the tasks necessary to succeed, but also need to develop a
mastery goal orientation towards pursuing their goals. Yough and Anderman (2006) indicated
that individuals may embrace multiple goals, prioritize them based on their perceived value, and
alternatively exhibit mastery and performance goal orientations simultaneously for different
tasks. In order to achieve their target performance goals, LEAP team members need to adopt an
intrinsically based, mastery goal orientation towards their work in an environment of competing
goals. Yough and Anderman (2006) and Rueda (2011) highlighted the TARGET conceptual
framework as one method of shifting individual’s approaches from a performance goal
orientation to a mastery goal orientation. The TARGET framework incorporates motivational
concepts into six aspects of instruction that can be controlled by the instructor and influence
motivation, including tasks, authority, recognition, grouping, evaluation, and time (Rueda, 2011;
Yough & Anderman, 2006). CVCC’s LEAP team is composed of a variety of individuals
participating in its NFE initiative, ranging from a full-time education coordinator who oversees
the LEAP program to house parents who provide ongoing support to residential children on daily
40
homework assignments. As a result, the educational attainment and teaching experiences of
LEAP team members widely vary, requiring the active involvement of the Education
Coordinator in developing a mastery goal orientation among team members. A TARGET
approach would allow the Education Coordinator to assist in defining tasks, promote
participation, encourage collaborative instructional development, and provide opportunities for
assessment for improving program performance. Simultaneously, LEAP team members can
apply TARGET principles in the context of their own instructional approaches.
Table 3 presents CVCC’s organizational mission, LEAP’s performance goal, and the
stakeholder goal for the organization’s LEAP team members. In addition, it presents the two
motivation influences described in the previous section accompanied by sample assessment
approaches and interview prompts
Table 3
Motivation Influences and Assessments for Analysis
Organizational Mission
CVCC’s mission is to provide for the safety, security, well-being, and development of
disadvantaged Filipino children through the provision of residential care services, education,
and community outreach.
Stakeholder Performance Goal
The goal of CVCC’s LEAP team is to improve the academic achievement of 100% of CVCC’s
residential children through non-formal education by 5% as measured by academic
performance in formal schools by June 2021.
Assumed Motivation Influences Motivation Influence Assessment
LEAP team members need to believe they are
capable of developing effective curriculum
and instructional approaches.
Sample interview prompts:
Walk me through a typical educational
interaction between you and the children.
41
Tell me about your experiences in teaching
prior to joining CVCC.
Describe the most challenging aspect of your
work with the LEAP program.
Walk me through your process of developing
a lesson plan for an educational session.
LEAP team members need to develop a
mastery level goal orientation towards NFE
provision for marginalized children.
Sample interview prompts:
Tell me about a time, if ever, that you
modified an aspect of your instructional
approach.
Probe: What led you to make those changes?
How often, if ever, do you have opportunities
to compare instructional approaches with
other educators?
Tell me about a time where a CVCC child
demonstrated improved academic
achievement in formal schools.
Probe: What do you believed the key factors
were that contributed to the child’s success?
Organizational Influences
Along with knowledge and motivation, organizational influences represent the third
potential cause of performance gaps in organizations (Clark & Estes, 2008). According to Clark
and Estes (2008), performance is influenced by organizational processes, material resources,
value streams and chains, and culture. Organizational goals are achieved through the application
of knowledge, skills, and motivation; appropriation of adequate material resources; and
integration of organizational processes (Clark & Estes, 2008). Each of these factors is important
in identifying and diagnosing performance gaps and are in turn influenced by culture, which
encompasses the underlying beliefs, values, emotions, and processes that define the organization
(Clark & Estes, 2008; Gallimore & Goldenberg, 2001; Rueda, 2011; Schein; 2017). As a result,
42
understanding an organization’s culture provides visibility into the underlying values and beliefs
that inform and drive organizational behavior. However, culture eludes static, simplistic, and
unidimensional representations.
Culture is a complex concept. Schein (2017) defined culture as accrued, shared learning
that successfully responds to challenges of external adaptation and internal integration. Over
time, successful strategies for addressing organizational challenges are internalized as de facto
responses to internal and external demands on the organization (Schein, 2017). However, culture
is multidimensional and dynamic (Erez & Gati, 2004; Schein, 2017). As opposed to a static
concept, culture should be viewed as the evolving response of an organization to internal and
external stimuli (Schein, 2017). While this perspective of a responsive and evolving culture
synchronizes well with Clark and Estes (2008) gap analysis conceptual framework, Rueda
(2011) noted the complexity of operationalizing culture due to a lack of organizational
transparency and the automated nature of cultural knowledge.
The concept of culture is also elusive. While certain components of culture are easily
identifiable, the underlying assumptions driving organizational culture are often difficult to
ascertain (Clark & Estes, 2008; Rueda, 2011; Schein, 2017). Schein (2017) stratified culture into
three categories including artifacts, espoused values, and underlying assumptions, based on their
level of observability and descriptive power. Artifacts, which are readily observable components
of culture such as language and clothing, are of limited utility in understanding organizational
culture (Schein, 2017). On the opposite end of the spectrum, Schein (2017) described
organizations’ underlying assumptions as difficult to identify, but capable of providing an in-
depth understanding of an organization’s behavior.
Gallimore and Goldenberg (2001) developed a framework for analyzing organizations
based on the culture underlying the organization’s structures, processes, and dynamics.
43
According to the authors, organizational culture can be broadly aggregated into “cultural
models” and “cultural settings.” Cultural models define a group’s understanding of how the
world works, while cultural settings describe interactions between individuals within specific
organizational contexts (Gallimore & Goldenberg, 2001; Rueda, 2011). Rueda (2011) also
identified the reciprocal relationship between cultural models and settings, noting that
individuals both inhabit and influence cultural settings with direct consequences for the
overarching cultural model that defines organizational norms. Erez and Gati (2004) expanded on
the dynamic nature of this relationship, describing it as a top-down, bottom-up process in which
higher-level cultural models influence individual behavior and, in turn, cultural models are
influenced by individual interaction and sharing. Cultural models and settings provide a
framework for identifying, interpreting, analyzing and understanding organizational behavior.
Although multiple instances of cultural models and settings may simultaneously exist in an
organization, this study focuses on one cultural model and two associated cultural settings that
are integrally related to the problem of practice.
CVCC Needs to Align Its Key Stakeholder and Organizational Goals
Organizations need to clearly articulate their goals and align them with the organizational
mission as a precursor to developing a learning organization that strives for continuous
improvement (Clark & Estes, 2008). Goal development occurs within what Gallimore and
Goldenberg (2001) refer to as a cultural setting, in which key stakeholders in the organization
interact within a specific organizational context. Rueda (2011) emphasized the importance of
establishing goals, including long-term, intermediate, and day-to-day performance goals, as a
first step in implementing Clark and Estes (2008) gap analysis model. In a systematic review of
street child intervention programs, Berckman et al. (2012) reported that a common shortcoming
among the interventions was the lack of a long-term objective, which resulted in difficulties in
44
evaluating the program efficacy. An organization’s goals also need to account for the realities of
organizational capacity with goals being moderated as necessary to make them attainable given
organizational limitations (Hentschke & Wohlstetter, 2004; Marsh & Farrell, 2015). Goals must
also align with an organization’s espoused values and philosophy (Schein, 2017).
In order to demonstrate the efficacy of the LEAP program, CVCC needs clearly defined
long-term, intermediate, and day-to-day performance goals to ensure that the goals cascade from
the organizational mission and that key stakeholders’ performance goals support the overarching
goal of the organization (Clark & Estes, 2008). Without clearly defined goals, as noted by
Berckmans et al. (2012), it will be challenging for CVCC to measure and improve performance
or demonstrate the efficacy of its LEAP program. As a result, establishing goals is an essential
initial step in demonstrating program efficacy and accountability.
CVCC Needs to Optimize Program Efficacy and Accountability
A paucity of evidence-based research exists related to the efficacy of non-formal
education initiatives in delivering on targeted learning outcomes (Shephard, 2014). Non-profit
organizations, however, increasingly face pressure to demonstrate a return-on-investment to both
internal and external stakeholders (Campbell, 2002). Developing effective assessments serve
these two functions. Effective assessments provide evidence-based research that supports the
identification of organizational performance gaps and design of solutions (Clark & Estes, 2008;
Rueda, 2011). Concurrently, effective performance assessment serves as a mechanism for
responding to the demands of external accountability. Ebrahim (2010) indicated the importance
of balancing demands for accountability with a commitment to fostering a learning organization
that delivers on the organization’s espoused values and established goals. As a result, effective
assessments can support an organization’s efforts to develop a learning mindset, while at the
same time demonstrating return-on-investment to external stakeholders (Clark & Estes, 2008;
45
Ebrahim, 2010; Rueda, 2011). Like goal setting, performance assessment occurs within what
Gallimore and Goldenberg (2001) would characterize as a “cultural setting,” involving multiple
stakeholders engaged around a targeted process. CVCC should foster this cultural setting and
evaluate performance assessment strategies to determine their potential for responding to internal
and external accountability demands.
CVCC Needs to Prioritize Collaboration and Foster Integration with Internal and External
Stakeholders
Organizations operate in a complex matrix of vertical and horizontal relationships that
have the potential to exert internal and external pressures on the organization (Schein, 2017).
Schein (2017) described how mental models result in standardized responses to internal and
external challenges based on precedents of success and anchored in the organization’s underlying
assumptions. However, these models are resistant to change and serve as a barrier to
organizational learning. Senge (1990) emphasized the importance of challenging existing mental
models at every level of the organization to foster systems thinking. A core component of
culture, as defined by Schein (2017), is accrued, shared learning that adapts to external
challenges and internal integration. As a result, culture is not a static concept. It is dynamic and
responsive to internal and external stimuli requiring adaptation in response to emerging
environmental and systemic challenges.
CVCC resides at the intersection of an evolving system in the Philippines designed to
more holistically address the needs of marginalized children. The evolution of DepEd’s ALS
program and the DSWD’s 4P conditional cash transfer program have altered the landscape of
non-formal education in the Philippines (World Bank, 2018; Yasunaga, 2014). Prior to the
launch of these programs, NFE provision was predominantly in the NGO domain. As a result,
CVCC needs to assess the role its NFE initiative fulfills within a broader educational context.
46
This assessment should prioritize systems thinking to determine ways in which CVCC’s nascent
services align with, overlap, complement, or supplement these emergent programs. In order to
achieve this goal, the organization needs to create a culture collaboration with a broad cross-
section of internal and external stakeholders capable of leveraging existing programs, as well as
informing the development of new solutions to shared challenges.
CVCC Needs to Provide LEAP Team Members Dedicated Time to Develop and Collaborate on
NFE Curriculum and Instructional Design
Student academic achievement is being increasingly linked to quality educators (Owen,
2014). Prenger et al. (2014) emphasized the importance of collaboration in educators’
professional development and ability to meet the needs of diverse student populations. Owen
(2014) underscored the benefits of providing educators with opportunities to develop and refine
their skills through professional learning communities (PLC) that encourage innovation, shared
experience, and reflective dialogue. According to Owen (2014), the collaborative nature of
professional learning communities (PLC) provide educators opportunities to improve as
educators, particularly when PLCs transcend collegial or superficial interactions and promote
goal setting, continuous improvement, student achievement, performance measurement, and
outcomes. Sargent and Hannum (2009) indicated that collaboration between educators “… leads
to increased involvement, ownership, innovation, and leadership among teachers” (p. 258). As a
result, the provision of dedicated time to develop and collaborate is essential to develop
educators capable of driving higher levels of student academic performance.
Table 4 presents CVCC’s organizational mission, LEAP’s performance goal, and the
LEAP team members’ goals. In addition, it summarizes the key organizational influences,
assessments, research-based recommendations, and proposed solutions for the three
47
organizational influences described in the previous section from the perspective of the
organization.
Table 4
Organizational Influences and Assessments for Analysis
Organizational Mission
CVCC’s mission is to provide for the safety, security, well-being, and development of
disadvantaged Filipino children through the provision of residential care services,
education, and community outreach.
Stakeholder Goal
The goal of CVCC’s LEAP team is to improve the academic achievement of 100% of
CVCC’s residential children through non-formal education by 5% as measured by
academic performance in formal schools by June 2021.
Assumed Organizational Influences Organizational Influence Assessment
CVCC needs to align its key
stakeholder and organizational
performance goals.
Individual interview questions requiring
respondents to verbalize organizational
performance goal, describe stakeholder
responsibilities for attaining that goal.
Document analysis of LEAP team members’
access of student performance data and CVCC
quarterly survey results.
CVCC needs to optimize program
efficacy and accountability.
Individual interview questions requiring
respondents to describe efforts to promote and
measure NFE program efficacy.
Focus group questions discussing CVCC
approaches to demonstrating program efficacy
and accountability.
CVCC needs to prioritize and foster
collaboration between internal and
external stakeholders.
Individual interview questions requiring
respondents to describe interactions related to
student academic achievement and
supplemental NFE development with local
DepEd stakeholders.
48
Focus group questions eliciting information
about frequency and quality of collaboration
between internal and external stakeholders
across shared goals.
CVCC needs to provide LEAP team
members dedicated time to develop and
collaborate on NFE curriculum and
instructional design.
Individual interview questions requiring
respondents to describe their experiences
developing NFE programs and instruction.
Focus group questions eliciting interactions
between LEAP team educators and external
stakeholders (e.g., DepEd teachers) to
collaborate on curriculum and instructional
design.
Conceptual Framework: The Interaction of Stakeholders’ Knowledge and Motivation and
the Organizational Context
Merriam and Tisdell (2016) describe a conceptual framework as the intersection between
what researchers believe about the world and the ways in which those beliefs influence their
research design. A conceptual framework describes a phenomenon as a researcher understands,
experiences, and constructs it based on existing theory and research, the researcher’s experiential
knowledge, exploratory research, and thought experiments (Maxwell, 2013). Moving from
theory to application, Maxwell (2013) delineated multiple uses of the contextual framework,
such as the justification for the research, assessment and refinement of research goals, selection
of appropriate methods, and interpretation of the findings. In the context of researching the
efficacy of CVCC’s non-formal education (NFE) in delivering on targeted educational outcomes,
the research is grounded in Clark and Estes (2008) gap analysis conceptual framework which
emphasizes the primary roles that knowledge, motivation, and organizational influences play in
organizational and stakeholder performance. While knowledge, motivation, and organization are
presented separately within the construct of the literature review, they are integrally related in
practice.
49
According to Maxwell (2013), a researcher’s own experiences influence their underlying
approach to framing and understanding the research question, design, and results. In this case,
the researcher views NFE initiatives from the perspective of basic education as a fundamental
human right that is best approached using pedagogy grounded in constructivist theories of
learning such as Kolb’s Experiential Learning Theory (ELT) and Vygotsky and Feuerstein’s
Mediated Learning Experience (MLE) (Tzuriel, 2012). As a result, the interaction between
LEAP team members as a cognitive guide or mediator and marginalized children as active
learners or “sense makers” (Mayer, 2011) is critical to the attainment of organizational goals.
50
Figure 1: Conceptual Framework of Stakeholder Knowledge, Motivation, Organizational, and Systemic Influences.
51
Figure 1 visually represents the relationship between organizational factors and LEAP
team members’ knowledge, skills, and motivation. Additionally, the figure illustrates the
organization’s cultural model and the primary cultural setting (Gallimore & Goldenberg, 2001)
in which LEAP team members and CVCC’s children interact. CVCC’s cultural model stems
from a human rights-based educational perspective in which education is viewed as a right as
opposed to a privilege, a perspective that mirrors international and national trends in education
(Robeyns, 2006). In order to provide non-formal education that is available, accessible,
acceptable, and adaptable (Tomaševski, 2001), the organization provides significant autonomy to
the LEAP team, but also demands accountability for program performance. The organization
also emphasizes flexible, holistic approaches that address both academic and non-academic
barriers to universal primary education and broader systemic integration with formal school
settings. LEAP team members play a pivotal role in actualizing these organizational goals.
LEAP team members are represented as both operating within and without the
organization due to their high level of autonomy in responding to the needs of marginalized
children. The centrality of the LEAP team’s role in developing and evaluating acceptable and
adaptable curriculum for marginalized children places them at a critical intersection between the
organization and its primary client (Datta & Banik, 2014). According to the World Health
Organization (2000), educators need the knowledge, skills, and attitudes to design and assess
instructional curriculum that considers both the academic and non-academic barriers facing
marginalized children. A significant component in developing an adaptive curriculum is
educators’ inclusion of marginalized children in the development of the curriculum (Grover,
2007; Rabasso & Rabasso, 2014), represented in the illustration by the cyclical arrows
connecting these key stakeholders. CVCC’s children, as reflected by the overlay on the arrow
flowing from the stakeholder to stakeholder goal, also play an integral role in stakeholder goal
52
attainment, reflecting the children’s role in knowledge construction and as active sense makers
(Mayer, 2011).
Motivation, including LEAP team members’ self-efficacy and goal orientation,
influences their ability to identify and address performance gaps to improve organizational
outcomes (Clark & Estes, 2008). CVCC’s cultural model emphasizes the importance of setting
standards for establishing challenging, achievable goals, providing accurate task-oriented
feedback, and promoting increased collaboration between LEAP team members to promote self-
efficacy. Pajares (2016) indicated that self-efficacy leads to higher use of self-regulation and
metacognition that promote improved organizational outcomes. The pursuit of mastery goals also
promotes continuous process improvement (Yough & Anderman, 2006). CVCC LEAP team
members need to demonstrate self-efficacy and goal orientation due to the highly autonomous
nature of their work, as well as to demonstrate learning outcomes that support the organizational
mission.
Conclusion
The purpose of this evaluation study is to determine the efficacy of CVCC’s supplement
NFE initiative in achieving targeted educational outcomes for marginalized children. The
literature review provided context for the emergence of an international human rights-based
agenda in education, the consequent challenges posed to traditional educational systems, and
NFE’s evolution as a response to meeting the demands of marginalized populations. The
challenge of providing universal basic education was framed in terms of academic and non-
academic barriers to educational availability, access, acceptability, and adaptability (Tomaševski,
2001). Chapter Two concluded by grounding the Philippine-based, non-profit CVCC’s LEAP
initiative within the broader context of challenges facing NFE initiatives internationally, as well
as aligning its stakeholder goals to the assumed influencers on performance covered in this
53
chapter. The literature highlighted the challenges experienced by NFE providers in terms of their
knowledge, motivation, and organizational capacity (Clark & Estes, 2008) to respond holistically
to marginalized children’s needs as a prerequisite to educational attainment. Consideration of
factual, conceptual, procedural, and metacognitive influences encompassed knowledge of
marginalized children’s academic and non-academic barriers to education, the development of
curriculum designed to address those barriers, and the approaches to measuring program
efficacy. Motivational influences focused on LEAP team member’s self-efficacy and goal
orientation in developing effective NFE instruction. Organizational influences addressed the
creation of an organizational culture that emphasizes clarity of purpose, accountability for
outcomes, and collaboration amongst internal and external stakeholders. Chapter Three describes
the study’s methodological approach, which represents the intersection of the extant literature,
researcher’s philosophical perspective, and selected research design.
54
Chapter Three: Methods
The purpose of this project was to conduct a qualitative evaluation study using an
inductive research approach to describe LEAP team members’ KMO influences in the provision
of supplemental NFE to CVCC residential children. The research questions guiding the design of
this evaluation study included:
1. What are LEAP education team members’ knowledge and motivation related to
achieving the organizational goal of ensuring that all CVCC students demonstrate
improved academic achievement in formal schools through the provision of non-formal
education?
2. What is the interaction between organizational culture and context and LEAP team
educator knowledge and motivation related to the provision of non-formal education?
This chapter presents the research design and methodology used to collect and analyze the data
required to respond to these research questions. This section presented an overview of the
purpose of the study and research questions. The following section provides a description of the
stakeholder group being studied and the rationale for their inclusion in the study. The chapter
will next describe the qualitative research strategies selected to collect the data, including a
discussion of the logic underlying those choices, a detailed description of the methods used, and
their connection to the study’s KMO influences, research questions, and conceptual framework.
Next, the chapter will describe the use of rich data, respondent validation, and triangulation
strategies to promote the credibility and trustworthiness of the proposed research design. A
discussion of ethics will follow, describing the methods used to protect respondent privacy and
confidentiality, as well as prioritize the respect for persons, beneficence, and justice throughout
the study. Finally, the chapter will present a summary of the research study’s major limitations
and delimitations.
55
Participating Stakeholders
CVCC’s LEAP team is the stakeholder population of focus for this research evaluation.
The LEAP team is composed of 16 individuals from diverse educational backgrounds with
widely varying teaching experiences. The roles of CVCC’s LEAP team is also diverse and
includes positions such as Education Coordinator, ESL Instructor, Tutor, Social Worker, and
House Parent. Table 5 provides a detailed breakout of the number of LEAP team members by
role and their accompanying education-related responsibilities.
Table 5
LEAP Team Composition and Education-Related Responsibilities
Role Number Description of Responsibilities
Center
Director
1 Responsible for supervision of all LEAP team staff and
relationships with local DepEd and private schools.
Education
Coordinator
1 Oversees the LEAP, including coordinating LEAP programming
and activities; relationships with schools; and maintenance of
student performance and survey data.
Tutor
(Part-time)
2 Provides targeted homework support in core subjects with
emphasis on English, Mathematics, and Science. Offer ESL
courses on weekends and additional programming, as requested.
Social Worker 4 Provides holistic support to children, including counseling
services related to career guidance, academic success, and
disciplinary issues.
House Parent 8 Monitors children’s performance, identifies performance issues,
and refers children to NFE support services. Provides basic
homework support (predominantly elementary level) in evenings
during academic year and conducts occasional learning activities
during intersessions. Serves as a primary point of contact for
schools in regard to children’s performance and discipline.
Attends parent-teacher meetings and student events.
56
The percentage of each role’s time commitment to education also varies, with the sole Education
Coordinator committed full time to the program, while House Parents commit approximately
10% of their time to supporting educational activities. As a result of the relatively small team
size and diversity of its members’ roles and experiences, individual interviews will be conducted
with all LEAP team members, while a representative sample of LEAP team members will be
interviewed in the subsequent focus group session, as described below.
Interview and Focus Group Sampling Criteria and Rationale
Criterion 1
The interview sample targeted all currently employed CVCC LEAP team members actively
engaged in providing educational support services to CVCC’s residential children. The rationale
for this selection was the relatively small sample of relevant stakeholders, ease of their
identification, and resultant representativeness of the final sample (Burke Johnson &
Christensen, 2014). Note that while all LEAP team members were invited to participate in
individual interviews, three House Parents were unable to participate due to their remote
location, complications stemming from the COVID-19 pandemic, or difficulty in connecting via
web-based communication applications (e.g., Zoom). As a result, 13 LEAP team members
participated in the individual interviews.
Criterion 2
The focus group sample included a representative sample of LEAP team members (n=5) actively
engaged in providing educational support services to CVCC’s residential children, as well as two
local DepEd teachers. The LEAP team members were selected from those individuals who
participated in the initial round of individual interviews. The rationale for this selection was to
provide a more holistic account of the relational and systemic complexity involved in providing
57
supplemental NFE to CVCC children and the integrated roles of CVCC LEAP team members
and local DepEd teachers in supplemental NFE provision (Creswell, 2014).
Interview and Focus Group Strategy and Rationale
The researcher conducted anchored, semi-structured, online interviews with thirteen
LEAP team members to discuss, interpret, and clarify data based on document analysis of
student performance data (Merriam & Tisdell, 2016). A single focus group interview composed
of five LEAP team members and two local DepEd teachers and DSWD social workers was also
conducted to elicit additional information on both educators’ knowledge and motivational
influences, as well as insights into broader organizational and systemic influences impacting
supplemental NFE program efficacy. The composition of the group reflects the
interconnectedness of stakeholders engaged in NFE provision and support, but also variation in
terms of each role’s perspective on the challenges influencing CVCC children’s academic
achievement (Krueger & Casey, 2009).
Explanation for Choices
The selected sampling approach reflected the centrality of the LEAP team members’
roles within the conceptual framework, while the selected methods respond to the study’s
research questions related to knowledge, motivation, and organizational influences on NFE
program efficacy. The use of document analysis promoted an emergent, inductive, case study
design that focuses on depth of understanding and the meaning made by participants in the NFE
program (Creswell, 2014). The combined use of document analysis, interviews, and focus group
supported a deeper understanding of the correlation between espoused attitudes and observed
behavior (Burke Johnson & Christensen, 2014). Combined with document analysis, the
methodology provided multiple perspectives of NFE program efficacy, as well as the ability to
triangulate the study’s findings.
58
Qualitative Data Collection and Instrumentation
This evaluation study used two qualitative data collection methods to respond to the
project’s research questions, including document analysis and interviews. Two CVCC data
sources were referenced as part of the document analysis process, including the results of a
quarterly survey focused on student self-efficacy, expectancy, and attribution, as well as
aggregated academic performance data garnered from quarterly report cards. Document analysis
of these data was used to provide insights into CVCC children’s behavioral characteristics and
academic performance, as well as further inform the research design. The underlying rationale
for the analysis of these documents included the provision of context; clarification of the
research design; supplementation of the research data; tracking of program performance; and
triangulation of research findings (Bowen, 2002). The results of this analysis also served as a
basis for discussion during subsequent individual and focus group interviews with CVCC’s
LEAP team members.
Standardized open-ended interviews were used to respond to research questions related to
the knowledge and motivation of LEAP team stakeholders, as well as the ways in which
organizational culture and context interact with these influences. The primary reason for using
interviews was to acquire rich data regarding the research topic based on the perspectives of
individuals who are intimately involved in the work (Bogdan & Biklen, 2007; Maxwell, 2013;
Merriam & Tisdell, 2016). Patton (2002) stated that the use of standardized open-ended
interviews also provides research consumers access to the original evaluation instrument; focuses
the interview; and facilitates analysis of the data. Individual interviews served as a basis for
understanding the individual KMO influences on student academic achievement, but also
informed a final focus group interview that emphasized identifying organizational and systemic
challenges to program performance.
59
Following the completion of individual interviews, a focus group interview comprised of
a broader cross-section of key internal and external stakeholders was used to elaborate on issues
addressed in the initial interviews and to surface new information. In their discussion of focus
groups, Merriam and Tisdell (2016) underscored the importance of focus groups in identifying
new data previously unidentified in individual interviews. The researcher used an interview
guide that provided core questions and a topical framework for the interview, while still
supporting the inductive nature of the research. Patton (2002) indicated that the use of an
interview guide is essential in conducting focus groups as it provides structure without restricting
the expression of individual experiences and perspectives.
Figure 2 illustrates the progressive nature of the research design process. Note that the
research was inductive and each phase built upon the former, progressing from an analysis of the
outcomes of the program to the KMO influences of the primary stakeholders and, finally, the
ways in which systemic factors influence both stakeholder perspectives and outcomes.
60
Figure 2: Applications of Selected Methods to Conceptual Framework
61
Documents and Artifacts
The initial phase of the research study analyzed two primary document sets, including
CVCC’s quarterly surveys on student academic behaviors and aggregated student academic
performance data as reflected in quarterly grade reports. CVCC’s first student survey launched in
April 2020 and focused on better understanding student self-efficacy, expectancy, and attribution
in respect to their academic performance and achievement. CVCC collects survey data in the
month immediately following the issuance of quarterly grade reports in the Philippines, which
occurs in June (Q1), September (Q2), January (Q3), and April (Q4). CVCC’s LEAP team
provided aggregated survey and academic performance data to the researcher in compliance with
the Republic of the Philippines’s regulatory policies related to the dissemination and use of
student data. While CVCC provided the data in an aggregated format, the researcher also
conducted an independent review of the data to ensure student confidentiality and privacy per the
University of Southern California’s Institutional Review Board (IRB) guidance.
CVCC uses the quarterly surveys and student grade reports as a measure of LEAP’s
effectiveness in improving the level of student self-efficacy and academic achievement. As a
result, the data’s inclusion in the research was essential to discuss program performance within
the context of individual and focus group interviews related to KMO influences. Bowen (2009)
underscored the importance of using a systematic process for the analysis and interpretation of
documents in order to make meaning. Data from both the surveys and grade reports informed the
interviews and discussions of the LEAP team’s KMO influences, specifically as they relate to
student achievement, while interviews reciprocally helped bring deeper meaning to the survey
and performance data. The researcher provided a summary of the data to interview participants
no less than one week prior to their scheduled interview session for review.
62
Interviews
Interview Protocol
The research project used both standardized, open-ended individual interviews and a focus group
interview to collect rich, descriptive data. As Patton (2002) discussed, a standardized interview
approach may restrict flexibility in identifying individual variations among respondents but
strengthens the comparability of responses and facilitates the analysis of the data collected. The
researcher selected this approach based on several criterion. First, the researcher anticipated that
the diversity of the LEAP team members’ roles and experiences would reflect variation in their
perspectives and responses to the standardized interview prompts. Second, the open-ended nature
of the questions allowed for a certain degree of variance within individual responses to
standardized interview prompts. Finally, the subsequent focus group session was designed to
surface previously unidentified challenges and issues, providing a purposeful sample of the
respondents an opportunity to address issues not covered in the standardized interviews. As a
result, the research design addressed some of the weaknesses of the standardized, open-ended
approach by simultaneously ensuring a higher degree of comparability in participant responses
and facilitating the analysis of the data collected.
The types of questions asked during individual interviews focused on LEAP team
members’ knowledge and motivation influences and their relationship to organizational culture
and context (See Appendix A). The categories of interview questions varied, incorporating
questions related to respondents’ experience and behavior, opinions and values, feelings,
knowledge, sensory experiences, and demographics (Patton, 2002). While the researcher used a
variety of question types, every question allowed for an open-ended response. Additionally, each
question was mapped to a specific aspect of one or more of the project’s research questions.
63
The 60-minute focus group session emphasized eliciting new issues and challenges,
identifying broader organizational and systemic factors that may influence program efficacy, and
presenting participants an opportunity to compare their perspectives. The researcher used an
interview guide approach to conduct the focus group session (See Appendix B). While an
interview guide approach may have resulted in the omission of relevant topics and reduced data
comparability with the individual interviews, it allowed for a more conversational approach that
promoted interactive discussions linked to core topics identified in the research (Patton, 2002).
As a result, the interview phases of the research design reflected different, but complementary,
interview approaches. Bogdan and Biklen (2007) emphasized that researchers should not feel
restricted to using a single approach to interviewing, but rather implement the most effective
approach based on the demands of the research initiative.
Interview Procedures
The researcher conducted both individual and focus group interviews across the course of
a two-week period in August 2020. The decision to conduct the interviews in this timeframe was
based on three factors, including the Philippine Department of Education’s (DepEd) academic
year, the issuance of students’ final grade reports, and the availability of CVCC’s tabulated
survey data. As noted, information garnered from the document analysis phase informed both the
individual and focus group interview sessions, requiring the availability of the aggregated
quarterly and annual survey and performance data prior to conducting the interviews.
The researcher conducted recorded, online individual interviews with thirteen LEAP
team members, including the Center Director, Education Coordinator, English-as-a-Second
Language instructor, evening tutor, 4 Social Workers, and 5 House Parents, each of whom have
specific responsibilities and goals related to the children’s educational outcomes. Interview
sessions were limited to 60 minutes in duration for a total of approximately 13 hours of recorded
64
interviews. In limited cases, the researcher scheduled a follow-up interview to complete a session
or to clarify questions related to a participant’s earlier responses. The relatively small size of the
LEAP team and diversity of individual roles in supporting students’ academic achievement drove
the decision to conduct individual interviews of all LEAP team members. While three House
Parents were unable to participate in the individual interviews, their role was well-represented by
the five House Parents who could participate. The primary goal, as indicated in Merriam and
Tisdell (2016), is to reach a saturation point at which no new information is being derived from
the interviews.
The researcher also conducted one 60-minute focus group session, which included 7
participants. The composition of the focus group was based on a purposeful sample of five of
CVCC’s LEAP team members, as well as two teachers from local schools in which CVCC
children are enrolled. The LEAP team members included the Education Coordinator, a House
Parent, Social Worker, ESL instructor, and evening tutor. The two teachers were selected local
schools where CVCC children attend, including an elementary teacher from Lorenzo Ruiz
Elementary School and secondary teacher from Bonifacio Science High School. Due to COVID-
19 and the demands placed upon teachers to pivot from onsite to remote learning, only one
teacher was selected to participate in the focus group from the elementary and secondary school
level. While Merriam and Tisdell (2016) indicated that six to ten attendees is frequently cited in
research as an optimal focus group size, the primary factor in determining the focus group size
for this study was interviewing a purposeful sample with maximum variation.
The researcher conduct all interviews online using Zoom, emphasizing the importance of
providing a balance between convenience, privacy, and safety for respondents, as well as optimal
conditions for the recording of the interviews. All interviews were conducted in English.
However, in limited cases where Cebuano or Tagalog was used in the context of the focus group
65
interview, the researcher had access to a local interpreter who was also a participant in the focus
group. Any instances of Cebuano and Tagalog in transcripts derived from the recordings were
retained along with the translated text of the response. Maxwell (2013) indicated that recording
is essential in ensuring that the thick, rich description of the interview session is captured, as well
as promoting the credibility and trustworthiness of the researcher’s findings.
Credibility and Trustworthiness
Maxwell (2013) underscored the importance of addressing challenges to the credibility
and trustworthiness of a research initiative as an integral component of the research design. The
failure to address these challenges, according to Maxwell (2013), evokes questions related to
bias stemming from an individual researcher’s interpretation of the research findings and the
failure to acknowledge the possibility of alternative interpretations. Three primary strategies
were used to address these challenges to the credibility and reliability of the research, including
the collection of rich data, respondent validation, and triangulation.
The research design called for the collection of rich, descriptive data from intensive
individual interviews of LEAP team members, as well as a subsequent focus group interview of a
broader cross-section of internal and external stakeholders. The researcher generated verbatim
transcripts from the Zoom-based recordings of interviews. Maxwell (2013) indicated the
importance of long-term involvement and intensive interviews in order to develop a deep
understanding of the situational context. Rich, descriptive data, according to Merriam and Tisdell
(2016), is essential for consumers of research to not only validate the quality of the research, but
also to draw analogies in terms of the generalizability of the results to their own context.
The researcher also implemented a respondent validation or member checking process to
provide participants the opportunity to provide feedback on the interpretations and conclusions
drawn from the data. Maxwell (2013) and Merriam and Tisdell (2016) indicated that respondent
66
validation is essential in minimizing researcher bias and ensuring that research findings
accurately reflect the perspective of the study participants. Study participants had the opportunity
to review and provide feedback on the analytical findings derived from the study following the
interviews but prior to publication. The researcher reviewed respondent feedback to verify that
participants’ perspectives are accurately reflected. The ultimate goal of this process was to
ensure that respondents can recognize their own lived experience and perspectives in the
researcher’s final interpretations and conclusions (Merriam & Tisdell, 2016).
The final strategy used to support the credibility and trustworthiness of the research was
triangulation. Denzin (1978) identified four primary approaches to triangulating research,
including the use of multiple investigators, data sources, methods, and theories. For the purposes
of this research, the researcher used multiple sources of data as represented by the diverse roles
and perspectives of the LEAP team members and external stakeholders, as well as multiple
methods including document analysis and interviews. While the study did not employ multiple
investigators, the dissertation was subject to peer review as a standard component of the
dissertation process. In particular, the focus group session was designed to encompass a broader
range of perspectives on the efficacy of LEAP and the ways in which team members’ KMO
influences impact program outcomes.
Ethics
A key measure of a research study’s quality is whether it is conducted ethically and with
integrity (Merriam & Tisdell, 2016). Respect for persons, beneficence, and justice are three
central tenets to the ethical conduct of human subject research as reflected in Federal regulations
and institutional review board (IRB) guidelines (Glesne, 2011). However, Rubin and Rubin
(2012) indicated that ethical responsibility for the protection of human subjects extends beyond
compliance with Federal regulations and IRB guidance. According to the authors, researchers are
67
ultimately accountable for the ethical treatment and protection of research participants (Rubin &
Rubin, 2012). This research initiative strove to incorporate measures designed not only to
comply with Federal and IRB guidance, but to respond to the deeper ethical obligation of
protecting the well-being of all study participants. These measures included ensuring that
individuals understand their rights as participants, the ways in which the data they provide will
be used, and were treated as collaborators with, as opposed to objects of, the study.
Informed consent provides research participants information related to the nature of the
research, the risks involved in participation, and the voluntary nature of participation (Rubin &
Rubin, 2012). As a component of the informed consent process, LEAP team members were
informed that their participation was voluntary, confidential, and could be terminated at any time
during the course of the study without penalty. Additionally, the informed consent
documentation informed participants that they retained ownership of the data collected during
individual and focus group interviews and could restrict or prevent its use at any time (Rubin &
Rubin, 2012). The researcher ensured confidentiality for both the organization and study
participants in any external reports or presentations related to the study’s findings. The research
also used an open autocratic research approach (Scott & Usher, 2011), allowing participants to
review and provide feedback on the research findings prior to dissemination.
All individual and focus group interviews were recorded and transcribed. As required,
elements of the interview sessions were translated due to the use by respondents of one of the
multiple languages in which CVCC operations are conducted, including Filipino, Cebuano, and
English. The researcher requested separate permission to record and use participants’ responses,
as well as provided participants the opportunity to terminate the recording at any time or request
that their response not be used (Rubin & Rubin, 2012). Prior to and during the recording of any
session, respondents were also be reminded that the session was being recorded. Additionally,
68
respondents were afforded the opportunity to review transcripts and request the modification or
deletion of selected data.
In order to ensure respondent confidentiality, the researcher minimized collection of
personal-identifying data, assigned pseudonyms to all study participants, and disassociated stored
respondent data from respondents to protect participants’ privacy (Rubin & Rubin, 2012). As
necessary, the researcher edited or modify transcripts and data to ensure respondent
confidentiality (Rubin & Rubin, 2012). Finally, all research data, notes, transcriptions, and other
data associated with the research was stored and disposed of in compliance with the University
of Southern California’s Institutional Review Board’s (IRB) guidelines.
Conducting research in the Philippines poses distinct cultural and ethical challenges,
including the potential for a biased research design, participant exploitation, and a lack of
reciprocity (Glesne, 2011). In order to minimize a Western-centric research design bias, the
researcher collaborated with key stakeholders at CVCC to define the research problem and its
relevance, assess its potential risks and benefits, and establish guidelines on accountability and
reciprocity (Glesne, 2011). Although the researcher has no direct managerial or oversight
responsibilities at CVCC, the researcher gave special consideration to the power relationship
between the researcher and participants, as well as acknowledged the relevance of U.S.
colonialism as an element of Filipino identity in the design and conduct of the research.
The goal of the aforementioned measures was to protect the well-being of research
participants across the spectrum of procedural, situational, and relational challenges encountered
in the conduct of the research (Tracy, 2010). While the research design provided a procedural
framework for ethical research, the researcher also prioritized the respect for persons,
beneficence, and justice in response to situational and relational issues encountered in the field.
The targeted outcome of the research involved transcending the requirement of “do no harm” and
69
emphasized collaborative, mutually beneficial research designed to advance CVCC’s efforts to
meet the educational needs of marginalized children.
Limitations and Delimitations
The proposed research design and methodology presented a number of key limitations
and delimitations for consideration. Several of the key limitations included the truthfulness of
respondents during individual and focus group interviews; the reliability and validity of CVCC’s
survey data used in document analysis; and the timeliness of student performance data provided
by formal schools. Key delimitations included the potential for CVCC’s LEAP team to influence
measurable gains in student academic achievement over a relatively brief research period, as well
as the time constraints placed on data collection due to the lack of proximity of the research site.
Two major considerations also arose from the perspective of the conceptual framework. The first
consideration rested in the assumption that LEAP team members’ KMO influences were directly
tied to CVCC’s students’ academic achievement. The second assumption related to the
assumption that systemic influences directly influenced LEAP team performance and goal
acquisition, extending Clark and Estes (2008) framework to a model that incorporates not only
internal influences under the organization’s control, but also external influences stemming from
systemic influences (i.e., a KMOS model).
70
Chapter Four: Results and Findings
The purpose of this study was to evaluate the KMO influences of CVCC’s LEAP team
members in the provision of supplemental NFE to improve the academic achievement of its
residential children in formal schools. The guiding questions for the study included:
1. What are LEAP education team members’ knowledge and motivation related to
achieving the organizational goal of ensuring that all CVCC students demonstrate
improved academic achievement in formal schools through the provision of non-formal
education?
2. What is the interaction between organizational culture and context and LEAP team
educator knowledge and motivation?
Assumed KMO influences were developed based on a review of the extant literature, informal
observations during two initial visits to the research site, concurrent visits to local schools
attended by CVCC children, and preliminary conversations with LEAP team members and local
DepEd principals and teachers. Informed by these efforts, it was assumed that LEAP team
members required knowledge of how to identify student performance gaps and develop NFE
approaches to addressing those challenges. More specifically, this entailed knowledge of the
relationship between student performance data and student achievement; knowledge of how to
identify gaps in students’ ability to succeed in formal schools; and knowledge of how to apply
learning theories and NFE instructional approaches to promote higher academic achievement.
Assumed motivational influences were similarly derived and included LEAP team members’
need to develop their sense of self-efficacy and mastery goal orientation in designing,
developing, and delivering effective NFE curriculum and associated instructional content.
Finally, assumed organization influences were identified that focused on cultural models and
cultural settings impacting LEAP team members’ knowledge and motivation in NFE provision.
71
The identified organizational influences emphasized creating greater alignment between the
organization’s espoused values regarding education and the expression of those values in
CVCC’s operations. These organizational influences included aligning stakeholder and
organizational performance goals; prioritizing increase collaboration between internal and
external stakeholders; optimizing program accountability and efficiency; and providing LEAP
team members dedicated time to develop and collaborate on the design of NFE curriculum and
instruction.
Qualitative research was conducted to validate these influences, including document
analysis, individual interviews, and a focus group interview, with each phase of the qualitative
research informing subsequent phases. Document analysis of aggregated student performance
data and a quarterly student survey were used to inform discussions during individual interview
sessions, which were in turn used to inform the final focus group discussion. The research
findings are presented in this chapter, organized by research question and the relevant KMO
influences.
Stakeholders
Central Visayas Children’s Center’s LEAP team members were the key stakeholders and
focus of this study. CVCC’s LEAP team is composed of an Education Coordinator, English-as-
a-Second Language (ESL) Instructor, evening tutor, social workers, and house parents. Each of
these individuals dedicate varying percentages of time and fulfill varying roles in addressing the
children’s academic development, ranging from the overarching coordination of the education
program to the relatively focused task of supporting the children with daily homework
assignments. The Education Coordinator is dedicated full-time to LEAP and holds a bachelor’s
degree in education. As a component of their broader roles, social workers focus on school
placement and children’s socialization issues. The ESL instructor provides supplemental English
72
language instruction sessions on weekends, while the evening tutor provides multidisciplinary
support to the children in the evenings during the school year. House Parents provide homework
assistance during the evenings, but also develop intermittent learning activities to supplement the
children’s formal education or address identified challenges in core subjects. The House Parents’
educational attainment ranges from high school graduate to a bachelor’s degree.
Due to the relatively small sample of relevant stakeholders, the ease of their
identification, and the resultant representativeness of the sample, the study aimed to conduct
individual interviews with all member of the LEAP team. Ultimately, 13 of the 16 LEAP team
members were interviewed for this study with representation from each of the major roles
accounted for in the individual interviews. Table 6 presents the number of each category of
LEAP team member included in the study and a brief description of their education-related
responsibilities.
Table 6
Composition of LEAP Team Sample and Education-Related Responsibilities
Role Number Description of Responsibilities
Center
Director
1 Responsible for supervision of all LEAP team staff and
relationships with local DepEd and private schools.
Education
Coordinator
1 Oversees the LEAP, including coordinating LEAP programming
and activities; relationships with schools; and maintenance of
student performance and survey data.
Tutor
(Part-time)
2 Provides targeted homework support in core subjects with
emphasis on English, Mathematics, and Science. Offer ESL
courses on weekends and additional programming, as requested.
Social Worker 4 Provides holistic support to children, including counseling
services related to career guidance, academic success, and
disciplinary issues.
House Parent 5 Monitors children’s performance, identifies performance issues,
and refers children to NFE support services. Provides basic
homework support (predominantly elementary level) in evenings
73
during academic year and conducts occasional learning activities
during intersessions. Serves as a primary point of contact for
schools in regard to children’s performance and discipline.
Attends parent-teacher meetings and student events.
A cross-section of LEAP team members and local DepEd teachers participated in the
focus group interview, including the Education Coordinator, one House Parent, a Social Worker,
the evening tutor, the English-as-a-Second-Language tutor, and two DepEd teachers representing
local elementary and secondary schools. Both tutors also work as DepEd teachers at the
elementary and high school level. Participation in the individual interviews and focus group
session was voluntary.
It is important to note that this research study was conducted during LEAP’s nascent
stages and within the context of the COVID-19 pandemic, resulting in delays in LEAP’s design,
development, and delivery. Prior to the pandemic, CVCC successfully laid the foundation for
performance benchmarking by tracking and aggregating student performance data, collecting
information on student motivation, and developing its first program based on an analysis of the
resulting data. The findings reported here focus on KMO influences essential to developing
LEAP to the next implementation level, such as the conceptual and procedural knowledge
needed to use performance data to develop student achievement plans and measure academic
progress. As a result, the findings primarily focus on the needs of an evolving program and
opportunities for future growth.
Research Question 1: What are LEAP education team members’ knowledge and
motivation related to achieving this organizational goal?
Results and Findings for Knowledge Influences
74
The assumed knowledge influences for this study span conceptual, procedural, and
metacognitive knowledge needs and emphasize the LEAP team’s ability to identify student
achievement gaps and design, develop, and deliver supplemental NFE to address those gaps. In
order to achieve that goal, it is expected that LEAP team members understand the relationship
between student performance and achievement and can identify achievement gaps based on a
review of students’ performance data and CVCC’s quarterly student survey (See Appendix C).
Once identified, LEAP team members require the procedural knowledge to develop NFE
programs using appropriate learning theories and instructional approaches. Finally, LEAP team
members need to adopt a metacognitive approach to assessing the efficacy of NFE instruction
and focus on the continuous improvement of NFE strategies to promote student achievement.
The research results related to each knowledge influence appear below categorized by type of
influence and associated findings.
Knowledge Findings
LEAP team members’ knowledge of the relationship between student performance
measures and academic achievement, identification of achievement gaps, and ability to design,
develop, and deliver NFE was explored using qualitative interviews and document analysis. The
analysis revealed that, while most LEAP team members were adept at identifying performance
gaps, they lacked the ability to design, develop, and deliver supplemental NFE to effectively
address those gaps. As a result, the analysis identified two critical procedural knowledge gaps.
The first procedural knowledge gap related to the LEAP team’s ability to identify appropriate
learning strategies to address students’ performance issues. The second gap related to LEAP
team members’ ability to deploy NFE solutions in response to those needs. LEAP team
members, with several notable exceptions, did not consistently exhibit the requisite
75
metacognitive knowledge necessary to evaluate current NFE learning strategies and the selected
strategies’ efficacy in addressing student academic challenges in formal schools.
Table 7 presents an overview of the assumed knowledge findings from the study which
will be discussed in detail in the following sections. Table 8 presents a summary of the findings
from the qualitative interviews related to LEAP team members’ proficiency across the four
knowledge influences evaluated for this study.
Table 7
Validation of Conceptual, Procedural, and Metacognitive Knowledge Findings
Assumed Knowledge
Influence
Knowledge Type
(i.e., factual, conceptual,
procedural, or
metacognitive)
Validation
(i.e., Asset or Need)
LEAP team members need
knowledge of the relationship
between student performance
data and student achievement
Declarative (Conceptual) Need
LEAP team members need
knowledge of how to identify
gaps in students’ ability to
succeed in formal schools
Procedural Need
LEAP team members need
knowledge of how to apply
learning theories to promote
higher academic achievement
Procedural Need
LEAP team members need
knowledge of how to improve
NFE instructional approaches
to promote higher academic
achievement
Metacognitive Need
Table 8
Summary of LEAP Team Members Proficiencies in Knowledge Influences by Role
76
Role
(Number of LEAP
team members
serving in role)
Exhibits
knowledge of
of the
relationship
between student
performance
data and student
achievement
Exhibits
knowledge of
how to identify
gaps in students’
ability to
succeed in
formal schools
Exhibits
knowledge of
how to apply
learning
theories to
promote higher
academic
achievement
Exhibits
knowledge of
how to improve
NFE
instructional
approaches to
promote higher
academic
achievement
Administrators (2) 1 0 0 0
Tutor
(Part-time) (2)
2 2 2 1
Social Worker (4) 0 0 0 0
House Parent (5) 0 0 0 0
Conceptual Knowledge
LEAP Team Members Demonstrated Limited Conceptual Knowledge regarding the
Relationship between Performance Data and Student Achievement. LEAP team members
demonstrated the ability to gauge student performance based on a review of individual formative
and summative assignments completed during the quarter, as well as quarterly grade reports.
However, their assessment of student performance varied according to their own standard of
student achievement, which did not always align with the organization’s goal of improving
academic outcomes. As a result, LEAP team educators were able to identify performance
challenges based on their own standards of achievement, but those standards of achievement
were frequently misaligned with LEAP’s objectives of improving academic outcomes. Dowd
(2005) underscored the importance of using appropriate benchmarking strategies to measure
student learning and progress, including performance, diagnostic, or process benchmarking.
Whereas performance benchmarking emphasizes the comparison of basic metrics, diagnostic
benchmarking focuses on program evaluation and improvement (Dowd, 2005). Process
benchmarking, the most intensive and expensive approach, compares performance across
comparable organizations. As Dowd (2005) indicated, organizations should implement
77
benchmarking approaches appropriate to their resources, needs, and capabilities. CVCC
implemented performance and diagnostic benchmarking strategies in 2020, but LEAP team
members have not yet utilized the benchmark data to assess student performance or guide NFE
design and development. Of the 13 LEAP members interviewed, only one team member
indicated that they had used the student performance data to develop or modify an NFE program.
In this one instance, the part-time ESL tutor used student performance data to identify a learning
gap in English among elementary students at Lorenzo Ruiz Elementary School and modified his
instructional approach to address that challenge. The lack of clarity regarding the definition and
measurement of academic achievement has direct implications for the acquisition and application
of the knowledge needed to identify student performance issues.
The primary method for assessing student performance was the daily examination of
students’ notebooks, which contain information related to students’ homework assignments and
feedback on formative and summative assessments administered during the quarter. The
assessment of student performance entails House Parents reviewing children’s notebooks each
evening following school, noting challenges, and referring children to appropriate NFE
resources. Each House Parent is assigned to review the work of a subset of children from specific
schools. Additionally, House Parents receive and review student grade reports each quarter. A
key finding related to the House Parents’ assessment of student performance highlighted the
individual differences in the definition of acceptable levels of academic achievement. Rowena, a
House Parent who supports children from Leyte National High School (LNHS), when asked
about the academic performance of students, stated,
In school, like I have our high school students. They're all good. Some of our students,
some of the students in elementary are still developing, but our high school students
today are all good. And the elementary students only need to focus.
78
Based on document analysis of student performance data, however, the performance of students
in LNHS in core subjects are comparable to the grades of other CVCC students across all
elementary and secondary schools attended. For example, the average grade in English across all
grade levels and schools is 86.45%, while the average English grade at LNHS is 87.19%. The
elementary children at Juna Luna Charter School are actually outperforming LNHS students in
English with an average grade of 88.45% (CVCC Quarterly Student Survey, 2020). LEAP team
members encountered similar difficulty in quantifying the performance of individual students.
In general, respondents had greater awareness of performance data for high and low
performing students, but had difficulty in identifying performance issues with students whose
grades were closer to the mean. Of the five House Parents whose primary responsibility is to
monitor student performance, all of them (5) could recall instances of high- or low-performing
students. However, none (0) could quantify the overall performance of their assigned students as
an average or provide details about individual student performance outside of these high- and
low-performing categories. When asked in individual interviews to describe either a time when a
student experienced a significant challenge or success, House Parents (5) were readily able to
relate detailed stories about specific student’s experiences. However, LEAP team members (0)
could not respond to questions related to shared challenges at a specific grade level or within a
core subject and, despite the collection of quarterly grade and student survey data, did not
implement performance, diagnostic, or process-oriented benchmarking strategies to track student
performance or demonstrate program efficacy.
Of the questions asked during the individual interviews, the question that posed the most
difficulty for respondents related to defining academic success. When asked about the meaning
of “academic success,” few LEAP team educators referred to quantifiable measures. In response
to interview questions regarding student performance, four LEAP team members responded that
79
the students were “passing” their courses, emphasizing the importance of preventing a child from
falling below a performance threshold, as opposed to advancing academic achievement. LEAP
team members (6) also approached responding to this question from a more holistic perspective,
emphasizing the importance of life skills and moral standards as substantive outcomes related to
academic achievement, as opposed to formal academic measures. Due to the lack of awareness
about the availability and use of performance benchmarks, LEAP team members encountered
difficulty providing context for program efficacy beyond the anecdotal level.
While all five House Parents reported that they consistently reviewed student formative
and summative assignments and grade reports, concerns existed regarding the frequency,
consistency, and quality of those reviews. When asked about House Parents’ knowledge
regarding academic performance, Angelo, another LEAP team member, stated,
Let me answer honestly. Some of these house parents, they didn't mind the grades of the
children and even they don’t know where their cards is. They don't know the progress of
the children, especially in their monitoring reports. In our monitoring reports, one of the
important part is … the number one part … school and academic performance of this
children or the child. So, it should be written their progress. His performance, his grades,
so on, and so forth in dealing with the subjects, but there's … no … they will just say …
just the shallow report. They will just write the reports that children are going to school,
wearing the school uniforms, going to school early. […] So, sometimes we failed because
to monitor the progress of the children.
This response indicates that, while LEAP team members can identify student performance
barriers, questions remain about the frequency, accuracy, and consistency of their evaluations, as
well as the resultant alignment with LEAP objectives.
80
Marsh (2012) emphasized the importance of developing performance measures that
aligned with the technical and cultural capabilities of the organization to support those measures.
While CVCC has established some basic performance benchmarks using grade data and student
surveys, LEAP team members have not effectively integrated those benchmarks into their
evaluation of student barriers to success. As a result, LEAP team members can identify poor
performance based on subjective criteria on an assignment-by-assignment basis, but have trouble
tracking long-term performance trends for individual students, performance across grades and
core subjects, or within schools. These difficulties express themselves in limitations in LEAP
team members’ procedural knowledge of how to identify student gaps, identifying effective
learning strategies, and designing NFE that promotes students’ success in formal schools.
Procedural Knowledge
LEAP Team Members Have Limited Procedural Knowledge Related to Identifying
Gaps in Students’ Ability to Succeed in Formal Schools. LEAP team members have limited
procedural knowledge related to the use of performance benchmarks in evaluating student’s
ability to succeed in formal schools. Despite the availability of 2 years of performance data in
the form of quarterly and annual grade reports, there is minimal evidence to date to indicate that
the data has been used to identify gaps in student performance. Due to its relative simplicity and
cost-effectiveness (Dowd, 2005), performance benchmarking aligns well with CVCC’s existing
technical and cultural capabilities. However, during the individual and focus group interviews,
none of the interviewees indicated that performance benchmarks were used to identify barriers to
student achievement. However, an early initiative based on the analysis of survey data resulted in
CVCC implementing a weekend English language program to address identified gaps in English
language learning among CVCC’s elementary students. Based on this initiative, early adopters of
81
a performance-based approach to identifying student achievement gaps exist within CVCC’s
leadership team. As indicated by CVCC’s Director,
Honestly, the survey forms [sic] is really very helpful to us staff and children, most
especially […]. Actually, Angelo used it in [sic] one of his client, Jaime. We can always
go back [sic] why certain goals were not achieve in school. Which the child also gave
specific reasons why... The forms is a 2-way understanding between the staff and the
child of what we and they need to achieve. Mariel [Education Coordinator] and I are
thinking of ways how other staff (like the traditional houseparent) can use this forms [sic]
as reference to fully understand what CVCC is trying to achieve. But anyway, we will
improve along the way. (Personal communication, May 12, 2020)
However, both the Center Director and Education Coordinator indicated that the broader LEAP
team has not received training or guidance in accessing and applying the student performance
data. Of the five House Parents interviewed, none had accessed or used the performance data,
which is maintained by the Education Coordinator. As noted in the previous section on
conceptual knowledge, fundamental programmatic challenges exist in terms of LEAP team
members’ definition of academic achievement, alignment with LEAP team objectives, and the
corresponding subjectivity of LEAP team members’ assessment of student performance. In
addition to this conceptual knowledge gap, LEAP team members did not demonstrate the
procedural knowledge needed to fully access, analyze, and apply CVCC’s existing performance
benchmarks. The conceptual and procedural knowledge gaps related to evaluating student
achievement and identifying performance gaps pose a significant barrier to applying learning
theories and designing NFE programs to address barriers to student performance.
LEAP Team Members Experienced Difficulty in Applying Learning Theories to
Address Student Achievement Gaps. LEAP team members did not demonstrate the procedural
82
knowledge necessary to apply specific learning theories to address student performance
problems, even in cases where anecdotal information or hypotheses were used to project student
performance gaps. In his revision of Bloom’s Taxonomy, Krathwohl (2002) emphasized the
importance of using appropriate learning strategies to address each of the four knowledge
dimensions, including factual, conceptual, procedural, and metacognitive knowledge. Mayer
(2011) suggested that an understanding of these knowledge dimensions is valuable in informing
instructional approaches and helps ensure that learning approaches align with the type of
knowledge students need to acquire. LEAP team members, with the exception of the English
tutor, did not volunteer any specific criterion used to validate their proposed instructional
approaches. Both the House Parents (5) and Social Workers (4) reported that instructional
approaches were based on subjective decision-making, as opposed to evidence-based learning
theories. Follow-up interview questions exploring the selection of learning theories identified not
only a procedural knowledge gap of how to apply learning theory, but also a lack of conceptual
knowledge related to the learning theories themselves.
Due to the fact that LEAP team members’ understanding and identification of student
achievement barriers was limited, significant challenges existed with identifying learning
strategies that address student performance barriers. For example, when asked about the selection
of learning strategies for improving students’ academic achievement in core subjects, Mariel
indicated that students needed to “study hard” and focus on mathematics and English. However,
she could not provide context for focusing on those subjects or detail specific activities that
would improve student outcomes in those disciplines. When asked about the process of
determining supplemental instructional content for a given week during the summer break,
Justine, a House Parent, stated,
83
At the beginning of the week, the, the House Parents just have a meeting in the sala and
decide … ummm … what are they going to teach that week. Everyone just talks and
people decide what they want to do and who will do it.
Jasmine, another House Parent, indicated that House Parents also independently design
instructional sessions without consulting the Education Coordinator, referring to student
performance data, or coordinating with other learning activities. Student learning activities are
selected based on discussions between the House Parents, individual preference, or, as one
House Parent noted, by topics the children want to learn. Based on these findings, it is evident
that LEAP team members are not currently using evidence-based learning theories to inform the
design of NFE activities and may, in fact, lack the conceptual knowledge of the learning theories
necessary to do so.
LEAP Team Members Experienced Difficulty in Designing NFE Curriculum and
Instructional Approaches to Address Student Achievement Gaps. LEAP team members did
not consistently demonstrate the procedural knowledge to design NFE curriculum and
instructional approaches targeted at student performance gaps. Of those instructional programs
described during individual interviews, only the English language instruction program was
designed and developed to address identified student achievement gaps using performance
benchmarking.
A key challenge to House Parents’ identification of learning theories is inadequate
training in designing and developing NFE instructional approaches that addresses students’
individual needs, an issue that will be further discussed under organization influences. Reynaldo,
a House Parent, described his difficulty in identifying the best instructional approach to
addressing student performance gaps when he stated,
84
It's a bit difficult. Well, once in a while, one child may need something, but then the
other, they will uhh … so it's like balancing of, it's a bit difficult to balance. So, what one,
what a particular child needs, the other they would not see that. When we give out
activities, one finds it interesting, but then the others might not. So, it would be like a bit
difficult in balancing those things.
Reynaldo’s response illustrates the challenges experienced by House Parents in segregating
children based on their performance challenges and providing relevant, focused content that
meets student needs. Due to the lack of performance benchmarking and subsequent identification
of effective learning theories, House Parents indicated difficulty in developing instructional
approaches that both responded to student needs and resonated with the children. To further
illustrate this finding, a local DepEd teacher, when asked to share strategies that CVCC could
use to improve academic achievement, commented,
Yeah, I think they [the House Parents] need to have the background on teaching
strategies and helping the kids, because you can’t just simply teach them right away.
They need to … to understand the level of this student, the capability of this to them.
Before they need to … they need to know the approach they’re going to use according to
the level of the kids.
This finding is relevant to LEAP team members’ conduct of both formal, structured instructional
sessions, as well as informal tutoring sessions conducted during the evenings. As a result, the
success of the LEAP team relies on addressing the conceptual and procedural knowledge gaps
that prevent them from identifying student performance gaps and designing, developing, and
delivering NFE programs to address those gaps.
85
Metacognitive Knowledge
LEAP Team Members Need to Improve Their Metacognitive Knowledge of How to
Improve NFE Instructional Approaches to Promote Higher Academic Achievement. An
assumed motivational influence in this study reflected the ability for LEAP team members to
evaluate their own learning processes, specifically as a means of improving the design,
development, and delivery of NFE programs. Based on an analysis of interview responses, LEAP
team members need to develop cognitive strategies that enable them to be self-regulated learners
capable, as Mayer (2011) suggests, of developing learning strategies that produce targeted
outcomes and knowing when to use them. While LEAP team members (n=8) regularly referred
to “learning as we go” in discussions about improving NFE instruction at the Center, most LEAP
team could not articulate specific strategies for achieving this end. When asked about approached
to improving instructional approaches, LEAP team members focused on developing NFE
programs that were more “fun” or engaging, but failed to connect their instructional approach to
addressing student performance gaps of promoting higher academic achievement.
Results and Findings for Motivation Influences
Self-efficacy and mastery goal orientation theory were identified as assumed influences
in the design of this study. The influence of self-efficacy and mastery goal orientation on LEAP
team member’s ability in improving NFE programs was explored using qualitative interviews.
The analysis of these assumed influences revealed that, while the majority of LEAP team
members were committed to supporting the children’s learning needs, they did not believe they
were capable of developing effective NFE instructional materials and lacked a mastery goal
orientation towards improving NFE curriculum and instructional approaches. These motivational
influences have a direct impact on the LEAP team’s ability to effectively deploy of a
supplemental NFE program at CVCC.
86
Table 9 presents an overview of the assumed motivation findings from the study which
will be discussed in detail in the following sections. Table 10 presents a summary of the findings
from the qualitative interviews related to LEAP team members’ proficiency across the two
motivation influences evaluated in this study.
Table 9
Validation of Motivation Findings
Assumed Motivation
Influence
Motivation Influence Type Validation
(i.e., Asset or Need)
LEAP team members need to
believe they can develop
effective curriculum and
instructional approaches
Self-efficacy Need
LEAP team members need to
develop a mastery level goal
orientation towards NFE
provision for marginalized
children
Goal orientation Need
Table 10
Summary of LEAP Team Members Proficiencies in Motivation Influences by Role
Role
(Number of LEAP team
members serving in
role)
Exhibit belief in ability to
develop effective curriculum
and instructional approaches
Exhibit a mastery level goal
orientation towards NFE provision
for marginalized children
Center Director (1) 0 0
Education Coordinator
(1)
0 0
Tutor
(Part-time) (2)
2 1
Social Worker (4) 0 0
House Parent (5) 0 0
87
Motivation Results
LEAP Team Members Doubt Their Capacity to Develop Effective Curriculum and
Instructional Approaches. LEAP team members demonstrated a commitment to the mission of
providing children with a quality education, but lacked self-efficacy in their ability to develop
effective NFE curriculum and instructional approaches. Of the 13 LEAP team members
interviewed, only the part-time tutors indicated a belief in their ability to develop effective
curriculum and instructional approaches. According to Pajares (2006), a high level of self-
efficacy is essential in driving choice, persistence, and mental effort, the core components of
motivation. The ability to develop high-quality NFE instruction is critical to address quality
issues in DepEd’s educational provision (PIDS, 2009a) and the barriers that traditional schools
present to marginalized children (Morpheth & Creed, 2012).
Based on an analysis of qualitative interviews, LEAP team members’ lack of self-
efficacy was expressed in their difficulty in supporting older children in completing complex
assignments in the core subject matter of English, mathematics, and science. Of the five House
Parents interviewed, none indicated confidence in their abilities to support children in core
curriculum areas. A second factor was the impact of the children’s perception of LEAP team
members’ abilities as teachers. As a result, self-efficacy challenges exhibited themselves in terms
of both subject matter content and instructional competencies. To compound this issue, LEAP
team educators did not demonstrate strategies for addressing these challenges or explore
possibilities related to improving their subject matter or instructional expertise.
LEAP team members did not believe they are capable of supporting students in specific
subject matter areas, particularly at the secondary level. When asked about their ability to
support student achievement in mastering core subjects in formal school, the most common
answer from social workers (3) and house parents (4) was “we do our best.” This challenge
88
primarily exhibited itself in discussions about the House Parents’ role in supporting children in
completing homework assignments. CVCC provides tutoring support in the form of a part-time,
secondary level DepEd teacher and three House Parents on school nights. While House Parents
(4) indicated a strong belief in their ability to support elementary age children across all topics,
all of the House Parents expressed reservations about their ability to support secondary level
students, particularly in the topics of English, mathematics, and science.
Children experiencing challenges in more complex topics were referred to the evening
DepEd tutor, resulting in a backlog of requests and the need for the tutor to limit support sessions
to a maximum of 15 minutes per child. In contrast to House Parents, the evening tutor
demonstrated a high level of self-efficacy in terms of supporting the children, particularly in
mathematics and science. LEAP team members differed in their assessment of the ability of the
evening tutor to handle the demands of children for support, but seven respondents noted that the
evening tutor invested a significant portion of his time tutoring the children in mathematics at the
expense of other core subjects. Despite these demand on the tutor’s time, four House Parents
indicated that they consistently referred students to the tutor as opposed to focusing on
developing their own ability to support the secondary students in these subjects. Based in these
findings, a motivational gap exists related to LEAP team members’ level of self-efficacy in
supporting students in core subject matter content, including English, mathematics, and science
at the secondary level. This motivational gap influences LEAP team members’ sense of self-
efficacy not only in terms of supporting children in these subjects, but also in developing the
subject matter and instructional expertise required to support secondary level students in these
subjects.
A key challenge to LEAP team members’ sense of self-efficacy is the children’s
perception of the teaching abilities of LEAP team members. Except for the Education
89
Coordinator, LEAP team members’ responsibilities span an array of tasks ranging from the
provision of medical care to emotional support. Excepting the two part-time tutors, no LEAP
team members have taught full-time in formal schools. When asked about the amount of time
spent working on educational tasks in a given day during the school year, House Parents (4)
indicated that their commitment to that role consisted of the 2 hours they assisted children with
homework assignments during evening shifts. Of the nine LEAP team members who were not
tutors, but provided instructional support, seven indicated that children had trouble viewing them
as educators, which influenced their belief in their own abilities. These same LEAP team
educators indicated that their lack of an educational background, knowledge regarding
instructional approaches, and formal authority as an educator contributed to their low self-
efficacy. Highlighting one difference between the children’s perception of LEAP team educators
versus DepEd teachers, CVCC’s Director commented,
If you're in a, in a, what's it called this one? You're an athlete or a choir member of the
schools, you have certain criteria or you have certain demands or you have to set rules
and regulations, that you have to be at least on 85% upgrade or, or you have this, you can
only have this scholarship if you maintain this one. We don't have. Their [the children’s]
stay here in [CVCC] is like a home with a lot of mouth of the House Parent who tells
them, do this, do that. But when it comes to the teachers. Okay. Behave or misbehave,
you're just on the tip of the ball pen.
In addition to lacking the extrinsic incentives available to DepEd teachers, four of the five House
Parents indicated that children view them as parental figures and, as a result, have difficulty
viewing them seriously as teachers. Three social workers shared the same experience and, of
those, two indicated that children had explicitly referenced the distinction between house parents
and teachers and CVCC as a home as opposed to a school. Six LEAP team educators responded
90
to the children’s perception by shifting focus away from instruction related to core academic
subjects and focusing on fun activities with academic content relegated to a peripheral objective.
In contrast, staff indicated that the weekend English language instructor, a DepEd teacher from a
local elementary school, delivered highly engaging instructional activities that focused on
developing competency in conversational English. LEAP team members could differentiate
between the quality and relevance of the English language tutor’s instruction and their own, but
their key takeaway (4) was that the students respected the teacher who they considered more
adept at instructing the children. LEAP team members did not indicate a belief in their ability to
develop the requisite instructional skills to replicate these sessions, even when prompted by
follow-up questions prompting them to describe strategies that would allow them to strengthen
their instructional capabilities. Bandura (1997) underscored the importance of self-efficacy as a
foundational component of choice, persistence, and mental effort. As a result, LEAP team
members’ lack of self-efficacy is a core challenge in implementing of effective NFE initiatives at
CVCC.
LEAP Team Members Lack a Mastery Level Goal Orientation Towards NFE
Provision for Marginalized Children. LEAP team members lack a mastery level goal
orientation towards NFE provision for marginalized children. Of those ten LEAP team members
who either oversee or actively develop NFE programs and instructional content, only one part-
time tutor described a reflective process that leveraged performance data and evidence-based
outcomes to evaluate program efficacy. As reflected in the previous discussions related to self-
efficacy and metacognition, LEAP team members indicated that, while they would “do their
best” to support children in improving performance in formal schools, they experienced
difficulties in connecting appropriate instructional approaches to identified performance gaps and
designing NFE initiatives to address those needs. When asked about a time when they modified
91
an instructional approach based on their experiences, six out of nine LEAP team members were
unable to identify an instance where they made a modification targeted at improving academic
outcomes, although they adjusted NFE activities to make them more engaging and enjoyable for
the children. The remaining three interviewees, when asked to describe their process for
evaluating instructional efficacy, were unable to connect it to quantifiable outcomes or describe
the ways in which instructional changes addressed students’ academic achievement gaps. As a
result, their responses indicated an effort to reflect on their past performance, but did not make
explicit connections between changes in instructional approaches and student outcomes.
Research Question 2: What is the interaction between organizational culture and context
and LEAP team educator knowledge and motivation?
Results and Findings for Organizational Influences
In addition to knowledge and motivation influences, this research study evaluated four
assumed organizational influences in relation to LEAP team members’ knowledge and
motivation. These assumed organizational influences were evaluated through the lens of
Gallimore and Goldenberg’s (2001) cultural models and cultural settings. The analysis of these
organizational influences was conducted through qualitative individual and focus group
interviews and the findings, presented by assumed influence, are presented below.
Table 11 presents an overview of the assumed organization findings from the study
which will be discussed in detail in the following sections. Table 12 presents a summary of the
findings from the qualitative interviews related to CVCC’s proficiency across the three cultural
models and one cultural setting evaluated in this study.
Table 11
Validation of Organization Findings
92
Assumed Organization
Influence
Influence Type Validation
(i.e., Asset or Need)
CVCC needs to align its key
stakeholder and
organizational performance
goals.
Cultural Model Need
CVCC needs to prioritize and
foster collaboration between
internal and external
stakeholders.
Cultural Model Need
CVCC needs to optimize
program efficacy and
accountability.
Cultural Model Need
CVCC needs to provide
LEAP team members
dedicated time to develop and
collaborate on curriculum and
instructional design.
Cultural Setting Need
Table 12
Summary of CVCC’s Performance in Addressing Organization Influences
Organization Influence Summary of Findings
Alignment between key stakeholder and
organizational performance goals
LEAP team members’ perception of
stakeholder goals are not clearly aligned with
organizational goals, specifically as they relate
to academic achievement, performance
benchmarking, and evidence-based outcomes.
Collaboration between internal and external
stakeholders
Collaboration between LEAP team members
and DepEd focuses on responding to
unsatisfactory student performance, as opposed
to proactive efforts to improve academic
achievement through NFE programs and
instruction.
Optimization of program efficacy and
accountability
Program efficacy and accountability is sub-
optimal due to stakeholder challenges in
connecting student outcomes to NFE
programs.
93
Provision of dedicated time for LEAP team
members to develop and collaborate on
curriculum and instructional design.
LEAP team members’ time is primarily
committed to responding to immediate student
needs, as opposed to developing and
collaborating on curriculum and instructional
design.
Organizational Results
CVCC Needs to Align Its Key Stakeholder and Organizational Performance Goals.
A critical role of leadership is establishing clear goals and objectives (Kotter, 2012). Yukl (1981)
underscored the importance of leaders clarifying roles and objectives, emphasizing the necessity
of establishing performance goals and approving plans for achieving them. CVCC established a
goal for the LEAP team of improving students’ academic performance by 5% on average by
June 2021. However, findings from the qualitative interviews indicated that LEAP team
members’ standards of performance frequently did not align with their established stakeholder
goal. In responses to questions related to targeted performance outcomes, LEAP team members
(13) did not reference specific, measurable performance objectives. Instead, respondents
provided responses that reflected a different standard (e.g., “passing”) or generalized goals, such
as “developing the whole person” that aligned poorly with measurable results.
CVCC Needs to Prioritize Collaboration Between Internal and External
Stakeholders. In order to achieve successful organizational transformations, organizations need
to align the critical behaviors of key stakeholders to drive targeted organizational outcomes
(Kirkpatrick & Kirkpatrick, 2016; Waterman et al., 1980). For the purposes of this research
study, an assumed organizational influence was collaboration between internal and external
stakeholders in implementing effective NFE program initiatives. Based on the findings from
individual interviews and the focus group session, LEAP team members interactions with local
DepEd schools are predominantly reactive. Each House Parent is tasked with serving as a liaison
94
for a defined set of children attending a specific school. While all five of the House Parents
interviewed indicated that they attend orientation sessions, parent-teacher meetings, award
ceremonies, and disciplinary meetings, none of them indicated that they regularly meet with
local DepEd personnel to discuss strategies for designing or improving supplement NFE
provision. Of the five House Parents interviewed, only one indicated that they proactively
reached out to a local teacher to discuss the children’s performance and behavioral issues in
school, primarily as a result of a longstanding association with a DepEd teacher. While the
inclusion of two DepEd teachers as tutors on the LEAP team addressed some of this need,
neither of the tutors taught at schools attended by CVCC’s children. During the focus group
session, the local DepEd teachers reinforced the fact that the interactions between CVCC and
DepEd staff had been limited, but that potential exists to expand those interactions in the future
to include substantive discussions regarding effective approaches to non-formal education. The
Education Coordinator did not schedule regular meetings with DepEd personnel to identify or
address higher-level student performance issues, discuss available DepEd resources for CVCC
children, coordinate NFE programming with DepEd curriculum, or promote deeper interactions
between CVCC and DepEd educational personnel.
CVCC Needs to Optimize Program Efficiency and Accountability. Non-formal
education programs need to demonstrate their efficacy through the establishment of performance
benchmarking and measurable outcomes (Shephard, 2014). Document analysis of CVCC’s
descriptive statistics on student performance and aggregated results of student survey data
indicated CVCC’s increased interest in performance benchmarking. CVCC has also produced
summary reports of these data sources describing the children’s performance in English,
mathematics, and science. However, interviews with LEAP team members revealed that use of
performance data to inform NFE strategies is limited. Of the 13 LEAP team members
95
interviewed, 11 had never used the performance data to inform program or instructional design
and seven noted that they had either not seen or did not have access to the data. Since 2019, only
the weekend ESL program has been created based on performance data and it is too early to
evaluate the program’s efficacy.
CVCC Needs to Provide LEAP Team Members Dedicated Time to Develop and
Collaborate on NFE Curriculum and Instructional Design. Educators need to devote time to
reflecting on their teaching practices, assess the efficacy of their instructional approaches, and
adopt evidence-based teaching strategies that deliver results (Clark & Estes, 2008; Sargent &
Hannum, 2009; Vescio et al., 2008). Based on the study’s finding, LEAP team members are not
provided with dedicated time to design instructional approaches or collaborate with other
educators to build their competency in NFE provision. LEAP team members (11) indicated that,
on average, less than 6 hours per week are dedicated by House Parents to supporting the
academic achievement of over 70 children. According to each of the House Parents interviewed,
this time is spent almost exclusively on tutoring students in the evenings. Three of the five House
Parents interviewed indicated that no dedicated time was provided for developing proactive
instructional sessions and the remaining two indicated that minimal time was allowed for the
preparation of supplemental educational activities. In addition, based on both the individual and
focus group interviews, none of the LEAP team educators interacted with local DepEd teachers
to share teaching strategies or align NFE activities with topics being covered in the children’s
classrooms.
Summary
Central Visayas Children Center’s LEAP is in a nascent stage of development. Since its
inception in 2019, CVCC has implemented performance benchmarking centered on descriptive
and summary statistics derived from grade reports and quarterly student surveys. However,
96
CVCC has yet to educate LEAP team members regarding the availability, access, and application
of these data to evaluating students’ academic achievement. The lack of this information has
significant implications for procedural knowledge in applying learning theories to the
development of NFE to address identified challenges and developing NFE programs, as well as
metacognitive strategies for evaluating and improving program efficacy. LEAP team members
also demonstrated a lack of perceived self-efficacy in developing NFE programs to support
student achievement, primarily stemming from challenges related to their level of subject matter
and instructional expertise. Based on the limited findings from individual interviews, LEAP team
members have not adopted a mastery level goal approach to improving NFE programs. However,
this finding should be qualified due to the underlying conceptual and procedural knowledge gaps
identified in the LEAP team’s efforts to design, develop, and delivery of NFE programs.
Each of the assumed organizational influences were validated. CVCC needs to address
the misalignment between its espoused and actual stakeholder goals to ensure that LEAP team
members understand their objectives and CVCC’s expectations for meeting them. Organizational
gaps also exist between internal and external stakeholders, primarily as expressed in the lack of
collaboration between LEAP team members and DepEd educators in designing and developing
effective NFE programs that align with formal school learning objectives. Additionally, CVCC
does not provide LEAP team members dedicated time to devote to enhancing their abilities to
develop NFE programs. Finally, and perhaps most significantly, the LEAP team has not
demonstrated results, an outcome that will be addressed in the recommendations presented in
Chapter Five.
97
Chapter Five: Solutions and Recommendations
This study evaluated the Central Visayas Children Center’s (CVCC) Learning and
Educational Assistance Program (LEAP) using Clark and Estes (2008) gap analysis framework.
Document analysis, individual interviews, and a focus group were used to validate assumed
knowledge, motivation, and organization (KMO) influences of CVCC’s LEAP team, the primary
stakeholders in implementing supplemental NFE programs to improve CVCC students’
academic outcomes. In this chapter, recommendations will be provided to address the validated
KMO challenges identified in Chapter Four. This chapter will present the recommended
solutions and evaluation approaches organized by each validated KMO influence using
Kirkpatrick and Kirkpatrick’s (2016) New World Model.
Introduction and Overview
In order to achieve the primary stakeholder goal of improving the academic performance
of marginalized Filipino children in formal schools through NFE provision, CVCC will need to
address the critical knowledge, motivation, and organizational gaps serving as barriers to the
LEAP team’s implementation of the program. Based on document analysis and interviews with
key stakeholders, addressing these challenges will require CVCC to provide LEAP team
members with the requisite knowledge to identify student performance challenges, as well as
design, develop, and deliver a curriculum and associated instructional approaches capable of
assisting student in overcoming those challenges. Beyond these knowledge influences, the
success of the NFE initiative will be reliant on the exhibition of higher levels of self-efficacy and
a mastery goal orientation toward NFE development. Finally, CVCC will need to provide the
organizational context for success, including improved alignment between organizational and
stakeholder goals, increased collaboration with external stakeholders, enhanced program
accountability, and the allocation of adequate resources to the LEAP.
98
The recommendations for addressing these knowledge, motivational, and organizational
influences include conducting an initial training session targeted at addressing LEAP team
members’ knowledge, skills, abilities, commitment, and confidence in identifying and
responding to identified academic performance gaps through NFE provision. Additionally,
CVCC will institute a professional learning community to provide LEAP team educators
ongoing opportunities to increase their level of competency and confidence in NFE
implementation. These quarterly sessions will focus on the development of critical skill sets
needed to implement effective NFE programs, afford LEAP team members an opportunity to
engage external DepEd and DSWD stakeholders on issues of education, and assess the LEAP’s
efficacy based on established benchmarking and accountability measures.
An integral component of the program’s implementation is an in-depth, iterative, and
ongoing evaluation of the program’s efficacy as defined by the Kirkpatrick New World Model
(Kirkpatrick & Kirkpatrick, 2016). Using blended and delayed evaluation methods, CVCC will
assess program efficacy at each of the four evaluation levels identified in the Kirkpatrick New
World Model, including reaction, learning, behaviors, and results. Core to the evaluation process
is the attainment of Level 4 results that demonstrate the program’s return on expectations
(Kirkpatrick & Kirkpatrick, 2016). CVCC will focus on assessing the efficacy of the
recommended performance solutions in closing the knowledge, motivation, and organization
gaps serving as barriers to the LEAP team’s success.
Recommendations for Practice to Address KMO Influences
Knowledge Recommendations
Introduction
The assumed knowledge influences in Table 13 reflect those knowledge influences were
identified based on a review of the extant literature and informal discussions with, and
99
observations of, Learning and Educational Assistance Program (LEAP) team members during
site visits to the Philippines in 2018 and 2019. Additionally, the recommendations reflect
approaches prioritized to make the most significant contributions in assisting stakeholders to
achieve their goals. The knowledge influences were subsequently validated using document
analysis and the collection of interview and focus session data, as detailed in Chapter Three:
Methods. The assumed knowledge influences in Table 13 are identified using Krathwohl’s
(2002) framework for categorizing knowledge-related performance issues, including declarative,
procedural, and metacognitive knowledge. Additionally, the table connects the knowledge
influences to relevant principles related to knowledge acquisition derived from evidence-based
research, as well as providing context specific recommendations targeted at reducing knowledge
gap and driving performance outcomes. These recommendations are based on Clark and Estes’
(2008) framework for reducing knowledge gaps and improving job performance using
information, job aids, training, and education.
Table 13
Summary of Knowledge Influences and Recommendations
Assumed
Knowledge
Influence*
Validated
as a Gap?
Yes, High
Probability
or No
(V, HP, N)
Priority
Yes, No
(Y, N)
Principle and
Citation
Context-Specific
Recommendation
LEAP team
members need
knowledge of
the relationship
between student
performance
data and student
achievement
(D)
V Y How individuals
organize
knowledge
influences how
they learn and
apply what they
know (Schraw &
McCrudden,
2006).
Provide a job aid that lists
available student
achievement measures and
how to access them.
Additionally, the job aid
provides prior examples of
effective strategies used to
apply student achievement
data/information to measure
student performance. This
job aid will be used as a
100
Information
learned
meaningfully and
connected
with prior
knowledge is
stored
more quickly and
remembered
more accurately
because it is
elaborated with
prior learning
(Schraw &
McCrudden,
2006).
component of the training
detailed below.
LEAP team
members need
knowledge of
how to identify
gaps in
students’ ability
to succeed in
formal schools
(P)
V Y Modeling to-be-
learned strategies
or behaviors
improves,
learning, and
performance
(Denler, Wolters,
& Benzon, 2009).
Effective
observational
learning is
achieved by first
organizing and
rehearsing
modeled
behaviors, then
enacting them
overtly (Mayer,
2011).
Modeled
behavior is more
likely to be
adopted if the
model is credible,
similar (e.g.,
gender, culturally
appropriate), and
the
Provide training that utilizes
the analysis of student
achievement data and the
observation of student
achievement assessment
sessions to demonstrate
effective practice and
procedures to identify gaps
in student performance.
Local DepEd teachers and
the LEAP Education
Coordinator will
demonstrate the process,
verbalizing the process and
the rationale used for
identifying gaps. This
process will be followed by
DepEd teacher and LEC
observation, feedback, and
recommendations for
improvement. The job aid
would also be used to
support the acquisition of
procedural knowledge in
training sessions.
101
behavior has
functional value
(Denler, Wolters,
& Benzon, 2009).
LEAP team
members need
knowledge of
how to apply
learning theories
to promote
higher academic
achievement (P)
V Y Modeling to-be-
learned strategies
or behaviors
improves,
learning,
and performance
(Denler, Wolters,
&
Benzon, 2009).
Effective
observational
learning is
achieved by first
organizing and
rehearsing
modeled
behaviors, then
enacting them
overtly (Mayer,
2011)
Modeled
behavior is more
likely to be
adopted if the
model is credible,
similar (e.g.,
gender, culturally
appropriate), and
the
behavior has
functional value
(Denler, Wolters,
& Benzon, 2009).
Provide training that allows
LEAP team members to
observe local DepEd
teachers and the LEAP
Education Coordinator
(LEC) apply learning theory
to develop a Student
Achievement Plan (SAP),
followed by providing
opportunities for the team
member to practice
designing their own SAP
with feedback from the
DepEd teachers and LEC.
Each objective on the
student achievement plan
will indicate how learning
theory connects to the
designated objectives and
promotes student
achievement.
LEAP team
members need
knowledge of
how to improve
NFE
instructional
V Y Learning and
motivation are
enhanced when
learners set
goals, monitor
their performance
Provide training that allows
learners to think aloud as
they work through an
existing instructional
approach designed to
improve student
102
approaches to
promote higher
academic
achievement (M)
and evaluate their
progress
towards
achieving their
goals.
(Meyer, 2011)
The use of
metacognitive
strategies
facilitates
learning
(Baker, 2006).
performance (e.g., a lesson
plan), while evaluating how
effective the approach is in
achieving targeted
educational outcomes.
* (D)=declarative knowledge; (C) = conceptual knowledge; (M) = metacognitive knowledge
Increasing LEAP team member knowledge of approaches to identifying and
diagnosing gaps in student performance. LEAP Team members need knowledge of
approaches to using student achievement data to identify gaps in student performance. This
finding was validated through the conduct of individual and focus group interview sessions and
document analysis. Information processing theory (IPT), which describes the cognitive processes
used by humans to integrate new information into existing mental schema, provides a basis for
addressing this challenge. Schraw and McCrudden (2006) underscored the importance of
connecting new knowledge to existing mental schemas in order to facilitate learning, the transfer
of knowledge, and its application. In order to master the procedural knowledge necessary to
address academic achievement gaps, LEAP team members must first master conceptual
knowledge of the performance metrics used to identify and measure achievement gaps. Based on
this information, the proposed recommendation entails creating a job aid that provides a list of
performance indicators available to LEAP team members, details the process for accessing them,
and supplies examples of how they are applied to diagnosing student achievement gaps. This job
aid will also support training initiatives related to the acquisition of procedural knowledge, as
referenced later in this section.
103
Mayer (2011) suggests that information processing theory provides a framework for
understanding how learners select, organize, and integrate new information for later retrieval and
application. In their seminal work on human memory, Atkinson and Shiffrin (1968) emphasized
the importance of “control processes” in managing the classification, storage, and retrieval of
knowledge. According to the authors, learners must first prioritize new information as relevant
before “rehearsing” with the data to organize it and subsequently connect it to existing mental
schema (Atkinson & Shiffrin, 1968). This process, which Atkinson and Shiffrin (1968) referred
to as “encoding,” enables learners to regulate the acquisition of new knowledge by forcing the
prioritization of incoming information within sensory and short-term memory prior to
committing it to long-term memory. According to Clark and Estes (2008), the commitment of
conceptual knowledge to mental schema associated with long-term memory will improve the
automaticity of its retrieval and probability of its application. Job aids are self-help resources that
provide employees with essential information for a specific job-related task (Clark & Estes,
2008). Using a job aid will facilitate the acquisition of the conceptual knowledge related to
measurements of academic achievement, resulting in a higher likelihood of retention and
increased likelihood of its effective application in designing instructional interventions (Schraw
& McCrudden, 2006). Frederiksen et al. (2011) indicated that the use of “causal maps” that
illustrate the relationship between key variables in an interconnected network substantially
improved the ability of decision-makers to process and apply new knowledge in complex,
dynamic environments. Reducing LEAP team member’s conceptual knowledge gap related to
the availability, access, and application of performance measures related to academic
achievement is foundational to addressing procedural knowledge gaps related to identifying and
addressing academic performance issues.
104
Increasing LEAP team member knowledge of strategies to promote higher academic
achievement among marginalized Filipino children in formal schools. Based on qualitative
interviews and document analysis, LEAP team members need procedural knowledge of how to
apply learning theories to promote higher academic achievement among students. Eleven out of
13 LEAP team members lack the procedural skills necessary to identify student achievement
gaps and develop instructional approaches to respond to those gaps. Denler et al. (2009) stated
that learners benefit from first observing and subsequently practicing modeled behaviors. This
learning approach, which is grounded in social cognitive theory, is particularly effective when
the behaviors are viewed as useful and modeled by individuals that learners find relatable and
credible (Denler et al., 2009). The principles highlighted in Denler et al.’s (2009) work
underscore the importance of providing LEAP team members the opportunity to observe and
practice modeled behaviors. Based on these principles, the proposed recommendation involves
conducting training that allows LEAP team members to observe local DepEd teachers and
CVCC’s LEAP Education Coordinator (LEC) develop a student achievement plan (SAP) and
subsequently practice developing their own SAP with feedback from both the DepEd teachers
and LEC. The trainers will review each objective on the student achievement plan and assess the
efficacy of each learning strategy in achieving targeted student achievement outcomes.
Bandura (2005) underscored the importance of social modeling as a central component in
learning. Social modeling involves experts demonstrating desired behaviors, which learners can
then emulate and refine through corrective feedback (Bandura, 2005). Bandura (1977) indicated
that direct observation of modelled behaviors reduces errors, assists with the construction of
knowledge, and accelerates learning (Bandura, 1977). Clark and Estes (2008) suggest that such
training must be intimately tied to specific performance outcomes and result in the acquisition of
procedural knowledge that is automated and unconscious. Based on these findings, LEAP team
105
members would benefit from observing the LEC and local DepEd teachers identify student
performance gaps and design instructional interventions to address those gaps. LEAP team
members would subsequently practice that behavior and receive corrective feedback with the
goal of reinforcing their understanding of the procedural knowledge required to design
instructional approaches that improve student performance.
Increasing LEAP team members’ self-regulation to improve NFE instructional
strategies. An analysis of interviewees responses in qualitative interviews and review of CVCC
student surveys and performance data indicates that LEAP team members need knowledge of
how to improve NFE instructional approaches to promote higher academic achievement.
According to Mayer (2011), an effective approach to enhance learning involves learners
establishing their own goals, reviewing their progress, and assessing their progress against the
established goals. Based on Mayer’s principles, LEAP team members would benefit from the use
of metacognitive approaches in establishing and evaluating goals to improve their instructional
approaches targeted at promoting higher academic achievement among students. In order to
realize this outcome, the recommended strategy entails the use of training that allows learners to
think aloud as they work through an existing instructional approach designed to improve student
performance (e.g., a lesson plan) while evaluating how effective the approach is in achieving
targeted educational outcomes. Using this approach, a LEAP team member could develop deeper
knowledge of the ways in which specific instructional approaches support learning and promote
improved academic achievement.
In his seminal work on metacognition and cognitive monitoring, Flavell (1979)
highlighted the importance of metacognitive experiences in establishing new goals, as well
as activating goal-oriented cognitive and metacognitive strategies to achieve those goals.
According to Flavell (1979), metacognition entails the conscious or unconscious monitoring of
106
one’s own cognitive processes. Mayer (2011) suggests that metacognition is composed of
metacognitive awareness and metacognitive control, or knowing how to learn and how to
monitor and control one’s learning, respectively. Metacognition is critical to a learner’s cognitive
processing and the self-regulated behaviors that promote active learning (Mayer, 2011). As a
result, any training initiative targeting the acquisition of procedural knowledge would benefit
from the use of metacognitive experiences. Mastery of complex procedural knowledge requires
the use of metacognitive strategies and corrective feedback (Bandura, 2005). In fact, Bandura
(2005) indicated that the combination of declarative and procedural knowledge, in the absence of
metacognitive knowledge, is insufficient for solving complex problems. Based on these findings,
training that encourages LEAP team members to articulate their thought processes during
development of a Student Achievement Plan (SAP), accompanied by feedback from local DepEd
teachers and the LEC, would encourage self-regulatory behaviors that promote improvements in
instructional approaches to addressing student performance issues.
Motivation Recommendations
Introduction
Motivation, according to Pintrich and Schunk (2002), is the initiation and sustenance of
goal-oriented activities requiring the investment of significant mental effort to address complex
learning challenges. Clark and Estes (2008) refer to these three characteristics of motivation as
choice, persistence, and mental effort. Research on motivation has focused on a variety of
motivational dimensions, such as the roles of attribution, self-efficacy, value, interest, emotion,
goal-orientation, and expectancy theory in attaining learning goals. However, for the purposes of
this study, self-efficacy and goal orientation were selected to address the perceived motivational
challenges of CVCC’s primary stakeholders in promoting higher academic achievement for
marginalized Filipino children. Banduras (1997) indicated that self-efficacy, defined as the
107
beliefs that individuals hold about their own ability to complete a given task, is central to human
motivation and learning. Mastery goal-orientation, according to Yough and Anderman (2006),
emphasizes self-improvement based on comparisons with previous efforts. Both principles
directly correlate with motivational challenges experienced by CVCC’s primary stakeholders.
Based on an analysis of the data collected in this study, LEAP team members need to enhance
their belief that they can develop an effective curriculum and associated instructional
approaches, as well as develop a mastery level goal orientation towards NFE provision for
marginalized children. Table 14 details recommendations to address the motivation influences
and promote LEAP team members’ self-efficacy and mastery goal-oriented behaviors.
Table 14
Summary of Motivation Influences and Recommendations
Assumed
Motivation
Influence*
Validated
as a Gap
Yes, High
Probability,
No
(V, HP, N)
Priority
Yes, No
(Y, N)
Principle and
Citation
Context-Specific
Recommendation
LEAP team
members need
to believe they
can develop
effective
curriculum and
instructional
approaches.
(SE)
V Y Feedback and
modeling
increase self-
efficacy (Pajares,
2006).
Mastery
experience is the
most significant
contributor to
individual self-
efficacy (Pajares,
2006).
Self-efficacy is
enhanced when
goal-directed
practice is
Provide LEAP team
members multiple
opportunities to practice
designing curriculum and
associated instructional
approaches using a
scaffolded approach,
gradually removing
supports as learners
demonstrate increased
levels of proficiency. Local
DepEd teachers and the
LEC will provide accurate,
credible, targeted,
supportive, and private
progress feedback to LEAP
team members.
108
coupled with
frequent,
accurate,
credible,
targeted and
private feedback
on progress in
learning and
performance
(Pajares, 2006).
LEAP team
members need
to develop a
mastery level
goal orientation
towards NFE
provision for
marginalized
children. (GO)
V Y Focusing on
mastery,
individual
improvement,
learning, and
progress
promotes positive
motivation
(Yough &
Anderman,
2006).
Designing
learning tasks that
are novel, varied,
diverse,
interesting, and
reasonably
challenging
promotes mastery
orientation
(Yough &
Anderman,
2006).
Create a LEAP team
community of learners that
discusses best practices for
developing curriculum and
associated instructional
approaches and encourages
a collaborative, mutually
supportive learning
environment that promotes
individual improvement.
LEAP team members need to believe they can develop effective curriculum and
instructional approaches. Based on a review of data from this study, LEAP team members lack
adequate self-efficacy to develop NFE approaches to promote the academic achievement of
marginalized Filipino children in formal schools. Self-efficacy theory, which defines the
relationship between an individual’s beliefs and their willingness to act or persist at a task,
109
provides a basis for addressing this challenge. Pajares (2006) indicated that modeling and
feedback increase self-efficacy. Mastery experience, according to Pajares (2006), is the most
significant contributor to individual self-efficacy. These findings underscore the importance of
providing LEAP team members ample opportunity to practice and refine curriculum and
instructional design and development tasks. Based on this information, the proposed
recommendation involves providing LEAP team members with opportunities to practice
designing curriculum and associated instructional approaches to promote mastery experience. As
part of this process, the LEC will provide credible, targeted, supportive, and private feedback to
LEAP team members related to their progress in these areas. Examples include the design and
development of an instructional session under the guidance and with feedback from local DepEd
teachers and the LEC.
In his seminal work on social learning theory, Bandura (1997) highlighted the importance
of improving task-specific confidence as a means of increasing learners’ willingness to act,
persist, and exert effort on a given task. Rueda (2011) suggests that a learner’s degree of self-
efficacy is influenced by the success or failure of prior attempts to complete the task, as well as
the nature of the task-related feedback received. Bandura (2012) also distinguished between
learning efficacy and self-regulatory efficacy, noting that individuals must not only believe that
they can learn targeted content, but also manage the “... social, motivational, and affective
aspects of learning” (p. 26). In order to increase self-efficacy, Bandura (1997) indicated the
importance of fostering competence and confidence in instructional activities. Schunk and
Hanson (1985) indicated that individuals developed greater self-efficacy by observing peers
complete targeted tasks then watching content experts using the same strategies to complete
identical tasks. Individuals with higher levels of self-efficacy are more likely to act, persist, and
perform better when faced with challenging tasks (Pintrich & Schunk, 2002). Accordingly,
110
LEAP team members would likely benefit from practice sessions that allow them to
incrementally master tasks undertaken with local DepEd teachers’ and the LEC’s constructive
guidance and support. With scaffolded practice sessions and substantive support and guidance,
LEAP team members will acquire the confidence necessary to undertake the design and
development of curriculum and instructional approaches that support marginalized Filipino
children’s success in formal schools.
LEAP team members need to develop a mastery level goal orientation towards NFE
provision for marginalized Filipino children. LEAP team members, based on data derived
from qualitative interviews and document analysis, need to develop a mastery level goal towards
providing non-formal education (NFE) to marginalize Filipino children to promote success in
formal schools. Goal orientation theory, which emphasizes the importance of a learners’ reasons
for engaging in a task, provides a basis for evaluating and formulating recommendations in
response to LEAP team members’ design and development of NFE initiative in support of
marginalized Filipino children. According to Yough and Anderman (2006), a mastery goal
orientation, or one that emphasizes learning focused on mastering a specific task, promotes
positive motivation. In order to promote mastery goal orientation, learning tasks should be
diverse and relatively challenging (Yough & Anderman, 2006). Based on these principles,
CVCC should create a LEAP team community of learners that discusses best practices for
developing curriculum and associated instructional approaches and encourages a collaborative,
mutually supportive learning environment that promotes individual improvement. Examples of
the activities engaged in by the community of learners include monthly roundtables to discuss
current challenges, collaborative problem-solving of key challenges, and review of case studies
demonstrating successful strategies for overcoming student performance issues and promoting
academic achievement.
111
Promoting a mastery goal orientation entails focusing learners on their level of effort and
progress against individual self-improvement standards (Pintrich, 2003). According to Grant and
Dweck (2003), individuals with established learning goals are more likely to choose, persist, and
exert mental effort when faced with difficult learning challenges. In order to foster individuals’
adoption of a mastery goal orientation, Pintrich (2003) emphasized the importance of providing
safe learning environments that encourage learners to focus on individual improvement, as
opposed to performance or social comparisons to others. One approach to achieving this goal is
through the creation of a community of mutually supportive learners working collaboratively on
a diverse, challenging set of learning tasks (Yough & Anderman,2006). For example, Sargent
and Hannum (2009) found that teachers who participated in professional learning communities,
even in schools that suffered from severe resource shortages, developed a greater degree of
individual mastery in teaching. Mayer (2011) also suggests that individuals work harder to learn
when they believe that they are collaborating with others on a shared challenge. Based on these
findings, LEAP team members would benefit from participating in a community of learners
focused on the shared challenge of improving NFE initiatives to improve marginalized Filipino
children’s academic achievement in formal schools. The provision of a low-risk, safe learning
environment characterized by collaborative effort, social support, diverse learning experiences, a
focus on self-improvement, and private, credible, and targeted feedback would foster a mastery
goal orientation among LEAP team members towards developing high-quality NFE initiatives
that serve the needs of marginalized Filipino children.
Organization Recommendations
Introduction
The validated organizational influences reflected in Table 15 reflect key organizational
challenges faced by CVCC through the lens of Gallimore and Goldenberg’s (2001) cultural
112
models and cultural settings framework. Cultural models, according to the authors, define how
the world works within an organization, while cultural settings describe the interactions between
individuals within a given organizational setting (Gallimore & Goldenberg, 2001). The use of the
cultural models and settings framework supports the identification, interpretation, and analysis of
organizational behaviors (Gallimore & Goldenberg, 2001). According to Clark and Estes (2008),
an essential element in achieving organizational goals is ensuring that employees have the
appropriate knowledge, skills, and motivation necessary to perform in their roles, but also access
to adequate material resources and the advantages inherent in integrated organizational
processes. Based on the results of qualitative interviews and document analysis, it is evident that
CVCC’s cultural models and settings inhibit the LEAP team’s ability to successfully develop
effective NFE curriculum and associated instructional approaches. In particular, LEAP team
members are not allocated adequate time to develop curriculum, design effective instructional
approaches, or coordinate their activities with critical external stakeholders. The
recommendations presented in Table 15 for addressing these challenges are based on research on
the importance of providing employees with adequate resources (Clark & Estes, 2008), aligning
their performance goals with overarching organizational goals (Clark & Estes, 2008), and
fostering relationships with key external stakeholders (Denning, 2005; Lewis, 2011).
Table 15
Summary of Organization Influences and Recommendations
Assumed
Organization
Influence*
Validated
as a Gap
Yes, High
Probability,
No
(V, HP, N)
Priority
Yes, No
(Y, N)
Principle and
Citation
Context-Specific
Recommendation
CVCC needs to
align its key
stakeholder and
V Y People are more
productive
Align LEAP team
members’ long-term,
intermediate, and day-to-
113
organizational
performance
goals. (Cultural
model).
when goal setting
and
benchmarking are
essential to
evaluating
progress and
driving
organizational
performance in
accountability
(Dowd, 2005;
Levy & Ronco,
2012)
Organizations
need to clearly
articulate their
goals and align
them with the
organizational
mission as a
precursor to
developing a
learning
organization that
strives for
continuous
improvement
(Clark & Estes,
2008).
Effective leaders
use storytelling to
communicate the
organization’s
mission,
vision and goals to
its members and
stakeholders
(Denning, 2005)
day performance goals
with overarching
organizational goals.
Provide regular, consistent
communication from
CVCC’s leadership team
of the organization’s
primary goal, the ways in
which key stakeholder
performance goals support
that goal, and examples of
those performance
behaviors.
CVCC needs to
prioritize and
foster
collaboration
between internal
and external
V Y Effective leaders
know how to
create and
manage good
working
relationships with
Establish partnerships
with local schools,
regional Department of
Education (DepEd)
representatives, and
Department of Social
114
stakeholders.
(Cultural model).
stakeholders.
(Lewis, 2011)
Welfare and Development
(DSWD) social workers to
extend the CVCC’s reach,
expand on its capabilities,
coordinate its activities,
and inform its educational
practices.
CVCC needs to
optimize
program efficacy
and
accountability
(Cultural model).
V Y People are more
productive when
goal setting and
benchmarking are
essential to
evaluating
progress and
driving
organizational
performance in
accountability
(Dowd, 2005;
Levy & Ronco,
2012).
Data-driven
benchmarking is a
common approach
to benchmarking
(Dowd, 2005;
Levy & Ronco,
2012).
Connect LEAP team
member performance to
student academic
achievement outcomes
using student academic
progress reports and
CVCC’s Quarterly
Student Survey.
Implement diagnostic
benchmarking approaches
to promote identification
and diagnosis of
programmatic gaps.
CVCC needs to
provide LEAP
team members
dedicated time to
develop and
collaborate on
curriculum and
instructional
design. (Cultural
setting).
V Y Effective change
efforts ensure that
everyone has the
resources
(equipment,
personnel, time,
etc.) needed to do
their job,
and that if there
are resource
shortages, then
resources are
aligned with
organizational
priorities (Clark &
Estes, 2008).
Allocate LEAP team
members a dedicated
period of time each week
to develop and collaborate
on curriculum. Foster the
development of
professional learning
communities (e.g., time,
resources, funding) for
LEAP team members to
collaborate the
development of effective
NFE curriculum and
instructional approaches.
115
CVCC needs to align its key stakeholder and organizational performance goals.
Based on findings from this research study, CVCC needs to align LEAP team members’
performance goals with its broader organizational goals in order to improve the long-term
objective of improving the academic performance of its children. According to Clark and Estes
(2008), organizations need to align individual performance goals with broader organizational
goals in order to realize targeted performance outcomes. In addition to providing a direct link
between an individual’s day-to-day performance and the organization’s intermediate and long-
term objectives, the establishment of goals also provides a benchmark against which progress
can be measured (Clark & Estes, 2008; Rueda, 2011). As a result, CVCC would benefit from
connecting the day-to-day activities of LEAP team members to organizational goals and
communicating the importance between their contributions and the success of the organization.
Based on these principles, the recommended solution entails the alignment of short-term,
individual work goals with intermediate and long-term organizational goals, as well clear
communication of the relationship between LEAP team members’ daily activities and the
organization’s long-term goal of improving the educational outcomes of marginalized children.
CVCC leaders can foster a deeper understanding of the importance of individual contributions to
organizational outcomes by sharing stories connecting the efforts of individual LEAP team
members to student success.
Clark and Estes (2008) underscored the importance of aligning organizational and
stakeholder goals in realizing performance outcomes, noting that the gaps between business
goals and performance goals highlight opportunities for organizational improvements. Lewin
(2011) noted that goals are essential to conveying purpose, assessing performance against stated
116
goals, and legitimizing the organization’s rationale for pursuing an objective. Short-term
performance goals must be current, concrete, and challenging (Clark & Estes, 2008). According
to Clark and Estes (2008), daily or weekly goals motivate employees more effectively than
intermediate or monthly goals. Providing measurable, achievable, short-term performance goals
to employees, as a result, is critical to realizing long-term organizational goals. Communicating
the relevance of short-term goals to long-term outcomes is also essential. Denning (2006)
emphasized the importance of communications in reinforcing key stakeholders’ understanding of
the organization’s vision, mission, and goals. According to Denning (2006), leaders can use
narrative storytelling to share knowledge with their team related to challenges to improving
educational outcomes for marginalized children, solicit and highlight effective solutions, and
foster collaboration around the shared challenge. Based on the aforementioned principles, CVCC
could promote better educational outcomes for marginalized children by providing LEAP team
members with clear, challenging, and current goals that align with overarching organizational
goals. Additionally, these goals could be further reinforced through narrative storytelling that
emphasizes collaborative problem-solving strategies and causal connections between LEAP team
members’ efforts and student achievement outcomes.
CVCC needs to prioritize and foster collaboration between internal and external
stakeholders. Based on findings from this research study, CVCC needs to collaborate with local
K-12 schools, regional representatives of the Philippines’ Department of Education (DepEd), and
social workers from the regional Department of Social Welfare and Development (DSWD) to
promote improved academic achievement for its children. Each of these organizations qualify as
“definitive stakeholders,” which Lewis (2011) categorizes as stakeholders that have power,
legitimacy, and urgency in relation to the targeted outcome. Lewis (2011) identified the
importance of managing meaning, networks, and practice amongst definitive stakeholders and
117
directing each of these elements of stakeholder relationships towards organizational goals.
CVCC’s efforts to foster academic achievement of its children in formal schools using non-
formal, supplemental education would benefit from leveraging relationships with key external
stakeholders vested in similar outcomes. In order to foster these relationships, CVCC should
establish closer partnerships with definitive external stakeholders to extend CVCC’s reach,
expand its capabilities, and inform its educational practices. One example of this form of
collaboration might include quarterly meetings with local teachers from schools that CVCC
children attend to discuss challenges in student performance, strategies for addressing those
challenges, and proactive instructional approaches to improving student academic outcomes in
the future.
In his discussion of the importance of stakeholder relationships in organizational change,
Lewis (2011) emphasized the interconnectivity between organizations, their employees, clients,
partners, government organizations, and other definitive stakeholders. The complexity of these
relationships, according to Lewis (2011), makes it imperative for organizations to regularly
monitor their relationships with internal and external stakeholders to ensure that definitive
stakeholders are aligned in their efforts towards achieving targeted organizational outcomes.
Denning (2006) emphasized the value of “communities” who engage issues across a domain of
shared knowledge and shared purpose, as well as work groups that share a common objective.
Following on these principles, CVCC would benefit from establishing communities of practices
and working groups between internal and external stakeholders focused on addressing the shared
goal of improving children’s educational outcomes.
CVCC needs to optimize program efficacy and accountability. Based on derived from
document analysis and qualitative interviews with LEAP team educators, CVCC needs to
develop better measures for monitoring LEAP team performance against targeted program
118
outcomes. According to Levy and Ronco (2012), individuals are more productive and when
benchmarking and goal setting are used to evaluate organizational performance and
accountability. Dowd (2005) suggested the need for key stakeholders to analyze student
performance data, engage the data to identify barriers to student achievement, and correctly
interpret data to drive informed decision-making. In order to understand their progress against
identified performance goals and drive evidence-based decision making, LEAP team members
would benefit from access to benchmark data and the collection of relevant data to track progress
against those benchmarks. The recommendation for addressing this challenge includes requiring
LEAP team members to use existing student performance data derived from CVCC’s quarterly
student survey and student grade reports. The goal of this recommendation is to promote a
deeper understanding of the causal connection between program design and student outcomes,
driving both program efficacy and accountability.
Diagnostic benchmarking provides organizations with information about the efficacy of
their current practices and a means of identifying needed performance improvements (Dowd,
2005; Levy & Ronco, 2012). Dowd (2005) noted that diagnostic benchmarking is an iterative
process that emphasizes the importance of the relationship between educators and students. In
addition to the benefits derived in terms of program performance and student outcomes,
diagnostic benchmarking also assists organizations in allocating limited resources to the highest
performing programs that will result in targeted outcomes (Levy & Ronco, 2012). The research,
then, supports the potential benefits of implementing the recommended use of existing student
performance data and collection of relevant diagnostic benchmarks include improved decision-
making, increased program efficacy, greater accountability, and the optimization of scarce
organizational resources in achieving improved educational outcomes for marginalized children.
119
CVCC needs to provide LEAP team members dedicated time to develop and
collaborate on curriculum and instructional design. Based on the results of this study, CVCC
needs to allocate dedicated time for LEAP team members to design and develop curriculum and
associated instructional design. Clark and Estes (2008) stated that the misalignment of resources
with organizational goals can interfere with targeted outcomes even in cases where employees
have the knowledge, skills, and motivation to achieve the goal. Organizational policies,
according to Clark and Estes (2008), must promote processes that support employees’ efforts in
realizing targeted outcomes. Based on these principles, LEAP team members would benefit from
the allocation of dedicated design and development time for program curriculum and
instructional content. In order to address this gap, CVCC’s leadership team should allocate
LEAP team members a dedicated period of time each week to design and develop curriculum
and foster the development of professional learning communities that encourage collaboration
between LEAP team members in their efforts to improve educational outcomes for marginalized
children.
Clark and Estes (2008) indicated the importance of aligning resources and work
processes with organizational goals to ensure the attainment of targeted organizational outcomes.
According to the researchers, the discipline of education has been historically handicapped by an
“... almost exclusive conceptual patronage of indefensible, craft-based points of view” (Clark &
Estes, 1999, p. 5). To counter this tendency, Vescio et al. (2008) stressed the need for educators
to invest time to reflect on their own teaching practices and adopt evidence-based teaching
strategies to promote student outcomes. Researchers found that professional learning
communities were particularly effective in resource-constrained communities where teachers did
not have access to formalized professional development opportunities (Sargent & Hannum,
2009). However, the success of professional learning communities hinge on institutional support,
120
as well as key stakeholder engagement (Sargent & Hannum, 2009). As a result, CVCC, as a
resource-constrained organization, would potentially benefit from providing LEAP Team
members with institutional support and a dedicated time to collaborate on, design, and develop
curriculum and associated instructional content to promote CVCC’s educational outcomes.
Integrated Implementation and Evaluation Plan
Implementation and Evaluation Framework
The New World Kirkpatrick Model (2016) serves as a basis for this study’s integrated
implementation and evaluation plan. Kirkpatrick and Kirkpatrick’s (2016) model prioritizes the
attainment of targeted organizational outcomes as the primary basis for evaluating the efficacy of
training initiatives. While designed to evaluate training, the four-level program can also be used
to assess the efficacy of any initiative designed to promote stakeholder learning resulting in
targeted organizational change. This study utilized Clark and Estes (2008) gap analysis
framework to identify knowledge, motivational, and organizational influences that represent gaps
in CVCC’s efforts to provide supplemental NFE to marginalized Filipino in support of academic
achievement in formal schools. Kirkpatrick and Kirkpatrick’s (2016) four level evaluation model
will be used to evaluate the efficacy of CVCC programs designed to address these gaps. The
levels of the New World Kirkpatrick model, presented in the order of their implementation,
include results, behavior, learning, and reaction.
Each of the four levels of the New World Kirkpatrick Model addresses a distinct aspect
of program implementation and evaluation. Level 4 requires the identification of targeted
program outcomes, their alignment with organizational outcomes, and the leading indicators and
metrics used to evaluate targeted versus actual program outcomes. Level 3 focuses on the core,
critical behaviors required to achieve targeted outcomes, as well as the required drivers that
reinforce, monitor, encourage, and reward these behaviors. Level 2 is closely aligned with Clark
121
and Estes (2008) gap analysis, evaluating the degree to which participants acquire the
knowledge, skills, and motivation necessary to successfully perform tasks essential to the
achievement of targeted outcomes. Level 1 reflects participants’ receptivity to, engagement in,
and satisfaction with programs designed to support their professional development in their given
role. Taken collectively, these four steps serve as an effective framework for the implementation
and evaluation of the LEAP team’s NFE initiatives targeted at improving educational outcomes
for marginalized Filipino children.
Organizational Purpose, Need and Expectations
CVCC’s mission is to provide for the safety, security, well-being, and development of
disadvantaged Filipino children through the provision of residential care services, education, and
community outreach. In 2019, the Center’s founder and director prioritized education as a means
of assisting CVCC children break the endemic cycle of poverty (Personal communication, April
17, 2019). Currently, over 80% of CVCC students are one or more years behind grade level due
to socioeconomic circumstances, as opposed to ability. An analysis of student performance data
from Q1-Q3 of the academic year 2019-2020 indicates that, while students are passing their
courses, their average scores fall short of CVCC’s target goal of 90% in all core subjects. This
goal was set in coordination between the Center Director and LEAP Team members as a realistic
target that also supported students’ admission into post-secondary schools. In order to achieve
that goal, students on average would need to improve their performance in core classes by an
average of 5% over their current performance level.
In addition to challenges with academic performance, CVCC’s quarterly student survey
indicates relatively low levels of self-efficacy in CVCC children related to their ability to master
core subjects in school, including Araling Panlipunan (Social Studies), English, Filipino,
Mathematics, and Science. CVCC is currently providing informal education to students through
122
basic support in completing homework assignments, the provision of ESL instruction, and a
small number of intermittent, structured educational activities. However, these efforts are
uncoordinated, lack integration, and are poorly connected to metrics that connect the educational
activity to student achievement outcomes. As one of CVCC’s core goals is to improve academic
outcomes of 100% of CVCC’s children in formal schools through the provision of supplemental
NFE, the Learning and Educational Assistance Program (LEAP) Team was asked to build a
supplemental NFE program capable of improving academic achievement outcomes of CVCC
children in formal schools. This study included recommendations designed to address
knowledge, motivation, and organizational gaps in the LEAP Team’s ability to achieve this
objective, using the New World Kirkpatrick Model as a basis for implementation and integration.
Level 4: Results and Leading Indicators
Table 16 describes the Level 4 outcomes and leading indicators used to gauge CVCC’s
progress towards improving marginalized children’s academic outcomes in formal schools. It
also details the metrics and evaluation methods proposed to measure these outcomes. The
external outcomes reflect CVCC’s goal of enhancing student achievement in formal schools, as
well as enhancing the perception of CVCC as an integral component in partnering with local
schools to address the unique needs of marginalized students. The internal outcomes mirror these
leading indicators, but focus on student-centric metrics as indicators of LEAP Team performance
outcomes, as well demonstrating CVCC’s commitment to prioritizing education as one of their
three pillars of supporting the well-being and development of disadvantaged Filipino children.
Table 16
Outcomes, Metrics, and Methods for External and Internal Outcomes
Outcome Metric(s) Method(s)
123
External Outcomes
Improved educational
attainment of CVCC
students in formal
schools
Percentage of CVCC students
matriculating from elementary, middle,
and secondary school
Department of Education
(DepEd)/CVCC records
Improved academic
performance of
CVCC students in
formal schools
Percentage of students demonstrating a
5% grade improvement in one or more
core subject areas (Araling Panlipunan,
English, Filipino, Mathematics, and
Science) across the academic year
Analysis of quarterly grade
reports issued by
Department of Education
(DepEd) and private
schools
Perception of CVCC
as an organization
that prioritizes
education
Number of awards received by CVCC
children in recognition of academic
achievement in formal schools
Quarterly reporting of
student achievement by the
LEAP Team
Perception of CVCC
as an organization
that prioritizes
education
Amount of funding donated to CVCC
to support educational activities as
reflected in grants, private donations,
volunteer hours, and in-kind
contributions
Quarterly reporting of
financial and in-kind
contributions by CVCC’s
administrative team
Internal Outcomes
Increased
commitment to
education as a pillar
of CVCC’s mission
Percentage of time spent by LEAP
Team and associated CVCC on the
design, development, and delivery of
educational activities in support of
student academic achievement
Monthly reports of LEAP
Team members’
educational support
activities
Improved academic
performance of
CVCC students in
core academic
subjects
Percentage of students demonstrating a
5% grade improvement in one or more
core subject areas (Araling Panlipunan,
English, Filipino, Mathematics, and
Science) across the academic year
Analysis of quarterly grade
reports issued by
Department of Education
(DepEd) and private
schools
Increased self-
efficacy of CVCC
students in core
subjects
Percentage of students who believe that
they can succeed in core classes in
formal schools (Araling Panlipunan,
English, Filipino, Mathematics,
Science)
Analysis of CVCC’s
quarterly student survey
results
Level 3: Behavior
124
Critical Behaviors
In order to achieve the requisite internal and external outcomes and realize program
objectives, LEAP Team members will need to demonstrate three critical behaviors. First, LEAP
Team members must demonstrate their proficiency in identifying barriers to student academic
achievement. Second, LEAP Team members must demonstrate their ability to develop a
supplemental NFE curriculum to address these academic achievement barriers. Finally, LEAP
Team members must demonstrate their proficiency in designing, developing, and delivering
instructional programs that address CVCC students’ challenges and promote improved
performance in core courses in formal schools. Table 17 provides an overview of the metrics,
methods, and frequency used in the evaluation process to determine progress against LEAP’s
goal of improving CVCC students’ academic outcomes.
Table 17
Critical Behaviors, Metrics, Methods, and Timing for Evaluation
Critical Behavior Metric(s)
Method(s)
Timing
1. Leap Team members will
demonstrate increased
proficiency in identifying
the key barriers to student
academic success in formal
schools.
Number of individual
student development plans
providing targeted guidance
for student improvement
and alignment with current
NFE program offerings
Individual
student
development
plans
Monthly
2. LEAP Team members must
demonstrate they have the
knowledge, skills, and
motivation to develop an
NFE curriculum capable of
overcoming CVCC
students’ barriers to
academic success.
Number of CVCC students
who demonstrate academic
performance improvements
of 5% or more in core
subjects, including Araling
Panlipunan, English,
Filipino, Mathematics, and
Science
DepEd quarterly
grade reports
Quarterly
3. Leap Team members will
demonstrate increased
proficiency in designing,
developing, and delivering
supplemental NFE
instructional content to
Number of CVCC students
who demonstrate academic
performance improvements
of 5% or more in core
subjects, including Araling
Panlipunan, English,
Student
performance on
summative and
formative
assessments
Monthly
125
overcome CVCC students’
barriers to success.
Filipino, Mathematics, and
Science.
Required Drivers
Table 18 describes the drivers required to reinforce, encourage, reward, and monitor the
critical behaviors that LEAP Team members need to demonstrate in order to achieve the goal of
improved academic outcomes for CVCC’s children. Kirkpatrick and Kirkpatrick (2016)
underscored the importance of reinforcing, encouraging, rewarding, and monitoring critical
behaviors to ensure that they are not only adopted, but also embedded, in the organization’s
approach to achieving its targeted outcomes. These drivers are designed to align with Clark and
Estes (2008) gap analysis framework and emphasize the drivers requisite to address knowledge,
motivation, and organizational gaps that could prevent a successful implementation of the LEAP
program.
In order to support the acquisition and retention of these critical behaviors, CVCC will
provide LEAP Team members with reinforcers that include training related to the identification
of barriers to student success, as well as the design, development, and delivery of curriculum and
instructional approaches to overcome those barriers. CVCC will use a job aid and lesson
templates to supplement and reinforce the learning acquired during these training sessions, as
well as launch a professional learning community where LEAP Team members can discuss
shared challenges and strategies for promoting student success. In order to encourage the
adoption and maintenance of these critical behaviors, CVCC will acknowledge the LEAP
Team’s successes by promoting program outcomes in the organization’s newsletter and social
media feeds, as well as acknowledging and reaffirming the importance of their work in an annual
meeting between the LEAP Team and CVCC’s founder. Rewards will take the form of public
recognition the exemplary work of LEAP Team members in promoting student success by
126
CVCC’s Founder, Director, and local DepEd principals and teachers, Finally, CVCC will
monitor LEAP Team members’ progress through the institution of monthly review meetings
between LEAP Team member and administrators focused on evaluating and improving the
quality of students’ individual achievement plans, as well as the plans’ alignment with current
NFE programming. Additionally, LEAP Team members will meet with local DepEd educators to
determine the efficacy of current approaches in supporting student success in formal schools, as
well as anticipate the demands for supplemental NFE support based on DepEd’s curriculum.
Table 18
Required Drivers to Support Critical Behaviors
Method(s) Timing
Critical
Behaviors
Supported
1, 2, 3 Etc.
Reinforcing
Create job aid for LEAP Team members that
provides resources related to student
achievement data and examples of its use in
diagnosing academic performance challenges
Develop in the third quarter
of 2020 and update on a
quarterly basis. (Note this
document will also be used to
support training and
professional learning
community sessions).
1
Provide LEAP Team members with lesson plan
templates highlighting key components of an
effective lesson plan and built-in guidance on
how to complete each section
Develop in the third quarter
of 2020, (Note this document
will also be used to support
training and professional
learning community
sessions).
3
Provide LEAP Team members with training on
using student achievement data to diagnose
academic barriers and design, develop, and
deliver appropriate instructional approaches to
address those barriers
Conduct training at beginning
of academic year (May 2020)
and at mid-year break
(October 2020)
1, 2, 3
Professional learning community focused on
identifying student barriers to success and
brainstorming/showcasing approaches to
designing curriculum and associated
Conduct on a monthly basis
starting in September 2020.
1, 2, 3
127
instructional approaches to overcoming those
barriers
Encouraging
Annual LEAP Team meeting with CVCC
Founder to discuss LEAP Team activities, their
impact on student outcomes, and reaffirm
Founder’s commitment to NFE education
programs
Annually beginning in
September 2020
1, 2, 3
Inclusion of articles in quarterly newsletter and
social media feeds highlighting the ways in
which LEAP activities and individual
educators have improved academic outcomes
for marginalized children.
Quarterly beginning in
September 2020
1, 2, 3
Rewarding
CVCC leadership team recognizes s LEAP
Team members’ exemplary work in the areas
of instructional design, development, and
delivery
Monthly beginning in
September 2020
1, 2, 3
Presentations by local DepEd teachers and
principals detailing how LEAP Team’s
supplemental NFE activities contribute to
student outcomes
Quarterly beginning in
September 2020
1, 2, 3
CVCC Director publicly recognizes attainment
of LEAP Team incremental program
milestones
Upon attainment of key
program milestones.
Ongoing.
1, 2, 3
Monitoring
LEAP Team members schedule regular
meetings with local DepEd teachers to discuss
current alignment between supplemental NFE
programs and prepare for upcoming
coursework in formal schools
Quarterly beginning in
September 2020
2, 3
Review of sample of LEAP Team members’
individual development plans by Education
Coordinator and Center Director to ensure
proper diagnosis of student barriers and
alignment with proposed interventions
Monthly beginning in
September 2020
1, 2, 3
Organizational Support
LEAP’s successful implementation hinges on the support of CVCC leadership, both in
terms of its express commitment to the program and the allocation of adequate resources to
support the LEAP Team’s efforts to achieve targeted outcomes. In order to realize targeted goals,
128
organizations must align their structure, skills, staff, systems, skills, leadership styles, and shared
values to focus on targeted improvements (Waterman at al., 1980). In order to support the LEAP
Team’s efforts at improving educational outcomes for students, CVCC needs to align its key
stakeholder and organizational performance goals; prioritize and foster collaboration between
internal and external stakeholders; optimize program accountability and efficiency; and provide
LEAP team members dedicated time to develop and collaborate on curriculum and instructional
design. For example, CVCC should ensure that LEAP Team members’ position descriptions
reflect their responsibilities as educators, provide adequate time to perform those duties, and
establish incentive systems that recognize performance that drives outcomes. In order to ensure
that the LEAP Team is receiving adequate support, CVCC’s leadership team should schedule
quarterly meetings with the LEAP Team to discuss the team’s performance; identify alignment
with targeted critical behaviors; assess and review program performance metrics; and propose
approaches to enhancing organizational support for LEAP Team initiatives.
Level 2: Learning
Learning Goals
In order for LEAP Team members to exhibit the critical behaviors essential to achieving
the program goals as expressed in Table 18, the following learning objectives must be mastered.
These learning objectives align with the assumed knowledge and motivational influences
identified in Chapter Three and performance gaps validated through document analysis and
qualitative interviews in Chapter Four. Following the implementation of the program, LEAP
Team members will be able to:
1. Identify key sources of student performance data (Knowledge – declarative)
2. Apply student performance data to diagnose barriers to student academic achievement
(Knowledge – procedural)
129
3. Develop curriculum and associated instructional content to improve student
performance outcomes (Knowledge – procedural)
4. Apply learning theories to improve NFE curriculum to promote academic
achievement (Knowledge – procedural)
5. Describe their process for evaluating and improving their knowledge related to
designing, developing, and delivering supplemental NFE that improves student
performance outcomes (Knowledge – metacognitive)
6. Express confidence in their ability to design and develop NFE programs that improve
student academic achievement outcomes (Motivation - self-efficacy)
7. Describe their professional goals and process for improving their knowledge and
skills in designing, developing, and delivering supplemental NFE that promote
improved student performance outcomes (Motivation – goal orientation)
Program
In order to provide LEAP Team members with the requisite procedural knowledge
essential to identify and overcome barriers to student academic performance using supplemental
NFE, CVCC should conduct a training session for all LEAP Team members. CVCC employs a
full-time Education Coordinator, as well as maintaining contractual arrangements with three
local DepEd teachers. Collectively, these individuals can develop a training program targeted at
providing LEAP Team members with the requisite procedural knowledge to develop
supplemental NFE programs that support learning outcomes in formal schools. The focus of this
training would include identifying students experiencing challenges in performing in core
academic subjects in formal schools, developing individualized student improvement plans
outlining goals for improving academic performance, and matching appropriate NFE
interventions to identified student needs. Prior to implementing the training, the Education
130
Coordinator and DepEd educators should collaborate to develop the instructional program and
associated training materials needed to support the program. Within the context of the training
session, LEAP Team members should be allowed to develop a sample individual student
development plan, beginning with the identification of performance issues and working the case
through to a completed NFE instructional session targeted at improving one aspect of the
student’s performance. As part of this process, attendees would receive feedback in the form of
peer reviews and one-on-one evaluation sessions with the Education Coordinator and DepEd
teaching staff. Following the training, CVCC should conduct monthly meetings of the LEAP
Team professional learning community to reinforce and expand upon the lessons learned during
the formal training session.
Evaluation of the Components of Learning
Level 2 of New World Kirkpatrick Model stresses the importance of knowledge, skills,
attitude, confidence, and commitment in evaluating the efficacy of training (Kirkpatrick &
Kirkpatrick, 2016), Table 19 illustrates how LEAP Team member will be evaluated based on
their performance across these five dimensions to ensure that the training has achieved its
targeted objectives. In accord with the New World Kirkpatrick model, the efficacy of training is
measured prior to, during, and after the training session using measures that balance the need to
measure training outcomes, while maintaining limited resources for higher level evaluations.
Table 19
Evaluation of the Components of Learning for the Program
Method(s) or Activity(ies) Timing
Declarative Knowledge “I know it.”
Review the job aid detailing available student
achievement metrics
Prior to and at the start of the training
session.
Review the lesson template highlighting key
components of an effective lesson plan and built-in
guidance on how to complete each section
At the start of and during training
session
131
Knowledge checks Intermittently during training session
Procedural Skills “I can do it right now.”
Demonstration of ability to identify student
performance issues using student achievement
metrics
During training session and in follow-up
sessions with Education Coordinator
Demonstration of ability to design and deliver NFE
curriculum
During training session and in follow-up
sessions with Education Coordinator
Demonstration of ability to design, develop, and
deliver NFE instructional sessions
During training session and in follow-up
sessions with Education Coordinator
Demonstration of ability to match appropriate NFE
interventions with student needs
During training session and in follow-up
sessions with Education Coordinator
Attitude “I believe this is worthwhile.”
Likert scale survey Following the session
Observation of LEAP Team member engagement
during training session
During training session
Confidence “I think I can do it on the job.”
Likert scale survey Following the session
Observation of LEAP Team member engagement
during training session
During training session
Commitment “I will do it on the job.”
Likert scale survey Immediately following the session and 2
weeks after session
Group discussion End of session
Demonstration of ability to design, develop, and
deliver NFE instructional sessions
In follow-up sessions with Education
Coordinator and within context of
professional learning community
Level 1: Reaction
Kirkpatrick and Kirkpatrick (2016) identified three dimensions that require evaluation at
Level One, all of which relate to attendees’ reaction to the learning experience. These
dimensions include engagement, relevance, and customer satisfaction. Table 20 includes the
methods and tools designated to measure attendees’ reactions to the learning experiences across
these three dimensions. With the exception of a summative evaluation at the conclusion of the
session, the methods are consistent with Kirkpatrick and Kirkpatrick’s (2016) guidance that
emphasizes the importance of utilizing formative assessments to allow for adjustments during the
course of training delivery. The selected evaluation approaches also minimize the expenditure of
132
excess resources in capturing Level One information that could be better used in evaluating
higher level outcomes associated with Level 2 through Level 4 evaluation.
Table 20
Components to Measure Reactions to the Program
Method(s) or Tool(s) Timing
Engagement
Observation (Dedicated
observer)
During session
Pulse checks During training at specified points and as needed, based on
observation
Evaluation End of session
Relevance
Observation (Dedicated
observer)
During session
Pulse checks During training at specified points and as needed, based on
observation
Evaluation End of session
Customer Satisfaction
Observation (Dedicated
observer)
During session
Pulse checks During training at specified points and as needed, based on
observation
Evaluation End of session
Evaluation Tools
Immediately Following the Program Implementation
Immediately following the initial LEAP Team training session, CVCC will conduct a
survey to capture attendees’ reactions and evaluate initial learning outcomes in terms of
knowledge transfer. Kirkpatrick and Kirkpatrick (2016) emphasized the importance of
conducting Level 1 and Level 2 evaluation immediately following the program’s
implementation. Level 1 responses measure participants’ reaction to the training program,
including their degree of engagement and satisfaction, as well as the relevance of the training
content, while Level 2 focuses on participant’s acquisition of knowledge, skills, attitude,
133
confidence, and commitment (Kirkpatrick & Kirkpatrick, 2016). The proposed survey for this
evaluation, composed of 14 scaled and 5 open-ended prompts, is included in Attachment A.
Delayed for a Period After the Program Implementation
Each quarter following the initial training, CVCC will offer follow-up training as a
component of their quarterly professional learning community (PLC) sessions. During these
sessions, a follow-up survey will be conducted to assess the training component of the PLC
program, as well as overall progress against the leading indicators of critical behaviors.
Kirkpatrick and Kirkpatrick (2016) recommended using a delayed, blended approach to
measuring training efficacy that captured elements of all four levels of the New World
Kirkpatrick Model. In addition to gauging participants’ responses and learning, the blended
approach also gauges the degree to which participants are exhibiting targeted behaviors and the
extent to which those behaviors are producing desired outcomes. The proposed instrument for
this evaluation, composed of 21 open-ended questions, is included in Appendix Ds.
Data Analysis and Reporting
A core component of CVCC’s mission is providing educational opportunities to
marginalized Filipino children. As a result, CVCC’s leadership team’s primary interest rests in
the LEAP team’s ability to deliver effective NFE to children to support their success in formal
schools. In order to measure the efficacy of CVCC’s training efforts, a Level One and Level Two
evaluation will be conducted immediately after the training session to capture participants’
reactions to the training and self-assessment of their learning. While measuring Level 1 and 2
outcomes provides valuable insights into the relevance of training and serves as a gauge of the
participants’ reaction, engagement, and perceived learning, it is insufficient to demonstrate
changes to LEAP team members’ critical behaviors or program results. Kirkpatrick and
Kirkpatrick (2016) underscored the importance of conducting ongoing evaluation of progress
134
throughout program implementation to inform program development, make course corrections,
and maximize outcomes. To evaluate these higher-level outcomes, CVCC will conduct ongoing
blended Level 1 through 4 evaluations on an ongoing basis at each quarterly PLC meeting. The
PLC meetings will align with the release of quarterly student performance reports which, in
combination with the CVCC quarterly student survey and blended survey results, will provide
CVCC’s leadership team with a more holistic understanding of the LEAP team’s progress in
terms of critical behaviors and targeted outcomes.
Immediately following the initial training session, CVCC will conduct a survey to elicit
LEAP team members’ reactions to the training, as well as allow them to self-assess the training’s
impact on their knowledge, skills, attitudes, confidence, and commitment. Figure 3, which
utilizes fictional data, illustrates one approach for reporting the results of these evaluations to
CVCC’s leadership team. Using this graphic, CVCC’s leadership team can readily identify shifts
in LEAP team members’ perceived improvement across the five dimensions of Level 2
evaluation. In addition to providing the distribution of responses within a given response, the
graphic also allows leaders to compare LEAP team members’ perceptions prior to and following
the training. Table 21 provides a comparison of fictional data derived from the blended
evaluation conducted at quarterly PLC meetings and corresponding student quarterly
performance data.
135
Figure 3
Sample Report Extract depicting Participant Self-perception of Level 2 Outcomes Prior to
and Immediately following Initial Training
Table 21
Sample Report Extract detailing LEAP Team Members’ Perceptions of Program Efficacy versus
Metric-based Performance Outcomes
LEAP Team Self-Assessment Outcomes since
LEAP launch
Critical Behavior Average LEAP
Team Member Self-
Assessment
(1=Strongly
disagree,
5=Strongly agree)
Metric Outcome
We have been
successful at
identifying students
in need of
supplemental NFE
support.
3.5
Percentage of CVCC
students who have
received NFE
support based on
IEP review or staff
referrals
78%
136
We are effective at
matching students
with NFE support
resources.
2.9
Percentage of CVCC
students who have
accessed NFE
support and
demonstrated
increased academic
performance
43%
We have developed
individual
educational plans
for all CVCC
students.
4.7
Percentage of
students with
updated individual
education plans (<6
months old)
98%
We have witnessed
measurable
improvements in
student academic
performance.
3.2
Percentage
improvement in
academic
performance in core
subjects as reflected
in quarterly grade
reports
Araling Panlipunan
English
Filipino
Mathematics
Science
Overall
1.7%
8.9%
1.8%
3.6%
3.2%
3.8%
Summary
The New World Kirkpatrick Model (Kirkpatrick & Kirkpatrick, 2012) complements
Clark and Estes’ (2008) gap analysis framework by providing a schema for delineating leading
indicators, learning objectives, and critical behaviors designed to address identified gaps in KMO
influences. Having established the LEAP Team’s primary goal of improving student
performance outcomes in formal schools through supplemental NFE provision, the LEAP team’s
stakeholder goal was further delineated into targeted internal and external outcomes. Metrics and
leading indicators were articulated for each outcome in order to focus the integration and
evaluation plan on priority organizational outcomes, as well as to measure the LEAP team’s
progress against those goals. Level 3 was designed to assess the proposed program’s efficacy
across three critical behaviors that LEAP team members need to exhibit to provide supplemental
NFE, including the ability to identify student performance barriers, design NFE curriculum, and
design, develop, and deliver NFE instructional sessions capable of assisting students in
137
overcoming those barriers. Specific measures were developed to recognize, encourage, reward,
and monitor LEAP team members’ exhibition of these behaviors. Level 2 emphasized improving
LEAP team members’ knowledge, skills, attitudes, confidence, and commitment in enacting the
critical behaviors identified in Level 3. In order to promote the critical behaviors needed to
develop effective NFE, an initial training program was proposed that delineated specific learning
objectives targeted at developing LEAP team members’ knowledge, skills, attitude, confidence,
and commitment in developing the LEAP. Finally, Level 1 defined the need for developing
training that was engaging, relevant, and satisfying for attendees. The combination of Clark and
Estes’ (2008) gap analysis and the New World Kirkpatrick Model (Kirkpatrick & Kirkpatrick,
2012) provides a structured approach to identifying organizational challenges and developing
result-oriented solutions. Clark and Estes (2008) emphasized the importance of using evidence-
based approaches to select performance solutions. The New World Kirkpatrick Model provides a
framework for evaluating a performance solution throughout its design and implementation,
allowing for course corrections based on metrics that serve as indicators of the solution’s
efficacy in achieving targeted organizational outcomes.
Strengths and Weaknesses of the Approach
The Clark and Estes (2008) gap analysis framework was used to conduct this study. The
objective of the gap analysis framework is the identification of primary stakeholders’ knowledge,
motivation, and organizational (KMO) gaps serving to inhibit the attainment of organizational
goals. Once identified, the model proposes deploying corresponding solutions designed to
address those gaps, evaluating the efficacy of the solutions, and refining the solutions based to
promote continuous performance improvements.
The gap analysis framework had several advantages in assessing CVCC’s LEAP
program. The gap analysis model provided a step-by-step approach to evaluating performance
138
challenges, while at the same time acknowledging the need for an iterative, cyclical model that
emphasizes continuous, incremental improvement. In CVCC’s case, the model’s emphasis on
setting and analyzing performance benchmarks was critical to addressing knowledge barriers that
emerged during initial site visits and review of the extant literature. An additional advantage of
the model was its portability and the ease in which the process model could be communicated to
and understood by organizational leaders and stakeholders. The simplicity of the model,
however, stands in stark contrast to the complexity of organization, resulting in limitations of the
model in representing organizational and systemic complexity.
Two key weaknesses were identified in using the gap analysis framework, both of which
reflected the difficulty of representing systemic complexity in organizations. The first weakness
was the model’s focus on one stakeholder group within the organization. While the benefits of
focusing on one stakeholder group, ceteris paribus, allows for clearer assessments of the impacts
of performance enhancements, it falls short of representing the complexity of organizations, the
role of external stakeholders in influencing programmatic outcomes, and the environments in
which they operate. For example, the partnership between CVCC, DepEd, and the DSWD
reflects multiple stakeholders whose performance goals differ, but all of whom influence NFE
provision. A second and related challenge entails the weakness of the KMO framework in
acknowledging systemic complexity and the influences of external variables that influence
organizational performance in attaining desired outcomes. An example of systemic complexity
In the final analysis, Clark and Estes (2008) gap analysis framework proved effective in
identifying an array of KMO gaps inhibiting CVCC’s implementation of the LEAP program.
While time consuming and resource intensive, the framework provided CVCC with insights into
barriers that, when viewed without the support of the framework, were difficult to identify. It
also provided CVCC with a foundation for evaluating solutions and demonstrating the efficacy
139
of their supplemental NFE program in serving the needs of previously marginalized children in
succeeding in formal schools.
Limitations and Delimitations
In addition to the limitations detailed at the conclusion of Chapter Three, three additional
limitations should be noted. First, CVCC’s supplemental NFE program, while still emerging, is
innovative and caution should be used in generalizing the findings and solutions recommended
in the study to other settings. As opposed to supplanting formal education, the program
emphasizes supplementing marginalized children’s education in formal schools and, as a result,
more closely aligns with the UNESCO Institute for Statistics (UIS; 2012) definition of NFE as a
structured alternative or supplement to formal education.
Second, delays in the implementation of the LEAP program and school closures resulting
from the COVID-19 pandemic prevented a more comprehensive analysis of the efficacy of
CVCC’s NFE program. Due to the COVID-19 outbreak, international travel to and inter-city
travel within the Republic of the Philippines was restricted. Schools scheduled to be opened in
June 2020 remained shuttered or repurposed as COVID-19 quarantine and treatment facilities
during the data collection phase of the study. In-person individual and focus group interviews
were conducted online via videoconferencing applications, providing an additional layer of
complexity to communications with respondents. Despite these challenges, the revised study
identified substantive challenges to the successful launch of the LEAP program and serves as a
point of comparison for future research.
Finally, the influence of cultural factors is difficult to quantify. The United States has a
complex relationship with the Republic of the Philippines, including a legacy of colonialism
spanning from 1899 to 1946 that continues to resonate today. While every measure was taken to
minimize bias in the design and conduct of the study, the role and identity of the researcher as
140
the primary research instrument should be considered in reviewing the findings and
recommendations. Research that clarifies or improves these findings is encouraged.
Recommendations for Future Research
The paucity of research demonstrating the efficacy of NFE initiatives in delivering
measurable outcomes offers significant opportunities for future research (Shepherd, 2014). Based
on limited funding and greater demand for accountability from NFE programs, the basis for
future research should focus on measurable, evidence-based performance results. Three
opportunities for future research, selected based on their relevance to demonstrating efficacy and
expanding on the findings from this research study, are presented in this section.
The first recommendation for future research requires the expansion of the existing
approach to evaluate the efficacy of a cross-section of NFE initiatives, such as street-based
education, alternative learning programs (e.g., ALS), and public-private NFE initiatives.
Pursuing an evaluation of differing approaches to NFE provision would allow for comparisons of
the efficacy of programs in promoting academic achievement for marginalized children.
A second recommendation involves conducting longitudinal studies to determine the
long-term impact of NFE initiatives. While the design of the current study emphasizes short-term
returns in terms of academic performance, the long-term goal of education is support individual’s
efforts to realize their full human potential. Longitudinal data would provide information outside
the scope of the current study, such as information on educational attainment and employment
statistics that serve as comparisons to comparable statistics collected on the general population.
The final recommendation for research focuses on continuing the evaluation of the
LEAP’s efficacy in improving student achievement in formal schools. As noted in the preceding
limitations section, additional research is required to assess the trajectory of CVCC’s LEAP
initiative and determine its long-term efficacy, both in terms of addressing stakeholders’
141
performance gaps and measuring improvement in student academic achievement. Based on the
assessment of LEAP’s limited implementation, it is difficult to determine the efficacy of the
program, although initial results related to the use of performance benchmarking in creating the
Center’s ESL program are promising. Due to emergence of the COVID-19 pandemic, NFE has
temporarily supplanted formal education as the primary means of educating CVCC’s children.
As a result, understanding NFE’s quality and impact on students’ academic achievement has
broader implications both for CVCC’s children and students worldwide.
Conclusion
Education is a basic human right. CVCC plays a central role in guaranteeing that right to
abandoned, abused, and neglected children in the Philippines through the provision of holistic
residential care, education, and community outreach services. The socioeconomic realities of
implementing international educational ideals at the national level has placed heavy demands on
developing nations in providing available, accessible, adaptable, and acceptable education to all
children. Based on the knowledge that formal schools are often poorly equipped to fully meet
these demands, CVCC established the Learning and Educational Assistance Program (LEAP) in
2019, a supplemental non-formal education program designed to support CVCC students’
academic achievement in formal schools.
CVCC launched LEAP in the autumn of 2019, establishing an initial goal of improving
children’s academic performance by 5% by June 2021. LEAP team educators were selected as
the primary stakeholders in the study due to their role in the provision of NFE education and as a
direct interface between the students and local schools. An assessment of their capacity to fulfill
the targeted performance objective using Clark and Estes (2008) gap analysis framework
identified knowledge, motivation, and organizational gaps inhibiting LEAP’s implementation,
142
including gaps that prevented the identification student performance barriers and the creation of
NFE programs to address those barriers.
In response to these performance gaps, solutions were recommended that included the
use of job aids, training, and the development of a community of learning to improve
stakeholders’ proficiency in developing NFE programs. Kirkpatrick and Kirkpatrick’s (2016)
New World Model was proposed to assess the efficacy of these recommendations across four
levels of evaluation, including reaction, learning, behavior, and result. Ultimately, the goal of the
solution is to enhance the capabilities of LEAP team members to support the academic
achievement of marginalized Filipino children in formal schools. Attaining this goal, while small
in scale in comparison to global challenges in providing education for all, represents a step
towards fulfilling the international community’s commitment to providing an accessible,
equitable, and quality education for every child.
143
References
Atkinson, R., & Shiffrin, R. (1968). Human Memory: A Proposed System and its Control
Processes (Vol. 2, pp. 89–195). Elsevier Science & Technology. https://doi.org/10.1016/
S0079-7421(08)60422-3
Bano, M. (2008). Non-profit education providers vis-à-vis the private sector: Comparative
analysis of non-governmental organizations and traditional voluntary organizations in
Pakistan. Compare: A Journal of Comparative Education, 38(4), 471–482.
https://doi.org/10.1080/03057920701420932
Baker, L. (2006). Metacognition. http://www.education.com/reference/article/ metacognition/
Bandura, A. (1977). Social learning theory. Prentice Hall.
Bandura, A. (1997). Self-efficacy: The exercise of control. W. H. Freeman.
Bandura, A. (2005). The evolution of social cognitive theory. In K. G. Smith & M. A. Hitt
(Eds.), Great minds in management (pp. 9–35). Oxford University Press.
Bandura, A. (2012). On the functional properties of perceived self-efficacy revisited. Journal of
Management, 38(1), 9–44. https://doi.org/10.1177/0149206311410606
Baum, D. (2012). Education service contracting in the Philippines: Human rights as trumps,
goals, or policy talk? Educational Research for Policy and Practice, 11(3), 189–206.
https://doi.org/10.1007/s10671-011-9118-5
Berckmans, I., Velasco, M., Tapia, B., & Loots, G. (2012). A systematic review: A quest for
effective interventions for children and adolescents in street situation. Children and Youth
Services Review, 34(7), 1259–1272. https://doi.org/10.1016/j.childyouth.2012.02.014
Bogdan, R. C., & Biklen, S. K. (2007). Qualitative research for education: An introduction to
theories and methods (5th ed.). Allyn and Bacon.
144
Boholano, H. B. (2013). Learning skills of the street children in Metro Cebu. European Scientific
Journal 9(34), 160-176.
Bowen, G. A. (2009). Document analysis as a research method. Qualitative Research Journal
9(2), pp. 27-40.
Boykin, A. W. (1986). The triple quandary and the schooling of Afro-American children. In U.
Neisser (Ed.), School achievement of minority children: New perspectives (pp. 57-92).
London: Lawrence Erlbaum.
Brennan, B. (1997). Reconceptualizing non-formal education. International Journal for Lifelong
Education 16(3), 185-200. https://doi.org/10.1080.0260137970160303
Burke Johnson, R. & Christensen, L. (2014). Educational research: Quantitative, qualitative,
and mixed approaches (5
th
ed.). Los Angeles: Sage.
Campbell, D. (2002). Outcomes assessment and the paradox of nonprofit
accountability. Nonprofit Management and Leadership, 12(3), 243–259.
https://doi.org/10.1002/nml.12303
Catubig, M., & Villano, R. (2017). Conditional cash transfer and school outcomes: An
evaluation of the Pantawid Pamilyang Pilipino Program in Davao Oriental,
Philippines. Asian Economic Journal, 31(4), 403–421. https://doi.org/10.1111/asej.12137
Charvon, G., & Chase, E. (2016). Understanding “Education for All” in Contexts of Extreme
Poverty: Experiences from Burkina Faso. Journal of International and Comparative
Education, 5(2), 103–114. https://doi.org/10.14425/jice.2016.5.2.103
Chinapah, V., Cars, M., Grinberg, S. (2013). Global efforts towards quality education for all:
Evidence and reflections from an international and comparative educational perspective.
Journal of Education and Research 3(2), 39-58. http://doi.org/10.3126/jer.v3i2.8397
145
Clark, R. E. and Estes, F. (1999). The development of authentic educational technologies.
Educational Technology, 39(2), 5-16.
Clark, R. E. & Estes, F. (2008). Turning research into results: A guide to selecting the right
performance solutions. Information Age Publishing, Inc.
Coombs, P. H, & Ahmed, M. (1973). Attacking rural poverty: How nonformal education can
help. Johns Hopkins University Press.
Creswell, J. W. (2014). Research design: Qualitative, quantitative, and mixed methods
approaches (4
th
ed.). SAGE Publications.
Datta, D. & Banik, D. (2014). “If the child cannot come to the school, then the school must go to
the child”: The railway station platform schools in Odisha, India. Education 3 - 13, 42(4),
369–385. https://doi.org/10.1080/03004279.2012.709530
Denler, H., Wolters, C., & Benzon, M. (2006). Social cognitive theory.
http://www.education.com/reference/article/social-cognitive-theory
Denning, S. (2006). Effective storytelling: strategic business narrative techniques. Strategy &
Leadership, 34(1), 42–48. https://doi.org/10.1108/10878570610637885
Denzin, N. (1978). The research act: A theoretical introduction to sociological methods.
McGraw-Hill.
Dowd, A. (2005). Data don’t drive: Building a practitioner-drive culture of inquiry to assess
community college performance. Lumina Foundation for Education.
Ebrahim, A. (2010). The many faces of nonprofit accountability. In D. O. Renz & associates
(Eds.), The Jossey-Bass handbook of nonprofit leadership and management (pp. 101–
123) (3rd ed.). John Wiley & Sons/Jossey-Bass.
Eccles, J. (2006). Expectancy value motivational theory. http://www.
education.com/reference/article/expectancy-value-motivational-theory/
146
Erez, M. & Gati, E. (2004). A dynamic, multi-level model of culture: From the micro level of the
individual to the macro level of global culture. Applied Psychology: An International
Review, 53(4), 583-598.
Flavell, J. (1979). Metacognition and cognitive monitoring: A new area of cognitive-
developmental inquiry. American Psychologist, 34(10), 906–911.
https://doi.org/10.1037/0003-066X.34.10.906
Frederiksen, C., Kehoe, E., & Wood, R. (2011). Effects of instructional aids on the acquisition of
dynamic decision-making skills. Learning and Instruction, 21(5), 601–613.
https://doi.org/10.1016/j.learninstruc.2011.01.002
Ganimian, A., & Murnane, R. (2016). Improving education in developing countries: Lessons
from rigorous impact evaluations. Review of Educational Research, 86(3), 719–755.
https://doi.org/10.3102/0034654315627499
Gallimore, R., & Goldenberg, C. (2001). Analyzing cultural models and settings to connect
minority achievement and school improvement research. Educational Psychologist,
36(1), 45–56. https://doi.org/10.1207/S15326985EP3601_5
Glesne, C. (2011). Becoming qualitative researchers: An introduction (4th ed.). Pearson.
Grant, H., & Dweck, C. (2003). Clarifying achievement goals and their impact. Journal of
Personality and Social Psychology, 85(3), 541–553. https://doi.org/10.1037/0022-
3514.85.3.541
Grover, S. (2007). The education rights of street-involved children. Brock Education 16(2), 21-
28.
Guererro, C. (2007). Country Profile commissioned for the EFA Global Monitoring Report 2008,
Education for All by 2015: Will we make it? https://unesdoc.unesco.org/
ark:/48223/pf0000155532
147
Heckman, J., Humphries, J., & Veramendi, G. (2018). The nonmarket benefits of education and
ability. Journal of Human Capital, 12(2), 282–304. https://doi.org/10.1086/697535
Hentschke, G. C. & Wohlstetter, P. (2004). Cracking the code of accountability. University of
Southern California Urban Education, Spring/Summer, 17-19.
Jiang, Y., Ma, L., & Gao, L. (2016). Assessing teachers’ metacognition in teaching: The Teacher
Metacognition Inventory. Teaching and Teacher Education, 59, 403–413.
https://doi.org/10.1016/j.tate.2016.07.014
Kaiman, J. & De Leon, S. (2016, May 28). The Philippines has 1.8 million abandoned children.
Here’s what keeps many from adoption. The Los Angeles Times.
https://www.latimes.com/world/asia/la-fg-philippines-orphans-adv-snap-story.html
Kayani, M., Akbar, R., Faisal, S., Kayani, A., & Ghuman, M. (2017). Analysis of socio-
economic benefits of education in developing countries: A [sic] example of
Pakistan. Bulletin of Education and Research, 39(3), 75-92.
Kirkpatrick, J., & Kirkpatrick, W. (2016). Kirkpatrick’s four levels of training evaluation. ATD
Press.
Kolb A.Y. & Kolb D.A. (2012). Experiential learning theory. In Encyclopedia of the Sciences of
Learning. Springer.
Kotter, J. (2012). Leading change. Harvard Business Review Press.
Kramarski, B., & Michalsky, T. (2009). Investigating Preservice Teachers’ Professional Growth
in Self-Regulated Learning Environments. Journal of Educational Psychology, 101(1),
161–175. https://doi.org/10.1037/a0013101
Krathwohl, D. R. (2002). A revision of Bloom’s Taxonomy: An overview. Theory into Practice,
41, 212–218. https://doi.org/10.1207/s15430421tip4104_2
148
Krueger, R. A. & Casey, M. A. (2009). Focus groups: A practical guide for applied research
(4th ed.). SAGE Publications.
Levy, G., & Ronco, S. (2012). How Benchmarking and Higher Education Came Together. New
Directions for Institutional Research, 2012(156), 5–13. https://doi.org/10.1002/ir.20026
Lewis, L. (2011). Organizational change: Creating change through strategic communication .
Wiley-Blackwell.
Marsh, J. (2012). Interventions promoting educators’ use of data: Research insights and
gaps. Teachers College Record, 114(11).
http://www.tcrecord.org.libproxy1.usc.edu/library/content.asp?contentid=16805
Marsh, J. A. & Farrell, C. C. (2015). How leaders can support teachers with data-driven decision
making: A framework for understanding capacity building. Educational Management
Administration Leadership, 43(2), 269–289. https://doi.org/10.1177/1741143214537229
Maxwell, J. (2013). Qualitative research design: An interactive approach (3rd ed.). SAGE
Publications.
Mayer, R. E. (2011). Applying the science of learning. Pearson Education.
McCowan, T. (2011). Human rights, capabilities and the normative basis of “Education for
All.” Theory and Research in Education, 9(3), 283–298.
https://doi.org/10.1177/1477878511419566
Merriam, S., & Tisdell, E. (2016). Qualitative research: A guide to design and
implementation (Fourth edition.). Jossey-Bass.
Morpeth, R. & Creed, C. (2012). Reframing basic education to deliver Education for All:
Flexible provision and enabling frameworks. Open Learning, 27(3), 201–214.
https://doi.org/10.1080/02680513.2012.716654
149
Nasir, M., Khalid, A., & Shoukat, A. (2014). Maslow theory of human development and
emergence of street children phenomenon in Pakistan. Pakistan Vision, 15(2), 98–123.
http://search.proquest.com/docview/1652185189/
National Economic and Development Authority. (2017). Philippine development plan: 2017-
2022 (Abridged Version).
Owen, S. (2014). Teacher professional learning communities : Going beyond contrived
collegiality toward challenging debate and collegial learning and professional
growth. Australian Journal of Adult Learning, 54(2), 54–77.
Pajares, F. (2006). Self-efficacy theory. http://www.education.com/reference/ article/self-
efficacy-theory/
Patton, M. Q. (2002). Qualitative research & evaluation methods (3rd ed.). SAGE Publications.
Pekrun, R. (2011). Emotions as drivers of learning and cognitive development. In R. A. Calvo &
S. K. D’Mello (Eds.), New Perspectives on Affect and Learning Technologies (pp. 23-
39). New York: Springer. https://doi.org/10.1007/978-1-4419-9625-1_3
Philippine Institute for Development Studies. (2009a). Governance in the education sector.
Development Research News, Volume XXVII (No. 4). Philippine Institute for
Development Studies: Makati City, Philippines.
Philippine Institute for Development Studies. (2009b). Improving Local Service Delivery for the
MDGs in Asia: The Case of the Philippines. IDEAS Working Paper Series from RePEc.
https://search.proquest.com/docview/1698934796/
Philippine Statistics Authority. (2017). Nine percent of Filipinos aged 6 to 24 years are out of
school (Results from the 2017 Annual Poverty Indicators Survey).
https://psa.gov.ph/content/one-of-every ten-filipinos-aged-6-24-years-out-school-child-
and-youth
150
Pintrich, P. R. (2003). A motivational science perspective on the role of student motivation in
learning and teaching contexts. Journal of Educational Psychology, 95, 667–686.
https://doi.org/10.1037/0022-0663.95.4.667
Pintrich, P. R., & Schunk, D. H. (2002). Motivation in education: Theory, research and
applications (2nd ed.). Prentice Hall.
Prenger, R., Poortman, C., & Handelzalts, A. (2017). Factors influencing teachers’ professional
development in networked professional learning communities. Teaching and Teacher
Education, 68, 77–90. https://doi.org/10.1016/j.tate.2017.08.014
Prytula, M. (2012). Teacher Metacognition within the Professional Learning
Community. International Education Studies, 5(4). https://doi.org/10.5539/ies.v5n4p112
Rabasso, C. & Rabasso, J. (2014). Responsible education and multiple learning identities by the
Mamanwas in Surigao del Norte, Mindanao, Philippines. Journal of Global
Responsibility, 5(1), 68–81. https://doi.org/10.1108/JGR-02-2014-0002
Rana, H. & Chaudry, H. (2012). Domestic conflicts and social inequalities compel children to
walk on street. Science International, 24(4).
http://link.galegroup.com.libproxy1.usc.edu/apps/doc/A321510697/AONE?u=usocal_ma
in&sid=AONE&xid=839e0a27
Read, L. & Atinc, T. (2017). Investigations into Using Data to Improve Learning. The Brookings
Institution. http://search.proquest.com/docview/1884426467/
Robeyns, I. (2006). Three models of education: Rights, capabilities and human capital. Theory
and Research in Education, 4(1), 69–84. https://doi.org/10.1177/1477878506060683s
Rose, P. (2009). NGO Provision of Basic Education: Alternative or Complementary Service
Delivery to Support Access to the Excluded? Compare: A Journal of Comparative and
International Education, 39(2), 219–233. https://doi.org/10.1080/03057920902750475
151
Rubin, H. J. & Rubin, I. S. (2012). Qualitative interviewing: The art of hearing data (3rd ed.).
SAGE Publications.
Rueda, R. (2011). The 3 dimensions of improving student performance. Teachers College Press.
Rufino, C. (2015). The joint estimation of Filipino child’s participation in schooling and
employment and new stylized facts on the Philippine child labor situation. De La Salle
University Business & Economics Review 25(1), 119-142.
Sargent, T., & Hannum, E. (2009). Doing more with less: Teacher professional learning
communities in resource-constrained primary schools in rural China. Journal of Teacher
Education, 60(3), 258–276. https://doi.org/10.1177/0022487109337279
Schein, E.H. (2017). Organizational culture and leadership, 5th edition. San Francisco: Jossey-
Bass.
Schraw, G., & McCrudden, M. (2006). Information processing theory.
http://www.education.com/reference/article/information-processing-theory/
Schunk, D. H., & Hanson, A. R. (1985). Influence of peer-model attributes on children’s beliefs
and learning. Journal of Educational Psychology, 81: 431-434.
Scott, D., & Usher, R. (2011). Researching education data, methods and theory in educational
enquiry (2nd ed.). Continuum International Pub. Group.
Senge, P. (1990). The leader’s new work: Building learning organizations. Sloan Management
Review, 32(1), 7-23.
Shephard, D. D. (2014). Nonformal education for improving educational outcomes for street
children and street youth in developing countries: A systematic review. International
Journal of Social Welfare 23(4), 349-361. https://doi.org/10.1111/ijsw.12080
Son, H. Y. (2013). Inequality of human opportunities in developing Asia. Asian Development
Review 30(2), 110-130. https://doi.org/10.2139/ssrn.2103001
152
Stanton-Salazar, R. (2011). A social capital framework for the study of institutional agents and
their role in the empowerment of low-status students and youth. Youth & Society, 43(3),
1066–1109. https://doi.org/10.1177/0044118X10382877
Steer, L., Gillard, J., Gustafsson-Wright, E., & Latham, M. (2015). Non-state actors in
developing countries: A framing paper for discussion. Washington, DC: Brookings.
Tan, E. A. (2017). Quality, inequality, and recent education reform. The Philippine Review of
Economics 54(2), 110-137.
Thompson, E. J. D. (2001). Non-formal education in urban Kenya: Findings of a study in
Kisumu, Mombasa and Nairobi. German Agency for Technical Co-operation.
Tomaševski, K. (2001). Human rights obligations: Making education available, accessible,
acceptable and adaptable. Right to Education Primers, 3. https://www.right-to-
education.org/sites/right-to-education.org/files/resource-
attachments/Tomasevski_Primer%203.pdf
Tracy, S. (2010). Qualitative quality: Eight “big-tent” criteria for excellent qualitative
research. Qualitative Inquiry, 16(10), 837–851.
https://doi.org/10.1177/1077800410383121
Tzuriel, D. (2012). Mediated learning experience (MLE) and cognitive modifiability. In
Encyclopedia of the Sciences of Learning. Boston, MA: Springer.
UNESCO (2007). A human-rights based approach to Education for All. United Nations
Educational, Scientific, and Cultural Organization.
UNESCO. (2015). Education for All 2000-2015: Achievements and challenges. United Nations
Educational, Scientific, and Cultural Organization.
UNESCO. (2018). What you need to know about the right to education.
https://en.unesco.org/news/what-you-need-know-about-right-education
153
UNESCO Bangkok, Asia-Pacific Regional Bureau for Education. (2015). Regional education for
all (EFA) report: Asia and Pacific: 2015 (UNESCO-Bangkok Publication No.
TH/APL/15/009-300).
UNESCO Institute for Statistics. (2012). The international standard classification of education
2011. UNESCO Institute for Statistics.
http://uis.unesco.org/sites/default/files/documents/international-standard-classification-
of-education-isced-2011-en.pdf
UNESCO Institute for Statistics. (n.d.). Glossary: Out-of-school children.
http://uis.unesco.org/en/glossary-term/out-school-children
United Nations. (1989). Convention on the Rights of the Child. United Nations.
United Nations. (2017). Quality education: Why it matters. https://www.un.org/
sustainabledevelopment/wp-content/uploads/2017/02/4.pdf
Veenman, M., Van Hout-Wolters, B., & Afflerbach, P. (2006). Metacognition and learning:
Conceptual and methodological considerations. Metacognition and Learning, 1(1), 3–14.
https://doi.org/10.1007/s11409-006-6893-0
Vescio, V., Ross, D., & Adams, A. (2008). A review of research on the impact of professional
learning communities on teaching practice and student learning. Teaching and Teacher
Education, 24(1), 80–91.
Waterman, R., Peters, T. & Phillips, J. (1980). Structure is not organization. Business Horizons
23(3), 14-26.
World Bank. (2018). Philippines: Support for basic education reform, 2006-2012 (English).
World Bank Group. https://documents.worldbank.org/curated/
en/764931528917588989/Philippines-support-for-basic-education-reform-2006-2012
154
World Bank. (2018). The Philippines alternative learning system: A second chance to develop
the human capital of out-of-school youth and adults.
https://www.worldbank.org/en/country/philippines/publication/the-philippines-
alternative-learning-system-a-second-chance-to-develop-the-human-capital-of-out-of-
school-youth-and-adults
World Health Organization. (2000). Module 2: Responsibilities of street educators. In Working
with street children: Training package on substance abuse, sexual and reproductive
health including HIV/AIDS and STDs (WHO/MSD/MDP/00.14).
https://apps.who.int/iris/bitstream/handle/10665/66756/WHO_MSD_MDP_00.14_Modul
e2.pdf?sequence=3&isAllowed=y
Yasunaga, M. (2014). Non-formal education as a means to meet learning needs of out-of-school
children and adolescents. http://ais.volumesquared.com/wp-
content/uploads/2015/01/OOSC-2014-Non-formal-education-for-OOSC-final.pdf
Yough, M., & Anderman, E. (2006). Goal orientation theory. http://www.
education.com/reference/article/goal-orientation-theory/
Yukl, G. (1981). Leadership in organizations (4
th
ed.). Prentice Hall.
155
Appendix A
Interview Protocol
Good morning/afternoon/evening [Respondent Name]. My name is Kevin Cross and I am a
doctoral student at the University of Southern California. The purpose of my research is to
understand how Central Visayas Children Center’s (CVCC) supplemental non-formal education
program, LEAP, is helping CVCC children succeed in traditional schools. Thank you again for
agreeing to discuss your work at CVCC. While I am conducting this research to support my
dissertation for the University of Southern California, I hope to make the information I gather
available to educators globally to support their efforts in providing quality education to
marginalized children.
As you may remember from our initial conversation, I am planning to record today’s
interview. During the interview, I am going to ask you a series of questions about your
experiences working with the LEAP program. If you do not want to answer a question, please let
me know and we can skip that question. You may also ask me to pause or stop the recording at
any point during the interview. Your participation is voluntary and you can withdraw from the
study at any time. There are no consequences for withdrawing from the study at any time. While
your participation is extremely valuable to my research, I want to ensure that you feel
comfortable during the interview and understand how I plan on using the information you
provide to me.
The information I collect during our interview will help me understand the strategies that
LEAP team members use to help marginalized children improve their academic performance in
formal schools. As a researcher, my role is to learn from your experience and expertise. At the
same time, I want to protect your privacy. Your name will not be associated with the information
you provide to me. A pseudonym will be used in any reference I make to information that you
156
provided to ensure your confidentiality. Following the interview, the recording of your voice will
be transcribed into written format and the recording will be deleted. Finally, all the information
associated with my research will be securely stored to ensure your privacy and the confidentiality
of your responses.
I am providing you two copies of a letter that provides the same information I just shared
with you. One copy is for you to keep and the second is for my records to acknowledge that I
shared this information with you. I would ask that you sign the second copy to acknowledge that
we have discussed your rights as a participant in the study. Please take a moment to review the
letter and let me know if you have any questions.
Interview Questions
I would like to start the interview by asking you a few questions about your typical workday and
experiences as a LEAP team member.
Table 1
Experiential and Motivational Questions
Interview Question RQ/KMO
Influence
Type of Question
1. What factors influenced your
decision to accept a position at
CVCC?
Probe: What aspect of your position
at CVCC do you find most
interesting?
RQ1, M1, M2 Values/opinion
question (Patton,
2002)
157
Probe: What aspect of your job at
CVCC do you find least interesting?
Probe: If you were to rank the tasks
you perform at CVCC, which would
you list as the most important?
Probe: If you were to rank the tasks
you perform at CVCC, which would
you list as the least important?
2. Describe your role in LEAP.
Probe: Describe the types of day-to-
day activities that you typically
perform as part of the LEAP team.
Probe: How much time do you
estimate you spend working on
educational activities each day?
RQ1, M1, M2 Experience/behavior
question (Patton,
2002)
3. Walk me through a typical
educational interaction between you
and the children.
RQ1, K3 Experience/behavior
question (Patton,
2002)
4. Tell me about your experiences in
teaching children prior to joining
CVCC.
RQ1, K1, K2, K3,
K4
Experience/behavior
question (Patton,
2002)
158
5. Describe the most rewarding aspect
of your work with the LEAP
program.
RQ1, M1, M2 Opinion/value
question (Patton,
2015)
6. Describe the most challenging aspect
of your work with the LEAP
program.
RQ1, M1, M2 Opinion/value
question (Patton,
2015)
This next set of questions focuses on the academic achievement of the children you serve as a
LEAP team member.
Table 2
Student Achievement Questions
Interview Question RQ/KMO
Influence
Type of Question
7. How do you define academic
success for CVCC’s children?
RQ1, RQ2, O1
Value/opinion
(Patton, 2015)
8. Walk me through the steps used to
place a child in one of the local
schools.
Probe: Describe your interactions
with the local school systems
regarding the placement of children
who were previously out of school.
RQ1, K1, K2 Experience/behavior
(Patton, 2015)
159
9. How do you determine a child's
educational needs when they first
join LEAP?
RQ1, K1, K2 Experience/behavior
(Patton, 2015)
10. How do you measure a child's
educational progress?
Probe: What criteria do you use to
evaluate academic performance?
Probe: How frequently do you
review a child’s academic
performance?
RQ1, O1, O2, O3 Experience/behavior
(Patton, 2015)
11. Describe a scenario where a CVCC
child demonstrated improved
academic achievement in formal
schools.
Probe: What do you believe the key
factors were that contributed to the
child’s success?
RQ1, K1, K2, K3,
K4
Experience/behavior
(Patton, 2015)
12. Tell me about a time when a CVCC
child struggled in school or
demonstrated decreased academic
achievement in formal schools.
RQ1, K1, K2, K3,
K4
Experience/behavior
(Patton, 2015)
160
Probe: What factors do you believe
influenced the child’s reduced
performance?
13. How are children enrolled in LEAP
similar or different to other
children?
Probe: [Only if respondent indicates
differences]. Do those differences
influence their performance in
school and, if so, how?
Probe: [Only if respondent indicates
differences and resultant
performance influences]. Describe
the ways that LEAP team members
RQ1, K1, K2 Value/opinion
(Patton, 2015)
14. Some people believe that
marginalized children are destined to
underachieve. How do you feel
about that statement?
RQ1, K1
Devil’s advocate
question (Strauss,
Schatzman, Bucher,
and Sabshin, 1981)
The next several questions relate to your teaching approach and will help me understand your
approach to teaching.
161
Table 3
Curriculum Design Questions
Interview Question RQ/KMO
Influence
Type of Question
15. Walk me through your process of
developing a lesson plan for an
educational session.
RQ1, K1, K2, K3,
K4, M1
Knowledge question
(Patton, 2015)
16. Tell me about a time, if ever, that
you modified an aspect of your
instructional approach.
Probe: What led you to make those
changes?
RQ1, K4, M2 Experience/behavior
question (Patton,
2015)
17. How often, if ever, do you have
opportunities to compare
instructional approaches with other
educators?
RQ2, K4, M2, O3 Experience/behavior
question (Patton,
2015)
Finally, I would like to ask you a few final questions about the ways, if any, you feel CVCC can
help children succeed in school.
162
Table 4
Organizational Culture and Context Questions
Interview Question RQ/KMO
Influence
Type of Question
18. What are the key barriers, if any, to
CVCC children accessing
educational support?
RQ2, K1, O1, O2,
O3, O4
Opinion/value
question (Patton,
2015)
19. What are the most valuable
resources, if any, for supporting
CVCC children’s academic
achievement??
RQ1, K2, M1, O4 Knowledge question
(Patton, 2015)
20. What additional resources, if any,
are essential to helping CVCC
children achieve academic success?
RQ2, O1, O2, O3 Opinion/value
question (Patton,
2015)
21. Describe your interactions, if any,
with the Department of Social
Welfare and Development (DSWD).
Probe: What measures do you feel
the DSWD could implement to
improve the academic performance
of CVCC children?
RQ2, O3, O4 Experience/behavior
(Patton, 2015)
22. Describe your interactions, if any,
with the local schools.
RQ2, O3, O4 Experience/behavior
(Patton, 2015)
163
Probe: What measures do you feel
the local schools could implement to
improve the academic performance
of CVCC children?
23. Is there anything else you feel we
should have discussed during this
interview?
All Open-ended
question to allow
respondent to
elaborate on earlier
responses or identify
additional issues.
Thank you again for participating in this interview and taking the time to share your experiences
as a street educator with CVCC.
164
Appendix B
Focus Group Interview Protocol
Good morning/afternoon/evening. My name is Kevin Cross and I am a doctoral student at the
University of Southern California. The purpose of my research is to understand how Central
Visayas Children Center’s (CVCC) supplemental non-formal education program, LEAP, is
helping CVCC children succeed in traditional schools. Thank you again for agreeing to discuss
your work with CVCC’s students. While I am conducting this research to support my
dissertation for the University of Southern California, I hope to make the information I gather
available to educators globally to support their efforts in providing quality education to
marginalized children.
As you may remember from our initial conversation, I am planning to record today’s
focus group session. During the session, I am going to ask you a series of questions about your
experiences working with CVCC students. You may choose not to respond to a question or ask
me to pause or stop the recording at any point during the session to provide a response that you
would not like recorded. Your participation is voluntary and you can withdraw from the study at
any time. There are no consequences for withdrawing from the study at any time. While your
participation is extremely valuable to my research, I want to ensure that you feel comfortable
during the session and understand how I plan on using the information you provide to me.
The information I collect during our discussion will help me understand the strategies
that CVCC, local schools, and the Department of Social Welfare and Development use to help
marginalized children improve their academic performance in formal schools. As a researcher,
my role is to learn from your experience and expertise. At the same time, I want to protect your
privacy. Your name will not be associated with the information you provide to me. A pseudonym
will be used in any reference I make to information that you provided to ensure your
165
confidentiality. Following the focus group session, the recording will be transcribed into written
format and deleted. Finally, all the information associated with my research will be securely
stored to ensure your privacy and the confidentiality of your responses.
I am providing you two copies of a letter that provides the same information I just shared
with you. One copy is for you to keep and the second is for my records to acknowledge that I
shared this information with you. I would ask that you sign the second copy to acknowledge that
we have discussed your rights as a participant in the study. Please take a moment to review the
letter and let me know if you have any questions.
166
Focus Group Interview Questions
The following focus group interview questions will be administered during the 90-minute focus
interview session.
Table 1
Focus Group Questions
Interview Question RQ/KMO
Influence
Type of Question
1. In what ways do you feel that CVCC
students are similar or different from
other students?
Probe: [Ask only if respondents
indicate differences. Otherwise, skip
to Question 2]. In what ways, if any,
do you feel any differences influence
their performance in formal schools?
RQ1, K1, O1, O3 Value/opinion (Patton,
2015)
2. Based on CVCC’s descriptive
summary statistics on CVCC
students’ academic achievement,
how do you feel that CVCC children
perform in comparison to other
students in formal schools?
Optional Probe: How would you
explain any similarities or
RQ1, RQ2, K1, K2,
M2
Value/opinion (Patton,
2015)
167
differences in academic
achievement?
3. Describe the process a CVCC
student would use to get support if
they were having difficulty at school.
Optional Probe: Tell me about a
time when a student used a non-
formal education program to support
their formal schoolwork.
RQ1, RQ2, K1, K2,
K3, K4, O2, O3
Experience/behavior
question (Patton, 2002)
4. Describe the process that you use, if
any, to identify and support students
who are struggling academically.
RQ1, RQ2, K1, K2,
K3, K4, O2, O3
Experience/behavior
(Patton, 2015)
5. What supplemental non-formal
educational strategies, if any, do you
believe are most and least effective
in improving CVCC students’
academic achievement in formal
school systems?
RQ1, RQ2, K1, K2,
K3, K4, O2, O3, O4
Value/opinion (Patton,
2015)
6. Describe the ways in which CVCC,
the local schools, and the regional
Department of Social Welfare and
Development work together to
RQ2, O3, O4 Experience/behavior
(Patton, 2015)
168
Thank you again for participating in this focus group and taking the time to share your
experiences with CVCC’s students.
promote improved academic
achievement for CVCC children.
169
Appendix C
Central Visayas Children’s Center Student Survey: Q3 2020
Question Responses
Q1. What grade
are you in?
(Grade 1-12)
☐ Grade One
☐ Grade Two
☐ Grade Three
☐ Grade Four
☐ Grade Five
☐ Grade Six
☐ Grade Seven
☐ Grade Eight
☐ Grade Nine
☐ Grade Ten
☐ Grade Eleven
☐ Grade Twelve
Instructions: Please rate how certain you are that you can do each of the things described below by
checking the appropriate checkbox. Check only one box per question.
Task Cannot do
at all
1
2
Moderately
can do
3
4
Highly
can do
5
I am not
studying this
subject.
Q2. Learn
Filipino.
☐ ☐ ☐ ☐ ☐ ☐
Q3. Learn
English.
☐ ☐ ☐ ☐ ☐ ☐
Q4. Learn
Mathematics.
☐ ☐ ☐ ☐ ☐ ☐
Q4. Learn
Science.
☐ ☐ ☐ ☐ ☐ ☐
Q6. Learn Araling
Panlipunan
(Social
Studies).
☐ ☐ ☐ ☐ ☐ ☐
Q7. Finish my
homework
assignments
by deadlines.
☐ ☐ ☐ ☐ ☐
Q8. Get myself to
study when
there are
other
interesting
things to do.
☐ ☐ ☐ ☐ ☐
Q9. Always
concentrate
on school
subjects
during class.
☐ ☐ ☐ ☐ ☐
Q10. Take good
notes during
class
instruction.
☐ ☐ ☐ ☐ ☐
170
Q11. Use the
library to get
information
for class
assignments.
☐ ☐ ☐ ☐ ☐
Q12. Plan my
schoolwork
for the day.
☐ ☐ ☐ ☐ ☐
Q13. Organize my
schoolwork.
☐ ☐ ☐ ☐ ☐
Q14. Remember
well
information
presented in
class and
textbooks.
☐ ☐ ☐ ☐ ☐
Q15. Arrange a
place to
study
without
distractions.
☐ ☐ ☐ ☐ ☐
Q16. Get myself to
do
schoolwork.
☐ ☐ ☐ ☐ ☐
Q17. Get teachers
to help me
when I need
help with my
homework.
☐ ☐ ☐ ☐ ☐
Q18. Get another
student to
help me
when I need
help with my
homework.
☐ ☐ ☐ ☐ ☐
Q19. Get a house
parent to
help me
when I need
help with my
homework.
☐ ☐ ☐ ☐ ☐
Q20. Get a tutor to
help me
when I need
help with my
homework.
☐ ☐ ☐ ☐ ☐
Q21. Live up to
what my
☐ ☐ ☐ ☐ ☐
171
teachers
expect of me.
Q22. Live up to
what my
peers expect
of me.
☐ ☐ ☐ ☐ ☐
Q23. Live up to
what I expect
of myself.
☐ ☐ ☐ ☐ ☐
Q24. Express my
opinions
when other
classmates
disagree with
me
☐ ☐ ☐ ☐ ☐
Q25. Stand up for
myself when
I feel I am
being treated
unfairly
☐ ☐ ☐ ☐ ☐
Q26. Get others to
stop
annoying me
or hurting my
feelings
☐ ☐ ☐ ☐ ☐
Q27. Stand firm to
someone
who is asking
me to do
something
unreasonable
or
inconvenient
☐ ☐ ☐ ☐ ☐
Q28. Rank your
favorite
subjects from
1-5
(1=Favorite,
2=Next most
favorite …
5=Least
favorite)
____ Araling Panlipunan
____ English
____ FiIipino
____ Mathematics
____ Science
Q29. Rank each
subject from
hardest to
easiest
(1=Most
difficult,
____ Araling Panlipunan
____ English
____ FiIipino
____ Mathematics
____ Science
172
2=Next most
difficult …
5=Least
difficult)
Q30. What grade
do you
anticipate
receiving in
each of the
following
subjects on
your final
report card
(0-100)? (If
you are not
taking one of
the subjects,
please place
an “X” next
to the
subject).
_____ Araling Panlipunan
_____ English
_____ FiIipino
_____ Mathematics
_____ Science
173
Appendix D
Survey to Administer Immediately Following the Training Session
(Level 1 and 2 Blended Assessment)
For questions 1 through 7, please place an “X” in the one box that best describes your level of
agreement with each statement.
Strongly
disagree
Disagree Neither
agree nor
disagree
Agree Strongly
agree
1. This program held my
interest.
2. I actively participated
in the training session.
3. The facilitator’s
presentation style
contributed to my
learning experience.
4. The presentation
content was easy to
follow.
5. The presentation
content was applicable
to my work as a LEAP
Team member.
6. The presentation
content will help me
support CVCC students
better.
7. I am clear about what
is expected of me as a
LEAP Team member
as a result of taking this
training.
174
The following questions ask you to rate your before and after the training session. Each question
requires a response for your rating prior to the training session and after the training session.
For questions 8 through 14, please use the following rating scale:
1.
None or very
low level
2.
Low level
3.
Moderate level
4.
High level
5.
Very high level
Before training
session
After training
session
1 2 3 4 5 8. Knowledge of how to access and
use student performance and
CVCC survey data.
1 2 3 4 5
Comments:
1 2 3 4 5 9. Knowledge of how to identify
student performance problems in
formal schools.
1 2 3 4 5
Comments:
1 2 3 4 5 10. Knowledge of how to help
students perform better in school.
1 2 3 4 5
Comments:
1 2 3 4 5 11. Knowledge of how to design and
develop effective NFE programs.
1 2 3 4 5
Comments:
1 2 3 4 5 12. Knowledge of how to design,
develop, and deliver effective
instructional sessions.
1 2 3 4 5
175
Comments:
1 2 3 4 5 13. Confidence in ability to develop
the LEAP.
1 2 3 4 5
Comments:
1 2 3 4 5 14. Commitment to implementing the
LEAP.
1 2 3 4 5
Comments:
15. What were the most useful aspects of this training session?
16. Where do you feel that the training could have done a better job of supporting your role as a
LEAP Team member?
17. Please share two ways that you plan on using the information you learned today within the
next two weeks within your current role as a LEAP Team member.
18. What topics would be most useful to include in the monthly meetings of the LEAP Team’s
professional learning community?
19. Please provide us with any additional feedback regarding the training session.
176
Appendix E
Survey to Administer Following Each Quarterly PLC Session
(Level 1 through 4 Delayed Assessment)
Please place an “X” in the one box that best describes your level of agreement with each
statement.
Strongly
disagree
Disagree Neither
agree nor
disagree
Agree Strongly
agree
1. This PLC session
held my interest.
2. This PLC session
was relevant to my
role as a member of
the LEAP team.
3. I actively
participated in this
PLC session.
4. We have increased
the use of student
performance and
survey data in
diagnosing student
performance
challenges.
5. We have been
successful at
identifying students
in need of
supplemental NFE
support.
6. We have developed
individual
educational plans for
all CVCC students.
177
7. We have developed
a curriculum to
support Araling
Panlipunan.
8. We have developed
a curriculum to
support English
language studies.
9. We have developed
a curriculum to
support Filipino
language studies.
10. We have developed
a curriculum to
support
Mathematics.
11. We have developed
a curriculum to
support Science.
12. We are effective at
matching students
with NFE support
resources.
13. We have witnessed
measurable
improvements in
student academic
performance.
14. We have been
successful at
identifying students’
motivational
challenges.
15. We have been
successful
addressing students’
motivational
challenges.
178
16. We have been
successful in
partnering with local
schools and the
DSWD in
supporting CVCC
students.
17. Our students see the
value in the LEAP.
18. We are confident
that we can
implement the
LEAP.
19. I personally see the
value of the LEAP.
20. I personally see the
importance of my
role in implementing
the LEAP.
21. I am committed to
implementing the
LEAP at CVCC.
Abstract (if available)
Linked assets
University of Southern California Dissertations and Theses
Conceptually similar
PDF
An evaluation of project based learning implementation in STEM
PDF
Measuring the impact of short-term campus exchange programs: an evaluation study
PDF
Compliance and regulatory efficacy and sustainability in specialty academic medicine: a longitudinal evaluation study
PDF
Federal grant approval at a state education agency: an evaluation study
PDF
Online graduate-level student learning and engagement: developing critical competencies for future leadership roles: an evaluation study
PDF
Quality literacy instruction in juvenile court schools: an evaluation study
PDF
Civic learning program policy compliance by a state department of higher education: an evaluation study
PDF
Implementing comprehensive succession planning: an improvement study
PDF
Sustained mentoring of early childhood education teachers: an innovation study
PDF
Raising special needs: an evaluation study of respite care for medically fragile children living with autism and other illnesses
PDF
Increasing post-secondary education among police officers: an innovation study
PDF
Performance management in government: the importance of goal clarity
PDF
Affluent teens and school stress: an evaluation study
PDF
Health care disparities and the influence of nurse leader cultural competency: an evaluation study
PDF
Impact of mindfulness on early education teacher well-being: an evaluation study
PDF
Teacher retention influences: an evaluation study
PDF
High attrition rate of preschool teachers in Hong Kong: an evaluation study
PDF
Organizational agility and agile development methods: an evaluation study
PDF
Evaluation study: building teacher efficacy in K8 computer science integration
PDF
One to one tablet integration in the mathematics classroom: an evaluation study of an international school in China
Asset Metadata
Creator
Cross, Kevin Eric
(author)
Core Title
Efficacy of non-formal education programs in educational outcomes of marginalized Filipino children: an evaluation study
School
Rossier School of Education
Degree
Doctor of Education
Degree Program
Organizational Change and Leadership (On Line)
Publication Date
11/01/2020
Defense Date
10/07/2020
Publisher
University of Southern California
(original),
University of Southern California. Libraries
(digital)
Tag
alternative education,education for all,educational access,equity and quality,human rights,non-formal education,OAI-PMH Harvest,out-of-school children and youth,sustainable development goals
Language
English
Contributor
Electronically uploaded by the author
(provenance)
Advisor
Min, Emmy (
committee chair
), Datta, Monique (
committee member
), Krop, Cathy (
committee member
)
Creator Email
kcross3@jhu.edu,kevinecr@usc.edu
Permanent Link (DOI)
https://doi.org/10.25549/usctheses-c89-390135
Unique identifier
UC11665956
Identifier
etd-CrossKevin-9093.pdf (filename),usctheses-c89-390135 (legacy record id)
Legacy Identifier
etd-CrossKevin-9093.pdf
Dmrecord
390135
Document Type
Dissertation
Rights
Cross, Kevin Eric
Type
texts
Source
University of Southern California
(contributing entity),
University of Southern California Dissertations and Theses
(collection)
Access Conditions
The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the a...
Repository Name
University of Southern California Digital Library
Repository Location
USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA
Tags
alternative education
education for all
educational access
equity and quality
non-formal education
out-of-school children and youth
sustainable development goals