University of Southern California Dissertations and Theses

Practitioner reflections and agency in fostering African American and Latino student outcomes in STEM

(USC Thesis Other)

Practitioner reflections and agency in fostering African American and Latino student outcomes in STEM

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 1

Practitioner Reflections and Agency in Fostering African American and
Latino Student Outcomes in STEM

by
Lorena Patton

A Dissertation Presented to the
FACULTY OF THE USC ROSSIER SCHOOL OF EDUCATION
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulfillment of the
Requirements for the Degree
DOCTOR OF EDUCATION

May 2014

Copyright 2014 Lorena Patton

STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 2

Dedication
I dedicate this dissertation to my Lord and Savior Jesus Christ. With him and through
him all things are possible. This journey was not an easy one; it required much faith and
patience. I thank the Lord for never leaving my side in the midst of it all. There were days I
wanted to give up, but he saw fit that I see it through unto completion. For that, I say thank you.

STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 3

Acknowledgements
I would like to acknowledge my uncle, Floyd Smith, for always believing in me and
being my biggest supporter. I would like to thank him for helping me accomplish major
milestones in my life and for never doubting my ability to succeed.
I would also like to thank my mother, Bridgett Smith, for being the best mom anyone
could ever have. She worked tirelessly to take care of five children and made sure we were all
educated. To my siblings, Dock, Bridgett, Lisa, and Karmen, I am grateful to have been blessed
with such a beautiful family. You guys believed in me when I didn’t believe in myself and
encouraged me to keep going. For that I say thank you.
I would also like to thank my mentor Dr. Sharma Henderson. Whether we were near or
far, you were always just a phone call or text message away. When I wanted to give up, you
gave me words of encouragement and were never too busy to lend your ear. To my best friend,
Dr. LaTesha Hagler, I thank God for sending me such an awesome friend like you. I will never
forget taking this journey with you.
To my thematic group, Erin, Svetlana, Rashitta, Chelvi, Christiane, Peggy, Tomas and
Leeann, thank you for all of your support and friendship.
Lastly, I would like to thank my chair, Dr. Alicia Dowd, for your patience, support, and
commitment to my academic success. Thank you for never giving up on me and never allowing
failure to be an option for me. To my co-chair and committee member, Dr. Estela Bensimon and
Dr. Robert Rueda, thank you for all of your guidance during this process. Your feedback has
made me a stronger student, writer, and working professional. I am grateful to your contribution
to higher education and look forward to working with you in the future.
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 4

Table of Contents
Dedication 2
Acknowledgements 3
List of Tables 6
List of Figures 7
Abstract 8

Chapter One: Introduction 9
Overview of the Study 9
Background of the Problem 12
Accountability for Latino Participation in STEM 14
Challenges in Accountability 15
Accountability in the Context of California 16
Measuring Institutional Accountability and Effectiveness 19
Statement of the Problem 21
Purpose of Study 21
Significance of Study 22
Definition of Terms 24
Chapter Two: Theoretical Framework 26
Practitioners’ Beliefs 27
How Practitioner Beliefs Impact Racial-Ethnic Minorities 29
The Equity Scorecard™ as Part of Action Inquiry 32
Organizational Learning and Cultural Historical Activity Theory 33
Institutional Agents as Drivers of Organizational Change 44
Chapter Three: Methods 48
Developmental Evaluation in Organizational Change 48
Developmental Evaluation Research at DCC 51
Action Research Driving Inquiry 52
Addressing Accountability and Assessment Measures 53
Action Inquiry at Dynamic Community College 55
Data Collection 56
Participant Demographics: CUE Action-Inquiry Study 56
Participant Demographics: DCC Action-Inquiry Study 59
Unit of Analysis 62
Action-Inquiry Activities and Data Collection 64
Data Collection Method 70
Data Analysis 71
Debriefing and Cognitive Interviews for Data Collection 75
Data Analysis Procedures 76
Code Categories 78
Attitudes/Beliefs 78
Knowledge 79
Social Interaction 79
Reflection 79
Problem Identification 80
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 5

Experimentation/Problem Solving 80
Action or Experience 80
Uncoded Data 80
Data Reporting 80
Ethical Concerns 83
Standards of Review 84
Credibility 84
Transferability 89
Dependability and Confirmability 91
Limitations 91
Reporting Results 94
Chapter 4: Findings 96
Settings and Participants 97
Participant Engagement 99
Participant Satisfaction 101
Valuing Dialogue 102
Purposeful Data Use 105
Engaging in “The Cycle of Inquiry” 109
Questioning Practices as a Result of Action Inquiry 111
Supporting Students 113
Raising Awareness of Equity-Minded Practices 117
Examining Beliefs Regarding Equity 121
Steps toward Organizational Change 125
Prompting Organizational Change through Equity Inquiry 128
Perceived Control 131
Organizational Change: Accountability for
Institutional Improvement 135
Lack of Institutional Structures to Ongoing Inquiry 137
Discussion 139
Chapter 5: Recommendations 141
The Cycle of Inquiry and Practitioner Knowledge 141
Connecting Theory and Practice 141
Sustaining Change 143
Recommendations 145
Define Expectations for Accountability 145
Align Program Objectives with Institutional Mission 146
Expose Campuses to Equity-Oriented Artifacts Early 148
Create Opportunities on Campus to Discuss Ways to
Lessen Racial-Ethnic Achievement Gaps 148
Strengthen Faculty Buy-In Final Recommendation 150
Personal Recommendations for CUE and Similar
Action Researchers 151
References 152
Appendix A 164
Appendix B 165
Appendix C 167
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 6

List of Tables
Table 1: CUE’s Developmental Evaluation S tudy Field Sites 57
Table 2: Timeline of Activities at DCC 61
Table 3: Participation at Each Stage 62
Table 4: CUE’s Proposed Roles and Activities at DCC 68
Table 5: Timeline for 2012-2013 Activities at DCC 69
Table 6: Summary of Data Collection Methods 72
Table 7: Deductive Data Analysis Codes 78
Table 8: Asset-Mapping Protocol
TM
127

STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 7

List of Figures
Figure 1 Developmental Evaluation Methods Inform Multiple Modes
of Research 52

Figure 2: Reflective Practice (Inquiry) as Driver of Racial-Ethnic Equity in
Postsecondary 54

Figure 3: Participant Engagement Levels 99

Figure 4: Effectiveness of CUE’s Facilitators 100

Figure 5: Participant Satisfaction with Topics 101

Figure 6: Cycle of Inquiry 111

Figure 7: Dynamic Community College, BESST Data 120

Figure 8: Beliefs about Campus Climate Surrounding Equity 123

Figure 9: Impact of STEM Symposium on Equity Awareness 124

Figure 10: Practitioner Beliefs Regarding Personal Responsibility to Impact
Equity 125

Figure 11: Asset-Mapping Protocol
TM
126

Figure 12: Symposium: Practitioner Beliefs Regarding the Impact of CUE’s
Tools on Influencing Equity 129

Figure 13: Practitioner Perceived Control Surrounding Equity 132

STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 8

Abstract
This study examined the experience of practitioners while engaged in equity-focused inquiry at
Dynamic City College (DCC), a public 2-year California institution. The study investigated
whether or not the tools developed by the Center for Urban Education (CUE) at the University of
Southern California raised equity-minded practices among study participants and prompted
organizational change. The case study revealed that increasing racial-ethnic equity in STEM
Outcomes at Dynamic City College would have required faculty buy-in, pre-exposure to equity
oriented artifacts, collective agency, and ongoing efforts to maintain on-campus dialogue
regarding racial-ethnic inequities.

STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 9

Chapter One: Introduction
Overview of the Study
Institutional practices that hinder the academic success of racial-ethnic minorities are
common at many institutions of higher education. Some institutions are not aware that they are
maintaining a campus culture that is set up for the failure of racial-ethnic minorities. Institutions
often make sure that they adhere to federal and state regulations, but little is done for the
subpopulations of students who struggle with adjusting to college and university life (Bowen,
Chingos, & McPherson, 2009).
The number of minority graduates in Science, Technology, Engineering and Mathematics
(STEM) fields, which will be a particular focus of this study, is even more disheartening. Major
reports show that minority participation in higher education has more than doubled over the past
20 years (Dowd, 2011). Even with this increase in participation, minority completion of STEM-
focused degrees remains relatively low. In 2007, congress directed the National Science
Foundation (NSF) to develop a Hispanic Serving Institutions Undergraduate Initiative “with the
goal of increasing Latina and Latino degree completion in science, technology, engineering and
mathematics (STEM) fields” (Dowd, Malcom, & Bensimon, 2009, p. 1). In addition, national
leaders recognized the importance of equity in STEM and continued to support and fund
initiatives that emphasized minority representation and equality in STEM. In March 2010, the
Health Care and Education Reconciliation Act (HCERA) signed by President Barack Obama
approved a billion dollar investment for Hispanic Serving Institutions (HSIs) to increase degree
completion in STEM fields in hopes of producing more Latino scientists, engineers, and
mathematicians (Malcom, Dowd, & Yu, 2010).
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 10

An overarching problem that exists in higher education is that systems are prepared to
teach a certain type of student and racial-ethnic minorities often do not fit this profile. The
higher education system is designed for students of the dominant culture and is not always
sensitive to the needs of racial-ethnic minorities or the socially and economically
disenfranchised. Bensimon (2007) stated that “Inequalities in educational outcomes for
historically underserved groups have been one of the most urgent and intractable problems in
higher education” (p . 99). This statement highlighted scholars’ recognition of disparities.
Existing research supports scholars’ efforts to keep racial-ethnic inequalities at the forefront of
higher education. The persistence of these inequalities has in part led to the rise of a new
accountability agenda and a number of national and state initiatives. There has been a great
demand to create systems of accountability that use evidence to guide and support the
development and implementation of new policies and practices (Dowd & Bensimon, 2009a). As
these systems are created and implemented, research must be conducted to monitor and assess
what is effective.
There also has been a massive push by the federal government to hold institutions of
higher education accountable for student success by measure of degree completion. President
Obama’s goal is that America will regain lost ground by having the highest proportion of
students graduating from college in the world by 2020 (The Whitehouse, n.d.). The national goal
is that all Americans be prepared to enroll in at least 1 year of higher education or job training
after high school to better prepare America’s workforce for the 21st century ( The Whitehouse,
n.d.). Racial-ethnic minorities will be impacted by this goal because they currently enter college
and the workforce at disproportionately lower rates. As previously mentioned, racial-ethnic
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 11

minorities are dramatically underrepresented in the fields of computer science, engineering, and
technology (Grubb & Badway, 2005).
Despite accountability efforts on the national and state levels, inequities in higher
education access, experiences, and completion rates still persist. Some scholars argued that
institutions of higher education often uphold policies and practices that can be harmful to
students of color (Bensimon, 2007). For example, transfer pathways ending in degree
completion are not always clear and accessible to all students. In community colleges in
particular, students who test below college-level mathematics and English upon entry are
required to complete several basic skills courses. These courses are not always transferrable
between campuses and racial-ethnic minorities often get stuck in the pipeline. For many
campuses, basic skills courses are impacted and over flowing with racial-ethnic minorities
(Dowd, Bensimon, Watford, & Malcolm, 2008). Sometimes students aren’t able to enroll from
one semester to the next because sequential courses are full. This is an example of existing
organizational structures that impact student success rates and may reflect poor institutional
policies.
Other scholars criticized current accountability and assessment practices for being
ineffective in eliminating inequities and in meeting Obama’s goal (Baldwin, Bensimon, Dowd,
& Kleiman, 2011; Bauman, 2005; Bensimon, 2005; Dowd, 2005). They believed that in order
to improve equity in student outcomes as well as assessment of institutional effectiveness, it is
necessary to focus on organizational learning and a culture of inquiry to bring about
organizational change (Baldwin, et al., 2011). Dowd and Tong (2007) presented evidence-
based inquiry as an accountability structure that can promote effective assessment processes.
Cultures of inquiry allow leaders within an institution to develop their capacity for
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 12

organizational learning by prompting them to actively engage in dialogue regarding
institutional practices that impact student outcomes. Evidence-based inquiry aims to improve
student outcome equity by engaging faculty, administrators, and counselors in institutional data
analysis, reflective inquiry, and self-assessment of instructional practices to help develop the
practical wisdom and professional expertise necessary to recognize and counter discriminatory
practices and address inequities in student outcomes (Dowd & Tong, 2007).
Background of the Problem
Persisting inequalities in educational attainment for historically underserved groups have
shifted the priorities of national stakeholders. Trow (1996 as cited in Dowd & Tong, 2007)
defined accountability as “the obligation to report to others, to explain, to justify, and to answer
questions about how resources have been used and to what effect” ( p. 62). Recent national
accountability indicators require institutions to provide equitable access to underserved
communities, racial-ethnic minorities, and students from low socio-economic statuses.
Initiatives have been put in place requiring institutions to increase services, efforts, and programs
to racial-ethnic minorities. These initiatives also hold institutions of higher education
accountable for producing equitable student outcomes for racial and ethnic minorities and have
provided criteria and benchmarks for institutions so they may gauge their effectiveness.
The way institutions of higher education should be held accountable, however, has been
the topic of significant debate (Dowd & Tong, 2007). This debate on accountability has
prompted questions regarding the state and federal legislation and the role they should play; to
whom colleges are accountable, what they are accountable for, and the measurement guidelines
for establishing such criteria; and whether or not accreditation reviews be made accessible to the
public (Dowd & Tong, 2007). The Spellings Commission report and the Education Testing
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 13

Service advocated for using systems of accreditation and culture of inquiry to increase
accountability in higher education (Dowd & Tong, 2007). Previous accreditation systems in
higher education focused on the assessment of educational processes, but the new accountability
uses the “bottom line” results to create both “cultures of accountability” and “cu ltures of
evidence” (Dowd & Tong, 2007, p. 61)
Federal movements in accountability have begun mandating educational institutions at
both the K-12 and higher educational levels to service their students and their publics in more
effective ways (Grubb & Badway, 2005). Several initiatives have been put in place to address
the inequalities in higher education that exist amongst racial-ethnic minorities. The Equity
Scorecard™ (2010) as designed by the Center for Urban Education identifies gaps in educational
outcomes for racial-ethnic minorities and promotes inquiry into instructional and academic
practices and supports “purposeful changes in practices and policies based on the results of
systematic inquiry, and evaluation of the implemented changes” (para. 3). Other postsecondary
examples include the Making Excellence Inclusive Initiative, COMPASS project, and the Health
Care and Education Reconciliation Act (HCERA). HCERA made “investments in federal
financial aid and Hispanic-Serving Institutions” (Malcom et al., 2010, p. 1) to increase minority
participation in STEM. These initiatives require institutions to increase services, efforts, and
programs designed for racial-ethnic minorities. These initiatives also aim to hold institutions of
higher education accountable for providing access and equality to students of color at the post-
secondary level. This is done by providing institutions with specific criteria and benchmarks so
they may gauge their effectiveness.
The Making Excellence Inclusive initiative as implemented by the Association of
American Colleges and Universities (AAC&U, 2007) is designed to impact student success and
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 14

high quality learning. Under the guidelines of Making Excellence Inclusive, colleges and
universities are assisted to incorporate equity, diversity, and educational quality efforts into their
missions and institutional operations (AAC&U, 2007). The objective of such initiatives is to
increase equity in student outcomes, staff, and resources. For example, the COMPASS project
as part of Making Excellence Inclusive focuses on the educational needs of students who have
historical disadvantages in higher education such as students from low-income families, first
generation students, and racial and ethnic minorities. Initiatives such as Making Excellence
Inclusive help institutions identify effective educational practices that can be sustained over time.
These practices are hoped to substantially increase the graduation rates of racial-ethnic
minorities.
Accountability for Latino Participation in STEM
In 2007, Congress authorized the America COMPETES Act which requires the National
Science Foundation (NSF) to develop an initiative for Hispanic-Serving Institutions (Dowd,
Malcom, Macias, 2010).
This act was in response to the underrepresentation of Latino students in Science,
Technology, Engineering and Mathematics (STEM) fields. The goal of the America
COMPETES Act is to increase baccalaureate degrees for racial-ethnic minorities. Data showed
that Latinos are one of the fastest growing demographic groups and expected to make up about
30% of the US population by 2040 (Dowd et al., 2010). Given the large number of Latino
students enrolled in community colleges and the emerging industry needs in STEM, it is evident
as the workforce becomes more diverse who will occupy these positions. In fact, “Estimates by
the National Science Board (2008) indicate that science and engineering jobs are expected to
grow by 26% from 2004 to 2014, in comparison to a 13% growth in all other occupations”
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 15

(Dowd et al., 2010, p. 1). About 8% of all institutions are Hispanic Serving Institutions (HSIs),
which are defined by the fact that they have at least 25% Hispanic undergraduate full time
equivalent (FTE) student enrollment. Literature showed that Latino students tend to gravitate
toward these institutions; however their outcomes are disproportionate compared to other races
and ethnicities (Dowd et al., 2010). Specifically, the numbers of Latino students who earned
degrees in STEM related fields are disproportionately below the number of Latino students
enrolled and their success outcomes in STEM fields were also disproportionate. There has been
a major push to enhance the way that institutions of higher education serve racial-ethnic
minorities in STEM, specifically Latino students.
Challenges in Accountability
Accountability at the post-secondary level faces many challenges. Grubb and Badway
(2005) discussed the challenges in accountability by means of measuring institutional
effectiveness. State and federal mandates require institutions of higher education to develop
quality assessment and evaluation practices. However, attempts to measure educational process
and outcomes, both qualitatively and quantitatively, have been difficult because institutions have
very different goals, learning conditions, and “multiple potential outcomes” (Grubb & Badway,
2005, p. 4). For example, some institutions may work harder towards improving graduation rates
while placement in basic skills courses, persistence, and transfer rates may warrant significant
attention.
Institutions of higher education also face challenges in accountability when they
aggregate student data. Scholars argued that aggregated data, which is representative of the
entire student body, “hide disparities in stud ent outcomes” (Dowd , 2008, p. 416). Aggregated
data gives the illusion that institutions are performing better than they actually are in producing
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 16

equitable student outcomes. This data reflects the quantity at which institutions are educating
students but does not demonstrate the quality of education provided to students. Data showing
the number of racial-ethnic minorities completing college provides insight to the quality of
education being provided. Bensimon (2005) argued that disaggregating student outcomes by
race and ethnicity could create opportunities for creating equity in educational outcomes for
racial-ethnic minorities. She also stated that institutionalizing the practice of data disaggregation
could assist many colleges and universities in identifying gaps in student achievement early
enough to stop the perpetuation of inequitable student outcomes (Bensimon, 2005).
Accountability in the Context of California
Researchers, policy makers, and educators have started to recognize the major points of
discussion regarding student success at the post-secondary level (Moore & Shulock, 2007). For
example, the California Master Plan (n.d.) put into legislation during the 1960s informs this
discussion. By allowing each segment to create their own standards, colleges and universities
are able to design practices specifically for students of color. With the diversity in higher
education, each institution has its own unique mission. The overarching theme of education and
public policy has been for each university to strengthen efforts made towards accessibility for all
students including underrepresented racial and ethnic minorities.
With federal mandates plus those established by the California Master Plan (n.d.) and
initiatives such as the CSU graduation initiative, there has been a push for racial-ethnic
minorities to succeed at the post-secondary level. The CSU graduation initiative, for example,
was implemented to address the gap in degree attainment that currently exists for racial-ethnic
minorities (The California State University, n.d.). The CSU campuses have aimed “to raise the
freshman six-year graduation rate by eight percentage points” by 2015-2016 (The California
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 17

State University, n.d., Graduation Initiative, para. 1). The goal is to increase degree attainment
by half for racial and ethnic minorities and push CSU campuses to implement graduation rates
that are comparable to national averages of similar institutions. To further address the gaps in
degree attainment for racial and minorities, the CSU graduation initiative requires universities to
carefully plan activities for underrepresented minorities, and discover roadblocks that may
impede student academic achievement. As the CSUs are currently graduating a little over 50%
of students in six years, the initiative is geared toward increasing the number of African
American and Latino students who obtain degrees.
Studies have projected shortages of educated workers within the State of California
unless colleges and universities can increase racial-ethnic minorities’ c ompletion rates. Low-
degree productivity affects the State of California because the future of California’s economy is
linked to the number of residents who enter into college and successfully complete it (Moore &
Shulock, 2007).
Rapid changes in California’s economy require a greater production of educated workers
who are flexible and can easily adapt to sudden changes in the labor market. The number of
degrees awarded to racial-ethnic minorities within the state of California is significantly low
given the total number of college degrees granted by the state each year (Bensimon, Dowd,
Trapp, Alford, 2007).
This inequity in institutions of higher education is compounded since the population
growth is higher in underserved groups (Moore & Shulock, 2007). While California has in the
past relied on other states and countries to contribute to its economy, research conducted by the
Public Policy Institute (PPIC as cited in Johnson & Reed, 2007) stated that California cannot
import enough workers to keep up with the rapid changes in the economy and will soon have to
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 18

rely on its own educated workforce. The need for skilled workers requires California to improve
degree attainment rates. Since the California community colleges house the majority of the
state’s public undergraduates (roughly three -quarters) (Moore & Shulock, 2007), increases in
educated adults by means of degree completion is necessary to maintain the state’s social and
economic health.
With the majority of underrepresented students attending community colleges,
accountability continues to be at the forefront of discussions regarding that sector as well. Grubb
and Badway (2005) referred to community colleges as special institutions within American
higher education. They are considered “open door institutions and the clearest examples of
inclusiveness and equity in a system that is otherwise devoted to competitive entry,
exclusiveness, and elitism” (Grubb & Badway, 20 05, p. 4). This particular division of the higher
education sector prides themselves on being “teaching colleges, in contrast to elite universities
that focus on research” (Grubb & Badway, 2005, p. 4). According to Grubb and Badway,
community colleges have made their missions and goals more expansive.
Accountability at the community-college level has been a growing concern specifically
because the focus of community colleges is very broad. As open-access institutions, community
colleges are challenged because they service very diverse student populations and student goals
are not always easily defined. For example, some students attend community college directly out
of high school, some adults return to advance their careers, other students attend to take
prerequisite courses, etc. Community colleges have the broadest student population, yet they
receive the least amount of funding in comparison with other institutions of higher education and
K-12 schools (Dowd, 2008). They represent the main gateway into higher education for
underrepresented students of color, first generation, and low-income students (Bloom & Sommo,
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 19

2005) yet receive less money per student than both the California State University (CSU) and
University of California (UC) systems (Moore & Shulock, 2007). Even though community
colleges have a broad focus, statistics show that some students have more difficulty navigating
this system than others (Dowd, 2008). According to the Institute for Higher Education Policy,
73% of California college students are enrolled in community colleges; however, only 22.7% of
those that intend to transfer do so successfully (California Community Colleges, 2013, p. 10).
The statistics for transfer rates from 2-year to 4-year colleges are especially disappointing for
underrepresented racial-ethnic minorities.
Measuring Institutional Accountability and Effectiveness
Some institutions have begun to use student learning outcomes to help them define
services provided to students and to set benchmarks for academic and budgetary planning.
Student learning outcomes are based on established criteria and expectations that faculty have of
students upon course completion. In order to achieve greater rates of student success, campuses
nationwide are using student learning outcomes (SLOs) to assist them in achieving their
performance goals and reaching their overall missions. The student equity plan required
California community colleges to assess student performance and enhance practices to promote
equity. Current state accountability measures require campuses to submit SLOs that specify
what students will know and be able to do or demonstrate when they have completed or
participated in a program or course. Student learning outcomes are seen as part of the new
accountability practices. Student learning outcomes more recently are being used for
accountability purposes and focus on performance outputs. They also reflect student knowledge,
skills, attitudes, and values gained and can be used as indicators to measure institutional
effectiveness. Grubb and Badway (2005) indicated that student learning outcomes can be used
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 20

to align local colleges’ and universities’ missions and help to maintain state and federal
compliance measures. As colleges discover effective indicators of student success and share
those with other institutions and policy makers, local experiences can inform statewide policies
and local and state boards can understand the variety of compliance measures that are most likely
to be both effective and efficient (Grubb & Badway, 2005, p. 13).
There has been a great deal of time and interest invested as racial-ethnic populations
continue to increase and the labor market expands. For example, a research project entitled the
Missing 87 report, the result of an action-inquiry project between Long Beach City College and
the USC Center for Urban Education, was developed to examine student success rates
(Bensimon et al., 2007). The Missing 87 report consisted of a collaborative process among
faculty members, student affairs practitioners, and administrators and aimed to discover existing
gaps in student outcomes by using “a culture of inquiry” to assist practitioners in breaking down
institutional data. The significance of the Missing 87 report was to assist higher education
institutions discover their roles in producing equitable student outcomes. Inquiry into
institutional data, as facilitated by CUE, allowed the participants to identify barriers that existed
for racial-ethnic minorities. Inquiry into the outputs of colleges and universities can solicit
questions that many institutions may feel uncomfortable answering (Bensimon et al., 2007).
This type of inquiry is the type of process that may well push higher education to fulfilling its
mission of truly being accessible and equitable for all students. Yet, with a few exceptions, such
as the Missing 87 study, the effects of inquiry on institutional effectiveness have not been
extensively documented. To address that knowledge gap, this study will contribute to
understanding the potential of inquiry and its relationship with institutional accountability for
improving equity.
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 21

Statement of the Problem
The aim of this study was to explore practices in place to address the equity gaps at
institutions of higher education and focused specifically on practitioners’ roles in impacting
student outcomes for racial-ethnic minorities in STEM. The issues of accountability regarding
educational outcomes in the national context and structures of accountability, assessment, and
institutional effectiveness in California provided the context for this study. In higher education,
many practitioners unknowingly operate from a Eurocentric perspective and do not realize they
were doing so. Research on effective organizational learning and reflection is necessary to fully
understand the process of creating a culture of inquiry. Previous research on accountability
structures has not been effective because it used aggregated student outcomes and did not design
policies drawing on practitioner knowledge. This study aimed to address these gaps in available
research.
Purpose of Study
Practices aimed at remediation of organizational policies, structures, and practitioner
beliefs have been the focal point of recent research addressing equity gaps in higher education.
In this study, I conducted a developmental evaluation of CUE’s (cue.usc.edu) efforts in helping
institutions remediate current practices and analyzed participants’ reactions to and experiences
with action-inquiry methods. Through a series of collaborative activities, such as the STEM
Student Success and Equity Forum and BESST Workshops, CUE worked with institutions to
provide a different framework or lens through which they could view students during this
research study. This lens caused practitioners to focus and reflect on their beliefs, opinions, and
ideologies regarding Hispanic and historically marginalized student populations. A series of
workshops, dialogue, inquiry, and interviews prompted practitioners to examine themselves and
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 22

their own internal beliefs and practices. More broadly, this study investigated whether action
inquiry using processes and methods such as CUE’s show the potential to bring about
organizational learning and change, improvements in institutional effectiveness, and greater
racial-ethnic equity in student outcomes.
My research questions for this study were as follows:
1. What is the experience of participation in equity-focused inquiry like for practitioners?
2. Do CUE’s tools raise awareness of equity -minded practices among participants in
inquiry?
3. In what ways, if any, do CUE’s tools prompt organizational change?
Significance of Study
This study is particularly important because of its focus on how practitioners reflect on
practices when examining disaggregated data by race and ethnicity and existing institutional
structures. The primary focus of CUE’s (STEM Toolkit, 2013) work is to equip educators and
institutions with the necessary tools to increase effectiveness in policies, structures, practices,
and curriculum to promote equity mindedness and increase the capacity of institutions to learn
and change organizationally. CUE’s framework prompts collaborative inquiry among
institutions.
Action inquiry is the process by which institutions analyze institutional data, internal
practices, and dialogue collectively to define problems and create solutions to address inequities
in student outcomes. In action inquiry, the practitioner is the researcher and an outside agency
assists the institution in making sense of the data. Research reveals the belief of institutional
stakeholders that data is sometimes compounded and hard to understand (Dowd, Malcom,
Nakamoto, and Bensimon, 2012). In action inquiry, the co-construction of knowledge takes
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 23

place amongst faculty, administration, and staff as they examine their own personal policies,
practices, ideas, and beliefs regarding racial-ethnic minorities.
Action inquiry prompts institutions to base decision making and policy development on
data and requires faculty, staff, and administration to negotiate the meaning of values and equity.
Using action inquiry can help practitioners address inequities in student outcomes by re-
mediating practices that hinder student performance and success. These practices may include
pre-conceived notions, beliefs, perceptions, and judgments held by practitioners. Action inquiry
prompts practitioners to become more equity minded.
Funds of knowledge are critical to assist practitioners in changing practices that
perpetuate inequalities in transfer rates and basic skills courses for underrepresented minorities.
The action inquiry process helps practitioners identify processes that hinder the learning and
development of racial-ethnic minorities.
The principles and practices of CUE (cue.usc.edu) reframe deficit-minded thinking to
equity-minded thinking by way of action inquiry. This research looked at action inquiry in this
process to prevent faculty, administration, and staff from treating the symptoms of a problem
without addressing its root causes. The Equity Scorecard™ (2010) is a tool designed by CUE
aimed to identify indicators that impede student progress. As part of this study, several CUE
Scorecard tools were used by practitioners to identify practices that will progress towards racial
equity. The process involved institutions in addressing and identifying gaps, and collaborate to
problem solve. This study aimed to understand the processes involved in the transformation of
institutional actors such as practitioners into empowerment agents. This study also aimed to help
institutions discover internal practices that inhibit the success of racial-ethnic minorities. It will
contribute to greater understanding of how racialized practices are perpetuated and how
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 24

practitioners can become more cognizant of their role in practices that are harmful to students of
color (Bensimon, 2007). Practitioners often place the responsibility of student success solely on
the individual student. From this stance, the significance of the practitioner in facilitating (or
impeding) the achievement of equitable educational outcomes is underestimated (Bensimon,
2007).
The dialectical framework used by Seo and Creed (2002) captured institutional change
and examined contradictions between institutions and praxis. Similar to CUE, this framework
views praxis as the force that drives institutional change. In action inquiry, the practitioner is
viewed as the researcher. This means that the practitioners examine institutional data with the
intent to discover discrepancies in educational attainment amongst students. With the
practitioner as researcher, institutions are able to improve current practices by examining old
ones. In the project that is the subject of this developmental evaluation, CUE’s tools were used
to prompt practitioners to examine their beliefs and practices. Changes in organizational
structure and function are necessary to change the social and cultural practices instituted by
colleges and universities that are harmful to racial-ethnic minorities. Therefore, it is of interest
and scholarly significance to investigate the implementation and outcomes of strategies, such as
action inquiry, that are designed to change organizational structures.
Definition of Terms
 Empowerment agents are typically people who hold high status positions or positions of
authority and possess a high degree of human, cultural, and social capital. They may also
share common cultural, racial, or ethnic characteristics with students (Stanton-Salazar,
2011).
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 25

 Action inquiry is the process by which institutions analyze institutional data, internal
practices, and dialogue collectively to define problems and create solutions to address
inequities in student outcomes.
 Accountability is “the obligation to report to others, to explain, to justify, and to answer
questions about how resources have been used and to what effect” (Trow , 1996 as cited
in Dowd & Tong, 2007, p. 62).

STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 26

Chapter Two: Theoretical Framework
The purpose of this study was to determine whether CUE’s (cue.usc.edu) action-inquiry
processes that actively engage practitioners in an inquiry process prompts individual learning
that can bring about organizational learning and change. The Center for Urban Education (CUE,
cue.usc.edu) within the Rossier School of Education at the University of Southern California
seeks to accomplish such learning by placing stakeholders, namely counselors, administrators,
and staff, in activity settings where they are encouraged to examine their individual ideas,
beliefs, attitudes, and behaviors. The action inquiry facilitated by CUE, which will be discussed
in greater detail later in this chapter, was analyzed in this study to investigate whether it could be
used to aide institutions in discovering internal policies and practices that contribute to gaps in
outcomes for racial-ethnic minorities. This study discussed whether action inquiry was effective
to help practitioners realize discriminatory practices towards racial-ethnic minorities. These
practices, specifically, may have stemmed from deficit beliefs and perspectives regarding racial-
ethnic minorities and their achievement at the post-secondary level. Therefore, the study
examined changes in beliefs as well as in practices.
In this chapter, I will discuss action-inquiry design and how CUE (cue.usc.edu) draws on
Cultural Historical Activity Theory (CHAT, Dowd, Bishop, Bensimon and Witham, 2012a) to
help practitioners realize their role in producing equitable student outcomes amongst racial and
ethnic minorities. I will also discuss the concept of “in stitutional agency” as a driver of change
for outcomes amongst racial and ethnic minorities.
Institutional agency refers to the role practitioners play in the success of underrepresented
students. Through the examination of current beliefs and practices, practitioners may gain
knowledge to impact their ability to act as institutional agents and student advocates (Stanton-
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 27

Salazar, 2011). Understanding the theoretical basis for the efficacy of CUE’s tools, such as the
tools found in the STEM Toolkit (2013) tools, was the focus as I aimed to uncover whether or
not action inquiry helped to bring about organizational change. CUE’s tools were the subject of
study as I examined how they were used for action inquiry in the field site of my study.
Practitioners’ Beliefs
In this study, tools that are associated with CUE’s Equity Scorecard™ (2010) were used
as part of the action-inquiry process to help institutions uncover inequities in student outcomes.
Action inquiry in this study consisted of a collaborative approach to assessing and responding to
racial and ethnic disparities in student outcomes. The inquiry process asked practitioners to
become the researcher to uncover, identify, collect, and reflect on institutional data to make
decisions regarding practices that affect racial and ethnic minorities on their campus. This
process is known as action inquiry. Dowd and Bensimon (2009a) defined action inquiry as “a
systematic process of problem identification, data collection, reflection, decision making, and
action” ( p. 2). Within CUE’s participatory critical action inquiry, CUE acts as “institutional
outsiders . . . who create processes and assessment tools for action inquiry for institutional
insiders” (Dowd & Bensimon, 2009a, p. 2), who used them to assess their own practices and
organizational settings. In this study, CUE’s research team act ed as teachers, equity advocates,
and observant participants. As Gutierrez and Vossoughi (2010) described,
The researcher as a collaborative, reflective ‘observant participan t’ may help make
visible the practices, meanings, and contradictions that often become invisible to those
closest to the action . . . in this sense neither the outsider or the insider is granted
immaculate perception. (p. 101)
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 28

Within the action-inquiry process, practitioners can learn from one another in their own
environment. This approach aides in the organizational learning process as inquiry teams
collaborate to make sense of institutional data. Dowd, Malcom, Nakamoto, et al. (2012) argued
that inquiry teams are essential for informing learning in higher education. Dowd, Bishop,
Bensimon, and Witham (2012b) wrote, “Inquiry teams are valuable organizational structures for
creating strategic intelligence about the problems of racial and ethnic inequities in student
outcomes” ( p. 4).
It can be intimidating for institutions and sometimes non beneficial when outside
agencies identify organizational problems in institutions of higher education regarding
educational outcomes, specifically when recommendations or mandates are made from external
sources (Greenwood & Levin, 2005). Practitioners view themselves as the experts when it
comes to identifying issues that may exist on their campuses. Action inquiry is used to prompt
practitioners to examine themselves and the role that they play in racial-ethnic inequalities. This
process is not always easy for practitioners because it requires investigation, remediation, and
some constructive criticisms of themselves (Greenwood & Levin, 2005). While the process
prompts self-reflection, it also allows practitioners to mutually negotiate meanings, values, and
goals (Dowd & Bensimon, 2009b). Colleges and universities can build both the language to talk
about race and the ability to inquire into causes of inequity and potential solutions using action
inquiry as a process of accountability and change (Dowd, Bishop, Bensimon, & Witham, 2012a).
Reason (in press) emphasized the action-inquiry process as being “fundamentally about
the transformation of power relations in the direction of greater democracy” ( p. 3). Asking
practitioners to assume the role of action researchers and to conduct action inquiry into their own
practices (Reason, 1994) is one approach to remediating the roles and community of practice in
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 29

higher education (Dowd, Malcom, Nakamoto, et al., 2012). The process of action inquiry
prompts practitioners to take responsibility and ownership of personal values, beliefs, and
actions. Once this process has occurred, practitioners are able to see the power that they have in
impacting the success of racial-ethnic minorities. Scholars support reflection that takes place
during the inquiry process in ways that help practitioners realize their roles as mediators and
their responsibility to racially and ethnically diverse students (Dowd & Bensimon, 2009a).
Action inquiry prompts institutional actors to question the normative views that they hold and
examine the basis by which they measure racial and ethnic minorities against the normative or
dominant paradigm of what it means to be a college student. It is harmful to students of color
when institutional actors have preconceived attitudes, beliefs, and perceptions about who they
are and do not challenge stereotypes that depict racial-ethnic minorities in a negative light (Dowd
& Bensimon, 2009b).
Action inquiry is designed to help educational institutions across the country become
more accountable to students from underserved racial and ethnic communities and aims to
transform institutional data into “actionable” knowledge . As a result of action inquiry,
practitioners may develop new knowledge to drive organizational practices. Action inquiry
emphasizes the full integration of action and reflection, “so that the knowledge developed in the
inquiry process is directly relevant to the issues being studied” (Reason, in press, p. 3).
How Practitioner Beliefs Impact Racial-Ethnic Minorities
There are often cases of prejudice and discrimination in institutions of higher education
(Jones, 2000). Racialized practices have become so common that they are often considered
neutral or normative even though they further promote educational inequities. As stated by one
faculty member, “some students are doomed to failure when they come in . They have no
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 30

motivation in them to succeed and they have no background in the first place” (Bustillos, Rueda ,
& Bensimon, 2011, p. 199). This statement reflected beliefs that many practitioners hold
regarding racial-ethnic minorities. Prejudice causes practitioners to make differential
assumptions about the abilities and capabilities of students of color; specifically, when they enter
college less prepared than their peers. Practitioners often make assumptions about racial-ethnic
minorities based on prior education, language differences, socioeconomic status, and access to
resources (Bustillos et al., 2011). Prejudice can also cause practitioners to question the motives
and intentions of racial-ethnic minorities based on their race (Jones, 2000).
Educational research shows historical patterns where practitioners attribute inequitable
outcomes to student deficits. Dowd and Bensimon (2009b) supported that practitioners often
hold deficit beliefs regarding racial and ethnic minorities. These beliefs demonstrate a type of
personally mediated racism. Personally mediated racism can be both intentional and
unintentional; Jones (2000) defined it as “prejudice and discrimination, where prejudice means
differential assumptions about the abilities, motives and intentions of others according to their
race, and discrimination means differential action towards others according to their race”
(p. 1213). Members of the dominant culture may also intentionally or unintentionally exhibit
internalized racism in their respective practices. Internalized racism often reflects systems of
privilege, societal values, erodes individual sense of value, and weakens collective action. When
practitioners lack cultural knowledge, they may be oblivious to noticing how they are and the
ways in which they create the conditions that prevent students from behaving according to their
expectations (Pena, Bensimon, & Coylar, 2006; Steele, 1997). Many scholars and observers of
higher education argued that it is now time for institutions to examine themselves in order to
determine how they are perpetuating inequities.
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 31

Researchers argued that the persistence of inequalities amongst racial-ethnic minorities in
higher education often reflect a learning problem of the practitioners (Harris III & Bensimon,
2008). Patterns of educational research demonstrate practitioners’ tendency to place the
responsibility of learning solely on the student. Bensimon (2007) supported these findings. Her
research discussed how in the dominant paradigm students are viewed as the sole authors of their
successes and failures. In the dominant paradigm, “the significance of the practitioner in
facilitating (or impeding) the achievement of equitable educational outcomes is underestimated”
(Bensimon, 2007, p. 446). When students are made the authors of learning, emphasis is taken
off institutions to collectively act on behalf of the students and advocate for equitable student
outcomes. It is a very common practice in higher education for practitioners to see students as
autonomous and committed, engaged, and self-motivating (Bensimon, 2006).
Widely accepted theories such as those proposed by Tinto (1993) and Astin (1985) are
also student centered and focus on the motivation of the student and the amount of effort he or
she expends in the college acclimation process (Dowd, Sawatzky, & Korn, 2011). Practitioners
ask questions that focus specific attention on the student often times when students aren’t
succeeding academically and racial-ethnic disparities exist. The capital that racial-ethnic
minorities bring to university settings is often considered deficient by practitioners who are
unfamiliar with who they are and where they come from. As emphasized by Gutierrez, Morales,
and Martinez (2009),
Artiles’s (1998) analysis of the deficit framework . . . points to the field’ s inattention to
the sociohistorical contexts of development of these students; historically, attention has
focused on comparing and holding members of nondominant communities against a
normative view that can only render them only ‘ different.’ (p. 222)
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 32

This quote coincides with Bernal’s (2002) literature on critical race theory because it
references American culture being based in Eurocentric standards and ideologies. The culture of
racial-ethnic minorities differs from what is considered the Eurocentric norm in America;
however, the cultures are unique and can be used to inform learning in higher education.
The Equity Scorecard™ as Part of Action Inquiry
CUE (cue.usc.edu) uses critical participatory action inquiry and The Equity Scorecard™
(2010) process to assist colleges and universities in discovering inequitable practices. A first
step is disaggregating student outcomes by race and ethnicity and asking practitioners to analyze
it for equity gaps. Disaggregation of data by race and ethnicity is important because aggregated
data is often presented as a mass and hides racial patterns of inequity (Dowd & Bensimon,
2009a). Aggregated data also gives the illusion that institutions are performing better in
producing equitable student outcomes. Scholars supported that when provided disaggregated,
data inquiry teams can interact in an action-inquiry process bringing together various ideas and
beliefs that are essential in creating solutions for addressing inequalities among racial-ethnic
minorities (Dowd, 2005). From this view, the practitioners have the power and hold the key to
organizational learning and change.
The Equity Scorecard™ (2010) as designed by CUE (cue.usc.edu) is intended to foster
institutional change through the identification and elimination of practices that create racial
disparities among college students. Viewed as both a “process” and a “data” tool that combines
a theoretical framework with practical strategies, it initiates institutional changes that lead to
equitable outcomes for racial-ethnic minorities (CUE, cue.usc.edu). The purpose of action-
inquiry tools is to help practitioners keep track of how effectively they are performing based on
the production of successful outcomes of minority students (Bensimon, Hao, & Bustillos, 2006).
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 33

Organizational Learning and Cultural Historical Activity Theory
The performance of racial and ethnic minorities when entering college is often adversely
affected by the organizational culture (Moore & Shulock, 2007). Harper and Hurtado (2007)
discussed how institutions often don’t deal with issues of race and discrimination until they are
forced. In an example pointed out by Harper and Hurtado, an institution in the Midwest was
scrutinized for its discriminatory practices against African American students and faculty who
advocated against unfair institutional practices. In this example, it was not until students
protested and the campus was audited that the administration felt the pressure to publicly address
student concerns and use audit findings to guide institutional change. In many instances, issues
of race and discrimination are neutralized until severe events occur in which institutions are
forced to confront them (Harper & Hurtado, 2007).
The theory and processes of organizational learning help researchers and practitioners
understand and address the structural and cultural obstacles that prevent colleges and universities
from producing equitable educational outcomes among racial and ethnic minorities (Bensimon,
2005). Researchers argued that organizational learning is both effective in theory and practice
by making un-discussable practices, such as those leading to inequitable outcomes for racial and
ethnic minorities discussable (Bensimon, 2005). Changes in organizational structure and
function are recommended to change the social and cultural practices instituted by colleges and
universities that are harmful to students of color. Disaggregated data often has the potential to
impact campus practitioners and motivate them to become agents of change at their respective
campuses. It also “reveals inequities in student participation and outcomes in the colleges” and
prompts practitioners to become more equity minded (Dowd, Malcom, Nakamoto, & Bensimon,
2012, p. 7).
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 34

Ogawa, Crain, Loomis, and Ball (2008) suggested that “all learning takes place in
settings that have particular sets of social and cultural norms and expectations” (p . 83). In this
study, each field site demonstrated a unique set of social and cultural norms. The Cultural
Historical Activity Theory (Roth & Lee, 2007) provided the framework for understanding this
study by demonstrating how learning has taken place socially, historically, and within specific
contexts. CHAT (Roth & Lee, 2007) was a framework that also provided a focus on outcomes
allowing us to see how learning takes shape over time. Dowd, Bishop, et al. (2012a) provided an
excellent example of this. Dowd, Bishop, et al. discussed how communities often segregate
based on factors such as race, ethnicity, social, and cultural differences in Accountability for
Equity in Postsecondary Education. Often times, communities are segregated for so long,
certain areas which the literature referred to as “geographies of difference” (Dowd, Bishop, et al.,
2012a, p. 170) become independent silos. They function and operate on their own becoming less
inclusive of other cultures, races, and ethnicities. It may be unintentional, but racialization
continues when communities remain comfortable in environments that do not value the diversity
of others. With continued separation and segregation of communities and cultures over time,
certain norms are adopted and accepted. For example, behaviors are often learned. Once a
behavior is practiced enough it becomes a habit, and habits are often hard to change. It takes
effort to change habits. Segregation in America has been a practice that has been sustained over
time, and many are comfortable functioning apart from others.
Dowd, Bishop, et al. (2012a) made a relevant connection between understanding histories
of racialization in America and understanding racial dynamics in post-secondary education.
Racialized practices have become learned over time and commonplace in higher education.
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 35

CHAT (Roth & Lee, 2007), as a framework for this study, demonstrated how knowledge can be
learned, unlearned, and relearned in activity settings.
Activity settings are the units of analysis in the Cultural Historical Activity Theory
(CHAT, Roth & Lee, 2007). In these settings, new knowledge can be generated. The CHAT
(Roth & Lee, 2007) diagram demonstrated that the relationship between the subject and the
object can change depending on the mediating artifact and the variable being manipulated.
Through activity settings, practitioners may discover various factors that impact student
outcomes. The factors that impede student outcomes may vary from student to student and
CHAT emphasized that practitioners should not generalize behaviors to specific racial and ethnic
groups. CHAT allowed practitioners to see the variables that impacted outcomes among racial-
ethnic minorities in higher education. The theory also allowed practitioners to see how certain
factors affect individuals independently. In this study, cultural norms and expectations were
analyzed through the lens of CHAT with immediate focus on remediating professional practices
and cultural artifacts leading to equity in student outcomes for racial and ethnic minorities
(Dowd & Bensimon, 2009b).
Ogawa et al. (2008) wrote, “From the CHAT perspective, learning is the process by
which people master and appropriate cultural tools and meanings while engaged in activity”
(p. 84). For example, the Benchmarking Equity and Student Success Tool™ (2010) designed by
the Center for Urban Education (CUE, cue.usc.edu) allowed practitioners to make meaning of
institutional data while engaged in activity settings. CHAT (Roth & Lee, 2007) acknowledged
the factors that influence people’s cognitive and social development . BESST (2010) is an
example of a tool that influences practitioner’s cognitive and social development within social
contexts.
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 36

CHAT (Roth & Lee, 2007) can be used as a conduit for remediating practice in higher
education because it focuses on the “activity as the site where individuals negotiate meaning and
where communities develop, reify, and transform common practices” (Ogawa et al., 2008, p. 84).
An example of this would be the practice of data disaggregation. This practice would allow
practitioners to identify disparities in student outcomes among racial-ethnic minorities and create
opportunities for institutional learning and sustained organizational change. CHAT also
emphasized how activity settings have both social and historical contexts by taking into account
people’s physical surroundings, social relationships , and behaviors. These are often factors that
get over looked when norms are created, rationalized, and left unchallenged. The CHAT lens
allowed people to understand their behaviors, values, and cognizance of certain behaviors.
CHAT emphasized the development of human activity and explained how meaning is developed
over time. In higher education, the learning process in itself can be multifaceted and as a result
of action inquiry, pursued with designs based in CHAT, practitioners may realize they have to
unlearn and relearn processes, practices, and information.
Contextual learning and culturally responsive pedagogy improve and enhance the
experiences for all students (Gutierrez & Vissoughi, 2010). Cultural Historical Activity Theory
(CHAT, Roth & Lee, 2007) addressed the way practitioners learn and construct knowledge
within activity settings (Ogawa et al., 2008). More specifically, it emphasized the factors that
impact the social and cognitive development of individuals and the systems that connect them.
CHAT (Roth & Lee, 2007) is one example of many theories that may be used by institutions of
higher education to implement the types of changes they need to bring about equity in practice.
Utilizing CHAT as a perspective for change, practitioners were asked to see how “cultural
artifacts” such as language, historical legacies , and objects currently found in education impact
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 37

student learning. CHAT contradicts the dominant paradigm. Within CHAT, there is an
emphasis on learning that takes place in social environments where both the subject and the
object impact one another. CHAT stated that learning and knowledge is neither imparted or set
but ever changing and co-constructed between the student and practitioner. Knowledge is
contextual and can be changed due to environmental factors that have little to do with the student
and/or learning, “therefore, the learning setting is recognized as dynamic and reflexive
characterized by various changing parts that may affect one another” ( Dowd & Bensimon,
2009b, p. 5).
The CHAT (Roth & Lee, 2007) perspective allowed collective and individual members
of a group to master and appropriate cultural tools and meanings. Within activity settings,
practitioners could share and assist one another in learning and professional development.
CUE’s (cue.usc.edu) framework for action, research, and evaluation focused on critical action
inquiry and inquiry theory to drive decision making. CHAT provided the framework for guiding
CUE’s action-inquiry methodology which focused on remediating the artifacts of professional
practice in colleges and universities to achieve equity in outcomes among racial-ethnic groups
(Dowd, Malcom, Nakamoto, et al., 2012). Using action inquiry alongside CHAT can help
practitioners address inequities in student outcomes by re-mediating practices that hinder student
performance and success. These practices include pre-conceived knowledge, beliefs,
perceptions, and judgments.
CHAT has been used to inform educational issues in the areas of language, language
learning, and literacy (Roth & Lee, 2007). The theory also examined the relationship and
interaction between group members, goals, rules, norms, and division of labor within the activity
settings. These dynamics are reflective of dichotomies between the practitioner, the student,
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 38

learning environments, and spoken and unspoken beliefs. All of these factors can impact student
learning outcomes. CHAT (Roth & Lee, 2007) helped inform this study because it showed that
learning occurs as a result of simultaneous actions. The theory showed how elements work
together to present a larger picture. In higher education, student deficits are often seen as single
isolated events. This viewpoint of practitioners does not take into account the elements
presented in CHAT that may influence student behavior and outcomes. CHAT can offer
practitioners a different lens through which to view students because it explains the elements that
impact student learning and development.
Roth and Lee (2007) supported that the elements of activity settings are not mutually
exclusive and should be considered together in the organizational learning process. For instance,
practitioners may assign certain characteristics or deficits to racial-ethnic minorities. They may
even believe that low graduation, retention, and transfer outcomes are the result of poorly
prepared and unmotivated students. This viewpoint looked at students as mutually exclusively
elements and does not provide insight to how students arrive where they are. CHAT (Roth &
Lee, 2007), used as a framework for this study, provided practitioners with a more holistic
approach in which to view students. The holistic approach includes six elements of activity
settings: object, subject, mediating artifacts, community, rules, and division of labor.
The object element of the activity setting “guides or directs individu al actions” (Ogawa et
al., 2008, p. 85). In CUE’s (cue.usc.edu) participatory action-inquiry project, the object being
considered and/or acted upon was the institutional data and the subject was the practitioners who
can “influence their social relations, objects of action, and artifacts which may also affect their
own capacities for action” (Ogawa et al., 2008, p. 86). Through their participation in activity
settings, subjects can create socially and culturally produced tools, transform tools and
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 39

structures, and impact activity systems. This is done through language, communication, inquiry,
and reflection. Roth and Lee (2007) discussed the role of collaboration in learning and
development. The relationship between the subject and object in this study was mediated by the
artifacts provided by CUE that prompted social interaction, community, and activity. Mediating
artifacts act by connecting subjects to one another. The BESST tool is an example of a
mediating artifact that aides in organizational learning in CUE’s work . The relationship that
occurs between the object, subject, and artifact takes place within the activity settings and/or
small communities of practitioners.
Wenger (1998) referred to communities in terms of their social and cultural practices,
which are characterized by mutual engagement of members, shared resources, language, and
actions. The rules of the activity system were established in action-inquiry settings when the
outside facilitator, in this instance CUE (cue.usc.edu), set the tone for and facilitated the
discussion on racial and ethnic inequalities in higher education. The division of labor is the
process by which the action researchers (CUE, cue.usc.edu) and practitioners work together to
uncover racial and ethnic inequities and set incremental steps towards the elimination of these
inequities.
One of the interesting concepts Roth and Lee (2007) pointed out was that when people
choose the purpose of the activity they are more interested and become engaged emotionally.
For example, practitioner participation in action inquiry increases practitioner responsibility to
take action and enhances their ability to recognize faulty infrastructures. The culture of inquiry
may create a sense of awareness for practitioners and engage them by allowing them to look into
their own institutional data and practices. Some practitioners do not realize their role in
contributing to institutional data. They look at student outcomes independent of the roles that
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 40

they play in generating those outcomes. By engaging practitioners in inquiry cultures, their
action possibilities are enhanced (Roth & Lee, 2007).
CHAT (Roth & Lee, 2007), as the framework for this study, theorized that people are
continuously shaping and being shaped by their social contexts which implies that action inquiry
can prompt learning that results in organizational changes. CHAT is also viewed as a theory for
praxis because it focuses on activities that occur when individuals interact in social settings. The
experiences of practitioners in activity settings can reflect the consequences of their incorrect
thinking and beliefs about racial and ethnic minorities. I have taken an excerpt from Roth and
Lee (2007) that shows an example of learning by participating in legitimate activity. I use this
example to demonstrate how knowledge is constructed among practitioners in the field sites
using activity theories such as CHAT. They wrote:
One day, the two coteachers of a seventh-grade class brought a newspaper article to class
describing the efforts of an environmental group concerned with the health of the local
watershed in which the village lies and its major water carrying body, Henderson Creek.
Besides a plea for improving the sorry state of the polluted creek, the article called for a
better understanding of the ecosystem as a whole. The teachers asked the students
whether they were interested in doing something about it. Excited by the challenge, the
students immediately began to brainstorm what they could do, including cleaning up and
documenting the litter that had been discarded there. To help students in framing viable
projects, the teachers organized an exploratory field trip, assisted by parents and
environmentalists, biologists, water technicians, first nation’s elders, and local residents.
The students in groups of three to four individuals then designed their own projects that
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 41

concretely realized the general call of the environmentalists to generate specific
knowledge to save the creek. (Roth & Lee, 2007, p. 192)
In this example the 7th graders, when given the opportunity to save the creek independent
of specific instruction from the teachers, were able to use the guidance, support, and knowledge
of community resources and individuals to create their own unique interventions of ways to save
the creek. This example shows that learning and the impartation of knowledge is not always
directly teacher to student and that knowledge can occur in other ways, particularly in social
activity settings. This example went on to further explain how the students enjoyed the freedom
of being able to design and implement their own studies. Some students decided to take
photographs of the creek to show its state while others took samples. In the end, the local
newspaper featured a story about the efforts of the children and emphasized their contributions to
community-relevant knowledge.
This example demonstrated how impactful knowledge generated within activity settings
can be. It also showed how theories such as the Cultural Historical Activity Theory (Roth &
Lee, 2007) can be used to problem solve, resulting in dimensional solutions all of which address
a unique element of the actual problem. When practitioners engage in action inquiry using data
provided by CUE’s (cue.usc.edu) research team, they are asked to examine inequitable outcomes
for racial and ethnic minorities. As a result, they are expected to be able to problem solve and/or
make recommendations. The freedom of practitioners to openly dialogue in discussions
facilitated by action researchers such as CUE allows for the creation of new knowledge that will
ultimately impact “community -relevant knowledge.” The knowledge generated in this example
ultimately resulted in multiple ways to “save the creek .” When action inquiry is designed well,
the activity settings created by action inquiry allow for individuals to actively problem solve and
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 42

engage in the solution. CHAT encourages the participation of institutional stakeholders in the
generation of new knowledge. In the example, the students learned science because they were
able to choose the method by which they learned it. Each student was allowed the creativity to
express him or herself and learn both as an individual and as part of a unit. In this particular
activity, there was no wrong answer or wrong method. In fact, each group’s presentation on the
creek contributed to the overall knowledge about the creek. Each group took a different
approach in discovering ways to save the creek which demonstrated that there is more than one
solution to a problem. This approach also showed the knowledge produced when individuals
work together in activity settings. The results that each group obtained as a part of their project
contributed to the knowledge of the community and those wanting to learn more about the creek
itself. Working in activity settings expanded the students’ action possibilities and allowed them
to produce knowledge and artifacts that ultimately benefited their community. The artifacts
produced were posters, pictures, digital media, etc. Each artifact presented a different
perspective demonstrating how the creek was being misused. Using this theoretical framework
from a higher education perspective, all practitioners’ contributions were deemed important,
impactful, and ultimately impacted decisions made regarding policies and practices geared
toward the successful outcomes of racial-ethnic minorities.
As a theoretical framework, CHAT (Roth & Lee, 2007) shows how the identity of
practitioners can change from institutional actors to citizens enacting concern for equitable
outcomes amongst racial and ethnic minorities. Using this framework, practitioners have
freedom to problem solve based on what they believe to be important. Active learning may help
them to remove labels and barriers placed on racial-ethnic minorities (Dowd & Bensimon,
2009b). Using CHAT as the framework for understanding the significance of action inquiry in
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 43

this study may help to make “learning that is normall y invisible amenable to deep reflection and
analysis” (Roth & Lee, 2007, p. 195). A unique outcome of using CHAT as the framework was
that the collaboration and collectivity of practitioners provided opportunities beyond those that
can be established by a single individual.
In the early 1900s, a Russian psychologist named Vygotsky (1962) pointed out
researchers’ failure to study learning as a series of events as opposed to single isolated moments
in time. Vygotsky argued that learning should be studied as a collection of events and activities.
CHAT, as a theory for praxis, demonstrates how individuals shape and are shaped within social
contexts (Roth & Lee, 2007). CHAT connects theory and praxis by allowing those involved to
construct and co-construct knowledge based on the activity setting. The theory praxis gap is
addressed in CHAT when practitioners are placed in environments where their values and
knowledge are created, recreated, and renegotiated. From a CHAT perspective, practitioners are
able to inform one another’s beliefs and create solutions that may impact institutional change .
There are several theoretical aspects of CHAT (Roth & Lee, 2007) that suggest why
action inquiry works. Roth and Lee (2007) gave an analogy of threads, strands, and fibers to
discuss how “a unit can be analyzed in terms of component parts, but none of the parts can be
understood or theorized apart from the others that contribute to defining it (Levins & Lewontin,
1985; Valsiner, 1998)” (Roth & Lee, 2007, p. 196). Their analogy pointed out that a strand of
thread cannot exist by itself but consists of several strands. The interwoven fibers that make up a
single strand of thread do not presuppose one another (Roth & Lee, 2007); rather all fibers work
together in a cohesive unit which allows the thread to carry out its function. The function of a
strand of thread far exceeds the function of the individual fibers. CHAT can be used to inform
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 44

this study because it shows how practitioner beliefs and practitioner activity can work together,
inform one another, and ultimately impact educational outcomes for racial and ethnic minorities.
Institutional Agents as Drivers of Organizational Change
In this study, I focused on how CHAT (Roth & Lee, 2007) coupled with CUE’s
(cue.usc.edu) tools impacted the transformation of institutional actors to empowerment agents.
Transitioning to college life can be difficult for racial and ethnic minorities and scholars argue
that socialization plays a big role in contemporary adolescent development (Stanton-Salazar,
2010). The practitioners in this study were oriented with tools designed to impact the way that
they see themselves as well as the students that they serve. The Benchmarking Equity and
Student Success Tool™ (2010) specifically allowed practitioners to see the ways in which they
were failing racial-ethnic minorities. The dialogue that occurred between CUE’s research team
and the practitioners at the field sites also allowed institutions to see how prejudiced practices
occur daily in higher education and often go over looked. In this study, institutional actors acted
as both empowerment agents and equity advocates by participating in assessment practices that
“frame problems of institutional effectiveness in terms of equity” ( Dowd, Malcom, Nakamoto, et
al., 2012, p. 4).
Recent theoretical and conceptual frameworks such as CHAT (Roth & Lee, 2007) and
Social Learning Theory studied the socialization, social networks, and educational attainment of
African America, Latino, Asian, and other ethnicities of students who may be socially and
economically disenfranchised. These frameworks were critical because they demonstrated how
the effort to articulate attainment among racial minority youth, including the multiple roles
played by resourceful and committed socialization and institutional agents can help to design
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 45

interventions and school environments that can authentically empower both youth and agents
(e.g., faculty, staff, counselors, administrators, etc.) (Stanton-Salazar, 2010).
Stanton-Salazar (2010) addressed the ways in which practitioners can impact student
success. It demonstrated that institutional resources can work together to impact and potentially
transform the lives of racial-ethnic minorities. Stanton-Salazar’s literature on institutional
agency supported the idea that it takes a village to raise a child. This ideology implies that if
everyone plays his or her role in a child’s upbringing it increases their chances at success . An
institutional agency in higher education can also strengthen relationships and build a sense of
community among practitioners. Many racial-ethnic minorities can identify with a strong sense
of community based on family history and experience. Institutional agents may hold high status
positions or positions of authority and hold a high degree of human, cultural, and social capital.
When placed in a social network an institutional agent “manifests his or her potential role . . .
when, on behalf of the adolescent, he or she acts to directly transmit, or negotiate the
transmission of, highly valued resources . . .” (Stanton-Salazar, 2010, p. 5). Highly-valued
resources can be degree-completion requirements or transfer-course requirements but can also be
the tools needed by the student to successfully navigate the college campus as an entering
freshman.
According to Stanton-Salazar (2010), young adults participated in multiple sociocultural
roles and non-familial adult agents played a critical role in their social development and
educational attainment. The role of the empowerment agent in this study served as a resource for
racial- and ethnic-minority students. In this study, I examined the impact of giving practitioners
tools aimed to increase their funds of knowledge, sparking the dialogue where they could openly
communicate about their roles in student development and educational attainment.
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 46

Institutional agents can use their reputation, status, or authority in strategic and
supportive fashions and change organizational systems in higher education. For instance, a
student’s first point of contact when arriving on university campuses is generally faculty or staff
in administrative roles. The experience of a first-time freshman is dependent upon their
interaction with institutional actors which demonstrate the power held by individuals in
authority. Some institutional actors argued against viewing educational inequities through the
lens of race and ethnicity. Dowd, Malcom, Nakamoto, et al. (2012) argued that although
differences in student preparation may exist they should not be pre-assigned based on cultural
characteristics (p. 25). Empowerment agents as proposed by Stanton-Salazar (2010) go against
deficit frameworks because they counteract established and hierarchical social structures that
have been deeply rooted in higher education. They strived to empower low status youth and are
“willing to disembed themselves from the reprod uctive practices of their institution or
environment, and to become a moral agent for positive change in the world that both agent and
youth inhabit” (Stanton -Salazar, 2010, p. 36). Empowerment agents widen the pipeline for
racial-ethnic minorities while acknowledging that institutionalized racism may exist in their own
institutions presenting barriers to underrepresented students.
Data gathered during this study sometimes revealed the belief held by practitioners that
racial-ethnic minorities lack the capital needed to succeed in institutions of higher education. On
the contrary, Stanton-Salazar (2010) argued that practitioners have the power and the authority to
enhance the social capital of students. Empowerment agents can enhance the empowerment
social capital of students by “transforming the consciousness of those they support, and at
encouraging them to also become moral and caring agents devoted to changing the world” ( Ward
2008 as cited in Stanton-Salazar, 2010, p. 36).
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 47

Empowerment agents work to give students the knowledge to transform and ultimately
impact their communities. Some scholars who use frameworks such as the social capital
framework hold stakeholders in higher education accountable for the role that they play in the
social development and educational attainment of racial-ethnic minorities (Bowles & Gintis,
1976; Sewell & Hauser, 1980). Stanton-Salazar (1997; 2001; 2004 as cited in Stanton-Salazar,
2010) defined “social capital as consisting of resources and key forms of social support
embedded in one’s network or associations , and accessible through direct or indirect ties with
institutional agents” (p. 5). Agents demonstrated their potential to impact the socialization
process through direct engagement with students (Stanton-Salazar, 2011). In this study, part of
the action-inquiry process required practitioners to examine whether or not they were equity
minded. Bensimon (2007) described equity-mindedness as a characteristic held by practitioners
that allows him/her to become institutional agents of minority student success. Institutional
agents act as advocates for students and practice equity-mindedness. “Equity-minded individuals
are more cognizant of exclusionary practices, institutional racism, and power asymmetries that
impact opportunities and outcomes for Black and Latina/o students” (Bensimon, 2007, p. 446).
They often reflect on their own practices, roles, and responsibility in achieving student success.
In this study we used CHAT (Roth & Lee, 2007) as the theoretical lens for understanding
how CUE’s (cue.usc.edu) tools impacted practitioner beliefs and organizational change.
Practitioners were given tools to help them discover inequities, examine beliefs, and discuss
equity mindedness when it comes to racial-ethnic minorities. In Chapter Three, developmental
evaluation to investigate how action inquiry can be used to bring about organizational change
will be discussed.
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 48

Chapter Three: Methods
This study was part of an ongoing collaboration between CUE and Dynamic City
College. The specific research procedures and methods that I used to conduct my study will be
presented in this chapter. The first part of the chapter details the use of developmental
evaluation to investigate the impact of action inquiry and how it was instrumental to
organizational change. The second section includes a detailed description of data collection
procedures and methods, the sample and field site of my individual study as well as of a larger
collective study of which this study is a part, and data analysis procedures. The third section
addresses standards of review, including credibility and limitations, and reporting of results and
findings.
Developmental Evaluation in Organizational Change
I used developmental evaluation to investigate and examine the manner in which
practitioners become aware of their cultural beliefs, assumptions, and values as they participate
in action inquiry to bring about organizational changes. More specifically, I focused on
practitioners’ beliefs regarding the outcomes of racial-ethnic minorities at the post-secondary
level in Science, Technology, Engineering, and Mathematics (STEM) related fields. Through
the developmental evaluation method, I explored how practitioners recognize their role in
student outcomes. I chose developmental evaluation as my methodology since developmental
evaluation seeks to establish social and personal interactions that encouraged cooperative
relationships amongst the participants involved in the project. Secondly, developmental
evaluation seeks to provide transparency, whereby all participants seek to agree on the processes
and procedures related to the project and how they will determine the kinds of information
necessary to move the project along. In this case, practitioners examined trends in student
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 49

outcomes in STEM-related courses and dialogued about how to create equitable outcomes for
culturally diverse students. Lastly, developmental evaluation encourages all participants to
continuously work towards establishing an on-going process of observation, reflection, and
action and then evaluation of the plan (Stringer, 2007). The developmental evaluation process
has the potential to inform institutions of higher education about how to incorporate action
inquiry into the assessment of institutional effectiveness and equity (Dowd & Bensimon, 2009a;
Patton, 2011). Developmental evaluation helps evaluators to fully participate in all aspects of the
evaluation process in “decision making, discussing how to evaluate the project, interpreting
findings, analyzing implications and applying the next stage of development” (Patton, 2011,
p. 20). In this study, developmental evaluation was used to help institutions analyze data and
dialogue about courses that function as gate keepers for racially and ethnically diverse students.
Moreover, institutions considered student progression in STEM-related disciplines and tracked
student progress to determine what can be done to generate equitable outcomes for racial-ethnic
minorities in STEM courses.
Developmental evaluations aid innovators to bring about “systems change under
conditions of complexity” (Patton, 2011, p . 20). In developmental evaluation, the goal is to
provide avenues for further inquiry by asking evaluative questions, using evaluation logic, and
gathering data from these questions, which then makes on-going decision making possible
(Patton, 2011). The primary function of developmental evaluation is to elicit discussions that
bring about data-based decision making. The reservoir of knowledge that practitioners bring to
the table can then be used to design new social innovations (Patton, 2011). Practitioners can then
evaluate the effectiveness of institutional programs as they continue on-going assessments of the
programs. The field setting for my study and the larger collective study consisted of colleges
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 50

and universities involved in action-inquiry projects conducted by CUE (cue.usc.edu). My site,
specifically, was a recipient of the federally funded Hispanic Serving Institutions (HSI) STEM
grant. The college worked alongside CUE to evaluate and assess its program efforts. Therefore,
CUE researchers and other higher education researchers involved in this inquiry project were the
primary group of practitioners to benefit from the results of this study because the findings
informed better designs of action inquiry tools and process. The broader field of higher
education will also benefit because action research and action inquiry are promising strategies to
improve equity in STEM, but these processes have not be studied extensively.
CUE’s (cue.usc.edu) research involved understanding how practitioners can incorporate
the language of equity and the characteristics of equity-mindedness as they create institutional
assessment tools that can be used to purposefully design collaborative activities that address the
problems of equity existing at 2- and 4-year colleges. For this study, I examined practitioner
reactions to and behavior during action inquiry facilitated by CUE at the Hispanic Serving
Institute (HSI) that I selected as my field site. During my study, CUE facilitators used tools,
such as those found in CUE’s STEM Too lkit (2013), that “help teams and individuals reflect on
how their own actions and behaviors, as well as institutional practices and resources, affect
Latina and Latino students’ success” ( Bensimon & Dowd, 2012, p. 12). The Equity Asset
mapping protocol is an example of a CUE tool. The Equity Asset mapping activity can be used
to help institutions evaluate how they deal with issues of inequity. It allows practitioners to see
themselves in light of institutional problems and examine their roles in student success. The
Equity Asset mapping activity also helps practitioners realize their ability to act as student
advocates and to promote and support success in STEM-related fields for Latino students.
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 51

This study specifically examined the processes of developmental evaluation as well as the
experiences and reactions of practitioners in regards to the action-inquiry process. It
supplemented CUE’s action -inquiry tools that facilitated inquiry into the attitudes, beliefs,
behaviors, and practices of a small sample of faculty, counselors, and administrators at a single
Hispanic Serving Institution.
Developmental Evaluation Research at DCC
Figure 1 illustrates the relationship of this study, which used developmental evaluation
methods, in relation to other types of research conducted by CUE researchers. Developmental
evaluation supplemented the development of CUE’s action-inquiry tools, which are designed to
foster equity among racial-ethnic groups’ experiences and outcomes in higher education. The
process of developmental evaluation supplemented various modes of research. First, the activity
setting was made up of institutional (campus-based) participants who collaborated on a joint
activity to identify and address institutional problems. Next, action inquiry was conducted by the
campus-based participants themselves who were involved in activity. Participants became action
researchers themselves as they looked from the outside at the problem that existed within their
institution as they proposed to change the system within, as well as enacted changes in practices
within their immediate control. Additionally, developmental evaluation allowed for a third-
person perspective whereby an outside observer, not one involved in the action research or in
action inquiry as a facilitator, studied the activity setting and its action-inquiry operations as
institutional practitioners created tools to remediate practices. Finally, numerical data collected
through evaluation questionnaires was analyzed through correlational analysis by an additional
collaborating researcher who was not involved in facilitating the action-inquiry activities or in
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 52

collecting observational data. These numerical questionnaire results were available to me as a
secondary data set for my analysis.

Figure 1: Developmental Evaluation Methods Inform Multiple Modes of Research
Source: Dowd & Bensimon (2009a)

Action Research Driving Inquiry
CUE (cue.usc.edu) conducts action inquiry to facilitate practitioner inquiry. As action
researchers, CUE creates activity settings with the aim of remediating educational practices that
are harmful to racial-ethnic minorities. CUE’s focus relies on the tenets that learning and
knowledge are constructed and co-constructed in collaborative activities. During the inquiry
process, specific discourse occurred to bring about acceptances and disagreements which then
led to a shared understanding of the problem. The inquiry process promoted contradictions
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 53

between participants’ views abou t their personal beliefs, assumptions, and values. These
contradictions motivated participants to gather data that revealed the beliefs, assumptions, and
values held by participants in the activity settings. CUE seeks to conduct its action-inquiry
projects at a “critical point of intervention.” This critical point of intervention began a cyclical
process that involved purposeful social interactions that allowed for the exchange of knowledge
and beliefs in specific cultural contexts. During the interactions, the knowledge that the
participants espoused revealed participants’ beliefs, assumptions , and values which in turn
provided for reflection amongst the participants. The reflection then allowed for problem
identification which was conducted through data analysis. Problem solving began through action
implementation once the problem was identified. Evaluation of the implemented plan called for
further evaluation and assessment of the intervention (Stringer, 2007). CUE’s action -inquiry
process emphasized how, when practitioners apply knowledge obtained through participatory
inquiry, they impact the way instruction and other educational practices improve equity and
effectiveness (see Figure 2).
Addressing Accountability and Assessment Measures
This study contributed to the efforts by institutions of higher education by addressing
state and federal accountability and assessment measures. Action inquiry provided practitioners
the tools to combine the inquiry process with the available accountability data as practitioners
learned how their personal beliefs, assumptions, and values contributed to the underachievement
of minority students, more specifically Latino students in STEM. In addition, action inquiry
created the demand for accountability regarding the success rates for minority students pursuing
STEM-related degrees and careers. Through the use of assessment processes and tools,

STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 54

Institutional
Structures
Instructional
Practices
Equitable
Student
Outcomes
CUE EQUITY MODEL: IMPACT
Intervention
Point
Mediating
Outcomes
Ultimate
Impact
State Policies
Adaptive
Expertise
Knowledge
Experimentation/
Problem Solving
Social
Interaction
Action/Experience
Beliefs
Problem Identification
Through Data Analysis
Reflection
Adaptive
Expertise
Knowledge
Experimentation/
Problem Solving
Social
Interaction
Action/Experience
Beliefs
Problem Identification
Through Data Analysis
Reflection

Figure 2: Reflective Practice (Inquiry) as Driver of Racial-Ethnic Equity in Postsecondary
Outcomes
Source: Dowd & Bensimon (2009b)

practitioners were brought together in a social setting to collect, observe, interpret, and make
meaning of institutional data that fostered reflection, problem identification, and action (Dowd &
Bensimon, 2009a). CUE’s (cue.usc.edu) equity model (see Figure 2) sought to provide
practitioners tools showing how external and internal mediating outcomes affect equitable
outcomes. The cycle of inquiry is, therefore, not static and continues to evolve as state and
federal policies challenge institutional structures and instructional practices (Dowd & Bensimon,
2009a).
The difference between traditional evaluation processes and developmental evaluation is
that developmental evaluation allows innovators to understand the problem as they experience it.
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 55

Unlike traditional program evaluation, developmental evaluation develops measures and tracking
mechanisms as outcomes emerge and the measures can be modified along the way as changes
emerge. Traditional evaluation programs are controlled by the evaluator and input from other
stakeholders is limited; however, in developmental evaluation, the evaluator collaborates with all
stakeholders and together they co-create evaluations that reflect institutional goals (Patton, 2011,
p. 23-26). Evaluation was essential in this study because literature shows that often times,
practitioners hold students accountable for their own failures (Bensimon, 2005). During my
study, some faculty and administrators stated that they did all that they could do to aid student
success; they didn’t believe that any institutional change could remedy the problem of student
underachievement. Based on my own experiences and future opportunities to act on what I
learned through this study, I am particularly interested to examine the following questions:
1. What is the experience of participation in equity-focused inquiry like for practitioners?
2. Do CUE’s tools raise awareness of equity -minded practices among participants in
inquiry?
3. In what ways, if any, do CUE’s tools pr ompt organizational change?
Action Inquiry at Dynamic Community College
Like other field sites in the collective study, Dynamic Community College (DCC)–my
field site–was chosen because of its designation as a Hispanic Serving Institute (HSI), because
several of the team’s participants had attended previous CUE (cue.usc.edu) workshops, and
because individual practitioners at the campus were willing to engage in inquiry. Table 1 lists
the institutions that were involved in the collective study and the various CUE tools that were
used at each site. Participants were selected based on their participation with the Center for
Urban Education at USC. Most participants were grant recipients to whom USC had been
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 56

indicated as a collaborator on the grant or an external evaluator. For my study specifically, CUE
was written into the stem grant as an external evaluator. The first process in this action-inquiry
project was to have all participants gather information related to the problem that they were to
investigate. The participants then expounded on how they viewed the problem and reflected on
their personal experiences and perspectives in small group settings. By doing this, they were
able to make sense of the problems in their own way (Stringer, 2007, p. 65) based on their
beliefs, assumptions, and values. The process participants used in this action-inquiry project was
to:
1. Examine institutional baseline data, and then based on the data, identify the equity gaps
in STEM course and degree progress.
2. Analyze what was happening and theorize why the problem existed.
3. Develop a plan, implement the plan, and then evaluate if the plan successfully addressed
the problem (Stringer, 2007).
Data Collection
Participant Demographics: CUE Action-Inquiry Study
As stated previously, my study was part of CUE’s (cue.usc.edu) larger action inquiry and
evaluation agenda that was based on several major research and action initiatives that began as
early as 2004. These projects conducted by CUE included the Diversity Scorecard, Equity for
All, the “Missing 87 ,” A Study of Transfer Ready Students Who Do Not Transf er, the California
Benchmarking Project, the Institute for Equity and Critical Policy Analysis, and the Wisconsin
Transfer Equity and Accountability Study.
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 57

The larger developmental evaluation study of CUE’s action inquiry, of which my study
was one part, included four field sites: one California State university and three California
community colleges (see Table 1). The majority of these colleges are designated by the federal

Table 1
CUE’s Developmental Evaluation Study Field Sites
Study Sites
(Pseudonyms
Used)

Type of Institution
CUE Tools Used at Workshops
Involving Individuals from this
Field Site in Spring 2012
Amarillo Community
College (ACC)

Community college
Defining Equity CUE Module;
Benchmarking Equity and Student Success
Tool™ (BESST)

Dynamic Community
College (DCC)

Community college with
federal designation as a
Hispanic Serving
Institution

CUE Modules; Benchmarking Equity and
Student Success Tool™ (B ESST); HSI
STEM Toolkit
Las Flores Community
College (LFCC)

Community college with
federal designation as a
Hispanic Serving
Institution

Benchmarking Equity and Student Success
Tool™ (BESST); Action Planning Tool;
Equity Scorecard Report

Monarch State University
(MSU)

Selective state university
with emphasis on science
and technology education
and a predominantly white
and Asian student body

Benchmarking Equity and Student Success
Tool™ (BESST); Action Planning Tool;
Document Analysis Protocol
The Racism cartoon “Who helped you
through college” Microaggressions
informational handout

government as Hispanic Serving Institutions (HSIs) because their Hispanic enrollment meets or
exceeds the HSI designation threshold of 25% of the student body. The institutions that
participated in the study were all California public higher education campuses. Monarch State
University (MSU) is a 4-year degree granting university offering bachelors and master degrees
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 58

that is located in a small city. It serves just fewer than 20,000 (U.S. Census Bureau, 2010). In
2009, there were 12% Hispanics and 1% African American students enrolled at Monarch State
University. The racial-ethnic breakdown in the region where Monarch State University is
located was as follows: White 91%, Hispanic 20%, Asian 3% and African American 2% (U.S.
Census Bureau, 2010). The study included three California community colleges which offer
certificates and associate degrees. Las Flores Community College (LFCC) is a 2-year institution
that offers certificates and associate degrees. In 2009, LFCC enrollment was just under 17,000
(U.S. Census Bureau, 2010). The racial and ethnic breakdown at LFCC was as follows: White
32%, Hispanic 27%, African American 18%, and Asian 10% (IPEDS, n.d.). The college’s
service area population was just over 65,000. The racial and ethnic makeup of LFCC’s service
area was as follows: Hispanic 41%, White 21%, African American 18%, and Asian 16%
(IPEDS, n.d.).
The second community college in the collective study is Amarillo Community College
(ACC) which has an enrollment of 17, 888 students (U.S. Census Bureau, 2010). It is located in
a small northern California city with a population of 10,003. The college’s service area
encompasses the communities of Benicia, Dixon, Fairfield, Suisun, Vacaville, Vallejo, and
Winters as well as Travis Air Force Base. The racial-ethnic breakdown of all the service area
communities’ population is: White 23%, Asian12%, Hispanic43%, and African American 4%
(IPEDS, n.d.). The racial makeup was 45.0% White (36.2% non-Hispanic), 21.0% African
American (20.5% non-Hispanic), 0.8% Native American, 15.2% Asian, 1.3% Pacific Islander,
9.1% from other races, and 7.6% from two or more races (U.S. Census Bureau, 2010). The
Hispanic population was 21.7% (U.S. Census Bureau, 2010). The racial-ethnic breakdown of the
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 59

college’s service area in 2009 was: White 38%, Hispanic 29%, Asian 16%, and African
American 17% (U.S. Census Bureau, 2010).
Dynamic Community College (DCC), my field site, is located in a large city with 31,000
students (U.S. Census Bureau, 2010). The racial-ethnic student breakdown of DCC is as
follows: Hispanic 42%, White 31%, Asian 11%, and African American 2% (IPEDS, n.d.). The
White population has decreased by 52%, while the Hispanic population increased by 8% (U.S.
Census Bureau, 2010). DCC had 11% fewer White students and a 4% increase in Asian students
(U.S. Census Bureau, 2010).
Participant Demographics: DCC Action-Inquiry Study
During my study, DCC, a recipient of a Hispanic Serving institution (HSI) grant, worked
with CUE (cue.usc.edu) to examine their institutional data disaggregated by race and ethnicity
and to engage faculty members and institutional stakeholders in analyzing STEM outcomes
among matriculating students. This process was identified as the STEM Pathways project. DCC
was also part of a collaborative grant that included one 4-year public institution and one other 2-
year public institution during this study.
The purpose of the collaboration between the three institutions mentioned above was to
address equity gaps seen among racial-ethnic minorities pursuing majors in the areas of Science,
Technology, Engineering, and Mathematics (STEM). More specifically, the collaboration served
to: (1) increase the number of Hispanic and other low-income students attaining STEM degrees;
(2) develop model transfer and articulation agreements between the institutions; (3) enable more
data-based decision making, and (4) increase the number of math and science teachers. CUE’s
(cue.usc.edu) role in the STEM Pathways project assisted DCC in reviewing student data on
STEM gatekeeper courses; collected, disaggregated, and converted data into CUE’s tools to
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 60

assess student progress in the STEM pipeline; and worked with the three participating
institutions to develop and implement annual and long-term benchmarks and goals to increase
the number of Latino and other low-income students entering the STEM fields and transferring
to STEM programs. While this study involved observations taking place during a collaborative
process between three institutions, the study had a particular focus on findings from DCC, the
primary recipient of the HSI STEM grant.
During the collaboration between CUE (cue.usc.edu) and DCC, I served as a member of
the developmental evaluation team. As a member of the developmental evaluation team and
participant observer, I attended the Kick-off meeting and introduction of the Benchmarking for
Equity Student Success Tool™ (2010) in June, 2012. Following this meeting, a leadership team
was identified consisting of grant staff, administrators, and faculty from STEM departments on
campus (see Table 2). The leadership team mainly consisted of people directly involved in the
grant and those in positions of authority. The leadership team selected the milestones to be used
for the benchmarking process prior to the STEM Student Success and Equity Symposium, which
took place in October, 2012. In the spring of March, 2013, part two of the STEM Student
Success and Equity Symposium was held, integrating culturally relevant instructional strategies
into STEM courses. During part one and part two of the STEM Student Success and Equity
Symposium, the leadership team at DCC participated in the benchmarking and inquiry processes
as facilitated by CUE (see Table 3).
During the study, inquiry workshops served as the primary activity settings that began the
dynamics between CUE and DCC. The workshops also encouraged social interaction between
colleagues which was the starting point of the cycle of inquiry for DCC. Part one of the STEM
Student Success and Equity Symposium, which took place in October, 2012, was a forum for

ACTIVITY
Kick-off
Meeting &
BESST
Introduction
Selection of
Milestones
for STEM
Courses
HSI STEM
Symposium
& BESST
Workshop
Asset
Mapping
Activity
Post-
Workshop
Cognitive
Interviews
Expert
Interviews
Part Two:
STEM
Symposium
Post-
Symposium
Cognitive
Interviews
DATES
May
2012
October
2012
October
2012
October
2012
October-
November
2012
November
2012
March
2013
March-April
2013

CUE TOOL
CUE Action
Research
Protocol BESST BESST
Defining Equity
Tool & Asset
Mapping
Cognitive
Interview
Protocol
Expert
Interview
Protocol
Equity Asset
Mapping tool
Cognitive
Interview
Protocol

DATA
COLLECTED None None
Observation;
Student
persistence
and success
rates
Observation
and
Workshop
Evaluation
Survey

Interviews

Interviews
Observation
and
Workshop
Evaluation
Survey Interviews

NUMBER OF
PARTICIPANTS
6
participants
4
participants
20
Break out by
institution
14
Break out by
institution
6
participants
1
participant
19
participants
5
participants

FACILITATED
BY CUE CUE CUE CUE ME ME CUE ME

MY ROLE Not Present Not Present Observer Observer Interviewer Interviewer Observer Interviewer

47
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 61
Table 2

Timeline of Activities at DCC
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 62

institutional stakeholders to address achievement gaps as well as structural and organizational
barriers for students. Part two of the STEM Student Success and Equity Forum which occurred
during the spring of 2013, focused on culturally relevant instruction strategies and institutions’
ability to act as institutional agents.
During my expert interview with a member of CUE’s (cue.usc.edu) action-inquiry team
(see Table 2), we discussed BESST (2010) being the primary tool for creating conversations
around the data and its ability to begin the inquiry process within institutions. During the STEM
symposium process, I observed practitioners utilizing BESST to investigate institutional gaps.
During my observations, I watched the interactions between practitioners and the BESST tool.

Table 3
Participation at Each Stage
Activity Date Members Present
Leadership Team June 2011 3 participants, 2
Administrators and 1
Staff

STEM Symposium:
Part I

October 2012 15 participants
STEM Symposium:
Part II
March 2013 19 participants, 12
staff, 4 administrators,
3 faculty members

Unit of Analysis
Criterion sampling, a predetermined criterion, was established in this study. For
example, this study included three institutions that were part of a HSI-STEM project and had
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 63

agreed to use CUE’s BESST (2010) and other action-inquiry tools. The unit of analysis was
higher education practitioners (faculty, staff, and administrators) at DCC who engaged in action
inquiry facilitated using CUE’s BESST and Equity Asset mapping tools (The Equity
Scorecard™, 2010) at the practitioner’s institutions . The participants all participated in CUE
(cue.usc.edu) workshops and planning meetings. Since developmental evaluation builds on the
idea of innovation and adapting to emergent and dynamic realities in a consistently changing
environment, the initial BESST workshop investigated further inquiry into racial-ethnic
inequities in STEM as participants asked evaluative questions, used evaluation logic, and
gathered data from these questions, which then brought about on-going decision making. The
sample size per institution ranged between 10 and 15 participants.
For the purposes of this study, the activities under study in the design experiment
included planning workshops and meetings that involved the use of CUE’S BESST (2010) and
Equity Asset mapping tools (The Equity Scorecard™, 2010) required the institutions to review
data regarding Latino students’ achievement in STEM-related coursework. Design experiments
evaluated the effects of intervention programs and other processes that impacted both individuals
and objects that may be considered units of experimentation. The design experiment also
allowed for practitioners and researchers to bridge theory and educational practice (Design-
Based Research Collective). The action-inquiry design also included subsequent team meetings
and focused on the multiple factors that have the potential to impact institutional change. Data
collected was used to start the process of inquiry for practitioners to critically analyze
institutional accountability and develop evaluative questions to further on-going discussions
ultimately bringing about data-based decision making. The focus for my data collection was to
document how participants experienced the action-inquiry design and interacted with each other
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 64

while in activity settings. Data analysis consisted of compiling information from action-inquiry
activities conducted by CUE (cue.usc.edu) which included documenting observations conducted
during workshops at DCC and conducting cognitive interviews with participants at each field
site. An observer was present at each field site to record dialogue between the action researcher
(CUE, cue.usc.edu) and the expert (practitioner). The methods for data collection consisted of
informal pre-workshop dialogue with activity-setting participants, observation in activity
settings, post-workshop debriefing interviews using unstructured questions, post-workshop
cognitive interviews with participants using the STEM Toolkit (2013), and expert interviews
with CUE facilitators.
Action-Inquiry Activities and Data Collection
As part of developmental evaluation, CUE (cue.usc.edu) action researchers conducted a
design experiment which engineers particular forms of learning to systematically study forms of
learning within the context of a particular project design (Cobb et al., 2003). The project design,
for example, consisted of workshops and subsequent team meetings. As suggested in Cobb et al.
(2003), the research team needed to: (a) have a clear understanding of the learning pathways,
(b) establish an ongoing committed relationship with the practitioners, and (c) support emphasis
on reciprocal learning. Learning in this sense, according to CHAT (Roth & Lee, 2007) as
discussed in Chapter Two, took place when people created new artifacts from existing artifacts
which then changed and regulated their previous behavior (Dowd, Bishop, Bensimon, &
Witham, 2012a; Lantolf & Thorne, 2006; Salazar-Romo, 2009).
By means of action inquiry, CUE (cue.usc.edu) researchers sought to “engage higher
education practitioners–faculty, administrators, counselors, and institutional researchers–in
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 65

‘action inquiry,’ a systematic process of problem identificat ion, data collection, reflection,
decision making, and action” (Dowd & Bensimon, 2009a , p. 37).
For the purposes of this study, the design experiment included workshops using CUE’s
BESST tool. BESST (2010) was designed to guide practitioners in examining successful course
completion rates, persistence rates, and entering student cohort migration rates from basic-skills
classes to transfer classes, disaggregated by race and ethnicity (Dowd & Bensimon, 2009a). The
data that was used at the BESST workshop showed trends in existing disparities in transfer rates
and placement in STEM courses for Latino students. This data was gathered after faculty
identified gatekeeper STEM courses and other courses that may hinder student progression,
retention, and transfer rates. Using the BESST tool, DCC and partnering campuses involved in
the HSI STEM initiative analyzed data from a 5-6 year period that identified potential problems
in transfer pathways or degree completion in STEM. Since data informs understanding in
activity settings and can inform action inquiry, practitioners at field sites used information from
CUE’s (cue.usc.edu) research team to assess campus cultures, policies, and practices. This initial
data tool permitted the workshop participants to contextualize and define institutional problems
by asking questions and deciding what other data needed to be analyzed to answer their
questions. CUE’s research team present ed disaggregated data by race and ethnicity to faculty,
administration, and staff. While BESST (2010) was designed to show student migration through
the basic skills pipeline, this study focused on student progression through STEM courses
specifically. Specifically, the BESST tool showed cohort data for STEM students and
coursework completion. This tool developed by CUE assisted workshop participants in
determining where barriers exist in student success; it also permitted workshop participants to
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 66

explore “what -if” s cenarios, and helped them determine short- and long-term benchmark goals
focused on racial and ethnic equity.
DCC identified a group of leaders consisting of faculty, administration, and staff during
the planning phase of my study. After this group had been identified, CUE (cue.usc.edu)
researchers reintroduced BESST (2010); DCC was already familiar with this tool since they had
been collaborating with CUE within the past year regarding evaluation and assessment. During
the planning phase, the team members learned about action inquiry during a workshop and
received an introduction to the equity-assets activity diagram. This diagram is an action-inquiry
protocol designed to allow teams to “‘peel the layers’ of routine practices, probe more deeply
into administrative and academic structures, and examine institutionalized policies from an
equity perspective” (CUE, 201 3, p. 5)
1
. During the spring of 2012, the team met and examined
student success data from DCC that were disaggregated by race and ethnicity. Teams then
participated in breakout sessions to form evidence-based answers about how institutional polices,
practices, and structure impact racial-ethnic equity. Student success was tracked at specific
milestones and transitions as an indicator of institutional progress and effectiveness. CUE’s
research team required DCC to establish performance benchmark goals for these specific
milestones and transitions. Through the use of diagnostic and process benchmarking, the team
was able to develop an action plan that supported student success in STEM-related courses.
Developmental evaluation methods permitted the examination of how participants
experienced activity settings, their reactions, and how their beliefs and practices are impacted by
action inquiry and research.

1
Copyright 2013, University of Southern California, Center for Urban Education Rossier School of Education. All Rights Reserved. The contents
cannot be copied or disseminated without express written permission from the Center for Urban Education
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 67

Data collected by CUE (cue.usc.edu) during the fall 2012 BESST workshops were
utilized as a secondary data source to characterize what participants in CUE’s action-inquiry
projects were expected to experience and do. BESST (2010) was designed to guide practitioners
in examining successful course completion rates, persistence rates, and entering student cohort
migration rates from basic skills classes to transfer classes, disaggregated by race and ethnicity
(Dowd & Bensimon, 2009a). At my field site, DCC, the participants were asked to identify and
define STEM courses, then review their institutional data regarding student success rates,
primarily success rates among Latino students in these courses. From this data, stakeholders
were able to see on average how many of their students persist to earning an Associate’s degree
or complete enough credits to transfer to a 4-year university in a STEM-related field. The data
also reflected potential barriers for Latino students in STEM and gatekeeper courses that affected
retention and transfer rates among Latino and other underrepresented students. The participants
were given the opportunity to manipulate the BESST tool for action planning by using baseline
numbers and then adjusted the numbers to determine how many students could be moved on
through the pipeline specifically in mathematics and science courses.
Table 4 outlines timeline snapshots from 2011-2015 and an overview of CUE’s
involvement with DCC as action researchers. DCC agreed to work collaboratively with CUE
during this 5-year span to assess and evaluate its progress as a Hispanic Serving Institute and its
capacity to serve Latino students in STEM. Table 4 shows a preview of the kinds of activities
that took place at DCC, when data was collected and when document analysis occurred during
the action-inquiry project. My study consisted of activities occurring in the 2012-2013 academic
year as shown in Table 5.

STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 68

Table 4

CUE’s Proposed Roles and Activities at DCC

Grant
Year

Fall Semester

Spring Semester
Year 1:
2011-2012
 Prepare and facilitate DCC’s HSI -
STEM Grant: STEM Select Kick-
Off Meeting on May 14, 2012

Year 2:
2012-2013
 Prepare and conduct a 4-hour workshop
for participants to increase their
expertise in data collection and analysis,
using CUE's BENCHMARKING
EQUITY AND STUDENT SUCCESS
TOOL™ (BESST) .
 In preparation for this workshop, CUE
will introduce its BESST via a webinar.
This will allow participants to gain
foundational knowledge for the BESST
workshop.
 Conduct a campus workshop
entitled “Using Inquiry to Define
the Problem(s) and Assess Potential
Interventions.”
 Workshop will train participants on
the use of structured inquiry to
understand the barriers to student
success and what can be done to
address institutional practices that
are not working for students of
color.
 End of the year summary report.

Year 3:
2013-2014
 Conduct a campus workshop entitled,
“Moving towards Equity.”
 Prepare and conduct a BESST 4-hour
workshop using new or updated data.
For example, the client and their partner
four-year institution may decide to
reexamine the cohort or milestones
selected to define their BESST transfer
model(s).

 Conduct a campus workshop
entitled “Using Inquiry to Define
the Problem(s) and Assess Potential
Interventions.”
 In preparation for this workshop,
CUE will provide a 1-2 hour
webinar that will prepare team
members to conduct structured
inquiry into college policies and
practices that hinder/facilitate
student success.
 End of the year summary report.

Year 4:
2014-2015
 Conduct and facilitate BESST workshop
with a focus on monitoring progress and
outcomes using the previously selected
BESST model(s).
 Conduct a workshop entitled, “From
Inquiry to Action Planning.”
 End of the year summary report.

The data collection timelines shown in Tables 4 and 5 covered a period in which CUE
engaged in identifying a planning group at DCC, hosted a BESST workshop where faculty, grant
personnel, and Institutional Researchers were present; created faculty teams and held a
symposium for STEM faculty and staff. During the symposium, STEM leaders discussed
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 69

institutional data, identified problems, and decided on what issues they wanted to collaboratively
address with CUE’s research team . During this symposium, leaders identified possible “next
steps” in serving underrepresented students in STEM more effectively .

Table 5
Timeline for 2012-2013 Activities at DCC
Activity Semester Stakeholders Involved
Introduction to BESST Spring 2012 Leadership team (3-4
participants)

Leadership team discusses
milestone selection/STEM
courses

Spring 2012 Leadership team (3-4
participants)
BESST Workshop
(Reviewing STEM courses)
Fall 2012 Leadership team, STEM
faculty, Administration

STEM Symposium
Expansion of Inquiry
Spring 2013 Leadership team, STEM
faculty, Administration

Tables 4 and 5 provide a description of CUE’s (cue.usc.edu) involvement with DCC. My
study consisted of the first part of CUE’s involvement . In this study, CUE worked alongside
DCC to examine and evaluate Latino student preparedness and transfer readiness in STEM-
related courses. Collected data also consisted of the following: (1) Data practices and Campus
Self-Assessment (How are students selected for admissions and do admissions staff review
disaggregated admissions data?); (2) Special Programs (What pre-collegiate programs are
available to STEM students? Does DCC offer STEM Transfer Bridge Academies and Transfer
Peer Mentors?); (3) Educational Core information (What does the educational core of DCC look
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 70

like? Does it consist of culturally inclusive pedagogy? How is curriculum mapping and
sequencing determined and how does curriculum prepare Latino students for transfer?);
(4) Information of institutional structures (How are students advised about the transfer process?);
and (5) Information on institutional policies (Are there incentives for Latino students pursuing
STEM majors and articulation agreements between DCC and neighboring 4-year institutions?).
Table 5 denotes a timeline of activities for DCC in which a leadership team was identified,
established which STEM courses to collect data, reviewed student academic progress in those
courses, and participated in further inquiry activities.
Data Collection Method
Table 6 shows the various methods of data collection that were used in this study. The
collected data represented practitioner beliefs, behaviors, social interactions, norms, knowledge,
and CUE’s (cue.usc.edu) analytical objectives. The data were summarized into categorical
summaries through deductive and thematic analysis and elaborated on using descriptive texts that
were summarized in a narrative form. The summary of data collection table (see Table 6)
identifies the kinds of data sources that were used to collect data, what the collected data
represented, the timeline for data collection, and how the data were summarized. Documents,
conversations between CUE researchers regarding the action-inquiry project, pre- and post-
workshop cognitive interviews, post-workshop debriefing interviews, activity setting
observations, and workshop evaluation surveys, were the methods used for data collection in this
study.
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 71

Document Analysis
The data sources I reviewed in the fall of 2012 and spring of 2013 were documents that
contain institutional policies, discourse, espoused beliefs, and environment factors that relate to
the institution. The documents that I reviewed included, for example:
1. Organizational/Administrative Practices of the 2012-13 ESL/Basic Skills Action Plan;
2. DCC District’s Board of Trustee meeting of Monday, February 7 , 2012, that provided the
2012-13 vision and goals for the district, accountability reporting for the community
colleges (ARCC), Presentation of the 12 Measures of Success, Progress toward 2011-
2012 Vision and Goals, Community Responses to Survey on Board Vision and Goals,
and finally the approval of the 2011-2012 Vision and Goals by the Board of Trustee;
3. DCC’s Strategic Plan for 2007 -2015
By studying documents and records, researchers gain a significant amount of information
(Stringer, 2007). Documents for analysis in this study were of two types. First, the documents
included meeting notes and agendas from the BESST workshops and team planning meetings.
Additionally, worksheets were used by participants during the workshops. The worksheets were
added to the audit trail and instrumental to support the integrity of the research process. All
practitioner input and documentation were added to CUE’s tools to serve as future artifacts for
practitioner knowledge. Patton (2002) stated that “documents prove valuable not only because
of what can be learned directly from them but also as stimulus for paths of inquiry that can be
pursued only through direct observation and interviewing” (p. 56). Analysis of the documents
included the language that was used in the meeting descriptions and whether agendas were
followed in regards to presenters and topics. By analyzing the documents, evidence pertaining to
the institution as well as what was valued about the project itself can be gathered.
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 72

Table 6
Summary of Data Collection Methods

Data Source

Data Represents
When Data was
Collected
How Data was
Summarized
Documents Institutional Context Fall 2012,
Spring 2013

Descriptive Text
Observations in
activity settings
(workshops)
Behaviors
Social Interactions
Knowledge
Reflections
Problem Identification
Social Interactions
Actions/Inactions

During workshops

Fall 2012
Descriptive text
(triangulating
interview data)
Post-workshop
debriefing interviews
using unstructured
questions
Behaviors
Social Interactions
Norms
Knowledge
Reflections
Division of Labor
Problem Identification
Action/Inactions

2 weeks after
workshop

(Fall 2012)
Categorical
summaries;
Summary tables and
text
Post-workshop
cognitive interview
with activity setting
participants
Behaviors
Social Interactions
Norms
Knowledge
Reflections
Division of Labor
Problem Identification
Action/Inactions
2 weeks after
workshop

(Spring 2013)
Categorical
summaries;
Summary tables and
text

In addition to analyzing institutional policies, notes, agendas, and worksheets, other kinds of
documents were analyzed. For example, various evaluative questionnaires were used across the
sites of the collective study. The evaluation questionnaires consisted of 19 items set on a 4-point
Likert scale with the following response categories: 1-Strongly Disagree, 2-Disagree, 3-Agree,
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 73

and 4-Strongly Agree. Examples of questions used to analyze the effectiveness of the workshop
included: (1) I expect to reflect more deeply on the conversations I had at this workshop;
(2) Prior to this workshop, I have played an active role in addressing issues of racism on campus;
(3) I feel that my own beliefs about equity are similar to those of most of my colleagues on
campus; (4) I have a strong sense of my own personal responsibility surrounding racial and
ethnic issues on my campus; (5) I believe most people on my campus feel it is risky to discuss
issues of race or equity; and (6) I feel apprehensive about my abilities to really make sense of the
data presented here today. Open-ended questions were given to workshop participants so they
could provide additional comments and feedback on information presented during the
benchmarking workshop. Following each workshop, an evaluation form was provided to each
participant. The evaluation tool created by the CUE (cue.usc.edu) research team was used to
gather quantitative data that revealed participants’ behavior, opinions, feelings, and other kinds
of knowledge, including the participants’ motivation to becom e involved in the inquiry project.
The workshop evaluation questionnaire was also used to gather data that reflected participants’
beliefs and their self-reported practices. It provided the researcher information that could not be
seen through observations. In alignment with the main goals of the study, the workshop
evaluation form was used to inform action-inquiry practices within the Center for Urban
Education (cue.usc.edu).
I continued collecting qualitative and quantitative data at subsequent workshops that were
held during the fall of 2012. Observational data (see Appendix A for the data collection
protocol) collected from the workshops represented behaviors, social interactions, norms,
discourse, and knowledge of the activity-setting participants. Following each of the observation
workshops, I scheduled cognitive interviews (see Appendix B for the cognitive interview
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 74

protocol) 2 weeks after each summer and fall workshop with activity-setting participants.
Cognitive interview questions were developed by CUE’s (cue.usc.edu) staff and PhD students,
who captured what practitioners learned from their experiences during their engagement in
previous workshops. Data collected from these cognitive interviews represented the attitudes,
beliefs, self-reported practices, and knowledge of the activity-setting participants. All
participants were given the opportunity to describe their personal experiences as they dialogued
about issues related to their practices (Stringer, 2007). Qualitative observations data were
collected that represented changes in practices, environmental factors, social interactions, norms,
discourse, and changes in discourse. Additional data were collected by conducting cognitive
interviews with the practitioners who participated in the BESST workshops. Notes were taken
during cognitive interviews. The researchers took reflective analytical notes at completion of
each interview. Each instrument was added to the researchers’ ability to triangulate the data and
to add to the reliability of the study. Each of the cognitive interview questions were designed to
address one or more of the research questions and were supported by concepts studied in the
literature review.
Patton (2002) described program evaluation as “the systematic collection of information
about the activities, characteristics, and outcomes of programs to make judgments about the
program, improve program effectiveness, and/or inform decisions about future programs”
(p. 10). Evaluative-based research and measurement of the effectiveness and impact of action
inquiry on learning and change among higher education practitioners was conducted through the
use of multiple data collection procedures. In this study, data collection consisted of pre and post
surveys, including an intake survey, observations, document analysis, participant interviews, and
a final survey. Reflections also took place following each stage of the activity settings. In
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 75

addition to reflection, evaluation of the effectiveness of the project and its goals were measured
at regular intervals. If changes in the process were necessary, extended dialogue and inquiry
occurred among practitioners and CUE’s (cue.usc.edu) research team to modify documents and
procedures.
Debriefing and Cognitive Interviews for Data Collection
Cognitive interviews were carried out with several of the workshop participants from
DCC. The interviews were conducted in the offices of the participants and lasted 30 to 45
minutes in length. The purpose of cognitive interviews was to probe respondents to: see if they
understood the closed-ended questions on a survey in the intended manner; observe how they
worked through the items; assess if a response category was appropriate for the item; and see if
rephrasing the item made it clearer (Ouimet, Bunnage, Carini, Kuh, & Kennedy, 2004).
In this study, cognitive interviews were used to support or dispute the data that was
collected from observations’ and documents’ analysis. The cognitive interviews were also used
to assess participants’ beliefs about equity and the impact of action-inquiry activities.
Cognitive interviews were designed based on the process described by Willis (2005).
The procedure used encompassed: (1) developing a testing plan; (2) creating a cognitive testing
protocol that included both target and probe questions; (3) setting up interview times with
members of the DCC team; (4) utilizing appropriate cognitive interviewing patterns (cognitive
probes and subjects thinking aloud as they answered questions); (5) documenting and reviewing
notes, with the subject’s consent; (6) writing a test report; (7) reviewing results and making
modifications as needed; and (8) carrying out more testing and then reevaluating the
questionnaire form (Willis, 2005).
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 76

Concurrent probing and retrospective probing was used during the cognitive interview
process. Concurrent probing was a part of the actual interview and occurred while the interview
was taking place. Retrospective probing occurred at the end of the actual interview. The format
for the cognitive questions (see Appendix B) was modeled after Willis (2005). The cognitive
interview protocol was designed to evaluate the analytic objectives of the evaluation protocol
and, by extension, of the action-inquiry processes and protocols themselves. Several steps were
taken to identify the analytical objectives. First, the objectives were based on observations made
during the fall 2012 workshops. Second, the objectives were derived from CUE’s (cue.usc.edu)
publications. Third, theories in Chapter Two provided the basis for the kinds of questions to
formulate. Extensive notes were taken during the interview to be sure to capture vital details.
Immediately after the interviews, the notes were transcribed and analyzed for common themes.
This helped to ensure the accuracy of details.
Data Analysis Procedures
CUE’s (cue.usc.edu) action-inquiry tools posits that as practitioners engaged in action
inquiry, existing knowledge and beliefs would be examined and reconstructed through social
interaction and reflections and that their roles would be remediated. It is through these
reflections that practitioner’s become aware of their beliefs, assumptions, and behaviors that
impact their students’ success .
Figure 2 illustrates the learning and change model informing CUE’s (cue.usc.edu) action-
inquiry methods. This model guided my data analysis. The model was informed by the
theoretical frameworks described in Chapter Two, and included practice theory and Cultural
Historical Activity Theory (CHAT, Roth & Lee, 2007). Practice theory calls attention to a cycle
of inquiry among individual practitioners, as well as the role of social interactions in shaping
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 77

opportunities for practitioner learning and experimentation with new educational practices.
CHAT (Roth & Lee, 2007) emphasizes the role of social interaction and cultural artifacts in
shaping practices. Following each workshop, all notes were transcribed and analyzed for
common themes. In order for the transcripts to be analyzed, shared categories were used in an
effort to determine major themes. The transcripts were coded according to eight different
deductive codes: workshop participants’ attitudes/beliefs (A/B), knowledge (K), social
interaction (SI) in activity settings, reflection (R), problem identification (PI), experimentation/
problem solving (EXP), action (inaction)/experience (A/E), or other (not coded above) (see
Table 7).
According to Participatory Action Researchers (PAR), the community or institutional
plane emphasizes the historical dynamics that involves “language, rules, values, beliefs, and
identities which are institutionalized artifacts that disclose the institution’s culture, beliefs,
assumptions, and values that practitioners bring to the activity setting” (Rogoff 1994 as cited by
Nasir & Hand, 2006, p. 459; Schein, 1985). When there is social interaction, participants learn
that their beliefs may contradict the next person’s and that this dissonance may lead to an
awareness of the differences. Through dialogue and inquiry, participants will be able to arrive at
the problem, which may then enable them to collaboratively formulate solutions. Therefore, the
codes listed in Table 7 enabled the researcher to classify and code qualitative data that supported
the conceptual theory discussed in Chapter Two. Any behavioral change that was documented
may have indicated that CUE’s (cue.usc.edu) tools contributed to the change. During the inquiry
workshops at my field site, data was captured and organized based on theme and frequency. The
codes described in the Code Categories section indicate the most common data categories
presented in Chapter Four, the data analysis chapter.
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 78

Code Categories
Attitudes/Beliefs
In this study, attitudes/beliefs were examined because of their importance in influencing
behaviors (Fishbein & Ajzen, 1975). From this framework, attitudes were developed from the
beliefs individuals hold about an object or stimuli. Together attitudes and beliefs influence
behaviors and outcomes (Fishbein & Ajzen, 1975). Data regarding attitudes/beliefs were
collected through evaluation questionnaires, interviews, and observing activity settings and
through the verbal and non-verbal language used during activity settings.

Table 7

Deductive Data Analysis Codes

Code Category Examples
Attitudes/Beliefs (A/B) “Success is entirely possible”

Knowledge (K) “We already know all of this (low
success/transfer). . . we’ve known this for
three years.”

Social Interaction (SI) “Your questions are so great!”

Reflection (R) “What are we doing wrong in the classroom to
help these students?”

Problem Identification (PI) “We haven’t paid attention to the first
generation students.”

Experimentation/Problem Solving (EXP) “This is nosebleed information . . . what I want
to focus on are the student learning
outcomes.”

Action (Inaction)/Experience (A/E) “It’s really time to have a cautious
conversation . . . we’re missing those
conservations.”

Other (not coded above)
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 79

Knowledge
One of the tenets of CUE’s (cue.usc.edu) work was to assist practitioners in developing
the knowledge needed to increase equity for students of color. In this study, CUE specifically
worked with DCC to increase equity for Latino students in STEM. It is not that practitioners did
not want to increase the students’ success outcomes for students of color; it is possible they
lacked the knowledge necessary to increase outcomes for racial and ethnic groups. Knowledge
was constructed through collaborative and “productive activities” ( Dowd & Bensimon, 2009a).
The tools developed by CUE may have had the potential to assist practitioners at DCC to
develop new knowledge and facilitate inquiry.
Social Interaction
Social interactions refer to how people participate or choose not to participate and how
they interact with others. Social interactions occur at three levels; the personal plane,
interpersonal, and community/institutional (Rogoff 1955 as cited in Nasir & Hand, 2006).
The personal plane involves individual cognition, emotion, behaviors, values, and
beliefs. The interpersonal or social plane includes communication, role performances,
dialogue, cooperation, conflict, assistance, and interactions with important social others.
. . . The community or institutional plane involves shared history, language, rules, values,
beliefs, and identities. (Nasir & Hand, 2006, p. 459)
All of these concepts were included to analyze social interactions.
Reflection
Reflection is a thought process that occurs when practitioners examine their practices.
Some argue reflection is necessary for individuals to become more skillful (Nasir & Hand, 2006)
and it is also needed for organizational learning and change (Dowd & Bensimon, 2009a).
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 80

Problem Identification
Problem identification occurs when practitioners identify an issue that needs to be
addressed at their institution.
Experimentation/Problem Solving
Experimentation/problem solving is when practitioners begin developing solutions to
address current issues at their institution.
Action or Experience
Data was coded as reflecting practitioner action or experience when participants were
observed in taking action or in gaining new experiences (or they describe such actions or
experiences, particularly new ones influenced by participation in the inquiry process) to assist
them in utilizing more equitable practices.
Uncoded Data
The data analysis process for uncoded data focused on understanding the attitudes/beliefs,
perceived control, and intent of practitioners; and, its main purpose was not to develop new
themes. However, any uncoded data was also analyzed to illuminate opportunities for expanded
or new themes, revision of evaluation forms, changes to the action-inquiry tools, and changes to
the way CUE (cue.usc.edu) workshops are facilitated. Analytical memos were also referenced to
reveal emergent themes, tensions, and unresolved issues.
Data Reporting
Data was reported from observations, cognitive interviews, and member checks.
Observational data reports included descriptions of the activity settings and CUE’s (cue.usc.edu)
tools. In this study, activity settings consisted of CUE led meetings, inquiry teams’ meetings,
and breakout team meetings. CUE tools included BESST and the Equity Assets mapping
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 81

protocol. Descriptive text and tabular summaries based on categorical data was used to conduct
deductive and thematic analysis. Table 7 illustrates code categories with examples of data that
codes represent.
The quantitative analysis included counting each code found in all the data collected.
Proportions were calculated to inform CUE’s research team of the most frequent areas of
participant reaction, experience, or response. Qualitatively, all codes that were utilized provided
evidence of what took place in the design experiment using the action-inquiry tool.
This study focused on the disproportionate numbers of minority students graduating from
community colleges, transferring to 4-year institutions, or earning 4-year degrees in STEM-
related fields. If practitioners do not have knowledge about their beliefs, then they will not be
willing to change their practices and agency in bringing about institutional changes as they
address the achievement gaps that exist at their institutions.
At each field site, practitioners’ attitudes toward action inquiry was examined as the
project revealed beliefs about the legitimacy of action inquiry and perceptions of self-efficacy
and collective efficacy to carry out projects. It is important to measure beliefs and changes in
beliefs because beliefs are important drivers of behavior (Fishbein, 2000; Patton, 2011).
Fishbein and Azjens’ (2010) framework was utilized to examine practitioners’ behavioral beliefs
(beliefs surrounding the consequences, positive or negative, or a behavior), normative beliefs
(perceived norms or the expected approval or disapproval from others), and control beliefs
(beliefs about personal perceived self-efficacy and environmental factors that may aide or hinder
the ability to carry out a behavior).
If there are no incentives or motivation for practitioners to change their behavioral
beliefs, they will continue the same practices of blaming students for their own shortcomings or
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 82

failures. Also, there are situations whereby practitioners, because of institutional policies or
because they feel that they are not capable of introducing change, may refrain from changing
their behavior. Finally, pressure from others at the institution may also stifle practitioners from
changing practices because of a fear of reprisal. Thus, even if practitioners are aware that they
can do more to narrow minority achievement gaps, they may still refrain from making changes.
After compiling information from planning meetings and cognitive interviews, analytical
memos were created to summarize results from revealed emergent themes, tensions, and
unresolved issues that were gathered through the code categories in Table 7. Additional themes
were added as part of the deductive process and were used to categorize themes that arose during
the data analysis that were not part of the themes previously established by CUE (cue.usc.edu).
We proposed a hypothesis based on the literature which suggested that equity-oriented, cultural
artifacts designed for use in “remediated” so cial learning environments influence educator’s
beliefs, attitudes, and behaviors in regard to equity in postsecondary education. The theory of
developmental evaluation examined and analyzed relationships between activity settings,
practitioners, and CUE’s researchers . With CUE’s research team presenting institutional data,
campus practitioners sought to determine if engaging in inquiry moved them from a place of
problem identification to problem solving, and whether action-inquiry methods prompted
practitioners to examine existing knowledge and beliefs. Using action-inquiry methodology,
CUE examined institutional artifacts, such as language and discourse, which shape practitioners
understandings, interactions, and abilities to take purposeful action (Dowd & Bensimon, 2009a).
The language and discourse that practitioners used demonstrated how their beliefs, assumptions,
and values hinder institutions from implementing polices that promote equitable outcomes in
STEM specifically. CUE’s cen tral tenet is for practitioners to learn through social interactions in
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 83

joint productive activities. The learning and change model was used to inform CUE’s action-
inquiry methods (see Figure 2 and Table 2). As practitioners engage in action inquiry, existing
knowledge and beliefs were examined and reconstructed through social interaction and reflection
(Dowd & Bensimon 2009a). The work of Dewey (1958 as cited in Schön, 1988) suggested that
individual learning cannot take place without reflecting on our interactions. The study
investigated if social interaction prompted reflections and conversations to identify problems of
equity in STEM pipelines, which in turn may lead to new opportunities for practitioner learning
and experimentation with new educational practices. The evaluation form analysis included
basic descriptive statistics characterizing (non-identifiable, pooled, site, and activity specific)
respondents’ experiences and impact. Participant responses to evaluation questions indicated
receptiveness to new concepts such as equity mindedness, action inquiry, and performance
benchmarking.
Ethical Concerns
In research, ethical concerns are very important. In qualitative research, participants may
give very personal responses, and their privacy and rights are important (Patton, 2002). The
rights and privacy of the participants in the study were protected in a number of ways.
Interviewees were provided the “Dear Colleague” letter ( see Appendix C) to inform them about
the study and provide contact information. Pseudonyms were used to protect the privacy of the
participants and and their respective institutions throughout the study. In order to ensure human
subject protection, all researchers completed Institution Review Board (IRB) training and the
study proposal was submitted and approved before the research began.
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 84

Standards of Review
Credibility
Social constructivists are aware of the bias that exists in the world and therefore seek
multiple perspectives in understanding how things work and do not rely on a single truth or
perspective (Patton, 2002). What social constructivists consider important is to understand the
operations of particular contexts and settings without making generalizations over time and
space. Their focus is on dialogue and interaction that reveals the world as people see it. The
focus is to understand how each person’s view or perspective is based on his/her experiences and
background and that it is through these differences that he acts in everyday life. This
phenomenon also includes how individuals act in inquiry (Patton, 2002). Furthermore, Patton
(2002) stated that “Quality and credibility are conn ected in that judgments of quality constitute
the foundation for perceptions of credibility” ( p. 542), which means that if one does not find the
study to be of high quality, then it will cause one to question the study’s credibility.
In the past, action researchers have argued that generalizable knowledge through
scientific research failed to provide remedies or solutions to the problems that individuals
experience (Patton, 2002). Action researchers argued that generalized solutions could not be
applied to all situations and that the dynamics of inquiry led to finding solutions that could be
applied in particular contexts or settings. Therefore, the primary purpose of systematic inquiry is
for individuals to experiment with solutions and to collect data to reflect and improve in the
cycle of inquiry that can help solve the problems individuals face. At the same time, the inquiry
can lead to evaluating the effectiveness of the tool in use.
Action researchers believed the quality of the study had to be derived from “credibility,
impartiality, and independence of judgment; confirmability, consistency and dependability of
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 85

data; and explainable inconsistencies or instabilities (GAO 1987:53)” (Patton, 2002, p . 93).
Credibility is established when the researcher takes a neutral stance to the phenomenon that is
being studied. Therefore, as a qualitative researcher, the study involved gathering data as it
emerged and reporting the findings with no regard to an ultimate conclusion. To prepare for data
collection, the research team practiced observing the process of action inquiry as practitioners
from various institutions participated in the BESST workshops. Additionally, credibility is
reached when the researcher carefully reports data as it unfolds, whether it supports or
contradicts the study’s hypothesis. The establishment of systematic data collection procedures,
rigorous training of observers and interviewers involved in the study, collection of multiple data
sources, and triangulation of sources added to the credibility of the study (Patton, 2002; Stringer,
2007). When action researchers do not take a neutral stance, they bring their own perceptions
and interpretations to the problem that is studied. This may lead to issues of trustworthiness in
the researcher. To avoid this problem, the researcher must state his preconceived notions about
the problem ahead of time and return to the data frequently to establish that his prior perceptions
and interpretations are not clouding the study’s findings ( Patton, 2002).
Triangulation of data sources are required to increase credibility of findings (Patton,
2002). Patton (2002) believed that the greater the triangulation of the data sources, the more
rigorous the supporting evidence, which then validates the study. Triangulation of multiple data
sources and the use of diverse sources increase the credibility of the study by avoiding bias and
distortion during data analysis (Patton, 2002; Stringer, 2007). Triangulation can be
accomplished using both quantitative and qualitative approaches. Four types of triangulation
exist that can establish credibility of a study: data triangulation, investigator triangulation, theory
triangulation, and methodological triangulation.
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 86

Data triangulation involves the use of multiple data sources using the same method. The
data that was collected at different times or in different settings are cross-checked for
consistency. Cross-checking was conducted comparing observation and interview data,
comparing what participants saw in public and in private, comparing what was said over a period
of time, comparing data collected from different stakeholders who had different perspectives,
and comparing for consistency between interviews and documents or other written evidence
(Patton, 2002). Investigator triangulation provided credibility through the use of a number of
researchers or investigators who gather data. Multiple observers and/or interviewers were
involved in gathering data so as to reduce bias that could occur if only one person observed or
conducted interviews for a study. Similarly there was analyst triangulation whereby several
different people independently analyzed the data to arrive at findings. Analyst triangulation
allowed researchers to see data from multiple perspectives, without seeking consensus. A
variation of this form of triangulation is team triangulation which compares the results of a
goals-based team with a goals-free team. The goals-based team assesses the outcomes of a
particular program, and the goals-free team gathers data related to the problem and then
compares the data as it relates to the program. Action researchers use goals-free concepts as they
work off of hunches. The researcher is not evaluating an existing problem but why, those
involved in the problem, are unaware of what is causing the program to fail. The goals-based
approach assesses the stated outcomes or effectiveness of the program; this is summative in
nature (Patton, 2002). Developmental evaluation is focused supporting on-going decisions that
are made when changes are needed; it involves thinking and acting as new reactions or
discoveries are made by participants in an activity setting. There is continuous and on-going
development and rapid feedback involved in developmental evaluation (Patton, 2011).
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 87

Therefore, triangulating data gathered from a goals-based team data and a goals-free team can
improve the credibility of the study.
Theory triangulation entails the use of multiple lenses or perspectives to interpret data.
Theory triangulation helps in understanding how different assumptions and perspectives affect
the findings and interpretations of the findings (Patton, 2002). Two figures, Figure 1 and 2,
provide a visual representation of how theory triangulation occurs. Figure 1, the developmental
evaluation methods model, allows for various ways to look at what is being studied. For
example, the activity setting in the figure identifies the culture that exists at the setting. Figure 2,
the reflective practice model, shows that a particular culture at the setting will produce social
interactions which create different results depending on the knowledge and belief systems of the
participants.
Finally, methodological triangulation uses multiple methods to study the problem
(Denzin, 1978a and 1978b). Usually, a study can be studied either qualitatively or quantitatively
or as a combination of both. Multiple methods triangulation complements the data or shows
divergence in data which provides additional insights to the qualitative researchers. My study
used both quantitative and qualitative data collection methods. Quantitative data was collected
through evaluation questionnaires, and qualitative data was gathered through cognitive
interviews and observations.
Credibility can also be attained through referential adequacy whereby the study’s r esults
are drawn from the participants’ or stakeholders’ experiences and perspectives and not
interpreted from existing theoretical knowledge. Therefore, when concepts or ideas in the study
are supported, “the evidence will reflect the participant’s perspective and not what has already
been grounded in the form of a theory” (Stringer, 2007, p . 58). After observation data was
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 88

collected, follow-up cognitive interviews were conducted to strengthen evidence that what was
observed during the workshops and is in fact what the participants truly felt about the workshops
and is objective.
In qualitative research, the researcher is the instrument in the inquiry. Therefore,
essential information about the researcher and his/her training and the purpose for the research
study is required to establish credibility. As a researcher in CUE’s (cue.usc.edu) action-inquiry
study, training on how to complete observations and cognitive interviews was critical before
actual research began. Training on how to conduct and use the observation and cognitive
interview protocols was conducted prior to field site visits. Several practice sessions and three
cognitive interviews were required before actual field site visits were scheduled. Additionally,
practice on how to document and complete observations were completed at two different field
sites. Access to the field sites were initiated by CUE’s staff who communicated with site
participants who attended other workshops conducted by CUE. Arranging workshops at new
field sites were less challenging owing to the rapport that had been established between CUE’s
staff and the field site administrators who attended prior workshops. The field site
administrators then were tasked to select the project team’s participan ts based on institutional
goals.
Member check or debriefing involved participants being provided the opportunities to
verify that the data that had been gathered were accurate and if additional information was
required to make the data collection complete. Furthermore, when participants were given the
opportunity to clarify information, it allowed them the opportunity to deal with emotions and
feelings that may have clouded their initial interpretations of the workshop and the purpose of
the study (Stringer, 2007). This too established credibility in the study as the participants were
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 89

not coaxed or pressured to reveal incorrect information. Through the probing technique involved
in cognitive interviews, member checking and debriefing confirmed that the data collected were
accurate.
Transferability
Nationally, higher education policy makers are concerned that the number of minority
students they enroll and the number of minority students who earn degrees or transfer from 2-
year to 4-year institutions in STEM-related fields is largely disproportionate. Similarly,
California community colleges are beginning to address the problem that a large number of
minority students are enrolled in basic skills courses. Overrepresentation of Latino students in
basic skills courses may provide insight as to why Latino students do not progress in STEM
pathways the way they should. The persistence and retention rates for these students to progress
on to college-level work or transfer to 4-year institutions has prompted community colleges to
seek solutions to improve the inequities that minority students experience. While this study
focused on Latino students at a Hispanic Serving Institution, minorities experience inequities all
the time at institutions of post-secondary education. Many of these institutions are taking
measures to address these inequities by implementing action-inquiry projects to study their
transfer and completion policies. Action-inquiry studies that have been conducted can be used to
provide these institutions the ability to extrapolate the findings from one study to another context
or group. Extrapolation allows for speculation that findings from one study can possibly be
applied to other similar situations which may not have identical conditions. Therefore, findings
from one study, even though they are contextual, may provide potential for best practice
applications. Transferability of findings from one context to another can be achieved through
descriptive narratives. The information-rich samples and designs that were gathered in the study
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 90

can then be used to target other similar research studies at other institutions. My study involved
documenting and describing all data collection methods and procedures, including summarizing
observations and cognitive interviews. Data analysis procedures and reporting mechanisms were
also described for transferability purposes.
For example, the tools used in my study, CUE’s Benchmarking Equity and Student
Success Tool™ (2010) and Equity Assets Mapping tool involved using individual institutional
data to facilitate dialogue and decision making amongst institutional practitioners. The data that
was shown at the BESST workshop at DCC were used to identify the number of students who
have or have not taken the placement tests, or have been placed in basic skills math and English
classes. The data also reflected which STEM courses tend to keep Latino students from
progressing, transferring, and continuing in pathways. Based on this data, the various pipelines
of cohort students were easily seen. At the conclusion of the BESST workshop, participants
completed evaluation questionnaires to reflect on the workshop. When vivid descriptions and
details are available, other institutions can use this information–rich sample to begin their inquiry
projects. The setting, too, can be replicated since the study can be conducted in any educational
institution, as long as the required institutional information is available to begin the process of
inquiry. The inquiry process will aid institutions of higher education begin their assessments on
student outcomes and also address the accountability issues that stem from poor student
outcomes. Similarly, the participants who were involved in the inquiry process at DCC can be
found in any institution. At DCC, the participants who were involved in the workshops hold
many roles and responsibilities and the data collected from their interactions during the
workshops may provide other practitioners insights on how they too can begin the inquiry
process at their own institutions. Other action-inquiry projects conducted by CUE (cue.usc.edu)
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 91

that can be used for transferability purposes are Research and Planning Group’s Bridging
Research Information and Culture Initiative (BRIC, The RP Group, 2009-2012), Basic Skills
Initiative (BSI, California Community Colleges System Office, 2008), and Research and
Planning Group’s Research & Evaluation Services (RP Group, n.d.).
Dependability and Confirmability
Dependability of a study can be attained when a systematic approach to data collection is
conducted. This systematic approach can be completed using an inquiry audit where the
procedures are described in detail and the processes and procedures are documented and
available for review. Confirmability suggests that an audit trail is present to show that all
documents pertaining to the research study have been compiled and are readily available to
support the procedures that were actually conducted.
Audit trails essentially have a two-fold purpose. One is to establish the rigor of the
fieldwork and the second is to confirm that the data collected is accurate and that, based on the
data collected, bias can be reduced in presenting the findings. The audit trail also improves
accuracy and removes the researchers’ personal perspectives, which in turn provides grounded
empirical findings (Patton, 2002). Transparency, another manner to establish dependability and
confirmability, can be attained when all documents, data, and other evidence are compiled and
available to support the validity of the study. Documentation of the methods used and its rigor
will show that a high-quality study was carried out.
Limitations
Several limitations may exist within the study. I have only recently been involved in the
field of research and action inquiry. My participation in the action inquiry was limited to three
workshops during the fall of 2012 and spring of 2013. Since my participation in this study takes
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 92

place during year two of a 5-year collaboration project between CUE (cue.usc.edu) and DCC, the
data that I was able to gather in a 1-year time frame, may not reflect the impact that CUE’s
action-inquiry tools have at my field site. The data, however, may reflect some commonalities
between my field site and institutions who also participated in CUE’s action-inquiry activities.
As a researcher, my credibility was acquired through training and experience. Role playing in
workshops, as a participant and an observer, provided additional insight to action inquiry. To
overcome my limited involvement in the field of action inquiry, extensive reading on literature
involving action and participatory research was conducted.
Another limitation to the study involved the observation data source. Participants in the
workshop, even though their activities were not recorded, may have felt uncomfortable when
unfamiliar faces were present in the setting and/or answering questions during cognitive
interviews. Therefore, they may not have conducted themselves in a typical manner and the data
that was collected may not have been accurate. Sampling, too, may have been another limitation
in the case of my one-community-college case study. Limited sampling may have affected the
credibility of the study since small or limited samplings as in the one-community-college case
study, or the limited selective participants, may have seemed biased for those who did not
understand the reasons for the intended design (Patton, 2002). However, the multiple member
participants whom I interviewed compensated for the one-case study sampling. Time was
another constraint when coordinating interviews with the action-inquiry participants. Educators
have busy schedules teaching classes and attending to other faculty obligations like serving on
committees and participating in campus-wide meetings. They may not have had the time to
participate in cognitive interviews after their participation in the workshops. Additionally,
administrators, too, have many additional responsibilities on and off campus that they are
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 93

committed to that limited their availability to be interviewed. Other participants may have been
afraid to be interviewed since they may have believed the interviews may force them to divulge
information that they were not comfortable revealing.
Best practices and evidence-based practices work best when implemented to simple
problems. However, in this case study, the research surrounded the belief systems that existed
with practitioners. In cases where practitioners are unaware that their beliefs, values, and
assumptions affect the success of their students, there cannot be a simple best practice that can be
applied to address the problem. The context of the study led the reader to understand the
problem being studied and the reasons for the study, and the reader then was able to interpret
meanings to the study and how it was significant to his/her context (Patton, 2002; Denzin and
Lincoln, 2000). Finally, the credibility of findings related closely to the credibility of the
researcher. If the researcher is not trained in the process and procedures used in data collection
and the process developed to analyze the data to lead to the findings, then the credibility of the
findings will be diminished (Patton, 2002). If methodological rigor is absent from the findings,
credibility then becomes an issue. Methodological rigor includes the methods used in fieldwork,
data collection, and the systematic and consistent method of data analysis. The method of
coding establishes a classification system so the researcher can carefully record all findings
based on established themes which can be easily verified and explained. If the method for data
analysis is absent or is not clearly defined, the study will lack credibility. Additionally,
credibility limitations exist when fieldwork data derived from purposeful sampling are not
systematically analyzed to answer the research questions.
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 94

Reporting Results
Field notes were collected by the researcher and transcribed. Reporting the results took a
variety of forms, including descriptive text, tabular summaries focused on categorical data, and
thematic analysis based on emerging issues/new themes revealed by the data. Typical and
atypical data was reported, including the range of participant experiences. The observational
data was collected from a variety of activity settings that provided a contextual view of the
campus, but was then narrowed to a core group of stakeholders. The contextual view of the
activity setting at DCC was obtained through presenting the BESST tool, reviewing college
policy documents, attending to the discourse of the participants, and tabulating the evaluations of
workshop participants.
Data was tabulated into categories that reflected distinct views or thought processes that
the workshop participants experienced while they worked with a CUE developed tool, namely
BESST (2010). For the first category, attitudes/beliefs, it was vital to identify the attitudes and
beliefs expressed through emotive comments made by the workshop participants. Their
knowledge was made explicit since they verbalized it. Their social interaction in the activity
setting was established in the way they interacted with each other, their use of non-CUE
mediating artifacts (e.g., reports), the nomenclature used, the apparent roles, the cultural rules
and norms the workshop attendees adhered to, the community itself, and the division of labor.
The next classification was that of reflection which was often, but not always, exhibited in the
form of questions. CUE’s (cue.usc.edu) tools are designed to encourage the workshop
participants to ask “what if” and reflective statements that are essential for the next two
categories to occur–problem identification and experimentation/problem solving. These final
steps focused on the comments participants made that exhibited they were contextualizing
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 95

institutional problems and establishing steps to solve the issues. The penultimate category was
that of action (inaction)/experience where short narratives expressed prior attempts or non-
attempts to address problems. The final category was statements that are not coded which were
an array of actions and statements that revealed some shared themes and issues, such as
leadership, as well as policies of the institution, state, or nation.

STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 96

Chapter Four: Findings

This chapter includes findings from my developmental evaluation of CUE’s (cue.usc.edu)
involvement with the STEM Pathways Project at my field site, Dynamic Community College
(DCC). Findings describe interactions between members of CUE’s action research project and
DCC in addition to the two collaborating institutions that were part of the study. This study was
motivated by concern for equitable outcomes for racial-ethnic minorities in STEM. The purpose
of this study was to investigate the following research questions:
1. What is the experience of participation in equity-focused inquiry like for practitioners?
2. Do CUE’s tools raise awar eness of equity-minded practices among participants in
inquiry?
3. In what ways, if any, do CUE’s tools prompt organizational change?
This study reports data from my observations of two STEM Student Success and Equity
Forums, a series of cognitive interviews, and evaluation surveys. Participants were asked to
evaluate their experience interacting with CUE’s (cue.usc.edu) tools and discuss their roles in
producing equitable outcomes for racial-ethnic minorities at their respective institutions. My
results are a combination of triangulated data between my study and similar studies done by
colleagues. I used CHAT (Roth & Lee, 2007) as the conceptual framework and the cycle of
inquiry to explain the impact of CUE’s tools on practitioner beliefs.
Themes are related to willingness to adopt equity-minded practices. They were generated
based on practitioner responses using CHAT (Roth & Lee, 2007) to investigate the social
interactions that took place within activity settings. The themes that arose in response to
research question number one, “ What is the experience of participation in equity-focused inquiry
like for practitioners?” were as follows: Engagement, Valuing Dialogue, and Purposeful Data
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 97

Use. Similarly, the themes that arose in response to research question number two, “Do CUE’s
tools raise awareness of equity-minded practices among participants in inquiry?” were as
follows: Engaging in the Cycle of Inquiry, Questioning Practices, and Examining Beliefs about
Equity. Lastly, the themes that arose in response to research question number three, “ In what
ways, if any, do CUE’s tools prompt organizational change?” were as follows: Perceived
Control, Lack of Time, and Lack of Institutional Structures for Ongoing Inquiry.
Settings and Participants
Observations from this study took place over two different settings and periods (fall 2012
and spring 2013). In the first setting (fall 2012), there were 17 study participants present
consisting of mainly administrators, counselors, and staff. During the second setting, which took
place in spring 2013, there were 19 study participants present consisting of faculty,
administration, and staff. Two sets of main data were collected during this study, both revolving
around the two primary settings, which I will refer to as the STEM Symposiums in this chapter.
The first set consisted of staff and administrative perspectives and the second set consisted of
faculty, administrative, counseling, and staff data. The perspective of each classification of
employee helped identify strategies for impacting student outcomes for racial-ethnic minorities
in STEM. While the viewpoints of faculty, administration, and staff differed, they provided
insight regarding what could be done at each level to impact outcomes for minority students in
STEM.
During both STEM Symposiums, the meeting room consisted of several large
roundtables of eight. CUE (cue.usc.edu) staff was positioned in the front of the room lecture
style with a screen and projector. Pre-assigned seating was not required at either opening
session; however, most of the guests sat with people that they already knew. Part one of the
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 98

STEM Symposium which occurred during the fall of 2012 set the foundation for the STEM
Equity and Student Success Symposium which directly followed in the spring of 2013. Since
most of the inquiry discussions began during part one of the STEM Symposium, the content
below previews the initial dialogue that prompted inquiry discussions at DCC beginning in the
fall 2012.
During part one of the STEM Symposium, the atmosphere was very positive and
collegial. Attendees greeted one another and expressed their excitement about their
collaboration with CUE (cue.usc.edu) and the discussion regarding equity in STEM. There were
a few administrators who stood out during the very beginning of the symposium due to their
activity within the group. After an initial welcome and presentation by the Dean of the
Engineering and Computer Science department at DCC regarding the national need for STEM,
CUE staff facilitated the discussion by asking participants to define equity on their respective
campuses. The CUE action-inquiry team presented on “Moving from Equity Aspirations to
Equity Outcomes” and introduced the group to the terms “equity mindedness” and “deficit
mindedness.” The lead facilitator of CUE’s action inquiry team began to describe when
practitioners look at outcomes and disparities they often talk about and focus on students. She
then went on to describe when placing the blame on students takes away the power of the
institution to act on behalf of its students. CUE’s lead facilitator ended with a discussion on
holding institutions accountable for what they “can control,” and often time that does not include
the students. The team then introduced the concept of institutional agency and asked participants
to discuss their roles in promoting transfer access on campus for racial-ethnic minorities. The
forum convened during the STEM Equity and Student Success Forum created a safe space for
the inquiry process to occur. The configuration of the room and roundtables appeared to ease the
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 99

facilitation of dialogue as each group around each table discussed the data presented by CUE’s
action research team.
Participant Engagement
As a participant observer, I was able to witness participant engagement in these settings
and to see what the experience of inquiry was like for participants. Figure 3 illustrates
participant engagement levels in the spring 2013 workshop. Over half of the workshop
participants indicated that they felt engaged during the workshop.

Figure 3: Participant Engagement Levels
Source: Center for Urban Education (2013, slide 6)

STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 100

Figure 4 demonstrates the perceived effectiveness of CUE (cue.usc.edu) facilitators
during the spring 2013 STEM Equity and Student Success Symposium. Figure 4 reflects that 16
out of 19 symposium participants felt that CUE’s facilitators were effective in their presentation
of the information covered throughout the symposium. This data is relevant and useful because
participant satisfaction with the presentation of information demonstrates that they were paying
attention. Paying attention is a form of engagement. It is not an active form of engagement but
shows that participants felt that the content of the workshops was important enough to listen and
pay attention.

Figure 4: Effectiveness of CUE’s Facilitators
Source: Center for Urban Education (2013, slide 8)

STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 101

Participant Satisfaction
Figure 5 reflects participant satisfaction with the overall discussions facilitated during the
spring 2013 STEM Student Success and Equity Forum. According to Figure 5, 88% of STEM
symposium participants reported satisfaction with the spring 2013 Symposium by indicating that
they felt the discussions were meaningful. These results are significant because satisfaction
indicates that participants were listening and engaged. Satisfaction also indicated that the
content of the discussions met their expectations. Similar to Vines (2012), this study also found
that there was a desire for campus participation in equity and diversity discussions due to the
teams’ satisfaction with the inquiry that was taking place (p. 119).

Figure 5: Participant Satisfaction with Topics
Source: Center for Urban Education (2013, slide 9)

STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 102

Valuing Dialogue
The cycle of inquiry was evident during a roundtable discussion and showed how
practitioners went about using data provided by CUE to identify trouble spots that impeded
student success and recommended intervention points. As introduced in Chapter Three, the
cycle of inquiry creates a way for practitioners to examine their beliefs, unlearn old ways of
thinking, and participate in dialogue and social learning activities that lead to the discovery of
new knowledge (Dowd & Bensimon, 2009a; Tharp & Gallimore, 1988). The roundtable settings
created social interaction between departments at DCC and encouraged collaboration and
investigative inquiry. DCC participants began to discuss common practices and asked one
another, “Are we relaying the same messages to all students?” Another participant added, “We
often don’t tell students the courses that they need to take for their majors until it is too late”
(DCC Participant, Symposium Comment, March 26, 2013). Participants expressed concerns
about how departments worked in “silos” and didn’t make the connection between student
services and academic affairs.
After CUE’s (cue.usc.edu) research team provided each roundtable with campus data,
participants came together collectively to share what they discovered during their discussion.
When asked to report back to the group, one faculty member stated, “this is our first time seeing
this data, I don’t think other faculty on our campus have seen these numbers, at least not like
this” (DCC Faculty Member, Symposium Comment, March 26, 2013). This quote was in
response to the disaggregated BESST data that allowed practitioners to more transparently see
disparities for students of color. Typically, participants said, the data stayed in the department or
in the realm of the institutional researcher. The BESST tool prompted practitioners to question
how data was collected at the institution and the measures used for determining its accuracy.
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 103

Participants were able to connect the available data to their ability to make institutional
decisions. For example, one workshop participant stated, “The BESST tool allows us to see
students differently as they progress through the math pathway” (DCC Workshop Participant,
Symposium, March 26, 2013). Participants dialogued about how institutional decisions
regarding scheduling were not typically made based on students’ success and retention rates in
difficult math classes. Participants discussed decisions being made based on enrollment data and
felt more attention should have been paid to student progression through the math pipeline.
Another participant stated, “Now that we see what is happening and where, we can put things in
place to help students who are falling through the cracks” (DCC Faculty Member, Cognitive
Interview, April 8, 2013). In this study, inquiry promoted awareness and awareness came
through discussion using inquiry tools. For example, one practitioner stated, “The BESST has
been the primary tool and it has definitely created conversations around the data and has already
influenced people to start thinking about their questions to find out what is going on with the
data” (DCC Workshop Participant, Cognitive Interview Comment, April 2, 2013). Another
workshop participant commented, “I think this is one of the first times where we disaggregate the
data. The discussions were helpful as well as the understanding of what went well and what
didn’t go well ” (DCC Workshop Participant, Cognitive Interview, April 2, 2013). Ultimately,
BESST (2010) was perceived by participants as a tool to help keep the inquiry process going at
DCC. Similar to Brown’s (2012) findings , this study found that CUE’s presentation of equity
oriented artifacts, such as the BESST tool, brought about awareness of inequities through social
interaction at DCC.
In this study, initially, despite engagement and awareness, there appeared to be hesitancy
among some practitioners to focus on one particular group of students. When asked if
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 104

discussions surrounding racial and ethnic inequalities took place on campus, participants of DCC
replied, “We strategize and design our programs to meet the needs of our student population .
We know who our students are” (DCC Workshop Participant, Cognitive Interview, April 8,
2013). One DCC participant commented, “We have been diverse for a long time, we don’t target
specific groups of students. My son is Asian and he isn’t good at math . Inequalities can exist in
any race/ethnicity” (DCC Workshop participant, Cognitive Interview, April 8, 2013).
The experience of participation in equity-focused inquiry prompted practitioners to
reflect on what was occurring inside theirs classrooms and inside their departments. When
trigonometry was identified as a gatekeeper course for Hispanic students, a faculty member
stated,
I realize that DCC is a Hispanic Serving Institution but never once have I asked myself,
where are all of the Latinos in my class? I try to remain neutral with issues of race and
try not to see color. I treat all students the same. (DCC STEM Faculty Member,
Symposium Comment, March 26, 2013)
This statement was reflective of a common belief among faculty members. During the
symposium, a math faculty member who is a minority and also a woman, spoke about how she
stratified her class as most students did not learn math at the same pace nor did they enter her
class with the same background knowledge. This dialogue sparked much conversation among
practitioners at DCC.
As mentioned in Chapter Two, Ogawa et al. (2008) discussed how communities are often
characterized by shared sets of rules and norms. During my experience as a participant at DCC,
I learned that there was an unspoken rule for faculty to dismiss issues of race and barriers posed
by historical disadvantages and view all students as “the same. ” This viewpoint was believed to
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 105

not give any “one group of students an unfair advantage over the others. ” While I understood
this perspective, it did not address the more salient concern that certain groups of students
already had unfair advantages due to social, historical, and cultural geographies of difference and
dominant privilege.
Part of CUE’s (cue.usc.edu) commitment to DCC was to provide a different lens for
looking at data. This approach facilitated conversations in which members of DCC could see
disparities in student outcomes and select focal points for interventions. As a result of the
BESST workshop facilitated by CUE, practitioners at DCC noticed patterns of students dropping
out of the math sequence at trigonometry. During the BESST workshop, one workshop
participant commented, “we can get students to trigonometry but they aren’t stayi ng there” (DCC
Faculty Member, Symposium Comment, March 26, 2013). Trigonometry was identified as a
critical point of intervention and participants began to inquire about why there were very few
Latinos. During the investigative inquiry process in fall, 2012, participants discovered that one
of the reasons for the attrition of students at the trigonometry level was because trigonometry
was not a UC transferrable course so students avoided it when possible.
Purposeful Data Use
After the STEM symposium in the fall of 2012, three participants returned to their
campus to present the BESST data to math and science chairs. The outcome of this exchange
was a concentrated effort between faculty and staff to determine where students were falling
through the cracks in the math pipeline. After these barriers were discovered, department chairs
then took the information back to their respective disciplines. Here is what one administrator
reported:
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 106

After we had the symposium, I was able to go back and present the same model on
campus. I did a 20-minute presentation using BESST (2010) to a couple of math faculty
members who could not be there at the symposium. They were very interested in the data and
looking at places where the students were falling through the math pipeline. The math faculty
chair then suggested taking the data back to her department, “we are interested in using the
BESST tool in course redesign for STEM students to see if it provides data to measure against”
(Administrator at DCC, Cognitive Interview, April 8, 2013).
Both the sharing of new knowledge collectively and the dissemination back to other
departments on campus showed the influence of CUE’s (cue.usc.edu) tools on practitioners’
behaviors. The inquiry discussions prompted practitioners to return to their campuses and
discuss disaggregated data and student outcomes. This example demonstrated one characteristic
of CUE’s tools, which was to influence institutional agency . Reporting back to the institution
demonstrated participants’ willingness to continue the cycle of inquiry, which was a key
component of the STEM Student Success and Equity Forum. Following the BESST workshop,
DCC wanted to make sure everyone saw the campus data and had an opportunity to contribute to
the discussion on closing equity gaps.
Findings from cognitive interviews also indicated that other campuses present during the
STEM Student Success and Equity Forum took their BESST data back to their campus and
presented it at STEM department meetings. During this meeting there were plans made to have
follow-up meetings and create action plans. Here is an excerpt from one cognitive interview, “ I
discussed it at a project leads meeting and we discussed that we would have another meeting to
actually take action and create action plans” (DCC Administrator, Cognitive Interview, April 6,
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 107

2012). By addressing equity gaps, workshop participants put themselves in a position to act as
student advocates on a broader scale.
While there were complaints that the faculty voice was missing, workshop participants
felt that they were able to evaluate and understand some of the barriers for STEM students but
needed more faculty input to create sustainable changes. For example, participants believed that
the faculty members worked on the front line with students, were involved directly with the
coursework, created, designed, and modified the curriculum. They also believed that faculty
members were the ones who knew best what was working and what was not. Faculty members
knew “what it might take to improve or create the necessary curricula for students in a STEM
major to be successful in it” (DCC Workshop Participant, STEM Symposium, October 12,
2012).
This study showed that inquiry could be used to focus on barriers to success in STEM
courses. One participant commented, “Nothing beats h aving the data to show you areas that you
need improvement in” (STEM Faculty, Cognitive Interview, April 6, 2013). During the
cognitive interview process, participants discussed how they were able to take the disaggregated
data in STEM outcomes back to campus and share the information with other faculty members.
The inquiry process encouraged continuing discussions on campus regarding math outcomes for
STEM students. When I followed up, practitioners informed me that they continued to hold
discussions reflecting on the patterns of data revealed during the STEM symposium. The data
presented at the STEM symposium helped practitioners formulate discussions with other
program leaders on campus and prompted discussions around what could be done in the future to
decrease racial-ethnic equity gaps in STEM.
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 108

Participants expressed that they cared about student progression and believed that all
students should have the same chance. This is one of the reasons I was told that DCC wanted to
disaggregate their data, “to see how everyone was doing, ” and possibly redesign courses to
become more pedagogically inclusive to racial-ethnic minorities in STEM majors. During the
Success and Equity evaluation, seven of eight respondents agreed that CUE’s Equity Scorecard
Workbook effectively guided their understanding of using BESST (2010) to identify equity gaps.
As the participants of the STEM symposium explored their disaggregated data,
discussion began around naming the stakeholders responsible for making changes or
improvements to the data. One participant commented, “I think that it is very helpful for faculty
and also for the whole college to have a discussion and create places to investigate further
obstacles along a student’s journey ” (DCC Faculty, Cognitive Interview, April 2, 2013). This
comment revealed an aspect of this practitioner’s beliefs as well as the typical roles and division
of labor on campus. It shows that this participant believed in a holistic or team approach to
discussing student barriers in STEM. Participants also discussed the need to get math and
science students additional help outside of the class and to work with students directly.
Results also revealed that participant interaction with BESST (2010) was found to be
purposeful. Many participants stated that prior to BESST, they didn’t have the capability to
manipulate the data and set equity goals. Participants also commented that interaction with
BESST showed them exactly which students were “falling through the cracks.” One major
characteristic of CUE’s BESST in prompting practitioners ’ desires to adopt beliefs and practices
that will promote equity is the disaggregation of data by race and ethnicity. The disaggregation
of data helped facilitate the action inquiry process during the STEM symposium by giving
practitioners a different lens through which to view data. Once practitioners had the raw,
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 109

disaggregated numbers showing students success and retention rates, they began asking inquiry
questions about internal campus practices and policies. One workshop participant discussed how
during the BESST activity, he was able to identify places where math interventions were
necessary. He commented about noticing data trends that were unrealized before and how
Hispanic students were not persisting in trigonometry. During the STEM symposium,
trigonometry was identified as a gatekeeper course for DCC. The inquiry process prompted
participants to investigate further reasons for this. As a tool, BESST was believed to allow
practitioners to see students differently as they progress through the pathway. One participant
commented, “It seemed like the BESST tool actually brought into light the areas where students
were getting stuck, it felt like an ‘aha’ moment” (DCC Workshop Participant, Cognitive
Interview, April 8, 2013).
Engaging in “The Cycle of Inquiry”
This section explains the various aspects of question posing, data use, and reflection on
educational practices involved in a cycle of inquiry. The cycle of inquiry below demonstrates
how the point of intervention, (in this case the two STEM symposiums and discussion forums),
had the potential to impact institutional structures and practices and ultimately impact student
outcomes. “If students are not aware of the importance of assessment, what are our offices doing
to communicate these beliefs to students” (DCC Staff, STEM Symposium, March 26, 2013)?
This question was a good example of an inquiry question posed by a workshop participant. It
demonstrated the reframing of a cognitive lens when the practitioner questioned his own
behavior and not the behavior of the students. Comments made during the symposium, such as
this one, reflected participants moving from a cognitive deficit frame to asking questions that
truly allowed the cycle of inquiry to really take place. Participants began to identify gaps in
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 110

service, reflect on current practices, and discuss ways to communicate the importance of the
assessment test to students. Using CUE’s (cue.usc.edu) equity model (see Figure 6), the cycle of
inquiry began when group settings were created during the STEM Student Success and Equity
Forum. After participants were asked to identify gaps in service, they began to reflect on
institutional and individual practices that were perpetuating inequalities. One of the practices
they reflected on was data use.
Workshop participants expressed discontentment with the ways they had been measuring
student outcomes prior to the STEM Symposium and exposure to CUE’s Equity Minded
processes. The main concern of some faculty and staff was that the institutional methods being
used to capture data were somewhat misleading. One participant commented,
We typically don’t look at that type of research . We tend to look more at snapshot data,
one thing at a time, passing classes or retention rates over time but never following one
whole cohort from one point in time from college entry until they reach their goal, trying
to identify where the gaps are. (DCC Faculty, STEM Symposium, March 26, 2013)
This statement spoke to the institutional culture at DCC, the rules and norms for typical
processes, and evaluation methods. Participants suggested that aggregated data not displaying
equity gaps for marginalized groups was a more comfortable discussion. Some participants
discussed knowing that gaps existed but not knowing how big they were or how to make
concentrated efforts to close them.
Participants seemed to enjoy dialoging about institutional responsibility and problems.
There was a focus on making purposeful solutions and general consensus for creating impactful
practices. During the inquiry process, the institutional researcher at DCC said she would go back
and pull certain data because she noticed gaps in the data during the process. This action
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 111

Figure 6: Cycle of Inquiry
Source: Dowd & Bensimon (2009b)

demonstrated the cycle of inquiry, as the participants reflected and asked questions regarding
their campus data, they began to identify missing information that they needed to go back and
pull in order to make meaningful changes to student policy. During the STEM Equity Forum,
the institutional researcher at DCC asked, “ Do we have the appropriate data we need to find out
information or to make the right decisions” (Workshop Participant, STEM Equity Forum, March
26, 2013). This question is reflective of an inquiry question prompted by CUE’s (cue.usc.edu)
tools that exemplifies the desired effects of the tools on workshop participants through the cycle
of inquiry. This example demonstrated the BESST tool as a probe to elicit questions from
workshop participants that may have otherwise gone unanswered.
Questioning Practices as a Result of Action Inquiry
After using the BESST tool and witnessing the disparities in mathematics courses for
Latino and African American students, participants began to dialogue about how institutional
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 112

data had been utilized and reported in the past. The disaggregated data not only revealed equity
gaps, it also revealed possible service gaps that the groups focused on during the asset mapping
activity. Service gaps described the areas where practitioners realized certain structures needed
to be in place but were not. For example, not all students were being advised by a counselor
upon admission, nor did all students receive scheduling information prior to enrolling in STEM
courses. Participants’ noted that due to poor communication with students, circu mstances arose
where students wasted time taking courses that they didn’t really need and/or were
underprepared for the appropriate STEM courses. During the asset mapping activity, DCC
selected a focal point and/or area to benchmark to creating equity goals. At that time,
participants identified campus resources designed to support the identified focal effort. During
the asset mapping activity, participants discussed partnering with other offices on campus to
communicate clearer, effective messages to students. Participants seemed to enjoy this activity
because it helped them identify what, if any, services were missing from their respective
campuses. My findings support that the facilitation of the inquiry process was a characteristic of
CUE’s (cue.usc.edu) tools that prompted equity mindedness and awareness among participants.
The activity setting, inquiry process, the dialogue among participants, roles, and division of labor
created the atmosphere for practitioners to reevaluate and reassess their beliefs, actions, and
practices. For example, while participants were discussing the assessment testing, one
participant stated, “If students are not aware of the importance of assessment, what are our
offices doing to communicate these beliefs to students” (DCC Staff, STEM Symposium, March
26, 2013). This comment showed the participant moving from a perspective focused on student
deficits to a reframing of the problem. This comment was reflective of an organizational practice
that DCC found could be improved during the inquiry practice.
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 113

Based on the work with CUE’s (cue.usc.edu) tools, several equity-minded practices came
up in participants’ dialogue and brainstorming . Findings showed that participants had a desire to
engage in actions outside the symposium and have more campus dialogue. One participant
stated that he wanted to have more meaningful conversations about the data and moving students
through the STEM pipeline at DCC but that it had been very difficult to get all of the necessary
stakeholders in the room at the same time. He stated, “We kind of look at data in silos within our
departments, but there is no mechanism to have campus-wide or institution-wide discussions on
big picture things such as moving students through the STEM pipeline at DCC” (DCC Faculty,
Cognitive Interview, April 2, 2013). Participants also suggested improvements in themselves
acting as advocates for equity, providing positive feedback and encouragement for students to
move forward in STEM courses, providing clarity in course syllabi, and encouraging student
self-advocacy.
Supporting Students
During the inquiry process, participants began to discuss the importance of students
having someone to “believe in them.” T hey began to identify strategies to improve racial-ethnic
outcomes in STEM. One suggestion that was given was to identify students early on in STEM
pathways who do well and make them course teaching assistants (TAs). By making Latino and
other underrepresented students course TAs, it was believed to empower them, draw out their
strengths, and reinforce a positive identity. Integrating racial-ethnic minorities into the system
by creating working relationships with faculty and mentoring relationships with other minorities
was also believed to give them a positive identity and encourage success in STEM. Identity was
identified as a factor that could drive or impede student success. Once participants realized that
it was important for their students to have a positive identity, they began discussing the roles that
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 114

they played in creating positive identity for STEM students. There was discussion about giving
underrepresented students in STEM a “science” identity . One faculty member stated, “ We can
tell students that they are good in math and science. We can emphasize areas were they have
strengths” (DCC Faculty, STEM Symposium, March 26, 2013). Participants then discussed how
they received their science identity. Common themes that arose during this conversation were
supportive teachers and parents who believed in them and encouraged their participation in
STEM courses.
During the inquiry process, both reframing of cognitive frames and the concept of
institutional agency and advocacy arose. For example, Jane Eyers (2013) of DCC spoke about
the importance of the syllabus as both a cultural and institutional artifact for students. She stated
specifically, “Students take behavioral cues and facu lty have a lot of influence. Students believe
that what is reflected on the syllabus is all they need to do to be successful.” She then went on to
say that “our attitudes may reflect that certain elements are optional ” (Jane Eyers, STEM
Symposium, March 26, 2013); therefore, creating an environment for students to fail. This
comment was an example of reflection, re-evaluation, and reassessment on behalf of the
practitioner. After Jane spoke, the group began discussing the messages communicated to
students by both faculty and offices on campus. There was a suggestion made to make the
course syllabus a document or artifact that students actually wanted to read. One participant
commented, “Students only do what they are required to do by f aculty” (STEM Faculty, STEM
Symposium, March 26, 2013). She then went on to explain that if students were not told that
they needed to see a counselor for scheduling, then they would not go see one, “ Students think
that if they do what is stated on the syllabus then they will be successful, and this is where
faculty can use their influence to impact student outcomes” (DCC Administrator, STEM
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 115

Symposium, March 26, 2013). Participants dialogued about the syllabus potentially being a
misleading artifact and discussed ways of supporting students through welcoming and
contextualizing language on the syllabus. There were comments made about students taking it at
face value and saying, “This is all I have to do to pass this course.”
One participant commented faculty should make more of an effort to reach students when
courses are offered online and that one example of this is a welcome statement. “Syllabi should
have a welcome statement” (DCC Faculty, STEM Symposium, March 26, 2013). An example
was then given about a faculty member who put together a short video for students to watch prior
to taking her course. This video complemented the course syllabus, familiarized students with
the faculty member ahead of time, and gave them the opportunity to get to know her history and
course expectations. Some faculty struggled with this concept. A Caucasian faculty member
stated, “I am a scientist and I didn’t have any of that ” (STEM Symposium, March 26, 2013)!
Some faculty members embraced the idea providing higher levels of student support for students
who needed it in addition to empowering students through verbal affirmations and by providing
additional resources. Other faculty members did not feel it was necessary. I observed that the
two faculty members who had the most difficulty adjusting to the concept of institutional agency
were Caucasian, one male and one female. The minority faculty members present seemed to
have less difficulty in embracing the need for institutional agency in this area. Since some
faculty members did not have these opportunities they could not empathize with current STEM
students who may need them. There continued to be a struggle during the discussion. One
faculty member stated, “Not all STEM students are cut out to be STEM students” (Faculty
Member, STEM Symposium, March 26, 2013). Another mathematics professor disagreed
stating,
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 116

We are weeding out our own students. We will not be able to meet the demands of the
workforce if we do not prepare our students. We have got to do better. Are we not going
to help students because we didn’t have the specialized services? (Faculty Member,
STEM Symposium, March 26, 2013)
During the discussion on student advocacy, participants also started to talk about their
responsibility to make information available to students and to clearly define transfer pathways.
Participants at DCC discussed how students were undecided at the community college level and
often faced other challenges. One participant then raised the question, “Students don’t know
what they want to do, so how can we help” (DCC Staff, STEM Symposium, March 26, 2013).
Another participant chimed in, “ We can help them identify their strengths and weaknesses.
Sometimes students don’t know what their strengths and weaknesses are ” (Workshop
Participant, STEM Symposium, March 26, 2013). This is an example of how the inquiry process
helped practitioners identify ways to improve their practices and better support students. During
this discussion, participants also talked about the importance of using students to help or impact
other students. They discussed faculty input into the development of STEM students and how
positive feedback encouraged students.
As institutional agents, workshop participants began to discuss how to contextualize
trigonometry to appeal to students in STEM majors. They realized that STEM students needed
trigonometry and were not taking it and also began to question whether the usage of assessment
tests would be beneficial to students. Participants did not feel the assessment tests were adequate
reflections of students’ skills and that an alternative type of measurement might have b een more
effective for students.
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 117

DCC participants agreed that by not encouraging students or pointing them to specific
resources or special programs, they were setting students up for failure. Participants began to
discuss how students needed to be able to do more than pass a class to be able to get through the
transfer process at a community college. Through the process of inquiry, practitioners realized
transfer channels were not clearly identified and that there was not a connection between the
information presented on the syllabus and the steps that STEM students needed to take in
preparation for transfer. The process of inquiry continued to prompt the following questions for
practitioners at DCC: (1) What avenues teach students how to take advantage of special
resources?; (2) Can faculty do a better job of informing students, and if so, is this their job?;
(3) Can faculty require students to seek resources?; and, (4) What is the institutional
responsibility to help students with the navigation and transitional processes?
Raising Awareness of Equity-Minded Practices
The previous section illustrated ideal characteristics of a cycle of inquiry. This section
shows the variety of participants’ responses to inquiry using CUE’ s (cue.usc.edu) tools. During
the breakout sessions, I observed CUE’s facilitator ask ing participants to take out their Defining
the Problem Workbooks. Each group had a laptop in the center of a roundtable and gathered
around and followed along as she explained the milestones. The participants watched as the
CUE facilitator explained the tabs in BESST (2010). For the most part, they appeared to be
interested and engaged. During the discussion, the CUE facilitator showed participants how to
interact with BESST. She hovered with her mouse over the columns and tabs displaying data by
race and ethnicity. She then went through DCC’s data to further demonstrate to participants how
to use the tool. Within the small groups, I could hear some shocked reactions related to the last
column that showed cohort success. The group at DCC seemed surprised with the low numbers
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 118

and low success rates of racial-ethnic minorities in STEM courses. It seemed like a “telling”
experience for them to see how many students actually persisted through the math sequence, see
Figure 7. The BESST tool showed participants starting cohort sizes, persistence and succession
in STEM Gateway courses, highest group rates, and cohort succession rates. Practitioner
reaction and surprise when looking at their campus data supports the finding that CUE’s tools
raised equity awareness among practitioners. This is one example of the types of reactions the
tools elicited. Other reactions took a more serious tone.
Reactions to the data in Figure 7 reveal a deeper level of concern. Figure 7 shows the
progression of freshmen in mathematics at DCC. While the numbers started off high and ended
progressively low, completion rates for racial-ethnic minorities were even more eye opening for
workshop participants once they began disaggregating milestones by race and ethnicity and
manipulating BESST. During the inquiry process, participants of DCC began asking themselves,
“Who is experiencing the largest gap? ” and “Where should we focus our energy?” DCC
participants seemed to understand how to manipulate the BESST tool and began looking at
patterns and trends in mathematics completion and retention for Hispanic and African American
students. Equity-minded inquiry caused participants to question whether or not their
organizational structure accommodated the needs of students of color.
Reflective practice also occurred within the cycle of inquiry. Practitioners began to
question how many sections of trigonometry were offered, who taught those sections, and how
accessible they were to students. There appeared to be a level of concern among participants and
they began discussing the data they found with other departments on campus. One participant
commented, “everyone needs to have access to a tool like the BESST ” (Workshop Participant,
STEM Symposium, March 26, 2013). During the BESST workshops that prompted inquiry
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 119

among participants, social interaction, knowledge building, reflection, analysis, problem
identification, action, and planning all took place within the activity settings. Overall, BESST
(2010) prompted social interaction by creating the forum for practitioners to discuss racial-ethnic
inequalities. The equity-asset mapping tool, which will be discussed in greater detail later on in
the chapter, created a forum for practitioners to discuss focal efforts in small discussion groups.
During part one of the STEM Student Success and Equity Forum, DCC spent quite some
time “defining the problem .” Although outcomes were apparent, participants chose to focus on
how they arrived at the problem. For example, there was some discussion about the math
pipeline and the sequence of courses that STEM students were required to take in order to be
“transfer ready.” There was dialogue among participants about why students fell out of the math
pipeline. Figure 7 displays the progression of first-time freshman who were placed in
Intermediate Algebra at DCC. After setting equity goals and creating benchmarks, there was
dialogue around Hispanic students. One participant commented, “We can get Hispanic students
into calculus but they aren’t st aying there” (DCC Faculty, STEM Symposium, March 26, 2013).
Faculty expressed concern regarding the time that passed between when a student was placed in
a course and the time the course was actually available. Participants also expressed concern
about STEM students’ skills diminishing and forgetting important math concepts during this
lapse in time period. BESST (2010) created opportunities for DCC to discover equity gaps.
During a cognitive interview, one workshop participant explained that the BESST tool gave them
a new way of looking at data. BESST assisted participants to single out milestones and gauge
how students were progressing throughout the pipeline in STEM courses. In particular, it
allowed them to see where they could make the most impact.

Figure 7: Dynamic Community College, BESST Data
Source: Presentation at CUE: STEM Student Success and Equity Forum, October, 2012
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 120

STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 121

Based on pooled data from surveys, interviews, and observations of activity settings of
inquiry teams, participants at DCC reported that significant changes surrounding racial-ethnic
equity were necessary at their institutions. Eighty-eight percent of survey respondent’s indicated
that CUE’s (cue.usc.edu) tools effectively guided their understanding of using BESST (2010) to
identify equity gaps. Ninety-percent of survey respondents, most of whom were not faculty
members, indicated that they had a strong sense of their own personal responsibility surrounding
racial and ethnic issues on their campus but that interventions regarding improving student
outcomes from racial-ethnic minorities should be faculty driven.
Organizationally, the participants of DCC supported the use of BESST (2010) for
evaluative performance. Participants felt the BESST tool could serve as a valuable resource to
help them set benchmarks for STEM students and track them to completion. Evaluation survey
results indicated that 10 out of 10 workshop respondents indicated that: They expected that the
BESST tool would be valuable for their institution. The BESST tool also caused practitioners to
look at and actually use the data to set institutional benchmarks. There was a shift from “here’s
what my department is doing” to here is what the institution is doing and what my depar tment
can do to impact what is going on at the institutional level.
Examining Beliefs Regarding Equity
After the STEM Student Success and Equity Forum, participants were asked to complete
an evaluation regarding the day’s activities and to participate in cognitive interviews. The
purpose of the evaluations was to determine if CUE’s (cue.usc.edu) tools prompted inquiry and
discussions that would cause institutions to create purposeful changes. During the cognitive
interviews, participants were asked to talk about their beliefs and feelings about equity on their
campus. They were asked to identify their roles and state whether or not they believed they had
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 122

the power to influence equity on their campuses. As shown above, my findings show that CUE’s
tools made participants aware of inequities in STEM courses. However, concentrated efforts to
continue the discussions sparked during the STEM symposiums were lacking. Figure 8 shows
over half of the symposium participants felt that their beliefs about equity was similar to their
peers. Most participants felt that equity discussions were necessary to continue at DCC but the
organizational structure was not set up for these types of discussions. As a participant observer, I
did not see much integration or collaboration across faculty, staff, administration, or disciplines.
Participants stated awareness of equity gaps but did not realize how big the gaps were for
specific racial-ethnic groups until exposure to CUE’s eq uity oriented artifacts. Findings from
both this study and Vines (2012) research on organization change suggested that action research
was effective in helping institutions uncover equity gaps and influence the interventions,
attitudes, beliefs, and behaviors of practitioners when addressing equity issues. Figure 8
provides insight to DCC participants’ beliefs about equity and campus climate surrounding
equity.
Figure 9 further demonstrates the impact of CUE’s equity orientated t ools on STEM
Symposium participants at DCC. Twelve workshop participants (N=16) responded that their
understandings of equity issues were deepened through the STEM Symposiums. This question
specifically addresses research question number two, “ Do CUE’s t ools raise awareness of equity-
minded practices among participants in inquiry?” Participant responses reflected beliefs that the
dialogue and facilitated activities led by CUE boosted their overall awareness of equity and
racial-ethic barriers on their campus. Findings from similar studies conducted on action research
supported the need for practitioners to take responsibility for equity in student outcomes. A first
step in this study was assisting DCC in uncovering inequalities in STEM for students of color.
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 123

Figure 8: Beliefs about Campus Climate Surrounding Equity
Source: Center for Urban Education (2013, slide 15)

Figure 10 captured participants’ sense of personal responsibility surrounding racial and
ethnic issues at DCC. As previously stated, there was evidence of individual accountability
among the leadership team and workshop participants, but cognitive interviews and follow-up
activities showed little evidence of conversations and interventions continued beyond the STEM
Symposiums. Interviews with workshop participants showed little evidence of the type of follow
through required for substantive organizational changes on a broader scale to be made.

STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 124

Figure 9: Impact of STEM Symposium on Equity Awareness
Source: Center for Urban Education (2013, slide 11)

Although participants were able to brainstorm ideas for improvement, there was a lack of
individual accountability for taking such action. When asked about individual roles in STEM
student success, most practitioners, unless they were directly involved in the management of the
HSI STEM grant, believed they did not play a significant role in racial-ethnic outcomes for
STEM specifically. One participant commented, “I don’t feel like I have an individual role in
discussing racial-ethnic equity or student outcomes, do I feel we should all work on it, sure”
(DCC Staff, Cognitive Interview, April 2nd, 2013). Other practitioners stated that they simply
didn’t have the time to follow -up on the discussions held during the STEM symposium if they

STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 125

didn’t have a direct role with the managing of the grant . Actually, participants stated that they
had very little involvement with the HSI-STEM grant outside of the discussion forums. Data
from this study revealed very little being done on an individual level to impact student outcomes
outside of Title V initiatives.

Figure 10: Practitioner Beliefs Regarding Personal Responsibility to Impact Equity
Source: Center for Urban Education (2013, slide 13)

Steps toward Organizational Change
Figure 11 shows the asset-mapping tool used by DCC to identify service gaps on campus.
The inside of the diagram is the area where workshop participants were asked to write down

STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 126

areas of concern regarding inequities on their campus that they wanted to focus on. These areas
of concern were identified as “focal efforts .” The outside of the diagram is divided into five
categories: structures, policies, educational core, programs, and data practices surrounding equity
gaps.

Figure 11: Asset-Mapping Protocol
TM2

Source: CUE (2013, p. 4)

2
Copyright 2013, University of Southern California, Center for Urban Education Rossier School of Education. All Rights Reserved. The contents
cannot be copied or disseminated without express written permission from the Center for Urban Education

STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 127

This activity served to be a crucial part of the STEM Student Success and Equity Forum
because it allowed participants at DCC to identify existing practices, services, and resources
impacting student outcomes. Participants were able to identify practices that they wanted to
change and all of the structures currently in place to support those practices. Based on the five
categories, Table 8 is included as developed by CUE (cue.usc.edu) that includes examples of the
policies, structures, educational core, special programs, and data practices investigated by DCC
during the inquiry practice. When focusing on the BESST data and the math sequence,
participants reflected on curriculum, pedagogy, formal communications to students, assessment
policies, and instructional practices. This reflection can be seen throughout the chapter.

Table 8
Asset-Mapping Protocol
TM3

Categories Examples of Assets
Policies Admissions/Assessment/Course Placement
Enrollment/Add/Drop/Advising Policies
Hiring Guidelines
Compensation Policies
Financial Aid

Structures Schools, Colleges, Learning Centers
Course Offerings and Course Schedules/Section Mix
Teaching Loads
Multicultural Centers
Advising Centers
Formal Communications

3
Copyright 2013, University of Southern California, Center for Urban Education Rossier School of Education. All Rights Reserved. The contents
cannot be copied or disseminated without express written permission from the Center for Urban Education

STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 128

Table 8 (Cont’d.)

Categories Examples of Assets
Educational Core Curriculum
Pedagogy
Instructional Practices
Faculty Professional Development
Deans and Department Heads Leadership Development
Student Services
Service Learning

Special Programs Summer Bridge
TRIO
Transfer Academy
Peer Mentoring
Supplemental Instruction
Research Internships
Study Abroad

Data Practices or Campus
Self-Assessment

Routine Use of Data Disaggregated by Race/Ethnicity
Peer Benchmarking
Institutional Self-Assessment/Inquiry
Developmental Evaluation

Source: CUE (2013, pp. 1-2)

Prompting Organizational Change through Equity Inquiry
Figure 12 shows the potential of CUE’s (cue.usc.edu) tools to impact organizational
learning and change. In response to research question number three, “In which ways, if any , do
CUE’s tools prompt organizational change?” s urvey responses reflect that workshop participants
were optimistic in taking steps to improve equity at DCC after participating in CUE’s action
inquiry processes and learning about equity-minded practices. Findings also supported that
action research methods increased individual awareness and accountability amongst
practitioners.

STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 129

Figure 12: Symposium: Practitioner Beliefs Regarding the Impact of CUE’s Tools on
Influencing Equity
Source: Center for Urban Education (2013, slide 12)

My findings indicated that an environment or forum was created that did not exist before,
which correlates with Stanton-Salazar’s (1997) research regarding using campus networking
characteristics and resources to support and empower underrepresented students. For example,
one faculty member stated during a cognitive interview,
We don’t have opportunities to mix with other faculty members across STEM disciplines
and certainly not across institutions to talk about our students. This workshop gave us a
chance to discuss challenges we face both inside and outside of the classroom when it
comes to our students. (DCC Faculty, STEM Symposium, March 26, 2013)

STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 130

By having three institutions present during the STEM symposium, practitioners were able to
share ideas and best practices as well as institutional successes and failures.
During the process of institutional inquiry, the following questions were addressed:
(1) What are the institutional challenges that result in inequitable educational outcomes?;
(2) How are our current practices failing our students?; (3) What is going on in and outside of the
classrooms that we can change?; and, (4) Why do we need to make a change? CUE’s
(cue.usc.edu) tools created new institutional settings for dialogue to take place. During the
dialogue, institutional stakeholders questioned practices that could be improved for the success
of racial-ethnic minorities. One participant commented that it was beneficial to have all of the
colleges present, “ I thought it was good to have different perspectives coming into this type of
discussion. Some colleges may have solutions that we are not familiar with and can bring those
to the table” (Workshop Participant, STEM Symposium, March 26, 2013). Participants believed
they should provide positive feedback and encouragement for students to participate in STEM
courses, create and implement course syllabi that were meaningful, and develop student
advocacy. In this last case, practitioners expressed interest in working with STEM students as
early as high school. One participant commented, “We can do some work with the high school
classes; we are currently doing counseling there” (Workshop Participant, STEM Symposium,
March 26, 2013). Another participant commented, “We need to put emphasis on the high school
target population and encourage students to take math during their senior year” (DCC Faculty,
STEM Symposium, March 26, 2013). During this dialogue, participants discussed ways to
prepare historically disadvantaged students prior to attending community college.
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 131

Perceived Control
The perceived benefits of CUE’s (cue.usc.edu) action inquiry processes were that the
disaggregated data caused workshop participants to think beyond their data and formulate
questions, hypothesis, and in some cases, create possible strategies to solve the equity gaps for
students falling through the cracks in STEM. In regards to individual beliefs surrounding
perceived control around equity (see Figure 13), one participant commented:
It’s hard to make the time for these types of activities . We know that they are beneficial
for our institution, and we know that certain issues exist among our student population,
. . . but these issues don’t always fall within the scope of our day-to-day job functions.
(Workshop Participant, STEM Symposium, March 26, 2013)
When asked if they felt if the changes surrounding equity were under their control,
participants had the following responses: (6) Somewhat under my control, (3) Not under my
control, and (1) under my control. One workshop participant indicated, “ I can only impact
ethnic/racial equity related to those projects within my immediate involvement” (DCC Staff,
STEM Symposium, March 26, 2013). This participant discussed his role in advocating for
equity through involvements in campus committees and projects. Seven out of the 10
respondents indicated that significant changes surrounding equity were necessary at their
institution. Participants’ response to “feelings of control over equity” provided insight to the
organizational learning and change process at DCC. Respondents helped inform this study by
providing cultural and historical perspectives that had been embedded into the institutional
culture at DCC. One example of this was a comment made by one workshop participant who
said, “We typically don’t focus on race, we know who our students are” (DCC Workshop
Participant, Cognitive Interview, April 8, 2013). By not focusing on race, there appeared to be
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 132

an unspoken belief that opportunities were created equal for all students. After reviewing the
BESST data, it was apparent this was not so.

Figure 13: Practitioner Perceived Control Surrounding Equity
Source: Center for Urban Education (2013, slide 14)

During the inquiry process, both reframing of cognitive frames and the concept of
institutional agency and advocacy arose. For example, Jane Eyers, an Administrator at DCC
spoke about the importance of the syllabus as both a cultural and institutional artifact for
students. She stated specifically, “Students take behavioral cues and faculty have a lot of
influence. Students believe that what is reflected on the syllabus is all they need to do to be

STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 133

successful” (Jane Eyers, STEM Symposium, March 26, 2013). She then went on to say that “our
attitudes may reflect that certain elements are optional;” therefore creating an environment for
students to fail. After Jane made this comment, the group began to discuss the messages
communicated to students by both faculty and offices on campus. Engagement in inquiry
resulted in DCC using CUE’s (cue.usc.edu) tools and equity oriented artifacts to gain insight into
organizational practices that could be improved to positively impact student success.
The STEM Student Success and Equity Symposium in 2013 also focused on culturally
relevant instructional strategies and the institutions’ capacity to act as institutional agents .
Cognitive interview respondents interviewed after the event expressed concerns about the lack of
faculty involvement with the HSI grants and lack of efforts to improve outcomes in STEM for
underrepresented student groups. Here is an excerpt from one study participant:
I think as a team we were looking at the math sequence and we needed the math faculty
present. I was kind of disappointed that our college didn’t have more people . I don’t
believe all of the players were there. We only had two Deans there and a Vice President
who came and went because she had to leave due to pressing matters. I don’t think
anyone can sit on one side and say ‘I am going to make changes for the other side,
without really knowing what’s going on inside .’ I mean maybe the faculty have never
seen this data. That day we saw something needed to be connected from intermediate
algebra and trig so the faculty and administration need to dialogue. (Workshop
Participant, STEM Symposium, March 26, 2013)
Figure 13 demonstrates the impact of CUE’s BESST tool in aiding practitioners to
identify solutions for lessening equity gaps in the math sequence. Recommendations were made
during the first STEM symposium to get faculty buy-in and involve them in developing
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 134

interventions for STEM students in the future. Cognitive interviews also revealed that
participants expressed feelings that they didn’t have as much influence over impacting student
outcomes in STEM as faculty members. Since many of the participants in the first symposium
did not deal directly with STEM students on a day-to-day basis, as STEM faculty do, many of
them believed that their influence was limited. The STEM counselors believed that their
services could impact students but that the faculty ultimately had the power to influence change
in STEM outcomes for racial-ethnic minorities.
There was much discussion around how to get the academic arm of the college involved
in the HSI-STEM grant. Participants at DCC felt that students would be the most impacted by
faculty-driven interventions but expressed frustration with their inability to get faculty involved
in addressing achievement gaps in outcomes for STEM students. An administrator at DCC
stated the following regarding faculty involvement,
I think benchmarking, well from my perspective, has to be faculty driven. Faculty are the
ones on the front line with our students and involved in the coursework. They create,
design, and modify the curriculum. They are the ones who I believe know best what is
working and what isn’t . They would know what it might take to improve or create the
necessary curricula for students in a STEM major to be successful in it. (DCC
Administrator, STEM Symposium, March 26, 2013)
The director of the HSI-STEM grant at DCC had the following input regarding
institutional capacity to set equity benchmarks,
The hardest thing is getting the time to sit down with people. That’s been the hardest
thing even with this first time around, getting enough people in the room at the same time
to really have meaningful discussions about data. We kind of do it in silos within our
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 135

departments, and we do it in other small places on campus but there’s no like
mechanisms to have campus-wide or institution-wide discussions on big picture things
such as moving students through the STEM pipeline. (DCC Administrator, Cognitive
Interview, April 8, 2013)
This statement reflects a lack of institutional structure and capacity for dialogue at DCC.
Therefore, the STEM Symposium created the space for practitioners at DCC to collectively
discuss institutional structures that impact student outcomes.
In Figure 13, participants appeared torn in their responses and unsure of their roles in
equitable student outcomes. Survey responses suggested about half (N=9 out of 17) did not
believe that changes surrounding racial-ethnic equity were within their immediate control. It
would seem that both faculty and administrators would perceive that they held more control over
student outcomes at DCC, but findings from this study suggest otherwise.
Organizational Change: Accountability for Institutional Improvement
The process of inquiry that took place during the STEM symposium was to determine if
institutions could manage the resources they already had and fill in the gaps on what was missing
based on institutional context. Participants were able to define what the problems were and
leverage their interventions based on what the data was telling them. For example, one of the
goals of DCC, a public 2-year institution was to increase the number of transfer students who
transferred to the local 4-year institution in STEM. The discussion forums that occurred, as a
result of various stakeholders being present, allowed institutions to examine structures, policies,
and educational beliefs. The asset mapping activity, for example, created a dialogue for
practitioners to discuss societal and institutional culture, values, rules, and norms. One math
faculty at DCC stated the following during this activity:
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 136

When I sit in on discussions like this, it reemphasizes to me how unequal some things
are. It doesn’t even explain why they are that way, just the fact that they are . I don’t
think people realize how unequal things are and how far out of whack society is and that
hurts everyone. (DCC Faculty, STEM Symposium, March 26, 2013)
After DCC identified the area they wanted to focus on as it related to student outcomes,
they were then able to realize existing structures that impeded the transfer process for
underrepresented students in STEM. Specialized programs, such as Puente and Student Support
Services were found to positively impact transfer rates for Latino and African American
students.
Action-inquiry methods occurring during the STEM symposium sought to create
leadership within groups. The process of action inquiry prompted a few practitioners to return to
their institutions to become change agents and student advocates. I found that practitioners were
willing to become advocates and some did report information from the symposium back to their
campuses, but many reported that they simply “didn’t have the time” to follow up . The
following excerpt below is from a conversation with the institutional researcher at DCC:
This is what happens too . . . we are all so busy, I haven’t ref lected on the conversations
since the workshop, I’m not sure if the grants team has continued to have a discussion . I
have not been included in any further equity discussions and honestly it’s probably not
because anything other than time. I believe that we should continue this conversation,
have we? Not necessarily, I don’t think . I am not sure if they have but I haven’t. (DCC
Staff, Cognitive Interview, April 8, 2013)
Findings from this study provided little evidence of institutional capacity to carry out the work
that was initiated during the STEM Student Equity and Success Forums.
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 137

Lack of Institutional Structures to Ongoing Inquiry
When participants were asked to identify potential barriers to inquiry and/or equity goal-
setting activities in the future, they indicated the following:
(1) The campus is aware of the importance of data-based decision making; however change
sometimes occurs slowly due to various challenges;
(2) We need to engage a larger audience in the discussions including key faculty and
administrators. The biggest barrier will be finding a time that all stakeholders are
available to meet;
(3) Gathering accurate data;
(4) Creating strategies to move beyond talking, beyond concerns, etc.; and
(5) Identifying qualified teachers with proper training.
The potential value of sustained inquiry to address these problems was evident in this
study. For example, practitioners expressed the frustrations about faculty scheduling courses
during the times that they wanted to teach as opposed to the times that students actually needed
the classes. There was dialogue about limited course sections offered and lack of classes offered
on the weekend and during the evenings. One participant commented, “Our faculty don’t want
to teach on the weekends to accommodate students” (DCC Faculty, STEM Symposium, March
26, 2013). Unfortunately, there was no faculty present to discuss this further.
Once the dialogue started, participants began to discuss factors that they could control
such as course scheduling. This illustrated the potential value of inquiry on an ongoing basis.
One participant inquired, “When do the students want to be here? When do they want to take
classes? Do we consider that at all” (DCC Faculty, STEM Symposium, March 26, 2013)?
These questions reflect a sense of institutional accountability for creating schedules that work for
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 138

STEM students. Participants discussed the need to offer more nighttime classes to accommodate
working parents and minimize existing waiting lists. During a scheduling analysis, one DCC
participant found that they offered classes from 10am-1pm because that was when the faculty
wanted to teach. One participant commented, “Faculty are all gone after that time; there is no
class offered on the weekend . . . It is hard being a student these days” (DCC Faculty, STEM
Symposium, March 26, 2013). While course scheduling and instructor availability appeared to
be a big deal, recommendations for increasing course offerings often came with budget
implications.
This study found the need for more structures to exist to support inquiry at DCC. For
example, during the inquiry process practitioners discovered students were not being exposed to
STEM careers. During this discussion, practitioners dialogued about students not taking the
appropriate classes because they were unaware of the coursework required for various STEM
careers. Participants suggested that more students would enroll in math and science courses if
they knew early on the requirements for STEM careers. Participants discovered that lack of
adequate course preparation impeded success for STEM students because many students did not
find out the courses needed for their careers until after they invested several years at DCC. The
cycle of inquiry caused practitioners at DCC to reflect on practices and identify areas for
improvement. One participant commented that, “Students seem to be more clear about course
and career paths once they get to the 4-year university” (Workshop Participant, STEM
Symposium, March 26, 2013). Through the inquiry process, participants brainstormed ways to
expose students to STEM careers early on so they do not waste time taking unnecessary courses.
During the inquiry activities, practitioners also determined it was necessary for STEM majors to
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 139

declare a major and identify a career path early and that more counseling and faculty mentoring
was necessary at DCC.
Discussion
The findings of this study show that CUE’s (cue.usc.edu) tools raised awareness of
equity-minded practices among inquiry participants. Evidence of raised awareness was apparent
during the equity asset mapping and other activities that prompted DCC to question their
practices. For example, they questioned the messages being communicated to racial-ethnic
minorities in STEM majors. They also questioned the lack of guidance and career preparation
provided to students that was essential to scheduling. Participants showed appreciation for the
opportunity to dialogue with one another in inquiry settings where they could truly dissect their
institutional data. While CUE’s tools prompted participant desire for organizational changes,
there was little evidence to support that further consideration had been given to address
inequalities for racial-ethnic minorities in STEM beyond the actual symposiums. There were
complaints from DCC participants about not having enough stakeholders present, but more
dialogue was needed to show how DCC planned to move forward with the findings uncovered
during the STEM Symposiums. Also, this study is left with several unanswered questions as the
ambiguity in the practitions’ perceptions and the inability of DCC to decide who should be held
responsible for student outcomes. I think it would have been helpful for each class of employee
present at the STEM Symposium to define their individual and collective roles in student
outcomes. It often appears that the word accountability may bring about negative perceptions
when it is intended to bring about ownership. If the administrators, faculty, and staff present
could have identified and owned their individual roles in producing equitable outcomes, it may
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 140

have been easier for DCC to set clear expectations regarding “next steps” leading to
organizational changes.
Ultimately, the HSI-STEM grant, which was one part of this study, will need the support
of the entire institution to move toward making marked progress in closing equity gaps for
students of color in STEM. An apparent challenge of DCC in closing equity gaps in
mathematics specifically was getting staff to see beyond the individual goals of the grant and
into the bigger picture. The bigger picture captured how the institution as a whole continued to
serve racial-ethnic minorities and the effectiveness of DCC’s transfer efforts as it pertained to
historically marginalized student populations. In chapter five, implications of these findings, as
well as limitations of the research and suggestions for further research are presented.
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 141

Chapter Five: Recommendations
The Cycle of Inquiry and Practitioner Knowledge
During the STEM Student Success and Equity forums at DCC, institutional stakeholders
gathered in activity settings where learning, reflection, problem identification, analysis, action,
and planning took place. After practitioners (faculty, administrators, counselors, and
institutional researchers) were presented with institutional data regarding racial-ethnic outcomes
in STEM, the inquiry process was able to occur.
Practitioners at DCC expressed that they were very much aware of their campus
demographics even though they were unaware of the racial-ethnic background by grade
distribution and outcomes. Participants also expressed concerns regarding the rapid shift in the
campus culture and that of the surrounding community. There was dialogue regarding being
able to evolve at the same pace as the surrounding community. Cultural climate shifts included:
modifications to campus services and access, modifications to language, which included
messages communicated to students, and changes to the delivery of content and information
disseminated to students.
Connecting Theory and Practice
According to Gutierrez and Vossoughi (2010), when social processes are reproduced over
time, “they take on meaning as systematic patterns of social behavior, which becomes
‘naturalized’ as the ‘rules of the game’ of social behavior” (p. 88). Both Institutional Theory and
CHAT (Roth & Lee, 2007) explain how institutions are both socially and historically embedded
and shaped by dominant societal values. Based on the theory, the impacts of CUE’s
(cue.usc.edu) tools can only be sustained if the work that has begun at DCC becomes a part of
the institutional culture and simulated from year to year. According to theory, social practices
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 142

become naturalized behavior (Gutierrez and Vossoughi, 2010). Continuing the action-inquiry
process at DCC and creating activities each year where practitioners can discuss student
outcomes for racial-ethnic minorities and set goals for benchmarking driven by data could easily
become the norm for practitioners at DCC. Study participants revealed that discussions around
race and equity should occur more consistently across campus but that schedule constraints did
not allow for such meetings. Findings from this study reflect race consciousness and awareness;
however, the level of awareness doesn’t match the strategies aimed at increasing student
outcomes for racial-ethnic minorities in STEM specifically at DCC.
Cultural Historical Activity Theory (Roth & Lee, 2007) supports that when learning takes
place in activity settings, collaborative funds of knowledge are increased and add value in
creating more meaningful solutions. Roth and Lee (2007) stated that in order for practitioners to
have a direct impact on students, they must understand the student’s unique cultural norms,
beliefs, ideologies, etc. In a society where the number of Hispanic and racial-ethnic populations
is rapidly increasing, it is going to take a culture of practitioners who are able to validate the
experiences of these students. As campus demographics shift, faculty, administration, and staff
at DCC should be trained on addressing the needs of their current student population and be able
to adjust Eurocentric viewpoints and ideologies that do not apply to racial-ethnic minorities
based on social-historical differences. Today, decision-making resembles more shared
governance as opposed to the more traditional top down approach. With collaboration,
collective input is multidimensional. Multidimensional input based on faculty, administration,
staff, and student experiences make for meaningful interventions.
Findings from this study also support critical theories of race in addition to Cultural
Historical Activity Theory (Roth & Lee, 2007). My data revealed evidence that practitioners
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 143

often attributed inequitable outcomes to students’ educational backgrounds, com munities,
cultures, etc. While there was some individual ownership regarding accountability in student
outcomes, there was hesitancy to focus on one specific racial-ethnic group. My findings
indicated inquiry discussions on campus were beneficial to begin the dialogue regarding racial-
ethnic inequities in STEM.
Sustaining Change
According to (Somekh, 2006), CHAT and Action Research share the fundamental
assumption that knowledge emerges as aspects of practice–or “praxis” (p . 95). Findings from
this study revealed limited opportunities for new knowledge to emerge due to the limited time
and availability of faculty and staff. Findings also revealed that the settings created as part of
CUE’s (cue.usc.edu) action research project were not duplicated or emulated at the three
participating institutions. The foundation for CHAT addresses how individual units cannot
function autonomously. Based on theory, in order for CUE’s action research project be found
effective in this study, DCC would have needed to take the information presented back to their
respective campuses and build on it, continuing the discussions and using the data to drive
decision making. While findings from this study reflected that some campus dialogue occurred
following the STEM Student Success and Equity Forum, there was not enough dialogue to
impact each institution at the structural and organizational levels.
This study examined the impact of CUE’s (cue.usc.edu) tools on practitioners at DCC.
Ideally, faculty was expected to take the information learned from the equity discussions back to
their respective classrooms and continually develop culturally inclusive pedagogy for STEM
students. There was little evidence found to support that this occurred. On the contrary, four
faculty members stated that it was the administrator’s responsibility to increase success rates for
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 144

racial-ethnic minorities in STEM by providing additional resources outside of the classroom.
Faculty members at DCC held the belief that there was only so much they could do for students
on an individual basis outside of office hours.
This study found that practitioners’ presence and participation were necessary and vital to
initiate changes in institutional culture at DCC. Practitioners’ participation was critical during
this study; however, few institutional actors participated in the action research project. Cultural
Historical Activity Theory (CHAT, Roth & Lee, 2007) suggests that when communities are
actively involved in doing the work and involved first hand in producing knowledge/solutions,
then the results of the impacts on the surrounding community are greater. In this study, more
faculty were needed to determine the impacts of CUE’s (cue.usc.edu) action research processes
and more workshops were necessary in order to integrate CUE’s culture and equity minded
practices with those evident at DCC. Due to limited participation in the STEM Student Success
and Equity Forums, there was little evidence found to demonstrate long-term impacts on
practitioner attitudes, beliefs, knowledge, practices, and behaviors.
Findings from this study also revealed gaps between the HSI-STEM Grant at DCC and
faculty accountability. There was limited faculty buy-in and many were not available to witness
campus data disaggregated for racial-ethnic minorities in STEM. Cultural Historical Activity
Theory (CHAT, Roth & Lee, 2007) demonstrates activity settings consist of multiple
components. All components add to the creation and capacity to expand knowledge. Within
CHAT, each piece of information assists in the development of the big picture. A major
limitation of this study was that DCC’s collaboration with CUE was completed in fragmented
sections. The activities hosted during the STEM Student Success and Equity Forum were not
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 145

integrated into DCC’s campus culture . Instead, the symposiums occurred as single isolated
events.
Lastly, findings from this study may have been more significant if there was equal
representation from faculty, administration, and staff during the action research activities and
each representative body worked together to build long-term sustainable changes on campus at
DCC.
Recommendations
The recommendations outlined in this chapter are suggested to assist practitioners at
DCC, and institutions like DCC, to begin to close the racial and ethnic gaps in student outcomes
in STEM courses specifically. These recommendations have the potential to positively impact
overall transfer rates in STEM majors and practitioners’ responses to demographic shifts on
campus. Recommendations were informed by my findings, the work of other action researchers,
as well as my colleagues that participated in CUE’s (cue.usc.edu) action research processes and
methods at field sites similar to DCC.
Define Expectations for Accountability
My findings showed that action research helped practitioners to realize their power to
contribute to racial-ethnic student outcomes at DCC. The process posed questions and generated
settings, which required DCC to confront institutional failures, successes, and injustices that
affected students of color. The action research process appeared to be beneficial because it
allowed DCC to hone in on areas where it had previously excelled. The equity asset mapping
activity, for example, supported that the basic skills courses, the summer bridge programs in
STEM and Puente specifically produced better student outcomes than courses or pathways where
students were students not part of a cohort model. The action research process helped
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 146

practitioners at DCC inquire about their institutional effectiveness and long-term implementation
of processes, services, and programs.
After discussing practices that were harmful to underrepresented students, it was very
difficult for DCC to decide who was accountable for seeing marked improvements in student
progress. There was also debate between whether faculty or administrators ultimately held the
power to student success. While it is necessary to use data to drive decision making, this study
found communication should be clear and effective in expressing expectations for faculty,
administration, and staff regarding contribution to racial-ethnic student outcomes.
Align Program Objectives with Institutional Mission
Findings from this study revealed that often practitioners were unable to make the
connection between the overall grant objectives and the mission of the actual institution. There
appeared to be more dialogue needed between stakeholders at DCC to ensure that everyone
understood the purpose of the HSI-STEM grant. While each practitioner present during the
STEM workshops did not hold a specific role with the grant, every practitioner held a stake as it
pertained to institutional effectiveness and student outcomes. Data from cognitive interviews
revealed that three faculty members at DCC were not aware of the purpose of the STEM Grant
and that their first real exposure to equity gaps was during the STEM Student Success and Equity
forums. Since there were not many faculty members present, I am curious to know how many
other faculty members at DCC were not familiar with the intent of the HSI-STEM grant.
Cultural Historical Activity Theory (Roth & Lee, 2007) tells us that individuals are more likely
to participate in activities when they have direct involvement in the outcomes. Since learning
often takes place on campuses with social and cultural norms and expectations, exposing
practitioners at DCC to the STEM grant by allowing interface and collaboration across
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 147

departments might be a good way for stakeholders to make the connection between their
positions and the STEM grant. This practice, if repeated, may have the potential to change the
organizational culture at DCC. At the institution where I work as the HSI-STEM Director, even
when there is no direct connection between my department and other departments on campus, I
conduct presentations to familiarize the campus population with the grant. I also look for
opportunities to inform departments of how they can serve as a direct resource to STEM students
ultimately impacting student outcomes. Often the goals and objectives of the grant vary from
institutional goals and objectives. Sometimes the goals support one another but day-to-day
operations require staff to work compartmentally. If there is a way to tie institutional objectives
and processes like accreditation and program review to the grant, there may be more incentive
for faculty, administration, and staff to work daily on objectives that would eventually lead to
equitable outcomes in STEM for racial-ethnic minorities.
Specialized programs such as Student Support Services (SSS) and Extended Opportunity
Programs (EOP), which are housed on many campuses, provide students with transfer
information, admissions information, and financial aid assistance. These services assist students
with their overall transfer goals because the transfer process can be seen as a thread with many
interwoven fibers. Findings from this study reflect that it takes a village, and in this case many
departments on a college campus, to support the transfer process for STEM students. I try
incorporating the goals and objectives of the grant into the institutional mission around others
often as possible. I have found that when practitioners and faculty specifically understand the
bigger picture, they are more likely to be involved in smaller capacities such as: conducting
workshops, mentoring students, and chairing student clubs or organizations.
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 148

Expose Campuses to Equity-Oriented Artifacts Early
Practitioners completed surveys and cognitive interviews after attending the STEM
Student Equity and Awareness symposiums held at DCC during the period of this study. While
the survey captured participation satisfaction with CUE’s (cue.usc.edu) equity-oriented artifacts,
having the questions ahead of time might have made participants aware of CUE’s expectations to
be equity minded/orientation and culturally sensitive to students. I recommend that participants
at DCC are given questions regarding their views and beliefs about equity mindedness in
advance so that they can make a conscious effort to change or become cognizant of their
individual behaviors. Having the questions ahead of time might create the opportunity for self-
reflection and give practitioners a goal to work towards. For example, the following cognitive
interview question, “ I have a strong sense of my own personal responsibility surrounding racial
and ethnic issues on my campus” require d the practitioner to examine their role in equity gaps.
Having this question ahead of time might prompt a practitioner to define the role he/she wants to
play in racial-ethnic issues on campus and cause them to change how they interact with students.
Create Opportunities on Campus to Discuss Ways to Lessen Racial-Ethnic Achievement
Gaps
My recommendation for DCC is to continue discussions on equity and find ways to
improve transfer rates for racial-ethnic minorities in STEM. The dialogue that began as a result
of CUE’s (cue.usc.edu) action-research processes should continue. So far, findings from this
study have shown very little extended dialogue has taken place since the conversations began at
the STEM Student Success and Equity Forums.
From this study, I learned the importance of hosting inquiry groups on campus to discuss
student outcomes and possible barriers to racial ethnic student success. As a result of this study,
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 149

I hosted a STEM Faculty Equity Awareness Retreat in November of 2013 at the institution
where I am employed. The purpose of the faculty retreat was to discuss data trends for racial-
ethnic minorities on my campus. As a continual practice and as a follow up to the retreat,
participants have been asked to annually review the STEM pipeline and gatekeeper courses for
mathematics and chemistry. Mathematics and science courses were pre-identified because most
STEM majors are required to take them and they tend to be historically difficult subjects for all
students. The faculty retreat also addressed enrollment and completion rates for trigonometry,
pre-calculus, and calculus for Hispanic and other underrepresented student groups. We reviewed
graduation rates over a 6-year period for Hispanic students and identified completion gaps for
racial-ethnic minorities.
After the STEM Symposiums, three of the practitioners that I interviewed stated there
needed to be more follow-up discussions on their campuses. To aide in the facilitation of inquiry
groups on my campus, I organized an equity committee that meets monthly to discuss
enrollment, completion, and retention rates for all student groups. In addition, I formed inquiry
teams to review the impacts of specialized programs on underrepresented student groups using
performance reporting data and program objectives. During this study, and in order to increase
equity awareness on my campus, I sent a team of five practitioners: The College President, Vice
President, Dean of Institutional Effectiveness, Dean of Student Services and one faculty member
to the Institute for Equity, Effectiveness, and Excellence at Hispanic Serving Institutions hosted
by the Center for Urban Education (CUE) on October 10th and 11th of 2013. Following the
Institute for Equity, Effectiveness, and Excellence, the team and I brought the information back
to our campus and presented it during the Institutional and Strategic Planning Council (ISPC).
After our presentation, the team set dates to continue having inquiry discussions on campus
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 150

regarding equity minded practices and outcomes. Next, we identified the members to serve on
the equity committee. To date, six faculty members have expressed interest in this committee.
Strengthen Faculty Buy-In Final Recommendation
While the term racial-ethnic minority expands beyond African American and Latino
students, the general sense at DCC was that most programs and interventions were created with
racial-ethnic minorities in mind. There were instances during data collection when specific
questions regarding racial-ethnic student outcomes were evaded but 1/3 of the practitioners
stated that they simply didn’t know . They didn’t know how well their students were doing in
math, especially if it wasn’t their disc ipline. While there are many gatekeeper courses for STEM
students, math was identified in this study to be the discipline with the most required courses for
STEM majors.
Lack of faculty buy-in also continued to be a recurring theme among study participants.
From this feedback, sending teams to events such as the Institute for Equity, Effectiveness, and
Excellence struck me as an effective follow up. After sending a team of five administrators and
faculty across departments to the Institute for Equity, I was able to bring the dialogue back to
campus in order to facilitate future discussion groups regarding racial-ethnic equity. My
institution is now tracking the impact of support services on underrepresented students as a result
of the institute. The information presented during the Institute for Equity, Effectiveness, and
Excellence helped me to promote the STEM Faculty Equity and Awareness Retreat as my team
brought the information back and sparked the interest of other practitioners on campus who were
not in attendance.
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 151

Personal Recommendations for CUE and Similar Action Researchers
I recommend that CUE (cue.usc.edu) have more of a presence on the campuses that they
collaborate with. If the STEM Student Success and Equity Forums had taken place on campus,
there may have been greater faculty participation. In instances where faculty cannot attend
workshops, avenues such as webinars, videos, and workbooks would be helpful in creating
equity minded consciousness at collaborating institutions. Webinars could be offered on a
continual basis and faculty could be provided flex credit for attending.
My second recommendation is for CUE (cue.usc.edu) to train faculty and staff at DCC on
culturally responsive practices and cognitive reframing techniques. My recommendation is also
to broaden the culture of accountability at DCC by integrating CUE’s tools into the classroom
and into professional development settings. Cognitive interview findings revealed that the
impacts of CUE’s action research processes were limited to the actions of the inquiry teams
thereafter.
My third recommendation is that data driven interventions be supported by efforts such
as student focus groups and surveys so that practitioners really understand their students and
focus efforts on addressing student needs. Interventions should be data driven and student
centered but they will only be as successful as the students who benefit from them. My
recommendation is for practitioners to seek to understand the social and historical context of
their institution and familiarize themselves with their surrounding community.

STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 152

References
Association of American Colleges and Universities. (2007). Making excellence inclusive.
Retrieved from http://www.aacu.org/compass/inclusive_excellence.cfm
Astin, A. W. (1985). Achieving academic excellence. San Francisco, CA: Jossey-Bass
Publishers.
Baldwin, C., Bensimon, E. M., Dowd, A. C., & Kleiman, L. (2011). Measuring student success.
In Ronald B. Head (Ed.), Institutional effectiveness: New directions for community
colleges, (pp. 75-88). San Francisco, CA: Jossey-Bass Publishers.
Bauman, G. L. (2005). Promoting organizational learning in higher education to achieve equity
in educational outcomes. In A. Kezar (Ed.), Organizational learning in higher education:
New Directions for Higher Education, Number 131, (pp. 23-35). San Francisco, CA:
Jossey-Bass Publishers.
The Benchmarking Equity and Student Success Tool™. (2010). A brief guide on the tools of the
equity scorecard™ process. Los Angeles, CA: Center for Urban Education, University of
Southern California. Retrieved from
https://www.uwsp.edu/AcadAff/Pages/EqScorecard/CUE%20Tool%20Guide.pdf
Bensimon, E. M. (2005). Closing the achievement gap in higher education: An organizational
learning perspective. In A. Kezar (Ed.), Higher education as a learning organization:
Promising concepts and approaches, New Directions for Higher Education, Number 131,
(pp. 99-111). San Francisco, CA: Jossey-Bass Publishers.
Bensimon, E. M. (2006). Learning equity-mindedness: Learning equity-mindednes: Equality in
educational outcomes. The Academic Workplace, 1(17), 2-21
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 153

Bensimon, E. M. (2007). The underestimated significance of practitioner knowledge in the
scholarship of student success. The Review of Higher Education, 30(4), 441-469.
Bensimon, E. M., & Dowd, A. C. (2012). Developing the capacity of faculty to become
institutional agents for Latinos in STEM. Los Angeles, CA: University of Southern
California. Retrieved from
http://cue.usc.edu/assests/Bensimon_Developing%20IAs_NSF%20Report%20CUE_201
2.pdf
Bensimon, E. M., Dowd, A. C., Trapp, F., Alford, H. (2007). Missing 87: A study of the "transfer
gap" and "choice gap. A collaborative report by Long Beach City College and University
of Southern California Center for Urban Edcuation." Long Beach and Los Angeles, CA:
Long Beach City College and the Center for Urban Education, University of Southern
California.
Bensimon, E. M., Hao, L., & Bustillos, L. T. (2006). Measuring the state of equity in higher
education. In P. Gándara, G. Orfield, & C. Horn (Eds.), Leveraging promise and
expanding opportunity in higher education, (pp. 143-166). Albany, NY: SUNY Press.
Bernal, D. D. (2002). Critical race theory, Latino critical theory, and critical raced-gendered
epistemologies: Recognizing students of colors as holders and creators of knowledge.
Qualitative Inquiry, 8(1), 105-126.
Bloom, D., & Sommo, C. (2005). Building learning communities: Early results from the opening
doors demonstration at Kingsborough Community College. New York, NY and Oakland,
CA: MDRC. Retrieved from http://www.mdrc.org/publication/building-learning-
communities
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 154

Bowen, W. G., Chingos, M. M., & McPherson, M. S. (2009). Crossing the finish line:
Completing college at America's public universities. Princeton, NJ: Princeton University
Press.
Bowles, S., & Gintis, H. (1976). Schooling in capitalist America: Educational reform and the
contradictions of economic life. New York, NY: Basic Books.
Brown, R. Z. (2012). Equity for students of color through practitioner accountability (Doctoral
dissertation, University of Southern California). Retrieved from
http://gradworks.umi.com/35/51/3551453.html
Bustillos, L. T., Rueda, R., & Bensimon, E. M. (2011). Faculty views of underrepresented
students in community college settings: Cultural models and cultural practices. In P. R.
Portes, & S. Salas (Eds.), Vygotsky in 21st century society: Advances in cultural
historical theory and praxis with non-dominant communities, (pp. 199-213). New York,
NY: Peter Lang.
California Community Colleges. (2013). SB 1440: Associate Degrees for transfer. California
Community Colleges Chancelor’s Office & California State University. Retrieved from
http://www.sb1440.org/
California Community Colleges System Office. (2008). Basic skills initiative effective practices
database. Retrieved from http://bsi.cccco.edu/
California master plan for higher education: Major features. (n.d.). Retrieved from
http://ucfuture.universityofcalifornia.edu/documents/ca_masterplan_summary.pdf
The California State University. (n.d.). CSU graduation initiative. Retrieved from
http://graduate.csuprojects.org/
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 155

Center for Urban Education. (2013). Evaluation results: STEM student success and equity
symposium: Integrating culturally relevant instructional strategies into STEM courses.
Los Angeles, CA: Center for Urban Education, University of Southern California.
Cobb, P., Confrey, J., diSessa, A., Lehrer, R., & Schauble, L. (2003). Design experiments in
educational research. Educational Researcher, 32(1), 9–13.
CUE. (2013). Asset Mapping. Los Angeles, CA: Center for Urban Education, University of
Southern California.
CUE. (n.d.). Center for Urban Education. Los Angeles, CA: Center for Urban Education,
University of Southern California. Retrieved from http://cue.usc.edu
Denzin, N. K. (1978a). The research act: A theoretical introduction to sociological methods (2nd
ed.). New York, NY: McGraw-Hill.
Denzin, N. K. (1978b). Sociological methods. New York, NY: McGraw-Hill
Denzin, N. K., & Lincoln, Y. S. (2000). Handbook of qualitative research. Thousand Oaks, CA:
Sage Publications.
Dowd, A. C. (2005). Data don't drive: Building a practitioner-driven culture of inquiry to assess
community college performance. Indianapolis, IN: Lumina Foundation for Education.
Dowd, A. C. (2008). The community college as gateway and gatekeeper: Moving beyond the
access “saga” to outcome equity . Harvard Educational Review, 77(4), 407-419.
Dowd, A. C. (2011). Improving transfer access for low-income community college students. In
A. Kezar (Ed.), Recognizing and serving low-income students in higher education: An
examination of institutional policies, practices, and culture, (pp. 217-231). New York,
NY: Routledge.
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 156

Dowd, A. C., & Bensimon, E. M. (2009a). CUE frameworks for action, research, and
evaluation. Los Angeles, CA: Center for Urban Education, University of Southern
California.
Dowd, A. C., & Bensimon, E. M. (2009b). The social learning theory and tools underlying
CUE’s facilitation of action inquiry for equity. Los Angeles, CA: Center for Urban
Education, University of Southern California.
Dowd, A. C., & Tong, V. P. (2007). Accountability, assessment, and the scholarship of "best
practice." In J. C. Smart (Ed.), Handbook of Higher Education, 22, 57-119. New York,
NY: Springer Publishing.
Dowd, A. C., Bensimon, E. M., Watford, T., & Malcom, L. E. (2008). Does “2 + 2” still equal
four? Examining the “new math” of transfer access from community colleges to the
baccalaureate. Paper presentend at the Association for the Study of Highr Education,
Jacksonville, FL.
Dowd, A. C., Bishop, R., Bensimon, E. M., & Witham, K. (2012a). Accountability for equity in
postsecondary education. In K. S. Gallagher, R. Goodyear, D. J. Brewer, & R. Rueda
(Eds.), Urban education: A model for leadership and policy, (pp. 170-185). New York,
NY: Routledge.
Dowd, A. C., Bishop, R., Bensimon, E. M., & Witham, K. (2012b). Re-mediating accountability.
In K. Gallagher, R. Goodyear, & D. Brewer (Eds.), Introduction to urban education. New
York, NY: Routledge.
Dowd, A. C., Malcom, L. E., & Bensimon, E. M. (2009). Benchmarking the success of Latino
and Latina students in STEM to achieve national graduation goals. Los Angeles, CA:
University of Southern California.
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 157

Dowd, A. C., Malcom, L. E., & Macias, E. E. (2010). Improving transfer access to STEM
bachelor’s degrees at Hispanic Serving Institutions through the America COMPETES
Act. Los Angeles, CA: University of Southern California.
Dowd, A. C., Malcom, L., Nakamoto, J., & Bensimon, E. M. (2012). Institutional researchers as
teachers and equity advocates: Facilitating organizational learning and change. In
E. M. Bensimon, & L. Malcom (Eds.) Confronting equity issues on campus:
Implementing the equity scorecard in theory and practice (pp. 191-215). Sterling, VA:
Stylus Publishing.
Dowd, A. C., Sawatzky, M., & Korn, R. (2011). Theoretical foundations and a research agenda
to validate measures of intercultural effort. The Review of Higher Education, 35(1), 17-
44.
The Equity Scorecard™. (2010). A brief guide on the tools of the equity scorecard™ process.
Los Angeles, CA: Center for Urban Education, University of Southern California.
Retrieved from
https://www.uwsp.edu/AcadAff/Pages/EqScorecard/CUE%20Tool%20Guide.pdf
Fishbein, M. (2000). The role of theory in HIV prevention. AIDS Care, 12, 273–278.
Fishbein, M., & Ajzen, I. (1975). Belief, attitude, intention, and behavior: An introduction to
theory and research. Reading, MA: Addison-Wesley.
Fishbein, M., & Ajzen, I. (2010). Predicting and changing behavior: The reasoned action
approach. New York, NY: Psychology Press.
Greenwood, D. J., & Levin, M. (2005). Reform of the social sciences and of universities through
action research. In N. K. Denzin & Y. S. Lincoln (Eds.), Handbook of qualitative
research, (3rd ed., pp. 43-64). Thousand Oaks, CA: Sage Publications.
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 158

Grubb, W. N., & Badway, N. N. (2005). From compliance to improvement: Accountability and
assessment in California community colleges. City, ST: Prepared for and funded by
California Community Colleges Chancellor’ s Office, Carl D. Perkins Vocational and
Technical Education Act of 1998 (VTEA), Contract #03-0392. .
Gutierrez, K. D., & Vossoughi, S. (2010). Lifting off the ground to return anew: Mediated
praxis, transformative learning, and social design experiments. Journal of Teacher
Education, 61(1-2), 100-117.
Gutierrez, K. D., Morales, P. Z., & Martinez, D. C. (2009). Re-mediating literacy: Culture,
difference, and learning for students from nondominant communities. Review of Research
in Education, 33(1), 212-245. doi:10.3102/0091732x08328267
Harper, S. R., & Hurtado, S. (2007). Nine themes in campus racial climates and implications for
institutional transformation. In S. R. Harper, & L. D. Patton (Eds.), Responding to the
realities of race on campus. New Directions for Student Services, 120, 7-24. San
Francisco, CA: Jossey-Bass Publishers.
Harris, F., III, & Bensimon, E. M. (2008). The Equity Scorecard: A collaborative approach to
assessing and responding to racial/ethnic disparities in student outcomes. In S. R. Harper
& L. D. Patton (Eds.), Responding to the realities of race on campus. New directions for
student services #120, 77-84.San Francisco, CA: Jossey-Bass Publishers.
IPEDS. (n.d.). IPEDS data system. U.S. Department of Education. Washington, DC: National
Center for Education Statistics. Retrieved from http://nces.ed.gov/ipeds/datacenter/
Johnson, H. P., & Reed, D. (2007). Can California import enough college graduates to meet
workforce needs? Retrieved from http://www.ppic.org/main/publication.asp?i=750
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 159

Jones, C. P. (2000). Levels of racism: A theoretic framework and a gardener's tale. American
Journal of Public Health, 90(8), 1212-1215.
Lantolf, J. P., & Thorne, S. L. (2006). Sociocultural theory and second language learning. In
B. VanPatten, & J. Williams, (Eds.), Theories in second language learning, (pp. 201-
224). Mahwah, NJ: Routledge.
Levins, R., & Lewontin, R. (1985). The dialectical biologist. Cambridge, MA: Harvard
University Press.
Malcom L. E., Dowd, A. C., & Yu, T. (2010). Tapping HSI-STEM funds to improve Latina and
Latino access to the STEM professions. Los Angeles, CA: University of Southern
California.
Moore, C., & Shulock, N. (2007). Beyond the open door: Increasing student success in the
California community colleges. Sacramento, CA: Institute for Higher Education
Leadership & Policy, California State University.
Nasir, N. S., & Hand, V. M. (2006). Exploring sociocultural perspectives on race, culture, and
learning. Review of Educational Research, 76(4), 449-475.
Ogawa, R., Crain, R., Loomis, M., & Ball, T. (2008). CHAT-IT: Toward conceptualizing
learning in the context of formal organizations. Educational Researcher, 37(2), 83-95.
Ouimet, J. A., Bunnage, J. C., Carini, R. M., Kuh, G. D., & Kennedy, J. (2004). Using focus
groups, expert advice, and cognitive interviews to establish the validity of a college
student survey. Research in Higher Education, 45, 233-250.
Patton, M. Q. (2002). Qualitative research and evaluation methods. Thousand Oaks, CA: Sage
Publications.
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 160

Patton, M. Q. (2011). Developmental evaluation: Applying complexity concepts to enhance
innovation and use. New York, NY: The Guilford Press.
Pena, E. V., Bensimon, E. M., & Colyar, J. (2006). Contextual problem defining: Learning to
think and act. Liberal Education, 92(2), 48-55.
Reason, P. (1994). Three approaches to participative inquiry. In N. K. Denzin & Y. S. Lincoln
(Eds.), Handbook of qualitative research, (pp. 324-339). Thousand Oaks, CA: Sage
Publications.
Reason, P. (in press). Learning and change through action research. To appear in J. Henry (Ed.)
Creative Management. London: Sage. Retrieved from
http://www.peterreason.eu/Papers/Learning&Change_Through_Action_Research.pdf
Roth, W-M., & Lee, Y-L. (2007). "Vygotsky's neglected legacy": Cultural-historical activity
theory. Review of Educational Research, 77(2), 186-232.
doi:10.3102/0034654306298273
The RP Group. (2009-2012). Bridging research information and culture (BRIC) initiative.
Retrieved at http://www.rpgroup.org/projects/BRIC.html
The RP Group. (n.d.) Research & evaluation services. Retrieved at
http://www.rpgroup.org/about/css
Salazar-Romo, C. (2009). Remediating artifacts: Facilitating a culture of inquiry among
community college faculty to address issues of student equity and access (Doctoral
dissertation, University of Southern California. Retrieved from
http://digitallibrary.usc.edu/cdm/ref/collection/p15799coll127/id/406097
Schein, E. H. (1985). How leaders embed and transmit culture. In Organizational culture and
leadership (pp. 223-243). San Francisco, CA: Jossey-Bass.
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 161

Schön, D. A. (1988). Coaching reflective teaching. In P. Grimmett & G. Erickson, Reflection in
Teacher Education (pp. 19-29). New York, NY: Teachers College Press.
Seo, M. G., & Creed, W. E. D. (2002). Institutional contradictions, praxis, and institutional
change: A dialectical perspective. Academy of Management Review, 27(2), 222-247.
Sewell, W. H., & Hauser, R. M. (1980). The Wisconsin longitudinal study of social and
psychological factors in aspirations and achievements. Research in Sociology of
Education and Socialization, 1, 59-99.
Somekh, B. (2006). Action research: A methodology for change and development. New York,
NY: McGraw-Hill International.
Stanton-Salazar, R. D. (1997). A social capital framework for understanding the socialization of
racial minority children and youths. Harvard Educational Review, 67(1), 1-40.
Stanton-Salazar, R. D. (2001). Manufacturing hope and despair: The school and kin support
networks of U.S.-Mexican youth. New York: Teachers College Press.
Stanton-Salazar, R. D. (2004). Social capital among working-class minority students. In M. A.
Gibson, P. G ndara, & J. P. Koyama (Eds.), School Connections: U.S. Mexican Youth,
Peers, and School Achievement, (pp. 18-28). New York, NY: Teachers College Press,
Columbia University.
Stanton-Salazar, R. D. (2010). A social capital framework for the study of institutional agents
and their role in the empowerment of low-status youth. Youth and Society, 42(2), 1-44.
Stanton-Salazar, R. D. (2011). A social capital framework for the study of institutional agents
and their role in the empowerment of low-status students and youth. Youth and Society,
43(3), 1006-1109. doi:10.1177/0044118X10382877
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 162

Steele, C. M. (1997). A threat in the air: How stereotypes shape intellectual identity and
performance. American Psychologist, 52(6), 613–629.
STEM Toolkit: The Tools. (2013). The study. Los Angeles, CA: Center for Urban Education,
Rossier School of Education, University of Southern California. Retrieved from
http://cue.usc.edu/tools/stem/the_study.html
Stringer, E. T. (2007). Action research. Thousand Oaks, CA: Sage Publications.
Tharp, R. G., & Gallimore, R. (1988). Rousing minds to life: Teaching, learning, and schooling
in social context. Cambridge, England: Cambridge University Press.
Tinto, V. (1993). Leaving college: Rethinking the causes and cures of student attrition. Chicago,
IL: University Press.
U. S. Census Bureau. (n.d.). State & county quickfacts. Retrieved at
http://quickfacts.census.gov/qfd/states/06000.html
Valsiner, J. (1998). Developmental psychology in the Soviet Union. Bloomington, IN: University
Press.
Vines, E. (2012). Re-mediating practitioners’ practice for equity in higher education: Evaluating
the effectiveness of action research (Doctoral dissertation, University of Southern
California). Retrieved from
http://digitallibrary.usc.edu/cdm/ref/collection/p15799coll3/id/108140
Vygotsky, L. S. (1962). Thought and language. Cambridge, MA: MIT Press
Ward, Mary. (2008). An AVID program investigation: Examining an intervention program
within the context of social capital theory. (Unpublished dissertation): University of
Southern California, Los Angeles, CA.
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 163

Wenger, E. (1998). Communities of practice: Learning, meaning, and identity. New York, NY:
Cambridge University Press.
The White House. (n.d.). Education: Knowledge and skills for the jobs of the future. Retrieved
from http://www.whitehouse.gov/issues/education/k-12
Willis, G. B. (2005). Cognitive interviewing. Thousand Oaks: Sage Press.
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 164

Appendix A
Observational Data Protocol

TIME PERIOD/TASK
(#)

Site
(room temp, equipment,
environment, “artifacts”)

Mood
(emotions, general attitudes,
personality traits)

“Task” Performance
(CUE Tools; knowledge base for
engagement with presentation
or tool; expressed attitudes
towards or beliefs about tool;
e.g. use or value or design)

Social Context
(Who is there (social markers),
positions/ authority relations;
race relations; interactions, who
talking)

Behavioral Intentions
(expressed next steps, plans,
norms)

Environmental
Constraints
(expressed concerns or hopes,
perceived limitations of self,
team or resources)

Reflection/Analysis

STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 165

Appendix B
Cognitive Interview Protocol

Date: Interview # : Interviewer initials:
Start Time of Interview:

“For each following statement, you were asked to indicate the response that best reflects your opinion
and experiences. Please try and ‘think aloud” this time, as you answer each question.”

1. My understanding of equity issues was deepened through this symposium.
Strongly Agree /Agree /Neither Agree nor Disagree /Disagree /Strongly Disagree
Probes:

If agree: In what ways has your understanding deepened?

If neither or disagree: Please explain your reasoning.

Additional notes:

2. My participation in this symposium has made me optimistic about taking positive steps to
improve equity on my campus.
Strongly Agree /Agree /Neither Agree nor Disagree /Disagree /Strongly Disagree
Probes:

How did you arrive at your answer?

If agree: In what ways did the symposium make you more optimistic?

What positive steps are you considering? Have you taken any positive steps?

What specific symposium information, activities, or experiences made you feel more optimistic?

If neither or disagree: Please explain your reasoning.

In what ways could the symposium have been improved?
Additional notes:

STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 166

3. I have a strong sense of my own personal responsibility surrounding racial and ethnic
issues on my campus.
Strongly Agree /Agree /Neither Agree nor Disagree /Disagree /Strongly Disagree

Probes:

How did you arrive at your answer?

Do you feel that your sense of personal responsibility surrounding equity issues has changed since the
symposium? Or previous CUE symposiums?

How or how not?

Why or why not?

Do you feel that most of your colleagues at your institutions share your sense of responsibility?

In what ways?
Additional notes:

4. At my institution, the changes surrounding equity ethnic/racial equity are…
Not Under My Control /Somewhat Under My Control /Under My Control
Probes:

How did you arrive at your answer?

Do you feel that your experiences at the symposium impacted your ability to make equitable changes?
Why or why not?

In what ways do you feel you are able to make equitable changes?

What are any barriers to change that you can foresee in the future or have experienced personally in the
past?

Additional notes:

Thank you, again, for your time. Your feedback is very much appreciated.
STEM OUTCOMES: PRACTITIONER REFLECTION AND AGENCY 167

Appendix C
Recruitment Text and Ethical Commitments for interactions with Human Subjects

Dear Colleague,

The Center for Urban Education (CUE) at the University of Southern California is currently conducting a
developmental self-evaluation study. The study has three main goals. First to develop our evaluation capacity by
improving the validity of the inferences we draw from our workshop evaluation forms and other evaluative
processes; second, to improve our effectiveness in conducting action research for the purposes of improving equity
in higher education; and third to use the results to share what we learn with our action research partners.
The purpose of this letter is to let you know that as a participant in a CUE workshop or Equity Scorecard Evidence
Team, you may be asked to take part in an interview or a focus group conducted by a doctoral student who is part of
the CUE Evaluation Study research team. The doctoral students will also collect observational data at workshops
and team meetings. You can decline to participate in an interview or a focus group, or request to be omitted from the
data collected during workshop and team observations. Participation is strictly voluntary.
I am hopeful you are willing to support and contribute to CUE’s developmental evaluation study and our goal of
better understanding you and your colleague’s reactions, experiences, reflections, and action steps and the extent to
which these were facilitated through our action research processes and tools.
In conducting this study, we make the following commitments:

• To respect your professionalism and privacy by conducting the study in a confidential and ethical manner.
• To use your time wisely and well, and to minimize the “response burden” on any one individual.
• To report findings anonymously to external audiences, for example in dissertations or evaluation reports.
• To share findings with you and your colleagues in ways informative to your learning process. We will not
report findings in ways that would reveal the experience of any one individual (for example based on his or
her race, ethnicity, gender, or position). Instead, we will draw on findings from multiple participants on
your campus or aggregated across different field sites to communicate themes or issues that are pertinent in
your setting.

Should you have any questions or concerns, or should any arise as we conduct this study, please contact me by
phone or email: 213.740.5202, alicia.dowd@usc.edu.

Thank you for your consideration!

Yours sincerely,

Alicia C. Dowd
Associate Professor
Co-Director
Center for Urban Education

University of Southern California • Rossier School of Education • Waite Phillips Hall, Suite 702
Los Angeles, CA 90089-4037 • T 213.740.5202 • F 213.740.3889 • http://cue.usc.edu

Abstract (if available)

Abstract This study examined the experience of practitioners while engaged in equity‐focused inquiry at Dynamic City College (DCC), a public 2‐year California institution. The study investigated whether or not the tools developed by the Center for Urban Education (CUE) at the University of Southern California raised equity‐minded practices among study participants and prompted organizational change. The case study revealed that increasing racial‐ethnic equity in STEM Outcomes at Dynamic City College would have required faculty buy‐in, pre‐exposure to equity oriented artifacts, collective agency, and ongoing efforts to maintain on‐campus dialogue regarding racial‐ethnic inequities.

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

Action research as a strategy for improving equity and diversity: implementation constraints, outcomes

PDF

Evaluating the impact of CUE's action research processes and tools on practitioners' beliefs and practices

PDF

Equity for students of color through practitioner accountability

PDF

Re-mediating practitioners' practice for equity in higher education: evaluating the effectiveness of action research

PDF

Designing equity-focused action research: benefits and challenges to sustained collaboration and organizational change

PDF

Culture, politics, and policy implementation: how practitioners interpret and implement the transfer policy in a technical college environment

PDF

Evaluating the impact of CUE's equity scorecard tools on practioner beliefs and practices

PDF

Institutional researchers as agents of organizational learning in hispanic-serving community colleges

PDF

How community college basic skills coordinators lead organizational learning

PDF

Faculty learning and agency for racial equity

PDF

AB 540 community college students in Southern California: making connections and realizing dreams

PDF

Language and identity in critical sensegiving: journeys of higher education equity agents

PDF

Remediating artifacts: facilitating a culture of inquiry among community college faculty to address issues of student equity and access

PDF

Accumulating (dis)advantage? Institutional and financial aid pathways of Latino STEM baccalaureates

PDF

Encouraging student success: turning attention to practitioners and institutions

PDF

A developmental evaluation of action research as a process for organizational change

PDF

The growing gender gap among Latino students attaining a postsecondary education: a study of a minority male support program

PDF

The Transfer Academy: providing community college students with the informational, structural, relational, and cultural resources to transfer successfully to a four-year college

PDF

Making equity & student success work: practice change in higher education

PDF

Math faculty as institutional agents: role reflection through inquiry-based activities

Asset Metadata

Creator Patton, Lorena (author)

Core Title Practitioner reflections and agency in fostering African American and Latino student outcomes in STEM

School Rossier School of Education

Degree Doctor of Education

Degree Program Education (Leadership)

Publication Date 06/30/2014

Defense Date 08/20/2014

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag action inquiry,collective agency,faculty buy-in,Latino student outcomes,OAI-PMH Harvest,practitioner,STEM

Format application/pdf (imt)

Language English

Contributor Electronically uploaded by the author (provenance)

Advisor Bensimon, Estela Mara (committee chair), Dowd, Alicia C. (committee chair), Rueda, Robert (committee member)

Creator Email lorena.patton@norcocollege.edu,mrs.sophistic8@gmail.com

Permanent Link (DOI) https://doi.org/10.25549/usctheses-c3-425318

Unique identifier UC11286687

Identifier etd-PattonLore-2586.pdf (filename),usctheses-c3-425318 (legacy record id)

Legacy Identifier etd-PattonLore-2586.pdf

Dmrecord 425318

Document Type Dissertation

Format application/pdf (imt)

Rights Patton, Lorena

Type texts

Source University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Access Conditions The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the a...

Repository Name University of Southern California Digital Library

Repository Location USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA