University of Southern California Dissertations and Theses

Evaluating the impact of CUE's action research processes and tools on practitioners' beliefs and practices

(USC Thesis Other)

Evaluating the impact of CUE's action research processes and tools on practitioners' beliefs and practices

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content

EVALUATING THE IMPACT OF CUE’S ACTION RESEARCH PROCESSES
AND TOOLS ON PRACTITIONERS’ BELIEFS AND PRACTICES

by
Svetlana V. Levonisova

_____________________________________________________________________
A Dissertation Presented to the
FACULTY OF THE USC ROSSIER SCHOOL OF EDUCATION
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulfillment of the
Requirements for the Degree
DOCTOR OF EDUCATION

August 2012

Copyright 2012 Svetlana V. Levonisova

ii

DEDICATION
To my parents, Vitaliy and Valentina Levonisovy, and
to my husband, Armen Arakelov

iii

ACKNOWLEDGEMENTS
This dissertation has not been easy to write. It would have been impossible
without the help of many people: the guidance of my advisor and committee members,
help from my husband and family, support from my friends both in the U.S. and
Russia.
First and foremost I offer my sincerest gratitude to my chair, Dr. Alicia Dowd,
for her excellent guidance, caring, patience, and providing me with an exceptional
atmosphere for doing action research. Her wisdom, knowledge, and commitment to
the highest standards inspired and motivated me. I gratefully acknowledge my
committee members, Dr. Estela Bensimon and Dr. Jamy Stillman. Thank you for your
professional advice and insight.
A particular debt of gratitude is owed to my husband, Armen Arakelov, who
helped me realize my dream – this study in the U.S. I owe him for his heartfelt
support, dedication, love, and confidence in me.
More than is usually so, this dissertation has relied on work of other people
because it has been conducted as part of the collective Center for Urban Education’s
developmental self-evaluation study. I would like to thank all CUE’s researchers and
staff who participated in this research project with interest and enthusiasm: Misty
Sawatzky, Rosita Ramirez, Dominic Alpuche, Arlease Woods. I am very grateful to
my dissertation group members Chelvi Subramaniam, Peggy Smith, Rashitta Brown,
Lee Ann Cornell and others.
My mother, Valentina, elder brother, Vyacheslav, his wife and children deserve
special mention for their inseparable support and prayers. Living more than 6,000

iv

miles away from me, in Russia, they have been always encouraging me with their best
wishes and high expectations.

v

TABLE OF CONTENTS
DEDICATION ii
ACKNOWLEDGEMENTS iii
LIST OF TABLES vii
LIST OF FIGURES viii
ABSTRACT ix
CHAPTER 1: INTRODUCTION
The Challenges of Increasing Institutional Effectiveness 1
National Context: The Spellings Commission and Other
Voices of Accountability 3
Accountability in California Context 10
Statement of the Problem 19
Purpose of the Study 26
Research Questions 27
Importance of the Study 29

CHAPTER 2: CONCEPTUAL FRAMEWORK
Funds of Knowledge to Address Inequities 33
Action Research as Organizational Change 37
Designing New Forms of Educational Activities Using CHAT 42
Incorporating Critical Perspectives on Racialized Practices 49
Summary 53

CHAPTER 3: RESEARCH METHODOLOGY
Developmental Evaluation 60
Sample and Field Site 63
Data Collection 70
Data Analysis Procedures 80
The Action Research Process, Activities, and Design 90
Data Reporting 97
Standards of Review 99
Limitations 103

CHAPTER 4: RESULTS
Project Activities Analyzed Through CHAT and Action Research 107
Experiences in the Workshop and Webinars, and Reactions to CUE’s
Tools 115
Thematic Findings 118
Summary 143

vi

CHAPTER 5: CONCLUSION AND RECOMMENDARIONS
Significant Findings 146
Recommendations 150
Recommendations for CUE and Other Action Researchers 150
Recommendations for Higher Education Practitioners at MSU and
Similar Institutions 168
Recommendations for the Higher Education as a Field 170
Conclusion 171

BIBLIOGRAPHY 173

APPENDICES
Appendix A: Recruitment Text and Ethical Commitments for
Interactions with Human Subjects 183
Appendix B: CUE’s Tool or Workshop/Webinar Evaluation
Questionnaire Sample (Webinar Evaluation Form) 187
Appendix C: Observational Data Collection Template 188
Appendix D: Cognitive Interview Protocol (used after
the Document Analysis webinar) 189
Appendix E: Individual Interview Protocol 191
Appendix F: Document Analysis for Self-Assessment of
Culturally Inclusive Practices Protocol
(used at the Document (Syllabus) Analysis workshop) 195
Appendix G: Document Analysis for Self-Assessment of
Culturally Inclusive Practices for Students of Color
(used at Document (Syllabus) Analysis webinar) 203

vii

LIST OF TABLES

Table 3.1. Field Sites of the CUE Developmental Evaluation Study 66

Table 3.2. Summary of Data Collection Methods 74

Table 3.3. Data Analysis Codes and Summary Sheet for Analyzing CUE’s
Action Research Processes and Tools 89

Table 3.4. Deductive Data Analysis Codes 98

Table 4.1. MSU – Distribution and Sample Quotes of Tabulated Codes
From Data Collected at the Document Analysis Workshop
And Post-Workshop/Webinar Interviews 119

Table 4.2. MSU – Number of Statements in the Data Providing
Confirming or Disconfirming Evidence for Themes (N=6) 121

Table 5.1. Indicators Grouped by Perspective and Developed Over Time 154

Table 5.2. Proposed Indicators of the Syllabus Review Protocol for
Use During Problem Defining 161

Table 5.3. Proposed Indicators of the Syllabus Review Protocol for
Use During Problem Solving 163

viii

LIST OF FIGURES

Figure 2.1. An Activity Setting (CHAT) Framework 46
Figure 3.1. Developmental Evaluation Methods Inform
Multiple Modes of Research 56

Figure 3.2. Reflective Practice (Inquiry) as Driver of Racial-Ethnic
Equity in Postsecondary Education 59

Figure 3.3. MSU Field Site, Timeline, Activity Settings, and Data Collection 73
Figure 3.4. BESST Baseline Data: All MSU Students Who
Applied, Were Admitted, and Matriculated
in Fall 2010 (Simulated Data) 92

Figure 4.1. Activity Systems Model as Applied to the Document
Analysis Workshop 108

Figure 4.2. Aspects of Inquiry Articulated at the Document Analysis Workshop 114

Figure 4.3. Comfort with Talking about Race and Ethnicity with Colleagues 217

Figure 4.4. Participants’ Perceived Obstacles to Change at MSU 131

Figure 4.5. Views About Institutional Effectiveness to Enroll and
Graduate Students of Color 132

Figure 4.6. The “Racism” Cartoon as a Mediator of Thought 134

Figure 4.7. Views About the Workshop as Facilitating Reflection
on Pedagogical Practices 140

ix

ABSTRACT

This study examined the impact of the use of CUE’s action research processes
and tools in a social design experiment undertaken at one field site as part of a larger
developmental self-evaluation study conducted by CUE. Through the use of structured
inquiry activities and equity-oriented cultural artifacts (language, media, tools),
participants of this study were able to review and reflect on a key pedagogical tool:
course syllabi. The purpose of the study was to understand whether and how CUE’s
equity-oriented cultural artifacts (language, media, tools) mediate practitioners’ beliefs
and behaviors in regard to equity. It was found that involvement with CUE influenced
participants’ willingness towards behavioral changes to better serve their diverse
students. However, insufficient CUE’s tool applicability and a lack of support from
the wider university community, as perceived by participants in my sample, weakened
the prospectus of broader institutional impact beyond the participant group.

1

CHAPTER 1: INTRODUCTION
We seek not just freedom but opportunity.
We seek not just legal equity but human ability,
not just equality as a right and a theory but
equality as a fact and equality as a result.
Lyndon B. Johnson (from CUE’s mission)
The Challenges of Increasing Institutional Effectiveness
Since mid-20
th
century the persistence of inequality has been “the most durable
and robust problem” challenging higher education research and practice in the United
States (Luke, Green, & Kelly, 2010, p. vii). Disparate educational outcomes among
different racial/ethnic groups in postsecondary education have attracted scrupulous
attention giving rise to a number of national and state initiatives. The emphasis on
quality, productivity, and a return on public dollars led to the rise of performance
accountability for higher education (Dougherty & Hong, 2005). The goal behind this
movement was to make higher education institutions demonstrate their performance to
the public by showing gains in student learning, graduation rates, and placement in
good jobs; the hope was that performance accountability would influence institutional
behavior, thus leading colleges and universities to become much more effective and
productive in the process of educating all students (Dougherty & Hong, 2005). A
major thrust of federal and state accountability systems has become to improve
institutional productivity as demonstrated by graduation rates of students. However,
evidence of inequities in postsecondary access, experience, and outcomes among
racial-ethnic groups is still persistent. It appears that the formula: accountability =

2

institutional effectiveness = equity in student outcomes still does not work.
Scholars argue that current accountability structures and assessment practices are
not effective in bringing about change, racial-ethnic equity is missing from external
and internal accountability structures, and assessment as currently practiced is not
particularly race-conscious (Baldwin et al., 2011; Bauman, 2005; Bensimon, 2005;
Dowd, 2005; Dowd & Tong, 2007; Grubb & Badway, 2005; Gutierrez & Vossoughi,
2010). Importantly, they claim that the collection and reporting of data to demonstrate
productivity is not sufficient to promote practitioners’ understanding of how to create
more equitable learning outcomes for all students, “Data don’t drive; decision makers
do” (Dowd, 2005, p. 79). Scholars believe that in order to improve racial-ethnic equity
in student outcomes as well as assessment of institutional effectiveness in promoting
student success, it is necessary to focus on organizational learning and a culture of
inquiry – drivers of organizational change (Baldwin, et al., 2011).
This chapter explores the problem of inequitable educational outcomes from the
perspective of organizational learning. First, it discusses current purposes and
structures of accountability within the national context of the U.S. higher education. It
also highlights some prominent tensions and debates in regard to accountability as a
driver of institutional effectiveness. The chapter then examines structures of
accountability, assessment, and institutional effectiveness in California, identified at
the forefront of trends, “ … placing the economic health and social stability of the
nation at risk” (Moore & Shulock, 2007, p. 1). One of these trends is unequal
distribution of education and skills. Next, it provides discussion on organizational
learning and a culture of inquiry as alternative strategies to improve organizational

3

effectiveness and outcome equity. Scholars argue that these strategies keep the
promise to create change because they focus on practitioners, “ … the most influential
factor in student learning” (Gutierrez & Vossoughi, 2010, p. 103). In a culture of
inquiry practitioners are provided with opportunities to reflect on their own
pedagogical practices to learn more about them and their institution (Dowd & Tong,
2007). This provides the opportunity to identify where racial-ethnic inequities occur
and to design equity interventions. The chapter culminates with an introduction of
purpose, research questions, and importance of the present study. The study
investigated whether action inquiry using CUE’s action research processes and tools
brings about or shows the potential to bring about organizational learning and change,
improvements in institutional effectiveness, and as a result – greater racial-ethnic
equity in student outcomes. It contributes to the body of knowledge regarding
effectiveness of conducting action research using CUE’s tools for the purposes of
improving equity in higher education; thus increasing capacity for educational
innovation.
National Context: The Spellings Commission and Other Voices of Accountability
A 2006 report from the U.S. Secretary of Education Margaret Spellings’
Commission on the Future of Higher Education declared that among the “dramatic”
changes that postsecondary education needed was improvement of the “ … troubling
and persistent gaps between the college completion rates of low-income Americans
and their more affluent peers as well as between the nation’s growing population of
racial-ethnic minorities” (p. 17). Cook and Pullaro (2010) assert that while the U.S.
Secretary of Education’s Commission was not the first voice of concern about the

4

graduation rates of students at colleges and universities – in particular, disparities in
completion by race and income – it moved the conversation on postsecondary
accountability from input measures (e.g., access and enrollment) to output measures
(e.g., persistence and graduation rates). Since the time the commission report was
released, numerous articles, reports, and even some state and federal legislation have
called for more institutional accountability in general and for graduation rates in
particular.
In August 2008 The Higher Education Opportunity Act (Public Law 110-315)
(HEOA) was enacted. Reauthorizing the Higher Education Act of 1965, HEOA
contains “burdensome” new reporting, disclosure, and other requirements for
institutions that participate in Title IV (American Council on Education, 2008, p. 1).
They include, for example, posting on institutional websites a net price calculator that
provides estimated net price information to current and prospective students and their
families; campus safety reporting requirements; textbook cost containment, and others.
Beginning July 1, 2011, U.S. Department of Education (ED) plans to publish national
lists, naming the top 5 percent of institutions with (1) the highest tuition and fees; (2)
the highest “net price;” (3) the largest percentage increase in tuition and fees; and (4)
the largest percentage increase in net price. Institutions with large percentage increases
in either tuition and fees or in net price will be required to submit a report to ED
providing the reasons for the increase and the steps that will be taken to reduce costs.
ED will issue an annual report on those institutions and post their reports on the
College Navigator web site. This example illustrates the use of performance reporting
to motivate institutional accountability, in this case in regard to cost control.

5

In particular, HEOA calls for wider measuring and reporting of institutional
graduation rates:
The Secretary of Education, in coordination with the Commissioner for
Education Statistics and after consultation with representatives from diverse
institutions of higher education… and other stakeholders … , shall issue a report
with recommendations to Congress about alternative ways to measure and report
degree or program completion rates for institutions of higher education receiving
funds under title IV of the Higher Education Act of 1965 (20 U.S.C. 1070 et
seq.). … The alternative measures described … shall consider the number of
degrees awarded and the increase in number of degrees awarded disaggregated
by race, ethnicity, gender, and income for all students who have earned a degree;
and the increase in degrees awarded in high-need fields such as science,
technology, engineering, mathematics ... (p. 424).
As another more recent example, in December 2010 The National Center for
Higher Education Management Systems, and The National Center for Public Policy
and Higher Education released a policy report from The Delta Project on
Postsecondary Education Costs, Productivity, and Accountability. The report calls for
the federal government and states to promote productivity increases by establishing
accountability indicators and creating incentives for degree completions. Universities
and colleges, in turn, are asked to establish clear goals for increases in degree and
certificate production as well as to develop clear metrics for measuring success and to
widely communicate the results each year. In addition, these accountability reforms
have an uneasy relationship with accreditation. The U.S. higher education has a well-

6

developed system of voluntary accreditation to certify the institutions that meet
professionally established standards for providing collegiate instruction and awarding
degrees. This system has served as “… the principal external means to ensure
institutional quality … for more than a hundred years” (Head, 2011, p. 26). It obtained
substantial public policy significance when the federal government began to rely on
accreditation for determining which institutions qualify for federal student assistance
programs (National Commission on Accountability in Higher Education, 2005). In the
first phase of its development, the accreditation process generally sought to assure that
institutions had the resources (facilities, libraries, finances, and faculty) to provide a
quality higher education. Later on, in addition to such measures, accreditation
agencies have increasingly focused attention on institutional processes to achieve and
assess higher levels of student learning. Statements from CHEA (The Council for
Higher Education Accreditation) and The Higher Learning Commission identify
important issues related to the role of accreditation in higher education accountability
and its relationship to accountability practices of state and federal governments
(National Commission on Accountability in Higher Education, 2005). For example, a
recently launched CHEA Initiative has two primary goals: further enhancing
accountability and accreditation, and “ … focusing federal oversight of accreditation
more directly on issues related to … the use of federal funds, rather then the academic
policy issues …” (2009-2010 CHEA Annual Report, p. 3).
In nearly every conversation about accountability in higher education graduation
rates are increasingly viewed as a critical measure of both student success and
institutional effectiveness. Moreover, since President Obama announced, “ … by

7

2020, America will once again have the highest proportion of college graduates in the
world,” postsecondary graduation rates have received even more scrupulous attention
(Cook & Pullaro, 2010, p. vi). However, there is still little consensus on how the
postsecondary education system should be reformed to guarantee that colleges and
universities are held accountable for retaining and graduating their diverse students
(Dowd, 2005). This lack of progress reflects both “ … the magnitude of the challenge
and the failure of previous efforts to examine how the various parts of the system –
measurement of student outcomes, data quality, faculty productivity, accreditation and
quality control – may facilitate or impede meaningful accountability” (American
Enterprise Institute for Public Policy Research, 2009, retrieved from
http://www.aei.org/event/100134).
Prominent debates in regard to accountability as a driver of institutional
effectiveness highlight several major challenges. One of them stems from the fact that
no agreement exists how best to measure institutional effectiveness at the
postsecondary level in general, and at the level of community colleges in particular.
The decentralization of American higher education and the diversity of learning
objectives across institutions help explain why now there is no nationally comparable
basis for assessing student learning among the states (National Commission on
Accountability in Higher Education, 2005). Requiring uniform assessments of student
learning also has been controversial in higher education because disparities in student
preparation among institutions may have a greater effect on learning outcomes than
disparities in instructional quality (National Commission on Accountability in Higher
Education, 2005). In order to address this issue, ED has attempted to establish a

8

national accountability system that features better measurement of the way institutions
of higher education impact student success by using a value added approach. “This
accountability voice represents an opportunity for higher education institutions to
assist in the development of an accountability system that will take into account the
variability among higher educations institutions” (Gonzales, 2009, p. 16).
Debate regarding measuring effectiveness at the level of community colleges
(CC) emanates from the fact that they are open access institutions with multiple
missions (Grubb & Badway, 2005). Beyond traditional academic and occupational
education, their missions include developmental education, ESL for immigrant
students, non-credit education, advancement training, community service, and
guidance to “experimenters” trying to find a path (Grubb & Badway, 2005, p. 2).
Grubb and Badway claim that on the one hand, these unique features of CC missions
are crucial to the public service, but on the other, they complicate the process of public
accountability and assessment. Dowd (2007) argues that the calculation of
performance indicators at community colleges can be quite challenging. She provides
an example of the difficulty in calculating transfer rates in a manner satisfactory to all
stakeholders. According to the author, the debate concerns the inclusion of students,
who potentially had no intention of transferring, in transfer indicators. Here the
question of whether colleges are responsible for increasing students’ aspirations to
transfer arises. Due to the difficulty of collecting faithful measures of students’
academic aspirations, “Community college administrators and policy makers alike
have struggled in deciding how to include students who do not have clear educational
goals in measures of institutional effectiveness” (Dowd, 2007, p. 4).

9

Furthermore, Dowd and Tong (2007) argue that accountability standards tend to
emphasize the importance of quantitative indicators of student learning outcomes and
de-emphasize the necessity to understand educational processes within institutional
contexts. The scholars believe that although important to the study of institutional
effectiveness in general, the focus on student learning outcomes in itself is not
sufficient to bring about productive change. They strongly advocate for “ … equal
attention under accountability to educational processes and contexts alongside the
measurement of student outcomes” (p. 58).
In addition, the “austere funding environment” has led to a national productivity
agenda, one component of which involves performance-based funding (PBF) as a
means of improving institutional effectiveness (Harnisch, 2010). Tomas L. Harnisch, a
policy analyst from American Association of State Colleges and Universities, asserts
that despite certain advantages, such as greater awareness of campus performance,
enhanced transparency, and even some productivity gains, PBF has several important
impediments and controversial points. One of them deals with increased inequality
and instability. Harnisch highlights concern that PBF may hurt institutions that need
the most help, especially those serving disadvantaged students from racial-ethnic
groups because “the lack of resources, not university efforts may be the driver behind
poor performance” (p. 8). Another important PBF disadvantage refers to quality
reduction. Harnisch argues that because PBF may stress efficiency over quality, some
institutions may act to reduce academic rigor to achieve better outcomes. Thus,
academic quality of education might suffer. The analyst concludes that the concept of
linking institutional performance with state funding has been met with skepticism

10

from all stakeholders in higher education. Dougherty and Hong (2005) also express
concern that performance accountability systems have not taken into account many
obstacles to effective institutional performance, such as depressed local labor markets
and a lack of organizational recourses. As a result, they argue, some colleges and
universities “may not have a fair shot at successfully meeting state standards” (p. 7).
Accountability in California Context
California is an important place to explore the problem of inequitable educational
outcomes because the state has been identified at the forefront of trends “placing the
economic health and social stability of the nation at risk” (Moore & Shulock, 2007, p.
1). These trends deal with unequal distribution of education and skills, changes in the
economy, and demographic shifts. Indeed, substantial disparities in current
educational attainment and college enrollment exist across different racial-ethnic
groups in California. Although Latinos make up 43 percent of California’s college-age
population (Bensimon et al., 2007), only 15 percent of these have or are pursuing a
college degree (Moore & Shulock, 2007). African American adults have higher rates
of educational attainment – 32 percent; however, they also significantly lag behind
whites and Asians who show the biggest rates (47% and 54% respectively) (Moore &
Shulock, 2007).
In addition, California’s economy has been experiencing major changes in recent
years. Increasing globalization, the decline in defense and other manufacturing
industries, and the rise of industries related to information technology – all contribute
to shifts in the economy (Moore & Shulock, 2007). These changes require workers to
possess a greater ability to adapt their skills to changing labor markets. The

11

demographic composition of California’s population has also been encountering
dramatic changes. Latinos represented 29 percent of the population ages 25 to 64 in
2000, are expected to reach 40 percent by 2020 and 49 percent by 2040 (Moore &
Shulock, 2007). Policymakers and scholars argue that without solving the problem of
inequities in educational outcomes, the average education level of California’s
workforce will decline, thus leading to “ … a deteriorating tax base and increasing
difficulty for the state to provide services to its people …” (Moore & Shulock, 2007,
p. 2).
The California community college (CCC) system and California State University
(CSU) system are vital to reversing these troubling trends. Currently 112 campuses of
the CCC and 23 campuses of CSU enroll the lion’s share of California student
population and are closely tied to each other by the state transfer policies (The 1960
Master Plan for Higher Education; CCCCO, 2011; SB 1440). According to Moore and
Shulock (2007), 73 percent of all California public undergraduate students are enrolled
in community colleges; 18 percent attend CSU system, while only 9 percent of
students are enrolled in the University of California (UC) system.
California Community Colleges
The CCC is known to be the largest system of higher education in the United
Sates, serving more than 2.76 million students (CCCCO, 2009). Its colleges offer
certificates and occupational degrees, provide basic skills education, and prepare
students for transfer to four-year universities and colleges (CCCCO, 2011). Assisting
the underprepared student in attaining the basic skills needed to succeed in college has
been a core function of the California Community Colleges throughout history.

12

According to the Chancellor’s Office Basic Skills Accountability Report 2009,
Basic skills are defined as those foundation skills in reading, writing, mathematics,
and English as a Second Language, as well as learning skills and study skills, which
are necessary for students to succeed at the college level. Basic skills courses are
designed to develop reading or writing skills at or below the level required for
enrollment in English courses one level below freshman composition, computational
skills required in mathematics courses below Algebra, and ESL courses at levels
consistent with those defined for English (p. 1).
According to 2008 data, of those students enrolled in CCC in the 2006-2007,
who were assessed, 70 to 85 percent were placed into a basic skills course in one or
more of the basic skills areas (Center for Student Success, 2008). Compared to white
and Asian students, minority students are overrepresented in basic skills courses. For
example, while African American students represent 7% of students in the California
Community College system, they represent 11 percent of enrolled students in credit
basic skills courses.
Another essential function of CCC is transfer to four-year colleges and
universities:
UC and CSU are to establish a lower division to upper division ratio of 40:60 in
order to provide transfer opportunities into the upper division for community
college students. The goal was that UC and CSU would enroll at least one
community college transfer student for each two freshmen enrolled. All eligible
California Community College transfer students are to be provided a place in the

13

upper division and are to be given priority over freshmen in the admissions
process (University of California Office of the President, 2007, p. 2).
In 2008-2009, the California Community Colleges transferred 99,583 students to
four-year institutions: public, private, in-state, and out-of-state (ARCC, 2010).
Moreover, the California State University system continues to be “the most frequent
transfer destination for community college students with the enrollment of 49,770
students from the community colleges” (p. XI).
However, the rates of completing certificates, degrees and transfers to
universities in the CCCs remain extremely low (Moore & Shulock, 2007). In a study
of CCC, Moore and Shulock (2007) found that only one in four degree seekers in the
1999-2000 incoming cohort of CCC students earned a certificate or degree, transferred
to a four-year university or achieved some combination of those outcomes within six
years of enrolling in a community college. Seventy six percent of degree seekers did
not achieve any of these outcomes within six years of enrolling in community college.
Moreover, completion rates for this cohort of students varied according to student
characteristics, including race-ethnicity, with Asian and white students completing at
higher rates (33% and 27% respectively) than Latino and African American students
(18% and 15% respectively). These statistics confirm other research indicating that
rates of persistence and completion in CCC are low, “… likely too low to meet the
needs of the workforce and to ensure continued economic growth and prosperity for
individuals and state” (Moore & Shulock, 2007, p. V).
Similarly, the CSU has a low percentage of graduates leaving the university in
six years. According to the Chancellor’s Office Graduation Initiative Plan (2009), the

14

current graduation rate for the system is 46 percent. Moreover, most CSU campuses
have significant achievement gaps to close. African American, Hispanic, and
American Indian students have lower rates of degree completion than their white and
Asian peers, graduating at 41 percent rate (total for three groups) as compared to 48
percent for whites and Asians combined. Therefore, there is the need to focus on
student success particularly in CCCs, which enrolls the largest number of students
most at risk of not finishing college, as well as in CSUs, which later enrolls the largest
numbers of transfer students from CCC.
Initially, the California Community College system made attempts to address
the existence of differential outcomes through the implementation of four
accountability systems (Grubb & Badway, 2005). The state accountability
mechanisms have included the Partnership for Excellence (PFE) and the State Report
Card (SRC). The federal mechanisms have included performance measures required
by the Vocational and Technical Education Act and the Workforce Investment Act
(WIA) (Grubb & Badway, 2005). Subsequent efforts were originated by the Western
Association of Schools and Colleges (WASC), which developed “a series of new
standards requiring colleges to develop mechanisms of assessment for student learning
outcomes (with the unfortunate acronym of SLOs)” (Grubb & Badway, 2005, p. 5).
The newest accountability systems are known as Accountability Reporting for
Community Colleges system (ARCC) and The Basic Skills Initiative (BSI).
In 2004, Assembly Bill 1417 triggered the creation of a performance measurement
system for the CCCs (CCCCO, 2010). That legislation and ensuing budget action
authorized the CCCCO to design and implement a performance measurement system

15

that contained performance indicators for the system and its colleges. This
comprehensive system has become known as “ARCC” (CCCCO, 2010, p. XI). AB
1417 requires the CCCCO to submit an annual accountability report that serves as the
annual evaluation of college-level performance in meeting statewide educational
priorities.
The ARCC framework specifies that community college performance data should
be aggregated, analyzed, and reported at two levels: the individual college level and
across the community college system. According to CCCCO (2010), there are ten
college level performance indicators: student progress and achievement rate; percent
of students who earned at least 30 units; persistence rate; annual successful course
completion rate for credit vocational courses; annual successful course completion rate
for credit basic skills courses; improvement rates for credit ESL courses; improvement
rates for credit basic skills courses; career development and college preparation
progress and achievement rate; and college profile summaries, (e.g., headcounts,
percentages of student enrollments by various demographics; summary of the
college’s peer groups for each indicator).
On the system-wide level, indicators include: the annual number and percentage
of baccalaureate students graduating from UC and CSU who originally attended a
CCC; the annual number of community college transfers to baccalaureate granting
institutions; the transfer rate to baccalaureate granting institutions from the CCCs; the
annual number of degrees/certificates conferred by vocational programs; the increase
in total personal income following completion of a vocational degree/certificate; the
annual number of basic skills improvements; system-wide participation rates (by

16

selected demographics). The most current statistics are reported in the ARCC 2010
Report. According to the report, for example, despite a slightly increasing six-year
trend (from 2003-2004 to 2008-2009) of the annual number of CSU and UC
baccalaureate students who attended CCC, the annual number of CCC transfers to
four-year institutions across those years declined in 2008-2009 (to a total of 99,583
students). Also, the annual number of CCC transfers to CSU decreased in 2005-2006
(52,641), increased the subsequent two years 2006-2007 and 2007-2008 (54,391 and
54,971) before decreasing again in 2008-2009 (49,770).
One distinctive feature of the ARCC is that its indicators have unique definitions
(ARCC Report, 2010). Therefore, they cannot be compared to those generated for
other states or by other studies of the California Community Colleges, thus
complicating the process of productivity assessment. Yet another one is that although
the goal of the ARCC is to support accountability measures that lead to a college’s
self-assessment of practices as a way to promote a fair educational opportunity for
students (ARCC Report, 2007), the indicators on both levels do not include data
disaggregated by race and ethnicity. It implies that the ARCC does not promise to be
efficient at addressing unequal outcomes.
Another recent effort to hold community colleges accountable, the BSI is a
grant-funded initiative from the CCCCO, which began in 2006 as part of the strategic
planning process. The goal of the BSI is to improve student access as well as success.
The project addresses credit and noncredit basic skills as well as adult education and
programs designed to help underprepared students. One prong of the BSI allocates
colleges supplemental funding to specifically address basic skills needs. This funding

17

is guided by locally developed action plans documenting usage of the funding. The
outcomes of the BSI are tracked using the ARCC. They are available in Basic Skills
Accountability Supplement to the ARCC Report 2011. The objective of this report is
“to make policymakers aware of the system-wide efforts and outcomes in basic skills”
(BSAS, 2011, p. 1). As such, performance is measured by deploying four categories:
descriptive metrics or “demographic snapshots,” workload metrics, assessment and
placement metrics, and student progress metrics. The other prong takes the shape of a
Professional Development Grant, which is designed to address training needs for
faculty and staff in basic skills, and English as a Second Language (ESL). The
implementation of the BSI occurred in three phases: communication effective
practices; provision professional development to aid in assessment and evaluation of
basic skills and ESL efforts; and implementation plans to improve basic skill student
success (Gonzales, 2009). Community colleges were required to assess the success of
basic skills students and create plans to improve their success.
California State University
California State University (CSU) is the nation’s largest university system, with
450,000 students and about 90,000 graduates each year (Echeverria, 2010). Although
California does not have a formal accountability system for its public universities
(USC California Policy Institute, 2005), CSU is committed to accounting for its
performance by providing periodic reports to the public through the CSU
accountability process. The CSU Cornerstones Report (1998) first called for such a
system. In 1999 the CSU accountability process was instituted and has since evolved
through input provided by the individual campuses, the Alumni Council, the

18

California State Student Association, and the Academic Senate CSU. The system wide
process analyzes the CSU’s educational effectiveness and honors the diverse ways in
which its campuses help meet the state’s educational objectives. Both system wide and
campus outcomes are reported on a regular, cyclical basis, with system wide
performance reported annually and campus performance reported every other year.
The California State University system has also committed to providing data on
student learning, student engagement, enrollment, and graduation as part of a national
initiative called the Voluntary System of Accountability (VSA). The VSA was
developed in 2007 by a group of university leaders and is sponsored by two higher
education associations – the Association of Public and Land-grant Universities
(APLU) and the Association of State Colleges and Universities (AASCU). Each of the
23 CSU campuses develop a web-based page called the College Portrait that is
designed to specifically communicate accountability data to the public. A more recent
effort to increase institutional productivity is the CSU Graduation Initiative: Closing
the Achievement Gap, which “strives to raise the freshman six-year graduation rate by
eight percentage points by 2015, and cut in half the existing gap in degree attainment
by CSU’s underrepresented minority (URM) students” (Chancellor’s Office
Graduation Initiative Plan, 2009, p. 1). Launched in Fall 2009 by the CSU Office of
the Chancellor, the Initiative is part of the nationwide Access to Success project of the
National Association of System Heads (NASH) and The Education Trust (Echeverria,
2010). As part of the initiative, each campus has formed a leadership team to examine
policies and programs and develop plans to identify campus targets and goals, manage
progress and make strategic changes when necessary. Campuses are responsible for

19

presenting bi-monthly reports and hosting campus visits to exhibit progress toward
their goals. Although as a system, the CSU “is on track to meet its overall graduation
rate target, … it is unlikely to cut achievement gap in half by 2015. As a result, the
CSU is energetically looking for strategies that will disproportionately benefit students
from three underrepresented ethnic groups: African American, Latino, and Native
American” (AAC&U Summer Institute CSU Application, 2011, p. 1).
While these initiatives are well intended and bring attention to the problem of
student outcomes as well as disparities in outcomes, they may be limited mainly
because despite the opportunities practitioners may have had to be trained in the
collecting and reporting of data, they have not been trained to inquire what those
numbers mean and how they can improve institutional practices (Gonzales, 2009).
Bensimon (2005) argues that in order to bring about change in an institution,
practitioners must see, on their own, and “ … as clearly as possible, the magnitude of
inequities.” They must then analyze and integrate the meaning of these inequities, so
that “they are moved to act upon them” (p. 24).
Statement of the Problem
In a new economic, technological, and cultural milieu of the 21
st
century, diverse
racial-ethnic groups are facing enduring patterns of educational inequality, evidence of
which has been brought to light in numerous research studies and official reports
(Luke, Green, & Kelly, 2010). In the 2009 update to the biennial report Minorities in
Higher Education 2008, the American Council on Education (ACE) reveals that
despite a steady rise in college enrollment rates among young people in every racial-
ethnic group, racial-ethnic disparities in college participation worsened in the past two

20

decades because of disproportionate rates of improvement. Among whites, African
Americans, and Hispanics, whites had the highest rate as well as the largest gain,
rising from 31 percent in 1988 to 45 percent in 2007. In contrast, both African
Americans and Hispanics made smaller gains, from 22 percent to 33 percent, and from
17 percent to 27 percent, respectively. When Asian Americans and American Indians
are included, gaps become even larger. Asian Americans marked the highest rate of
college enrollment (63 percent) in 2007, while American Indians registered the lowest
rate (25 percent).
The current rates of degree completion for non-Asian minorities also lag behind
white students across postsecondary education. According to U.S. Department of
Education National Center for Educational Statistics, in nearly all the comparison
groups, white students graduate at higher rates than African American, Hispanic, and
Native American students. Thus, 59 percent of white students graduated from 4-year
institutions, while only 46 percent of Hispanics, 37 percent of African Americans and
the same percentage of American Indians completed their degrees (graduation rates for
2003 cohort). Another evidence of inequity is provided by the national report card,
Measuring Up, issued by National Center for Public Policy in Higher Education
(2008). It reveals that 59 percent of white students complete a bachelor’s degree
within six years of enrolling in college. In contrast, only 47 percent of Hispanic
students, 40 percent of African Americans, and 39 percent of Native American
students complete the same degree within six years.
The underrepresentation of three key minority groups – African Americans,
Hispanics, and American Indians – in high-need fields such as science, technology,

21

engineering, and mathematics (STEM) is particularly onerous. For example, report
from the National Action Council for Minorities in Engineering (NACME, 2008)
illustrates that the number of minority students pursuing STEM degrees and careers
has flattened out or even declined in recent years. The study Confronting the “New”
American Dilemma, Underrepresented Minorities in Engineering: A Data-Based Look
at Diversity, supported by the National Science Foundation, shows that the percentage
of bachelor's degrees in engineering awarded to African American, Hispanic, and
American Indian students declined significantly from 1995 to 2005. Thus, engineering
degrees accounted for 3.3% of bachelor’s degrees awarded to African Americans in
1995 but only 2.5% in 2005; 5.5% of Hispanics received engineering degrees in 1995
but only 4.2% received them in 2005. A similar pattern exists for American Indian’s
share of engineering bachelor’s degrees, “with a peak in 1998 followed by a long
period during which American Indians earned just under 1% of the bachelor’s degrees
awarded in engineering technology. Between 2004 and 2005, there is a noticeable
decline in their representation among recipients of these degrees” (Frehill, DiFabio, &
Hill, 2008, p. 96).
Statistics from community colleges are even more troublesome. Across the United
States, nearly 1,200 community colleges play a vital role in higher education enrolling
11.5 million students (CCCO, 2009). Despite their crucial role, community colleges
are facing enormous challenges in terms of student success. Nearly half of all students
enrolled in community colleges fail to complete their postsecondary education. Data
from a recent study by the Community College Research center shows that only 53
percent of students earned an associate’s degree or transferred to a four-year

22

institution within eight years of initial enrollment. According to the report Referral,
Enrollment and Completion in Developmental Education Sequences in Community
Colleges, released by the Community College Research Center of Teachers College,
Columbia University in 2010, “ … only between 3 and 4 of every 10 students referred
to a remedial education sequence to prepare for college-level work actually complete
the sequence. Most students who don’t complete the sequence abandon it early on –
with almost half failing to complete the first course in a sequence” (p. 1).
In response to these challenges, scholars argue that current accountability
structures are not effective to address the problem of inequitable educational outcomes
because they are not designed to promote organizational learning, they are sometimes
contradictory, and that there are not enough resources to conduct necessary reforms
(Baldwin et al., 2011; Dowd, 2005). Moreover, equity in educational outcomes is not a
measure of postsecondary institutional effectiveness tracked continually at the
national, state, or local levels (Bauman, 2005). Dowd, Bishop, Bensimon, and Witham
(2012) argue that although state and federal accountability systems attempt to focus
practitioner attention on achieving greater student educational outcomes through a
variety of reporting requirements, “It is still questionable whether the information
contained in mandated reports is meaningful to practitioners” and whether public
reporting requirements are promising to produce positive change in performance” (p.
183). The authors suggest that accountability indicators are not meaningful to
practitioners because they are measured at a highly aggregated level, far removed from
the classrooms and programs in which practitioners do their work. Gutierrez and
Vossoughi (2010) assert, “The cultural norms operating in educational practice often

23

obscure opportunities for improvement” (p. 103). Based on their own empirical work,
the researchers conclude that although practitioners are the most influential factor in
student learning they are rarely provided the opportunity to influence and reflect on
policies and their own practices, to examine their role in the change process, or to
understand the social and cognitive consequences of policies and practices. They
claim practitioners have even less opportunity to make sense of how the latter
“…operate in relation to one another or function collectively toward productive
change” (p. 103).
Scholars argue that organizational learning and self-assessment in the form of
inquiry are necessary to bring about improvements in institutional effectiveness and
inequitable educational outcomes, particularly for African American, Hispanic and
other minority students because of histories of institutionalized racism,
marginalization, and exclusion. In a culture of inquiry practitioners use their
experience to learn more about the institution through the collection of data (Dowd &
Tong, 2007). To create and study change in professional practice, Gutierrez and
Vossoughi (2010), for example, propose a praxis model of teacher education and
advocate the idea of “social design experiments,” which they define as a form of
inquiry, designed to promote learning among participants by providing regular
opportunities for reflection. In such experiments, the purpose is to engage practitioners
in joint activity to redesign the learning ecology so that ongoing opportunities for
robust learning practices become the norm. The social design experiment is a hybrid
methodology in which research is a co-construction of researchers and stakeholder
participants. One central design principle “ … privileges understanding and addressing

24

the contradictions that constrain opportunities or that give rise to inequitable learning
environments” (p. 102). The scholars highlight that this attention to contradictions
challenges the tendency toward simpler, less nuanced explanations of professional
practices and problems.
Another example of advancing “a culture of inquiry” is represented by action
research conducted by the Center for Urban Education at the University of Southern
California (CUE). CUE fosters research that “is helping higher education institutions
across the country become more accountable to students from underserved racial and
ethnic communities” (Dowd & Bensimon, 2009, p. 1). The question “What can
colleges and universities do to reduce inequities?” has led CUE researchers to explore
institutional effectiveness as the locus of change. Seeking to reframe the traditional
goal of the public sector from providing equal access and opportunity to achieving
equal educational outcomes among racial-ethnic groups, Bensimon (2012) and Dowd
(2007) emphasize institutional responsibility and cultures of inquiry. Dowd states that
colleges can become more equitable by developing the capacity for organizational
learning. Further, she argues institutions become more efficient when practitioners
have both actionable information and an orientation towards equity.
One of the CUE’s key principles is that organizational learning and change occur
when practitioners participate as researchers into their own culture and practices
(Bensimon, Polkinghorne, Bauman, & Vallejo, 2004), that is when they come together
to generate knowledge, to mutually define problems, priorities, and action. This
participation is facilitated through action inquiry, “a systematic process of problem
identification, data collection, reflection, decision-making and action,” which help

25

develop the professional expertise necessary to recognize and counter discriminatory
practices and to address inequities in student outcomes (Dowd & Bensimon, 2009). As
practitioners engage in this cycle of inquiry, existing knowledge and beliefs are
examined and reconstructed through social interaction and reflection. With problem
identification through data analysis, these social interactions generate new capacities
for experimentation and problem solving, which lead to a broader range of
professional experiences, the source of adaptive expertise (Dowd & Bensimon, 2009).
To initiate action inquiry CUE helps colleges and universities create campus
evidence teams through its unique form of participatory action research. The teams
consist of individuals including faculty, administrators, and staff. CUE trains those
individuals to take a practitioner-researcher role and actively work together to examine
their institution’s data and practices, pinpoint where racial inequities occur, and design
equity interventions tailored to fit the unique needs and culture of their institution.
Two central parameters of action research: “knowledge generation through action and
experimentation in context” and “participative democracy” (Greenwood & Levin,
2005, p. 53) are likely to ensure sustained change for at least two reasons. First, the
evidence teams’ locally produced knowledge leads to the generation of solutions that
are appropriate for the campus context. Second, locally generated knowledge is more
meaningful to those charged with addressing the problem.
The basis for CUE’s inquiry approach focuses on organizational learning
through the use of data to inform decision-making. The problem with only using data
to create change is that data alone cannot lead towards change at the organization.
Moreover, the process of organizational learning is often challenging since individuals

26

at the institution do not have the time or existing structures to use the data that already
exist at the institution. Therefore, even though the recent higher education
accountability movement, with its focus on data and outcomes, is aligned with the
goals of outcome equity (Dowd, 2007), Dowd and Bensimon (2009) question whether
it has the potential to bring about positive change in institutional performance. “The
theory of action of most accountability systems is that data drive change: that more
and better data will result in improved decisions” (Dowd & Bensimon, 2009, p. 9).
But, they argue, data alone will not produce change.
While the Center shares the interest of policy makers in obtaining the data
necessary to promote equity, it places a unique emphasis on the actual use of data
(Dowd, 2009). Bensimon (2005) argues that even though data are necessary to engage
in equity sense making, it is critically important for practitioners to understand that the
ultimate purpose of examining data disaggregated by race and ethnicity is to create a
context to talk about race and ethnicity. Organizational learning, she strongly believes,
is “particularly effective in making the invisible visible and the undiscussible
discussible, two conditions that describe the status of racial-ethnic unequal outcomes”
(Bensimon, 2005, p. 99).
Purpose of the Study
The purpose of the present study was to investigate whether action inquiry using
CUE’s action research processes and tools brings about or shows the potential to bring
about organizational learning and change, improvements in institutional effectiveness,
and as a result – greater racial-ethnic equity in student outcomes. The stated purpose
behind the action inquiry CUE facilitated was to promote learning among faculty,

27

administrators, and professional staff around the existence of differential learning
outcomes. CUE researchers expected that learning would be facilitated through
participation in evidence teams and the use of “remediated artifacts” of professional
practice: processes and assessment tools that CUE creates for action inquiry. Mediated
action is defined as a process, by which individuals and artifacts interact through goal-
oriented activity (Nasir & Hand, 2006). Remediation, or in other words, goal-oriented
change in artifacts is expected to lead to remediation of professional practice, “… in
order to remediate practice it is necessary to remediate the artifacts that are the tools
for the social construction of practice” (Dowd et al., 2012, p. 181).
CUE creates “remediating artifacts” in the form of different equity-oriented self-
assessment tools and cultural artifacts (language, media) designed to assist the
learning, equity-mindedness, and professional development of college administrators
and faculty members in ways that will enhance college capacity to achieve equitable
student learning outcomes (Dowd & Bensimon, 2009). This study examined the
impact of the use of CUE’s action research processes and tools in a social design
experiment undertaken at one field site as part of a larger developmental self-
evaluation study conducted by CUE.
Research Questions
Given the purpose of the present study, I explored the following research
questions:
1. What beliefs do practitioners hold about racial-ethnic equity for the purposes of
equity-oriented organizational change?

28

2. What social interactions (roles, rules/norms, communities, division of labor, power
relationships) mediate attitudes, beliefs, and behaviors related to racial-ethnic
inequities?
3. What influence do equity-oriented cultural artifacts designed for use in
“remediated” social learning environments have on postsecondary educators’
beliefs, attitudes, and behaviors in regard to equity in postsecondary education?
My focus was on a team consisting of faculty members at a 4-year public
comprehensive university. I investigated whether and how the practice of action
inquiry using a small number of CUE’s tools is transformational for the individuals in
the team. In addition I examined how these people strive towards increasing equity in
student educational outcomes through equity-minded practice. Furthermore, I
identified what equity-oriented cultural artifacts are associated with changes in a
practitioner’s beliefs and practices concerning equity. I also explored what social
interactions, norms, and rules mediated practitioners’ beliefs and behaviors towards
equity.
A number of focal areas for analysis emerged from the theories presented in the
next chapter. These are: (1) when practitioners are engaged in activity settings using
CUE’s tools, their attitudes and beliefs will be challenged and influenced towards
equity-mindedness; (2) participation in social interactions facilitated by action
research and equity-oriented tools will promote learning among participants; (3)
participation in action inquiry and reflection will influence a willingness in behavioral
changes.

29

Importance of the Study
The study is important for several reasons. First, its results contributes to the
growing body of knowledge regarding whether, how, and under what conditions
action research is an effective form of assessment, which allows practitioners to better
understand the nuances of their practices. Next, according to Patton (2011), the
purpose of developmental evaluation is to support adaptation of innovation in complex
and dynamic conditions, and to support making sense of the implications of adopting
particular innovations. This study enhances the understanding of how CUE’s tools
enable practitioners to assess their practices, to think about changing their practices to
better serve historically underrepresented students, and to advocate for changes
towards equity-mindedness in policies and procedures at their institution. Furthermore,
the results of the study inform CUE in the Center’s evaluation questionnaire
development. Finally, having conversations about race-ethnicity and equity through an
inquiry process may put practitioners in a position to actually address the issues of
inequity and make changes.

30

CHAPTER 2: CONCEPTUAL FRAMEWORK
To reiterate, the purpose of this dissertation was to investigate whether inquiry
using CUE’s action research processes and tools brought about or showed the
potential to bring about change in practitioners’ beliefs, attitudes, and behaviors.
Essentially, the study sought to understand whether and how social interactions (roles,
norms, communities, division of labor, power relations, racial relations, ethnic
relations) facilitated by action research and by equity-oriented cultural artifacts
(language, media, self-assessment tools) mediate practitioners’ attitudes, beliefs, and
behaviors related to racial-ethnic inequities. The challenge of reducing educational
inequities has led researchers at the Center for Urban Education, whose action
research projects provided the field settings for this study, to apply the concepts of
culture of inquiry, organizational learning, and institutional responsibility in their
work. A culture of inquiry places practitioners at the center of change efforts to
improve student outcome equity (Dowd et al., 2007). In conducting action research as
a form of intervention to bring about equity, CUE researchers are among others who
engage higher education practitioners in self-assessment activities in the form of
action inquiry aimed at remediation of professional practices.
Researchers at the Office of Community College Research and Leadership at the
University of Illinois (OCCRL) provide another example of the use of action research
and evaluation methods to address equity issues in community colleges in Illinois and
nationwide. In a recently launched Pathways to Results in Illinois (PTR) equity–
oriented project, they engage practitioners in a 5-phase inquiry process to identify
sustainable solutions to improve student outcomes. The five phases are represented by

31

engagement, outcomes and equity assessment, process assessment, process
improvement, and review and reflection. This process is aimed at examination of
institutional data and uncovering inequities residing in institutional practices that “ …
go unnoticed in day-to-day practice,” two conditions Debra D. Bragg, director of the
Office, believes are critical to identifying equity gaps and creating solutions (retrieved
from http://occrl.illinois.edu).
The necessity of focusing on practitioners and institutional practices in regard to
equity in student experiences and outcomes is supported by many theoretical and
empirical studies. Scholars believe that practitioners’ commonly held knowledge,
beliefs, and assumptions regarding students from racial-ethnic groups, embedded in
institutional structures, contribute to persisting inequities, assuming that
institutionalized racism in its subtle forms is still present (Bensimon, 2012; Gutierrez
& Vossoughi, 2010; Margolis, et al., 2008). Over the past ten years, CUE’s
researchers have studied practitioners’ conversations about race and equity to
understand which cognitive frames are commonly relied upon to understand inequities
in student educational outcomes. Bensimon (2012) concludes: “ … the prevailing
funds of knowledge that postsecondary educators draw on are inadequate to undo
racial patterns of inequity …” (p. 18). This implies that institutions of higher education
need to take responsibility for elimination of racial-ethnic inequities. That is,
practitioners need to develop new funds of knowledge, which would allow for
considering a problem of inequity as a problem of practice (Bensimon, 2012). These
new funds of knowledge would serve as a driving force providing motivation to
change one’s own practices and institutional norms (Bensimon, 2012).

32

Scholars agree that practitioners can develop their expertise to make equity
possible through participation in designed learning opportunities. Dowd and Tong
(2007) state that institutional change can be brought about through an inquiry into
organizational cultures enabling practitioners to collectively assess the problems on
their campuses. Action research is one approach that has the potential to realize this.
Greenwood and Levin (2005) strongly believe that action research facilitates
development of knowledge, enabling practitioners, through working together, to learn
how to act to achieve their goals. However, action research does not necessarily
address the problems of racism, inequities, or cultural change. Cultural historical
activity theory (CHAT) brings this focus.
Acknowledging human mental functioning as a mediated process organized by
cultural artifacts, social activities, and concepts (Ratner, 2002), CHAT considers
learning as situated across social settings inherently constrained by historical legacies
and hidden assumptions (Dowd et al., 2012). Of specific interest in this study were
histories of institutionalized racism, marginalization, and exclusion. CHAT focuses on
an activity system as a unit of analysis, in which different components (described later
in this chapter) influence each other and human learning. Since learning is recognized
as a culturally and socially mediated process, scholars argue that change in the
individual involves reorganization of the activity system as well as in the way
mediating artifacts regulate coordination with the environment (Gutierrez &
Vossoughi, 2010). From this point of view, in order to initiate change in the
individual, it is necessary to remediate artifacts and social interactions. Finally,
recognizing that the barriers to college success for racialized minority students can be

33

structural or interpersonal, it is useful to consider the meaning of the term race and
racism in the United States today. The chapter begins by introducing a concept of
funds of knowledge to address inequities developed by Bensimon (2012), followed by
a discussion of three overarching theoretical strands that inform this study: action
research, cultural historical activity theory, and critical perspectives on racialized
practices.
Funds of Knowledge to Address Inequities
Contrary to the theories, studies, and interventions of racial-ethnic gaps in student
outcomes, focusing mainly on the ability of students to cope with the opportunities
and barriers of higher education (Bensimon, 2012), Estela Mara Bensimon explores
the knowledge and behaviors of practitioners and institutions rather than the
knowledge and behaviors of students. She focuses on the knowledge and cultural gaps
that undermine practitioners’ capacity to be responsive to students from racial-ethnic
groups. Bensimon introduces the concept of funds of knowledge or cognitive frames
that are “ … bodies of knowledge that practitioners unconsciously draw on as they
decide which questions to ask, what information to collect, what to notice, how to
define problems, and what course of action to take” (p. 18). These bodies of
knowledge are “a mental map” of attitudes and beliefs that an individual maintains to
make sense of the world. They are developed through everyday practices, and
transmitted through social conversations and institution’s cultures. They are also
developed through dominant models of student success and student development.
According to Bensimon, the prevailing cognitive frames or funds of knowledge
that many practitioners draw on reflect widely accepted beliefs about student success,

34

mostly derived from psychological theories of motivation, self-efficacy, and self-
regulation as well as sociological theories of cultural integration. These funds of
knowledge, she writes, lead educators to expect self-directed students, and to label
those who fail to meet these expectations as “at-risk,” thus “ … reinforcing a logic of
student success that is detrimental to an equity change agenda” (p. 18). They also
produce stereotypical judgments and diminish responsibility and effectiveness on the
part of the institution and its practitioners to create change. She concludes that the
dominant cognitive frame widely operating in education leads to a deficit‐minded
approach, a perspective that places the responsibility for unrealized success solely on
students.
Stuck in the Shallow End: Education, Race and Computing by Jane Margolis and
her colleagues (2008) provides a compelling illustration of how educators’ deficit-
based views operate in an educational system, creating structures and ideologies
feeding inequities. Although Margolis’ study was conducted in the K-12 system, the
findings, revealing how structures and beliefs combine to perpetuate inequities, are
relevant to higher education as well. The book is the outcome of a study focused on
three different high schools in Los Angeles to explore reasons why so few African-
American and Latino/a high school students learn computer science. Despite the
differences between the schools, the results of the study demonstrate that, across all
three schools, students from different racial-ethnic groups simply do not receive the
same learning opportunities; instead, the more advanced coursework is available to
those students who are less in need (Parsons, 2010). In this situation, enthusiasm of
minority students for computer science is destroyed by the school system.

35

The authors reveal how the system of educational institutions limits the
achievement of students by investigating the “intricate interplay” between school
structures and belief systems, including psychological factors that affect African-
American and Latino/a students’ educational pathways (Margolis, et al., 2008, p. 16).
It can be assumed, therefore, that the potential solution to the problem of inequities
requires change in both institutional structures and, most importantly, the belief
systems of educators because the latter often influences the former. For example, in
one of the schools, the authors describe how structural constraints such as
overcrowded classes, insufficient teachers’ qualifications, accountability pressures and
funding limitations dramatically affect computer science curriculum. However, the
beliefs within the schools matter even more, “We quickly saw firsthand how belief
systems both contribute to this constrained curriculum and frame how educators make
sense of, as well as justify, schooling decisions” (p. 39). Teachers at the school
articulated deficit-based views toward Latino/a students in the following way, “They
do not have much interest in learning,” “They do not have that curiosity,” “They are at
a total loss” (p. 40). Consequently, those teachers designed their classes to match their
expectations for their students. As a result, the school’s curriculum concentrates on
teaching little more than clerical work and “cut and paste” technique rather than
anything stimulating and creative.
In addition, the structural inequalities along with the low expectations and
negative stereotypes toward minority students constantly shape student identities and
interests (Margolis, et al., 2008). They cause what Jones (2000) describes as
internalized racism, when, based on acceptance of negative messages about their own

36

abilities, stigmatized individuals do not believe in members of their racial group and in
themselves. The study contains examples showing how students drew on stereotypes
to communicate their thoughts. A Latino student described a computer scientist this
way, “A dude … they’ll probably be White or Chinese, cause they’re, like, smart and
like those things” (p. 60). As the authors visited different schools, they discovered
how deficit-minded approaches prevented practitioners from fully educating students
of color. They witnessed how structural inequities interacted with belief systems of
both teachers and students to “make a powerful mechanism that ends up preselecting
only a small number of students to receive the necessary preparation” for the modern
economy (p. 139). The findings of the book can also serve as indicators of the need for
research that addresses structural barriers to equity as well as the negative stereotypes
that create barriers to racial-ethnic equity, in computer science particularly, and in
education more broadly.
In higher education, structural barriers can be illustrated by the inefficient design
of remedial/developmental education in community colleges. In a study of the
monetary and time costs for transfer students in nine community colleges in the Los
Angeles district, Melguizo, Hagedorn, and Cypers (2008) found that the biggest
contributor to the cost of transfer is time students spend in remedial/developmental or
nontransferable courses. Student transfer productivity on a time-adjusted basis is
relatively low, “Despite the long time that the students remain enrolled in college, they
transfer only about one year’s worth of credits” (p. 423). The authors also found a
strong relationship between remediation and cost of transfer. Given the fact that
student population in California Community Colleges is represented by a substantial

37

majority of African American and Latino students with high remediation demands
(Center for Students Success, 2008), their costs of successful graduation or transfer
would be much higher than for students with lower remediation needs. This example
illustrates how college structures impede effective navigation of students from racial-
ethnic groups through challenges of remedial/developmental education.
Bensimon (2012) strongly believes that to bring about institutional
transformation for equity and student success, practitioners, including institutional
leaders, have to develop funds of knowledge for equity-minded expertise. She argues
that practitioners can make a noticeable difference in the educational outcomes of
minoritized students if they recognize that their practices are not working and if they
participate in designed situated learning opportunities to develop the funds of
knowledge necessary for equity-minded practice. These funds of knowledge include:
color-consciousness; awareness that beliefs, expectations, and practices assumed to be
neutral can have outcomes that are racially disadvantageous; willingness to assume
responsibility for the elimination of inequality; awareness that racialized policies and
practices can permeate higher education institutions and maintain racial hierarchies
despite increasing diversity. However, limited empirical evidence exists regarding
whether it really works. Therefore, practitioner’s beliefs, attitudes, and behaviors are
the subject of this study.
Action Research as Organizational Change
One method for developing funds of knowledge that would empower practitioners
with the expertise, know-how, and self-efficacy to produce equity in student outcomes
is through action research processes. Peter Reason and Hilary Bradbury (2001) define

38

action research as “A participatory, democratic process concerned with developing
practical knowing in the pursuit of worthwhile human purposes, grounded in a
participatory worldview ... It seeks to bring together action and reflection, theory and
practice, in participation with others, in the pursuit of practical solutions to issues of
pressing concern to people …” (p. 1). Given the emphasis on developing practical
knowing through participative inquiry, action research challenges many of the basic
assumptions and values of orthodox social science on combined epistemological,
methodological, and ethical/political grounds (Bensimon, Polkinghorne, & Bauman,
2004; Charles & Ward, 2007; Greenwood & Levin, 2005; Herr & Anderson, 2005;
Reason, 1994). The most distinctive challenge is epistemological, concerned with the
nature of knowledge as well as with the questions of what knowledge is and how it is
acquired (Bensimon, Polkinghorne, & Bauman, 2004; Charles & Ward, 2007).
According to Greenwood and Levin (2005), the conventional understanding of
knowledge tends to be grounded in its explicit forms: what can be recorded in words,
numbers, and figures and, thus, is explicitly available to people. Referring to Fuller’s
definition, Greenwood and Levin conclude, therefore, that knowledge, in conventional
understanding, tends to be treated as an individualistic, cognitive phenomenon formed
by a researcher’s ability to capture insights. It is highly theoretical and self-contained,
aiming “to remove as many concrete empirical referents as possible in order to achieve
the status of general truth” (p. 50). The primary purpose of traditional academic
research, consequently, is to contribute to an abstract body of knowledge available to
“third persons” (Reason, 2001). However, neither Greenwood and Levin nor Reason
assign any special priority to this conventional form of knowledge. In contrast, they

39

argue that the theoretical capability of knowledge is necessary, but “… no results ever
will be achieved unless practitioners learn how to act in effective ways and to use
necessary tools and methods” (p. 51). Hence, these scholars add more dimensions to
common concepts of knowledge in order to link “ … theoretical and practical
intelligence” (Greenwood & Levin, 2005, p. 50). They put emphasis on both knowing
how to act (Greenwood & Levin, 2005) or, in Reason’s (1994) words, developing
experiential and practical knowing through action and experimentation, as well as on
participative approaches to knowledge generation.
Regarding knowing how to act, Greenwood and Levin advocate the idea of
phronesis, formally defined by Aristotle as “internally consistent reasoning dealing
with all possible particulars” (p. 51). They describe it as the design of action through
collaborative knowledge generation in which the researcher’s knowledge is combined
with the knowledge of practitioners. Working together practitioners and researchers go
through an inquiry, or action research cycle. They explore and define the problem,
design actions to improve the situation together, and assess the results. If outcomes are
not satisfactory, they cycle through this process again and again until they achieve the
desired results. “Both researchers and local practitioners have rights to determine the
outcome,” state Greenwood and Levin (p. 50).
This kind of knowing is closely linked to action. Thus, knowledge is not a
passive form of reflection, but “ … emerges through actively struggling to know how
to act in real-world contexts with real world materials” (Greenwood & Levin, 2005, p.
51). Greenwood and Levin explain that to act means “to contextualize behavior” and
“to act skillfully implies that actions are appropriate to the given context” (p. 51).

40

They conclude that the actor needs to make sense of the context to enable appropriate
actions. The notion of “knowing how” thus implies knowing how in a given context in
which appropriate actions stem from contextual knowing. The traditional
understanding of knowledge with its inclination to universal applicability is of very
little use here because it is impossible to conceptualize action as taking place in a
“generalized” milieu (Greenwood & Levin, 2005). Thus, any study that wishes to
address a problem of practice, must examine how practitioners make sense of the
context using real world materials and tools.
Peter Reason (1994) follows a somewhat different path but ends up making
some of the same conclusions. He uses different descriptions of knowledge but with
similar intentions. While discussing three approaches to action research, Reason refers
to the works of Heron (1981), Tandon (1989), and Torbert (1991) in describing an
extended epistemology based on different forms of knowledge. Thus, in the co-
operative inquiry approach, extended epistemology involves the interplay of
experiential knowledge gained through experience of direct encounter with people,
places, or things; presentational knowledge emerged from experiential knowing, and
providing its first expression though forms of imagery; practical knowledge dealing
with knowing “how to” do something and expressed in skills and competencies; and
finally, propositional knowledge expressed in theories and statements and constituting
knowledge “about” something.
Another approach – participative action research – involves the reclaiming of
three ways of knowing – thinking, feeling, and acting. In action inquiry extended
epistemology is expressed in valid knowledge of four territories of human experience:

41

intuitive or spiritual knowledge of the system’s own purposes; cognitive knowledge
about its strategy; a practical knowledge of behavioral choices, “resting on an
awareness of oneself and interpersonal skill” (p. 330); and finally, empirical
knowledge of the outside world. Furthermore, Reason defines action inquiry as a form
of inquiry into practice concerned with development of effective action contributing to
the transformation of organizations toward greater effectiveness.
Reason seeks a solution to the current dilemmas of orthodox social science by
advocating a closer link to experiential and practical knowing, which arise through
participation with others. This participation he places “ … at the heart of inquiry
approaches” (p. 324). Similar to Greenwood and Levin (2005), Reason argues that in
research on persons, the propositional knowledge expressed in the research
conclusions needs to be derived from the experiential and practical knowledge of the
subjects in the inquiry. “If the propositions are generated exclusively by a researcher
who is not involved in the experience being researched and without consideration of
the practical and experiential knowledge,” he concludes, “the findings reflect neither
the experience of the researcher nor that of the subjects” (p. 326).
This literature pinpoints how researchers and practitioners, through working
together, can generate knowledge that enables them to take the appropriate actions
necessary to achieve their goals. In the words of Greenwood and Levin (2005), “The
core perspective is a conceptualization of knowledge as inscribed in actions that are
collectively developed and shared by individuals working together” (p. 51). The
authors argue strongly for the generation of knowledge, which involves practitioners
and is underpinned by continuing reflection.

42

Many scholars believe in the potential of action research to transform locally
situated understandings and practice. It is considered transformational to the
individuals involved and, by extension, to their teams and organizations (Burns, 2007).
Burns claims that by integrating “learning by doing” with deep reflection, action
research has always held the promise of an embedded learning process that can
simultaneously inform and create change. Being employed across many practitioner-
related disciplines in the health (Wang, 1999) and social sciences (Reason, 2001;
Greenwood & Levin, 2005), action research has garnered particular attention in the
field of education (Dowd et al., 2012; Mills, 2003; Noffke,1997; Suter, 2006). Gaining
increased interest in the past 20 years (Nolen & Putten, 2007), it is viewed as a
practical yet systematic research method that enables practitioners to investigate their
own practices and students’ learning (Bensimon, Polkinghorne, & Bauman, 2004). All
of this implies that action research may be a useful institutional self-assessment
strategy to bring about organizational change in higher education. However, action
research does not guarantee a focus on the problems of racism, inequities, or cultural
change. Cultural historical activity theory (CHAT) brings this focus.
Designing New Forms of Educational Activities Using CHAT
Integrating socio-cultural theories of learning with historical understandings of
racial discrimination, cultural historical activity theory (CHAT) emphasizes
remediating the artifacts of professional practice in education to achieve equity in
outcomes among racial-ethnic groups (Dowd, et al., 2012; Gutierrez & Vossoughi,
2010). CHAT acknowledges the social and cultural construction of knowledge and
addresses “ … how new learning takes place within a setting inherently constrained by

43

its historical legacies and the language, and objects present within its culture” (Dowd,
et al., 2012, p. 181). Therefore, learning is considered as situated and distributed
across “social settings” and “acting in concert with different artifacts” (p. 181).
However, CHAT considers not only these verbal and physical “cultural artifacts,” but
also the ways that individuals interact with these artifacts (Dowd, et al., 2012). The
group memberships, goals, rules, and norms, and division of labor within the setting
affect such interaction. Moreover, these elements are mutually constitutive and no
single element can be fully considered without including its connections with the other
elements. Therefore, “ … the learning setting is recognized as dynamic and reflexive,
characterized by various changing parts that may affect one another” (Dowd, et al.,
2012, p. 181).
In general, being a metatheory (Roth & Lee, 2007), CHAT incorporates several
central themes relevant to this study. First, the basic premise of CHAT is that learning
occurs in the context of culture, which mediates people’s interactions with each other
through values, assumptions, norms, and expectations (Gurierrez, Morales, &
Martinez, 2009). Consistent with this approach, Dowd and her colleagues (2012)
emphasize the importance of exploration of cultural factors that may facilitate or
hinder desired change within the study of organizational change.
Second, culture needs to be understood in historical perspective as people’s
“social inheritance” (Gutierrez, Morales, & Martinez, 2009, p. 216). This means that
people’s actions are influenced by the present state of cultures along with “the
histories of those cultures and the legacies of prior generations,” which often go
unnoticed and unquestioned (Dowd, et al., 2012, p. 182). “Historicized” (Gutierrez,

44

Morales, & Martinez, 2009, p. 103) understandings of current practices are critical in
order for change to occur. Dowd and her colleagues believe that active reflection
about how current practices are situated within history can help practitioners to
examine held assumptions and beliefs about their practices and allow for “a conscious
consideration of the benefits of ‘historically new’ forms of action” (p. 172).
Third, social interactional processes play an important role in the process of
learning and development. Learning is influenced by change in social relationships
and the social world (Nasir & Hand, 2006; Roth & Lee, 2007). Dowd, Bishop,
Bensimon, and Witham (2012) acknowledge that the context of a shared, mutually
engaging activity is the best way to learn new information and new actions, “As
participants negotiate roles and rules and embark upon ways of investigating or
solving problems, each brings back new discoveries” (p. 181). Roth and Lee (2007)
also emphasize the role of collaboration in enacting learning and development. They
argue that interplay between individual and society produces two “manifestations of
expansive learning” (p. 205). First, learning is expansive when “ … it contributes to an
enlarged room to maneuver for the individual whereby new learning possibilities are
formed” (p. 205). Second, learning is also mediated by the division of labor in
collaboration, which leads to learning outcomes and forms of societal activity.
The authors assert that learning occurs whenever a new practice, artifact, tool, or
division of labor within the activity system creates a new possibility for others, leading
to an increase in generalized action possibilities and, therefore, to collective learning.
Referring to Vygotsky’s concept, they conclude that it is collective work that allows
“new forms of actions, which are appropriated in what is termed the zone of proximal

45

development” (p. 205). This conclusion implies that creating organizational change
depends on creating social settings for collaboration, professional development and the
remediation of practice.
Fourth, CHAT provides the opportunity to examine the social construction of
practice holistically and in historical perspective (Dowd, et al., 2012). As shown in
Figure 2.1, it focuses on an activity system as a unit of analysis composed of six
elements: object, subject, mediating artifacts, community, rules, and division of labor.
A change in any of these facets may bring about changes in others, and may contribute
to change in the overall system, within which the various settings are embedded
(Ogawa, Crain, Loomis et al., 2008). Stillman (2011), in her study on how teachers
prepared to serve marginalized students respond to increased accountability demands,
shows how contexual factors, especially local leadership, mediated teacher learning in
the context of school change. She found that teachers’ professional learning and
instruction were improved when teachers were provided with opportunities to “grapple
with reforms they found objectionable and to apply innovations to their classroom
practices” (p. 134). Importantly, Dowd and colleagues (2012) note that such a model
allows for a focus on all of these facets within an action research methodology.
Fifth, foundational to CHAT is the principle of mediation – that is, that cultural
tools and artifacts fundamentally influence learning and development and are
mediators of thought (Nasir & Hand, 2006). According to Dowd and colleagues
(2012), a person’s understanding of what is valued at a given institution or
environment, what is accepted, what is celebrated or is to be striven for is
communicated in a number of ways: via language, signs, and via tools (the materials

46

of production and reproduction used in various activities). Nasir and Hand (2006)
suggest that artifacts can also be culturally held ideas, “ideational” artifacts, that
structure human interaction and activity (p. 466). Thus, mediated action is a process
by which individuals and their cultural tools interact through goal-directed action.
Individuals and the tools that they use to achieve their goals exist in an irreducible
tension: one cannot be separated from the other (Nasir & Hand, 2006).
Figure 2.1. An Activity Setting (CHAT) Framework

Finally, “activity systems harbor inner contradictions,” which are considered
facilitative as they present an opportunity for participants to question, challenge, and
experience conflict, all of which engender learning (Roth & Lee, 2007, p. 203;
Gutierrez & Vossoughi, 2010; Dowd et. al., 2012). Roth and Lee argue that when
inner contradictions are conscious, they become the primary driving forces that bring
about change and development within the activity system. Whereas inner
contradictions reveal themselves during analysis, the authors assert, they express
themselves as trouble in ongoing activity. When such trouble is available to
consciousness, it can be addressed by a change of goals (Roth & Lee, 2007).

47

CUE’s participative action research, by design, anticipates contradiction within
activity system. For example, the low numbers of students transferring is introduced as
contradiction. As the Scorecard participants’ early hunches about the reasons for such
inequities are explored, a period of questioning and challenging ensued (Dowd et. al.,
2012, p. ). The ensuing cycle of inquiry leads to further inquiry into factors and
potential solutions. Gutierrez and Vossoughi (2010) argue that change in the
individual involves change in the social situation itself – tools, relationships with
participants in a community, and division of labor. According to Gutierrez, Moralez,
and Martinez (2009), instead of emphasizing problems as located in the individual, the
remediation of learning involves the reorganization of the whole activity system,
including the social organization of learning, the social relationships in the community
of practitioners, the division of labor, and, more importantly, the artifacts in use. It
also includes a “shift in the way that mediating devices regulate coordination with the
environment” (p.102). This process entails a transformation in the social organization
of learning, the social relationships, the forms and uses of artifacts, and in roles. For
the reasons above, studies of organizational change have become focused on activity
systems as the unit of analysis (Dowd et al., 2012; Stillman, 2011; Bustillos, Rueda,
Bensimon, 2011).
Dowd et al. (2012) argue that one approach to remediating the roles and
community of practice of higher education practitioners is to ask them to assume the
role of action researchers and to conduct action inquiry into their practices. “The
CHAT framework makes it clear that, in order to remediate practice, it is necessary to
remediate the artifacts that are the tools for the social construction of practice” (Dowd

48

et al, 2012, p. 181). CUE, therefore, focuses on creating remediating artifacts in the
form of tools for action inquiry and action research. These include a scorecard (e.g.,
the Equity Scorecard), tools (e.g., the Benchmarking Equity and Student Success Tool
(BESST); the Equity-Based Assessment Toolkit; the STEM toolkit), and self-
assessment inventories (e.g., the Transfer Access Self-Assessment Inventory; the
Organizational Learning Inventory (OLI); the Syllabus Review Inventory).
These tools are designed to assist the learning, equity-mindedness, and
professional development of college administrators and faculty members in ways that
enhance college capacity to achieve equitable student learning outcomes (Dowd &
Bensimon, 2009). As illustrated by prior empirical studies, tools have the capacity to
remediate practitioners’ language (make it equity-oriented). According to Dowd and
colleagues (2012), the use of language in and through the tools of action research and
action inquiry is key in a number of ways: first, language introduces race-conscious
terms that can lessen the discomfort practitioners experience when talking about race
and ethnicity and facilitate their transition from that discomfort to an intentional
dialog. Next, the introduction of a new language may help induce the exploration of
reasons for the absence of certain racially explicit language on campus. In addition, a
language that provides a vocabulary for discussing race allows stereotypes and false
assumptions to surface so that they can be tested and invalidated in the process of
reframing attributions about student inequities. Finally, a new language plays a major
role “ … in the framing of inequitable student outcomes as the responsibility of the
institution and its practitioners, rather than as a result of what students lack” (Dowd et
al., 2012, p. 174). From this view, as language shapes people’s understandings,

49

interactions, and actions (Dowd & Bensimon, 2009), remediation of language will,
consequently, lead to the remediation of practitioners’ knowledge, beliefs, and
practices.
Data are also cultural artifacts of practice (Dowd & Bensimon, 2009). However,
data are not self-acting to produce improvements in institutional practices.
Practitioners must analyze, interpret and act on them. CUE aims to remediate the use
of data by emphasizing that the purpose of the data analysis is to reframe the problems
of student success from the perspective of student deficits to one of institutional
responsibility. Moreover, the process of organizational learning is often challenging
since individuals at the institution do not have the time or existing structures to use the
data that already exist at the institution. Therefore, even though the recent higher
education accountability movement, with its focus on data and outcomes, is aligned
with the goals of outcome equity (Dowd, 2007), Dowd and Bensimon (2009) question
whether it has the potential to bring about positive change in institutional performance.
While the Center shares the interest of policy makers in obtaining the data necessary
to promote equity, it places a unique emphasis on the actual use of data. Bensimon
(2012) argues that even though data are necessary to engage in equity sense making, it
is critically important for practitioners to understand that the ultimate “ … purpose of
examining data disaggregated by race and ethnicity is to create a context to talk about
race and ethnicity …” (p. 33).
Incorporating Critical Perspectives on Racialized Practices
Bensimon (2012) argues that a lack of race-consciousness in deficit-minded
attributions results in negative stereotypes based on race and ethnicity. Equity-minded

50

reflection on educational problems, she believes, requires practitioners to learn how
race is embedded in everyday practices. Towards that end, it is helpful to consider the
meaning of the terms “race” and “racism” in the U.S. today. Although race has no
biological meaning, it has social and political significance (Jones, 2000). Many race
scholars believe that racial distinctions matter because they are still prevalent in a
modern American society. Camara P. Jones (2000) writes, “The variable race is only a
rough proxy for socioeconomic status, culture, and genes, but it precisely captures the
social classification of people in a race-conscious society such as the United States.
The race … has a profound impact on daily life experiences in this country” (p. 1212).
Recognizing that the barriers to college success for racialized minority students can be
structural (inherent in systems and policies) or interpersonal (stemming from
conscious or unconscious stereotyped or discriminatory beliefs), it is useful to
distinguish among different forms of racism.
More than a decade ago, Jones (2000) elaborated a recognized theoretical
framework for understanding racism on three different levels. The first level is
associated with institutionalized racism, defined as “differential access to the goods,
services, and opportunities of society by race” (p. 1212). It is normative, structural,
sometimes legalized, and often manifests as inherited disadvantage. Although Jones
asserts that institutionalized racism is the most fundamental and must be addressed if
important change is to occur, it is difficult to underestimate the significance of the
other two levels. The next level refers to personally mediated racism, which manifests
as prejudice in the form of differential assumptions about the abilities and intentions of
others according to their race, and discrimination in the form of differential actions

51

toward others according to their race. It is noticeable that personally mediated racism
can be intentional as well as unintentional. Finally, the third level is represented by
internalized racism, defined as acceptance by members of the stigmatized races of
negative messages about their own abilities and is characterized by their not believing
in members of their racial group and in themselves. It involves accepting limitations to
one’s own full humanity, one’s right to self-determination, and one’s range of
allowable self-expression. All three levels are closely interconnected and may
influence each other, thus perpetuating inequalities.
In his recent book Microaggressions in Everyday Life: Race, Gender, and
Sexual Orientation, Derald W. Sue, “the most influential multicultural scholar in the
United States” (p. xxiii), also discusses how racism continues to exist in the 21
st

century. Sue distinguishes between “old-fashioned racism” and “modern racism” or
“racial microagressions” (p. 142). He asserts that while old-fashioned racism in its
blatant and visible forms at three levels, elaborated by Jones (2000), seems to
disappear, it “has manufactured a new face” (p. 143). It (a) morphed into a highly
invisible and subtle form that lies outside the level of conscious control; (b) resides in
the invisible assumptions and beliefs of people; and (c) is embedded in the policies
and structures of institutions. Sue argues that even individuals who believe in equality
and embrace democratic ideas may continue to harbor unconscious racist attitudes and
beliefs toward people of color. He maintains that racial microaggressions in the form
of everyday slights, insults, and invalidations (though invisible to perpetrator) toward
marginalized groups create disparities for them in education, employment, and
medical care. Therefore, speaking specifically about students of color, it is highly

52

likely that these racial “microaggressions” affect the opportunities available for them,
their integration into postsecondary education and into society in general. Jones and
Sue provide comprehensive frameworks for understanding racism and its dynamics.
The frameworks can be utilized as a lens for exploration of race-associated differences
in various organizations including educational institutions. In this respect, studies of
the discourses of race are needed to understand how to remediate racialized practices
in higher education as manifested in language and documents of higher education.
CUE’s race-conscious approach to data inquiry seeks to build practitioner
awareness of race-based assumptions, which attribute unequal outcomes to student’s
cultural predispositions, as practitioners are directed to look at institutional data
disaggregated by race and ethnicity (from CUE’s website). These assumptions are
based on stereotypes and are often used to rationalize inequalities as peculiar to
specific identities. When practitioners making these statements look at data showing
actual trends in student educational attainment, they usually find that they are over-
generalized and unsupported. Data based inquiry demonstrates that stereotypes and
assumptions about minority students are not sound reasons to not take action to
address inequities (Dowd & Bensimon, 2009). CUE researchers argue that such
inquiry begins with a willingness to speak about race, to develop common language
about race, and to make race an explicit focus. From this view, when practitioners
allow issues of race and equity into their conversations, become comfortable talking
about race-based gaps, and contextualize the inequities according to the historical and
current context of race, they are in a position to actually address the issues and make
changes (Bensimon, 2012). A product of such conversations, Bensimon believes, is

53

equity-minded practitioners who are color-conscious; aware that beliefs, expectations,
and practices assumed to be neutral can have outcomes that are racially
disadvantageous; willing to assume responsibility for the elimination of inequality;
aware that racialized policies and practices can permeate higher education institutions
and maintain racial hierarchies despite increasing diversity.
Summary
To understand whether and how practitioner’s beliefs, attitudes, and behaviors in
regard to equity are influenced by action research processes and tools, it is worthwhile
to investigate faculty members, administrators, and staff who participate in an action
inquiry using one of CUE’s tools. In regard to the present study, limited empirical
evidence exists about how social interactions, facilitated by action research and
equity-oriented cultural artifacts, mediate practitioner’s attitudes, beliefs, and
behaviors related to racial-ethnic inequities.
Action research approaches, engaging higher education practitioners in self-
assessment activities for reflective practice, can be used to investigate solutions to
inequalities. Calling attention to the role of social interaction and cultural artifacts in
shaping educational practices, CHAT emphasizes the influence that history has on
organizations. Using CHAT as a framework will allow an examination as to how an
institution’s culture and history may be impeding organizational change. The cultural
historical concept of remediation can serve as a starting point for reimaging what
higher education practices can look like in regard to equity.
This study involved investigations into whether CUE’s action research processes
and tools led practitioners to adopt equity-minded perspectives that would allow them

54

to address the diverse needs of students; whether they led practitioners to adopt equity-
minded practices; and whether they worked in helping practitioners become more
effective in educating underrepresented students. Chapter 3 presents the methodology
pertaining to this investigation.

55

CHAPTER 3: RESEARCH METHODOLOGY
The previous chapter explored the concepts and theories that inform the present
study. To review, this study examined the impact of action research processes and
tools on learning and change among higher education practitioners. Many
contemporary action research facilitators believe in action research’s potential to
transform locally situated understandings and practice (Burns, 2007; Greenwood &
Levin, 2005; 2007; Stringer, 2007). They believe it is transformational to the
individuals and organizations involved. Burns (2007) asserts that by integrating
“learning by doing” with deep reflection, action research simultaneously informs and
creates change. This change, however, is not revolutionary, but rather can be
characterized as “ … subtle transformations brought by the … modifications to
existing practices” (Stringer, 2007, p. 208).
This study was part of a larger research agenda being carried out by researchers
at USC’s Center for Urban Education (Baldwin et. al., 2011; Bensimon et al., 2004;
Bensimon et al., 2010; Bustillos, 2007; Dowd, 2005; Dowd & Bensimon, 2009; Dowd
et al., 2012; Enciso, 2009; Salazar-Romo, 2009). Specifically, it examined the impact
of the facilitation of action inquiry using CUE’s tools on the attitudes, beliefs, and
behaviors of a small sample of faculty at a single institution where CUE’s tools were
used. The multiple field sites involved in this collective study were purposefully
sampled based on their relationship with CUE. College and university faculty and
administrators at each of the field sites were engaged in action inquiry facilitated by
CUE researchers. The field site for this study was a state university in California with
selective admissions practices and an emphasis on science and technology education.

56

Figure 3.1 illustrates the relationship of this study, which used developmental
evaluation methods, in relation to other types of research conducted by CUE
researchers. Developmental evaluation is informing the development of CUE’s action
research processes and tools, which are designed to foster equity among racial-ethnic
groups in higher education experiences and outcomes. More broadly, this study
informs institutions of higher education about how to incorporate action research into
the assessment of institutional effectiveness and equity.
Figure 3.1. Developmental Evaluation Methods Inform Multiple Modes of
Research

The unit of the study was constituted by the “activity settings,” in which action
research was conducted (Tharp & Gallimore, 1998, p. 72). As illustrated in Figure 3.1,
the activity settings represented the workshop, the webinar or various other meetings
where action research was conducted. For this study, the meetings included faculty,
administrators, and staff who became practitioners researching disaggregated data
from their own institution using a variety of CUE’s tools, including, for example, the

57

Benchmarking Equity and Student Success Tool (BESST) and Document Analysis
Protocol (Document Analysis for Self-Assessment of Culturally Inclusive Practices).
Practitioners came to these meetings with their own values, beliefs, and assumptions.
Using these tools allowed for social interaction by collaboratively discussing what was
needed for change. The BESST Tool allowed the data to be manipulated to show how
changes in one milestone can influence greater student outcomes. The Document
Analysis Protocol helped faculty members explore and reflect on their syllabi; thus, it
facilitated their learning to become culturally responsive practitioners. This was the
1st person action inquiry stage, where an individual or group engaged in study for
reflective practice.
CUE engages higher education practitioners in action inquiry process through
participatory action research. 2nd person action research stage in Figure 3.1 represents
CUE’s relationship with practitioner colleagues in this work. As institutional outsiders
conducting action research, CUE researchers “create processes and assessment tools
for action inquiry on the part of institutional insiders, who use them to assess their
own practices” with the purpose to address inequities (Dowd & Bensimon, 2009, p. 2).
Dowd and Bensimon add that from this outsider status action researchers can
orchestrate a change process in ways that insiders cannot.
I worked in relationship to a larger CUE Evaluation Study Team as a
developmental evaluator conducting 3rd person case study. The team had two main
goals: first, to develop CUE’s evaluation capacity by improving the validity of the
inferences drawn from evaluation questionnaires, and second, to enhance CUE’s
effectiveness in conducting action research for the purposes of improving equity in

58

higher education. This case study provides CUE researchers with more nuanced
understandings of the experiences of the action inquiry team participants. That means
better understanding of the reactions, attitudes, reflections, and action steps taken by
practitioner participants in ways that create positive impact in their students’ lives
(Bensimon, et al., 2010).
As shown in Figure 3.2, CUE conducts action research to facilitate practitioner
inquiry. For the purposes of this study it is important to understand the types of
changes that might come about through action inquiry and through what types of
activities. As action researchers, CUE creates activity settings with the aim of
remediating educational practices that are harmful to racial-ethnic equity. Tharp and
Gallimore (1998) emphasize that problem solving and learning cannot be “ …
understood outside the complex social context” (p. 91). They explain that designing
learning experiments should include assisted performance activities with peer
consultants. The Center draws on this notion and explains that in CUE projects,
“learning and knowledge are socially constructed through joint productive activity”
(Dowd & Bensimon, 2009, p. 13). In joint productive activity settings, the members of
a group are not sharply divided into novices and experts, but rather are a combination
of people with different competencies that work together in a manner where an
individual member will assist others in the group depending on his or her own areas of
knowledge and skill (Tharp & Gallimore, 1998). Similarly, the CUE research model
theorizes that a critical point of intervention in a change process is created by social
activity (Dowd & Bensimon, 2009). Therefore, the CUE approach uses social

59

activities as a critical point of intervention for organizational change in colleges and
universities.
Figure 3.2. Reflective Practice (Inquiry) as Driver of Racial-Ethnic Equity
in Postsecondary Education

As shown in Figure 3.2, the action research inquiry model is cyclical. Adaptive
expertise is acquired through a social inquiry experience. The first part of the figure
demonstrates the cycle of reflective practice and shows how practitioners examine
their knowledge and beliefs. The social interaction creates a forum for practitioners to
be open to “different perspectives and problem framing through data analysis” and
allows for them to engage in experimentation and problem solving (Dowd &
Bensimon, 2009, p. 6). Action research contextualizes the problem and is intended to
support a broader range of professional actions and experiences, which are the source
of adaptive expertise. Through the systematic use of observation and data analysis in
this reflective inquiry cycle, untested assumptions can be tested in ways focused on

60

problem solving. Experimentation is valuable because it opens up possibilities for new
ways of seeing and acting (Dowd & Bensimon, 2009, p. 6).
The tenets of practice theory and social learning theory assert that the cycle of
inquiry creates a way for practitioners to examine their beliefs (such as deficit minded
beliefs) and in this process to un-learn old modes of thinking and participate in
learning activities that lead to new knowledge (Dowd & Bensimon, 2009; Tharp &
Gallimore, 1988). In this study, the social intervention point occurred during the
learning activity where practitioners as a group used CUE’s tools that “provided
impetus for reflection, problem identification, experimentation, etc.” (Dowd &
Bensimon, 2009, p. 13). It is difficult to definitively measure the impact of equity-
based inquiry activities, but Figure 3.2 illustrates the logic model for the expected
impact of this social activity when practitioners use knowledge gained through the
cycle of inquiry to make changes to institutional structures and practices. If positive
change occurs, practitioners create environments that are more equitable. However,
social learning is not the only point of impact on student educational outcomes. Figure
3.2 also illustrates that state policies and institutional structures and practices play a
role in changing practitioner behavior as well, even if it is just as a reaction to stay in
compliance with rules or mandates.
Developmental Evaluation
As it was mentioned previously, this study used developmental evaluation
methods. According to Michael Quinn Patton (2011), the best way to decide which
type of evaluation to use is to be clear about the purpose of the study. Once the
primary audience is established, researchers can make a specific design study to gather

61

data and analyze decisions that address the issues. Formative and summative are
known as the conventional forms of evaluation. Formative evaluation focuses on
improving a program or model and generates questions that enable researchers to
inform decision-makers about ways of improving effectiveness. Its primary purpose is
to get prepared for summative evaluation (Patton 2011). Summative evaluation, in
turn, encapsulates information to make judgments regarding programs or practices in
order to decide whether they should be continued or not. Its goal is “ … to make an
overall judgment about the merit and worth of the program” (Patton, 2011, p. 2). The
following paragraphs explicate the study’s use of developmental evaluation, a
methodology that is appropriate when the organization conducting the study operates
in a complex, dynamic environment and is interested in developing innovative and
responsive processes that will function successfully in those environments.
As a member of the team, I evaluated the effectiveness of CUE’s tools in
creating change of attitudes, beliefs, and behaviors in practitioners, so the form of
evaluation needed to be an on-going process. This allows researchers the opportunity
to analyze for continuous improvement. Patton (2011) defines developmental
evaluation as the processes and activities that support programs, projects, products,
personnel and/or organizational development (usually the latter). The evaluator is part
of a team whose members collaborate to conceptualize, design, and test new
approaches in a long-term, on-going process of continuous improvement, adaptation
and intentional change. The evaluator’s primary function in the team is to facilitate
organizational discussions and enable data based decision-making in the process
(Patton, 2011, p. 317).

62

Patton characterizes developmental evaluation as instrumental for social
innovators who are trying to bring about change. In this sense, he argues, there is a
necessity to realize that change does not follow a linear path. There are dynamic
interactions, unexpected, unanticipated divergences, tipping points and the traditional
evaluation approaches do not fit these situations properly. Developmental evaluation
tracks any emergent and changing realities using findings in real-time as well as
adapting to complex dynamics rather than trying to impose order and certainty into an
uncertain world. It attempts to make sense of what emerges under conditions of
complexity. In trying to create change, it is necessary to get beyond just identifying
the problem and finding a solution, what is commonly called single-loop learning.
Developmental evaluation allows participants to dig deeper into the assumptions,
policies, practices, values and system dynamics that led to the problem in the first
place and intervene in ways that involve the modification of the underlying system
relationships and functioning (Patton, 2011). This process allows for examination of
the effects of actions and for reflection on important questions such as: Are we
walking the talk? Does it work? How do we know? What are we observing that is
different, that is emerging? More questions posed by Dowd and Bensimon (2009)
regarding intervention include: Are these tools leading individuals to adopt equity-
minded perspectives that will allow them to address the diverse needs of students?
Does this work lead campuses to adopt equity-minded practices? Does it help
individuals and institutions become more effective in educating underrepresented
students? Does it produce more equitable student outcomes? While it does depend on
the individuals and institutions that are involved, developmental evaluation allows

63

assessment to be an ongoing process. That builds organizational capacity to carry out
innovative work.
Patton (2011) explains that organizations that become involved in
developmental evaluation are usually more willing to ask these difficult questions and
identify their shortcomings and failures. Kruse (2001) explains that the development
of continuous improvement planning that takes place in schools is a form of
collaboration centered on student outcomes and that creating a culture of collaboration
leads teachers to engage in problem solving. Developmental evaluation is the process
that measures and may encourage continuous improvement. In order to implement
change, an organization incorporates ongoing assessment and a solid evaluation plan
(Bensimon, et al., 2010). Patton (2011) reiterates, “The concept of developmental
evaluation isn’t a model. It is a relationship founded on a shared purpose:
development” (p. 313).
Sample and Field Site
The sample for my study and for the collective CUE developmental evaluation
study was recruited from among participants in CUE’s action research projects.
Although one of the field sites in the collective study was engaged in a multi-year
action research project with CUE, the remaining field sites were involved in shorter-
term projects (in duration of one year or less) consisting of a series of planning
meetings, workshops, and webinars. The workshops and webinars were conducted by
CUE under a variety of circumstances consistent with the Center’s mission and typical
practices. These shorter-term projects involved many of the aspects of action research,
such as an integrated planning, inquiry, problem framing and solution generation

64

process, but they can be best characterized as design experiments because of their
shorter duration. In effect, they were not full-fledged action research projects, because
the time span was not sufficient to support a complete cycle of inquiry.
Numerous definitions and stated purposes have been posited in the literature to
describe design experiments or design research in education. The Design-Based
Research Collective (2003) defines it as “an important methodology for understanding
how, when, and why educational innovations work in practice” (p. 5). Cobb et al.
(2003) articulate that design experiments entail both “engineering” particular forms of
learning and studying those forms of learning with the context, subject to test and
revision (p. 9). In this study, CUE action researchers sought to “engineer” the
environments where practitioners learnt how to counter institutionalized racism and
marginalizing practices in higher education.
Design experiments can be summarized as having the following unique features.
First, design experiments in education blend empirical educational research with the
theory-driven design of learning environments (the Design-Based Research Collective,
2003). The second “crosscutting feature” is the highly interventionist nature of the
design research (Cobb et al., 2003, p. 10). The goal is to explore the possibilities for
improvement by bringing about new forms of learning in order to study them. Third,
design experiments are characterized by iterative design (Cobb et al., 2003; Bannan-
Ritland, 2003; The Design-Based Research Collective, 2003). This iterative design
process features continuous cycles of design, enactment, analysis, and redesign. The
fourth feature of design experimentation emphasizes focus on authentic settings and
interactions that refine understanding of the learning issues involved (The Design-

65

Based Research Collective, 2003). Finally, in design experiments practitioners and
researchers “work together to produce meaningful change in contexts of practice”
(The Design-Based Research Collective, 2003, p. 6). The Design-Based Research
Collective sees four areas where design experiments provide the most promise:
exploring possibilities for creating novel learning and teaching environments;
developing theories of learning and instruction that are contextually based; advancing
and consolidating design knowledge; and increasing capacity for educational
innovation, which is consistent with the developmental evaluation methodology.
All of the colleges and universities selected as field sites for CUE’s
developmental evaluation study are located in California. Each institution sent faculty
and administrative representatives to workshops and webinars hosted by CUE in
Spring and Fall 2011, and in Winter 2012. A variety of CUE tools were used in these
settings that asked participants to engage in social interactions and reflection in a
manner such that professional development and learning around issues of race and
equity could be investigated using the developmental evaluation methods of this study.
Therefore, the activity settings created through the use of CUE’s tools during these
workshops had the characteristics of design experiments.
Some of the workshops involved participants in examining equity gaps in
student persistence and degree completion at their own institution through the use of
the BESST. In other settings a Document Analysis Protocol was used to engage
faculty in reflection about their pedagogical practices, particularly in relation to
minoritized racial groups on their campuses. An Action Planning Protocol provided
another example of a CUE tool employed at the workshops. It asked participants to

66

brainstorm about actions they could take to promote equity on their campus through
intervention at critical curricular or programmatic milestones. In sum, three
community colleges and one state university provided field sites for the collective
CUE developmental evaluation study. Table 3.1 presents the pseudonyms for each
institution and indicates CUE’s tools used at each one. As evident from the table, the
majority of colleges are designated by the Federal Government as Hispanic Serving
Institutions (HSIs) because their Hispanic enrollment meets or exceeds the HSI
designation threshold of 25% of the student body.
Table 3.1. Field Sites of the CUE Developmental Evaluation Study

Pseudonym Type of Institution CUE Tools Used at
Workshops/Webinars
Involving Individuals from
this Field Site
Amarillo Community
College
Community college Defining Equity CUE
Module;
Benchmarking Equity and
Student Success Tool
(BESST);
Dynamic
Community College
Community college with
federal designation as a
Hispanic Serving Institution
CUE Modules;
Benchmarking Equity and
Student Success Tool
(BESST);
Action Planning Tool;
Syllabus Reflection Protocol
Los Flores
Community College
Community college with
federal designation as a
Hispanic Serving Institution
CUE Modules;
Benchmarking Equity and
Student Success Tool
(BESST);
Action Planning Tool

67

Table 3.1, Continued

Monarch State
University (MSU)
Selective state university with
emphasis on science and
technology education and a
predominantly white and Asian
student body
CUE Modules;
Benchmarking Equity and
Student Success Tool
(BESST);
Action Planning Tool;
Document (Syllabus)
Analysis Protocol;
The Racism cartoon;
“Who helped you through
college;”
Microaggressions Tool;
Informational Handout

This study’s field site was Monarch State University (MSU). Monarch State
University is a 4-year public comprehensive university with an undergraduate
population of over 17,500 located in a small suburb in California. The following
information reported is from the Department of Education’s Integrated Postsecondary
Education Data System (IPEDS). To avoid reporting the exact school in the study,
statistics are rounded. Monarch’s primary educational programs award bachelor
degrees. The mission statement of the university highlights a commitment to academic
excellence and educational quality. The institution boasts a 32% admission rate for first
time freshman. The university website highlights high-test scores and grades as a part of
the admissions process. In 2009, transfer students made up 15 percent of the nearly
4,000 new students that entered that year.
The university had several trends that set it apart from peer institutions
(institutions that include other California state universities and other public state
universities with similar Carnegie classifications). For one, while peer institutions
reported an average of 30% of students receiving Pell grant aid, Monarch reports only

68

10 percent. Second, compared to the ethnic composition of other university peer
groups, Monarch’s ethnic representation showed marked differences. For instance,
peer institutions on average had a student population that was about 45% white, while
Monarch’s white student population is over 60%. Latinos or Hispanics make up about
16% of the student population at peer institutions, but only represent 12% at Monarch.
Peer intuitions have Black student populations of about 7 percent and Asians at about
5 percent, but Monarch's percentages are 1 and 11 percent, respectively. These
differences in ethnic composition highlight a tension that is prevalent in this study.
The institution prides itself on its commitment to educational excellence, which the
university has translated as requiring high academic standards. Yet, this focus on high-
test scores and grades in the admissions process, which can be characterized as a
selectivity agenda, may conflict with the pursuing an equity agenda.
For more than a decade, since the 2000 WASC accreditation, the university has
worked to improve the institution’s diversity issues, including the development of a
significantly revised program review process, adoption of a model of Inclusive
Excellence, and the creation of Diversity Learning Objectives (DLOs), which have
been used to assess the effectiveness of diversity content in curricular as well as
effects of improvement in the diversity climate of the campus (WASC Report, 2010).
Before engaging in action inquiry activities with CUE, many faculty members and
administrators at MSU were already participating in self-reflective and evaluative
research projects that focused on diversity issues on campus. For example, the
university put together an advisory council made up of students, administrators, and
faculty to exclusively focus on providing recommendations to the university president

69

about how to improve equity and diversity in areas such as admissions, retention,
curricular development, and the campus climate. Other initiatives on campus also
included a pilot study on improving teaching and learning in science, technology
engineering, and mathematics (STEM) disciplines as well as a learning experiment
involving engagement of engineering students. Both of these initiatives had
implications for diversity in terms of gender and ethnicity.
During 2011-2012, the years when the present study has been conducted, MSU
has been engaged in a wide range of activities related to diversity and inclusivity
issues. Its current Strategic Plan includes “helping California meet future challenges in
a global context” as part of its Vision. The concept of “Fostering diversity and cultural
competency in a global context” is one of President’s six Strategic Imperatives that
“will allow the university to achieve its vision of becoming the nation’s premier
comprehensive polytechnic university” and “an innovative institution that develops
and inspires whole-system thinkers to serve California and help solve global
challenges” (System Documentation, 2012). This strategic imperative was also
emphasized within the September 13, 2011 speech by President, in which he said that
diversity is the Monarch’s “Achilles heel” and concluded that Monarch’s students “ …
cannot be truly successful in an increasingly diverse society and in an increasingly
complex, global economy unless they experience, as students, what it takes to function
in the real world” including exposure to multiple cultures.
Finally, in February 2012 the MSU held a diversity colloquium “intended to
showcase the university … as an inviting place for people of all ethnic backgrounds
and sexual orientations” (retrieved from the university website). MSU President and a

70

newly hired Provost stated that the university and the community “unquestionably will
make overtures within California and across the nation that they want a student body,
faculty and staff, and community citizenry representative of the many races, ethnic
groups, and cultures of the state and nation” (retrieved from the university website).
The sample size for this case study comprised 6 MSU faculty members from
various departments. The larger data set of the collective study included 30-40
respondents. These participants were observed and interviewed across multiple points
of interaction throughout the study. The following section describes participants and
field settings of my study.
Data Collection
Participant Recruitment
All participants in this study were provided with a letter that outlined the study’s
ethical commitments for interactions with human subjects (see Appendix A). The
letter provided an overview of the study and explained that all data collected form the
participants would be treated confidentially and anonymously. Two professors, who
served as co-facilitators on the part of Monarch State University, initially recruited
study participants in Fall 2010. The composition of the other team members included a
vice-president, deans, faculty members, and student services personnel. Colleges and
departments represented at different meetings included: School of Education; College
of Science and Math; College of Liberal Arts; College of Engineering; Admissions
and Recruitment; Academic Success Office; Office of Diversity and Inclusion
Initiatives.
After initial planning meetings in February and April 2011, four leaders were
identified, who were responsible for recruiting participants during Summer 2011. As a

71

result of their effort, by September 2011 four inquiry groups involving a total of
approximately 25 people, were formed: the group of student affairs administrators; the
group of the project leaders; the group of STEM faculty; and the group of non-STEM
faculty. The focal group of the participants for my particular study included 6 faculty
members teaching non-STEM disciplines at MSU with the exception of an Assistant
Professor from a STEM field. In early September of 2011 I began email
communication with them. Three were male and three female, but to maintain
participant anonymity (and the results not being analyzed from the perspective of
gender), I use a possessive pronoun in the form of “he/she” when presenting results
and findings in the following Chapter 4.
Field Settings
This action research project (as implemented in the design experiments) began
in the Spring of 2011 and concluded in the Spring of 2012. During Spring 2011 the
main focus of the project was the testing and development of instruments for data
collection during workshops and for cognitive interviews. The months of Summer and
early Fall 2011 were used to recruit individual participants for this study.
As shown in Figure 3.3, I collected data in a variety of ways at or subsequent to
CUE-facilitated workshops. The first level of the figure provides a time line for the
data collection; the second level represents the focus of the action research activities;
the third level indicates my role and method of data collection; and, finally, the fourth
level indicates the number of participants at each event. Throughout the project, 15-25
individuals consisting of Monarch State University faculty, support staff, and
administrators met in a series of CUE conducted events. Overall, five activities

72

occurred: a community and planning meeting (late February 2011); Analyzing Data
Using BESST workshop (late April 2011); Document (Syllabus) Analysis workshop
(early October 2011); Document (Syllabus) Analysis webinar/seminar (early February
2012); and Assessing Students Reactions to Culturally Inclusive Practices (mid
February 2012). As it is evident from their prominent and highlighted position in the
figure, the Document (Syllabus) Analysis workshop and the Document (Syllabus)
Analysis webinar served as primary activity settings for my data collection. These two
events involved twenty-five and six participants, respectively. The number of
participants in the other events ranged between fourteen and eighteen.
Initially, at the first planning meeting and workshop, I was a participant
observer. While this allowed me to gain an introduction to the setting, I did not collect
data. My data collection began in the Fall of 2011 and was ongoing through to the
final activity of a webinar conducted in the Winter of 2012. As indicated in Figure 3.3,
I collected observational data at the two document (syllabus) analysis workshops and
then conducted cognitive interviews with participants immediately following the
workshops. Evaluation questionnaires were administered by CUE following each of
these activities, as well as following the final webinar/seminar held in mid-February.
Respondents to the questionnaires numbered 19 after the first workshop, and 20 total
(combined), after the second and third workshops. In addition, I conducted follow up
individual interviews with the participants in the first document (syllabus) analysis
workshop, to obtain more information from them concerning their experience and
reactions to this event after some additional time had elapsed.

73

Figure 3.3. MSU Field Site, Timeline, Activity Settings, and Data Collection

To communicate the context of the field settings in which I collected my data,
CUE’s action research activities and tools, as implemented at MSU, are described in
detail towards the conclusion of this chapter. The following section describes the data
collection methods and instruments
Data Collection Methods and Instruments
The following describes the data collection methods that were used for this
study. Table 3.3 illustrates the variety of data that were collected and how these
different forms of data provided evidence to answer the research questions. The first
column of Table 3.3 outlines the range of data sources, which can generally be
separated into five categories: (a) documents; (b) observations; (c) evaluation
questionnaires; (d) cognitive interviews; and (e) individual interviews. The second and
third columns illustrate the type of data represented by each source and the points in
the study when it was collected. The fourth column explains how the data collected

74

from the various sources were summarized. Most data, with the exception of
observations, were summarized as descriptive text, categorical summaries, and/or
tabular summaries. Observational and interview data were subjected to deductive and
thematic analysis.
Table 3.2. Summary of Data Collection Methods
Data Source Data Represents When Data Were
Collected
How Data
Were
Summarized
Documents Policies
Discourse
Espoused beliefs
Environmental
factors
Throughout study
(Fall 2011/Spring
2012)
Descriptive text
Observations in
activity settings
(workshops,
webinars)
Behaviors
Social interactions
Norms
Discourse
Knowledge
During workshop
(Fall 2011)
During webinar
(early February
2012)

Deductive and
thematic
analysis;
Numerical
tables, and
summarizing
text (mode,
range, strength
and direction of
impact)
Workshop/webinar
evaluation
questionnaires
Attitudes
Beliefs
Practices (self-
reported)
Immediately after
workshop and
webinars
(Fall 2011)
Bar graphs, line
graphs, counts,
means, tabular
comparisons of
means,
descriptive text
Post-
workshop/webinar
cognitive
interview
with activity
setting participants
Attitudes
Beliefs
Practices (self-
reported)
Knowledge
1 week after
workshop
(Fall 2011)
1 week after
webinar (Winter
2012)
Categorical
summaries;
Summary tables
and text

75

Table 3.2, Continued

Individual
interviews with
activity setting
participants
Changes in
practices
(self-reported)
Environmental
factors
Social interactions
Norms
Discourse
Changes in
discourse
3 months following
workshop
(Winter 2012)
Descriptive text
and thematic
summaries

It was possible to observe these factors because the workshops and webinars
facilitated by CUE researchers provided practitioners at MSU with both the
opportunity and resources to express their reactions and experiences. Data were
collected and analyzed keeping in mind the theoretically informed expectation, to be
assessed through the developmental evaluation, that CUE action research processes
and tools have a positive impact on practitioner attitudes, beliefs, and as a result,
behaviors.
Document collection. Documents are a rich source of cultural and historical
data of institutions and programs (Patton, 2002; Stringer, 2007). Documents provide
the researcher with information that cannot be observed and can “reveal things that
have taken place before the evaluation began” (Patton, 2002, p. 293). Documents
allow for a researcher to get a better sense of processes and culture not readily evident
in traditional fieldwork and may provide information to guide future inquiry paths that
can be pursued later through direct observation and interviewing (Patton, 2002). The
document data collected for this study included institutional reports, website
information, grant proposal submitted by participants after CUE’s workshop, and pre-
post webinar syllabi.

76

Observations. Observations were an important feature of this study and were
used to record interactions of the study participants. The purpose of conducting
observations was to be able to describe the activity settings, how the “design
experiment” unfolded, as well as the social interactions involved in using CUE’s tools.
Observational data provides the researcher with a more comprehensive understanding
of the interactions between study team members in a natural setting (Patton, 2002).
Observations were collected using the Observational Data Collection Template
protocol (see Appendix C) developed by CUE. In accordance with the protocol, the
observational notes included detailed descriptions of Site, Mood, and “Task”
Performance, Social Context, Environmental Constraints and reflections. Importantly,
observations regarding “Task” Performance focused on the ways to capture ways in
which joint productive activity in the form of social interaction, mutually negotiated
values and goals, and actions brought about learning and change in the individuals
involved (Dowd & Bensimon, 2009; Tharp & Gallimore, 1998). Patton (2002) notes
that observation allows the researcher to understand more fully the program of study
“to an extent not entirely possible using only the insights of others obtained through
interviews” (p. 23). Observations occurred at the October workshop and the first of
February webinars, where MSU faculty and administrators were engaged in inquiry
activities. Observations allowed the research team to take descriptive field notes to
observe patterns of behavior and culture through the analysis of verbal and non-verbal
cues.
Evaluation questionnaires. The evaluation questionnaires that were used
throughout the study were designed by CUE. Questionnaires are a way to capture a

77

respondent’s thoughts and feelings at a specific, static moment in time (Patton, 2002).
While the opinion of participants about their experiences in inquiry activities is
important, this dissertation study was also focused on studying more in depth the
patterns of change in thoughts and beliefs over a period of time, therefore
questionnaires also served as the reference point for the cognitive interviews that were
conducted.
At the end of the October workshop and of February webinars, participants were
emailed an online evaluation questionnaire to complete (for an example, see Appendix
B). The questionnaires asked respondents to answer questions based on their reactions,
beliefs, and experiences during an activity setting, as well as intended behaviors
afterwards. Questionnaires also served as the reference point for the cognitive
interviews that were conducted after the workshop. According to CUE Evaluation
Questionnaire Reports, of the 25 participants who attended the October workshop, 19
completed the questionnaire (response rate 76%). Of the 20 participants of both
February webinars, 15 completed questionnaires (response rate 75%).
Cognitive interviews. The cognitive interview is a way to “study the manner in
which targeted audiences understand, mentally process and respond to materials we
present” (Willis, 2005, p. 3). CUE develops evaluative questionnaires to improve the
validity of their conclusions about their effectiveness and impact in carrying out action
research. This process is an example of conducting 3rd person research, as denoted in
Figure 3.1. The cognitive interviews for this study prompted participants to think
aloud as they answer evaluation questionnaire items. Traditional surveys where
respondents answer questions independently can produce response errors that occur

78

for any number of reasons including because questions are difficult to understand,
misinterpretation of scales, and respondents not recalling information presented to
them, just to name a few. Therefore, the cognitive interview is utilized to understand
how these errors can occur.
On a conceptual level, the data from cognitive interviews allows CUE
researchers to assess if its evaluation questionnaire is measuring what they want to
measure in the ways they wish to measure it. It is difficult to assess changes in
practitioner beliefs and analysis if survey data does not inform researchers whether all
responses accurately represent the learning outcomes the survey designers intended
(Ouimet et al., 2004; Willis, 2005). Cognitive interviews can also help with
identifying logistical problems with survey items such as wording and flow.
Using CUE developed Cognitive Interview Protocols (see Appendix D), I
interviewed members of the MSU team I had observed at activity settings. In this
respect, the cognitive interviews assisted me to assess the impact of CUE action
research projects. In addition, the data collected through the interviews provided base
for understanding of the respondents’ beliefs, attitudes, behavioral referent groups
(e.g. who else seems to matter when they talk about whether they are likely to take
some step or not or hold some belief or not), perceived norms, and practices.
It should be noted that cognitive interview process incorporated a method called
stimulated recall (SR). SR is a method of stimulating a participant’s cognition
surrounding a past activity, during a recall situation allowing for the verbalization of
cognitive and metacognitive processes that occurred during the activity setting and/or
are occurring as a result of reflection on the activity in question (Kagan, Krathwhol, &

79

Miller, 1963). SR is known to be useful in the investigation of conceptual change (e.g.,
changes in beliefs). It also leads to continued reflection surrounding an activity. For
the purposes of this study SR consisted of additional questions added to the Cognitive
Interview Protocol. The purpose was to activate recall of the cognitive and
metacognitive processes experienced by participants during the activity setting.
In a week or two after the October workshop I conducted four phone cognitive
interviews with the participants from the breakout session. After both February
webinars participants also completed evaluation questionnaires to deliver personal
feedbacks on the reflective practices using CUE’s Document (Syllabus) Analysis
Protocol. I conducted follow up cognitive interviews with two participants by phone.
Individual interviews. Another way of obtaining data for this study was
through individual interviews. Patton (2002) suggests that researchers interview
people to uncover those things that cannot be directly observed. Those things include
feelings, thoughts, attitudes, and intentions, all of which are central to answer this
study research questions. According to Stringer (2007), an interview is a reflective
process that “enables the interviewee to explore his or her experience in detail and to
reveal the many features of that experience that have an effect on the issue
investigated” (p. 67). The purpose of my interviewing, therefore, was to capture the
other person’s perspectives and experience associated with CUE’s tools and
participation in the study. The following overarching questions framed my interviews:
What do the tools and the study look and feel like to the participants? What are their
experiences in the study? What changes do participants perceive in themselves as a
result of their involvement in the project?

80

To accomplish my purpose, I used the standardized open-ended interviews
developed by CUE (see Appendix E). This type of interview consists of a set of
questions “carefully worded and arranged with the intention taking each respondent
through the same sequence and asking each respondent the same questions” (Patton,
2002, p. 342). On the one hand, standardized format increases comparability of
responses, reduces bias, permits evaluation users to see the instrumentation used, and
facilitates analysis of the data (Patton, 2002). On the other hand, open-ended questions
enable respondents to describe and interpret experiences, thoughts, and insights in
their own words (Stringer, 2007). Patton suggests using this type of interviews when
doing action research or conducting evaluation that is relevant to the present study. It
is reasonable, he believes, to interview participants once and for a short period of time,
such as a half hour, so highly focused questions, arranged in advance, serve to
establish priorities for an interview. One of the limitations of the standardized open-
ended interviews is little flexibility in relating the interview to particular individuals
and circumstances (Patton, 2002). To address this limitation, I included probing in my
interviews. Overall, I conducted three individual interviews with October workshop
participants in February 2012.
Data Analysis Procedures
Figures 2.1 and 3.2 illustrate the learning and change model informing CUE’s
action research methods. The theoretical frameworks described in Chapter 2, including
action research and CHAT, inform this study. Action inquiry calls attention to a cycle
of inquiry among individual practitioners, as well as the role of social interactions in
shaping opportunities for practitioner learning and experimentation with new

81

educational practices. Other inquiry studies often take a constructivist, interpretive
approach to research. These studies focus on meaning making, examining interactions,
and enriching our understanding of social situations. They are rich with descriptions
and provide an understanding of the social realities of individuals and social context
(Phillips, Bain, McNaught, Rice & Tripp, 2000).
Kruse (2001), for example, conducted several constructivist interpretive studies
that dealt with issues that teachers and administrators encounter in the K-12 education
system. Kruse and Louis’s (1997) study examined interdisciplinary teacher teaming to
investigate if teaming produces opportunities for a professional community. They
found that while teaming was beneficial for those members within a team, teams often
operated separately, and members formed close bonds with individuals on their teams
but often worked in isolation from other teams.
In another constructive interpretive study, mentioned earlier in this paper,
Stillman (2010) examined “factors that support equity-minded teachers to navigate
accountability-driven language arts reform, barriers that impede teachers from serving
marginalized students, particularly, English language learners, and how particular
environmental factors mediate teachers responses to accountability pressures” (p.
133). These studies have several commonalities that are reflective of constructivist,
interpretive paradigms. While constructivist-interpretive studies can help bring
organizational issues to the forefront, they tend not to address areas, which need
change (Phillips et al., 2000). Constructivist-interpretive studies do not provide
practitioners with direct opportunities for learning or the opportunity to experiment
with new educational practices.

82

This study differed from these constructive studies of inquiry. It was a mixed
methods developmental evaluation study. While some of the methods for conducting
research such as interviews and observations were the same, there were several
dissimilarities in the purposes and methods in this study compared to the studies
conducted by Kruse, Louis, and Stillman. The purpose of this study was to understand
an organizational learning and change process, as catalyzed through action research.
The methodologies used to employ these studies also differ. In traditional studies,
researchers often act as outsiders, studying the participants. In this case, CUE
researchers simultaneously served as researchers, while working collaboratively with
participants, providing tools developed by CUE to facilitate the action research
process. It should be noted that CUE modified their tools throughout the study; for
example, it used a modified version of the Document Analysis Protocol in the
webinar. The protocol was changed based feedback and needs of participants at the
previous workshop.
In traditional studies researchers generally study individuals, groups or
phenomenon. This study required that participants also served as researchers into their
own practices and work collaboratively with CUE facilitators to develop the tools
necessary for organizational learning, change, and to increase institutional
effectiveness. This process allowed for an inquiry process to occur, giving
practitioners the opportunity to take a deeper look at institutional practices and
policies that could be hindering institutional change. Therefore, the data analysis
focused on documenting what took place in the design experiment and understanding
the subsequent behaviors and changes in beliefs rather than focusing on the lived

83

experiences or construction of meaning in the activity settings, as would be studied in
an interpretive analysis.
CHAT calls attention to the role of social interaction and cultural artifacts in
shaping educational practices. It also emphasizes the influence that history has on
organizations. Using CHAT as a framework allows CUE researchers and practitioners
at the focal institution to examine how the institution’s culture may be impeding
organizational learning and change. Culture can often be invisible. Bess and Dee
(2008) assert that uncovering cultural assumptions require a joint effort between
insiders and outsiders, in this case – practitioners at Monarch State University and
CUE researchers.
Coding
Data analysis took place after each data collection procedure and again once
data had been collected from all sources to triangulate insights from the data. Codes
for thematic coding were selected based on prior CUE evaluation studies. Data
analysis involved the use of deductive themes that were pre-determined and included:
attitudes/beliefs, knowledge and social interactions in activity settings, including non
CUE related mediating artifacts, language, roles, rules/norms, community and division
of labor. Reflection, problem identification, experimentation/problem solving and
action experience were also used to analyze the impact of CUE’s tools.
Attitudes/Beliefs. As it was mentioned previously, Ajzen’s (1991) theory of
planned behavior guided the analysis of the relationships between attitudes/beliefs and
behavior observed in the data. Ajzen asserts that behaviors are highly influenced by
intentions, attitudes, beliefs, and a person’s perceived control in performing a given

84

behavior. Therefore, the stronger the intentions are, the stronger the possibility of
actually performing the intended behavior is. Behavior achievement depends on
individuals’ intentions or motivation as well as their belief that they have the ability
(behavioral control) to reach the desired goal. In addition to intention as a motivational
factor, Ajzen found that people are more likely act upon an intended behavior when
they have the necessary resources and opportunities to perform desired behaviors.
Beliefs are also a factor in achieving intended outcomes. Therefore, if
practitioners believe and have the expectation that they have the power to increase
equitable outcomes at their institutions, they will. “Self efficacy activities can
influence choice of activities, preparation for an activity, effort expended during
performance, as well as thought patterns and emotions” (Bandura, 1982, 1991, as cited
in Ajzen, 1991). Ajzen’s (1991) posits that beliefs influence attitudes toward certain
behaviors. Attitudes, in turn, impact behavior in both positive and negative ways
(Ajzen, 1991). Thus, intentions, attitudes, perceptions of behavioral control, and
beliefs serve as predictors of behavior (Ajzen, 1991). Another factor that predicts
behaviors is subjective norms, which is social pressure to perform or not perform the
behavior. According to the theory of planned behavior, normative beliefs are
influenced by influential others in an individual environment and their approval or
disapproval of a specific behavior. The theory claims, “ … behavior is a function of
salient information, and beliefs” (Ajzen, 1991, p. 32).
In this case study, code categories such as attitudes/beliefs were used to analyze
whether CUE’s tools have an impact on practitioner attitudes/beliefs, behaviors, and
practices in ways expected to positively impact equity in racial-ethnic outcomes. From

85

this framework, attitudes are developed from the beliefs individuals hold about an
object or stimuli. Together attitudes and beliefs influence behaviors and outcomes
(Ajzen, 1991). Data regarding attitudes/beliefs were collected through observing
activity settings, through the verbal and non-verbal language used by during activity
settings, from evaluation questionnaires as well as cognitive and individual interviews.
Knowledge. One of the tenets of CUE’s work is to assist practitioners in
developing the knowledge needed to increase equity for students of colors. Even when
practitioners want to increase the outcomes of students of color, it is possible they lack
the knowledge necessary to increase outcomes for minoritized groups. Knowledge is
constructed through collaborative and “productive activities” (Bensimon & Dowd,
2009). The tools developed by CUE may assist practitioners at Monarch in the
facilitation of developing new knowledge. Therefore, data collected noted
“knowledge” statements. One example of such statement is: “We already know all of
this (low success/transfer) ... we’ve known this for three years.”
Social interaction. Social interactions refer to how people participate or choose
not to participate and how they interact with others. Social interaction occurs at three
levels: the personal plane, interpersonal, and community/ institutional (Nasir & Hand,
2006, as cited in Rogoff, 1995). “The personal plane includes individual cognition,
emotion, behaviors, values and beliefs. The interpersonal plane involves
communication, role performances, dialogue, cooperation, conflict, assistance, and
interaction. The third plane, community/institutional involves having a shared history,
language, rules, values, beliefs and identities” (Nasir & Hand, 2006). All of these
concepts were included to analyze social interactions.

86

Reflection. Along with social interaction, reflection allows for examination and
reconstruction of existing knowledge and beliefs (Bensimon & Dowd, 2009).
Reflection is a thought process that occurs when practitioners examine their practices.
Donald Schon (1983) describes reflection as a process, which involves looking into
one’s own experiences, connecting with one’s own feelings, and attending to one’s
own theories in use. It entails building new understandings to inform one’s actions in
the situation that is unfolding.
The practitioner allows himself to experience surprise, puzzlement, or confusion
in a situation, which he finds uncertain or unique. He reflects on the
phenomenon before him, and on the prior understandings, which have been
implicit in his behavior. He carries out an experiment, which serves to generate
both a new understanding of the phenomenon and a change in the situation
(Schon, 1983, p. 68).
Furthermore, reflection allows for revealing inner contradictions, which express
themselves as trouble in ongoing activity. When such trouble is available to
consciousness, it can be addressed by a change of goals (Roth & Lee, 2007). Roth and
Lee suggest that when inner contradictions become conscious during reflection, they
become the primary driving forces that bring about change and development. The
questioning, “What are we doing wrong in the classroom to help these students?” can
serve as an illustration of reflection.
Problem identification. Problem identification occurs when practitioners
identify an issue that needs to be addressed at their institution. It is “seeing” the

87

problem before trying to solve it. In other words, it is the first step in solving a
problem.
Experimentation/Problem solving. With openness to various perspectives and
problem framing through data analysis, social interactions bring about new capacities
for experimentation and problem solving (Dowd & Bensimon, 2009). Experimentation
opens possibilities for new ways of seeing and acting. Experimentation/problem
solving occurs when practitioners begin developing solutions to address current issues
at their institution.
Action/Experience. Experience is acquired through action, specifically new
actions brought about through the inquiry process. Action takes place when
participants can relate their own experiences directly or indirectly to the problem at
hand. The statement “It’s really time to have a cautious conversation...we’re missing
those conversations” illustrates this type of coded data.
The Data Analysis and Summary Sheet (Table 3.4) was used to analyze the data
collected through the observations and interviews with participants. It was divided into
two columns. The first column included code categories: attitudes/beliefs, knowledge,
social interactions in activity settings, mediating artifacts, language, roles,
rules/norms, community and division of labor, reflection, problem identification,
experimentation/problem solving and action experience. The second column was
divided into four sections. The first section guided a quantitative analysis of the data.
It called for a numerical summary of the frequency of the use of each code in order to
provide evidence from which to draw conclusions from this study about the typical
areas impacted by CUE’s tools.

88

The second section characterized action of the data to summarize the results of
the analysis in regard to the hypothesis that CUE’s tools remediate practice in ways
that bring about “equity mindedness” and practices believed to promote equitable
outcomes for racial-ethnic groups. In sections two, three and four the codes were
summarized categorically and qualitatively described. Section two provided evidence
of the impact of the construct codes interpreted as weak or strong based on the
frequency of the use of the code. Section three provided space to enter excerpts from
the data of the analysis to demonstrate the typical meaning captured by the code in
use. Section four described language that illustrates the variation/range of meaning
captured by the code.
In the last row of the data analysis codes and summary sheet is a space for an
analytic memo. An analytic memo summarized the results revealed by the coded data,
such as emergent themes, tensions and unresolved issues. The analytic memo served
as an ongoing document to capture and analyze the results of each code throughout the
data collection process. It was used not only for analysis, but also to inform the
following steps.
Table 3.3. Data Analysis Codes and Summary Sheet for Analyzing CUE’s
Action Research Processes and Tools
Attitudes/Beliefs (A/B)
CODE CATEGORIES

Attitudes/ Beliefs (A/B)

Knowledge (K)

Social Interaction (SI) in
Activity Settings
(1) Frequency, Tally # = the number of
times you have used this code in coding the
data from this particular activity setting
______

(2) Characterize the data…

Based on evidence of impact [weak/strong,

89

Table 3.3, Continued

(non CUE) mediating artifacts,
language,
roles, rules/norms, community,
and
division of labor

Reflection

Problem Identification

Experimentation/
Problem Solving
(includes perceived behavioral
control; environmental factors)

Action(Inaction)/
Experience
(includes perceived skills, ability,
self-efficacy)

Other (not coded above)

based on frequency of code count relative to
total code count for this activity setting]

(3) Data Excerpts: Typical
Quotes that illustrate the typical meaning of
the data coded in this category (include
reference #s with data note page and line
number, e.g. 10.5 means page 10, line 5)
(paste excerpts and let pages flow, retaining
the page breaks)

Insert data excerpts here

(4) Data Excerpts: Variation/Range
Quotes that illustrate the variation and
range of meaning of the data coded in this
category (include reference #s with data
note page and line number)
(paste excerpts and let pages flow, retaining
the page breaks)
Insert excerpts here
Analytical Memo

Evaluation Questionnaire Data
Evaluation questionnaires provided basic descriptive statistics characterizing
(non-identifiable, pooled, site- and activity-specific) respondent experiences and
impact (strength, direction, mode, range). In general, the types of questions included in
the questionnaire can be divided into three subgroups: questions related to the quality
and clarity of information presented during the workshop as well as to the presenter’s
expertise; questions concerning participant’s experience around the issues of equity

90

prior the workshop; and questions dealing with the participant’s intentions to address
the issues of equity after the workshop.
The Action Research Process, Activities, and Design
Community and Planning Meeting: Laying the Groundwork
The initial 4-hour planning meeting (late February 2011) was organized on
MSU campus by the co-director of CUE (leading this study) in collaboration with two
MSU co-facilitators of the project. It was in this meeting where my first exposure to
the project occurred. I participated in the event as a participant-observer and did not
collect observational and interview data. Overall, 18 individuals from Monarch
University participated. The composition of the invited individuals included deans,
faculty, student services personnel, and college of education graduate students.
The meeting’s primary goal was to plan the subsequent action inquiry project in
collaboration with CUE. The MSU co-facilitators described it as “an open mapping
process.” Co-director of CUE briefly introduced CUE’s tools and made packages
available so people got a sense of what was available. The foundational questions
framing the meeting were the following: How can this process support MSU
organizational learning goals? What data are available to support inquiry? What types
of action inquiry activities are MSU people interested in participating in (CUE tools
are available to guide action inquiry)? What current planning, inquiry, faculty
development activities, assessment or accountability reporting requirements provide a
foundation or entry point into this project? My review of the audit trail indicates there
were three emergent areas of interest as outcomes of the meeting: robust systems of

91

support and resources for students of color; creating welcoming culture for students of
color; and tracking pathways of students of color.
In two weeks after the planning meeting there was a planning call between co-
director of CUE and one of the MSU facilitators. It was decided to hold a workshop in
April. It was also suggested to involve an Associate Vice President for Inclusive
Excellence in the project and to house our joint effort within the Inclusive Excellence,
in order to “institutionalize what we were doing.” Ultimately, this idea was only
partially realized; the Inclusive Excellence Office provided some support such as
lunch for the April workshop participants, but the effort did not become known as
officially established as Inclusive Excellence.
Data Analysis using BESST Workshop: Defining the Problem
The late April workshop, entitled Using Data to Set Equity Benchmark Goals,
involved participants in examining equity gaps in student enrollment, persistence, and
degree completion at the Monarch State University through the use of the BESST. The
intent of the meeting was to enable cross-institutional groups to self-organize, plan and
enact, and assess changes needed for the purpose of greater equity for Latino/a and
African American students on Monarch campus. Facilitated by the co-director of
CUE, the event attracted a group of 15 MSU practitioners (some of whom were from
the previous planning meeting and some newcomers). It is worthwhile to note that
prior to the workshop, all its participants received, by email, web-based links to
animated PowerPoint presentations produced by CUE to explain key terms that
function as artifacts to remediate practitioner language from a “student deficit”
orientation to an “equity-minded” orientation. The content of the modules provided

92

explanations of the concepts of equity, equity-mindedness, inquiry, and disaggregated
data, as typically articulated in CUE’s action research. However, no data were
collected to determine participants’ use of these modules. The BESST provided a
snapshot of MSU’s enrollment, persistence and degree completion trends for white,
Latino, African American, and all students (cohorts 2005, 2010). The tool allowed
practitioners to see MSU data in a graphic form and to interact with the data findings
to set short- and long-term equity goals. Figure 4.2 illustrates one of the components
of the tool that shows how many students applied to MSU in 2010 (Starting Cohort),
how many students were admitted, and how many actually matriculated.
Figure 3.4. BESST Baseline Data: All MSU Students Who Applied, Were
Admitted, and Matriculated in Fall 2010 (Simulated Data)

After describing the functions of the tool and reviewing what story the data told
about equity at the MSU, CUE co-director, acting as the facilitator, stimulated a
dialogue among the participants by presenting several evaluative questions: “What if
Latinos were selected for admission in Fall 2010 at the same rate as the group with the

93

highest admission rate, how many more Latinos would have enrolled?” or “How many
more African American students would we need to bring to reach equity goals?”
During discussion CUE reframed the focus towards the content and quality of
institutional practices that might produce inequities. Recognizing problems, tensions
in practice, and recognizing that it is in practitioners’ hands to solve those problems as
well as questions, “What do we do wrong to enroll, retain or graduate more Latino/a
and African-American students?” - were the ways of reframing a problem of
institutional practices that were producing inequities. The tool served as a mediational
means for informed thinking about equity. Not only did it allow for analysis of data
disaggregated by race and ethnicity but it also stimulated the process of social
interaction, problem framing, and solution generation from the perspective of
institutional responsibility. At the end of the meeting, the practitioners were involved
in a discussion on what they would need to know and to do to increase equity
outcomes for students from minority backgrounds, which actions are within their
grasp, which actions are within their reach – with a stretch, and which actions are
beyond their reach (require collaboration, involvement or institutional leaders, or state
policy changes, etc.). These BESST data was not used again and no clear actions
emerged from planning.
Document (Syllabus) Analysis Workshop: Self-Assessment Using Inquiry Tool
The next event, the Document (Syllabus) Analysis workshop, entitled Fostering
a Culture of Equity and Diversity at the Monarch State University, served as the main
“activity setting” of my study. It was organized off campus and involved 25 MSU
practitioners – high-level administrators, student support services professionals, and

94

faculty members. Some of them had attended previous meetings and were familiar
with CUE people and tools; some of them came for the first time. The workshop had
its primary goal to initiate self-assessment in the form of inquiry through the
Document (Syllabus) Analysis Protocol.
The workshop agenda involved: the “Storyboard Tour” with the discussion of
what took place prior to the workshop; “Framing Concepts” part, including interactive
activity “Who helped you through college,” presentation about cognitive frames, and
activity using the comic strip “I’ve never benefited from racism;” four breakout
sessions; and the final part where participants of breakout sessions reported back to the
whole group and discussed the next steps. The “Storyboard Tour” opened the
workshop. The MSU co-facilitators began the conversation with the story about the
origin of the project. They also connected the workshop’s goals to the institution’s
mission and the President’s Vision. Co-director of CUE continued the “Storyboard
Tour” by the explanation of the poster “Stuck in a Shallow End” introducing the
book’s metaphor as a metaphor for inequities in education. The poster promoted the
development of new views on the reasons why minority students are underrepresented
in STEM disciplines.
The following interactive part, “Framing Concepts,” involved facilitation of
social interaction, self-reflection, and discussion among participants using a number of
different CUE-developed artifacts. One of the artifacts, a presentation slide with the
homonymous interactive exercise “Who helped you through college?” was introduced
first. People participated in the teams grouped by tables and then reported to the whole
group. They were asked to discuss what struck them and what differences and

95

similarities they had. The introduction of the notion of cognitive frames and how they
unconsciously drive peoples’ actions continued the activity. In a presentation a
facilitator intensively employed such symbolic artifact as CUE’s conceptual language
(e.g., “mental map of attitudes and beliefs,” “equity-minded knowledge,” “deficit-
minded dominant cognitive frame” etc.) and reframed discussion towards color
consciousness and institutional responsibility for underrepresented students’ success.
Facilitation of reflection using another tool, the cartoon “I’ve never benefited
from racism,” concluded the first part of the main session. The cartoon depicted
historic process of discrimination and structural racism, “differential access to the
goods, services, and opportunities of society by race” (Jones, 2000, p. 1212) with the
premise that generation after generation those inequities built up into deficit views that
still exist today, including education. The idea behind the activity was to highlight
how certain statements and what students see or feel can shape the way they
experience a campus community. The cartoon appeared on the screen and people also
had it in their packages. After one of the participants had read it aloud, people were
invited to provide their comments, thoughts, and reactions.
After a short lunch there was a transition to breakout groups where participants
were involved in a process of inquiry into own practices, using the Document
(Syllabus) Analysis Protocol for Culturally Inclusive Practices. The group I observed
(one of the four) consisted of five faculty members from various departments. Each
brought a syllabus for review. They were asked to assess a sampling of their own
documents of educational practice (syllabi, handouts from class, assignments, or
assessments of student learning), using nineteen indicators of culturally inclusive

96

practices prior to the workshop, and to bring their notes. The indicators, derived by
CUE from the relevant literature, included: (a) Respect for students; (b) Desire to help
students succeed; (c) Information students need to successfully act on the information
in the document; (d) Validation of racial diversity; (e) Expectation for active
involvement with diverse communities, real world problems, and social responsibility;
and others (see Appendix E). The response columns alongside each of the indicators
asked whether the culturally inclusive practice represented by the indicator is reflected
in the document, whether it should be (if not already), and whether the person,
completing the review, would be willing to take steps to change the document if she or
he believes changes are warranted.
From the beginning, the facilitator initiated social interaction by asking
participants to reflect on and to share with the group their experiences from
conducting a preliminary document review. Drawing on those initial reflections, the
facilitator suggested to collectively generate ideas and examples of how to incorporate
indicators into their syllabi. Building on their past experiences, participants generated
and shared positive examples or solutions how to strengthen culturally inclusive
practices. At the end of the session participants were suggested to brainstorm action
steps that can be taken to improve equity at the MSU. After the breakout sessions, all
participants gathered together to build on what was learned and what the next steps
would be.
Document (Syllabus) Analysis & Assessing Students Reactions Webinars
Two other activity settings were created in February 2012. They took the form
of webinars/seminars and were conducted on MSU campus. The first webinar,

97

Reflective Practice Using CUE’s Syllabus Analysis Protocol, involved six faculty
members who were not able to attend the October workshop. They were introduced to
methods of inquiry underlying CUE’s Syllabus Analysis Protocol (slightly modified
since the October workshop), the principles of culturally inclusive practices, and a
worked example – a sample of a culturally relevant syllabus. After the presentation
participants also reflected on their own syllabi and had the opportunity to discuss their
insights and questions with their colleagues. Participants in the second webinar,
Assessing Student Reactions to Culturally Inclusive Practices, 14 MSU faculty and
staff members were introduced to characteristics of culturally responsive pedagogy,
the concept of micro-aggressions, and engaged in a discussion of the strategies to
assess students’ reactions to practices adopted in an effort to be culturally inclusive.
Data Reporting
Data was reported from observations, evaluation questionnaires, cognitive and
individual interviews, as shown in Table 3.3. Observational data reports included
descriptions of the activity settings and of the CUE tools. Activity settings took the
form of CUE led meetings, inquiry team meetings, and breakout team meetings. CUE
tools included the BESST and the Document Analysis for Self-Assessment of
Culturally Inclusive Practices tool (two versions). Descriptive text, tabular summaries
based on categorical data were used to conduct deductive and thematic analysis. Table
3.5 illustrates different code categories and examples of data that the codes represent.

98

Table 3.4. Deductive Data Analysis Codes
Code Category Examples
Attitudes/ Beliefs
(A/B)

“I don’t give any syllabus at the beginning. First we
critique a society we live in, I ask questions about
their education ... And I think it’s important because
when you allow students and you trust them, they take
you in places, which you couldn’t ever imagine.”
Control Beliefs/Perceived
Behavioral Control
(C/B)/(PBC)

“I think on the interactional level I can do a lot and I
can also do much on the interactional level in my
classroom with the kind of material that I teach as well
advocating what I do in a larger university
environment.”
Perceived Norms
(PN)

“So you have a President’s vision, for me, after
experience I’ve had, it’s just a lip service, this is what
they need to say for public consumption …”
Knowledge
(K)

“I am always open for changes but if I would know
what those possible steps might be I would have done
them.”
Social Interaction
(SI)

Rachel shares that she chose the syllabi of colleagues
at another school: “I started by looking at syllabi that
are more interesting than my own so I could learn a lot
by doing that.”
Reflection
(R)

“I am just thinking we are sitting here and saying 'I
would never say something like that.' But when I
actually went through that exercise, I saw that I am
not very inclusive, mostly have 'nos,' I don't say those
things but I do a lot of other stuff that is more subtle.”
Problem Identification
(PI)

“Some of our policies are almost arcane so if students
don’t get help, they are not likely to understand that
there are consequences for not acting in a timely
way.”
Experimentation/Problem
Solving
(E/PS)
“The solution is that every faculty includes culturally
responsive pedagogy into their practices.”

99

Table 3.4, Continued

Intention
(I)
“This type of a dialog I will be trying to achieve in a
classroom in our discussions.”
Roles & Division of Labor
(R&DL)

Not found.
Cultural Artifacts
(CA)

“The comic strip was very clear and the people who
already knew could refer to it and the people who
didn’t know, they were learning.”
Language
(L)
“I am not very inclusive.”
Rules & Norms
(R&N)
“Many of those issues that concern students of color
are oftentimes silenced not only through the classroom
but also through the university events and through the
general administration.”
Community
(C)
“I have to go beyond my own department or even
school in order to have more comfortable
conversations about race and ethnicity ...”

Standards of Review
Credibility
The credibility of a research process, defined as “the plausibility and integrity of
the study,” is a fundamental issue in action research (Stringer, 2007, p. 57). Credibility
of action research is based on the standard of acceptance of the results of the study by
users in the setting and measured by their willingness to act on those results, “thereby
risking their welfare on the ‘validity’ of their ideas and the degree to which the
outcomes meet their expectations” (Greenwood & Levin, 2005, p. 54). It means that
knowledge co-generated by researchers and local stakeholders is considered as
credible and valid if it gives rise to actions on changes in practices or policies
(Greenwood & Levin, 2005).

100

The users of the present study are twofold: CUE and practitioners in the field
setting. As mentioned previously, the study informs CUE for development of
evaluation questionnaire and for enhancement of its efficacy in conducting action
research. It also informs MSU practitioners and practitioners at different institutions of
higher education for creation of self-assessment activities using action inquiry to
initiate the change process for more equitable outcomes.
The potential audiences who may search for credibility of the present study
include action researchers in general and CUE action researchers in particular, higher
education assessment professionals at similar and different institutions, as well as
higher education as a field. To meet the expectations of those audiences, the study
established credibility by way of four strategies that are proposed by Stringer (2007):
data triangulation, referential adequacy, researcher triangulation and debriefing.
First, the credibility of the study was enhanced using data triangulation
incorporating a range of sources of information available over time. The observations
during the workshops were a primary source of data. This study also relied on
evaluation questionnaires, cognitive interviews, document analysis, and data from
individual interviews. The inclusion of perspectives from different sources “enables
the inquirer to clarify meaning by identifying different ways the phenomena are being
perceived” (Stringer, 2007, p. 58).
Referential adequacy was another technique to amplify credibility of the study.
Referential adequacy, proposed by Lincoln and Guba (1986), refers to “an activity that
makes possible checking preliminary findings and interpretations against archived
“raw data” (p. 301). In regard to referential adequacy Stringer (2007) states that in

101

action research concepts and ideas within the study should reflect the experiences and
perspectives of participating parties and results should be drawn on their terminology
and language. To ensure that the study reflects the perspectives of participants, I
identified a portion of data to be archived (not analyzed) and conducted analysis on
the remaining data to develop preliminary findings. I then returned to the archived
data and analyzed it as a way to test the validity of my findings.
Next, my peers and CUE researchers, who served as facilitators, were additional
sources of insight in making sense of potential inconsistencies in data. I utilized
researcher triangulation - discussions of diverse interpretations, concrete examples of
actions, behavioral and attitudinal changes, and perspectives in a larger research team
- to establish credibility. In addition, debriefing with focus on peer/researcher feelings
and reactions was used. Thus, peers and researchers questioned, challenged, and
supported my interpretations of the data. Finally, the research team shared results with
inquiry team participants, welcoming their feedback and reactions.
Transferability
In general, action inquiry outcomes are applicable only to the particular groups
and contexts that were part of the study (Stringer, 2007). However, Stringer argues,
that it is not to say that nothing can be generalized to others. In order to enhance
transferability of research, he suggests reporting “detailed descriptions of the context,
activities, and events” (p. 59) as part of the research outcomes. Greenwood and Levin
(2005) frame transferability in action research as “necessitating a process of reflective
action rather as being based on structures of rule-based interpretations” (p. 55). They
suggest that the key to transferring context bound knowledge to a different setting is to

102

understand contextual conditions under which knowledge was created and those of a
new setting.
Given both perspectives, thick descriptions of the action research processes and
tools, setting itself, and social interactions provided in the present study, allow its
audiences mentioned above to infer relevance for their own practice settings. For
example, the outcomes of this study may be transferable to other public universities
that are interested in improving their instructional and administrative practices to
better serve students from racial-ethnic groups. Multiple sources of data also enhance
transferability. It is important to understand, however, that motivational intent of
practitioners to adopt knowledge generated in the study will be shaped by their beliefs
about the legitimacy of action inquiry activities and their perceptions of self-efficacy
and collective efficacy to carry out similar projects.
Dependability and Confirmability
According to Stringer, “dependability focuses on the extent to which people can
trust that all measures required of a systematic research process have been followed”
(p. 59). Dependability of the study is strong because detailed descriptions of all
research procedures are presented. That provides the basis for judging the extent to
which they are dependable. Confirmability, or “ability to confirm that the procedures
described actually took place” (Stringer, 2007, p. 59), was established through an audit
trail. It included the data collected, field notes, instruments, tools, and other artifacts
related to the study. As Kane and colleagues (2002) assert,
An audit trail provides the reader with evidence of trustworthiness in that she or
he can start with the raw data and continue along the trail to determine for her-

103

or himself if, in fact, the trail leads to the outcomes claimed by the researcher
(p. 199).
The items above and the analysis of the inquiry project provide an audit trail that
makes easily accessible for systematic review the evidence to ensure that the study is
trustworthy.
Limitations
Stringer (2007) notes, “Human inquiry, like any other human activity, is both,
complex and always incomplete” (p. 179). The study may have several limitations
arising from realities of investigation. First, participants selected themselves to engage
in the study. This self-selection may cause a biased sample. Additionally, observations
conducted can be characterized as truncated (limited to “activity settings”). Further,
the evaluation questionnaire and interview data used in the study represents self-
reported beliefs, attitudes, and practices, which may be inaccurate due to social
desirability bias. Respondents could answer questions in a manner that was viewed
favorably by their colleagues and CUE researchers. Therefore, impact of CUE’s
processes and tools may be fade or be isolated to “activity settings.”
Direct observations of the practitioners and their practices in the context of their
student services offices or their classrooms would provide richer data for triangulation.
In addition, data was not collected from students, whose experiences also could have
suggested evidence about real changes in the practitioner’s practice. Also, the short
duration of the study was another limitation because the inquiry cycle was not
complete. Finally, results of the study do not point out any one individual respondent’s
race or ethnicity because the number of participants in my individual sample was too

104

small and there was a risk of revealing someone. This created a limitation in that the
results do not speak to “racialized” differences in experiences and views and do not
allow to explore them. The study endeavored to maintain confidentiality and placed
priority on not harming someone by making their views known, especially given that
under developmental evaluation the expectation was to share the results with the field
site partners.

105

CHAPTER 4: RESULTS
This study maintains that racial-ethnic equity is missing from external and
internal accountability structures and institutional assessment as currently practiced is
not designed in ways that promote higher education practitioners’ understanding of
how to create more equitable educational outcomes for all students. Although
generally underutilized, practitioners’ self-assessment in the form of inquiry represents
an alternative and promising strategy to improve racial-ethnic equity in student
experiences and outcomes. The process of inquiry involves examination and
reconstruction of practitioners’ knowledge and beliefs about their own pedagogical
practices through social interaction and reflection (Dowd & Bensimon, 2009). The
events and inquiry activities, brought about by the project under study, facilitated
participants’ interactions with peers, CUE’s representatives and equity-oriented tools
in a design experiment (undertaken at one field site as part of a larger developmental
self-evaluation study conducted by CUE). This entailed participants’ reflection on the
ways they use (or could use) culturally inclusive practices to assist underrepresented
students in succeeding at their institution.
The study sought to answer the following research questions:
1. What beliefs do practitioners hold about racial-ethnic equity for the purposes of
equity-oriented organizational change?
2. Whether and how do social interactions (roles, rules/norms, communities, division
of labor, power relationships) mediate attitudes, beliefs, and behaviors related to
racial-ethnic inequities?

106

3. What influence do equity-oriented cultural artifacts designed for use in
“remediated” social learning environments have on postsecondary educators’
beliefs, attitudes, and behaviors in regard to equity in postsecondary education?
The results of the study provide evidence that participants at MSU were engaged
in social interactions through activity settings created by CUE facilitators using CUE’s
cultural artifacts (language, media, tools). Drawing on the conceptual framework of
action research, it was found that participation in social interactions, facilitated by
structured inquiry activities, was instrumental to the process of reflection among
participants, promoted problem identification, problem solving and, as a result,
practitioners’ learning from each other about teaching diverse students. Next, building
on CHAT, it was revealed that CUE’s equity-oriented cultural artifacts (language,
media, tools) mediated the way participants viewed their syllabi and pedagogical
practices. Participants’ attitudes and beliefs were challenged and influenced towards
color-consciousness and cultural inclusivity. Overall, participation in inquiry using
CUE’s tools influenced participants’ attitudes and willingness towards behavioral
changes to better serve their racially and ethnically diverse students. However,
consistent with Fishbein and Ajzen’s (2010) reasoned action approach, it came to light
that reported lack of opportunity for a dialog about equity with colleagues, a lack of
support from the university leadership (as perceived by the participants in my sample),
and participants’ insufficient knowledge of culturally inclusive pedagogy affected the
degree of a net impact on practitioners’ behavior.
In this chapter, I first present the context for the inquiry activities and equity-
oriented tools exerted at MSU employing Cultural Historical Activity Theory and

107

action research theoretical frameworks. This description will provide a foundation of
where and how the thematic findings arose. Next, I provide participants’ reactions to
the activities and tools derived from the evaluation questionnaire data. Thereupon, I
present results summarized by the thematic findings that arose from the data analysis:
multicultural diversity perspective in understanding equity; sense of isolation among
colleagues; perceived lack of administrative support; moving towards color-
consciousness; learning through social interaction and reflection; and promoting
institutional change through self-assessment.
I clustered the thematic findings based on their relevance to either MSU
institutional culture or to CUE’s activities and tools. The first three thematic findings,
related to MSU culture, provide evidence that racial-ethnic equity appears to be a
value for MSU participants and they are willing to take responsibility for making their
practices more culturally relevant. However, their thinking about racial-ethnic equity
as a multicultural diversity prevails over understanding it as a goal of achieving equal
outcomes. Sense of isolation among colleagues and a perceived lack of support from
senior administration are factors affecting participants’ behaviors related to racial-
ethnic equity. The last three themes, associated with CUE’s activities and tools,
explain in what ways and to what degree participants’ attitudes and behaviors were
mediated towards equity and cultural inclusivity as a result of involvement with CUE.
I conclude this chapter by providing summary of results and findings.
Project Activities Analyzed Through CHAT and Action Research
As shown in Figure 3.3, the project facilitated participants’ inquiry using CUE’s
tools through a series of activities: a community and planning meeting; Analyzing

108

Data Using BESST workshop; Document (Syllabus) Analysis workshop; Document
(Syllabus) Analysis webinar/seminar; and Assessing Students Reactions to Culturally
Inclusive Practices. The Document (Syllabus) Analysis workshop and the Document
(Syllabus) Analysis webinar served as primary activity settings for data collection. At
those activity settings participants were engaged in social interaction and reflection
initiated by a number of CUE’s equity-oriented tools. Examples include equity-
oriented language, the poster “Stuck in a Shallow End,” the cartoon “I’ve never
benefited from racism” and others presented further. The process of inquiry involved
another tool – the Document Analysis Protocol. Based on the Activity Systems Model
employed by CHAT, Figure 4.1 provides a diagram to illustrate, from the theoretical
perspective, what took place at the activity settings.
Figure 4.1. Activity Systems Model as Applied to the Document Analysis
Workshop

In this model, the subject is represented by practitioners, who participated in
the activity setting (a workshop). The object that is always “the purpose behind

109

activity” (Ogawa, Crain, Loomis et al., 2008, p. 85) was to enable practitioners to
understand the importance and components of culturally inclusive practices, thus
leading them to remediate their documents (syllabi) through a change in their tone and
content. Relationships between subjects and the object were enabled through the use
of tools or mediating artifacts, represented in both forms, symbolic and material.
Symbolic artifacts included nonmaterial tools such as CUE language, including terms
such as “prioritizing equity,” “culturally inclusive practices,” “structural racism,”
“equity-minded knowledge,” “inquiry,” “action research” and others.
Round tables located in a big U-shape, colorful packages with CUE’s equity-
minded symbolic and mission statement, “Storyboard Tour” posters describing main
milestones of the project in a metaphorical way and most importantly the Document
Analysis Protocol constituted material artifacts of the event. These materials were
intended to mediate social interactions and cognitive processes that occurred at the
activity setting. For example, the arrangement of tables and chairs allowed participants
across all tables to see each other, thus supporting facilitation of social interactions
promoting collaboration among participants. When new participants were invited to
place their names on the “Storyboard Tour” posters, they symbolically joined the
project community in the following way. Disclosing the timeline of our progress
leading up to the workshop, the CUE co-director used various storyboards to illustrate
how and when various participants became involved in the on-going project. This
allowed everyone to catch up with what had already been done and what direction we
were going to. Another example of material artifacts, the Document Analysis Protocol,
allowed for reflection using indicators of culturally inclusive practices. Results from

110

this study, presented further in this chapter, provide evidence that this led practitioners
to remediate the way they viewed their previous syllabi.
The people participating in the workshop represented community. It should be
noted, however, that the element of community operated in two dimensions at the
setting. One dimension represented community of the workshop. Another dimension,
the wider university community with its rules, norms, and traditions still operating in
people’s minds, was also represented at the workshop. CUE’s major effort of
remediation occurred in the top segment of the model, enabling relationships between
subjects, object, and artifacts.
As presented in Chapter 3, during the October workshop there were several
activities performed. The “Storyboard Tour” opened the workshop. The MSU co-
facilitators began the conversation with the story about the origin of the project. Co-
director of CUE continued the “Storyboard Tour” by the explanation of the poster
“Stuck in a Shallow End” that served as an artifact that promoted the development of
new views on the reasons why minority students are underrepresented in STEM
disciplines. Its function was strengthened by the reflection of African American senior
administrator to the community of the workshop participants. He shared his personal
educational experiences affected by racism when he was on the swim team at his
college. He told that they competed with plenty of other schools and only a few let his
team use showers or dorms at their schools, “I remember the labels with the ‘no swim
for Blacks,’ I recall that experience.” Also, he told about the role of his coach in his
swimming career that helped him to get a scholarship along with “ambition and
drive.” In response one of the participants expressed a belief that “just putting

111

computers in the classrooms doesn’t mean access to quality education and computer
science. In science there are assumptions who can do computer science … so
resources are not enough to solve the problem, there should be cultural intervention.”
Another activity involved the introduction of the notion of cognitive frames and
how they unconsciously drive peoples’ actions. In a presentation a facilitator
intensively employed such symbolic artifact as CUE’s conceptual language (e.g.
“mental map of attitudes and beliefs,” “equity-minded knowledge,” “deficit-minded
dominant cognitive frame” etc.) and reframed discussion towards color consciousness
and institutional responsibility for underrepresented students’ success. Facilitation of
reflection using another tool, the cartoon “I’ve never benefited from racism,”
concluded the first part of the main session. The idea behind the activity was to
highlight how certain statements and what students see or feel can shape the way they
experience a campus community. After a short lunch there was a transition to breakout
groups.
The group I observed (one of the four) consisted of five faculty members from
various departments, each of whom brought a syllabus for review. They were asked to
assess a sampling of their own documents of educational practice (syllabi, handouts
from class, assignments, or assessments of student learning) prior to the workshop,
and to bring their notes. At the workshop participants were involved in a process of
inquiry using the Document Analysis Protocol for Culturally Inclusive Practices.
Essentially, they examined the syllabi and reflected on the ways they use or could use
culturally inclusive practices to assist underrepresented students succeed.

112

From the beginning, the CUE facilitator initiated a dialog by asking participants
to reflect on and to share with the group their experiences from conducting a
preliminary document review. The initial reflection revealed that not everyone was
able to complete the assessment. Three of the five faculty members in my group
reported certain challenges associated with the understanding of particular indicators
as well as with their application. One shared, “I read through all the indicators but I
stopped because my responses were ‘No,’ ‘No,’ ‘No’ … and I didn’t entirely
understand what some of the things were getting at, for example, validation of racial-
ethnic diversity (indicator #4 in the Protocol). Another professor expressed the
concern, “I didn’t understand how and why I need to change something in my
syllabus.” The third asserted his openness for changes upon the condition he/she,
“would know what those possible steps might be,” “So I am here in the group to learn,
to look with ‘different eyes’ on my work,” he/she concluded. Those reflections might
signal participants’ insufficient understanding of underlying theoretical construct of
the Protocol derived from the principles of culturally responsive pedagogy. They also
might communicate certain weakness in the design of the tool itself as well as activity
around the tool. Participants were not completely ready to conduct self-assessment
prior to the workshop without being introduced to the notion, principles, and strategies
of culturally responsive teaching.
Drawing on those initial reflections, the facilitator suggested to collectively
generate ideas and examples of how to incorporate theoretical indicators into their
syllabi and teaching. People were divided into three pairs (one participant worked with
the facilitator) and went through several indicators. Building on their past experiences,

113

they generated and shared positive examples or solutions how to strengthen their
culturally inclusive practices. Observational data shows examples associated with the
“respect for students” indicator (#1 in the Protocol) that included the use of 1
st
person
language, making explicit the tasks and expectations, and having “a full outline of the
quarter with when things are due with the idea that instructor understands that the
students are busy people too.” Another perspective was humor “that respects
intelligence and assumes that the students are serious-minded. It is a high-minded stuff
that sends the student a message ‘you have a capacity to understand this’ and we’re
going to work hard together. This particular humor assumes that students are smart
and intelligent.” Other ways to show respect for students were “to ask them to write on
the topic they choose” and to let students “define a certain percentage of the class and
of the grade.” These examples show, however, that during discussion of the first
indicator participants’ focus shifted from culturally inclusive perspective towards
general pedagogical principles.
To draw participants’ attention to culturally inclusive teaching, the facilitator
suggested reviewing of “validation of racial-ethnic diversity” indicator (#4 in the
Protocol). Examples discussed in regard to this indicator included diversity in the
content of class, role models, quest speakers representing different racial-ethnic
groups, contextual references, the use of specific quotes such as “I respect the right of
others to express their opinion” in the syllabus and others. However, it was still left
not very clear how to validate racial-ethnic diversity in STEM disciplines, for
example, in electrical engineering classes. The session ended by reviewing one of the

114

last indicators from the Protocol: expectation for active involvement with diverse
communities, real world problems, and social responsibility.
While performing this activity, the practitioners per se went through several
phases of the Inquiry Cycle, articulating Knowledge, Beliefs, Reflection, Problem
Identification, and Problem Solving. Figure 4.2 graphically represents the aspects of
inquiry that were observed at the activity setting.
Figure 4.2. Aspects of Inquiry Articulated at the Document Analysis
Workshop

CUE initiated social interaction using the Protocol to catalyze conversation and
reflection about the ways of incorporating principles of culturally inclusive pedagogy
into the syllabi and practice. Observational data captured consistent and substantial
reflections that led participants to introspect and, benchmarking against the indicators
of the Protocol, to identify weaknesses and problems in their current practices and
syllabi. The tool appeared to be effective in bolstering participants’ reflections on the

115

ways to be more culturally inclusive. As one of the participants noted, “I thought I had
a good syllabus but then after reading this (showed the Protocol), I really see it in a
different light!”
Collective reflections on their practices let participants generate potential
solutions to the problems they identified. This process will be illustrated further in the
following section under the thematic finding “learning through social interaction and
reflection.” At the end of the session participants were suggested to brainstorm action
steps that can be taken to improve equity at the MSU. The ideas that emerged were: to
bring together departments and colleges; to create exemplars and opportunities to
demonstrate self-reflection in practice; getting beyond “usual suspects;” moving
beyond “lip service;” talking about “presence of absence.”
Experiences in the Workshop and Webinars, and Reactions to CUE’s Tools
Drawing exclusively on the evaluation questionnaire data, I looked at the
participant’s experiences in the Document Analysis workshop and webinars, and their
immediate reactions to CUE’s tools. It helped me to gain a preliminary understanding
about the impact of the events and tools on participants’ attitudes and beliefs. I first
addressed the feedback of participants regarding workshop format, organization, and
engagement. Overall, there was neutral reaction from participants to workshop format.
Forty seven percent of respondents (N=15) neither agreed nor disagreed, and the rest
were equally split between agreement and disagreement that that “they would have
preferred an in-person presentation to the webinar.” Clearly, the format of workshop
was not a factor to interfere with participants’ beliefs and attitudes.

116

However, there were some common complaints about both events. One
respondent argued that he/she would have liked to dedicate more time to focus on
objective discussions as some members tended “to be off target and misguided.”
Another one mentioned that CUE could do more to help bridge what was perceived to
be weak coordination between university administration, faculty, and student services,
which should lead to broader participation in such events. Additionally, people had
some complaints about more practical CUE tool application and training, away from
discussion of abstract concepts. Overall the workshop and webinars were considered
well planned and executed with some respondents praising facilitators for helping the
group to feel comfortable and safe from the outset.
Regardless of the weaknesses, activities succeeded in facilitation of personal and
group activities along with participant engagement. Almost 90% of respondents
(N=19) found CUE’s workshop and webinars “to be engaging” and more than 90%
pointed to “highly encouraging group activity.” Ninety five percent of respondents felt
that their personal engagement in workshops was “Fair” to “Extremely High”, and
almost all respondents described their perception of the engagement of others as
“Fair” to “High.” Such feedback supports the argument about positive impact of
CUE’s workshops in terms of participant engagement.
Strong engagement by participants during workshop and webinars carried over
into conspicuous impact on their beliefs and attitudes in regard to equity in
postsecondary education. In their text responses many respondents manifested “much
more awareness and higher enthusiasm” toward equity issues discussed. Others

117

reported post-workshop “deep thinking” about their classes. As one of the participants
responded to an open-ended question,
Before the meeting, I kind of felt like, “here we go again, another failed attempt to
workshop the problems at Monarch.” I have become cynical, because many of the
administrators who are supposed to care about racial diversity have never experienced
it in their personal lives. So how can they create something that they have never
experienced? But, after the meeting, I went home, washed my car and found myself
thinking deeply about my classes, and things I can do to improve my interactions. If
nothing else, this reflection was worth the time, and I would be willing to do it again.
Respondents also showed the tendency to revive their attitudes and beliefs
toward “cultural inclusiveness.” Prior to attending the workshop 53% of respondents
admitted “rare or some reflection on ways to make their pedagogical materials as
culturally inclusive as possible.” However, when asked about what it meant to be
“culturally inclusive ” all respondents reported expectations “to dedicate more time to
thinking about the issue.” If we add over 90% of responses (N=15) signaling new
ways of reflection among respondents on their pedagogical practices due to
workshop/webinars, it is quite safe to argue that events had solid positive impact on
participants’ beliefs and attitudes toward both equity and cultural inclusiveness.
Regarding the impact on respondents’ intentions, the data was weaker but still
positive. Although the majority of respondents found both CUE’s Document Analysis
Protocol used in October (78%, N=19) and Syllabus Analysis Protocol used in
February (74%, N=15) to be valuable to their work, only 60% (N=15) of respondents
agreed or strongly agreed on intention to make changes to their own syllabus and/or

118

course materials. The weakness could be attributed to some lack of knowledge and
understanding, and the need for additional instruction on application of such tools. In
their text responses some respondents suggested inclusion of broader discussion and
more examples of existing tools and studies that “improved inclusiveness and
addressed cultural, racial, other context” focused on STEM and non-STEM fields.
Respondents were interested in “more discussion about positive steps that faculty can
take to improve student success.” There was also reported some weak communication
of document analysis matrix as it had to address every time in every document. As one
respondent mentioned “ it made it confusing and frustrating to do the assignment
ahead of time without knowing that it was not expected that every document analyzed
contained all elements from the matrix.”
Thematic Findings
Six thematic findings emerged from the results of this study as discussed below.
The first one refers to multicultural diversity perspective in understanding equity.
There is evidence that racial-ethnic equity is a value for most MSU practitioners who
participated in the project and they expressed willingness to take personal
responsibility for reducing inequities. However, there is also evidence that
multicultural diversity perspective prevails in their thinking about the concept of
racial-ethnic equity. Sense of isolation among colleagues and perceived lack of
administrative support (as perceived by participants in my sample) are two thematic
findings reflecting the challenges related to the promotion of equity and cultural
inclusivity at Monarch. Further, data provides evidence that reflection catalyzed by
CUE’s cultural artifacts and tools appeared to have a mediational effect on

119

participants’ attitudes and beliefs moving them towards color-consciousness. Next,
social interaction with peers where roles, norms, and rules were made visible, and
collective reflection promoted participants’ learning about their teaching. Finally,
participants reported changes in their professional practices due to the positive impact
of CUE’s workshops and tools. Promoting institutional change through self-
assessment is a thematic finding that describes those changes.
Table 4.1 below summarizes the distribution of the data I collected through
interviews and participant observation at the workshop/webinar according to the codes
I assigned to it during the data analysis. The distribution of the data codes illustrates
the nature of the inquiry process structured by CUE’s approach, as well as my
approach to coding the data. For example, 18% of the data were coded as reflecting
attitudes and beliefs. This is consistent with the structure of activities surrounding the
“I never benefited from racism” cartoon and the “Who helped you through college”
discussion, which explicitly asked participants to engage in a discussion of their
reactions and thought processes surrounding these materials.
Table 4.1. MSU – Distribution and Sample Quotes of Tabulated Codes from
Data Collected at the Document Analysis Workshop and Post-
Workshop/Webinar Interviews
Code Category as
Represented in Combined
Data Sources
Sample Quotes
Attitudes/ Beliefs
Count: 41
Proportion of total: 18%

“I think first-generation students are often challenged
by institutional constraints … and usually it’s just a
question of inexperience.”

120

Table 4.1, Continued

Control Beliefs/Perceived
Behavioral Control
Count: 10
Proportion of total: 4%
“I think within my classroom I have a great deal of
control … On the departmental level – somewhat
control … And I have absolutely none at the
university level.”
Perceived Norms
Count: 6
Proportion of total: 3%
“So you have a President’s vision, for me, after
experience I’ve had, it’s just a lip service, this is what
they need to say for public consumption …”
Knowledge
Count: 20
Proportion of total: 9%
“So I struggled how I would modify my syllabus to
include this sensitivity …”
Social Interaction
Count: 18
Proportion of total: 8%
“I have always found that collaborating with others
and working with others has always been a success.”
Reflection
Count: 34
Proportion of total: 15%
“We’ve been trying to talk about ‘presence of
absence’ in our teacher education program and how
our candidates are overwhelmingly white. What does
it mean in their preparation to teach in California
schools?
Problem Identification

Count: 39
Proportion of total: 17%
“Some of our policies are almost arcane so if students
don’t get help, they are not likely to understand that
there are consequences for not acting in a timely
way.”
Experimentation/Problem
Solving
Count: 29
Proportion of total: 12%
“The solution is that every faculty includes culturally
responsive pedagogy into their practices.”
Intention
Count: 9
Proportion of total: 4%
“I’d like to incorporate more humor into my syllabus
because ‘it’s pretty cut and dry,’ it can be described as
a legal contract.”
Cultural Artifacts
Count: 8
Proportion of total: 4%
“The comic strip was very clear and the people who
already knew could refer to it and the people who
didn’t know, they were learning.”
Rules & Norms
Count: 10
Proportion of total: 4%

“Many of those issues that concern students of color
are oftentimes silenced not only through the classroom
but also through the university events and through the
general administration.”

121

Table 4.1, Continued

Community
Count: 5
Proportion of total: 2%
“I have to go beyond my own department or even
school in order to have more comfortable
conversations about race and ethnicity ...”

The thematic findings emerged from the data analysis were further supported by
confirming and disconfirming statements. Table 4.2 summarizes the data I interpreted
as providing support for the thematic findings. It shows distribution of the MSU
participants providing confirming and disconfirming data across observational and
interview sources.
Table 4.2. MSU – Number of Statements in the Data Providing Confirming
or Disconfirming Evidence for Themes (N=6)
Theme Respondents (N=6) Providing Confirming and Disconfirming Data
Confirming Disconfirming No Data
Multicultural
Diversity
Perspective in
Understanding
Equity
5 1 0
Sense of Isolation
Among Colleagues
4 1 1
Perceived Lack of
Administrative
Support
3 0 3
Moving Towards
Color-
Consciousness
5 0 1
Learning Through
Social Interaction
and Reflection
6 0 0
Promoting
Institutional
Change Through
Self-Assessment
3 3 0

122

Multicultural Diversity Perspective in Understanding Equity
Although results from the observations, interviews, and evaluation
questionnaires (N=34) provide evidence that the significant majority of the
respondents has a sense of commitment and confidence in their ability to bring about
changes in regard to equity, it was found that their understanding of racial-ethnic
equity tends to be limited by multicultural perspective or diversity. The finding
emerged helps address the first research question of this study: What beliefs do
practitioners hold about racial-ethnic equity for the purposes of equity-oriented
organizational change?
Data from the activity setting observations as well as interviews with
participants shows that participants are concerned about existing inequities on campus,
willing to take responsibility, and believe in their own ability to reduce inequities. Five
of the six respondents made statements consistent with this interpretation. For
example, during the breakout session of the workshop one of the participants – a
faculty member – acknowledged with regret that MSU does not reflect California’s
diversity and, consequently, does not adequately prepare teaching candidates for
teaching in real diverse classrooms. Another participant noted in a cognitive interview,
“I have extremely few students of color in my class. So if we were supporting their
success, I would see more students of color, if the institution was doing all it can.” Yet
another one shared,
As a teacher, I feel sorry about the incoming students of color because we have
admission process here that is really bad, we admit and benefit white students at
the expense of students of color. On top of that we do a much better job of

123

recruiting African American students for athletics but not for academics so that
communicates that they are fine here as long as they are willing to sacrifice their
bodies … We need to diversify our campus; it’s embarrassing when you look at
our numbers here.
Analysis of participants’ control beliefs and perceived behavioral control reveals
their willingness to take personal responsibility for reducing inequities and confidence
in their own ability to make a positive shift. As one of the participants exclaimed
during the workshop, “What are we doing for California in teacher education? I think
it’s our responsibility that we have as a campus and we are not standing up to that.”
Later in the interview she continued this theme, “I think there is much more under our
control than we are willing to admit … I just was thinking that often we are looking
for someone/thing to put blame on but there is a way, in which we all have power to
shift it. We are powerful enough to change it.” Typically, faculty members expressed a
belief that they are capable of making a positive impact through such types of daily
behaviors as communication of affirming attitude to their students, “we are
communicating an attitude, openness, and affirmation and those are important parts of
equitable campus.” They also mentioned intervention in inappropriate behaviors
related to racial-ethnic issues, “If someone else says or does something inappropriate,
I can intervene.” Most importantly, one respondent highlighted reshaping curriculum
and the method of teaching based on students’ diverse needs, “when students come to
my office, I need to be able to listen to their concerns … and reshape the way I interact
with them, curriculum, and the way I teach, based on listening to their concerns.”

124

When asked whether at the institution the changes needed to improve equity are under
their control, three respondents of the four indicated that they have “a great deal of
control” in their classrooms, “somewhat control” on the departmental level, and “less
to no control” on the level of institution.
Evaluation questionnaire data provides further evidence to substantiate results
from observations and interviews. Mostly (although not uniformly), the respondents
felt comfortable with the racial-ethnic equity focus of CUE workshops, had a sense of
responsibility in regard to equity issues on their campus, and believed that their
individual behavior can make a positive difference. Seventy four percent of
respondents to questionnaires (N=19) indicated that they “felt comfortable talking
about issues of race and ethnicity with their colleagues at the Document (Syllabus)
Analysis workshop.” The absolute majority of respondents demonstrated a positive
attitude toward prioritizing “equity-mindedness” at their institution. Seventy four
percent of respondents demonstrated a belief that the MSU’s “commitment to racial-
ethnic equity should be prioritized as “High priority,” along with 11% who chose
“Moderate priority” response option. When asked about whether they “have a strong
sense of personal responsibility surrounding racial-ethnic issues on their campus,”
93% of respondents (N=19) agreed or strongly agreed (40% and 53% respectively).
Respondents of the questionnaires were also confident about their ability to
bring about changes in regard to equity. When assessed in regard to perceived
behavioral control, most of the MSU practitioners demonstrated high self-efficacy and
a belief in their own ability to improve racial-ethnic equity on their campus. A
hundred percent of respondents (N=19) agreed and somewhat agreed (74% and 26%

125

respectively) that they “can make a positive difference to reduce racial-ethnic
inequities on campus through their daily behavior.” Analogously, 93% (N=15) of
respondents agreed and strongly agreed that along with similarly minded colleagues
they had an ability to make a positive impact surrounding equity at their institution
(thus also validating the existence of an institutional equity problem). Furthermore,
80% (N=15) believed that the culturally inclusive changes in their classrooms could
have significant effects on their students’ success.
However, data from interviews revealed that respondents’ understanding of
“racial-ethnic equity” was primarily defined by the concept of providing access and
creating an environment welcoming of differences. Five of the six respondents made
statements supporting this conclusion. When asked what “racial-ethnic equity” meant
for them, one of the interviewees identified it as “having environment that is
welcoming for people from all racial-ethnic backgrounds.” For another respondent it
was “a question of representation,” access to the institution, “our state is one of the
most diverse in the country and our campus doesn’t reflect that, … so as a state
institution, we should be more representative.” Yet the other respondent defined
racial-ethnic equity as “self-identified classifications that students apply to themselves
when they first enter the university.” Only one participant of the six defined racial-
ethnic equity as not just a matter of welcoming and treating everybody equally but
also as “understanding the life experiences of those students,” and “giving sometimes
less resources to white students, who have far more resources outside of university,
and more resources to students of color, who may lack those resources.” Nevertheless,
even this definition does not incorporate “outcome equity.”

126

In addition, there was a tendency among respondents to unconsciously substitute
the notion of “racial-ethnic equity” for “diversity” when answering the questions. For
instance, when asked whether he noticed changes in how he/she viewed issues related
to racial-ethnic equity, one of the respondents replied, “ … I am focusing less on
diversity these days, … I mean I still believe diversity is important but I am involved
in different projects …” Answering the question concerning changes surrounding
equity issues at the level of institution, another faculty confirmed, “ … there are
changes … diversity now is everyone’s job …”
The overall results provide evidence that generally racial-ethnic equity appears
to be a value for most MSU practitioners who participated in the project, particularly
to the extent that diversity and equity are interrelated concepts. They expressed
willingness to take personal responsibility for reducing inequities and a belief in their
ability to make a positive difference on the classroom level. However, there is also
evidence that multicultural diversity perspective, as applied specifically to their
campus, prevails in MSU practitioners’ thinking about the concept of racial-ethnic
equity. Support for equity, defined as a goal of achieving equal educational outcomes,
is less clear.
Sense of Isolation Among Colleagues
Building on observational, interview, and evaluation questionnaire data (N=19),
it was revealed that one of the challenges related to the promotion of equity and
cultural inclusivity at Monarch is that, too often, faculty members experience a sense
of isolation within their community, represented by departments and colleges. This
prevents them from having a sense of positive insight or peer support for addressing

127

issues of racial-ethnic equity. Quantitative data revealed that 74% of respondents
(N=19), who completed questionnaires after the Document Analysis workshop,
indicated that they definitely felt comfortable talking about issues of race and ethnicity
with their colleagues at the CUE workshop. Fewer participants, only 26% (Figure 4.5),
answered that they “always” did at their institution and 74% answered “sometimes”
and “rarely” (53% and 26% respectively).
Figure 4.3. Comfort with Talking about Race and Ethnicity with Colleagues
7. I generally feel comfortable talking about issues of race and ethnicity with my
colleagues.
# Answer

Response %
1 Never

0 0%
2 Rarely

4 21%
3 Sometimes

10 53%
4 Always

5 26%
Total 19
100
%

These results signal two things. First, that the MSU institutional community
does not sustain such conversations and, second, that CUE’s workshop and the tools
provided an opportunity for MSU practitioners to make the conversation possible and
convenient, suggesting a different social norm established or communicated at the
workshop. Also the data revealed some weakness in respondents’ expectations about
their continuing work together to promote equity on MSU campus. Only 32% of
respondents (N=15) were optimistic and 63% had a somewhat positive outlook when
responding to the item “My similarly minded colleagues and I have the ability to make
a positive impact surrounding equity at my institution.”

128

Qualitative data from the follow up cognitive and individual interviews with
participants substantiates this finding providing more details. Four of the six
respondents made statements supporting this interpretation. One of the respondents
shared,
A kind of liberal perspective is acceptable at the Monarch State but radical
perspective on racial equality is not accepted as much … that generally tends to
silence people … when you exist in a monocultural department there are certain
things that you just don’t feel people would understand so that silence you a bit
… I feel definitely silenced in that regard even though I try to do my best not to
act on that feeling.
When asked to provide additional comments in the questionnaire, several respondents
wished more their colleagues had been at the workshop that do not usually think about
equity issues, “I wish more faculty were involved. Often in these meetings it’s the
same dedicated people. The few of us who care deeply about increasing the racial
diversity on campus are often burned out.” One of the interviewees reported
experiences of having conversations about issues of race and ethnicity with the
colleagues who were “hostile” and “ … there is no imperative to do something about it
… other than if you are having difficulty with these people, just stay away from them
…” Another faculty member reported necessity to go beyond own department to find
similarly-minded colleagues,
For me those are issues that are alive and ever present and I don’t necessarily
see that with my colleagues, within a department and even school, who would
prefer to avoid topics of race and ethnicity and for me that’s unfortunate. I have

129

to go beyond my own department in order to have more comfortable
conversations about race and ethnicity.
In an individual interview a faculty member also shared that the people who
were in the workshop, made comments that they couldn’t go back to their departments
and talk about equity and inclusivity because they would be just themselves and they
would feel uncomfortable bringing up these issues in a large group setting. Yet
another participant noted,
I think that the idea of valuing diversity and inclusivity is something that a lot of
people support theoretically but then when it comes to implementation, actually
making changes, I think it’s something that a lot of my colleagues unfortunately
don’t put a lot of effort toward. So they don’t necessarily agree that diversity is
important because it doesn’t improve the quality of education. They even might
think that diversity would decrease the quality of education.
This thematic finding was strengthened by a response to the question about changes in
a respondent’s post-workshop behaviors, when the respondent explained that he/she
did not change behavior and it had nothing to do with the workshop but with the
“structural and cultural constraints” at MSU,
In some ways the method that I use, trying to be more inclusive in my syllabus
and my teaching, represents a more democratic methodology ... and it deals
with the marginalized students on this campus. Traditional methods of teaching
completely silence them. And I think lecture model silence those students
heavily. And I try to have more democratic model, one that would involve
discussion and dialog, one that really brings a lot of input from students and …

130

we actually create the syllabus together. There are institutional obstacles to this;
one is that you have to have a syllabus in the first day of class, so I have to get
around that rule; and two, a different method of teaching … some people may
critique that it’s not being rigorous, if you don’t have enough exams and that
students really need to hear a lecture...
It appeared that despite participants’ apparent commitment and willingness to
take responsibility for reducing inequities, their effort might be affected by the deficit
of the opportunity for a dialog with their fellow colleagues. This finding is informative
and signaling a lack of a shared value among MSU practitioners in regard to racial-
ethnic equity as well as structures that would support collaboration and promotion of
such value.
Perceived Lack of Administrative Support
Another finding that emerged from data analysis deals with a perceived lack of
adequate institutional support, particularly support of senior administrators that MSU
participants revealed. Data from observations, interviews (three of the six), and
evaluation questionnaires (N=19) supports this theme. In some cases, MSU
participants reported explicit disregard from administration related to their efforts to
create change associated with this topic. In other cases, a lack of support for the time
and resources necessary for pedagogical innovation functioned as an indirect form of
discouragement. Fishbein and Ajzen’s (2010) reasoned action approach (Figure 4.6.)
illustrates how normative beliefs and perceived norms affect participants’ intentions
and ultimately behaviors, thus impeding change to bring about equity.

131

Figure 4.4. Participants’ Perceived Obstacles to Change at MSU
Norma&ve)
Beliefs:)
Administra*on,
does,not,
support,this,
Behavioral,
Beliefs,
A5tude,
Toward,
Behavior,
Perceived)
Norms:)
Equity,is,not,
a,value,,
Perceived,
Behavioral,
Control,
Inten&on,
is,
weakened,,
BEHAVIOR:)
Ins*tu*onal,
structures,
affec*ng,
students,of,
color,,
,
Control,Beliefs,
Actual,Control:,
Reality,D,
Unknown,
Background)
Factors:)
Personality,
Mood,
Emo*on,
Values,
Stereotypes,
Gen,A5tudes,
Perceived,risk,
Past,behavior,
Educa*on,
Age,,
Gender,
Income,
Religion,
Race/
Ethnicity,
Culture,
Knowledge,
Media,
Interven*onD
CUE,
,

Source: Reproduced from Fishbein and Ajzen (2010) Reasoned Action
Approach.
This theme helps answer the second research question as related to norms/rules: What
rules/norms mediate attitudes, beliefs, and behaviors related to racial-ethnic equity?
One of the respondents wrote in the “additional comments” section of the
questionnaire,
It is my belief that Monarch State administration and our legal counsel in
particular, take a conservative view on Proposition 209. By doing so, under-
represented students cannot even be identified and therefore, help cannot be
given when needed. Each student success liaison in each college CANNOT have
access to identifying underrepresented students, therefore CANNOT provide

132

help in retention for those students. Until this view changes from the top down, I
don’t believe MSU will ever be successful in consistently retaining under-
represented students. It essentially ties the hands of the practitioners to provide
help.
Figure 4.7 below illustrates practitioners’ beliefs regarding institutional efforts
to enroll and graduate students of color. An overwhelming majority of respondents,
95%, agreed and 5% somewhat agreed (N=19) that the MSU could do more to enroll
and graduate more students of color. Also, answering the questions concerning the
institution’s efforts in providing access and supporting success of students of color,
the majority of respondents disagreed that the MSU is doing all it can (53% and 68%
respectively). These responses demonstrate that in the practitioners’ perceptions, the
institution has not accomplished much in terms of promoting racial-ethnic equity.
Figure 4.5. Views About Institutional Effectiveness to Enroll and Graduate
Students of Color
14. Our institution can do more to enroll and graduate more Students of Color.
# Answer

Response %
1 Disagree

0 0%
2
Somewhat
disagree

0 0%
3
Somewhat
agree

1 5%
4 Agree

18 95%
Total 19 100%

One of the participants described how senior administration discouraged his/her
involvement in the committee assessing student achievement with respect to diversity
learning. “The committee spent three years on collecting data and writing a report,”

133

the participant shared, “but when we gave the report to the administration, it was
largely ignored.” After this experience President’s vision for him/her is “just a lip
service, this is what they need to say for public consumption, I don’t think they really
believe in that … maybe I am cynical at this point, I just don’t believe anymore.”
Ultimately, this participant withdrew from the communication with me. “I still believe
that diversity is important” he/she shared in an individual interview three months later,
“but I guess I am getting to the point where … I was very energized but now I am not
so energized, and I am getting just worn out and I am very disappointed, again, going
back to the assessment committee when not much attention was paid to it. So I guess I
am at the point of disappointment and lack of the energy, where I would like to move
to other things.” Finally, one of the respondents added that faculty involvement in
diversity and equity is not a value in institutional promotion to tenure or retention
process, which, for him/her, communicates that diversity and equity are not the values
of first importance for the institutional leadership.
Moving Towards Color-Consciousness
This thematic finding emerged drawing on data from observations, cognitive
interviews, and evaluation questionnaires (N=19) collected after the Document
Analysis workshop. Data provided evidence that reflection catalyzed by CUE’s
cultural artifacts and tools appeared to have a mediational effect on participants’
attitudes and beliefs moving them towards color-consciousness, “the attribute of
equity-minded individuals” (Bensimon, 2012, p. 36). Five of the six respondents
provided statements confirming this finding. Observations from the Document
Analysis workshop captured that reflection (initiated by the cartoon “I’ve never

134

benefited from racism”) about how current practices are situated within history helped
practitioners to examine held assumptions and beliefs about communities and their
practices. As participants reported later, the tool had promoted their learning, raising
awareness of unconscious bias. Essentially, this theme provides the answer to the third
research question (in regard to attitudes and beliefs): What influence do equity-
oriented cultural artifacts designed for use in “remediated” social learning
environments have on postsecondary educators’ beliefs, attitudes, and behaviors?
The Figure 4.8 below represents a top segment of the Activity Systems Model
demonstrating the mediational effect of the cartoon on practitioners’ cognitive
processes.
Figure 4.6. The “Racism” Cartoon as a Mediator of Thought

The intended outcome represents color-consciousness that, according to
Bensimon (2012), allows issues of race and ethnicity into conversations,
contextualizes inequities according to the context of race, and creates a level of

135

comfort when talking about race-based institutional outcome gaps. When the cartoon
appeared on the screen, it moved the audience into silence for several minutes. People
fixed their eyes on the comic strip. All data suggests that some people were thinking
and speculating on the issues in ways that they, perhaps, had never thought before. A
comment of one of the participants made in the interview, supports this conclusion,
“… not everybody already understands privilege, so the people in the room, who have
never seen it before, it was a great place for them to catch up … the people who didn’t
know, they were learning.” When asked by the facilitator what they were thinking
about while looking at the cartoon, one of the participants, white faculty-administrator,
finally voiced in an undertone, “Privilege.” Many people nodded. Silence extended.
“Privilege,” echoed back in the room. When asked to share what they had learned
from the exercise, a white faculty member noted, “For me it means racism is not
necessarily about our personal relationships but about organizational structures …
there is an institutional component in it.” Her colleague, also a white faculty member
joined the conversation, “Racism is not interpersonal and it is, there is a way in which
it is still shared ... We all can participate in it and it’s weird because if we don’t call
anyone a name, and yet we can participate in an invisible, unconscious way … but I
think that we are powerful enough to change it.” White faculty at my table reflected
aloud,
I just think we are sitting here and thinking ‘I would never say something
offensive.’ But when, for example, I actually went through the protocol
(document analysis) and saw the ways in which I am not very inclusive, mostly

136

had ‘No,’ I realized that I don’t say something inappropriate but I do a lot of
stuff that is more subtle.
Approximately 70% of the questionnaire respondents (N=19) indicated that the
cartoon added a “necessary dimension to the discussion about racial-ethnic equity in
education.” During cognitive interviews, the respondents unanimously (four of the four)
acknowledged the tool’s usefulness and efficacy as a good reminder about a lot of
things that people experience and benefit from but are not always aware. As one of the
interviewees said, “… it was an interesting way to get us thinking more about, without
embarrassing people, the privileges we enjoy at the expense of others.” Yet another one
mentioned, “I loved the comic strip and I am looking for it right now to use it in my
classroom.” Indeed, in the follow-up interview the respondent confirmed the tool’s
application in own classroom.
Overall, these data provide evidence of how collective reflection initiated by an
equity-oriented artifact uncovered for people a way in which the “legacy of prior
generations” can go unnoticed in their current actions. Thus, the tool allowed for “a
conscious consideration of the benefits” of new forms of action (Dowd, Bishop,
Bensimon et al., 2012, p. 6). A remark of the participant in my breakout session
substantiated this conclusion, “I go back to one of those slides … I am still thinking
about it. I feel that we share responsibilities to counteract the stereotypes. I teach …
classes and I personally feel I will try to make an effort.”
Learning Through Social Interaction and Reflection
This finding emerged from observational, interview, and evaluation
questionnaire data (N=15) that suggests that social interaction with peers where roles,

137

norms, and rules are made visible, and collective reflection promotes participants’
learning about their teaching. Six of the six participants made statements confirming
the finding. While discussing the indicator “respect for students,” one participant
reflected, “Mostly we don’t invite students to think with us. So one perspective to
show respect for students would be to incorporate humor into the syllabus … the kind
of humor that respects intelligence and assumes that the students are serious-minded
… a high-minded stuff that sends the student a message ‘you have a capacity to
understand this’ and we’re going to work hard together.” A faculty member, sitting
next, caught up, “Right, I’d like to incorporate humor in my syllabus because it is
pretty “cut and dry” now. I think humor can break the ice a little bit and set more
comfort for students.”
Another participant suggested a different idea of how to show respect for
students,
I think it’s important to us to communicate to students that we are in a learning
process too. I think about when students write about topic of their choice, how
much I learn from them? There are 34 people in my class and knowledge comes
not only from my head. All of us came from different experiences and for me in
my class all those experiences are important.
At some points of discussion people expressed disparities in opinions, thus
making their interaction energetic and even more meaningful. For example, one of the
participants expressed a belief that education for him is collaborative and instructors
are “simply facilitators, who help students understand … and I think it’s a two-way
street, where I have responsibilities and students have responsibilities too … so I think

138

it’s only fair when they understand what those responsibilities are.” One of his/her
colleagues disagreed,
If you are saying to students what their responsibilities are, these students would
resist and I think some professors structure the lack of interest in students …
there are so many ways when we dehumanize students. I don’t teach just
subject, I teach human beings. Listen to students; don’t impose on them solely
what you want them to do … ability to understand students’ needs is very
important. It amazed me even through the grad school how many professors use
the same model of education over and over again.
These examples show that collective benchmarking of the current practices
against the protocol indicators, accompanied by reflection, allowed for defining the
problems and generating potential solutions about how to become more culturally
relevant instructor. As practitioners engaged in two aspects of inquiry, problem
defining and problem solving, their existing knowledge and beliefs were examined
and reconstructed. As one of the participants shared in the session,
Now my syllabus starts with my name and a course name, it can be described as
a legal contract. But I really like your comment ‘Welcome to this class!’ I also
thought about a message that it’s not easy class, you need to work hard to
succeed but I know you are capable to succeed. I think setting this tone at the
beginning will be very helpful and welcoming all students. I am not doing that
now but I will.
Social interaction with peers appeared to be instrumental to the process of
reflection and learning among participants. Data from interviews substantiated this

139

conclusion. Feedback of one of the respondents provided details about the ways in
which social interaction bolstered his/her reflection,
I think, based on conversations with other faculty and listening to some of the
things that they did in their classes, whether they talked about things that worked
in positive ways or negative ways, you know, I kind of tried to make analogies
to my own classroom experiences … Those examples made me try to think: is it
something that I am doing in my class? … I took notes, thinking about asking
questions to myself like how would this work in such and such class, you know.
I think there was much of use that caused me to reflect …
Another respondent confirmed the effect of interaction with colleagues on
his/her learning, “I gained a lot by working in an activity with people, more than alone
just having a presentation done or things to read … collaborative effort is always
successful, you know.” Yet another one shared,
One of those things for me, as a professor, that strikes me is how little people
actually talk about teaching … and so when I was going through the document
analysis, it helped me think categorically about some of the issues that I need to
improve in my teaching and I liked having that as a discussion …
Having that collective feedback was really good and I wanted that collective
collaboration. I enjoyed the session because it allowed more collective and
reflective thinking about teaching and usually this doesn’t happen here.
The results of evaluation questionnaires support this interpretation of the
observational and interview data. Fifty percent of respondents (N=15) acknowledged
rare or some reflection on ways to make their pedagogical materials as culturally

140

inclusive as possible prior to attending the workshop, with the other 50% reflected
“Quite Often” and “Very Often.” However, when asked about whether the workshop
encouraged reflective environment for group activity among colleagues, 90% of
respondents agreed and strongly agreed. Also, respondents almost unanimously agreed
and strongly agreed that overall the workshop caused them to reflect in new ways on
pedagogical practices (Figure 4.9).
Figure 4.7. Views About the Workshop as Facilitating Reflection on
Pedagogical Practices
7. Overall, today's workshop caused me to reflect in new ways on my own
pedagogical practices.
# Answer

Response %
1
Strongly
Disagree

0 0%
2 Disagree

1 7%
3
Neither
Agree
nor
Disagree

0 0%
4 Agree

9 60%
5
Strongly
Agree

5 33%
Total 15 100%

Promoting Institutional Change Through Self-Assessment
Drawing on interview and document analysis data, I explored the impact of
CUE’s action research processes and tools on practitioners’ behaviors after they used
the Document (Syllabus) Analysis Protocol. This impact produced several behavioral
changes. Three of the six participants reported post-workshop/webinar changes.
Although only one of those changes involved changes in the syllabus, they still
reflected the impact on practices. The theme is informative to answer the third

141

research question in regard to behaviors: What influence do equity-oriented cultural
artifacts designed for use in “remediated” social environments have on postsecondary
educators’ beliefs, attitudes, and behaviors?
One of the respondents reported the use of the cartoon “I’ve never benefited
from racism” (one used by CUE at the workshop) in his classes. Another participant
was highly impressed by the Syllabus Analysis Protocol as well as the worked
example provided in the presentation, exclaiming, “I am sold! I like it!” Later he
continued in the interview, “I think that the syllabus is a whole different thing to me
now, I see it as a whole different starting point, it is a lot more important to me now.”
Immediately after the webinar, he reported changes in his syllabus stemming from his
participation in the webinar, “I was able to add some inclusive images to my syllabus
and now I try to format it more like a newsletter format, when I have time, I am going
to do that. I am thinking about showing inclusiveness from the very beginning of the
class.”
However, the strongest evidence of CUE’s tools impact on participant behaviors
can be illustrated by the post-workshop-organized professional learning community
(PLC). Notably, its title highly reflects the title of the Document Analysis workshop.
One of the originators of the community, a faculty member, highlighted the influence
of CUE workshop, sharing in an individual interview,
The workshop gave me an idea to start a learning community. When I walked
out of the meeting in October, my idea was to create a continuing conversation
and one that would invite two people from each department to come and to
continue the conversation. So the people who were uncomfortable going back,

142

… would be in a learning community and continue that conversation and then
when they knew enough, they would go back to their departments and there
would be two of them, and they would have more strength.
Together with another faculty he/she wrote a proposal (soon after the
workshop), which was funded by the university Center for Teaching and Learning.
Subsequently, two sections of the professional community successfully functioned and
met every week during the Spring 2012 semester. Their ultimate goal was “to combine
and integrate the support systems attention to demographics (recruitment and retention)
with classroom-based practices (e.g., syllabi and assignment construction, team
creation, etc.).” Over 20 participants from both the student support network and STEM
faculty were involved and there were plans to continue on ongoing basis.
According to the Proposal document, one of the components of this PLC was
that participants would together develop, enact, and then analyze the success/failure of
one or more pedagogical changes in their classrooms in Spring 2012. Among
resources for reading and group discussions, there is a book by Margolis (2010) Stuck
in the Shallow End: Education, Race and Computing, the one that was used twice
during CUE workshops, “So things that you guys talked about are now homework
assignments for two sections of learning community,” the participant shared. In
addition, my interviewee reported application of the Document Analysis Protocol in
PLC, “This week we also talked about the document analysis. We talked about the
syllabus and how to make it more culturally relevant … so you want to show that you
are taking people into account and trying to make an environment that is inclusive.” At
the end of our conversation, she expressed a strong belief that those people who were

143

involved “are going to be different at the end of PLC” and that this learning
community would create more learning communities, “these people will go and create
learning communities on their own.”
It appeared that the discussed events and tools influenced practitioners’
behaviors in positive direction; however, the strength of the impact was not at its
utmost. The results of the study provide evidence that partially this can be attributed to
some shortcomings in the Document Analysis Protocol design. The shortcomings
were mainly associated with the weak tool applicability. The participants did not feel
confident in how to apply certain indicators. In addition, certain indicators did not
reflect specificity of cultural inclusivity, thus sometimes diverting the focus of
discussions. Data revealed the necessity of presenting samples of the pre- and post-
syllabi, reflecting the changes that had been adopted because it would promote
people’s understanding of the indicators themselves and their application.
Summary
This study examined whether and how inquiry facilitated by CUE action
research and by equity-oriented cultural artifacts (language, media, tools) mediated
practitioners’ attitudes, beliefs, and behaviors related to racial-ethnic inequities. It was
found that when practitioners were engaged in activity settings using CUE’s cultural
artifacts and tools, their attitudes and beliefs were challenged and influenced towards
equity-mindedness and cultural inclusivity. Participation in action inquiry and
reflection influenced their willingness towards behavioral changes. However, the
impact on behaviors was moderate, which might be attributed to perceptions of an
unsupportive university environment, lack of knowledge how to incorporate principles

144

of culturally responsive teaching into practice as well as to the Document Analysis
Protocol design.
CUE initiated social interaction and reflection on roles, rules, norms among
practitioners using the Document Analysis Protocol to catalyze conversation and
reflection about the ways of incorporating principles of culturally inclusive pedagogy
into the syllabi and practice. Collective benchmarking of the current practices against
the Protocol indicators accompanied by reflection, allowed for defining the problems
and generating potential solutions about how to become a more culturally relevant
instructor. As practitioners engaged in those inquiry aspects, their existing knowledge
and beliefs were examined and reconstructed. Social interaction with peers appeared
to be instrumental to the process of learning among participants. Participants
expressed willingness to change their professional practices due to the positive impact
of CUE’s Protocol on their beliefs and attitudes. However, the impact was moderate
because participants reported lack of knowledge concerning incorporating protocol
indicators into syllabi and a perceived negative pressure on the side of wider
university community. The study also found that the effectiveness of inquiry process
depends on the support of university leaders as well as their understanding of why
equity is important. MSU campus culture has instilled a sense of distrust among
administrators, faculty, and staff. “Without formal institutional support, equity
advocates and leaders can emerge through action inquiry, but their potential for impact
is likely to be diminished” (Dowd & Bensimon, 2009, p. 8).

145

CHAPTER 5: CONCLUSION AND RECOMMENDATIONS
This final chapter provides a brief overview of the impetus to the study and
summarizes the significant findings, which are followed by recommendations for the
related audiences and a conclusion. To review, the study was conducted throughout
2011 – 2012 as part of a larger research agenda and developmental self-evaluation
being carried out by researchers at USC’s Center for Urban Education. With a mission
to improve college graduation rates among different racial-ethnic groups on a national
level, CUE conducts participatory action research by engaging higher education
practitioners in self-assessment through a cycle of inquiry using CUE developed
equity-oriented tools (Dowd & Bensimon, 2009). CUE is a nationally recognized
research and action center whose work has been funded over the years by numerous
philanthropic foundations, educational institutions, and state systems.
Some of the recent CUE projects include the Equity Scorecard with the
University of Wisconsin that contracted CUE to implement three phases of the Equity
Scorecard process on its campuses through 2005 – 2012. Another example involves
CUE collaboration with the Pennsylvania State System of Higher Education
(PASSHE), which retained the Center in 2011 for a two-year project aimed at
improving access, retention, and success of minority students at 14 PASSHE
campuses. CUE’s work has been featured at national meetings of the largest higher
education associations and it has been disseminated through articles in publications for
practitioners and in peer-reviewed academic journals. More information about CUE
can be obtained through their website at www.usc.edu/cue.

146

The impetus to the collective developmental self-evaluation study (which this
study was part of) was stimulated by CUE leadership’s continuous effort to improve
the quality of their work with institutions of higher education. Results of the collective
evaluation are intended to inform the development of CUE’s action research processes
and tools, designed to foster equity among racial-ethnic groups in higher education
experiences and outcomes. Aligned with the collective goal, the purpose of my
individual study was to investigate whether and how CUE’s activities and tools enable
practitioners to assess their own beliefs and practices, to think about changing their
practices to better serve historically underrepresented students, and to advocate for
changes towards equity-mindedness at their institution. A complementary goal was to
help develop CUE’s evaluation capacity by improving the validity of the inferences it
draws from the workshop evaluation questionnaires. Given the design of the study, the
primary audience for its findings is represented by CUE researchers. However, the
results are also informative for action researchers in general, the practitioners at my
field site (MSU) and peer institutions, and higher education practitioners, particularly
those engaged in projects involving assessment, accountability, equity, diversity, anti-
racism, and organizational change, because accreditors and policy makers are asking
all of these stakeholders for change.
Significant Findings
To summarize, the multiple field sites involved in the collective study were
purposefully sampled based on their relationship with CUE and included three
community colleges and a comprehensive four-year university. The field site for this
study was Monarch State University, a public university with selective admissions

147

practices located in California. MSU was purposefully selected due to two reasons;
first, the institution has been experiencing challenges related to diversity and equity
issues. Second, it is one of the few institutions of higher education that is interested in
participating in action research. Speaking in Patton’s (2002) language, the purposeful
sampling allowed for selecting an information-rich case that illuminated the questions
under study.
Within the scope of the study, CUE researchers created activity settings
(workshops) for a group of practitioners at MSU and facilitated their learning to
become culturally responsive educators through the process of action research and
self-assessment activities using the Document (Syllabus) Analysis Protocol, a CUE
tool that assists individuals and teams to explore and to reflect on their syllabi (as well
as other documents of professional practice). The activity settings were designed as
joint ventures that allow practitioners to interact and collaborate on issues including
cultural inclusivity in teaching, responsiveness to the needs of diverse students, and
providing equitable learning opportunities for minority students (Moll, 2000; Tharp &
Gallimore, 1988). The settings became the cultural devices for thinking and learning
(Moll, 2000), where learning occurred through assisted performance (Tharp &
Gallimore, 1988). The goals of the activity settings were to direct the action and
collaborative interaction towards developing a structure that encourages a common
understanding of equity-mindedness.
Consistent with the action research and CHAT theoretical framework, the first
key finding reveals that participation in social interactions facilitated by action
research and by equity-oriented cultural artifacts (language, media, tools) promoted

148

learning among practitioners. Using the tools allowed for social interaction by
collaboratively discussing what was needed for change. Participation in self-
assessment and a discussion, directed by CUE facilitators, provided structured
opportunity for reflection that enabled participants in my sample to learn how they
could improve their pedagogical practices to promote success of their
underrepresented students. Tharp and Gallimore (1998) emphasize that problem
solving and learning cannot be “understood outside the complex social context” (p.
91). They explain that designing learning experiments should include assisted
performance activities with peer consultants. In joint productive activity settings, the
members of a group are not sharply divided into novices and experts, but rather are a
combination of people with different competencies that work together in a manner
where an individual member will assist others in the group depending on his or her
own areas of knowledge and skill (Tharp & Gallimore, 1998).
At the workshop CUE researchers created a forum for practitioners to be open to
different perspectives and problem framing and allowed for them to engage in
experimentation and problem solving (Dowd & Bensimon, 2009). It was found that
collective benchmarking of the current practices against the protocol indicators,
accompanied by reflection, allowed for defining the problems and generating potential
solutions about how to become a more culturally relevant instructor. As practitioners
engaged in two aspects of inquiry – problem defining and problem solving – existing
knowledge and beliefs were examined, questioned, and reconstructed towards cultural
inclusivity. However, as participants engaged in reflection, the process did not always
result in learning. There were situations when participants experienced a lack of

149

knowledge related to principles and strategies of culturally responsive teaching as well
as to the ways of how to incorporate them into practice. One of the progenitors of the
culturally responsive teaching theory, Geneva Gay (2000) states, “Teachers need
guidance in their attempts to do multiculturally responsive teaching” (p. 212). The
design of the Document (Syllabus) Analysis Protocol and activity around it were not
always effective in providing that guidance.
Another key finding emerged drawing on Fishbein and Ajzen’s (2010) reasoned
action approach. In alignment with this approach, I assumed that participation in
action inquiry and reflection would influence willingness towards behavioral changes.
Indeed, in the Monarch State University case, reflection played an important role in
allowing project participants to think about changing their practice to better serve their
racially and ethnically diverse students. However, despite the fact that they openly
expressed willingness to make changes in their syllabi and behaviors, their ability to
act on this intention was limited. This limitation might be attributed to a lack of
knowledge and understanding of how to incorporate conceptual indicators of the
Protocol into real syllabi as well as to a number of factors related to the university
culture such as perceived lack of support from university administration and
colleagues. Robinson-Armstrong et al. (2012) argue that institutional change around
equity and perspectives on diversity are political phenomena and therefore, it is vital
for practitioners “at the bottom” to have support of the institutional leadership.
Participants at MSU expressed a sense of powerless to bring about change campus-
wide because in their perception, equity was not a priority at MSU and they did not
feel the higher administration’s buy-in.

150

Recommendations
Recommendations presented here stem from the results and findings of the study
and are offered for consideration of related audiences including: action researchers in
general and Center for Urban Education’s action researchers in particular; higher
education practitioners at the Monarch State University and at similar institutions;
higher education professionals overall. The immediate users of the present study are
twofold: researchers at CUE and practitioners at institutions of higher education like
MSU. As mentioned previously, the study aimed to inform CUE for enhancement of
its efficacy in conducting action research and for development of the evaluation
questionnaire. It also aimed to inform higher education practitioners for creation of
self-assessment activities using action inquiry to initiate the change process for more
equitable outcomes. I begin by providing several recommendations for CUE and other
action researchers. Next, I offer recommendations for MSU practitioners and their
colleagues at similar institutions. Recommendations for higher education professionals
conclude this section.
Recommendations for CUE and Other Action Researchers
Recommendations Related to Action Research Processes and Tools
First, given the revealed insufficiency in participants’ understanding of how to
incorporate the Document (Syllabus) Analysis Protocol indicators into practice, it is
necessary to strengthen the Protocol capacity of promoting practical knowledge,
“knowing how to act,” among its users – higher education practitioners. One way to
do it would be by providing guidelines for incorporation of culturally inclusive
pedagogy into syllabi with focus on race-ethnicity or remediation of color-blindness.

151

The Document Analysis Protocol showed the potential to be a valuable mediating tool
that allows faculty to reflect on how effective or ineffective their current syllabi and
practices are. However, it did not allow for implementation of the improvement
strategies as required to promote the idea of cultural inclusiveness. Most faculty
members reported that they lack the knowledge, skills, and experiences needed to
teach ethnically diverse students. Therefore, there is a demand for a tool that would
address this lack of practitioners’ knowledge and help them become more effective in
working with culturally diverse categories of students.
Greenwood and Levin (2005) argue that phronesis constitutes the necessary
“know-how” for organizational change. The sources of phronesis are collaborative
arenas for knowledge development in which “professional researcher’s knowledge is
combined with the local knowledge of the stakeholders in defining the problem to be
addressed” (p. 51). When CUE researchers and MSU practitioners worked together,
they successfully identified problems. However, the notion of phronesis goes further,
“together, they design the actions to improve the situation” (Greenwood & Levin,
2005, p. 51). It appeared that the tool had a strong theoretical capability, initiating
reflection based on theoretical principles of culturally inclusive pedagogy. At the same
time, knowledge “is not a passive form of reflection but emerges through actively
struggling to know how to act …” (p. 51). Although practitioners revealed intentions
to modify the current syllabi based on the tool’s indicators, the design of the tool itself
and activity around the tool did not provide them with this necessary “knowing how to
act.” Additionally, it would be valuable to organize activities around the tool in such a
way that would provide practitioners with the background, core principals, and

152

methods of culturally responsive teaching theory. These activities would enhance their
understanding of the underlying constructs of the theory itself, creating a knowledge
base for implementation.
Also, it is necessary to incorporate into the Protocol more indicators that would
reflect specificity of culturally responsive pedagogy with the focus on ethnically
diverse students and clearly distinguishing from those reflecting critical race
perspectives. As it was mentioned previously, the Protocol went through certain
modifications during this study based on emergent results and feedback from
practitioners. It was revealed that certain current indicators could not be considered as
“culturally relevant” because they could be applied to all students universally. Such
indicators increase a risk of diverting the focus of discussions from equity or race
consciousness to discussion of diversity more generally.
To address this design weakness, I recommend CUE adopt a revised protocol
and utilize one version during the problem-defining phase and another during the
problem-solving phase of the cycle of inquiry. The one for problem defining would
utilize indicators to prompt reflection, similar to those used in the prior CUE protocols
and with a similar facilitation strategy. The version to be utilized during the problem-
solving phase would be annotated to provide guidance as to specific pedagogical or
curricular strategies faculty can take to effectively incorporate the tenets of culturally
inclusive pedagogy. These annotations would include references to the relevant
knowledge base in the educational literature to ensure that those faculty, who wish to
learn more would have a key resource immediately available for accessing that
literature. The optimal timing for use of the problem solving version of the tool – and

153

the optimal facilitation strategy for introducing the knowledge base and prescriptive
guidance – should be the subject of additional developmental evaluation, as the use of
prescription should be introduced in a manner consistent with the expectation of
practitioner expertise and knowledge generation, which is the foundation of CUE’s
work and inquiry strategies more broadly.
The tables below illustrate these recommended revisions to the Syllabus Review
Protocol. Whereas the Document Analysis Protocol included more general indicators,
given the clear emphasis on classroom instruction and curriculum here, alternative
indicators (and the accompanying knowledge base) would need to be developed for
other arenas of educational practice, for example student advising and admissions.
Table 5.1 first provides a summary of changes that took place during the period of my
fieldwork. Table 5.2 presents the version I recommend for use during problem
defining, restricted to presentation of the indicators only. As noted above, the
complete protocol would include instructions for use to the practitioner and ground
rules for facilitation, much like the original protocol. Table 5.3 presents the annotated
version for use during the problem-solving phase of inquiry.
The indicators on each of these three tables are categorized under the headings
of I. Cultural Responsiveness/ Diversity/ Multiculturalism; II. Active Learning; and
III. Critical/Equity Perspective. These categories are utilized as a means to distill the
various tenets of culturally inclusive pedagogy in a manner that distinguishes critical
race pedagogies, as a necessary aspect of inclusiveness which goes beyond recognition
of the value of diversity to recognition of the need for social change and remediation
of structural racism in educational institutions. The first column of the Table 5.1 below

154

presents indicators from both Protocols used in the activity setting in October and
February as well as those I propose for use. The second column provides comments
noting the changes that took place as well as changes in the proposed indicators.
Table 5.1. Indicators Grouped by Perspective and Developed Over Time

I. Perspective: Cultural Responsiveness/ Diversity/ Multiculturalism
# Indicators Communicating Affirming
Attitude
Comments

1.

Respect for students (October);
Respect for students from all backgrounds
(February);

Proposed:

Communicates an environment in which there is
genuine respect for students and a belief in
their capability (Rendon, 1994):
• Treat students equally
• Communicate clear messages that all
students are capable of obtaining high
educational goals and expected to
succeed
• Provide descriptions that empower all
students to believe they can meet and
excel at the course goals and objectives
• Make it clear that student success will
be a collaborative effort among the
student, peers, faculty, administrators,
counselors, students’ families and
communities
• Work individually with students who
need extra help
• Provide meaningful feedback

Proposed indicators include
sample strategies to improve
the protocol applicability.

155

Table 5.1, Continued

2.
Desire to help students succeed (October);
Desire to help students from all backgrounds
succeed (February);

February indicator was
slightly modified to reflect the
inclusion of all students. It
was further modified and
included in the proposed
version (see above).
Indicators Communicating High
Expectations
Comments

3.

Belief that all students are capable of obtaining
high educational goals (October)
Expectation that student success will be a
collaborative effort among the student, peers,
faculty, administrators, counselors, students’
families and communities (October);

Expectation that students from all backgrounds
are capable of obtaining high educational goals
and expected to succeed. Student success will
be a collaborative effort among the student,
peers, faculty, administrators, counselors,
students’ families, and communities (February);

The two indicators from the
October protocol were
combined in one and used in
February. They were also
incorporated in the Proposed
version (Indicators
Communicating Affirming
Attitude).

4.

Information students need to successfully act on
the information in the document (October);

Information and resources students need to be
successful are available in the document
(February);

Proposed:

Communicates clear expectations (Gay, 2000):
• Be specific in what you expect students
to know and be able to do
• Provide information and resources
students need to be successful

The Proposed indicator
embraces and enhances ones
used in October and February.

156

Table 5.1, Continued

1

Rendón (1994) defines validation as “an enabling, confirming, and supportive process initiated by in- and
out-of-class agents that foster academic and interpersonal development.”
Rendón, L. I. (1994). Validating culturally diverse students: Toward a model of learning and student
development. Innovative Higher Education, 19(1), 33–51.

Indicators Communicating Validation Comments

5.

Validation of racial diversity (October);

Validation of diversity in terms of ethnicity
(shared ancestry, language, national heritage,
religious beliefs, community norms) (October);

Validation of other forms of diversity, such as:
Ability/Disability
Gender
Socio-economic status
Sexual orientation
Immigration status
Age (October);

Validation of the value of differences in prior
educational experiences (October; February);

Validation
1
of racial-ethnic diversity (shared
ancestry, language, national heritage, religious
beliefs, community norms) (February);

Proposed:

Shows connections of course to students’
cultural backgrounds (Gay, 2000):
• Include readings reflecting diversity of
students’ racial-ethnic experiences and
backgrounds (see Multicultural Review
for recommendations)
• Promote student engagement through
having students to generate lists of
topics they wish to study and/or
research and allowing students to select
their own reading materials
• Have students research aspects of a
topic within their community

The October indicators were
edited to maintain focus on
racial-ethnic diversity, to
reduce redundancy, and to
enhance usability.

Proposed indicators improve
overall applicability by
providing concrete examples
of pedagogical strategies
validating racial-ethnic
diversity.

157

Table 5.1, Continued

• Encourage students to interview
members of their community who have
knowledge of the topic they are studying
• Have members of the students’
communities speak to students on
various subjects, being sure to include
minority communities
• Ask members of the community to teach
a lesson or give a demonstration (in
their field of expertise) to the students,
being sure to include minority
communities

Bridges cultural differences through effective
communication (Gay, 2002):
• Teach and talk to students about
differences between individuals
• Show how differences among the
students make for better learning

Communicates an environment that encourages
and embraces culture (Gay, 2002):
• Allow students ample opportunities to
share their cultural knowledge
• Teach students to question and
challenge their own beliefs and actions
• Discuss current events from a
multicultural viewpoint
• Provide information to the students on
alternative viewpoints or beliefs of a
topic

Includes learning activities that are more
reflective of students’ backgrounsd Gay, 200):
• Assign students research projects that
focus on issues or concepts that apply to
their own community or cultural group
• Employ self-assessment and reflection
techniques that encourage sharing,
diverse perspectives and equity in
participation, and critical problem
solving

158

Table 5.1, Continued

• Communicate that the course prepares
students in specific ways to contribute to
their community
• Advocate student volunteerism in a local
culture as a “service learning”
experience (Gay, 2000)

Indicators Communicating Empowerment Comments

6.

Expectation that students will be empowered as
agents of social and community well being
through their education (October);

Expectation that students will be empowered as
agents of social and community well being
through their education to reduce racial-ethnic
inequities (February);

Proposed:

Communicates expectation of empowering
students through:
• Explaining their problem-solving
techniques to each other in small groups
• Learning the “cultural capital” of
school success (test-taking strategies;
self-presentation techniques; note-
taking; time management) (Gay, 2000)

February indicator adds
emphasis on racial-ethnic
equity. Proposed indicator
provides sample strategies.

7.

Recognition that aspects of the educational
experience are challenging but attainable
(October; February);

No changes were stated in this
indicator; it is incorporated in
the Proposed version
(Affirming Attitude).
Indicators Communicating Rigorous
Learning
Comments

8.

Expectation that learning takes place in with
authentic application to and engagement with
real-world problems (October);

159

Table 5.1, Continued

Expectation that education starts from students’
experiences and builds upon those experiences
to expand student learning (October);

Expectation of milestone and culminating
experiences that connect general, major, and
field-learning contexts (October);

Proposed: None

These indicators were not
included in the February
protocol.
The Proposed version does
not incorporate Rigorous
Learning indicators because
they do not reflect the concept
of culturally relevant
pedagogy.

9.

Expectation that broad integrative learning will
take place, focused by engagement with big
questions, both contemporary and enduring
(October; February);

Expectation that analytical, applied and
integrative learning will occur across all major
fields (October; February);

Proposed: None

No changes were made in the
February protocol.
The Proposed version does
not incorporate Rigorous
Learning indicators because
they do not reflect the concept
of culturally relevant
pedagogy.

II. Perspective: Active Learning

10.

Expectation that students from all
backgrounds will actively participate in
different aspects of their educational
experience (February)

Expectation for active involvement with
diverse communities, real world problems,
and social responsibility (October; February)

Proposed:

Includes a wide variety of instructional
strategies to promote active learning
activities (collaborative & cooperative
learning, group/individual presentations,
discussions, lectures, etc.) (Gay, 2000)
The Proposed indicators
make the knowledge base
for active learning
strategies evident,
suggesting sample
approaches of how to
incorporate the tenets of
active learning into
practice.

160

Table 5.1, Continued

Signals the use of teaching approaches to
accommodate diverse learning preferences
and language proficiency:
• Promote student engagement by
requiring that students play an active
role in crafting curriculum and
developing learning activities
• Initiate cooperation and
collaboration among students to
achieve common learning outcomes
• Use student-directed discussion
groups
• Promote learning by doing (Gay,
2000)
• Incorporate different types of skill
development (intellectual, emotional,
moral) in teaching and learning

III. Perspective: Critical/Equity Perspective

11.

Expectation that the educational community
will engage in respectful discussion of the
history and contemporary experiences of
discrimination, racism, and marginalization
(October; February);

Expectation that students will be empowered as
agents of social and community well being
through their education to reduce racial-ethnic
inequities (February);

Proposed:

Reflects inclusion of the experiences of
marginalized students by including such
methods as storytelling, family histories,
biographies, scenarios, etc. (Yosso, 2005)

Addresses the inequality of the curriculum
before addressing unequal educational
outcomes:

161

Table 5.1, Continued

Table 5.2. Proposed Indicators of the Syllabus Review Protocol for Use
During Problem Defining

Incorporate into lectures and discussions a
challenge to the dominant social and cultural
assumptions regarding culture and intelligence,
language and capability, objectivity and
meritocracy (Yosso, 2002)

Reflects the use of a variety of pedagogical
approaches to teach about race and racism (as
cited in Nikol Alexander-Floyd, 2008):
• Provide a substantial amount of data
establishing the reality of social
inequality at the beginning of the course
and tell students ahead of time that they
will be engaging in difficult subjects
(Samuels, Fervber, & Herrera, 2003)
• Set ground rules for discussion to
ensure respect in the classroom (Allen,
Floyd, & Gillman, 2001)
• Assign privilege inventories and/or
other exercises that encourage self-
examination (Sue, 2003)
• Create opportunities for self-generated
knowledge (e.g., assigning interviews on
racial topics) (Tatum, 2001)

Demonstrates expectations of reflection as a
means of incorporating issues of equity into
teaching, thinking, and practice (Howard,
2003)

The Proposed indicators
incorporate several central
themes of a critical race
theory extended to
education: the challenge to
dominant ideology,
experiential knowledge of
students from marginalized
groups, and the
commitment to social
justice (Yosso, 2005).
They also improve
applicability of the
indicators from October
and February protocols.
I. Perspective: Cultural Responsiveness/ Diversity/ Multiculturalism
# Indicators Communicating Affirming Attitude

1
.

Communicates an environment in which there is genuine respect for students
and a belief in their capability;

162

Table 5.2, Continued

Indicators Communicating High Expectations

2.

Communicates clear expectations;

Indicators Communicating Validation

3.

4.

5.

6.

Shows connections of course to students’ cultural backgrounds;

Bridges cultural differences through effective communication;

Communicates an environment that encourages and embraces culture;

Includes learning activities that are more reflective of students’ backgrounds;

Indicators Communicating Empowerment

7.

Communicates expectation of empowering students;

II. Perspective: Active Learning

8.

9.

Includes a wide variety of instructional strategies to promote active learning
activities (Gay, 2000);

Signals the use of teaching approaches to accommodate diverse learning
preferences and language proficiency;

III. Perspective: Critical/Equity Perspective

10.

11.

12.

13.

Reflects inclusion of the experiences of marginalized students (Yosso, 2005);

Addresses the inequality of the curriculum before addressing unequal
educational outcomes (Yosso, 2002);

Reflects the use of a variety of pedagogical approaches to teach about race and
racism (Nikol Alexander-Floyd, 2008)

Demonstrates expectations of reflection as a means of incorporating issues of
equity into teaching, thinking, and practice (Howard, 2003).

163

Table 5.3. Proposed Indicators of the Syllabus Review Protocol for Use
During Problem Solving

I. Perspective: Cultural Responsiveness/ Diversity/ Multiculturalism
# Indicators Communicating Affirming Attitude

1.

Communicates an environment in which there is genuine respect for students
and a belief in their capability (Rendon, 1994):
• Treat students equally
• Communicate clear messages that all students are capable of obtaining
high educational goals and expected to succeed
• Provide descriptions that empower all students to believe they can meet
and excel at the course goals and objectives
• Make it clear that student success will be a collaborative effort among
the student, peers, faculty, administrators, counselors, students’ families
and communities
• Work individually with students who need extra help

Indicators Communicating High Expectations

2.

Communicates clear expectations:
• Be specific in what you expect students to know and be able to do
• Provide information and resources students need to be successful
•
Indicators Communicating Validation

3.

Shows connections of course to students’ cultural backgrounds (Gay, 2000):
• Include readings reflecting diversity of students’ racial-ethnic
experiences and backgrounds (see Multicultural Review for
recommendations)
• Promote student engagement through having students to generate lists
of topics they wish to study and/or research and allowing students to
select their own reading materials
• Have students research aspects of a topic within their community
• Encourage students to interview members of their community who have
knowledge of the topic they are studying
• Have members of the students’ communities speak to students on
various subjects, being sure to include minority communities
• Ask members of the community to teach a lesson or give a
demonstration (in their field of expertise) to the students, being sure to
include minority communities

164

Table 5.3, Continued

4.

5.

6.

Bridges cultural differences through effective communication (Gay, 2002):
• Teach and talk to students about differences between individuals
• Show how differences among the students make for better learning

Communicates an environment that encourages culture (Gay, 2002):
• Allow students ample opportunities to share their cultural knowledge
• Teach students to question and challenge their own beliefs and actions
• Discuss current events from a multicultural viewpoint
• Provide information to the students on alternative viewpoints/beliefs of a
topic

Includes learning activities reflective of students’ backgrounds (Gay, 2002):
• Assign students research projects that focus on issues or concepts that
apply to their own community or cultural group
• Employ self-assessment and reflection techniques that encourage
sharing, diverse perspectives and equity in participation, and critical
problem solving
• Communicate that the course prepares students in specific ways to
contribute to their community
• Advocate student volunteerism in a local culture as a “service learning”
experience (Gay, 2000)

Indicators Communicating Empowerment

7.

Communicates expectation of empowering students through:
• Explaining their problem-solving techniques to each other in small
groups (Gay, 2000)
• Learning the “cultural capital” of schools success (test-taking
strategies; self-presentation techniques; note-taking; time-management)
(Gay, 2000)

II. Perspective: Active Learning

8.

9.

Includes a wide variety of instructional strategies to promote active learning
activities (collaborative & cooperative learning, group/individual presentations,
discussions, lectures, etc.) (Gay, 2000)

Signals the use of teaching approaches to accommodate diverse learning
preferences and language proficiency:

165

Table 5.3, Continued

• Promote student engagement by requiring that students play an active
role in crafting curriculum and developing learning activities
• Initiate cooperation and collaboration among students to achieve
common learning outcomes
• Use student-directed discussion groups
• Promote learning by doing (Gay, 2000)
• Incorporate different types of skill development (intellectual, emotional,
moral) in teaching and learning

III. Perspective: Critical/Equity Perspective

10.

11.

12.

13.

Reflects inclusion of the experiences of marginalized students by including such
methods as storytelling, family histories, biographies, scenarios, etc. (Yosso,
2005)

Addresses the inequality of the curriculum before addressing unequal
educational outcomes: Incorporate into lectures and discussions a challenge to
the dominant social and cultural assumptions regarding culture and
intelligence, language and capability, objectivity and meritocracy (Yosso, 2002)

Reflects the use of a variety of pedagogical approaches to teach about race and
racism (as cited in Nikol Alexander-Floyd, 2008):
• Provide a substantial amount of data establishing the reality of social
inequality at the beginning of the course and tell students ahead of time
that they will be engaging in difficult subjects (Samuels, Fervber, &
Herrera, 2003)
• Set ground rules for discussion to ensure respect in the classroom
(Allen, Floyd, & Gillman, 2001)
• Assign privilege inventories and/or other exercises that encourage self-
examination (Sue 2003)
• Create opportunities for self-generated knowledge (e.g., assigning
interviews on racial topics) (Tatum, 2001)

Demonstrates expectations of reflection as a means of incorporating issues of
equity into teaching, thinking, and practice (Howard, 2003)

166

Recommendations Related to the Evaluation Questionnaire
Regarding revealed difficulties dealing with subjects’ comprehension of certain
terms in the evaluation questionnaire items and making judgments, the following
change are recommended:
1) Original form of evaluation questionnaire item:
My institution’s commitment to racial/ethnic equity should be prioritized as…
Not a priority Low priority Moderate priority High priority
2) Probes:
What do you believe is meant by “commitment to … equity”?
What is meant by “racial-ethnic equity” on your campus specifically?
3) Results:
Comprehension problems: The meaning of the term “racial-ethnic equity” was not
clear for subjects. Different subjects understood this question in different ways.
Furthermore, they understood it not in the same way CUE researchers understand it.
Because of the comprehension problems, I suggest the following change:
4) Suggested revision:
My institution’s commitment to racial-ethnic equity, defined as equal educational
outcomes among all racial-ethnic groups, should be prioritized as…
Or CUE may want to consider a different form of question and response, such as:
CUE advocates equity, defined as equal outcomes among all racial-ethnic groups, as a
standard for measuring institutional performance. To what extent do you support this
view?

167

1) Original form of evaluation questionnaire item:
My institution is doing all it can to support success of students of color.
Disagree Somewhat Disagree Somewhat Agree Agree
2) Probes:
How did you arrive at your answer?
How did you define “success” when answering this question?
3) Results:
Judgment problems: Subjects found it difficult to answer the question because they
were not able to assess the full extent of what their institution was doing to support
success of the students of color: not enough information to make judgment. Some of
them felt “frustrated.”
4) Suggested revision:
Do not include questions that require making uninformed judgments. Or limit to
certain categories of respondents that have enough information to make a judgment.
1) Original form of evaluation questionnaire question:
I feel that I have a lot to learn before I can impact racial/ethnic equity issues on my
campus.
Not at all Somewhat Definitely
2) Probes:
What types of learning do you feel you or your colleagues need to accomplish in order
to make an impact?
3) Results:

168

Comprehension problems: Subjects found it difficult to identify what types of
learning are assumed in the item; CUE researchers do not gain information to address
subjects’ needs.
4) Suggested revision:
In order to make an impact, I feel I need to learn more about (please choose all that
apply):
(a) cultures of students of color (b) CUE’s tools
(c) culturally inclusive pedagogy/practices (d) structural racism
(e) equity/equity indicators (f) other (please indicate):
(g) I don’t feel I need to learn to make an impact
Recommendations for Higher Education Practitioners at MSU and Similar
Institutions
My principal recommendation for higher education practitioners at institutions
similar to MSU pertains to professional learning communities, defined by McLaughlin
and Talbert (2006) as communities of educators who work collaboratively to reflect on
their practices, on the relationship between their practice and student outcomes, and
make changes in practice based on these reflections. The findings of this study point
out that one of the challenges related to the advancement of cultural inclusivity and
equity is that, too often, faculty members experience a sense of isolation within their
community, represented by departments and colleges. Faculty members may become
inclined to incorporation of new practices responsive to their ethnically diverse
students. However, a lack of shared equity value among colleagues tends to affect
these inclinations, ultimately impeding change. Furthermore, scholars argue that

169

teaching is more than a deep understanding of a subject matter (Gay, 2002). Effective
teaching “infuses content with an understanding of the cultural nature of learning”
(King, Artiles, & Kozleski, 2009, p. 5). The results of the study provide evidence and
coincide with Gay’s (2002) assertion that too many educators are inadequately
prepared to teach racially and ethnically diverse students. There is a necessity of
responding to the strengths and needs that students from all cultural backgrounds bring
to the classrooms.
This suggests that at institutions like MSU, it is necessary to create cross-
institutional inquiry groups providing an opportunity for practitioners to collectively
review their practices, to reflect, to share ideas on how to improve them, to develop a
shared value, and to break a “silo” of higher education. Creating a supportive
environment where administrators and faculty members can discuss equity and
cultural inclusivity issues openly is important to assist them in learning more about
their own pedagogical practices and students. Therefore, my primary recommendation
to practitioners would be to develop structures in the form of inquiry teams or learning
communities, dedicated to the improvement of pedagogical practices that would
promote organizational learning and change in instruction, and as a result, improve
institutional effectiveness.
The Equity Alliance at Arizona State University (King, Artiles, & Kozleski,
2009) generates the following set of principles to guide experiences of learning in
professional learning communities that foster culturally responsive practices:
1. Professional learning communities are “focused on improving learning within a
racially and ethnically diverse community;”

170

2. Professional learning communities engage practitioners “in joint, productive
activity through discourse and inquiry. Effective professional learning is reached
by collaborative interaction with colleagues through discussion, knowledge
development and inquiry around professional practice;”
3. Professional learning communities foster professional learning that results “in
improved educational outcomes for students who have been marginalized from the
academic and social curricula of the U.S. educational system;”
4. “Professional learning communities influence decisions about what is taught and
why;”
5. Professional learning “is embedded within everyday practice” and “becomes part
of daily discourse, discussions about student learning, study groups, and
examination of evidence from inquiry cycles” (p. 6).
As a tool for learning, professional learning communities offer educators the
opportunity “… to explore their own practice and look for ways to build bridges
between the curriculum, the culture of the institution, and the experiences of their
students” (King, Artiles, & Kozleski, 2009, p. 8). PLC would provide individuals
involved with the necessary institutional data as well as support as they move toward
their goals. They would also offer practitioners the opportunity for continuous inquiry
and self-assessment, reflective thinking and bringing about change in their practices.
Recommendations for the Higher Education as a Field
The primary recommendation for the field of higher education would be to
actively adopt action research methods and tools, “aimed to solve pertinent problems
in a given context through inquiry and collaboration of professional researchers and

171

practitioners” (Greenwood & Levin, 2005). The results of this study show evidence
about the potential of action research processes and tools to bring about change in
educational practices related to racial ethnic equity. As an institutional outsider
conducting action research, CUE activated capacities for organizational learning by
initiating processes and assessment tools for action inquiry on the part of institutional
insiders, who have used and continue to use them in order to assess their own practices
in their pursuit for equity. This study shows that researchers and practitioners, through
working together, can generate knowledge that enables them to take the appropriate
actions to achieve their goals. It is necessary to continue developing a body of
knowledge regarding whether, how, and under what conditions action research is an
effective form of assessment, which allows practitioners to better understand the
nuances of their practices.
Conclusion
In 2010 President Barack Obama, Lumina Foundation, the Bill & Melinda Gates
Foundation, and others committed themselves to “The Big Goal” – increasing the
proportion of American adults with college degrees to 60% by the year 2025 (Harper,
2012). However, just getting more people to college is a necessary but not sufficient
measure to improve rates of postsecondary education attainment (Harper, 2012). As
noted earlier in this paper, the current rates of degree completion for non-Asian
minorities conspicuously lag behind white students across U.S. postsecondary
education. The existence of disparate educational outcomes among different racial-
ethnic groups undermines current efforts to increase the number of college-educated
adults in the United States. Current accountability structures and assessment practices

172

are not effective in bringing about change because racial-ethnic equity is missing from
external and internal accountability structures, and assessment as currently practiced,
is not particularly race-conscious. This study provides evidence that self-assessment in
the form of action inquiry carries potential as an alternative strategy to improve
organizational productivity and outcome equity because it provides practitioners with
the opportunity to learn more about their practices.

173

BIBLIOGRAPHY
Ajzen, I. (1991). The theory of planned behavior. Organizational Behavior and
Human Decision Processes, 50, 179-211.
ARCC/AB 1417. (2010). Retrieved from California Community College System
Office, ARCC.
ARCC/AB 1417. (2007). Retrieved from California Community College System
Office, ARCC.
American Council on Education. (2008). ACE analysis of Higher Education Act
Reauthorization. Washington, D.C.
American Council on Education. (2009). Minorities in Higher Education: 2009
Supplement. Washington, DC.
Baldwin, C., Bensimon, E.M., Dowd, A.C., & Kleiman, L. (2011). Measuring student
success. In Head, R.B. (Ed.), Institutional effectiveness. New Directions for
Community Colleges, 153, 75-88.
Balley, T., Jeong, D.W., & Cho, S.-W. (2010). Referral, enrollment, and completion in
developmental education sequences in community colleges. Economics of
Education Review. 29(2), 255-270.
Bannan-Ritland, B. (2003). The role of design in research: The integrative learning
design framework. Educational Researcher, 32(1), 21-24.
Basic Skills Accountability Supplement to the ARCC Report. (2011). Retrieved from
California Community Colleges Chancellor’s Office: http://cccco.edu

174

Bauman, G.L. (2005). Promoting organizational learning in higher education to
achieve equity in educational outcomes. In A. Kezar (Ed.), Organizational
learning in higher education (Vol. 131). San Francisco: Jossey-Bass.
Bensimon, E.M. (2005). Closing the achievement gap in higher education: an
organizational learning perspective. In A. Kezar (Ed.), Higher education as a
learning organization: Promising concepts and approaches (Vol. 131), San
Francisco: Jossey-Bass.
Bensimon, E.M. (2012). The Equity Scorecard. In Bensimon, E.M. & Malcom, L.
(Ed.), Confronting equity issues on campus. Stylus Publishing, LLC.
Bensimon, E.M., Dowd, A.C., Alford, H., & Trapp, F. (2007). Missing 87: A study of
the “transfer gap: and “choice gap”. Long Beach and Los Angeles: Long
Beach City College and the Center for Urban Education, University of Southern
California.
Bensimon, E. M., Dowd, A.C., Daniels III, J. E . & Walden, D. (2010). Long-term
partners for serving Los Angeles’ African American and Latino students: USC’s
Center for Urban Education and Los Angeles Southwest College. College
bound: Strategies for access and success for low-income students. Los Angeles:
USC Office of Government and Civic Engagement.
Bensimon, E.M., Polkinghorne, D.E., Bauman, G.L., & Vallejo, E. (2004). Doing
research that makes a difference. Journal of Higher Education, 75(1), 104-126.
Burns, D. (2007). Systemic action research: A strategy for whole system change. The
Policy Press University of Bristol.

175

Bustillos, L.T. (2007). Dissertation: Exploring faculty beliefs about remedial
mathematics students: A collaborative inquiry approach. Rossier School of
Education, University of Southern California.
Bustillos, L.T., Rueda, R., & Bensimon, E.M. (2011). Faculty views of
underrepresented students in community college settings: Cultural models and
cultural practices. In P.R. Portes & S. Salas (Eds.), Vygotsky in 21
st
century
society: Advances in cultural historical theory and praxis with non-dominant
communities. New York: Peter Lang.
Center for Student Success. (2007). Basic skills as a foundation for success in
California community colleges. Sacramento, CA: California Community
Colleges Chancellor’s Office. Retrieved from
http://www.cccbsi.org/publications
Center for Urban Education. (2011). About CUE. Retrieved from www.usc.edu
Charles, L., & Ward, N. (2007). Generating change through research: Action research
and its implications. Centre for Rural Economy Discussion Paper Series, 10.
University of Newcastle upon Tyne.
Cobb, P., Confrey, J., diSessa, A., Lehrer, R., & Schauble, L. (2003). Design
experiments in educational research. Educational Researcher, 32(1), 9-13.
Cook, B. & Pullaro, N. (2010). College graduation rates: Behind the numbers.
American Council on Higher Education, Washington, D.C.
Council for Higher Education Accreditation 2009-2010 Annual Report. Washington,
DC.

176

Dowd, A.C. (2005). Data don’t drive: Building a practitioner-driven culture of
inquiry to assess community college performance. Indianapolis, Ind.: Lumina
Foundation for Education.
Dowd, A.C. (2007). The community college as gateway and gatekeeper: Moving
beyond the access “saga” to outcome equity. Harvard Educational Review,
77(4), 407-419.
Dowd, A.C., & Bensimon, E.M. (2009). CUE frameworks for action, research, and
evaluation. Los Angeles: Center for Urban Education.
Dowd, A.C., & Bensimon, E. M. (2009). The social learning theory and tools
underlying CUE’s facilitation of action inquiry for equity. Los Angeles: Center
for Urban Education.
Dowd, A.C., Bishop, R., Bensimon, E.M., & Witham, K. (2012). Accountability for
equity in postsecondary education. In K. S. Gallagher, R. Goodyear, D. J.
Brewer & R. Rueda (Eds.), Urban education: A model for leadership and policy.
New York: Routledge.
Dowd, A.C., Malcom, L.E., Nakamoto, J., & Bensimon, E.M. (2007). Institutional
researchers as teachers and equity advocates: Facilitating organizational
learning and change. Paper presented at the Associating for the Study of Higher
Education.
Dowd, A.C., & Tong, V.P. (2007). Accountability, assessment, and the scholarship of
“best practice”. In J.C. Smart (Ed.), Handbook of Higher Education (Vol. 22,
pp. 57-119): Springer Publishing.

177

Dougherty, K.J. & Hong, E. (2005). State systems of performance. Accountability for
community colleges: Impacts and lessons for policymakers. Community College
Research Center, Teachers College, Columbia University.
Echeverria, J. (2010). The CSU Graduation Initiative: Closing the Achievement Gap.
CSU Board of Trustees Meeting, January 26-27, 2010.
Enciso, M. (2009). Dissertation: The influence of organizational learning on inquiry
group participants in promoting equity at a community college. Rossier School
of Education, University of Southern California.
Fishbein, M., & Ajzen, I. (2010). Predicting and changing behavior: The reasoned
action approach. New York: Psychology Press.
Frehill, L.M., DiFabio, N.M., and Hill, S.T. (2008). Confronting the “New” American
Dilemma, Underrepresented Minorities in Engineering: A Data-Based Look at
Diversity. National Action Council for Minorities in Engineering (NACME).
New York.
Gay, G. (2000). Culturally responsive teaching: Theory, research, & Practice. New
York: Teachers College Press.
Gonzales, R. (2009). Dissertation: Achieving equity in educational outcomes through
organizational learning: Enhancing the institutional effectiveness of community
colleges. Rossier School of Education, University of Southern California.
Greenwood, D.J., & Levin, M. (2005). Reform of the social sciences and of
universities through action research. In N. K. Denzin & Y. S. Lincoln (Eds.),
Handbook of qualitative research (3rd ed., pp. 43-64). Thousand Oaks: Sage
Publications.

178

Grubb, W.N., & Badway, N.N. (2005). From compliance to improvement:
Accountability and assessment in California community colleges: Higher
Education Evaluation and Research Group.
Gutierrez, K. D., Morales, P. Z., & Martinez, D. C. (2009). Remediating Literacy:
Culture, Difference, and Learning for Students from Nondominant
Communities. Review of Research in Education, 33, 212-245.
doi:10.3102/0091732X08328267
Gutierrez, K. D., & Vossoughi, S. (2010). Lifting off the ground to return anew:
Mediated praxis, transformative learning, and social design experiments.
Journal of Teacher Education, 61(1-2), 100-117.
doi:10.1177/0022487109347877
Harnisch, T.L. (2011). Performance-based funding: A re-emerging strategy in public
higher education financing. A Higher Education Policy Brief. American
Association of State Colleges and Universities.
Harper, S.R. (2012). Black male student success in higher education: A report from
the National Black Male College Achievement Study. Philadelphia: University
of Pennsylvania, Center for the Study of Race and Equity in Education.
Head, R.B. (2011). Institutional effectiveness (Vol. 153 (Spring)). San Francisco:
Jossey-Bass.
Herr, K., & Anderson, G.L., (2005). The action research dissertation. Thousand Oaks:
Sage Publications.
Howard, T.C. (2003). Culturally relevant pedagogy: Ingredients for critical teacher
reflection. Theory Into Practice, 42(3), 195-202.

179

Higher Education Opportunity Act. (2008). Public Law 110-315.Washington, DC.
Jones, C.P. (2000). Levels of racism: A theoretic framework and a gardener's tale.
American Journal of Public Health, 90(8), 1212-1215.
Kagan, N., Krathwohl, D.R., & Miller, R. (1963). Stimulated recall in therapy using
video tape: A case study. Journal of Counseling Psychology, 10(3), 237-243.
Doi: 10.1037/h0045497
Kane, R., Sandretto, S., & Heath, C. (2002). Telling half the story: A critical review of
research on the teaching and practices of university academics. Review of
Educational Research, 72(2), 177-228.
King, K.A., Artiles, A.J., & Kozleski, E.B. (2009). Professional learning for culturally
responsive teaching. Retrieved from www.equityalliaceatasu.ord
Kruse, S.D. (2001). Creating communities of reform: Continuous improvement
planning teams. Journal of Educational Administration, 39(4), 359-383.
Kruse, S.D., & Louis, K.S. (1997). Teacher teaming in middle schools: Dilemmas for
a school-wide community. Educational Administration Quarterly, 33(3), 261-
289. Doi: 10.1177/0013161X97033003002
Lincoln, Y.S. & Guba, E.G. (1985). Naturalistic inquiry. Newbury Park, CA: Sage
Publications.
Luke, A., Green, J., and Kelly, G.J. (2010). What counts as evidence and equity?
Review of Research in Education, 34, vii-xvi. doi:10.3102/0091732X09359038
Margolis, J. (2008). Stuck in the shallow end: Education, race, and computing.
Cambridge, MA: The MIT Press.

180

McArdle, K.L., & Reason, P. (2007). Action research and organizational development.
In T.C. Cummings (Ed.), Handbook of organizational development. Thousand
Oaks, CA: Sage Publications.
McLaughlin, M.W., & Talbert, J.E. (2006). Building school-based teacher learning
communities. New York: Teachers College Press.
Melguizo, T., Hagedorn, L.S., & Cypers, S. (2008). Remedial/developmental
education and the cost of community college transfer: A Los Angels county
sample. The Review of Higher Education, 31(4), 401-431.
Mills, G. (2003). Action research: A guide for the teacher researcher (2
nd
ed.). Upper
Saddle River, NJ: Merrill/Prentice Hall.
Moll, L. C. (2000). Inspired by Vygotsky: Ethnographic experiments in education. In
D. D. Leer and P. Smagorisky (Eds.), Vygotskian perspectives on literacy
research Cambridge: Cambridge University Press.
Moore, C., & Shulock, N. (2007). Beyond the open door: Increasing student success
in the California community colleges. Sacramento: Institute for Higher
Education Leadership and Policy, California State University.
Nasir, N.S., & Hand, V. M. (2006). Exploring sociocultural perspectives on race,
culture, and learning. Review of Educational Research, 76(4), 449-475.
National Center for Public Policy and Higher Education. (2008). Measuring up 2008:
The state- by-state report card for higher education. Washington, DC.
National Center for Public Policy and Higher Education & National Center for Higher
Education Management Systems. (2010). Strengthening college opportunity and
performance: Federal, state, and institutional leadership. A policy report from

181

The Delta Project on Postsecondary Education Costs, Productivity and
Accountability. San Jose, California.
National Commission on Accountability in Higher Education. (2005). Accountability
for better results: A national imperative for higher education.
Nikol Alexander-Floyd. (2008). Critical race pedagogy: Teaching about race and
racism through legal learning strategies. PS: Political Science & Politics,
41, 183-188 doi:10.1017/S1049096508080281
Noffke, S.E. (1997). Professional, personal, and political dimensions of action
research. Review of Educational Research, 22, 305-343.
Nolen, A., & Putten, V. (2007). Action research in education: Addressing gaps in
ethical principals and practices. Educational Researcher, 36(7), 401-407. doi:
10.3102/0013189X07309629
Ogawa, R., Crain, R., Loomis, M., & Ball, T. (2008). CHAT-IT: Toward
conceptualizing learning in the context of formal organizations. Educational
Researcher, 37(2), 83-95.
Parsons, S. (2010). Review of Margolis Jane, Estrella Rachel, Goode Joanna, Holme
Jennifer Jellison, and Nao Kimberley “Stuck in the Shallow End: Education,
Race and Computing.” The Knowledge Engineering Review, 25, 109-110.
doi:10.1017/S0269888909990373
Patton, M.Q. (2002). Qualitative research and evaluation methods. Thousand Oaks,
CA: Sage Publications.
Patton, M.Q. (2011). Developmental evaluation: Evaluating complexity concepts to
enhance innovation and use. New York: The Guilford Press.

182

Phillips, R., Bain, J., McNaught, C., Rice, M., & Tripp, D. (2000). Handbook for
learning-centered evaluation of computer-facilitated learning projects in higher
education. Murdoch University.
Ratner, C. (2002). Cultural psychology: Theory and method. New York: Plenum.
Reason, P. (1994). Three approaches to participative inquiry. In N. K. Denzin & Y. S.
Lincoln (Eds.), Handbook of qualitative research (pp. 324-339). Thousand
Oaks: Sage Publications.
Reason, P. (2001). Learning and change through action research. In J. Henry (Ed.),
Creative Management (pp. 182-194). London: Sage Publications.
Reason, P., & Bradbury, H. (2001). Handbook of action research. London: Sage
Publications.
Rendón, L.I. (1994). Validating culturally diverse students: Toward a model of
learning and student development. Innovative Higher Education, 19(1), 33–51.
Robinson-Armstrong, A., Clemons, A., Fissinger, M., & Sauceda, M. (2012). The
diversity scorecard at Loyola Marymount University: An exemplary model of
dissemination. In Bensimon, E.M. & Malcom, L. (Ed.), Confronting equity
issues on campus. Stylus Publishing, LLC.
Roth, W.-M., & Lee, Y.-L. (2007). “Vygotsky's neglected legacy”: Cultural-historical
activity theory. Review of Educational Research, 77(2), 186-232.
doi:10.3102/00 34654306298273
Salazar-Romo, C. (2009). Dissertation: Remediating artifacts: Facilitating a culture of
inquiry among community college faculty to address issues of student equity
and access. Rossier School of Education, University of Southern California.

183

Schon, D.A. (1983). The reflective practitioner: How professionals think in action.
Basic Books, Inc.
Stillman, J. (2011). Teacher learning in an era of high-stakes accountability:
Productive tension and critical professional practice. Teachers College Record,
113(1), 133-180.
Stringer, E.T. (2007). Action research. Thousand Oaks, CA: Sage Publications, Inc.
Sue, D.W. (2010). Microagressions in Everyday Life: Race, Gender, and Sexual
Orientation. John Wiley & Sons, Inc., Hoboken, New Jersey.
Suter, W.N. (2006). Introduction to educational research: A critical thinking
approach. Thousand Oaks, CA: Sage.
Tharp, R.G., & Gallimore, R. (1988). Rousing minds to life: Teaching, learning, and
schooling in social context. Cambridge: Cambridge University Press.
The Design-Based Research Collective (2003). Design-based research: An emerging
paradigm for educational inquiry. Educational Researcher, 32(1), 5-8.
University of California Office of the President. (n.d.). Major Features of the
California Master Plan for Higher Education. Retrieved from
www.ucop.edu/acadinit/mastplan/mpsummary.htm
USC California Policy Institute. (2005). Measuring equity in higher education: An
accountability framework for California colleges and universities. Retrieved
from http://www.usc.edu
U.S. Department of Education. A test of leadership: Charting the future of higher
education. Washington, D.C.: U.S. Government Printing Office, 2006.

184

Wang, C.C. (1999). Photovoice: A participatory action research strategy applied to
women’s health. Journal of Women’s Health, 8(2), 182 –192.
Willis, G. B. (2005). Cognitive interviewing. Thousand Oaks: Sage Press.
Yosso, T. J. (2002). Toward a critical race curriculum. Equity & Excellence in
Education, 35(2), 93-107.
Yosso, T. J. (2005). Whose culture has capital? A critical race theory discussion of
community cultural wealth. Race, Ethnicity and Education, 8 (1), 69-91.

185

APPENDIX A
Recruitment Text and Ethical Commitments for Interactions with Human
Subjects
Dear Colleague,

The Center for Urban Education (CUE) at the University of Southern California
is currently conducting a developmental evaluation study. The study has two main
goals. First to develop our evaluation capacity by improving the validity of the
inferences we draw from our workshop evaluation forms and other evaluative
processes and, second, to improve our effectiveness in conducting action research for
the purposes of improving equity in higher education. Therefore, we are interested in
gaining a more nuanced understanding of the experiences of workshop participants
and Equity Scorecard evidence team members who engage with us in action research
projects.
As a participant in a CUE workshop or evidence team, you may be asked to take
part in an interview or a focus group conducted by a doctoral student who is part of the
CUE Evaluation Study research team. The doctoral students will also collect
observational data at workshops and team meetings. You can decline to participate in
an interview or a focus group, or request to be omitted from the data collected during
workshop and team observations.
That said, I am hopeful you are willing to support and contribute to CUE’s
developmental evaluation study and our goal of better understanding you and your
colleague’s reactions, experiences, reflections, and action steps and the extent to
which these were facilitated through our action research processes and tools.
In conducting this study, we make the following commitments: (1) To respect
your professionalism and privacy by conducting the study in a confidential and ethical
manner; (2) To use your time wisely and well, and to minimize the “response burden”
on any one individual; (3) To report findings anonymously to external audiences, for
example in dissertations or evaluation reports; (4) To share findings with you and your
colleagues in ways informative to your learning process. We will not report findings in
ways that would reveal the experience of any one individual (for example based on his
or her race, ethnicity, gender, or position). Instead, we will draw on findings from
multiple participants on your campus or aggregated across different field sites to
communicate themes or issues that are pertinent in your setting.

Should you have any questions or concerns, or should any arise as we conduct this
study, please contact me by phone or email: 213.740.5202, alicia.dowd@usc.edu.
Thank you for your consideration!

Yours sincerely,
Alicia C. Dowd

Associate Professor
Co-Director
Center for Urban Education

186

APPENDIX A

CUE’s Developmental Evaluation Study – At a Glance

Q. If I participate in this study, what would I be asked to do?
A. You would be asked to do one or more of the following:
Complete an evaluation form at the conclusion of a workshop or evidence team
meeting in which you were a participant (approximately 15 minutes)
Participate in one or more individual telephone or in-person interviews following a
workshop or team meeting (approximately 40 minutes each)
Participate in a focus group (approximately 1.0 to 1.5 hours long)
Provide non-confidential materials from your work to illustrate professional practices
on your campus (e.g. a syllabus, an application form, an assessment form, a campus
report) and changes that take place over the course of the study.

Q. What if I participate in one of the activities indicated above, but don’t want to
participate in the others.
A. That is fine. You may decline to participate at any time.

Q. What methodology are you using?
A. Our study is best characterized as developmental evaluation, a methodology that is
appropriate when the organization conducting the study operates in a complex,
dynamic environment and is interested in developing innovative and responsive
processes that will function well in those environments.
Developmental evaluators use a variety of methods, data, and analysis techniques. We
will triangulate data from evaluation forms, observations, interviews, focus groups and
documents. The interviews will take a particular form called “cognitive interviewing.”
These are “think aloud” interviews where you explain how you interpreted and
answered the questions on the evaluation form. This will enable us to improve the
quality of the data we collect from this simple and quick evaluation tool.

Q. Who else is involved in this study?
A. Currently, faculty, administrators, and counselors at twelve community colleges,
two state universities, and two liberal arts colleges (all in California) are being invited
to participate. We anticipate having 10 to 30 participants per site at 10 of these sites,
with the number depending on the total number of participants in CUE workshops or
evidence teams. It is not necessary for everyone who has participated in a workshop or
team meeting at a particular campus to participate in the evaluation study

187

APPENDIX B

CUE’s Tool or Workshop/Webinar Evaluation Questionnaire Sample
Webinar Evaluation Form

1. I found CUE’s webinar/workshop to be engaging.

Strongly Disagree Disagree Neither Agree Nor Disagree Agree Strongly Agree

2. The webinar provided a helpful understanding of CUE’s syllabus review tool.

Strongly Disagree Disagree Neither Agree Nor Disagree Agree Strongly Agree

3. I would have preferred an in-person presentation to the webinar.

Strongly Disagree Disagree Neither Agree Nor Disagree Agree Strongly Agree

4. I have a strong sense of my own personal responsibility surrounding racial and
ethnic issues on my campus.

Strongly Disagree Disagree Neither Agree Nor Disagree Agree Strongly Agree

5. The group activity encouraged an environment of reflection for most of my
colleagues.

Strongly Disagree Disagree Neither Agree Nor Disagree Agree Strongly Agree

6. Overall, today’s workshop caused me to reflect in new ways on my own
pedagogical practices.

Strongly Disagree Disagree Neither Agree Nor Disagree Agree Strongly Agree

7. Prior to attending today’s webinar/workshop, I have reflected on ways to make my
pedagogical materials as culturally inclusive as possible.

Strongly Disagree Disagree Neither Agree Nor Disagree Agree Strongly Agree

8. Due to today’s webinar and activities, I intend to make changes to my own syllabus
and/or course materials.

Strongly Disagree Disagree Neither Agree Nor Disagree Agree Strongly Agree

9. I found CUE’s Syllabus Analysis Tool to be valuable to my own work.

Strongly Disagree Disagree Neither Agree Nor Disagree Agree Strongly Agree

188

APPENDIX C
Observational Data Collection Template
TIME PERIOD/TASK (#)
Site
(room temp, equipment,
environment,
“artifacts”)
Mood
(emotions, general
attitudes, personality
traits)
“Task” Performance
(CUE Tools; knowledge
base for engagement
with presentation or
tool; expressed attitudes
towards or beliefs about
tool; e.g. use or value or
design)
Social Context
(Who is there (social
markers), positions/
authority relations; race
relations; interactions,
who talking)
Behavioral Intentions
(expressed next steps,
plans, norms)
Environmental
Constraints
(expressed concerns or
hopes, perceived
limitations of self, team
or resources)

Reflection/Analysis

189

APPENDIX D
Cognitive Interview Protocol (used after the Document Analysis webinar)
Instructions to be read to the Participant (Willis, 2005):

Either read these instructions in their entirety or paraphrase them, however include
elements from each item. “Thank you for taking the time to talk with me today. Let
me tell you a little more about what we will be doing today.”
We’re evaluating the survey items you completed from the webinar/workshop survey
as well as your experience throughout this process.
I’ll ask you questions and you answer them, just like a normal survey.
However, my goal here is to get a better idea of how the questions are working.
Therefore, I would like you to think aloud as you answer the questions, in other words
tell me everything you are thinking about as you go about answering them.
At times I’ll stop and ask you more questions about the terms or phrases in the
questions and what you think a question is asking about. I will also be taking notes.
Please keep in mind that I want to hear all of your opinions and reactions. Do not
hesitate to speak up whenever something seems unclear, is hard to answer, or doesn’t
seem to apply to you.
Finally, we will do this for 20 to 30 minutes, unless I run out of thing to ask you
before then.
Do you have any questions before we begin?

Cognitive Interview Protocol

Date: Interview # : Interviewer initials:
Start Time of Interview:

“For each following statement, you were asked to indicate the response that best
reflects your opinion and experiences. Please try and ‘think aloud” this time, as you
answer each question.”

Q3. I would have preferred an in-person presentation to the webinar.
Probes:

How did you arrive at your answer?
What did you prefer or not prefer about the webinar?
Additional notes:

190

Q5. The group activity encouraged an environment of reflection for most of my
colleagues.
Probes:

How did you arrive at your answer?
What do you believe is meant by “reflection”?
If you agreed, in what ways did the activity encourage group reflection?
Additional notes:

Q6. Overall, today’s workshop caused me to reflect in new ways on my own
pedagogical practices.
Probes:

How did you arrive at your answer?
What are “new” ways of reflection that could be generated from a single workshop?
Additional notes:

Q7. Prior to attending today’s webinar/workshop, I have reflected on ways to make
my pedagogical materials as culturally inclusive as possible.

Probes:

How did you arrive at your answer?
What do you believe is meant by “culturally inclusive?”
What do you think is meant by your reflection of your own materials, specifically?
Additional notes:

Q9. I found CUE’s Syllabus Analysis Tool to be valuable to my own work.
Probes:

How did you arrive at your answer?
What do you believe is meant by “valuable” to your own work, specifically?
Additional notes:

Q10. I believe that culturally inclusive changes I make in my classroom can have
significant effects on my students’ success.
Probes:
How did you arrive at your answer?
What do you believe is meant by “culturally inclusive changes in your own
classroom”?
What is meant by “significant effects” for students?
Additional notes:

191

Q11. I have a strong sense of my own personal responsibility surrounding racial and
ethnic issues on my campus.
Probes:

How did you arrive at your answer?
What do you believe is meant by “personal responsibility”?
What would this look like on your campus specifically?
Additional notes:

Q12. My similarly minded colleagues and I have the ability to make a positive impact
surrounding equity at my institution.
Probes:

How did you arrive at your answer?
What do you believe is meant by “similarly minded colleagues”?
What is meant by “positive impact” on your campus specifically?
What would an “impact surrounding equity” at your institution look like?
Additional notes:

Lastly, is there anything else you can remember regarding your own thinking
processes while completing the evaluation?

During the workshop, what was your reaction to (BLANK: fill-in an incident, context,
etc., from your observational notes)?

What was your own thinking during this incident/activity/etc?

Is there anything else you would to share regarding your own processing during the
workshop?

(Paraphrase if you prefer) Thank you for taking the time to help CUE improve our
work with Cal Poly. Do you have any further questions or concerns?

Cognitive Interview Analysis

Major categories of outcomes that emerge from cognitive testing (Willis, 2005):

1. Item-specific recommendations for changes to working (cognitive,
logical/structural, culturally oriented defects).
Were there any items found to have consistent deficits across respondents? Across a
single respondent, however valid and helpful? Based on this feedback what changes
would you recommend in order to improve the survey? (e.g., specifying the term
“success” to mean graduation).

192

2. Need for further specification of objectives or the manner in which the questions
satisfy them.
Referring back to our analytic objectives of our evaluation, how well did the questions
map onto these objectives? Were the items too vague (e.g. items focusing on societal
issues when the focus should be focusing on institutional issues)? Were they
comprehensive, in other words, cover the span of the objectives? Were any items
outside of parameters of the analytic objectives? If so, do they add to the data, or are
they off base? What do you recommend surrounding their inclusion or removal?

3. Problems related to ordering (or items, sections, and so on) and other
interactions between survey questions.
Do you feel that the order of the items negatively affect how the respondents process
the overall instrument? Does one item lead to a possible bias for a later item? Critical
items and more specific items covering the same concepts should be included first. Is
this the case? If not, what do you recommend in terms of changing the order? Why?

4. Problems related to reduction in overall instrument length or burden.
Did you get the sense that the instrument was too long for your respondents? Did they
lose interest and attentiveness by the end of the process? If so, how do you
recommend shortening it? If so, in what ways could the instrument be less cognitively
demanding on your respondents?

5. Limitations on what can be asked of survey respondents using the intended
procedures.
Are the topics reviewed in the module too dense or abstract it would require an
unreasonable number of items in order to adequately collect the data? Were your
respondents able to process the information in the module or recall the information
enough to legitimately answer the survey items? If not, what do you recommend (e.g.
remove items).

[1] See Patton, M. Q. (2011) Developmental Evaluation: Applying Complexity
Concepts to Enhance Innovation and Use. New York: Guilford Press

193

APPENDIX E
Individual Interview Protocol

Date: Interview #: Interviewer initials:
Start Time of Interview:
Interviewee:

Instructions to be read to the participant. Either read these instructions in their entirety
or paraphrase them. It is more important that you work with what you hear as opposed
to following the protocol.

Opening Statement:

“Thank you for taking the time to talk with me today. As you know I am collecting
data from workshop participants, such as yourself, in order to improve the
effectiveness of CUE tools for future workshops. Additionally, I hope to gain insight
into your experience as a workshop and action research participant. This data may also
be used for my dissertation. However, similar to our last interview, data reported will
not reveal any identifying information. I’d like to use this opportunity to recap what
we discussed the last time we spoke and expand on changes that have occurred in the
meantime. I appreciate your sharing with me, it continues to be very helpful.”

Question:

“Are there any ways you have incorporated CUE’s work into your own? Any changes
in your own behaviors or perhaps intentions to do so?”

Probes:

If yes:
“Are there any documents that demonstrate these changes?
“Would you mind sharing them with me and CUE at some point?”

If no:
“Why do feel this might be the case?”

Question:

“Have you noticed any changes in how you view issues related to racial/ethnic equity
on your campus? For example, do you feel that you see things differently or have a
stronger sense or agency towards moving your campus forward on this level?”

Probes:

If yes:

194

“Can you describe what lead to these changes?”

Question:

“Since the last time spoke have you noticed any changes in the focus of your
colleagues around equity or other topics discussed at the workshops? For example, has
there been any noticeable discussion around this/these topics, or changes in attitudes
or behaviors?”

Probes:

If yes:
“In your opinion, what caused this change?”

If no:
“Why do feel this might be the case?”

Question:

“Similar to the last question, have you noticed any changes at the level of your
institution (culture or policy) surrounding equity issues or other areas touched upon at
the workshops?”

Probes:

If yes:
“In your opinion, what caused this change?”

If no:
“Why do feel this might be the case?”

Conclusion:

Is there anything else you would to share regarding your own processing or noticeable
changes at your colleague or campus level?

Thank you for taking the time to help CUE improve our work with Cal Poly. Do you
have any further questions or concerns?

195

APPENDIX F

Document Analysis for Self-Assessment of Culturally Inclusive Practices Protocol
(used at the Document (Syllabus) Analysis workshop)

Document Analysis for Self-Assessment of
Culturally Inclusive Practices

Purpose: This document is designed for individual and collective use to facilitate self-
assessment into the ways a college or university uses (or could use) culturally
inclusive practices
2
to assist underrepresented students, particularly Latinos, Latinas,
and African Americans (who are not represented among college graduates to an extent
proportional to their representation in the U.S. population), succeed in college. It is
designed to be used as part of a series of steps involving administrators, student affairs
professionals, and faculty in inquiry, self-assessment, and action planning.

At many campuses, this document analysis will follow a review of student persistence
and success at key milestones in a matriculation process (e.g. application, assessment
testing, admissions, placement in a curriculum) or in a curriculum (e.g. gateway
courses, credit accumulation, courses required for a major or degree completion,
timely graduation). A group of people in varying roles (e.g. admissions/matriculation,
administration, faculty, student affairs) agree to assess a sampling of the documents
they use in communicating with students as “artifacts of culture,” in other words to
look at the sampled documents “with fresh eyes” to consider what kinds of cultural
assumptions they hold. The group can be convened within a functional area (e.g.
admissions office, academic department, or student service program) or be cross-
functional.

The process involves each participant first in individually assessing a sampling of their
own documents of educational practice using the indicators in the table below and then
in talking with the group about what they learned or contemplated while conducting
the document review. The response columns alongside each of the indicators asks
whether the culturally inclusive practice represented by the indicator is reflected in the
document, whether it should or could be (if not already), and whether the person
completing the review would be willing to take steps to change the document if she or
he believes changes are warranted. Drawing on these reflections, each participant
volunteers ideas to draft an action plan to strengthen culturally inclusive practices on
the campus.

2
Term derived from concepts adapted from culturally responsive, culturally relevant, or culturally inclusive
pedagogy. See, for one key reference, Ladson-Billings, G. (1995). Toward a theory of culturally relevant
pedagogy. American Educational Research Journal, 32(3), 465-491.

196

Getting Started

1. Collect a sampling of materials that reflect your educational practices and that
communicate expectations and information to students; for example:

For admissions and enrollment professionals: Application forms, program recruitment
brochures, web pages, financial aid forms
For faculty: syllabi, handouts from class, assignments, assessments of student
learning, course or personal web sites (if provided to students)
For program administrators and institutional leaders: descriptions of student eligibility,
mission statements and program goals, speeches, PowerPoint presentations,
newsletters and e-blasts

2. Keep in mind that printed materials are only one important way that you and your
campus communicate with students. Individual discussions, in class presentations or
group discussions, campus gatherings, artwork and the physical space are other
important forms of communication. (The Center for Urban Education offers other
protocols for self-assessment of practices represented in other types of media and
settings.)

3. Scan the materials to re-familiarize yourself with their contents.

4. With the materials at hand, proceed to complete the worksheet provided by the table
below.

197

Self-Assessment using Indicators of Culturally Inclusive Practices

Use the indicators in the first column to reflect on the sample of materials you have
collected. Then, use the second column to jot down your notes based on your
reflections prompted by the indicators. Finally in the third column, note steps you
would be willing and able to take to make changes to the documents, in those cases
where you believe change is warranted.

Refer to the
following
indicators of
culturally
inclusive practices
and consider
whether the
documents in your
sample could be
characterized as
communicating…

Based on your review of the
sampled documents, would
you say that they can be
characterized by the
indicator in the first
column?
Would you be willing
and able to take steps
to make changes to
the materials you
reviewed, if you
believe changes are
warranted?
1. Respect for
students

Yes / No/Somewhat

If Yes or Somewhat,
How do they?

If No, Should they?
Yes / No
If Yes, How could they be
revised to communicate this?

Yes / No

If Yes, how?
Note possible steps:

If No, Why not?

198

2. Desire to help
students succeed.
Yes / No/Somewhat

If Yes or Somewhat,
How do they?

If No, Should they?
Yes / No
If Yes, How could they be
revised to communicate this?
Yes / No

If Yes, how?
Note possible steps:

If No, Why not?

3. Information
students need to
successfully act on
the information in
the document.
Yes / No/Somewhat

If Yes or Somewhat,
How do they?

If No, Should they?
Yes / No
If Yes, How could they be
revised to communicate this?
Yes / No

If Yes, how?
Note possible steps:

If No, Why not?

4. Validation of
racial diversity
Yes / No/Somewhat

If Yes or Somewhat,
How do they?

If No, Should they?
Yes / No
If Yes, How could they be
revised to communicate this?
Yes / No

If Yes, how?
Note possible steps?

If No, Why not?

5. Validation of
diversity in terms
of ethnicity
(shared ancestry,
language, national
heritage, religious
beliefs,
community norms)
Yes / No/Somewhat

If Yes or Somewhat,
How do they?

If No, Should they?
Yes / No
If Yes, How could they be
revised to communicate this?
Yes / No

If Yes, how?
Note possible steps:

If No, Why not?

6. Validation of
other forms of
diversity,
such as:
Gender
Socio-economic
status
Yes / No/Somewhat

If Yes or Somewhat,
How do they?

If No, Should they?
Yes / No
Yes / No

If Yes, how?
Note possible steps:

If No, Why not?

199

Sexual orientation
Immigration status
Age
If Yes, How could they be
revised to communicate this?
7. Validation of
the value of
differences in prior
educational
experiences
Yes / No/Somewhat

If Yes or Somewhat,
How do they?

If No, Should they?
Yes / No
If Yes, How could they be
revised to communicate this?
Yes / No

If Yes, how?
Note possible steps:

If No, Why not?

8. Validation of the
value of
differences in prior
life experiences
Yes / No/Somewhat

If Yes or Somewhat,
How do they?

If No, Should they?
Yes / No
If Yes, How could they be
revised to communicate this?
Yes / No

If Yes, how?
Note possible steps:

If No, Why not?

9. Expectation that
student success
will be a
collaborative effort
among the student,
peers, faculty,
administrators,
counselors,
students’ families
and communities
Yes / No/Somewhat

If Yes or Somewhat,
How do they?

If No, Should they?
Yes / No
If Yes, How could they be
revised to communicate this?

Yes / No

If Yes, how?
Note possible steps

If No, Why not?

10. Belief that all
students are
capable of
obtaining high
educational goals.

Yes / No/Somewhat

If Yes or Somewhat,
How do they?

If No, Should they?
Yes / No
If Yes, How could they be
revised to communicate this?
Yes / No

If Yes, how?
Note possible steps:

If No, Why not?
11. Expectation
that students will
be empowered as
Yes / No/Somewhat

If Yes or Somewhat,
Yes / No

If Yes, how?

200

agents of social
and community
well being through
their education
How do they?

If No, Should they?
Yes / No
If Yes, How could they be
revised to communicate this?
Note possible steps:

If No, Why not?

12. Expectation
that the educational
community will
engage in
respectful
discussion of the
history and
contemporary
experiences of
discrimination,
racism, and
marginalization
Yes / No/Somewhat

If Yes or Somewhat,
How do they?

If No, Should they?
Yes / No
If Yes, How could they be
revised to communicate this?
Yes / No

If Yes, how?
Note possible steps:

If No, Why not?

13. Recognition
that aspects of the
educational
experience are
challenging but
attainable

Yes / No/Somewhat

If Yes or Somewhat,
How do they?

If No, Should they?
Yes / No
If Yes, How could they be
revised to communicate this?
Yes / No

If Yes, how?
Note possible steps:

If No, Why not?

14. Expectation
that learning takes
place in with
authentic
application to and
engagement with
real-world
problems

Yes / No/Somewhat

If Yes or Somewhat,
How do they?

If No, Should they?
Yes / No
If Yes, How could they be
revised to communicate this?
Yes / No

If Yes, how?
Note possible steps:

If No, Why not?

15. Expectation
that education
starts from
students’
experiences and
builds upon those
Yes / No/Somewhat

If No, Should they?
Yes / No

If Yes, How could they be
Yes / No
If Yes, how?
Note possible steps:

If No, Why not?

201

experiences to
expand student
learning
revised to communicate this?

16. Expectation
that broad
integrative learning
will take place,
focused by
engagement with
big questions, both
contemporary and
enduring
3

Yes / No/Somewhat

If Yes or Somewhat,
How do they?

If No, Should they?
Yes / No
If Yes, How could they be
revised to communicate this?
Yes / No

If Yes, how?
Note possible steps:

If No, Why not?

17. Expectation
that analytical,
applied and
integrative learning
will occur across
all major fields,
both pre-
professional and
liberal arts and
sciences
Yes / No/Somewhat

If Yes or Somewhat,
How do they?

If No, Should they?
Yes / No
If Yes, How could they be
revised to communicate this?
Yes / No

If Yes, how?
Note possible steps

If No, Why not?

18. Expectation for
active involvement
with diverse
communities, real
world problems,
and social
responsibility

Yes / No/Somewhat

If Yes or Somewhat,
How do they?

If No, Should they?
Yes / No
If Yes, How could they be
revised to communicate this?
Yes / No

If Yes, how?
Note possible steps:

If No, Why not?

19. Expectation of
milestone and
culminating
experiences that
connect general,
major, and field-
learning contexts
Yes / No/Somewhat

If Yes or Somewhat,
How do they?

If No, Should they?
Yes / No
If Yes, How could they be
revised to communicate this?
Yes / No

If Yes, how?
Note possible steps:

If No, Why not?

3
Indicators 16-18 are adapted from indicators of High Impact Practices from AAC&U’s LEAP Campus Toolkit,
available at http://leap.aacu.org/toolkit/

202

Group Discussion and Action Planning

After group members have had a chance to assess their sampling of documents, the
group gathers to discuss their insights and to share possible action steps. These are
collected and displayed in the categories of “within reach,” “with a stretch” and
“beyond reach” and mapped to key milestones in a student’s educational progress at
the institution. Milestones where racial-ethnic groups experience substantial
differences in progress and success will be highlighted as potential intervention zones
for focused action by the group.

During Group Discussion, Keep in Mind:

Self-assessment differs from evaluation in that it is focused on problem-framing,
experimentation, and solution generation, not on evaluating your performance or the
performance of your colleagues.
In discussing any document or statement, use the same language you would if
your intent were to provide constructive feedback to a colleague sitting next to you.
To support your interpretations, point to specific language in the sampled
documents to provide “data” for your interpretation.
Refer to specific language as revealing or suggesting that the document
communicates something rather than inferring that the original creator of that
document (if you yourself did not create it) intended to communicate something. The
communication intent of the author may differ from your interpretation of what is
communicated.
Documents communicate educational practices that are shaped by campus
cultures and often communicate campus norms.
The things that most surprise you about documents may be those things that run
counter to prevailing cultural practices.
Printed and electronic media represent only one component of a wide variety of
communications with students. They may have characteristics similar or dissimilar to
other forms of communication.

203

APPENDIX G
Document Analysis for Self-Assessment of Culturally Inclusive Practices for
Students of Color (used at Document (Syllabus) Analysis webinar)

Document Analysis for Self-Assessment of
Culturally Inclusive Practices for Students of Color

Purpose: This document is designed for individual and collective use to facilitate self-
assessment into the ways a college or university uses (or could use) culturally
inclusive practices
4
to assist underrepresented students, particularly Latinos, Latinas,
and African Americans (who are not represented among college graduates to an extent
proportional to their representation in the U.S. population), succeed in college. It is
designed to be used as part of a series of steps involving administrators, student affairs
professionals, and faculty in inquiry, self-assessment, and action planning.

At many campuses, this document analysis will follow a review of student persistence
and success at key milestones in a matriculation process (e.g. application, assessment
testing, admissions, placement in a curriculum) or in a curriculum (e.g. gateway
courses, credit accumulation, courses required for a major or degree completion,
timely graduation). A group of people in varying roles (e.g. admissions/matriculation,
administration, faculty, student affairs) agree to assess a sampling of the documents
they use in communicating with students as “artifacts of culture,” in other words to
look at the sampled documents “with fresh eyes” to consider what kinds of cultural
assumptions they hold. The group can be convened within a functional area (e.g.
admissions office, academic department, or student service program) or be cross
functional.

The process involves each participant first in individually assessing a sampling of their
own documents of educational practice using the indicators in the table below and then
in talking with the group about what they learned or contemplated while conducting
the document review. The response columns alongside each of the indicators asks
whether the culturally inclusive practice represented by the indicator is reflected in the
document and whether the person completing the review would be willing to take
steps to change the document if she or he believes changes are warranted. Drawing on
these reflections, each participant volunteers ideas to draft an action plan to strengthen
culturally inclusive practices on the campus.

4
Term derived from concepts adapted from culturally responsive, culturally relevant, or culturally inclusive
pedagogy. See, for one key reference, Ladson-Billings, G. (1995). Toward a theory of culturally relevant
pedagogy. American Educational Research Journal, 32(3), 465-491.

204

Getting Started

1. Collect a sampling of materials that reflect your educational practices and that
communicate expectations and information to students; for example:

For admissions and enrollment professionals: Application forms, program recruitment
brochures, web pages, financial aid forms
For faculty: syllabi, handouts from class, assignments, assessments of student
learning, course or personal web sites (if provided to students)
For program administrators and institutional leaders: descriptions of student eligibility,
mission statements and program goals, speeches, PowerPoint presentations,
newsletters and e-blasts

2. Keep in mind that printed materials are only one important way that you and your
campus communicate with students. Individual discussions, in class presentations or
group discussions, campus gatherings, artwork and the physical space are other
important forms of communication. (The Center for Urban Education offers other
protocols for self-assessment of practices represented in other types of media and
settings.)

3. Scan the materials to re-familiarize yourself with their contents.

4. With the materials at hand, proceed to complete the worksheet provided by the table
below.

Group Discussion and Action Planning

After group members have had a chance to assess their sampling of documents, the
group gathers to discuss their insights and to share possible action steps. These are
collected and displayed in the categories of “within reach,” “with a stretch” and
“beyond reach” and mapped to key milestones in a student’s educational progress at
the institution. Milestones where racial-ethnic groups experience substantial
differences in progress and success will be highlighted as potential intervention zones
for focused action by the group.

During Group Discussion, Keep in Mind:

Self-assessment differs from evaluation in that it is focused on problem-framing,
experimentation, and solution generation, not on evaluating your performance or the
performance of your colleagues.
In discussing any document or statement, use the same language you would if
your intent were to provide constructive feedback to a colleague sitting next to you.
To support your interpretations, point to specific language in the sampled
documents to provide “data” for your interpretation.
Refer to specific language as revealing or suggesting that the document
communicates something rather than inferring that the original creator of that

205

document (if you yourself did not create it) intended to communicate something. The
communication intent of the author may differ from your interpretation of what is
communicated.
Documents communicate educational practices that are shaped by campus
cultures and often communicate campus norms.
The things that most surprise you about documents may be those things that run
counter to prevailing cultural practices.
Printed and electronic media represent only one component of a wide variety of
communications with students. They may have characteristics similar or dissimilar to
other forms of communication.

Self-Assessment using Indicators of Culturally Inclusive Practices

Use the indicators below to reflect on the sample of materials you have collected. Jot
down your notes based on your reflections on ways your documents provide culturally
inclusive environment for students of color. Would you say that the documents can be
characterized by the indicators bellow? Finally, note steps you would be willing and
able to take to make changes to the documents, in those cases where you believe
change is warranted.

Indicators

High Expectations:

1. Expectation that students from all backgrounds are capable of obtaining high
educational goals and expected to succeed. Student success will be a collaborative
effort among the student, peers, faculty, administrators, counselors, students’ families,
and communities.

2. Expectation that students from all backgrounds will actively participate in different
aspects of their educational experience.

3. Information and resources students need to be successful are available in the
document.

Reflections:
Next Steps:

Validation:

1. Validation
5
of racial-ethnic diversity (shared ancestry, language, national heritage,
religious beliefs, community norms).

5
Rendón (1994) defines validation as “an enabling, confirming, and supportive process initiated by in- and out-of-
class agents that foster academic and interpersonal development.”
Rendón, L. I. (1994). Validating culturally diverse students: Toward a model of learning and student development.
Innovative Higher Education, 19(1), 33–51.

206

2. Validation of the value of differences in prior life and educational experiences.

3. Expectation that the educational community will engage in respectful discussion of
the history and contemporary experiences of discrimination, racism, and
marginalization.

Reflections:
Next Steps:

Affirming Attitude:

1. Respect for students from all backgrounds.

2. Desire to help students from all backgrounds succeed

Reflections:
Next Steps:

Empowerment:

1. Recognition that aspects of the educational experience are challenging but
attainable.

2. Expectation that students will be empowered as agents of social and community
well being through their education to reduce racial-ethnic inequities.

Reflections:
Next Steps:

Rigorous Learning:

1. Expectation that broad integrative learning will take place, focused by engagement
with big questions, both contemporary and enduring
6
.

2. Expectation that analytical, applied and integrative learning will occur across all
major fields, both pre-professional and liberal arts and sciences.

3. Expectation for active involvement with diverse communities, real world problems,
and social responsibility.

Reflections:
Next Steps:

6
Rigorous Learning Indicators are adapted from indicators of High Impact Practices from AAC&U’s LEAP
Campus Toolkit, available at http://leap.aacu.org/toolkit/

Abstract (if available)

Abstract This study examined the impact of the use of CUE’s action research processes and tools in a social design experiment undertaken at one field site as part of a larger developmental self-evaluation study conducted by CUE. Through the use of structured inquiry activities and equity-oriented cultural artifacts (language, media, tools), participants of this study were able to review and reflect on a key pedagogical tool: course syllabi. The purpose of the study was to understand whether and how CUE’s equity-oriented cultural artifacts (language, media, tools) mediate practitioners’ beliefs and behaviors in regard to equity. It was found that involvement with CUE influenced participants’ willingness towards behavioral changes to better serve their diverse students. However, insufficient CUE’s tool applicability and a lack of support from the wider university community, as perceived by participants in my sample, weakened the prospectus of broader institutional impact beyond the participant group.

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

Re-mediating practitioners' practice for equity in higher education: evaluating the effectiveness of action research

PDF

Practitioner reflections and agency in fostering African American and Latino student outcomes in STEM

PDF

Evaluating the impact of CUE's equity scorecard tools on practioner beliefs and practices

PDF

Designing equity-focused action research: benefits and challenges to sustained collaboration and organizational change

PDF

A developmental evaluation of action research as a process for organizational change

PDF

Action research as a strategy for improving equity and diversity: implementation constraints, outcomes

PDF

The influence of organizational learning on inquiry group participants in promoting equity at a community college

PDF

Changing the landscape of institutional assessment on transfer: the impact of action research methods on community college faculty and counselors

PDF

Encouraging student success: turning attention to practitioners and institutions

PDF

Syllabus review equity-oriented tool as a means of self-change among STEM faculty

PDF

Equity for students of color through practitioner accountability

PDF

Remediating artifacts: facilitating a culture of inquiry among community college faculty to address issues of student equity and access

PDF

Making equity & student success work: practice change in higher education

PDF

Language and identity in critical sensegiving: journeys of higher education equity agents

PDF

Creating an equity state of mind: a learning process

PDF

Culture, politics, and policy implementation: how practitioners interpret and implement the transfer policy in a technical college environment

PDF

Addressing a historical mission in a performance driven system: a case study of a public historically Black university engaged in the equity scorecard process

PDF

The Transfer Academy: providing community college students with the informational, structural, relational, and cultural resources to transfer successfully to a four-year college

PDF

Institutional researchers as agents of organizational learning in hispanic-serving community colleges

PDF

Institutional agents' impact on tranfser student success through the avenue of social capital

Asset Metadata

Creator Levonisova, Svetlana V. (author)

Core Title Evaluating the impact of CUE's action research processes and tools on practitioners' beliefs and practices

School Rossier School of Education

Degree Doctor of Education

Degree Program Education (Leadership)

Publication Date 08/06/2012

Defense Date 04/03/2012

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag action inquiry,action research,CHAT,equity in educational outcomes,OAI-PMH Harvest,Race

Language English

Contributor Electronically uploaded by the author (provenance)

Advisor Dowd, Alicia C. (committee chair), Bensimon, Estela Mara (committee member), Stillman, Jamy (committee member)

Creator Email levonisova@yahoo.com,slevonisova@gmail.com

Permanent Link (DOI) https://doi.org/10.25549/usctheses-c3-88301

Unique identifier UC11289173

Identifier usctheses-c3-88301 (legacy record id)

Legacy Identifier etd-Levonisova-1141.pdf

Dmrecord 88301

Document Type Dissertation

Rights Levonisova, Svetlana V.

Type texts

Source University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Access Conditions The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the a...

Repository Name University of Southern California Digital Library

Repository Location USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA