University of Southern California Dissertations and Theses

Academic achievement of English -learning Latino students in relation to higher order thinking skills instruction

(USC Thesis Other)

Academic achievement of English -learning Latino students in relation to higher order thinking skills instruction

PDF

Download

Open document

Flip pages

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content ACADEMIC ACHIEVEMENT OF ENGLISH LEARNING
LATINO STUDENTS IN RELATION TO
HIGHER ORDER THINKING SKILLS INSTRUCTION
by
Susan Louise Johns
A Dissertation Presented to the
FACULTY OF THE ROSSIER SCHOOL OF EDUCATION
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulfillment of the
Requirements for the Degree
DOCTOR OF EDUCATION
August 2002
Copyright 2002 Susan Louise Johns
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
UMI Number: 3094342
UMI
UMI Microform 3094342
Copyright 2003 by ProQuest Information and Learning Company.
All rights reserved. This microform edition is protected against
unauthorized copying under Title 17, United States Code.
ProQuest Information and Learning Company
300 North Zeeb Road
P.O. Box 1346
Ann Arbor, Ml 48106-1346
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
UNIVERSITY OF SOUTHERN
CALIFORNIA
The Graduate School
University Park
LOS ANGELES, CALIFORNIA 90089-1695
This dissertation, written by
Susan Louise Johns
Under the direction o f her Dissertation
Committee, and approved by all its members, has
been presented to and accepted by The Graduate
School, in partial fulfillm ent o f requirements fo r
the degree o f
DOCTOR OF EDUCATl
\ u u j j u .
DISSERTATION COMMITTEE
/ f \ 0
/ _
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
DEDICATION
This dissertation is dedicated with love to my
husband, Darrell R. Corell; to my mother, Delphine L.
Johns; and in the memory of my father, Virgil L. Johns,
for their patience, support and sacrifices through the
years.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
ACKNOWLEDGMENTS
The author wishes to express her sincere appreciation
to the many people who have influenced and supported her
during the completion of this dissertation.
Deepest appreciation is expressed to Dr. Edgar
Williams, chair of my dissertation advisement committee,
for his insightful critiques, guidance, and encouragement
throughout the process. Additionally, I am grateful for
the advice and support provided by Dr. Stuart Gothold and
Dr. John Nelson who served as members of my dissertation
committee and professors for four of the doctoral courses.
A special word of thanks is due to the dedicated
administrators, teachers, and other professional
colleagues for their assistance with collection of the
data. Acknowledgment is given to Peggy Adin, Dr. Kathy
Sabine, Nadine Rodriguez, JoeAnn Bruzzo, Dr. Manuel
Fuentes, Diana Hernandez-Colwell, Kam Nip, and Chad
Hammitt for their extensive help with this process.
Finally, deepest gratitude is expressed to my
friends, Cena Buchannon and Kathleen Compton, and
especially to my loving husband, Darrell Corell, for their
infinite patience and continuous encouragement. It was
with the support of all of these wonderful people that my
dissertation and doctoral program were completed.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
TABLE OF CONTENTS
Page
DEDICATION........................................ ii
ACKNOWLEDGMENTS. ................................. .iii
LISTS OF TABLES .................................. vi
ABSTRACT..................... * ‘ viii
Chapter
1 INTRODUCTION. ............... 1
The Problem Situation
Statement of Problem
Purpose
Importance of the Study
Research Questions
Definition of Terms
Delimitations
Limitations
Organization of the Remainder
of the Dissertation
2 REVIEW OF RELEVANT LITERATURE. ............ 15
Major Contributions
Higher Order Thinking Skills Instruction
Studies of Higher Order Thinking Skills
Literature about Latino Students' Instruction
Summary
3 METHODOLOGY. ................ 49
Sample
Research Design
Instructional Procedures
Instrumentation
Data Collection Procedures
Statistical Analysis
Methodological Assumptions
Summary
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
V
4 ANALYSIS AND DISCUSSION OF FINDINGS. .... 64
Summary of the Research Questions and Null
Hypotheses
Analysis of Findings
Discussion of Findings
5 SUMMARY, CONCLUSIONS, AND RECOMMENDATIONS . .89
Summary of Study
Conclusions
Recommendations
B IH. H * HiR H a f t i l C i K S . . S . B O B O * . 0 . B • . ■ . • • .103
APPENDICES. .................. .118
A Higher Order Thinking Skills Program Forms
B Teacher Questionnaire
C Site Facilitator's Interview Questions
D Additional Statistical Tables
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
vi
LIST OF TABLES
Table page
1 NAEP 1998 Reading and 1996 Mathematics....... 3 .
Performances for Grades 4, 8, and 12
2 Subjects Grouped by Language Fluency,........ 51
. Treatment and Gender
3 Elements of Pretest-Posttest Control-Group .. 52
Research Design
4 Means and Standard Deviations for .......70
Mathematics Scores in NCE Points for Latino
Students by English Language Fluency Groups
5 ANOVA for Effect of Language Fluency o n ......71
Mathematics Scores Presented in NCE Points
for Latino Students
6 Learning Skills Means and Standard...........78
Deviations for Latino English Learners by
HOTS Treatment and Control Groups
7 ANOVA for Effect of HOTS Treatment o n ........79
Application of Learning Skills of English
Learning Latino Students
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
v i i
8 Individual Learning Skills Means and.........80
Standard Deviations for English Learning
Latino Students
9 Independent-Samples T Test for Significance ..81
of Learning Skills of English Learning
Latino Students
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
viii
ABSTRACT
A one year quantitative study of the impact of Higher
Order Thinking Skills instruction on achievement of
English learning Latino students was conducted in a large
urban school district. The major purposes of the study
were to investigate the effects of this curriculum on
reading comprehens ion and mathematics problem solving
achievement, as well as on the application of learning
skills to classroom lessons. The learning skills included
selecting, testing, and explaining problem solving
strategies and positive, task-oriented behavior.
Initially, the impact of the thinking skills was
investigated for differences, designated by English
language fluency, among Latino students. These groups
were English learners, fluent English proficient students,
and English only students. Also studied were gender
differences on academic tests for students receiving the
thinking skills instruction.
The researcher implemented the Pretest-Posttest
Control-Group Design in this study. A total of 821 Latino
fourth and fifth grade students were the subjects with 81%
designated as English learners. Two hundred eighty-eight
subjects received the thinking skills treatment and the
remaining 533 served as control subjects. The Higher
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
ix
Order Thinking Skills treatment is a general thinking
skills curriculum of information processing which includes
lessons employing Socratic questioning techniques and
application of the thinking skills through use of computer
software.
With statistically significant findings for all eight
learning skills, the Higher Order Thinking Skills program
appeared to effectively increase students' application of
learning skills to regular classroom assignments. The
findings, however, did not indicate that this curriculum
significantly improved reading or mathematics gains scores
on the Stanford Achievement Test (1996) for English
learning Latino students. In the comparison of Latino
language fluency groups, the only significant difference
was between English learners and fluent English proficient
students in mathematics with the latter group producing
higher scores. The curriculum did not yield differences
determined by gender.
The researcher suggested possible future studies
utilizing longitudinal formats and various assessment
instruments, as well as assessing thinking skills
instruction presented in the students’ dominate language.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
1
CHAPTER 1
INTRODUCTION
The credo of public education in the United States
includes the belief that all students have the right to
effective elementary and secondary school instruction
which will prepare them for their future. While
implementing this philosophy, educators are faced with the
challenge of preparing students to utilize increasingly
sophisticated skills. When America's society transformed
from an agrarian culture to the Industrial Age, schooling
was extended and new knowledge was needed to prosper in
occupations of the Industrial Revolution. As the United
States phases into a complex, information-based age in the
21st century, all students, not only the elite, need to be
more proficient in reading, mathematics and higher level
thinking skills to process information and solve problems
(Naisbitt, 1982; A. R. Odden & E. R. Odden, 1995; Paul,
1995; Toffler, 1995; U. S. Department of Education, 1996a;
Costa & Liebmann, 1997).
It is estimated that during the first decade of the
21st century, the United States will generate
approximately 10,000 new jobs a day. America's fastest
growing industries are information-intensive, such as
computer firms, biotechnology businesses, financial
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
2
institutions, media corporations, medical services, and
education (Toffler, 1995).
Combining the Information Age academic demands with a
second trend, the increasing number of American students
entering school with home languages other than English,
magnifies the challenge (U. S. Department of Education,
2001). Historically, America's public education systems
have not been very successful with culturally and
linguistically diverse students (Gonzalez, Brusca-Vega, &
Yawkey, 1997). Many of the students are learning English
as their second language and are underachieving when
compared to their English speaking peers. Reported in the
next section, on average, academic achievement is lower
for students who do not have English as their native
language. Identifying and implementing effective
curricula that will prepare all students, including
English learners, is a critical challenge for the
educational community.
The Problem Situation
To determine the level of student achievement, the
United States Department of Education created the National
Assessment of Educational Progress (NAEP). The NAEP
assessments provide information with a total score for each
subject area and student characteristics. The following
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
3
table provides scores for total students and specifically
Latino students (U. S. Department of Education, 2000a).
Table 1
NAEP 1998 Reading and 1996 Mathematics Performances for
Grades 4, 8, and 12
Reading Performance, 1998 Grade
_4_________8__________12
Average Total Score 217 264 291
Average Latino Score 196 244 275
Mathematics Performance, 1996 Grade
_4________8________ 12
Average Total Score 224 272 304
Average Latino Score 206 251 287
Over a period of six years, the average total score
has increased one point in reading and ten points in
mathematics. The average score for Latinos had decreased
by a point in reading and increased by almost nine points
for mathematics during the six-year period from 1990 to
1996, but remained below the average total scores.
Analyzing the NAEP results, the examiners indicate
that students are demonstrating a high degree of mastery on
very basic reading and mathematics skills. However, on
average, only 5% of the students reached the advanced
levels of proficiency in reading and mathematics which
required high levels of thinking. The NAEP reports
conclude that while most students at fourth, eighth, and
twelfth grades have mastered basic competencies, too few
with permission of the copyright owner. Further reproduction prohibited without permission.
4
have reached levels likely to be required for the 21st
century work place (U. S. Department of Education, 2000a).
Statement of Problem
in general, public educational systems in the United
States are not providing effective curriculum to assist all
students in learning the higher order thinking skills that
are needed to attain advanced proficiencies in reading and
mathematics. As is indicated by the NAEP results, this
lack of sufficient instruction is especially evident with
the Latino students. With increasing numbers of Latino
students, many who are learning English, entering schools
in the United States, it is crucial that educators utilize
effective thinking skills instruction so the students will
attain academic success. Beginning in the elementary
schools, where the highest numbers of English learning
Latino students are enrolled, the students need to be
challenged with thinking skills to help them learn and
monitor their understanding of reading and mathematics.
Such instruction is essential to serve as a basis for
further education, so all students will be prepared for the
Information Age.
Purpose
The primary purpose of this study was to determine
whether the teaching of higher order thinking skills would
contribute to increasing academic achievement of Latino
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
students who are learning English as a second language.
The students were in the fourth and fifth grades and were
in English reading groups. Standardized test scores were
analyzed to measure the academic levels of the students in
reading comprehension and mathematics problem solving.
The test scores were compared for the groups of students
as they were designated by English language fluency. The
three groups were English learners, fluent English
proficient, English only students.
The secondary purpose of the study was to determine
whether the higher order thinking skills enabled students
to apply learning skills to their classroom lessons. The
learning skills involved making predictions, selecting
strategies, explaining processes and using prior
knowledge. They also included willingness to participate
in discussions and cooperative groups, as well as
developing an attitude of persistence.
In addition, the study analyzed the academic data for
the factor of gender. The study sought to determine
whether the thinking skills teaching strategies were more
effective for male or female students.
Importance of the Study
The importance of a study that analyzes the
effectiveness of curriculum for Latino students who are
learning English is evident because it could assist such a
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
rapidly growing number of students (U. S. Department of
Education, 1995, 1996b, 2001). By 1998, 37% of public
school students in grades first through twelfth were
considered to be minority students. This figure
represents a doubling of the percentage since 1972. This
increase was largely due to the growth in population of
Latinos whose proportion grew from 5.8% in 1972 to 15.1%
in 1998. With diversity of ethnicity, multiple languages
have ensued. As reported by the National Clearinghouse
for Bilingual Education, in 1997 there were approximately
3.4 million English language learning students in the
United States (Macias, 1998). Of these students, about
three-fourths spoke Spanish as their first language (U. S.
Department of Education 2000b). These students' origins
represented many countries including areas of Central
America, South America, Mexico and the Caribbean.
Schools in the West are providing the education for
the majority of the Latino students with many students
learning English as their second language ( U. S.
Department of Education, 1997). In this region of the
United States, by 1998 Latino students accounted for 30%
of the student body, twice the percent of 15% in 1972.
More specifically, in 1999, 42.2% of California's 6.9
million public schools students were Latinos. Half of
these students are learning English as their second
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
7
language while in the school system. Including students
who speak a variety of languages, California experienced a
334% increase in the total number of English language
learning students between 1982 and 1998 (from 431,449 in
1982 to 1,442,692 in 1998). Eighty-one percent of these
English language learning students are of Latino origin.
In summary, as a result of linguistic diversity,
approximately one in four students in California's public
schools are learning English as a second language
(California Department of Education, 2000). Additionally,
demographic data indicate that the number of Latino
students who are English language learners continues to
rapidly increase. On a national basis, by the year 2020
it is projected that more than one in five children in the
United States will be of Latino origin (U. S. Federal
Government, 1998).
Research Questions
Research Question 1 Are there significant
differences in reading comprehension test score gains
among groups of Latino students defined as English
learners, fluent English proficient, and English only who
received the Higher Order Thinking Skills Program?
Research Question 2 Are there significant
differences in mathematics problem solving test score
gains among groups of Latino students defined as English
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
8
learners, fluent English proficient, and English only who
received the Higher Order Thinking Skills Program?
Research Question 3 Is there a significant
difference in reading comprehension test score gains
between Latino English learners receiving the Higher Order
Thinking Skills Program and students with these
characteristics who are not receiving this program?
Research Question 4 Is there a significant
difference in mathematics problem solving test score gains
between Latino English learners receiving the Higher Order
Thinking Skills Program and students with these
characteristics who are not receiving this program?
Research Question 5 Is there a significant
difference in reading comprehension test score gains
determined by gender of Latino English learners who
received the Higher Order Thinking Skills Program?
Research Question 6 Is there a significant
difference in the mathematics problem solving test score
gains determined by gender of Latino English learners who
received the Higher Order Thinking Skills Program?
Research Question 7 Is there a significant
difference in application of learning skills between
Latino English learners receiving the Higher Order
Thinking Skills Program and those students with the same
characteristics who did not receive the program?
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
9
Definition of Terms
For the purposes of this study, pertinent terms are
defined as follows:
Critical Thinking "Reasonable, reflective thinking
that is focused on deciding what to believe or to do"
(Ennis, 1992, p. 75).
Decontextualization "Generalizing ideas from one
context to another" (Pogrow 1997c, p. 103).
English Learners (EL) Students whose primary
language is not English and who lack English language
skills, according to state approved assessments, that are
necessary to succeed in the school's regular instructional
programs. The terms "limited English proficient" and
"English language learner" are synonyms for "English
learners" (California Department of Education, 1999).
English Only (EO) Students who speak only English
and, according to their Home Language Survey, to whom
English is spoken at home.
Fluent English Proficient (FEP) Students whose
primary language is not English and who have met the
district criteria for determining proficiency in English
(California Department of Education, 1999). For the
school district where this research was conducted, the
criteria was the 31st percentile or higher on the Stanford
9 and meeting proficiency standards.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
10
Higher Order Thinking Skills Complex processes
involving nonrountinized thinking in which complex
reasoning and nuanced judgment may produce responses not
previously designed (Resnick, 1987), specifically the
skills needed to develop metacognition, inference from
context, decontextualization, and information synthesis.
Inference from context "Figuring out unknown words
and information from the surrounding information" (Pogrow,
1997c, p. 103).
Information synthesis "Combining information from a
variety of sources and identifying the key pieces of
information needed to solve a problem" (Pogrow, 1997c,
p. 103).
Latino Students A general term for students whose
families were originally from Mexico, Cuba, and Central or
South America. In this study, 99% of the Latino students'
families were from Mexico. Most of the students either
were born in Mexico or are the first generation to be born
in the United States.
Metacognition The process of reflecting on one’s
thinking and utilizing monitoring and regulation, usually
to accomplish a task.
. Normal Curve Eguivalent (NCE) NCE scores are used
for data analysis in this study. NCEs have many of the
characteristics of percentile ranks but have the
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
11
additional advantage of being based on an equal-interval
scale. The scale has a range of 1 through 99 with a mean
of 50. A student's score of 50 in the fall and 50 again
the following fall on the next grade level test, indicates
a stable standing in comparison to other students across
grades.
Delimitations
The following delimitations narrow the focus of this
study.
1. size of Sample This study is limited to fourth
and fifth grade students. They were assessed in spring
1998 and 1999. The sample is composed of 288 students who
receive the Higher Order Thinking Skills Program and 533
students who served as the control group. Seven of the
experimental group and 17 of the control group were
English only students. Fifty-five of the experimental
group and 79 of the control group were fluent English
proficient. The remaining students were English learners.
2. Geographic Area This study is limited to a
single school district which is located in an urban area
of Southern California.
Limitations
The identification of the limitations of this study
provides a framework within which all conclusions must be
made.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
12
1. Attrition and History Because the study covers
one year, the attrition rates of the schools will probably
be an important factor. This attrition rate is assumed to
have an equal impact on both the experimental and control
groups.
2. Instrumentation The Stanford Achievement Test,
Ninth Edition, Form T, (1996) is a nationally normed
series of standardized tests that provide easily
understood results concerning reading comprehension and
mathematics problem solving.
3. Maturation Similar to the effects of attrition
and history, maturation is assumed to affect all groups of
students involved in this study in the same manner.
4. Personnel This study examines the programs
taught by teachers who naturally bring their own
personalities, strengths and weaknesses to their
instruction. Training will ensure some conformity in
lesson preparation and presentation, but it will not
completely control the effect of teacher nature.
5. Setting This study occurs in ten schools that
present very similar demographics. They will not
necessarily represent the demographics of other schools.
6. Socioeconomic Levels The socioeconomic levels of
the schools in this study are definitely below the
national norm as measured by the percent of students on
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
13
the Free or Reduced Lunch Program. Research shows,
however, that most English learners are in families living
in low socioeconomic conditions.
7. Statistical Regression Because this research
studies many students scoring at low levels on the
standardized tests, it is assumed that statistical
regression may be a factor in the posttest results.
Organization of the Remainder
of the Dissertation
Serving as a basis for this study of the relationship
between thinking skills instruction and academic
achievement, Chapter 2 presents a review of related
literature. Because higher order thinking skills are
nested in critical thinking skills, Chapter 2 begins with
the contributions to the development of critical thinking
skills instruction. That discussion is followed by
summaries of studies of thinking skills programs and
English learners' instruction in thinking skills.
The subsequent chapters focus on procedures and
findings of the study. Chapter 3 describes the student
sample and the research methodology including the
instruments and data collection procedures. Chapter 4
presents an analysis of the statistical findings
describing the standardized test scores and the
questionnaire about the application of learning skills.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
14
Finally, Chapter 5 provides a summary of the study,
outlines conclusions drawn from the findings, and suggests
recommendations for future studies.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
15
CHAPTER 2
REVIEW OF RELEVANT LITERATURE
The review of relevant literature includes a listing
of the major contributions to higher order thinking skills
instruction during the twentieth century. Due to the
evolving content of this literature, the discussion begins
with defining and classifying thinking including
metacognition and critical thinking. Subsequently, the
term "higher order skills" is coined as this more
encompassing concept emerges. With the development of
curricular projects, studies of thinking skills
instruction are summarized. Lastly, writings about
English learning Latino students 1 instruction in higher
order thinking skills are be described.
Major Contributions
As an introduction, one must acknowledge that work
toward improving the higher levels of thinking skills has
been emphasized by noted philosophers for centuries. For
example, 2400 years ago, Socrates involved his students in
challenging conversations about ideas. Through probing
questioning, he enabled them to discover and explore
concepts (1995, Paul). He queried with questions such as,
What is the evidence? Into which category, then, are we
to put that? and If that is true, then does it follow
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
16
that. . . . 7 Through a series of questions which lead to a
definite point, the students were stimulated to confront
their assumptions and logically developed their
conclusions (Cole, 1950).
For a historical perspective during the 21st century,
this section presents a description of the major
contributions toward the advancement of improving higher
order thinking skills. The leaders in the area of
thinking recognized and defined the concepts and skills;
identified the need for instruction in thinking skills;
developed evaluation criteria; and created curriculum.
Beginning in the 1880s and continuing through the
first half of the twentieth century, Dewey wrote about
educating children to think critically and the importance
of this education for our society. He was concerned about
the teaching methods that were dominated by rote
memorization of the disciplines' contents. In The Child
and the Curriculum (1902), he stated "the child's
reasoning powers, the faculty of abstraction and
generaliz at ion, are not adequately developed"(p. 26).
Instead of teaching passive acceptance and retention of
information, Dewey exclaimed "the important thing is that
thinking is the method of an educative experience" (1916,
p. 192). He described the elements of the method as the
problem, collection and analysis of data, projection and
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
17
elaboration of suggestions or ideas, experimental
application and testing, and the resulting conclusions.
Recognizing that students possess personal differences and
will proceed through their thinking processes in different
manners, Dewey continued to stress reflective thinking.
Whether the "problem" is an action, proposal, or belief,
Dewey stated that it must be carefully and persistently
examined and the possible solutions and consequences must
be analyzed. Dewey reiterated the importance of
reflective thinking as he stated, "While all thinking
results in knowledge, ultimately the value of knowledge is
subordinate to its use in thinking" (1916, p.178).
Through his publications and teaching, Dewey's philosophy
influenced the reforms in education of the 1930s and 1940s
and is again being recognized with the renewed interest in
thinking skills. Loughran (1996) observed that Dewey's
descriptions of reflective thinking could be appropriately
referenced in many modern day studies into the enhancement
of student learning.
Studying thinking ability from the viewpoint of its
development, Swiss psychologist, Piaget emphasized that
children gain knowledge and create concepts by interacting
with their environments (Costa & Liebmann, 1997). in his
findings from the 1920s to the 1970s, he stated that
children develop concepts by investigating to make sense
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
18
of the world around them. Children are not passive
recipients in this process; they are actively involved in
experiences where they experiment and organize ideas. By
interacting with their environment, children progress
through stages which Piaget identified as Sensorimotor,
Concrete Operations, and Formal Operations Periods.
Piaget's theories contributed to several revisions in
instruction, e.g. emphasizing hands-on experience with
materials for concept formation. Piaget's work provided a
basis for increasing the use of mathematics manipulatives
and elementary science curriculum that utilizes materials
for investigations (Phillips, 1969).
Glaser has served as one of the authorities in the
area of critical thinking since the 1940s. In 1941, his
book, "An Experiment in the Development of Critical
Thinking. laid the cornerstone for the critical thinking
movement" (Paul, 1995, p.i). Glaser extended the concepts
of critical thinking to guidelines for developing
responsible citizenship through problem solving. He
considered the following three key elements to be
critical: an attitude of thoughtfulness, a knowledge of
the methods of reasoning, and specific skill application
(Glaser, 1985). With these elements, citizens would be
able to think critically about issues presented through
various opinions. Applying his theories to student and
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
19
adult behavior, Glaser wrote about reinforcing positive
classroom behavior and increasing productivity of the
work force (Sarason, Glaser & Fargo, 1972).
As cognitive theories were impacting educational
practices, two psychologists lead the effort to develop
well known hierarchies. Bloom and a his colleagues
(1956), after eight years of discussions, developed the
Taxonomy of Educational Objectives. This taxonomy
presents the following six categories of intellectual
tasks on a continuum from simple to complex: knowledge,
comprehension, application, analysis, synthesis, and
evaluation. While focusing on the cognitive domain, the
contributors to this taxonomy envisioned its usefulness
for clarification of objectives, organization of
evaluation instruments, assistance with comparison of
educational programs, and aid in the research on learning.
The concepts and examples in the handbook continue to be
used by educational administrators, teachers, theorists,
and evaluators.
A more complex hierarchy, Structure of the Intellect,
was created and revised by Guilford (1959, 1988). Based
on this theory, 180 factors were classified by the
interactions of three dimensions (1) the five types of
processed contents, (2) the six types of operations
performed with the contents, and (3) the six kinds of
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
20
products of the processed information. Utilizing this
model and the various facets of intellect embedded within,
Guilford was one of the leaders in distinguishing
creativity from general intelligence (Dembo, 1994). He
analyzed concepts of both convergent and divergent
thinking. Based on Guilford's work, Meeker and her
associates developed several tests that have been useful
in identifying potentially gifted students, especially
minority students. The model in the Structure of the
Intellect has been the basis for diagnosing developmental
needs of gifted students and prescribing instructional
objectives (Glatthorn, 1987).
These hierarchies brought increased general awareness
that learning was more than memorizing facts and learning
routine procedures. Both convergent and divergent
thinking were valid, and curriculum development should
include and assess various levels of thinking. However,
at this point in time the higher levels of thinking and
problem solving were viewed as being isolated from basic
learning. Instead of being central to education, they
were considered to be advanced activities (Resnick, 1987;
Resnick & Klopfer, 1989; Brown & Campione, 1992; Paul,
1995).
As theories about thinking and the organization of
its facets continued to be created, the term
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
21
"metacognition" was coined. Flavell (1976) discusses
metacognition by stating:
"Metacognition" refers to one's knowledge
concerning one's own cognitive processes and
products or anything related to them, e. g., the
learning-relevant properties of information or
data.... Metacognition refers, among other
things, to the active monitoring and consequent
regulation and orchestration of these processes
in relation to the cognitive objectives on which
they bear, usually in the service of some
concrete goal or objective (p. 232).
Metacognitive knowledge includes understanding oneself as
a learner, the tasks at hand, and learning strategies.
Metacognitive control involves strategies for planning;
monitoring, including comprehens ion and focusing
attention; and regulation (Dembo, 1994). As will be
discussed in the following section of this paper,
instruction in metacognition has shown positive results
for students in reading comprehension.
As Bloom's and Guilford's taxonomies were constructed
and metacognition was being discussed, Ennis was also
contributing to the teaching of critical thinking. From
the 1950s through the 1980s, he published works on
defining, motivating, teaching, and testing critical
thinking. While Ennis served at Cornell University as the
director of the research project, Critical Thinking
Readiness in Grades 1-12, he and his associates reviewed
Piaget's work and developed operational definitions for
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
22
investigations and tests to assess critical thinking of
upper elementary through high school age students (Ennis
& Paulus, 1965). He concluded that without direct
instruction, classic and conditional logic were not
mastered by students ages 11-18, but he documented
accelerated improvement in logic with instruction.
Discussing the placement of instruction, he recommended
curriculum development and classes in middle and high
school rather than infusing these studies into traditional
subject matter courses (Ennis, 1985b).
Ennis (1992) globally defined critical thinking as
the "reasonable, reflective thinking that is focused on
deciding what to believe or do" (p. 75). He also directed
attention to the need to study the impact of direct
instruction of critical thinking skills on academic
performance. As a result of Ennis' (1985a, 1992)
contributions to the defining, investigating, instructing,
and evaluating critical thinking, it now had a solid base
for curriculum development for public school students .
As the theorists and educators provide structure to
the area of cognitive processes and thinking skills,
Presseisen (1986) divided thinking skills into three main
categories. The essential thinking skills are basic and
also serve as building blocks of thought development.
They would include classification, observation, and
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
23
sequencing. The category of higher order thinking skills
build on the basic skills. They are more complex and
involve problem solving, decision making, critical
thinking, and creative thinking. The third major category
is metacognitive processes which are "the learning to
learn skills aimed at making thinking more conscious and
the student more aware of the ways one can go about
problem solving or decision making" (p. 9).
Another major contributor to the field of thinking
skills education is Vygotsky, a Russian psychologist.
Although his writings were completed in the 1930s and
1940s, they were not translated into English and widely
read until the late 1970s and 1980s (Veer & Valsiner,
1991, 1994). Vygotsky (1962, 1978, 1997) emphasized that
the thinking required to gain knowledge and develop
concepts is based in social interaction. The zone of
proximal development (ZPD) is one of Vygotsky's most noted
theories. He defined the zone of proximal development as
"the distance between the actual developmental level as
determined by independent problem solving and the level of
potential development as determined through problem
solving under adult guidance" (Vygotsky, 1978, p. 86).
The adult or teacher assists the student through
questioning, activities, modeling, etc. of the targeted
learning to reach the potential level of development.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
24
Vygotsky also addressed the concept of metacognition
through his writing about inner speech. This is the
person's thoughts which monitor progress and provide self
guidance (Rieber 1997; Vygotsky, 1962, 1978, 1997). His
explanations serve as the basis for strategies, such as
scaffolding and reciprocal reading techniques. They also
support the use of Socratic questioning techniques which
mediate learning.
Paul, a colleague of Glaser and Ennis, is an
influential spokesperson for the philosophical perspective
of critical thinking. He served as the director of the
Center for Critical Thinking and the chairperson of the
National Council for Excellence in Critical Thinking.
Paul states, "Critical Thinking is a systematic way to
form and shape one's thinking. It functions purposefully
and exactingly. It is thought that is disciplined,
comprehensive, based on intellectual standards, and, as a
result, well-reasoned" (Paul, 1995, p. 20). He goes on to
explain the term "comprehens ive critical thinking" by
charting the following elements (p. 21).
• A unique kind of purposeful thinking
• In which the thinker systematically and habitually
• Imposes critical and intellectual standards upon
the thinking
• Taking charge of the construction of thinking
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
25
• Guiding the construction of the thinking according
to the standards
• Assessing the effectiveness of the thinking
according to the purpose, the criteria, and the
standards
Paul (1995) wrote that critical thinking skills
enable students to work through everyday problems. He
expressed concern that the skills taught through
traditional content curriculum do not teach students to
develop critical thinking. Students need to learn to draw
justifiable conclusions on the basis of good reasoning.
To become skilled in reasoning, students must be able to
make what is implicit explicit, so they can "check out"
what is going on "beneath that surface" of their thoughts.
A distinguishing feature of Paul's work was the
intellectual standards which he described as including
relevance, accuracy, precision, clarity, depth and
breadth. Utilizing these criteria he stressed thinking
that enables students to analyze not only their beliefs
and attitudes, but also opposing belief systems. He and
his colleagues have authored curriculum for students from
kindergarten through post graduate level. The methodology
includes teacher directed Socratic questioning to train
students to examine their views. Through the
international conferences and annual regional trainings
which he organizes, he leads staff development to empower
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
26
teachers so they can instruct their students in critical
thinking.
Costa's writings throughout the 1980s and 1990s
contributed to classification of thinking skills,
understanding of metacognition, and awareness of
instructional programs. Costa (1985) provided a
generalized definition that thinking is the receiving of
external stimuli through the senses followed by internal
processing. He then summarized thinking skills into four
tiers: (1) discrete skills of thinking, (2) strategies of
thinking, (3) creative thinking, and (4) the cognitive
spirit (Costa 1985; Costa & Liebmann 1997).
Higher Order Thinking Skills Instruction
From the 1960s through the 1990s, several programs
were created for presenting thinking skills. Costa
provided descriptions of more than fifteen of the
curricular programs. These programs included Feuerstein’s
Instrumental Enrichment (1978), Lipman's Philosophy for
Children (1968) and deBono's CoRT Thinking Program (1975).
Additional programs are Robert J. Swartz's Reading for
Thinking: A New Framework for Comprehension (1987); Paul,
Binker, Martin, Vetrano and Kreklau's Critical Thinking
Handbook(s) (1989); Marzano and Arredondo's Tactics for
Thinking (1986); Will's Great Books (1983); and Pogrow' s
Higher Order Thinking Skills (1986).
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
27
Of the curricular programs that Costa listed, three
are summarized as examples of materials developed from
the 1960s through the 1980s. These programs are
Philosophy for Children, CoRT Thinking Lessons, and
Instrumental Enrichment. Pogrow's Higher Order Thinking
Skills, initially developed in the 1980's, will be
explained later in this review of literature.
Philosophy for Children
Lipman developed the program Philosophy for Children
in 1968 in response to his observation that many incoming
college students had never been exposed to any
philosophical thinking. The program was designed to teach
children with average reading ability in grades three
through twelve how to think more broadly by improving
their reasoning skills and fostering independent thought.
The program uses novels geared toward specific age
levels. Each novel features a main character who becomes
the catalyst for philosophical thought and discussion
among the students. Teachers stress the use of language,
making dialogue a large part of the program. The
discussions are held to elicit and clarify views, seek
consistency in thinking, search for assumptions, indicate
fallacies, and request supporting reasoning (Chance,
1986).
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
28
CoRT Thinking Lessons
In 1975, CoRT Thinking was developed by deBono, the
director of the Cognitive Research Trust, from which CoRT
gets it's name. According to deBono, thinking occurs in
two distinct stages: perceptual and analytical.
Perceptual thinking precedes analytical thinking so that
an error in the perceptual stage is often the cause for
flawed analytical responses. Because the perceptual stage
of thinking is often overlooked, deBono created CoRT with
an emphasis on the perceptual stage of problem solving.
The purpose of CoRT lessons is to improve perceptual
thinking by using "tools": like PMI (Plus, Minus,
Interesting) and C&S (Consequences and Sequences), which
emphasize the importance of this first stage of thinking.
This program is most often used with students between the
ages of nine and twelve and is effective with all ability
levels. The strengths of this method of thinking are that
students begin to see more sides of an issue and are able
to develop more creative solutions to problems (Chance,
1986).
Instrumental Enrichment
Feuerstein created the Instrumental Enrichment
program in 1978. It is based on the assumption that
children who do not receive "mediated learning
experiences," which include an explanation of the
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
29
rationale behind a fact or command, develop cognitive
deficiencies causing them to be labeled as "slow" or
"retarded." These children then take a passive approach
to learning. Feuerstein's program is designed to change
the thinking patterns of individuals eleven years and
older who perform below their grade level, specifically
those with IQs of 75-90.
Instrumental Enrichment groups meet for one hour
three to five times a week for two or three years. In
addition to providing these students with "mediated
learning experiences," their teachers present fifteen
separate units of worksheets, each said to help overcome a
specific cognitive difficulty such as impulsivity or the
lack of appreciation for accuracy. Unlike traditional
academic activities, instruments focus on the process of
thinking rather than its results. This is especially
important when working with these students because many of
them have not succeeded in conventional classroom settings
and many have low estimations of their own abilities. It
helps them to begin to take an active part in thinking and
learning (Chance, 1986; Feuerstein, Feuerstein & Schur
1997).
As numerous curricula were being developed for
teaching higher order thinking skills, the debate became
not if thinking skills should be taught, but how they
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
30
should be taught (Marzano et al. 1988; Presseisen, 1992).
Several noted educators and researchers in the area of
thinking skills attested to the viewpoint that thinking
skills need to be presented, or at least initially taught,
as independent units of study (Beyer, 1988; Paul, 1995;
Pogrow 1996b, 1997a, 1997c), For example, Beyer (1988)
supported the development of separate thinking skills
curriculum "with higher order skills introduced in
simplified version in third, fourth, and fifth grades"
(p.16). Paul (1995) states critical thinking skills
instruction is more effective when organized as
independent lessons because these skills possess specific
content. Pogrow (1990b, 1995a, 1996a, 1997b) concurs
since the Higher Order Thinking Skills Program involves
techniques requiring a week of intensive staff development
and practice. without this training, classroom teachers
generally have not learned these techniques.
Other leaders in the area of thinking skills
v.
education declared that thinking skills should be
incorporated into the subjects taught by the classroom
teachers. Costa stated that higher level skills should
not be taught in isolation. Bareli, Liebman and Segil
(1988) stated that teachers need to teach programs which
help students recognize how to plan the accomplishment of
a task, to monitor their performance, and to evaluate the
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
31
results. Swartz and Perkins (1990) developed a thinking
skills program that assisted the teacher with including
thinking skills into established curriculum. He explained
that this program offered structure for infusing the
skills into the curricular areas.
As Resnick and Klopfer addressed the debate about the
form for delivery of thinking skills, they emphasized
reasons for all students receiving the instruction. In
the Association for Supervision and Curriculum
Development's Toward the Thinking Curriculum; Current
Cognitive Research (1989), Resnick and Klopfer write that
research has provided a new perspective on learning to
think. They state, "'thinking skills' are involved in
successful learning of even elementary levels of reading,
mathematics, and other subjects. Cognitive research on
children's learning of basic skills such as reading and
arithmetic reveals that cultivating key aspects of these
thinking processes can and should be an intrinsic part of
good instruction from the beginning of school" (p. 2).
Continuing their analysis, they explained that thinking
skills should not be reserved for selected students such
as those with excellent achievement. They called for
recognition that all real learning involves thinking and
that thinking ability can be nurtured and cultivated in
everyone.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
32
During the 20th century, philosophers, psychologists
and educators established an extensive compilation of
professional literature concerning the study of thinking
and its skills. Although the definitions vary, this body
of work has increased understanding of thinking processes
and of higher level thinking skills development. Based on
this accomplishment, curricula have been created to teach
thinking skills in the school systems. The next section
of this chapter addresses research on the effectiveness of
these curricula.
Studies of
Higher Order Thinking Skills
Studies specifically addressing the effects of higher
order thinking skills instruction on students' academic
performance have been conducted during the past ten years.
Examples of reports concerning the academic performance of
elementary and intermediate students in these studies are
described on the following pages.
In Detrick's research (1988), first, second, third,
fourth, and sixth grade students served as subjects for a
study utilizing seven of Marzano's Tactics for Thinking
Program. The skills taught in this study were as follows:
attention control, deep processing, concept attainment,
pattern recognition, analogical reasoning, non-linguistic
patterns, and solving academic problems. At the p < .05
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
33
level no significant differences in mean gains scores on
the Iowa Test of Basic Skills were found when comparing
experimental group students' paired scores of the prior
year's gains with gains during the treatment. But, when
the experimental classes 1 gains were compared to the
remainder of the student population, those receiving the
Tactics for Thinking Program in grades fourth and sixth
did show significance. Also, "when considering the
percentage of students below grade level at the beginning
of the year and at the end of the year, the experimental
groups were successful at bringing from 6% to 23% more of
those students below grade level to within expected grade
level than in the control group" (p. 77).
The ability of fourth and fifth grade students to
learn, retain, and transfer thinking skills was the focus
of a study by Riesenmy, Mitchell, Hudgins, and Ebel
(1991). This research was an extension of a series of
studies by Hudgins and Edelman beginning in 1986. These
two researchers concluded that students as young as nine
or ten years old could successfully learn to read
information and use it as evidence to support their
answers to higher order thinking skills questions.
In Riesenmy, et al.'s 1991 investigation, a study was
designed to determine whether the experimental group
retained and transferred their self-directed thinking
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
skills better than the control group. The study was
conducted in suburban areas and the students were
predominately Anglo with some African American students in
both groups. Ten students in fourth and fifth grade
classrooms were trained through 12 small group
discussions. During the treatment, the students learned
roles designated as task definer, strategist, monitor, and
challenger which they used in their group work. At a
significance level of p <.001, the experimental group
produced retention scores on the three variables
identified as self-directed thinking skills, amount of
information used in solutions, and quality of answers.
Also at the p <.001 significance level, the experimental
group scored better than the control group on the transfer
problems. To accomplish this transfer, these experimental
students were self-directed and applied their skills to
problems that demanded more difficult thinking abilities
than required in their discussion training.
In 1993, Allison completed a doctoral study of the
Super Math and Reading Thinking Skills (SMARTS) Program, a
critical thinking/problem solving skills program for
third, fourth, and fifth grade at-risk students. In this
study, at-risk students were defined as those students who
qualified for Title 1 instruction. At an alpha level of
.05, the subjects receiving the SMARTS Program in critical
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
35
thinking/problem solving did not indicate significance
gains after one year of instruction. However, more in-
depth analysis, which compared the SMARTS Program with the
regular classroom instruction, the reading gains did show
significance. Additionally, mathematics achievement was
above mean of the Iowa Test of Basic Skills for the
treatment and control groups which is atypical for most
Title I students.
Studies in several locations have been conducted on
the Higher Order Thinking Skills Program (Pogrow 1995b).
In Detroit a study was conducted using nationally
standardized tests. The gains exceeded national averages
and were significant at the p <.05 level. On spring to
spring measures, the experimental group gained 4.7 NCEs in
reading as compared to the national average of 3.1 NCEs.
On the mathematics section the experimental group performed
even higher gaining 8.1 NCEs as compared to the national
average of 3,9 NCEs.
In the Soldotna, Alaska's study of HOTS during the
1990-1992 school years, the spring to spring reading
results on the Iowa Test of Basic Skills reported that the
fifth grade students made gains of 8.22 NCEs for the
experimental group as compared to 4.41 gains for the
control group. On the mathematics test, the treatment
students averaged 12.12 NCEs while the control students
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
36
scored 7.67 NCEs. Twenty-five subjects received the HOTS
treatments in this study and forty-four served as control
subjects (Pogrow, 1995b).
At Mann Middle School in San Diego, California, the
1992-94 study indicated that seventh grade students
receiving HOTS earned improved report card grades. The
report from the HOTS organization states that Mann School
is one of the lowest socioeconomic status schools in San
Diego and 80% of the students qualify for the free or
reduced lunch program. The study does address transfer of
skills from the HOTS program to academic achievement in the
classroom, an important aspect of learning. In 1992-93,
14% of the 65 seventh grade students in the second year of
HOTS earned an A average, 49% a B average or better and 83%
a C average or better. In 1993-94, 21% of the first year
students (sixth graders) had an A average. Twenty-three
percent of the second year HOTS (seventh graders) had an A
average (Pogrow, 1995b).
A well designed quasi-experimental investigation of
the Higher Order Thinking Skills (HOTS) Program was
conducted by Eisenman (1995) in Georgia. A total of 175
fourth and fifth grade students were divided into groups
designated as HOTS instruction and the traditional Title 1
program. Utilizing the Iowa Test of Basic Skills and Ross
Test of Higher Cognitive Processes, findings suggest that
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
37
the HOTS Program produced increased higher order thinking
skills for fifth grade students. In this study the
program was more effective for girls than boys and two
years of the treatment lessons provided better results
than only one year. Included in the findings are the
reading results showing that both the HOTS and Title 1
instruction raised NCE scores above the state average.
However, the study failed to produce statistically
significant difference in academic achievement between the
HOTS and Title 1 programs.
In the study by Darmer (1995) of the effectiveness of
the HOTS Program for fourth and fifth grade students,
Native American and Latino students were the subjects.
The treatment group was composed of 53 subjects with 41%
Native American, 55% Latino, and 4% Anglo who received the
HOTS Program for one year. All of the subjects scored
between the 15th and 40th percentile on the standardized
reading test qualifying for Title 1 assistance. The
results of the pre and post tests indicated that the
treatment group significantly outscored the control group
in all of the 22 comparisons measured by the Nelson
Reading Comprehens ion Skills Test, Woodcock-Johnson
Cognitive Abilities Subtests, a metacognition
questionnaire, a set of novel problem solving tasks, etc.
In addition, 34% of the students in the HOTS Program made
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
38
the honor roll. The study presents consistent,
substantial academic growth of the treatment group
compared to the control group.
Depending on the subjects and methodology, the
results of research on higher order thinking skills do
vary. Overall, however, the teaching of higher order
thinking skills seems to enhance the students' thinking
skills and their academic performance.
Literature about Latino Students' Instruction
This section presents a discussion and studies of
Latino students' thinking skills instruction. Current
teaching practices in thinking skills and their
application to language arts and computers are addressed.
Initially, issues about instruction are presented.
As cited in the first chapter of this study, the
reading and mathematics achievement of English learning
Latino students is, on average, below grade level.
Carrasquillo (1996) reports, "Hispanics have the lowest
levels of education attainment of any major population
group" (p. 4). As an example, 20% of the Hispanic young
adults are dropouts and another 10% are non-enrollees.
This is compared to 8.6% of non-Hispanic white and 12.1%
non-Hispanic black young adults (Krashen, 1998).
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
As reported by Carrasquillo (1996), many factors
contribute to this low level of achievement. Two of these
factors are the limited English proficiency of the
students, as well as the limited ability of many teachers
to communicate in the students' dominate language.
Another factor is the inadequate training of teachers in
effective, specialized strategies for communicating in
English with the students. Also, teachers need training
in techniques for developing the English learning
students' academic English. Additionally, there are
issues concerning the accuracy of assessment of English
learning students when standardized achievement
examinations are written in English. "Because language
and content are intricately intertwined, it is difficult
to isolate one feature from the other in the assessment
process" (p. 7).
That many Latino students fall behind their peers and
are often placed in remedial programs, is a known fact.
For example, 35% of the English learning students in the
United States participate in Title 1 programs
(Carrasquillo & Rodriguez, 1996). Title 1 programs are
designated for low achieving students in low socioeconomic
areas.
As the literature reflects, many educators are
concerned that remedial programs are frequently
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
40
characterized by low level thinking skills and repetitive
tasks. Pertaining to remedial reading programs, Garcia
and Pearson (1991) express concern about the curriculum.
They observed that the low achieving readers often do not
have the opportunity to read real texts or to write
because they have not demonstrated complete mastery of
word-level processes.
This concern was echoed by Gersten (1996) when
studying instructional practices for reading. He found
that little attention is given to teaching or promoting
comprehension strategies in classrooms with many language
minority students. This trend continues even in the
middle and later elementary grades when such instruction
is crucial. Teachers tend to focus on word recognition
and pronunciation. In summary, several citings of
literature about English learning students indicate that
they are not receiving higher level thinking skills which
could assist them with improving their reading
comprehens ion.
In a large scale study, Jimenez, Garcia, & Pearson
(1996, p. 57) "suggest that successful bilingual readers
use certain strategies for comprehending both Spanish and
English texts. They focus on unknown words and use
cognates as one source of knowledge. They also monitor
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
41
their comprehension, make inferences, and actively use
prior knowledge" (August & Hakuta, Eds. 1997).
In Educating Latino Students: A Guide to Successful
Practice. Sosa (1998) briefly presents a description of
higher order thinking that is taught through reciprocal
interaction teaching. He explains that the students are
not passive recipients of information. They are active
participants in the learning of this program. This type
of lesson involves students in genuine dialogue with the
teacher. The teacher serves as a facilitator and guide.
Sosa states this model emphasizes higher order thinking,
meaningful language use by students, language integration
across the curriculum, and intrinsic motivation. In
Sosa1s description, Garcia (1987) reports on instructional
strategies used in effective bilingual classrooms. He
encourages student to student interactive discourse. He
states that such strategies are important to enhance
cognitive and linguistic development.
Carrasquillo and Rodriguez (1996) also report on the
social interact ions of the students. For Latino English
learners "small collaborative groups are recommended to
ensure oral participation. Large group discussions tend
to intimidate LEP students, and this lack of confidence or
intimidation often translates into silence" (p. 87).
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
42
Another term for English learning students is limited
English proficient (LEP) students.
Computers have become an important aspect in teaching
higher order thinking skills to Latino English learners.
According to a study by the President's Advisory
Commission on Educational Excellence for Hispanic
Americans (1996), computer literacy is a basic skill of an
information based economy, yet Latinos do not have the
same access to technology in their schools and homes as
their peers.
While computer-based learning programs often have
many of the same shortcomings as traditional instruction,
the increasing application of artificial intelligence
techniques to instructional computing designs promise to
alter this situation. New programs may offer tools for
reflective thought and planning that should assist wise
strategy use (Collins & Brown, 1988; Wenger, 1987).
Fitzgerald (1995) reviewed 67 studies of cognitive
reading processes of English learners analyzing
metacognitive strategies, vocabulary, prior knowledge, and
effects of types of text structures. One-fourth of the
reports were concerning fourth through seventh grade
students with most of the other subjects representing
eighth grade through college level students. From the
results of the studies, Fitzgerald concluded that English
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
43
learners did learn reading via the same principles of
instruction taught to native English speakers. The
differences were slower reading and depressed activation
of selected cognitive processes for English language
learning students. For example, Knight et al. (1986)
found third through fifth grade English learners used
fewer metacognitive strategies for comprehension
monitoring than did the native English speaking students.
Fitzgerald (1995) suggests that one of the main
instructional implications of the research surrounding
English learning Latino's cognitive reading process is
that teachers might display even more than normal patience
with English learners and that they take extra care when
wording questions and making interactive comments in order
to maximize the opportunity for activation of thought
process.
Nanez and Padilla (1995) conducted research on 49
youth who were 15 to 17 years old. The students ’ reaction
times were measured as they proceeded through the Raven
Advanced Progressive Matrices of cognitive information
processing tasks. Both groups responded with the same
accuracy. The results indicated that the students who
knew very little Spanish were significantly faster than
the other students. The other students were fluent
English proficient and knew about equal amounts of English
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
44
as Spanish. In the discussion, the researchers suggested
that the students with about equal amounts of English as
Spanish may have more information to mentally scan because
they have two languages. This information, along with the
findings from Fitzgerald, suggest that English learning
students may need additional time to process information
and apply it.
Rowe's technique (1974) of wait time may be of
additional value to English learning students and fluent
English proficient students. After studying 300 classroom
tape recordings, Rowe found that the average teacher's
wait time was one second after asking a question before he
called on a student to answer it. When the teacher
increased wait time, the students' answers were more
logical, their responses increased, and participation by
"slower" learners increased. Extending wait time would
seem to be beneficial, especially when English learning
students and English proficient students are encountering
complex, higher order thinking skills.
The importance of higher order thinking skills for
English learners was emphasized in a speech by
Carrasquillo at the American Educational Research
Association in April 1996. While expressing the need to
utilize the first language and the cultural heritage of
the students during instruction, Carrasquillo outlined
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
45
characteristics of a "first class curriculum" for
linguistically and culturally diverse student populations.
Included in these factors are comprehension, critical
thinking, problem solving and writing skills.
Comprehension is defined as an active and goal oriented
construction of coherent mental representation based on
newly acquired information and prior knowledge. The
"first class curriculum" utilizes problem-situated
learning for the development of intellect, learning to
learn, decision making, creativity and problem solving
which become the subject matter of instruction.
Higher Order Thinking Skills Program
The experimental treatment for the current study is
the Higher Order Thinking Skills (HOTS) Program, a general
thinking skills curriculum (Pogrow, 1990b, 1996a, 1997a).
The development of this instruction began in 1983 as a
mentally challenging program for Title I students to
replace remedial lessons. It contains a two-year
curriculum that can be taught to students in grade levels
from fourth through seventh. Under the direction of
Pogrow at the University of Arizona, the thinking skills
lessons and computer software are continually refined and
updated. The basic goal of the HOTS Program is to develop
general thinking skills that transfer to a variety of
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
46
learning outcomes. The goals include producing gains in
reading and mathematics standardized test scores, as well
as transferring thinking skills to general classroom
lessons (Pogrow, 1995b).
The HOTS treatment is scheduled so students generally
attend forty-five minute sessions four times a week
throughout the school year. With approximately half of
the students in the class, the teacher leads Socratic
dialogues while the other students are at the computers.
Then the students rotate from the computers to the teacher
(Pogrow, 1995b).
During the teachers' lessons, they question the
students' thinking by asking clarifying and probing
questions. In this approach, the teachers incorporate
longer wait time after their questions and the students 1
answers encouraging the students to think more intensely.
This type of mediated dialogue guides and extends the
students' thinking skills as they solve problems. The
students gradually learn the following information
processing skills: metacognition, inferring information
from context, decontextualizing (generalizing)
information, and information synthesis. The metacognition
includes not only the students' selection of strategies,
but also their explanation of the rationale for their
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
47
selection, as well as an evaluation of the effectiveness
of the strategy (Pogrow, 1990a, 1990b).
The curriculum is designed so the students are not
only taught the general thinking skills, but they are also
motivated to apply them in interesting and challenging
situations. Many of the lessons provide dramatic scenes
where the teacher portrays a character who poses an
intriguing problem. This is followed by the Socratic
dialogue in which the students learn and practice their
skills. The students also apply these skills as they use
computer software that requires problem solving. In the
HOTS Program, these computer programs are called Learning
Dramas. Because the Learning Dramas are interesting, but
very challenging, the students focus on the available
directions and information as they develop tenacious
behavior (Pogrow, 1997b).
The students work in pairs or small cooperative
groups at the computers (Pogrow, 1997c). This type of
instruction provides an environment for interaction with
other students and is less intimidating than a formal,
total class and/or didactic discourse. During the lessons
of this research, the students could also use their native
language as they discussed the computer programs.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
48
Summary
The review of the literature presented the
contributions of major scholars and practitioners to the
field of cognition, specifically thinking skills. Related
to this compilation of works, a variety of curricular
programs have been developed. Educators are seeking
challenging, effective higher order thinking skills
curriculum for students, especially Latino English
learning students. Instructional leaders and program
developers (C. Estrada, personal communication, November
17, 1994) desire research of the effectiveness of these
programs. The next chapter presents the methodology for
the present research of the Higher Order Thinking Skills
Program.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
49
CHAPTER 3
METHODOLOGY
The purpose of this chapter is to describe the
procedures that were utilized to investigate the effects
of higher order thinking skills instruction on reading
achievement, mathematics achievement, and learning skills.
The subjects in this study were Latino students in grades
fourth and fifth. This chapter presents the general
characteristics of the sample population followed by the
descriptions of the research design, instructional
treatment, instrumentation, data collection processes, and
statistical analysis procedures. The chapter closes with
the methodological assumptions of the study.
Sample
The population for this research was Latino students
in a large urban school district. This district was one
of the ten largest in California with a total enrollment
of 56,637 students during the 1998-1999 School year. The
percent of English learning students was 78% for the
school district. The student ethnicity of the school
district was as follows: 91% Latino, 5% Asian and
Pacific islander, 3% Non-Latino White, and 1% African
American.
Additional demographic data provided information on
the socioeconomic status. During the 1998-1999 school
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
50
year, 84% of the elementary students were eligible for the
Federal Free and Reduced Lunch Program, a circumstance
indicating that their parents meet the criteria for
poverty level incomes.
The subjects for this research, fourth and fifth
grade students, totaled 821. Two hundred eighty-eight of
these students received the Higher Order Thinking Skills
(HOTS) Program, a general thinking skills curriculum.
These were all of the students in the school district who
received the thinking skills program. The remaining 533
students participated in regular classroom instruction not
receiving a specific thinking skills treatment. These
subjects were randomly selected from eight elementary
schools and their data served as control information for
hypotheses comparing participation and non-participation
in the Higher Order Thinking Skills Program.
As was typical of the school district, a high
percentage of the subjects, 81%, were English learning
Latino students. These students were reading in English,
as well as receiving instruction in English language
development. Fluent English proficient Latino students
represented 16% of the subjects, and the remaining 3% of
the subjects were native English speakers designated as
English only in language proficiency. Table 2, which
follows, lists the number of subjects in the language
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
51
fluency groups partitioned by instructional treatment and
gender.
Table 2
Lancraaae Fluencv Treatment Group M F Total
English Learner HOTS 110 116 226
Fluent English HOTS 28 27 55
English Only HOTS 4 3 7
English Learner Control 229 208 437
Fluent English Control 37 42 79
English Only Control 5 12 17
Totals 413 408 821
Research Design
The research design for this study is in the quasi-
experimental category. The purpose of this category of
designs is "to approximate the conditions of the true
experiment in a setting which does not allow the control
and/or manipulation of all relevant variables" (Isaac &
Michael, 1995, p. 58). Using this approach, it was
possible to study the effects of an instructional program
of thinking skills that was in operation in school
settings.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
52
For the research about the effect of a higher order
thinking skills instruction on academic achievement, this
study utilized the procedures of the Pretest-Posttest
Control-Group Design. The chart below lists the major
elements of this design.
Table 3
Group Assignment Pretest Treatment Posttest
Experimental 01 T
02
Control R 01 02
Since the selection of the schools, teachers, and
students had already been completed for the experimental
groups, random assignment (R ) was not appropriate. The
control groups of students were randomly selected. The
pretest (Oj) and posttest (02 ) were conducted on the
dependent variables which included performance on the
reading comprehens ion and mathematics problem solving
sections of the Stanford 9. The treatment (T), an
independent variable, was the Higher Order Thinking Skills
instruction.
For the research about the effects of higher order
thinking skills instruction on the application of learning
skills, the model is a Posttest-Only Control-Group Design
(Borg & Gall, 1989). The teachers of students who
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
53
received the treatment and the teachers of the control
students completed questionnaires during spring of 1999.
The results of the two sets of questionnaires were then
compared. Characteristics of the testing and treatment
for these two research models are explained later in this
section.
The research design and the statistical procedures
were used to control the internal validity which addresses
the extent to which the independent variable produces the
change in the dependent variable. The control group
prevented mistaking effects of history, pretesting,
maturation, and instrumentation for the main effect of the
treatment. Because the control group was randomly
selected students in schools that were similar to the
treatment schools in language fluency and Stanford 9
scores, if regression toward the mean did occur, it should
be similar for both groups. Mortality effects were
controlled by checking pretest and posttest records.
The research design and statistical analysis provided
partial control of the external validity which is the
generalizability or representativeness of the experimental
findings. The extraneous variable of pretesting
interaction was lessened because the Higher Order Thinking
Skills Program is dissimilar from the Stanford 9. The
reflective effects of the study were reduced in comparison
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
54
to a true experiment due to the fact that the treatment
groups were already in operation and were not disrupted by
the experimental design.
Instructional Procedures
Pogrow’s program, entitled Higher Order Thinking
Skills (HOTS), served as the treatment for the students in
this study. The students attended HOTS instruction in the
computer laboratory during forty-five minutes sessions,
four days a week for a year. They used approximately half
of the time participating in the thinking skills lessons
with their teachers and the remainder of the period
applying these skills at the computers.
Higher Order Thinking Skills consists of two years of
curriculum that focuses on general thinking skills. It is
designed for students between fourth and seventh grades
when many students experience difficulty with reading.
The program was developed as an alternative to remedial
education of a repetitive nature. Instead of practicing
low level skills, through the teachers' lessons and
computer assisted instruction, it provides challenging,
thinking skills instruction. This instruction consists of
motivating Socratic questioning and problem solving via
computer simulation programs. From the Socratic
dialogues, the students develop metacognition, inferring
information from context, decontextualizing information,
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
55
and information synthesis. The students apply these
skills as they use computer software that requires problem
solving, such as Oregon Trail and Where in the World Is
Carmen San Diego (1990)?
To prepare to teach the Higher Order Thinking Skills
curriculum, all of the teachers attended five days of
staff development about the program. They were provided
notebooks of lessons for two years and use of the computer
software. The focus of the staff development was the
intensive instruction in the Socratic questioning methods
with guided practice in their use. After the HOTS
instructor modeled two lessons, each participant presented
three lessons and received constructive criticism from the
other participants and the instructor. A copy of the
agenda for the staff development is provided in Appendix
A. During the 1998-1999 school year, the HOTS Newsletter
and technical support were available to the teachers.
Instrumentation
Three instruments were used in this study. The first
was the Stanford Achievement Test Series (Stanford 9)
Ninth Edition, (1996). The second was a teacher
questionnaire to assess the application of the students'
learning skills. The use of this questionnaire is
described in the Data Collection Procedures section. The
third instrument was the site facilitator's interview
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
56
sheet, used to gather descriptive information for the
researcher. It yielded general information about the
teachers' training, grouping of students, and
instructional procedures which were implemented at the
sites.
The Stanford 9 is a nationally normed achievement
test administered to elementary and secondary students.
The tests include measures of reading comprehension, and
mathematics problem solving, as well as other content
areas. On the test levels for fourth and fifth grades,
74% and 78% of the reading comprehens ion items are
identified as thinking skills questions. At these same
grade levels, 52% and 54% of the mathematics problem
solving items are considered to be thinking skills
questions. Scores are typically reported as percentiles,
standard scores, and Normal Curve Equivalent points
(Harcourt Brace, 1997).
Standardization of the Stanford 9 involved testing
students from 49 of the United States and the District of
Columbia. The process began in 1995 with random samples
of 250,000 students during the spring and another 200,000
during the fall. In addition, a sample of 80,000 students
participated in the study for equating the levels, forms,
and the Stanford, Eighth Edition. The variables of
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
57
socioeconomic status, urbanicity, and ethnicity were
analyzed (Impara & Plake, 1998).
Information about the testing of English learners is
briefly included in the Stanford Achievement Test Series,
Ninth Edition; Technical Data Report (Harcourt Brace &
Company, 1997). The English learners are referred to as
"LEP" which means limited English proficient students. In
the section titled Participation by Students with
Disabilities and LEP Students, the authors state that
students receiving instruction as part of a regular
education classroom who would normally test with other
students in the regular classroom were asked to be part of
the standardization sample. The remainder of the section
discusses mentally handicapped students, but it closes
with the statement, "There are no definitive national
figures available for the number or percentage of students
with disabilities or LEP students" (p. 27).
Procedures were implemented to eliminate bias in the
content of the Stanford 9. An advisory panel of two
people of major ethnic groups, including Hispanic
representation, was established to review the passages and
items. The Mantel-Haenszel procedures were used to
examine differential item functioning between reference
(majority) and focal (minority) groups. An item was
considered to be potentially bias if its Chi-square was
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
58
greater than what would normally be expected by chance.
Items with differences greater than chance were flagged
for review and possible exclusion from the final forms of
the test (Harcourt Brace, 1997).
To determine reliability of internal consistency,
several measures were employed. The following is a list
of the procedures applied to the Stanford 9: Kuder-
Richardson Formula 20, coefficients and standard errors of
measurement; the Kuder-Richardson Formula 21 coefficients
and standard errors of measurement; and alternate-forms
coefficients and standard errors. Interrater coefficients
using the Pearson correlation and the Spearmean-Brown
Prophecy formula were used to assess the descriptive,
expository, narrative, and persuasive writing assessment.
The coefficients were in the acceptable range of the .80s
to .90s for most of the tests and subtests for the full
multiple-choice battery (Impara & Plake, 1998).
Validity, the extent to which the test is an
appropriate measure, is divided into the categories of
content, criterion related, and construct. Most critical
is that the content of the test represents the curriculum.
Harcourt and Brace provide the Stanford 9 Compendium of
Instructional Objectives for this comparison. The degree
to which the curriculum is effectively taught and the
extent that the students complete the tests impact the
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
59
validity. With the exception of the high school
mathematics and science tests, the completion rates for
the multiple-choice subtests were in the 90s (Impara &
Plake, 1998).
Data Collection Procedures
During spring 1998, the students in the ten schools
were pretested with the Stanford 9 using the examinations
for their grade levels. The same procedures were utilized
during spring 1999 for the posttesting. Their teachers,
as part of the school district's regular testing cycle,
tested the students in their classrooms. Security of the
tests and guidelines for directions were monitored by the
site administrators. The tests were machine scored by CTB
McGraw-Hill and results complied by computer CTB Test Mate
software. Aggregated means were compiled on a
spreadsheet.
After the research procedures for this study were
approved by the school district administration, the data
were gathered by the researcher from several schools and
departments. At the schools where the Higher Order
Thinking Skills (HOTS) Program was taught, the site
facilitators provided the teachers’ and students 1 names.
The names for the control group were made available
through lists from the Pupil Support Services Department .
and participants were randomly selected. The scores were
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
60
available through the Research and Evaluation Department.
The students1 grade levels and English proficiency levels
were recorded by the classroom teachers and bilingual
resource teachers as part of the students' regular
records. This information and students' genders were
available to this researcher via the Computer Accounting
Student Terminal System of the school district.
The teacher questionnaire was duplicated and
explained to the classroom teachers of the research
subjects at meetings at their school sites. The
questionnaire contained eight items with a response range
from "1" to "5" for "not yet" to "usually" indicating the
degree to which the student demonstrated the behavior
addressed by each item. The teachers completed a
questionnaire for each student they taught in either the
thinking skills treatment group or in the control group.
The questionnaire is included in Appendix B.
The information from the questionnaires was added
to an Excel database compiled for this study. The
questionnaires were maintained in a locked filing cabinet.
In the database, each subject's name was replaced with a
code number for anonymity. The questionnaires, lists of
students 1 names, and the database were destroyed when the
study was completed.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
61
The staff interviews were conducted with the Higher
Order Thinking Skills site facilitators at the two schools
using this program. These staff members were the resource
teachers who coordinated the program for their school.
They were asked to describe the program including the
utilization of instructional strategies, grouping of
students for lessons, training of staff, support of the
program and the physical environment. A copy of the
interview questions is included in Appendix C. The
information was used by the researcher to further
understand the implementation of the thinking skills
program at the two school sites.
Statistical Analysis
To assess the effects of the Higher Order Thinking
Skills Program on English language learning Latino
students' academic achievement gains and learning skills,
the analysis of variance (ANOVA) statistical tests were
administrated for each null hypothesis. The purpose of
this analysis is to assess whether means on the dependent
variable are significantly different for each group
(Green, Salkind, & Akey, 1997). Statistical significance
was set at a priori alpha level of <.05 which means that
the chance of an error in claiming an independent variable
explains or predicts the dependent variable is less than
five chances out of one hundred. The Statistical Package
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
62
for the Social Sciences (SPSS) was the software program
used to analyze the scores.
To determine the level of significance for each null
hypothesis, the tests of between-subjects effects yielded
data about the following statistics: the fit of the
model, the intercept, the weight of the independent
variables, as well as the error and the R Squared terms.
The sum of squares, degrees of freedom, mean square, F
value, and statistical significance were calculated for
the first four items. For Null Hypothesis 2, post hoc
pairwise comparisons were used to determine significance
between groups. For Null Hypothes is 7, t tests were
conducted because there were eight dependent variables
which were being examined. The t tests, called post hoc
multiple comparisons, analyzed the results to determine
which of these variables were statistically significant.
Methodological Assumptions
Six methodological assumptions were present in this
study. They are as follows:
1. The test instruments were relatively reliable
normed measures of the skills they represent to
measure.
2. The test administration was uniformly applied
according to the conditions necessary for normed
comparability.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
63
3- The data were correctly recorded and analyzed.
4. The teacher questionnaire was a valid tool for
recording the teachers' observations of the students'
classroom behavior.
5. The respondents to the interview questions
answered without bias.
6. The research design and statistical procedures,
as well as the standardized tests, were appropriately
used.
Summary
Chapter 3 presented information concerning the
methodology for this research about thinking skills
instruction. Initially, the characteristics of the
sample, totaling 821 upper grade elementary students, were
explained. Next, each of the elements of the quasi-
experimental research design was outlined, and its
utilization was described. The instrumentation components
including the Stanford 9, the teacher questionnaire, and
the site facilitators' interviews were presented.
Subsequently, the procedures for collecting the data and
statistically analyzing the findings were presented. The
chapter concluded with the recognition of the
methodological assumptions.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
64
CHAPTER 4
ANALYSIS AND DISCUSSION OF FINDINGS
. This study was designed to determine whether
instruction in higher order thinking skills increases the
academic gains and learning skills of Latino English
learning students. This chapter presents the statistical
findings related to each of the seven research questions
that were stated in the first chapter of this
dissertation. For research purposes, the seven questions
are presented in the form of null hypotheses in Chapter 4.
Following the report of the findings, a discussion of the
results is presented.
Summary of the Research Questions and
Null Hypotheses
The following are the seven research questions for
this study preceding their related null hypotheses.
Research Question 1 Are there significant
differences in reading comprehens ion test score gains
among groups of Latino students defined as English
learners, fluent English proficient, and English only who
received the Higher Order Thinking Skills Program?
Null Hypothesis 1 There are no significant
differences in reading comprehens ion test score gains
among groups of Latino students defined as English
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
65
learners, fluent English proficient, and English only who
received the Higher Order Thinking Skills Program.
Research Question 2 Are there significant
differences in mathematics problem solving test score
gains among groups of Latino students defined as English
learners, fluent English proficient, and English only who
received the Higher Order Thinking Skills Program?
Null Hypothesis 2 There are no significant
differences in mathematics problem solving test score
gains among groups of Latino students defined as English
learners, fluent English proficient, and English only who
received the Higher Order Thinking Skills Program.
Research Question 3 Is there a significant
difference in reading comprehension test score gains
between Latino English learners receiving the Higher Order
Thinking Skills Program and students with these
characteristics who are not receiving this program?
Null Hypothesis 3 There is no significant difference
in reading comprehension test score gains between Latino
English learners receiving the Higher Order Thinking
Skills Program and students with these characteristics who
are not receiving this program.
Research Question 4 Is there a significant
difference in mathematics problem solving skills test
score gains between Latino English learners receiving the
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
66
Higher Order Thinking Skills Program and students with
these characteristics who are not receiving this program?
Null Hypothesis 4 There is no significant difference
in the mathematics problem solving test score gains
between Latino English learners receiving the Higher Order
Thinking Skills Program and students with these
characteristics who are not receiving this program.
Research Question 5 Is there a significant
difference in reading comprehens ion test score gains
determined by gender of Latino English learners who
received the Higher Order Thinking Skills Program?
Null Hypothesis 5 There is no significant difference
in reading comprehension test score gains determined by
gender of Latino English learners who received the Higher
Order Thinking Skills Program.
Research Question 6 Is there a significant
difference in the mathematics problem solving test score
gains determined by gender of Latino English learners who
received the Higher Order Thinking Skills Program?
Null Hypothesis 6 There is no significant difference
in the mathematics problem solving test score gains
determined by gender of Latino English learners who
received the Higher Order Thinking Skills Program.
Research Question 7 Is there a significant
difference in application of learning skills between
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
67
Latino English learners receiving the Higher Order
Thinking Skills Program and those students with the same
characteristics who did not receive the program?
Null Hypothesis 7 There is no significant difference
in application of learning skills between Latino English
learners receiving the Higher Order Thinking Skills
Program and those students with the same characteristics
who did not receive the program.
Analysis of Findings
Null Hypothesis 1 There are no significant
differences in reading comprehension test score gains
among groups of Latino students defined as English
learners, fluent English proficient, and English only who
received the Higher Order Thinking Skills Program.
The purpose of the investigation of this hypothesis
was to analyze whether there were differences in reading
gains for Latino students according to their language
fluency when they participated in the thinking skills
curriculum for one year. An analysis of variance was
conducted on the standardized reading comprehens ion test
results for a total of 252 students. The students' scores
were divided into three groups according to the students'
English language fluency. One hundred ninety-one of the
students were Latino English learners. Fifty-four of the
subjects were fluent English proficient Latino students
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
68
and the remaining seven subjects were classified as
English only Latino students. This latter group spoke
English as their native language.
The analysis of variance calculated data about the
influence of the independent variable, i.e. the English
language fluency of the students. The dependent variable
was the 1999 Stanford 9 reading comprehension scores
measured in Normal Curve Equivalent (NCE) points.
The descriptive statistics for this hypothesis
present the means and standard deviations of the reading
comprehens ion examination f or each of the three English
language fluency groups. For the English learners, the
mean was 32.26 and the standard deviation was 12.16. The
fluent English proficient students scored a mean of 52.20
and a standard deviation of 12.98. The third group, the
English only students, had a mean of 46.03 and a standard
deviation of 11.96.
Using the information from the descriptive
statistics, the tests of between subjects effects were
completed. These findings present the proportion of
variance explained by the various sources and their levels
of significance. The analysis of variance model was a
very effective method for testing this hypothesis with its
statistical significance at p < .001. The findings
indicated that the proportion of variance explained by the
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
69
factors is a R Squared = .57. This means that 57% of the
variance in reading comprehens ion scores was accounted for
by the variables included in the model. However, the
independent variable, English language fluency, was not
found to be statistically significant at the level of p <
.05. These results included the mean square of 173.25, an
F value of 1.80 and the statistical significance of . 17.
Therefore, Null Hypothesis 1 was not rejected.
Null Hypothesis 2
There are no significant differences in the
mathematics problem solving test scores gains among groups
of Latino students defined as English learners, fluent
English proficient, and English only who received the
Higher Order Thinking Skills Program.
Data for this hypothesis were studied to determine
whether there were differences of gains in mathematics
problem solving scores according to language fluency
grouping of Latino students who received a year of
thinking skills curriculum. The independent variable was
the English language fluency level of the students. The
dependent variable was the 1999 Stanford 9 mathematics
problem solving gains scores reported in Normal Curve
Equivalent (NCE) points.
An analysis of variance was conducted using these
Stanford 9 mathematics scores for a total of 260 Latino
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
70
students. One hundred ninety-nine of the students were
English learners. Fifty-four of the students were fluent
English proficient, and seven others were classified as
English only students.
The descriptive statistics for this investigation of
mathematics scores are listed below in Table 4.
Table 4
Means and Standard Deviations for Mathematics Scores in
MCE Points for Latino Students by English Language Fluency
Groups
Lanauaqe Fluency Mean Standard Deviation Number
English Learners 35.64 13.40 199
Fluent Proficient 59.93 14.15 54
English Only 44.63 10.56 7
Total 40.93 16.68 260
As the data on this table show, differences did exist
among the groups of Latino students. The mean was 35.64
and the standard deviation was 13.40 for the students
designated as English learners. The fluent English
proficient students' mean was higher, 59.93, and their
test results indicated a standard deviation of 14.15. The
mean and standard deviation for the seven English only
students, who spoke English as their native language, were
44.63 and 10.56 respectively.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
71
Based on the descriptive data, the analysis of
variance procedures were completed to determine if
statistical significance was indicated at the p < .05
level. Table 5 displays the results for the calculations.
Table 5
ANOVA for Effect of Language Fluency on Mathematics Scores
Presented in NCE Points for Latino Students
Sum of Mean
Source 'Squares df Square F Siq.
Model
40992.77^
3 13664.26 112.77 .001
Intercept 4565.33 1 4565.33 37.68 .001
Math 98 15821.41 1 15821.41 130.57 .001
Language 1658.21 2 829.11 6.84 .001
Error 31019.55 256 121.17
Total 72012.32 259
a. R Squared = .56
This statistical testing analyzed the effect that
English language fluency had on mathematics achievement as
indicated by the Stanford 9 mathematics problem solving
examination. The data in Table 5 present the sources of
variation and the results that lead to their level of
significance. The model, with a F value of 112.77 and
level of significance at p <.001, indicated that it was a
valuable design for testing this data. The R Squared
value of .56, which appears below the columns, indicated
the proportion of variance explained by a combination of
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
72
intercept, mathematics 1998 scores and the language
fluency. Representing 56% of the variance in the
dependent variable, the 1999 mathematics scores, the R
Squared value was considered to be moderate. The results
of the independent variable, the level of English fluency,
included the mean square of 1658.21 and F value of 6.84
which produced statistical significance at the p < .001
level. This indicated that there is very little chance
that these results were derived from sampling error.
Thus, Null Hypothesis 2 was rejected.
To discern which of the pairs of language groups
statistically differed, the researcher conducted post hoc
comparison tests. The pairwise comparisons of the means
indicated that only the English Learners Group and the
Fluent English Proficient Group differed significantly
(p < .001).
Null Hypothesis 3
There is no significant difference in reading
comprehens ion test score gains between Latino English
learners receiving the Higher Order Thinking Skills
Program and students with these characteristics who are
not receiving this program.
An analysis of variance was conducted to evaluate the
relationship between participation in the Higher Order
Thinking Skills Program and improvement in reading
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
73
comprehension. The independent variable was the
instruction in the thinking skills curriculum and the
dependent variable was the 1999 Stanford 9 reading
comprehens ion test results.
One hundred ninety-one Latino students who were
English learners were in the treatment group that had
received three hours of the thinking skills treatment each
week for a year. Three hundred sixty-eight Latino English
learners who did not participate in thinking skills
program composed the control group for a total of 559
subjects involved in this segment of the study.
The descriptive statistics related information
concerning the Stanford 9 reading score gains reported in
Normal Curve Equivalent (NCE) points using scores from
1998 and 1999. The mean and standard deviation for the
treatment group were 32.26 and 12.16 respectively. The
mean and standard deviation for the control group were
31.28 and 11.06 respectively.
Tests of the between subject effects resulted in an F
(1,556) = 1.13 and thus a p value of .29. Concerning the
test model, the R Squared value was .30. Since the p value
was greater than p < .05, the results did not meet the
criterion of the level of significance. Thus the Null
Hypotheses 3 was not rejected.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
7 4
Null Hypothesis 4
There is no significant difference in mathematics
application and concepts test scores gains between Latino
English learners receiving the Higher Order Thinking
Skills Program and students with these characteristics who
are not receiving this program.
An analysis of variance test was utilized to
calculate the relationship between participation in the
Higher Order Thinking Skills Program and improvement in
mathematics for English learning Latino students. The
between-subject factors for this hypothesis included a
total of 566 Latino English learners. One hundred ninety-
nine subjects were in the treatment group that had
received three hours of thinking skills treatment each
week for a year while 367 students composed the control
group that did not participate in the thinking skills
program.
The descriptive statistics related information
concerning the dependent variable, the 1999 Stanford 9
mathematics application and concept gains score reported
in Normal Curve Equivalent points. The mean and standard
deviation for the treatment group were 35.64 and 13.40
respectively. The mean and standard deviation for the
control group were 34.48 and 13.90.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
75
The ANOVA for the between subject effects produced a
F (1, 563) = .56 and a p value of .453. The R Squared
result was .36. Since the p value was greater than .05,
it was not statistically significant. Therefore, Null
Hypothesis 4 was not rejected.
Null Hypothesis 5
There is no significant difference in reading
comprehens ion test scores gains that is determined by
gender of Latino English learners receiving the Higher
Order Thinking Skills Program.
An analysis of variance was conducted to determine if
there was statistically significant difference between the
scores of male and female subjects. The independent
variable was gender, and the dependent variable was the
reading scores. The between-subject factors included a
total of 191 students. This total represented 90 male and
101 female students who had received three hours a week of
thinking skills treatment for a year. The measurement for
reading was the Stanford 9 reading comprehension test
scores gains which were reported in Normal Curve
Equivalent points.
The results of the calculations showed the mean and
standard deviation for the male subjects were 32.72 and
11.97 respectively. The mean and standard deviation for
the female subjects were 31.97 and 12.33. Using these
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
76
figures, the tests of the between-subject effects resulted
in an F (1, 189) = .94 and a p value of . 76. Since this
was greater than p < -05, the criterion for statistical
significance was not met. Therefore, Null Hypothesis 5
was not rejected.
Null Hypothesis 6
There is no significant difference in the mathematics
problem solving test scores gains that is determined by
gender of Latino English learners receiving the Higher
Order Thinking Skills Program.
As with the tests for the prior hypotheses, the
analysis of variance test was conducted for Null
Hypothesis 6. The purpose was to determine if there was a
statistical significance between male and female students'
test scores gains in mathematics. All of the students
were Latino English learning students who had received
three hours a week of higher order thinking skills lessons
for a year. The independent variable was gender and the
dependent variable was mathematics tests results. The
improvement in mathematics was based on the mathematics
problem solving component of the Stanford 9 reported in
Normal Curve Equivalent points.
A total of 199 students met the criterion so that
their scores were utilized in this analysis. Ninety-four
were male subjects and 105 were female subjects. The
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
77
results of the analysis of variance stated that the mean
for the males was 36.24 and their standard deviation was
14.39. Similarly, the mean for the females was 35.24 and
their standard deviation was 12.76. Analyzing these
figures, the F value (1,196) was 2.44 resulting in g .12.
Because this was greater than the g < .05 criterion, these
data were not statistically significant. Thus, Null
Hypothesis 6 was not rejected.
Null Hypothesis 7
There is no significant difference in application of
learning skills between Latino English learners receiving
the Higher Order Thinking Skills Program and those
students with the same characteristics who did not receive
the program.
To investigate the relationship between receiving the
thinking skills curriculum and utilization of learning
skills for Latino English learners, an analysis of
variance test was conducted. The independent variable was
students' participation in the Higher Order Thinking
Skills (HOTS) curriculum and the dependent variable was
their application of learning skills.
To gather appropriate data for this hypothesis, the
classroom teachers of the students completed an eight-item
questionnaire. The teachers rated their students on
application of learning skills, such as making
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
78
predictions, selecting strategies to test the accuracy of
their predictions, and explaining their process for
arriving at their answers. The questionnaire is included
as Appendix B.
The teachers returned a total of 635 questionnaires
for their Latino English learners. Of these, 224
reflected the learning skills of students in the treatment
group, those who received Higher Order Thinking Skills
(HOTS) instruction three hours a week for a year. The
other 411 questionnaires were concerning the Latino
English learners in the control group, students who were
not instructed in Higher Order Thinking Skills lessons.
Table 6 presents a summary of the descriptive
statistics for this investigation. The results of the
questionnaires for the treatment group produced a mean of
28.57 with a standard deviation of 8.35. The control
group's questionnaires reflected a mean of 24.35 with a
standard deviation of 7.69.
Table 6
Learning Skills Means and Standard Deviations for Latino
English Learners bv HOTS Treatment and Control Groups
Instruction_________Mean____ Standard Deviation Number
HOTS Treatment 28.37 8.35 224
Control 24.35 7.69 411
Totals 25.84 8.18 635
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
79
From the information of the descriptive statistics,
the between subjects effects were completed. Table 7
presents the proportion of variance explained by the
sources and their levels of significance.
Table 7
ANOVA for Effect of HOTS Treatment on Application of
Learning Skills of English Learning Latino Students
Source
Sum of
Sguares df
Mean
Square F Sig.
Model 2579.66a
1 2579.66 41.02 .001
Intercept 405344.90 1 405344.91 6445.26 .001
Treatment 2579.66 1 2579.66 41.02 .001
Error 39683.86 631 62.89
Total 42263.52 632
a R Squared = .061
Table 7 presents the results of the analysis of
variance testing. With one degree of freedom, a mean
square of 2579.66, and an F value of 41.02, the results
indicated that the treatment produced p < .001. Since the
E value met the p < *05 criterion, it was statistically
significant and therefore the null hypothesis was
rejected.
To specifically determine which of the skills were
significantly increased, individual-samples t tests were
conducted to evaluate the application of each of the
specific learning skills. These results, listed in
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
80
Table 8 provide the mean and standard deviation for each
of the skills.
Table 8
Individual Learning Skills Means and Standard Deviations
for English Learning Latino Students
Standard
Item instruction________Mean____ Deviation______Number
1 Treatment 3.60 1.12 224
1 Control 3.07 1.09 411
2 Treatment 3.40 1.20 223
2 Control 2.78 1.08 409
3 Treatment 3.34 1.26 224
3 Control 2.78 1.12 411
4 Treatment 3.46 1.26 222
4 Control 2.94 1.13 410
5 Treatment 3.38 1.18 224
5 Control 2.84 1.13 411
6 Treatment 3.76 1.11 224
6 Control 3.23 1.15 411
7 Treatment 3.71 1.16 224
7 Control 3.31 1.14 410
8 Treatment 3.92 1.10 223
8 Control 3.44 1.13 409
The descriptive information from the independent-
samples t tests presented a pattern of higher means for
the treatment group which received the higher order
thinking skills curriculum than the control group of
students who did not participate in the lessons. The data
from the questionnaire provided a range in means for the
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
81
treatment group from 3.34 to 3.93 and standard deviations
from 1.10 to 1.26. By contrast, the results of the
control group provided a range in means from 2.78 to 3.44
and standard deviations from 1.08 to 1.15.
To investigate whether these ranges in standard
deviations represented equal or unequal variances in
means, Levene's Test for Equality of Variances was
conducted. The results of the Levene's Test showed that
equal variances could be assumed on all items of the
questionnaire.
The results of the independent-samples t tests are
presented in Table 9. The t values, degrees of freedom,
level of significance, and mean difference are included
for each of the eight items.
Table 9
Independent-Samples T Tests for Significance of Learning
Skills of English Learning Latino Students
Item t df Siq.(2-tailed). Mean Difference
1 5.87 451.24 .001 .54
2 6.44 419.25 .001 .62
3 5.54 414.50 .001 .53
4 5.21 422.00 .001 .52
5 5.63 440.80 .001 .54
6 5.68 471.51 .001 .53
7 4.16 450.67 .001 .40
8 5.20 463.81 .001 .48
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
82
As shown in Table 9, the findings of the independent-
samples t tests provided the statistical information for
each of the eight learning skills items. The results
indicated that the treatment produced p < .001. Since this
meets the p <.05 criterion, Null Hypothesis 7 was rejected
for each of the learning skills.
This first section of Chapter 4 has presented the
findings for each of the seven null hypotheses. Through
the statistical procedures of analysis of variance and
post hoc tests, the levels of significance were determined
for the variables of the hypotheses.
Discussion of Findings
The second section of Chapter 4 presents a discussion
of the findings. Initially, a brief summary of the
findings are presented. Subsequently, the findings are
discussed in relation to the studies presented in the
review of the literature.
In summary, the findings for only Null Hypotheses 2
and 7 met the p <.05 criterion of significance. The post
hoc tests for Null Hypothesis 2 indicated that there was a
significant difference in mathematics test score gains
between the English Learners Group and the Fluent English
Proficient Group. The post hoc t tests for Null
Hypothesis 7 provided findings indicating that on all
eight of the learning skills items, students who received
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
83
the Higher Order Thinking Skills instruction performed
significantly higher than students not receiving this
thinking skills instruction.
When comparing the Higher Order Thinking Skills
treatment group with the control group, who did not
receive the treatment, the results of this study about
English learning Latino students did not indicate that the
treatment produced significant findings in reading or
mathematics achievement. Nor was gender a significant
factor.
The remainder of the chapter is devoted to a
discussion of the findings. Each research question will
be addressed through the corresponding null hypothesis.
The data for Null Hypotheses 1 and 2 were concerning
the effect of higher order thinking skills instruction on
reading comprehension and mathematics problem solving.
For each hypothesis, the scores were compared among the
Latinos who were English only, fluent English proficient,
and the English learning students. There were no
statistical differences among the three groups on reading
scores, but there was statistical significance for the
overall mathematics scores. The results of post hoc
t tests showed that only the fluent English proficient
students' scores and the English learning students' scores
were significantly different. The fluent English
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
84
proficient (FEP) students' scores on mathematics problem
solving were significantly higher.
These findings may have been influenced by the
criteria for being identified as FEP. The criteria was a
score at or above the 31st percentile and meeting language
and academic proficiencies. The English learning students
had not met those criteria. As a result, it would be
likely that fluent English proficient students would score
higher on achievement tests than English learning
students.
The fact that English only students did not score
significantly higher than the other two language fluency
groups may have been due to the small sample size. Only
seven students were in this sample.
The data for Null Hypotheses 3 and 4 were concerning
the effect of the Higher Order Thinking Skills Program on
reading comprehens ion and mathematics problem solving
gains of the English learning students compared with the
control group. The findings were not significant for
either of the hypotheses. The Higher Order Thinking
Skills Program was not shown to be effective at increasing
the reading or mathematics performance as measured by the
Stanford 9.
These results are contrary to some of the other
research on the Higher Order Thinking Skills (HOTS)
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
85
Program. DelGiudice's extensive research (1994), which
included the schools in the current study, presented
specific findings on the HOTS Program. During the second
year of her study, the HOTS Program produced average gains
of 10.4 NCEs when 3 NCEs were considered to be significant
according to the federal regulations for Title I schools.
The results of a study in Soldotna, Alaska showed
significant findings on the Iowa Test of Basic Skills
recorded in Normal Curve Equivalent (NCE) points. The
HOTS students scores averaged 12.12 NCEs, almost double
that of the control group (Pogrow, 1995b). Additionally,
Marzano's Tactic for Thinking Program, which also teaches
higher levels of thinking, was found to produce
statistically significant results (Detrick, 1988).
However, the results of the current study do concur
with two other investigations concerning the findings of
standardized test scores. Eisenman (1995) evaluated the
effectiveness of the HOTS Program on fourth and fifth
grade students1 academic progress, attitudes and specific
thinking skills. The findings were statistically
significant concerning attitudes and thinking skills.
Even though the students' academic gains improved more the
second year, they were not significant as measured by the
standardized achievement tests. Allison's research (1993)
on the Super Math and Reading Thinking Skills indicated
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
86
that the initial results on the Iowa Test of Basic Skills
were not statistically significant, but stated in-depth
statistical analysis was significant in reading. The
current study and the investigations listed in the
literature suggest inconclusive results when discussing
statistical findings of the effects of thinking skills
instruction on standardized achievement tests scores for
reading and mathematics.
Of the research reviewed, only Darmer's investigation
(1995) studied Latino or Hispanic students. Her treatment
group consisted of Hispanic (55%), Native American (41%),
and Anglo (4%) fourth and fifth grade students. Using
Swanson's metacognitive questionnaire (1993) and Woodcock-
Johnson Revised Cognitive Abilities Subtests (1989), she
reported the treatment group made significant growth on
all 22 of the comparisons. The English language fluency
of the students, however, was not reported in Darmer's
study.
The factors for Null Hypotheses 5 and 6 concerned the
effect of Higher Order Thinking Skills on reading
comprehens ion and mathematics problem solving test score
gains according to students grouped by gender. Again, the
findings suggested no significant differences. These
findings were supported by the overall results of both
Allison’s (1993) and Eisenman's (1995) research on HOTS
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
87
with fourth and fifth grade students. In Allison’s study,
there were no significant differences for students who
received HOTS instead of the traditional program. Post
hoc analysis did suggest that female students who received
one year of traditional remedial instruction and one year
of HOTS outperformed the male students in mathematics.
Eisenman did report one subset of scores, the specific
thinking skills of abstract relations, in which the fifth
grade female students had statistically higher scores than
the male students. Overall, as suggested by the research
of this study and other investigations, the Higher Order
Thinking Skills Program did not positively or negatively
discriminate by gender.
The data for Null Hypothesis 7 were analyzed to
investigate the effect of the Higher Order Thinking Skills -
(HOTS) Program on the students' application of learning
skills. The data for this hypothesis were compiled from
635 questionnaires about English learning Latino students.
They were completed by the students' classroom teachers
for the HOTS treatment group and the control group. The
findings indicated that each of the eight learning skills
behaviors was statistically significant. This study
indicates that the Higher Order Thinking Skills Program
produces a positive effect on the students' ability to
apply learning skills to classroom assignments.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
88
These findings seem to confirm elements of studies
and the professional literature on thinking skills
instruction. The investigation by Riesenmy, Mitchell,
Hudgins, and Edel (1991) found that students in fourth and
fifth grades transferred learning skills from
instructional discussion groups to class assignments.
These students applied task oriented behavior, strategies
and prior information to challenging problems.
Additionally, the general descriptions of the schools
where HOTS was taught indicate that the instruction in
thinking skills transferred to improvement in students'
report card grades and metacognitive skills. In these
studies, the students scored higher than the three Normal
Curve Equivalent points that designated significance
according to Title 1 regulations (Pogrow, 1995b).
In the professional literature, Sousa (1998) explains
that asking thought provoking questions, requiring
students to explain their reasoning, and making
connections to prior learning, do increase learning.
Specifically for the Latino English learning students,
Carrasquillo (1996) stressed the English learning
students' need for thinking skills instruction. She
articulated the desire for instruction in learning to
learn, problem solving, processing information, using
prior knowledge, and decision making.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
89
CHAPTER 5
SUMMARY, CONCLUSIONS, AND RECOMMENDATIONS
This chapter presents a summary of the study,
conclusions, and recommendations for future research. .
The summary includes a brief synopsis of the background,
purpose of the study, research methodology, and the
findings. Based on these findings, conclusions are drawn.
The chapter closes with recommendations for future
research.
Summary of Study
Background
With the onset of the Information Age and its global
economy, higher levels of academic proficiency have become
essential for success in society. To prepare current
students with the concepts and skills needed for the
increasingly complex Information Age society, all students
must be provided the opportunity to learn higher order
thinking skills. Such skills aid students in attaining
academic proficiency and augment their preparation for the
expanding fields of employment which draw upon these
skills (Paul, 1995).
As reported by the National Assessment of Educational
Progress (1998), only 5% of all students reach advanced
levels of proficiency in reading and mathematics, which
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
90
involve higher levels of thinking. Compounding the
challenge of achieving higher level skills is the fact
that an increasing number of students are non-native
English speakers. They are learning English as students
in the school systems of the United States. Latino
students are the fastest growing ethnic group in America,
and increasing numbers of these students are not fluent
English proficient. It is projected that by 2020, 20% of
the students in the United States will be of Latino origin
(U.S. Federal Government, 1998).
The educational systems in America have not provided
successful instructional programs for the Latino students.
As reported by the National Research Council (1997),
Latino students, as a group, score below their non-Latino
peers in reading in elementary school and are an average
of four years behind in secondary school. Subsequently,
because many Latino students fall behind their peers, they
are often taught remedial curricula that stress discrete,
low level skills instead of higher order thinking skills
(Carrasquillo & Rodriguez, 1996; Garcia 1995; Garcia &
Pearson, 1991).
Purpose
The purpose of this study was to examine whether
instruction in higher order thinking skills would have a
positive effect on the academic achievement of Latino
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
91
students who were English learners. Specifically, the
study investigated performance in reading comprehension
and mathematics problem solving of students being
instructed in the Higher Order Thinking Skills. Also,
examined was the effect of gender on performance in these
areas when students received this thinking skills
curriculum. Additionally, the study investigated whether
the higher order thinking skills instruction would improve
the application of the students' learning skills during
content lessons taught in the classroom.
Research Questions
1. Are there significant differences in reading
comprehens ion test score gains among groups of Latino
students defined as English learners, fluent English
proficient, and English only who received the Higher Order
Thinking Skills Program?
2. Are there significant differences in mathematics
problem solving test score gains among groups of Latino
students defined as English learners, fluent English
proficient, and English only who received the Higher Order
Thinking Skills Program?
3. Is there a significant difference in reading
comprehens ion test score gains between Latino English
learners receiving the Higher Order Thinking Skills
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
92
Program and students with these characteristics who are
not receiving this program?
4. Is there a significant difference in mathematics
problem solving test score gains between Latino English
learners receiving the Higher Order Thinking Skills
Program and students with these characteristics who are
not receiving this program?
5. Is there a significant difference in reading
comprehens ion test score gains determined by gender of
Latino English learners who received the Higher Order
Thinking Skills Program?
6. Is there a significant difference in the
mathematics problem solving test score gains determined by
gender of Latino English learners who received the Higher
Order Thinking Skills Program?
7. Is there a significant difference in application
of learning skills between Latino English learners
receiving the Higher Order Thinking Skills Program and
those students with the same characteristics who did not
receive the program?
Methodology and Procedures
Sample The sample included a total of 821 fourth and
fifth grade Latino students. Of these students, 288
received the Higher Order Thinking Skills Program, the
treatment. The remaining 533 students did not receive the
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
93
thinking skills instruction and served as control
subjects. Of the treatment group, 226 students were
English learners and 437 of the control group students
were English learners.
Research Design The design was quasi-experimental.
This design was appropriate for studying the effects of
the Higher Order Thinking Skills Program that already had
been implemented in the school setting. The researcher
used the pretest-posttest control-group procedure (Borg &
Gall, 1989) to assess the effects of the Higher Order
Thinking Skills Program on academic achievement. For the
investigation of this thinking skills treatment on the
application of learning skills, the posttest-only control-
group process was implemented.
Instructional Procedures The curriculum of the
Higher Order Thinking Skills (HOTS) Program was designed
to develop general thinking skills for underachieving
students in fourth through seventh grades. Instead of
instructing these students in remedial programs
emphasizing low level cognitive skills, the HOTS lessons
taught metacognition, inference from context,
decontextualization, and information synthesis. The goal
of the curriculum was to develop thinking skills that
would enhance academic outcomes, including gains in
standardized test scores and the transfer of skills to
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
94
classroom lessons. Due to the past success of this
thinking skills instruction, it was chosen twice as a
Promising Practices Program by the National Diffusion
Network in the U.S. Department of Education.
The lessons included problem-solving activities that
utilized Socratic questioning to mediate the learning of
thinking skills. Challenging Learning Dramas
incorporated computer software and teacher presentations
to create motivation and an environment of teamwork.
Students applied their thinking skills while solving
problems in the context of these Learning Dramas.
The instructional strategies of the Higher Order
Thinking Skills Program related to the academic needs
of underachieving English learning Latino students. The
curriculum was challenging because it did provide higher
order thinking skills. As stated in the literature
review, these skills are often neglected in the English
learning Latino student’s instruction. The strategies
of Socratic questioning, teacher dramas, and learning
dramas in the computer software, increased active
participation and motivation. The Socratic questioning
taught the processes of analysis and synthesis, as well
as assisted the students in developing metacognition.
Through the Socratic questioning, the teachers were also
continuously monitoring the students’ understanding of
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
95
the content and skills. Higher levels of thinking,
challenging skills and concepts, active participation,
and monitoring of understanding, enhance the performance
of English learning students.
Concerning the affective needs of underachieving
English learning Latino student, the Higher Order
Thinking Skills supported the students. Since the
students were in the process of learning English, they
may have been mentally translating from English to
Spanish as they listened and vice versa as they spoke.
The use of extended wait time allowed the students time
for these mental processes and thus encouraged increased
participation. Additionally, because the students were
still learning English, they might have been hesitant to
answer questions in large group instruction. The
organization of the Higher Order Thinking Skills Program
divided the class in half for the Socratic instruction
and into small groups for application at the computers.
The affective filter was lowered during the time at the
computers because the students worked in small groups,
usually pairs, and could proceed at their own pace. Also
contributing to the positive affective environment, was
the fact that students could converse in Spanish as they
solved problems using computer software. Although the
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
96
teachers' lessons were in English, some of the teachers
were Spanish bilingual, and thus they were able to
explain major concepts and vocabulary in Spanish.
Staff Development During the five days of staff
development for the teachers and instructional
assistants, they learned the Socratic techniques for
teaching thinking skills. The organization of the
lessons, as well as the function of questions, were
explained and practiced so the participants could
effectively facilitate students' learning. The teachers
were specifically trained to extend wait time, which
would allow students more time to ponder the questions
and use their prior knowledge.
Instrumentation Both nationally standardized
examinations and teacher questionnaires were used to
gather data. The Stanford Achievement Test Series
(Stanford 9), Ninth Edition, (1996) was used to assess
the students' gains in reading comprehens ion and
mathematics problem solving. Additionally, a
questionnaire consisting of eight items was used to
collect information from the classroom teachers to
evaluate the students' application of learning skills.
Data Collection and Analysis Data were collected
from the spring 1998 and spring 1999 Stanford 9 test
results. These results, listed in normal curve
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
97
equivalents, were compiled for students who were
participating in the Higher Order Thinking Skills Program
and for the control group of students. Also, data were
collected from the teacher questionnaires about their
students' application of learning skills. Analysis of
variance, t-tests, and pairwise comparisons tests were
conducted. The level of statistical significance was
set at p = < .05.
Findings
The statistical findings were reported for the seven
null hypotheses that were directly related to the seven
research questions.
The data for Null Hypotheses 1 and 2 were concerning
the effect of the Higher Order Thinking Skills Program on
Stanford 9 test score gains in reading comprehens ion and
mathematics problem solving respectively. For each
hypothesis, the scores were compared among the Latinos
who were English only, fluent English proficient, and
the English learning students. There were no
statistical differences among the three groups on
reading scores, but there was statistical significance for
the overall mathematics scores. The results of post hoc
t-tests showed that only the fluent English proficient
students' scores and the English learning students' scores
were significantly different. The fluent English
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
98
proficient students' scores on mathematics problem solving
were significantly higher than the English learning
students.
The data for Null Hypotheses 3 and 4 were concerning
the effect of the Higher Order Thinking Skills Program on
the Stanford 9 reading comprehension and mathematics
problem solving test score gains of the English learning
Latino students. Their scores were compared to the scores
of English learning Latino students who did not receive
the Higher Order Thinking Skills Program. The finding
were not significant for either of the hypotheses. The
Higher Order Thinking Skills Program did not seem to be
effective for increasing the reading or mathematics
performances as measured by the Stanford 9.
The factors for Null Hypotheses 5 and 6 concerned the
effect of the Higher Order Thinking Skills Program on
reading comprehens ion and mathematics problem solving test
score gains according to students as grouped by gender.
Again the findings of the ANOVA procedure obtained no
statistically significant differences. The thinking
skills instruction did not seem to produce increased
scores for students according to gender.
The data for Null Hypothesis 7 were analyzed to
investigate the effect of the Higher Order Thinking Skills
Program on the students' application of learning skills.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
99
The ANOVA procedure obtained overall statistically
significant findings. The post hoc pairwise comparisons
tests obtained statistically significant findings for each
of the eight learning skills. The Higher Order Thinking
Skills Program appears to produce a positive effect on the
students' ability to apply learning skills to classroom
lessons.
Conclusions
The following conclusions are based on the
statistical findings. These findings suggest that the
Higher Order Thinking Skills Program produced positive
growth in the students' ability to apply learning skills
to classroom activities and assignments. As reported by
the classroom teachers, the students appeared to transfer
the skills that they learned in the general thinking
skills lessons to classroom studies. The Higher Order
Thinking Skills Program appears to have improved student
performance in each of the following areas: making
predictions, selecting strategies to test accuracy of
predictions, explaining the process by which their answers
were obtained, working at solving problems by thinking
rather than guessing, applying prior knowledge to new
situations, improving their attitude toward learning by
displaying tenacity when encountering difficult tasks, as
well as participating in classroom discussions and
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
100
cooperative group activities. In summary, the Higher
Order Thinking Skills Program appears to be an effective
program for teaching these learning skills.
Although the theoretical framework for thinking
skills education and prior research on higher order
thinking skills had suggested improvement in student
academic performance, the Higher Order Thinking Skills
Program did not produce significant results in reading
comprehension or mathematics problem solving for English
learning Latino students in this study. The fluent
English proficient (FEP) Latino students did make
significantly higher gains scores than the English
learning Latino students in mathematics. To be identified
as FEP, however, the criteria included Stanford 9 test
score levels and achieving proficiency guidelines not met
by the English learning students. Finally, when comparing
students by gender, male and female students made similar
gains.
Overall, for English learning Latino students, the
Higher Order Thinking Skills Program seemed to improve
learning skills that transferred to application in their
classroom assignments, but it did not appear to be
effective instruction for improving reading and
mathematics performance as measured by standardized
achievement tests.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
101
Recommendations
Based on the design and findings of the present
research, the following are recommendations suggested for
future research.
1. Future research could replicate this study with
Latino students in other geographic and socioeconomic
areas. Such studies could be designed to include more
fluent English proficient and native English speaking
students who are Latino.
2. Research about the effects of higher order
thinking skills on Latino students could be conducted with
students in middle schools and high schools. Many Latino
students in secondary school will have received more years
of English instruction. This English instruction may
assist the students in benefiting from the thinking skills
curriculum.
3. Future research could use a longer time frame in
which to study the results of the Higher Order Thinking
Skills Program and other thinking skills programs. This
extended time frame should be used to assess the
application of learning skills instruction.
4. Research on higher order thinking skills could
continue to include a variety of evaluation instruments.
Instruments could range from authentic assessment, such as
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
102
portfolios and learning logs, to standardized tests that
measure specific higher order thinking skills.
5. Research of higher order thinking skills
instruction could study curriculum and lessons presented
in the native language of the English learning student.
6. For future research concerning English learning
students and higher order thinking skills, in-depth
designs could incorporate the assessment of teaching
strategies that assist language comprehension for the non-
native English speaking students.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
103
REFERENCES
Allison, A. K. (1993)• Critical thinking/problem-solving
skills for the at-risk student. Unpublished doctoral
dissertation, Northern Arizona University,
Flagstaff.
August, D., & Hakuta, K. (Eds.). (1997). Improving
schooling for lanauaqe-minoritv children: A
research agenda. National Research Council,
Washington, DC: National Academy Press.
Bareli, J., Liebman, R., & Segil, I. (1988). Fostering
thoughtful self-direction in students. Educational
Leadership, 45. 14-17.
Beyer, B. K. (1988). Developing a scope and sequence for
thinking skills instruction. Educational Leadership,
45, 26-30.
Bloom, B. S., Englehart, N. D., Furst, E. J., Hill, W.
H., & Krathwohl, D. R. (1956). Taxonomy of
educational objectives: Handbook I. Cognitive
domain. New York: McKay.
Borg, W. R., & Gall, M. D. (1989). Educational research:
An introduction. New York: Longman.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
104
Brown, A. L., & Campione, J. C. (1992). Students as
researchers and teachers. In J. w. Keefe & H. J.
Walberg (Eds.), Teaching for thinking (pp. 49-58).
Reston, VA: National Association of Secondary
School Principals.
California Department of Education. Educational
Demographic Unit. (2000). Statewide enrollment
reports. Sacramento, CA: Author.
California Department of Education Proposition 227 Task
Force. (1999). Educating English learners for the
twentv-first century. Sacramento, CA: California
Department of Education.
Carrasguillo, A. L. (1996). Preparing all classroom
teachers to educate a linguistically and culturally
diverse school population. Paper presented at the
annual meeting of the American Educational Research
Association, New York.
Carrasquillo, A. L., & Rodriquez, V. (1996). Language
minority students in the mainstream classroom.
Philadelphia: WBC Book Manufacturers.
Chance, P. (1986). Thinking in the classroom: A survey
of programs. New York: Teachers College, Columbia
University.
Cole, L. (1950). A history of education: Socrates to
Montessori. New York: Rinehart.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
105
Collins, A., & Brown, J. S. (1988). The computer as a
tool for learning through reflection. In H. Mandl &
A. Lesgold (Eds.), Learning issues for intelligent
tutoring systems (pp. 1-18). New York: Springer-
Verlag.
Costa, A. L. (Ed.). (1985). Developing minds: A resource
book for teaching thinking. Alexandria, VA:
Association for Supervision and Curriculum
Development.
Costa, A. L., & Liebmann, R. M. (Eds.). (1997).
Supporting the spirit of learning: When process is
content. Thousand Oaks, CA: Corwin Press.
Darmer, M. A. (1995). Developing transfer and
metacognition in educationally disadvantaged
students: Effects of the Higher Order Thinking
Skills. Unpublished doctoral dissertation,
University of Arizona, Tuscon.
DelGiudice, L. (1994). Academic achievement of Chapter 1
Spanish and English speaking students and
specialized types of instruction. Unpublished
doctoral dissertation, University of Southern
California, Los Angeles.
Dembo, M. H. (1994). Applying educational psychology. New
York: Longman.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
106
Detrick, L. W. (1988). The impact of thinking skills
teaching on standardized test scores. Unpublished
doctoral dissertation, Brigham Young University,
Provo, Utah.
Dewey, J. (1902). The child and the curriculum. Chicago:
University of Chicago Press.
Dewey, J. (1916). Democracy and education. New York;.
Macmillan.
Eisenman, J. Jr. (1995). An evaluation of the higher
order thinking skills program with fourth and fifth
grade students. Unpublished doctoral dissertation,
University of Georgia, Athens.
Ennis, R. H. (1985a). Critical thinking and the
curriculum. National Forum. 65 (1), 28-31.
Ennis, R. H. (1985b). Goals for critical thinking
curriculum. In A. L. Costa (Ed.), Developing minds;
A resource book for teaching thinking (pp. 54-57).
Alexandria, VA: Association for Supervision and
Curriculum Development.
Ennis, R. H. (1992). Assessing higher order thinking for
accountability. In J. W. Keefe & H. J. Walberg
(Eds.), Teaching for thinking (pp. 73-92). Reston,
VA: National Association of Secondary School
Principals.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
107
Ennis, R. H., & Paulus, D. (1965). Critical thinking
readiness in grades 1-12. Ithaca, NY: Cornell
University.
Feuerstein, R., Feuerstein, R., & Schur, Y. (1997).
Process as content in education for exception
children. In A. L. Costa & R. M. Liebmann (Eds.),
Supporting the spirit of learning: When process is
content (pp. 1-22). Thousand Oaks, CA: Corwin Press.
Fitzgerald, J. (1995). English-as-a-second-language
learners' cognitive reading processes: A review of
research in the United States. Review of
Educational Research, 65(2). 145-190, 184.
Flavell, J. H. (1976). Metacognitive aspects of problem
solving. In L. Resnick (Ed.), The nature of
intelligence (pp. 230-239). Hillsdale, NJ: Eribaum.
Garcia, E. E. (1987). Effective schooling for language
minority students. New Focus, Occasional papers in
bilingual education, No. 1. Wheaton, MD: National
Clearinghouse for Bilingual Education.
Garcia, G. E. (1991). Factors influencing the English
reading test performance of Spanish-speaking
Hispanic students. Reading Research Quarterly. 26
(4), 371-392.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
108
Garcia, G. E., & Pearson, P. D. (1991). Modifying reading
instruction to maximize its effectiveness for all
students. In M. S. Knapp & P. M. Shields (Eds.),
Better schooling for the children of poverty:
Alternatives to conventional wisdom (pp. 31-59).
Berkeley, CA: McCutchan.
Garcia, G. N. (1995). Improving access to high quality
teaching and learning for linguistically and
culturally diverse students. National Association
for Bilingual Education News, 18 (7), 33-34.
Gersten, R. (1996). Literacy instruction for language-
minority students: The transition years. The
Elementary School Journal, 96 (3), 228-244.
Glaser, E. M. (1941). An experiment in the development of
critical thinking. New York: Teachers College,
Columbia University.
Glaser, E. M. (1972). Reinforcing productive classroom
behaviors A teacher's guide to behavior
modification. New York: Behavioral Publications.
Glaser, E. M. (1985). Educating for responsible
citizenship in a democracy. National Forum. 65 (1),
24-27.
Glatthorn, A. A. (1987). Curriculum leadership. Glenview.
IL: Scott, Foresman.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
109
Gonzalez, V., Brusca-Vega, R., & Yawkey, T. (1997 ).
Assessment and instruction of culturally and
linguistically diverse students with or at-risk of
learning problems. Boston: Allyn and Bacon.
Green, S. B., Salkind, N. J., & Akey, T. M. (1997). Using
SPSS for windows: Analyzing and understanding data.
(2nd ed.). Upper Saddle River, NJ: Prentice Hall.
Guilford, J. P. (1959). Three faces of intellect.
American Psychologist, (14) ,469-479.
Guilford, J. P. (1988). Some changes in the structure-of-
intelligence model. Educational and Psychological
Measurement. (48 ), 1-4.
Harcourt Brace. (1996). Stanford achievement test series:
Form T. (9th ed.). San Antonio, TX: Author.
Harcourt Brace. (1997). Stanford achievement test series:
Technical data report. (9th ed.). San Antonio, TX:
Author.
Impara, J. C., & Plake, B. S. (1998). The thirteenth
mental measurements yearbook. Lincoln, NB:
University of Nebraska Press.
Isaac, S., & Michael, W. B. (1995). Handbook in research
and evaluation: A collection of principles.
methods. and strategies useful the planning, design,
and evaluation of studies in education and the
behavioral sciences. (3rd ed.). San Diego, CA:
EdITS Publishers.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
1 1 0
Jimenez, R., Garcia, G. E., & Pearson, P. (1996). The
reading strategies of bilingual Latina/o students
who are successful English readers: Opportunities
and obstacles. Reading Research Quarterly, 31 (1),
107.
Krashen, S. (1998). Bilingual education and the dropout
argument. Discovery (4H. [On line]. Available:
http://www.ncbe.gwu.edu/ncbepubs/discover/04dropout.
htm
Loughran, J. J. (1996). Developing reflective practice;
Learning about teaching and learning through
modeling. London: Falmer Press.
Macias, R. F. (1998). Summary report of the survey of the
states' limited English proficient students and
available educational programs and services, 1996-
97. Washington, DC: National Clearinghouse for
Bilingual Education.
Marzano, R. J. (1992). A rationale and framework for
teaching thinking tactics. In J. w. Keefe & H. J.
Walberg (Eds.), Teaching for thinking (pp. 15-25).
Reston, VA: National Association of Secondary
School Principals.
Marzano, R. J., Brandt, R. S., Hughes, C. S., Jones, B.
F., Presseisen, B. Z., Rankin, S.C., & Suhor, C.
(1988). Dimensions of thinking: A framework for
curriculum and instruction. Alexandria, VA:
Association for Supervision and Curriculum
Development.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
I l l
Naisbitt, J. (1982). Megatrends: Ten new directions
transforming our lives. New York: Warner Books.
Nanez, E. Sr., & Padilla, R. V. (1995). Bilingualism and
processing elementary cognitive tasks by Chicano
adolescents. The Bilingual Research Journal. 19 (2),
249-260.
National Research Council. (1997). Improving schooling
for language-minoritv children: A research agenda.
Washington, DC: National Academy Press.
Odden, A. R., & Odden, E. R. (1995). Educational
leadership for America's schools. New York: McGraw-
Hill.
Paul, R. W. (1985). The critical thinking movement: A
historical perspective. National Forum, 6 5 (1),
2-3.
Paul, R. W. (1995). Critical thinking: How to prepare
students for a rapidly changing world. Santa Rosa,
CA: Foundation for Critical Thinking.
Phillips, J. L. (1969). The origins of intellect:
Piaget's theory. San Francisco: w. H. Freeman.
Pogrow, S. (1990a). Challenging at-risk students:
Findings from the HOTS program. Phi Delta Kappan. 71
(5), 389-397.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
1 1 2
Pogrow, S. {1990b). HOTS higher order thinking skills:
Using computers to develop thinking skills in
students at risk. New York: Scholastic.
Pogrow, S. (1994). Helping students who "Just don't
understand." Strategies for success. Educational
Leadership. 52 (3), 62-67.
Pogrow, S. (1995a). Making reform work for the
educationally disadvantaged. Educational Leadership,
52 (5), 20-25.
Pogrow, S. (1995b). A revalidation of the effectiveness
of the HOTS program. A paper prepared for the
National Diffusion Network. U. S. Department of
Education, Washington, DC.
Pogrow, S. (1996a). HOTS: Helping low achievers in
grades 4-7. Principal. 76 (2), 34-35.
Pogrow, S. (1996b). Reforming the wannabe reformers: Why
education reforms almost always end up making things
worse. Phi Delta Kappan, 77 (10), 656-668.
Pogrow, S. (1997a). Accelerated learning for
disadvantaged students. Speech presented at
California School Board Association Conference.
Anaheim, CA.
Pogrow, S. (1997b). For low-achievers in grades 4 to 7:
Some like it HOTS. (Higher Order Thinking Skills
Program). Education Digest. 62 (5), 25-29.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
113
Pogrow, S. (1997c). Using technology to combine process
and content. In A. L. Costa & R. M. Liebmann (Eds.),
Supporting the spirit of learning; When process is
content (pp. 98-116). Thousand Oaks, CA: Corwin
Press.
President1s Advisory Commission on Educational Excellence
for Hispanic Americans. (1996). Our nation on the
fault line: Hispanic American education. Washington,
DC: U. S. Government Printing Office.
Presseisen, B. Z. (1986). Thinking skills: Research and
practice. Washington, DC: National Education
Association.
Presseisen, B. Z. (1992). Thinking skills in the
curriculum. In J. W. Keefe & H. J. Walberg (Eds.),
Teaching for thinking (pp. 1-13). Reston, VA:
National Association of Secondary School Principals.
Resnick, L. B. (1987). Education and learning to think.
Washington, DC: National Academy Press.
Resnick, L. B., & Klopfer, L. E. (1989). Toward the
thinking curriculum: An overview. In L. B. Resnick
& L. B. Klopfer (Eds.), Toward the thinking
curriculum: Current cognitive research (pp. 1-18).
Alexandria, VA: Association for Supervision and
Curriculum Development.
Rieber, R. W. (1997). The collected works of L. S.
Vygotsky. Vol. 4. The history of the development of
higher mental functions. New York: Plenum Press.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
114
Riesenmy, M. R. , Mitchell, S., Hudgins, B. B., & Ebel,
D. (1991). Retention and transfer of children's
self-directed critical thinking skills. Journal of
Educational Research, 85 (1), 14-25.
Rowe, M., (1974). Relation of wait time and rewards to
the development of language, logic, and fate
control: a. Part 1: Wait time. Journal of Research
in Science Teaching. 11 (2), 81-94. b. Part 2:
Rewards. 11 (4), 291-308.
Sarason, I. G., Glaser, E. M., & Fargo, G. A. (1972).
Reinforcing productive classroom behavior; A
teacher's guide to behavior modifications. New Yorks
Behavioral Publications.
Sosa, A. S. (1998). Latinos in the United States: A
tapestry of diversity. In Gonzalez, M. L., Huerta-
Macias, A., Tinajero, J. V.(Eds.), Educating Latino
students: A guide to successful practice (pp. 197-
212). Lancaster, PA: Technomic.
Sousa, D. A. (1998). Brain research can help principals
reform secondary schools. National Association of
Secondary School Principals Bulletin. May 1998, 21-
28.
Swartz, R. J., & Perkins, D. N. (1990). Teaching
thinking: Issues and approaches. Pacific Grove, CA:
Midwest.
Toffler, A., & Toffler, H. (1995). Creating a new
civilization. Atlanta, GA: Turner.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
115
U. S. Department of Education. National Center for
Education Statistics. Office of Educational Research
and Improvement. (1995). The educational progress of
Hispanic students. NCES 95-767. Washington, DC:
U.S. Government Printing Office.
U. S. Department of Education. National Center for
Education Statistics (1996a). Reading literacy in
the United States: Findings from the IRA reading
literacy study. Washington, DC: U. S. Government
Printing Office.
U. S. Department of Education. National Center for
Education Statistics. (1996b). Schools and staffing
in the United States: A statistical profile, 1993-
94. NCES 96-124. Washington, DC: U. S. Government
Printing Office.
U. S. Department of Education. National Center for
Education Statistics. (1997). The inclusion of
students with disabilities and limited English
proficient students in large-scale assessments: A
summary of recent progress» NCES 97-482.
Washington, DC: u. S. Government Printing Office.
U. S. Department of Education. National Center for
Education Statistics. (2000). The condition of
education 2000. NCES 2000-062, Washington, DC: U. S.
Government Printing Office.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
116
U. S. Department of Education. National Center for
Education Statistics. (2001). Education statistics
quarterly. NCES 2001-605, Washington, DC: U. S.
Government Printing Office.
U. S. Department of Education. Office of Educational
Research and Improvement. National Center of
Education Statistics. (2000). Increasing the
participation of special needs students in NAEP: A
report on 1996 NAEP research activities. NCES 2000-
473. Washington, DC: U. S. Government Printing
Office.
U. S. Federal Government. (1998). America's children:
Key national indicators of well-being. Health
Resources and Services Administration Information
Center. [On line]. Available:
http://www.childstats.gov/americaschildren
Veer, R. van der, & Valsiner, J. (1991). Understanding
Vygotsky: A quest for synthesis. Cambridge, MA:
Blackwell.
Veer, R. van der, & Valsiner, J. (Eds.). (1994). The
Vygotsky reader. Cambridge, MA: Blackwell.
Vygotsky, L. S. (1962). Thought and language (E. Hanfmann
& G. Vakar, Trans.). Cambridge, MA: Massachusetts
Institute of Technology Press. (Original work
published 1934)
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
117
Vygotsky, L. S. (1978). Mind in society; The development
of psychological processes. M. Cole, S. Scribner,
V. John-Steiner, & E. Souberman, (Eds.). Cambridge,
MA: Harvard University Press.
Vygotsky, L. S. (1997). Educational psychology. (R.
Silverman, Trans.). Boca Raton, FL: St. Lucie Press.
Wenger, E. (1987). Artificial intelligence and tutoring
systems. Los Altos, CA: Morgan Kaufmann.
Where in the world is Carmen Sandiego? [Computer
Software]. (1990). Novato, CA: BrOderbund.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
118
Appendix A
Higher Order Thinking Skills
Program Forms
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
119
Date: M ay 4,1998
To: Selected Principals
From: Susan L. Johns, Coordinator
R e: Higher Order Thinking Skills Participants
Please send me the names of the students in the Higher Order Thinking Skills
Program and the teachers who are teaching it. I have attached a form for the
information. I need the form returned to me by May 18,1998.
The Superintendent's Cabinet has authorized my gathering of this information for
my dissertation. I will be compiling data, but the name of the school district,
individual schools, teachers, and students will not be mentioned.
If you have any questions, please call me at 433-3473. Thank you for your assistance.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
1 2 0
HOTS List of Students' Names
Directions: Please return this list to Susan Johns at Pupil Support Services (PPS)
by May 18, 1998. List the student number, student name, grade
level, and schedule of the HO TS program.
(A= 1 hr/4 days a week; B-l/2 hr./4 days a week; G= lhr/5 days a week;
D=1 /2 hr /5 days a week; Other=please explain the schedule)
Classroom Teacher_______ ' Name of School______________
HOTS Instructor_______________________
Grade Student Last First Schedule of
Level Number Name Name Program
(Example) 5 32527 Garcia Diana B
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
1 2 2
HOTS Training W orkshop
IMPORTANT: Times, other than the beginning session for each day, are
approximate, depending on the needs of the training group and the decision
of the trainer. Evening ...homework assignments are an integral part of the
training.
Lunch and coffee break times are set each day by the trainer.
Monday .(8;3.ftdL«M )
Daily Topics:
Introductions
Rationale, S tructure, and Organization of the H O TS P rogram : P art I
Workshop Overview
Computer Time
Teaching Techniques
H O M E W O R K : Lesson Preparation; Reading Assignments
Tuesday (8:30-4:00)
Daily Topics:
Lesson Presentations
Teaching Techniques (conL)
Lesson Assignments
H O M E W O R K : Lesson Preparation; Reading Assignments
Wednesday (8:30-4:00)
Daily Topics:
Teaching Techniques: Improvisations
H OTS Model
Lesson Presentations
Overview: C oaching Techniques
Technical Demonstrations
HO M EW O R K 1 Lesson P rep aration; Reading Assignments
Thursday (fcilOA’O O )
Daily Topics:
Teaching Techniques: Improvisations
Lesson Presentations
Linkage W orkshop for Classroom Teachers
Review: Teaching Techniques
HOM EW ORK: Lesson P reparation; Reading Assignments
Friday (8:30-4:00)
Daily Topics:
Rationale, Structure, and Organization of the HOTS Program II
(This is an all-group session for principals, coordinators,
other administrators, teachers, and educational assistants.)
Lesson Presentations
Sum m ary of Training Session
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
123
HOTS Training Workshop
$4jfGc£web
The HOTS training workshop is a five day, thirty-five hoar coarse conducted by a
certified HOTS trainer. Teachers will demonstrate competency in HOTS Program
tftchniqnes through the teaching of three sample lessons, as assigned during the week, for
critique and feedback by tie trainer and other trainees.
By the end of the HOTS training, teachers being trained w il have developed:
1.0 a knowledge and understanding of
1.1 the goals of the HOTS program ;
1.2 the target group for whom HOTS is appropriate;
1.3 the the need of a program sudi as HOTS;
1.4 the reasons for the effectiveness of the program, including theories of
cognition;
1.5 the key higher order thinking skills emphasized in the HOTS program;
1.6 the difference between good and poor thinkers;
1.7 the use of the HOTS curriculum ;
1.8 general school improvement techniques
1.9 HOTS computer lab / classroom linkage concepts and activities;
1.10 die role of the principal in die implementations of the HOTS program;
1.11 evaloaiioa techniques for HOTS sites;
1.12 the Soeratic model used in HOTS classes;
1.13 the importance of using learning dramas in the classroom;
' 2.0 computer sMOs, including
2.1 computer command operations;
2.2 computer / printer interface;
2.3 software use;
3.0 HOTS classroom conversation tedm iqaes, including
3.1 use of the curriculum guides;
3.2 general H O TS teaching techniques;
3.3 conducting and maintaining Socralic questioning environments;
3.3a phrasing of questions;
3.3b showing wait tim e;
3.3c responding to students' responses;
3.3d multiple probing;
3.3e students' articulation of strategies;
3.3f rhythm of a Somatic dialogue;
33g improvisation techniques;
33h using drama techniques;
3.4 preparation for teaching a HOTS lesson;
3.5 coadhing techniques;
3.6 supplementing curricular questions;
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
124
Appendix B
Teacher Questionnaire
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
125
Teacher Questionnaire
Student, .Date. School.
ClassroomTeacfaer. HOTSTeacher.
Evaluation of Higher Order Thinking Skills effectiveness includes this
questionnaire to be completed by the classroom teacher on the students’ use of
learning skills in their classroom. For the items below, please mark the number
that best describes the extent that the student demonstrates each behavior.
1. Makes predictions.
N o t Yet Sometimes Usually
2. Selects strategies to test accuracy of
predictions.
3. Explains the process by which an
answer was obtained.
4. Works at solving problems by thinking
rather than guessing.
5. Sticks with a problem, even
though difficult.
6. Participates in classroom discussions.
7. Takes an active role in cooperative
group activities.
8. Applies prior knowledge to new
situations
3
3
4
4
4
5
5
5
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
126
Teacher Questionnaire
Student.._________________________________ Date School______________
QassroomTeacher______________________ ___
Evaluation of Higher Order Thinking Skills effectiveness includes this
questionnaire to be completed by the classroom teacher on the students' use of
learning skills in their classroom. For the items below, please mark the number
that best describes the extent that the student demonstrates each behavior.
Not Yet Sometimes U sually
1. Makes predictions.
2. Selects strategies to test accuracy of 1 2 3 4 5
predictions.
3. Explains the process by which an 1 2 3 4 5
answer was obtained.
4. Works at solving problems by thinking 1 2 3 4 5
rather than guessing.
5. Sticks with a problem, even 1 2 3 4 5
though difficult.
6. Participates in classroom discussions. 1 2 3 4 5
7. Takes an active role in cooperative
group activities.
8. Applies prior knowledge to new
situations
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Appendix C
Site Facilitator's Interview Questions
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
128
Facilitator Interview
Higher Order Thinking Skills Program
Directions: Please answer the following questions about the operation of the
Higher Order Thinking Skills Program at your school. The name of the school
district, school and personnel will remain anonymous. Thank you for your
assistance with this research. Susan L . Johns
1 Training
Please describe the training you participated in for the HOTS
Program. Quality? How long ago? Any follow up sessions?
2. Scheduling
Please describe your scheduling for the HOTS Program? Number of
days a week and length of sessions? Scheduling of computer time
and instruction in Socratic dialogue?
3 . Student Groups
Please describe your grouping of students? Size of groups?
Composition? Rotation of groups? Accommodation for new
students?
4. Facilities
Please describe the computer learning center in your school?
Number of computers? Dependability of computers and of software?
Appropriateness of software?
5 . Personnel
Besides the classroom teacher, how are other adults' time utilized in
the HOTS Program? Instructional assistants? Resource teachers?
6 . Parent Involvement
How are parents involved in the HOTS Program? Information for
parent? Are there any other ways that parents could support the
instruction?
7 . Support
Please describe the support for the HOTS program? Other staff?
Staff development time? Teacher collaboration? Other?
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
Appendix D
Additional Statistical Tables
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
130
Table A
Means and Standard Deviations for Reading Scores in NCE
Points for Latino Students by English Language Fluency
Groups
Lanouaae Fluency Mean Standard Deviation Number
English Learners 32.26 12.16 191
Fluent Proficient 52.20 12.98 54
English Only 46.03 11.96 7
Total 36.92 14.83 252
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
131
Table B
ANOVA for Effect of Language Fluency on Reading Scores
Presented in NCE Points for Latino Students
Source
Sum of
Soruares df
Mean
Souare F Sia.
Model
31327.10*
3 10442.34 108.33 .01
Intercept 3122.31 1 3122.31 32.39 .01
READ 98 13990.85 1 13990.85 145.14 .01
LANGUAGE 346.50 2 173.25 1.80 .17
Error 23906.53 248 96.40
Total 55233.54 251
a. R Squared = .57
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
132
Table C
Pairwise Comparison of Mathematics Scores of Latino
Students Grouped by Language Fluency in HOTS Program
(I) Language
code
(J) Language
code
Mean
Difference Std. Error Sig.
English Only
English
Learners
3.428 4.261 .422
Fluent English
-4.697 4.518 .300
English
Learners
English Only -3.428 4.261 .422
Fluent Enqlish -8.125 ■ 2.204 .000
Fluent English
English Only
4.697 4.518 .300
English
Learners
8.125 2.204 .000
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
133
Table D
Means and Standard Deviations for Reading Scores in NCE
Points for Encrlish Learning Latino Students Comparing HOTS
Treatment and Control Groups
Variable Mean Standard Deviation Number
HOTS Treatment 32.26 12.16 191
Control 31.28 11.06 368
Total 31.61 11.44 559
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
134
Table E
ANOVA for Effect of HOTS Treatment on Reading Scores in
NCE Points for English Learning Latino Students
Sum of Mean
Source Scruares df Sauare F Siq
Model
22200.79a
2 11100.39 121.11 .01
Intercept 13049.55 1 13049.55 142.38 .01
READ 98 22079.23 1 22079.23 240.90 .01
TREATMENT 103.96 1 103.96 1.13 .29
Error 50959.79 558 91.65
Total 73160.57
a. R Squared = .30
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
135
Table F
Means and Standard Deviations for Mathematics Scores in
NCE Points for Enalish Learnina Latino Students Comnarincr
HOTS Treatment and Control Groups
Variable Mean Standard Deviation Number
HOTS Treatment
Control
Total
35.64 13.40
34.48 13.90
34.98 13.73
199
367
566
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
136
Table G
ANOVA for Effect of Treatment on Mathematics Scores
Presented in NCE Points for English Learning Latino
Students
Source
Sum of
Scruares df
Mean
Scniare F Sia.
Model 38221.36a
2 19110.68 157.60 .01
Intercept 20995.04 1 20995.04 173.14 9
©
MATH 98 38049.74 1 38049.74 313.78 .01
TREATMENT 68.43 1 68.43 .56 .45
Error 68271.74 563 121.26
Total 106493.10 565
a. R Squared = .36
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
137
Table H
Means and Standard Deviations for Reading Scores in MCE
Points for English Learning Latino Students bv Gender in
HOTS Program
Gender___________ Mean Standard Deviation Number
Male 32.72 11.97 90
Female 31.97 12.33 101
Total 32.26 12.16 191
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
138
Table I
ANOVA for Effect of Gender on Reading Scores Presented in
NCE Points for English Learning Latino Students in HOTS
Program
Source
Sum of
Souares df
Mean
Souare F Sia.
Model 9271.64a
2 4635.82 46.26 .01
Intercept 5107.65 1 5107.65 50.97 .01
READ 98 9246.16 1 9246.16 92.27 .01
GENDER .06 1 . 06 .01 .98
Error 18838.97 188 100.21
-
Total 28110.61 190
a. R Squared = .33
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
1 3 9
Table J
Means and Standard Deviations for Mathematics Scores in
NCE Points for English Learning Latino Students by Gender
in HOTS Program
Gender Mean Standard Deviation Number
Male 36.24 14.39 94
Female 35.24 12.76 105
Total 35.64 13.40 199
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
1 4 0
Table K
ANOVA for Effect of Gender on Mathematics Scores Presented
in NCE Points for English Learning Latino Students in HOTS
Program
Source
Sum of
Sauares df
Mean
Sauare F Sia.
Model 13071.29a
2 6535.51 56.95 .01
Intercept 4008.48 1 4008.48 34.93 .01
MATH 98 13023.34 1 13023.34 113.49 .01
GENDER 280.27 1 280.27 2.44 .12
Error 22492.32 196 114.76
Total 35563.34 198
a. R Squared = .37
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

Factors influencing minority parents to place their children in private schools

PDF

HP2 schools achieving in spite of: A case study looks at programs, culture, and leadership

PDF

An analysis of Long Beach Unified School District's EXCEL model of providing gifted programs for urban students and their effect on student achievement

PDF

A longitudinal comparative study of the effects of charter schools on minority and low-SES students in California

PDF

An evaluation of the current level of Korean parent involvement at Third Street Elementary School

PDF

An analysis of teachers' perceptions of the efficacy of an instructional method for English language development using music and singing paired with movement

PDF

Effective practices of a Spanish -bilingual and -bicultural principal vs. a non -Spanish -bilingual and -bicultural principal and the effects of their language proficiency and cultural knowledge ...

PDF

Alternative dispute resolution: An effective strategy for reducing special education due process hearings in California

PDF

Effective school practices that facilitate improved student achievement

PDF

An analysis of the implementation of content standards in selected unified school districts of California

PDF

A comparative case study: Tutoring in reading in two settings

PDF

How has the requirement to implement content standards affected the instructional program in schools and classrooms?

PDF

A comparative analysis of academic achievement for CalWORKs students in a K--12 public school system

PDF

Connecting districts and schools to improve teaching and learning: A case study of district efforts in the Los Coyotes High School District

PDF

Academic and social adjustment of Korean "parachute kids" in Southern California

PDF

Community college students' perceptions of collaborative learning in developmental writing classes: Identifying the factors that promote positive active learning

PDF

American Sign Language as a high school language elective: Factors influencing its adoption

PDF

Cognitive and motivational factors that affect the achievement of elementary students on standardized tests

PDF

Improving student achievement: An urban success story

PDF

A study of student retentiveness, achievement, and program completion in a school -to -career program

Asset Metadata

Creator Johns, Susan Louise (author)

Core Title Academic achievement of English -learning Latino students in relation to higher order thinking skills instruction

School Rossier School of Education

Degree Doctor of Education

Degree Program Education

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag education, bilingual and multicultural,education, curriculum and instruction,OAI-PMH Harvest

Language English

Contributor Digitized by ProQuest (provenance)

Advisor Williams, Edgar H. (committee chair), Gothold, Stuart (committee member), Nelson, John L. (committee member)

Permanent Link (DOI) https://doi.org/10.25549/usctheses-c16-539929

Unique identifier UC11339399

Identifier 3094342.pdf (filename),usctheses-c16-539929 (legacy record id)

Legacy Identifier 3094342.pdf

Dmrecord 539929

Document Type Dissertation

Rights Johns, Susan Louise

Type texts

Source University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Access Conditions The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the au...

Repository Name University of Southern California Digital Library

Repository Location USC Digital Library, University of Southern California, University Park Campus, Los Angeles, California 90089, USA