University of Southern California Dissertations and Theses

Visualizing historic space through the integration of geographic information science in secondary school curriculums: a comparison of static versus dynamic methods

(USC Thesis Other)

Visualizing historic space through the integration of geographic information science in secondary school curriculums: a comparison of static versus dynamic methods

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content i

Visualizing Historic Space through the Integration of Geographic Information Science in
Secondary School Curriculums:

A Comparison of Static versus Dynamic Methods

by

Jason Edward Martos

A Thesis Presented to the
Faculty of the USC Graduate School
University of Southern California
In Partial Fulfillment of the
Requirements for the Degree
Master of Science
(Geographic Information Science and Technology)

August 2016

ii

Copyright ® 2016 by Jason E Martos

iii

To my wife, April, and daughters, Lorraine and Natasha; thank you for all of your love and
support (and patience).

iv

Table of Contents
List of Figures ............................................................................................................................... vii
List of Tables ............................................................................................................................... viii
Acknowledgements ......................................................................................................................... x
List of Abbreviations ..................................................................................................................... xi
Abstract ......................................................................................................................................... xii
Chapter 1 : Introduction .................................................................................................................. 1
1.1 Motivation ............................................................................................................................2
1.2 Study Design ........................................................................................................................4
1.3 Research Questions ..............................................................................................................6
1.4 Study Organization ..............................................................................................................6
Chapter 2 : Related Work ............................................................................................................... 8
2.1 The Present Landscape of GIS in Secondary School Curriculums ......................................8
2.2 Related Work with Visualization Methods ........................................................................11
2.3 Opportunities for GIS Integration in History Classrooms .................................................14
Chapter 3 : Research Design and Methods ................................................................................... 17
3.1 Research Design: ...............................................................................................................18
3.1.1. Securing and Protecting the Research Population ...................................................18
3.1.2. Lesson Plan Development........................................................................................21
3.1.3. Exam Tool Data .......................................................................................................24
3.2 Selection of Appropriate Lesson Material .........................................................................26
3.2.1. Sources of Historical Data .......................................................................................27
3.2.2. Sources of Historical Maps, Photographs, and Materials ........................................29
3.3 GIS Datasets .......................................................................................................................30
3.4 Data Aggregation ...............................................................................................................31
3.5 Visualization Methods: ......................................................................................................32
3.5.1. Static Visualization Tools ........................................................................................32
3.5.2. Dynamic Visualization Tool ....................................................................................39
3.5.3. Data Uniformity .......................................................................................................44
3.6 Methodology for Analyzing Findings: ...............................................................................44
v

3.6.1. Determining student performance in knowledge extraction ....................................44
3.6.2. Determining measurable increases in student long term memory ...........................45
3.6.3. Evaluating increases in critical thought ...................................................................46
Chapter 4 : Findings ...................................................................................................................... 49
4.1 Classroom Based Knowledge Extraction Performance: ....................................................50
4.1.1. Effectiveness ............................................................................................................50
4.1.2. Efficiency .................................................................................................................54
4.2 Transition to Long Term Memory: ....................................................................................54
4.2.1. Effectiveness ............................................................................................................54
4.2.2. Efficiency .................................................................................................................57
4.3 Assessing Critical Thought: ...............................................................................................59
4.3.1. Effectiveness ............................................................................................................59
4.3.2. Participation and Engagement .................................................................................73
4.4 Qualitative Research Observations ....................................................................................76
4.4.1. Student Engagement ................................................................................................76
4.4.2. Student Interaction with the Study Tool ..................................................................76
4.4.3. Financial Considerations ..........................................................................................77
Chapter 5 : Discussion .................................................................................................................. 79
5.1 Discussion of Findings .......................................................................................................79
5.1.1. Student Performance on Knowledge Extraction Tasks ...........................................79
5.1.2. Impact on Transition to Long Term Memory ..........................................................81
5.1.3. Implications for Increasing Critical Thought in Students ........................................82
5.1.4. General Observations ...............................................................................................86
5.2 Assessment of Study Design Strengths and Weaknesses ..................................................88
5.2.1. Strengths ..................................................................................................................88
5.2.2. Weaknesses ..............................................................................................................89
5.3 Implications for Future Research .......................................................................................91
5.4 Conclusions ........................................................................................................................92
References ..................................................................................................................................... 95
GIS in Education Sources: .......................................................................................................95
Visualization & Cartography Sources: ....................................................................................96
vi

BIBLIOGRAPHY ......................................................................................................................... 98
History Sources ........................................................................................................................99
Primary Sources .................................................................................................................99
Secondary Sources .............................................................................................................99
Appendix A: Institutional Review Board (IRB) Documentation ............................................... 100
Appendix B: IRB Certified Youth Assent-Parental Permission Form ....................................... 102
Appendix C: Hawaii Data Sharing Agreement (DSA) and Work Plan ...................................... 106
Appendix D: Static Research Tool ............................................................................................. 109
Appendix E: Dynamic Research Tool ........................................................................................ 125
Appendix F: Study Examination Tool ........................................................................................ 126
Appendix G: Study Exam Tool Grading Rubric......................................................................... 133

vii

List of Figures
Figure 1 Bloom’s Taxonomy of Cognitive Learning Skills ........................................................... 5
Figure 2 Graphic Depiction of the Timeline for the Classroom Experiment ............................... 17
Figure 3 Map of the Arizona and New Mexico Territories .......................................................... 33
Figure 4 Map of Western and Chiricahua Apache Lands in 1861 ................................................ 34
Figure 5 Map Depicting the Arivaipa Apache Group of the Western Apache Band ................... 35
Figure 6 Macro Level View of the Key Sites and Routes Involved in the Bascom Affair .......... 36
Figure 7 Line of Sight Analysis Approaching Apache Pass ......................................................... 37
Figure 8 Micro Level View of the United States Positions in Apache Pass ................................. 38
Figure 9 Story Map Tab #1 - Title Page ....................................................................................... 39
Figure 10 Story Map Tab #2 - Identifying Our Space .................................................................. 40
Figure 11 Story Map Tab #3 - The Indeh Circa 1861................................................................... 40
Figure 12 Story Map Tab #4 - The Raid on the Ward Ranch ....................................................... 41
Figure 13 Story Map Tab #5 - Cultural Considerations ............................................................... 41
Figure 14 Story Map Tab #6 - The Approach to Apache Pass ..................................................... 42
Figure 15 Story Map Tab #7 - The Conflict Begins ..................................................................... 42
Figure 16 Story Map Tab #8 - What Went Wrong ....................................................................... 43

viii

List of Tables
Table 1 Study Population Demographics ..................................................................................... 20
Table 2 Relationship Between Study Design and Research Questions ........................................ 22
Table 3 Description of Time Allotment Used During the First Study Iteration ........................... 23
Table 4 Description of Time Allotment Used During the Second Study Iteration ....................... 24
Table 5 Description of Exam Tool Sections and Data Collected ................................................. 25
Table 6 Relationship between Content and Performance Standards, and the lesson plan ............ 28
Table 7 Spatial Data Used to Support the Study........................................................................... 31
Table 8 Description of Student Participation Scores .................................................................. 47
Table 9 Scoring Criteria for Essay Questions on the Exam Tool ................................................. 48
Table 10 Relationship Between Research Questions and Their Corresponding Hypotheses ....... 48
Table 11 Student Scores by Question Format............................................................................... 50
Table 12 Student Scores Amongst Male Students by Question Format ....................................... 51
Table 13 Student Scores Amongst Female Students by Question Format ................................... 52
Table 14 Student Scores Amongst Students Age 14-15 by Question Format .............................. 53
Table 15 Student Scores Amongst Students Age 16 by Question Format ................................... 53
Table 16 Student Scores Amongst Students Age 17-18 by Question Format .............................. 53
Table 17 Student Efficiency Scores .............................................................................................. 54
Table 18 Difference in Scores Between Study Iterations ............................................................. 55
Table 19 Difference in Scores of Male Students Between Study Iterations ................................. 56
Table 20 Difference in Scores of Female Students Between Study Iterations ............................. 56
Table 21 Difference in Scores of Students Age 14-15 Between Study Iterations ........................ 56
Table 22 Difference in Scores of Students Age 16 Between Study Iterations ............................. 57
Table 23 Difference in Scores of Students Age 17-18 Between Study Iterations ........................ 57
Table 24 Student Completion Times on the Second Iteration of the Study ................................. 59
ix

Table 25 Percentage of Students That Attempted the Essay Questions on the Exam Tool.......... 59
Table 26 Student Performance on Exam Tool Essay Questions ................................................... 61
Table 27 Difference in Average Scores Between Study Iterations ............................................... 61
Table 28 Student Performance By Count ..................................................................................... 62
Table 29 Male Student Performance on Exam Tool Essay Questions ......................................... 63
Table 30 Difference in Average Scores for Male Student Between Study Iterations .................. 64
Table 31 Male Student Performance By Count ............................................................................ 64
Table 32 Female Student Performance on Exam Tool Essay Questions ...................................... 65
Table 33 Difference in Average Scores for Female Student Between Study Iterations ............... 66
Table 34 Female Student Performance By Count......................................................................... 67
Table 35 Performance of Students Age 14-15 on the Exam Tool Essay Questions ..................... 68
Table 36 Difference in Average Scores Between Study Iterations for Students Age 14-15 ........ 68
Table 37 Performance of Students Age 14-15 By Count ............................................................. 69
Table 38 Performance of Students Age 16 on the Exam Tool Essay Questions .......................... 70
Table 39 Difference in Average Scores Between Study Iterations for Students Age 16 ............. 71
Table 40 Performance of Students Age 16 By Count ................................................................... 71
Table 41 Performance of Students Age 17-18 on the Exam Tool Essay Questions ..................... 72
Table 42 Difference in Average Scores Between Study Iterations for Students Age 17-18 ........ 72
Table 43 Performance of Students Age 17-18 By Count ............................................................. 73
Table 44 Student Class Participation and Engagement with the Research Tool .......................... 74
Table 45 Student Motivation for Engagement with the Research Tool........................................ 75
Table 46 Impact of Student Participation and Engagement on Critical Thought ......................... 75

x

Acknowledgements
I am indebted to a great many people for their support, without which this research would have
never been possible. I would like to begin by thanking both of my thesis supervisors. Professor
Warshawsky, thank you for your guidance and constant refinement throughout the early steps of
development. Professor Ruddell, your feedback, patience, and constant support throughout the
study, and particularly through developing the study’s findings, kept me focused and provided
the guidance I required to develop a professional product. I would also like to thank the members
of my committee, Professors Hasan, Oda, and Swift, for their critical review and feedback
throughout the process. Your shared experiences clearly strengthened my experiment design and
findings. In addition to my advisors and committee members, I would be remiss if I did not
mention the following people by name for their role in making this study possible: Professor
Kemp, Professor Fleming, Miss Vanessa Osborne, Professor Rogers (United States Military
Academy), Miss RoseAnn Fleming, Miss Ke’ala Fokuda, Miss Avis Nanbu, and Miss Isla
Young. I would like to issue special thanks to Mister Fred Murphy and Miss Sandy Webb. Thank
you both for your enormous support, and the trust you bestowed upon me to allow this study to
take place at Mililani High School. Finally, thank you to all of the students who participated in
the study, and their parents for allowing them to take part.

xi

List of Abbreviations
DSA Data Sharing Agreement
GIS Geographic information system
HIDOE Hawaii Department of Education
IRB Institutional Review Board
PBL Problem Based Learning
SSI Spatial Sciences Institute
USC University of Southern California

xii

Abstract
Spatial scientists spent the better part of the last three decades pushing for further integration of
Geographic Information Science (GIS) technologies in K – 12 curriculums. Their efforts to date
are leading to moderate breakthroughs in geography and physical sciences, but social studies
continue to neglect its use almost entirely. Unfortunately, little empirical evidence exists that
suggests students realize quantifiable gains from its inclusion in the classroom. In fact, the
findings from most research comparing visualization methods indicate that static mapping
methods outperform dynamic methods when assessed by the user’s ability to extract information
from the product. This study adds to existing literature by expanding upon current research into
static versus dynamic visualization methods. In contrast to previous visualization studies that
focus heavily on animation for their dynamic representations, this study tested static methods
against story maps to determine whether they provide teachers an advantage in the classroom.
To develop its findings, the study employed standard classroom instruction methods and
examination materials to identify which visualization method most effectively communicated the
material to students in secondary school history classrooms. The study divided students into a
control group using standard classroom static visualization tools, and an experimental group
using dynamic story maps. Written exams conducted immediately following initial instruction,
and again two weeks later, provided the basis for evaluation. The study failed to demonstrate that
dynamic products provide students a distinct advantage over traditional static products in a
classroom environment. Its findings suggest that students can use both tools equally effectively,
supporting the findings from previous research. Of note, this study suggests that among female
students, dynamic products may yield decreased learning outcomes. This indicates the need for
further research to identify how gender affects visualization strategies.
1

Chapter 1 : Introduction

Secondary school social studies teachers rely on cartographic tools to relay complex spatial-
temporal concepts to their students. In Hawaii, state standards require students to develop spatial
skills to analyze and interpret data from maps relating to people, places, and environments so
that they can explain the interactions between geographic regions, and various societies
throughout history (Hawaii Standards 2005). However, as Mares and Moschek argue, to fully
appreciate the relationship between humans and geographic space, students must recognize and
contend with the, “imaginative quality of their own views.” (Mares and Moschek 2013) In other
words, teachers and students must recognize and account for the ways that their personal
prejudices influence the manner in which they interpret spatial-temporal data.
This study seeks to determine how students interpret data through cartographic
visualization tools to determine if employing computer-based geographic information science
(GIS) technology makes sense at the secondary level, in social studies classrooms. The findings
are based on empirical comparisons of standard, static classroom visualization tools, and
emerging dynamic story map applications hosted by Esri through ArcGIS online. In the context
of this study, a story map is defined as a web based application that enables the author to fuse a
live, web-based map, with narrative text, photographs, timelines, and other sources of digital
media to enhance the delivery of information to the user. The study asks students to respond to a
series of written questions following a fifteen minute instructional period on the Bascom Affair,
using the specific visualization tool being implemented: static teaching aids or Esri story maps.
Through this classroom performance experiment, the study intends to understand which tool sets
yield more effective results in the classroom learning environment based on the students’ ability
to extract and interpret data from the maps provided. This study hypothesized that story maps
2

would provide a more effective means of relaying complex ideas, and that the increased student
interaction with the materials would lead to improved long term memory retention and greater
critical thinking skills.

1.1 Motivation

Pushing to integrate GIS into secondary school classrooms is not a new concept.
However, despite professional development courses geared toward further GIS integration in
secondary school curriculums, a significant gap exists between teachers trained to use GIS and
those implementing the tools in their classrooms. The theory that teachers who are trained to use
new methods will employ them to the benefit of their students has not borne fruit. In fact,
according to Lisner, much of the professional development received by teachers to implement
GIS in their classrooms has been wasted effort. In her dissertation at Northern Illinois, Lisner
articulated three primary barriers to integration that prevent teachers from willingly transitioning
to unproven methods: Learning to use the software requires too much time outside of the
classroom; schools lack funding for the hardware and software requirements; and a lack of
support from the administrative level. (Lisner 2008)
The limited gains that have been realized tend to center around integrating GIS into
teaching geography and physical sciences. Over the last 15 years, historians began recognizing
the relationship between history and geography, based on the way that humans perceive events in
space and time. Unfortunately, despite acknowledging the relationship, secondary schools have
made almost no progress when it comes to integrating GIS into history curriculums. (Knowles
2014) This may be influenced, in part, by the belief held by mainstream historians that GIS
offers no credible gains to their field of study. (Lunen and Travis 2013)
3

Although certain specializations in history welcome GIS, the major organizations that
dictate the direction of the professional field (Royal Historical Society, American Historical
Association, and the Deutsche Historikertag) demonstrate no inclination toward its widespread
adoption. Similar to the challenges faced at the secondary education level, historians perceive
that GIS poses too great of an initial investment in time, training, and resources to justify its use.
Much like the decision to abandon the use of quantitative analysis following its peak in the
1960s, the decision to ignore GIS reflects a conscious decision. Ultimately, professional
historians do not believe that GIS will forward their knowledge by answering the essential
questions in their discipline. (Lunen and Travis 2013)
Ironically, while this mentality creates a significant obstacle to integrating GIS in
secondary curriculums, it also provides its greatest justification. Spatial perception skills are
realized in most humans between the ages of 12 and 15 years. (Mares and Moschek 2013, 61) It
stands then to reason that incorporating GIS into secondary curriculums will further develop
these skills and reduce the initial investment required as adults to integrate spatial reasoning into
historical study. Thanks to advances in modern technology, and the current generation's
familiarity with web-based GIS applications and commercial global positioning system (GPS)
software, modern secondary school students already possess the will and many of the basic skills
required to leverage GIS in the classroom. (Lisner 2008, 7)
Lunen and Travis argue that it is more important to demonstrate why historians should
embrace GIS by providing concrete examples of benefits that the field stands to gain. Likewise,
from a curriculum development point of view, research must demonstrate that GIS technology
can enhance current lesson plans (2013). Professional development and Spatial History theory
will only advance the integration of GIS into secondary school history curriculums so far. For
4

widespread implementation to occur, GIS use must result in a quantitative increase in student
performance to justify the personal risk and accompanying costs that it carries.
Lisner identified that the body of teachers who are currently employing GIS in their
classrooms, despite the barriers to implementation, do so because they understand that GIS
increases their students' critical thinking and decision-making skills (2008). Unfortunately,
current visualization literature fails to demonstrate that GIS will enhance student performance in
social studies. Before widespread GIS integration in secondary schools can become a reality,
additional visualization research is needed to demonstrate why dynamic visualization tools
benefit students. This study represents an initial step toward providing that body of research.

1.2 Study Design

The study’s primary objective is to measure student performance in executing map based
knowledge extraction tasks, and their ability to accurately interpret and analyze spatial-temporal
relationships using cartographic visualization tools. The experiment design is influenced in part
by the work of Ben Anderson. Anderson's 2015 thesis at the University of Southern California
tested static maps against animated maps to determine the user's ability to extract criminal
activity data. This study adopts Anderson’s metrics of effectiveness and efficiency to determine
the overall benefit of each visualization method by recording the accuracy of student responses
on a written exam, and overall time required for each student to complete testing.
In addition to Anderson’s research, Liu et al.’s Problem Based Learning (PBL)
experiment influenced the research tool for this study. To assess critical thought and higher level
learning objectives, Liu et al. used Bloom’s taxonomy of cognitive learning skills demonstrated
in Figure 1 on page 5. (2010) Liu et al. determined that recall reflected the students’ ability to
5

remember important facts, numbers, or events; understanding indicated the students’ ability to
explain figures, tables, and concepts clearly outlined or provided in the lesson material; analyze
reflected consideration of cause and effect; evaluate judged that the student provided critical or
expansive comments that go beyond recall; and create applied to genuine or creative ideas of
interpretations or solutions to the questions that the student could not identify explicitly in the
lesson material. (Lui et al. 2010) This study employed the taxonomy developed by Liu et al to
produce the rubric used to define levels of critical thought in the research population.
This study does not propose to conclude the overarching superiority of either method of
visualization. Its sole intent is to determine which method produces improved learning results in
the classroom. In so doing, this study expands upon the broader body of visualization literature.
Through the empirical comparison of student test scores, the study provides focus on the
advantages and disadvantages of each form of visualization in a previously unstudied context.
However, its findings should not be accepted as universal.

Figure 1 - Bloom’s Taxonomy of Cognitive Learning Skills as revised by Krathwohl (2002)
(Liu et al. 2010, 154)
6

1.3 Research Questions

To reinforce the scope of this research, this study focused primarily on secondary school
level students’ ability to extract data using standard static classroom visualization tools, or
dynamic story maps hosted by Esri through ArcGIS online. This study was conducted to
determine whether story maps offer teachers an advantage in the classroom over traditional static
products by answering the following research questions:

1. Are there measurable increases in student performance in knowledge extraction?
2. Does the use of story maps increase the likelihood transitioning ideas from short term to
long term memory?
3. Do story maps facilitate critical thought through active participation with data as it is
presented?

The initial hypotheses reflect that story maps should provide a more effective means of relaying
complex ideas than traditional static products due to the assumptions that:

1. Students would perform knowledge extraction tasks with greater effectiveness and
efficiency with story maps.
2. Active engagement with the materials would increase the depth of processing that
students afford to new concepts, resulting in increased transition of ideas from short term
to long term memory.
3. Active engagement with the materials would generate classroom participation, leading to
increased critical thought and higher level learning outcomes.

1.4 Study Organization

After introducing the study in this chapter, chapter 2 further defines the visualization
methods compared in the study; identifies the present landscape of GIS in secondary school
curriculums; reviews related work into visualization methods; and articulates opportunities for
GIS integration in secondary school, social studies classrooms. Chapter 3 covers the research
design and methods used for the study to include: The steps required to secure and protect the
7

research population; lesson plan development; and the focus of data collected on the exam tool.
It also covers a selection of appropriate lesson material for the study; GIS datasets utilized to
develop the study tools; levels of data aggregation; a description of the visualization tools; and
the methods employed to analyze the study’s findings. Chapter 4 identifies the study’s findings,
broken down by each of the study’s three research questions, and concludes with a discussion of
qualitative findings produced during the classroom experiment. Chapter 5 provides discussion on
the study’s findings; an assessment of the study’s relative strengths and weaknesses;
recommendations for future research; and conclusions.

8

Chapter 2 : Related Work
As introduced in chapter 1, this study restricts its scope to comparing static and dynamic
cartographic visualization techniques in a standard classroom environment. This chapter reviews
the existing visualization literature that influenced the study design, and creation of the research
tools employed. However, before going further, two critical terms must be defined for the reader:
Static and Dynamic.
The study adheres to Mares and Moschek’s definitions for static and dynamic mapping
techniques. Static products imply conventional classroom tools employed in secondary school
social studies instruction: printed maps, pictures, texts, and timelines. In line with Mares and
Moschek, static maps are defined by the user’s inability to manipulate the data portrayed. As
such, students are forced to visualize the data as the cartographer envisioned. In contrast,
dynamic maps enable the user to manipulate how the data are represented and choose how they
wish to visualize the information provided. (Mares and Moschek 2013) Based on this
understanding, this study departs from previous definitions used by Anderson (2015) and
Baldwin (2014) in their research. Planimetric maps provide the user with the ability to
manipulate how they view the data through layered analysis techniques. Therefore, while this
study accepts the use of small multiple map displays as static products, it rejects planimetric
tools. For the same reason, this study asserts that digital GIS technologies are inherently dynamic
in nature because the user has the ability to control the data, and rapidly transform the
information visualized.
2.1 The Present Landscape of GIS in Secondary School Curriculums
Most research focused on the challenges facing further GIS integration into secondary
school curriculums leverage survey instruments as their main methodology for gathering their
9

data. (Lisner 2008) Because of this, the studies have excellent continuity and support each
other’s findings well. However, with few exceptions, the vast majority have focused almost
exclusively on reasons inhibiting further adoption. The research community has a very strong
understanding of the roadblocks ahead as a result. Unfortunately, less is understood about
opportunities for change or the potential benefits of GIS-enhanced curriculums
In 2013, Kerski, Demirci, and Milson, published their findings on the global landscape of
GIS in secondary education in the Journal of Geography. Principal among their findings was the
fact that despite claims otherwise, the technology gap remained a challenge to implementation.
President Obama’s 2014 ConnectEd initiative and Esri’s gift of free ArcGIS online
organizational accounts to K-12 teachers have largely settled issues related to hardware and
software availability. Schools now receive federal funding for classroom computers and wireless
internet services, and Esri’s software and web portals alleviate the requirement for schools to
manage their own spatial data repositories on site. However, software complexity remains a
concern for further integration. Teachers still require training on GIS practices and software, and
ongoing mentorship to effectively integrate the technology into their curriculums. (Richardson
and Solem 2014) Both endeavors require them to commit significant effort and personal time.
Software complexity challenges students in the classroom as well. Although students
approach learning through new technologies with high levels of excitement (often greater than
educators), the complexity of GIS software can quickly squander their initial interest. (Artvinli
2010) Most commercial GIS applications are not designed with classroom curriculums in mind.
The non-spatial workflows prove tedious to users over time and can inhibit visualization,
suggesting the need for a simplified, gesture based, instructional tool. (Blaser, Sester, and
Egenhofer 2000)
10

Most commercial and university outreach programs have focused on developing capacity
from the bottom, up. (Demirci, Karaburun, and Ünlü 2013, and Kerski, Demirci, and Milson
2013) There are many opportunities geared at professional development for teachers that assist
them with how to incorporate GIS into their lessons. However, educating teachers, and getting
them to translate their training into classroom practices are separate challenges. Remember,
based on her findings, Lisner called the GIS-based professional development that teachers
receive a wasted effort. (Lisner 2008) Unfortunately, as schools moved toward national common
core standards, opportunities for innovation in the classroom decreased and teachers became less
likely to search for solutions to incorporate new technologies. Teachers are trained to teach
students using proven methodologies to enhance student performance. (Chalmers 2010) GIS
remains unproven.
Demirci, Karaburun, and Ünlü argue that teacher-centered problems remain the most
important hurdle to bypass. (2013) Teachers ultimately decide what they will use in their
classrooms, and must be motivated appropriately to incorporate the new technology. Lisner
demonstrated that teachers moved forward with GIS on the grounds that it improves their ability
to teach the subject, or it improves their students’ abilities to think critically about the material.
Yet most teachers approach GIS from the perspective that it will be difficult to learn, time-
consuming to incorporate into their lesson plans, and could prove damaging to their careers
(should their students fail to demonstrate success) due to their decision to risk teaching with
methods that are not supported by their administrations. (2008) Similar findings regarding the
impact of insufficient administrative support in other research have led many to advocate
transitioning to a top-down approach. (Demirci, Karaburun, and Ünlü 2013, and Kerski, Demirci,
and Milson 2013) In particular, Kerski, Demirci, and Milson noted that GIS use spread most
11

rapidly in countries where its use was mandated as part of the national curriculum. Regardless of
the model selected, bottom-up or top-down, research must convince educators and administrators
that a transition benefits their field.
Very little research exists that links GIS to increases in student performance on
standardized tests or in classroom environments (Kerski, Demirci, and Milson 2013). In fact, in
2003, Kerski noted that GIS did not produce measurable increases in standardized performance.
More recently, in 2010, Liu et al. published their study on problem-based learning (PBL) using
GIS. They found that while GIS methods increased higher level thought in the experimental
group, the control group performed better at recall and standard memorization tasks.
Unfortunately, most standardized tests for the social sciences continue to require students to
memorize important dates and events as a large component of their evaluation. Once again,
research failed to demonstrate why teachers should transition to GIS in their classrooms.
In 2013, this evidence gap led Kerski, Demirci, and Milson to recommend establishing a
research base to demonstrate why GIS makes a difference in secondary education. They
recognized that to move forward with implementation, GIS had to prove itself relevant and
useful in the classroom setting. Research and training cannot stop at how to supplement lessons
with GIS. As Kerski, Demirci, and Milson articulate in their findings, it must demonstrate why
GIS is an improvement to traditional methods. (2013)
2.2 Related Work with Visualization Methods
Baldwin articulated the primary question at the center of most visualization research in a
manner that bears repeating. How do you communicate temporal and spatial change while
simultaneously, effectively communicating the story that the data has to tell? (2014) Complexity
increases as multiple sources of data are fused to form a common picture. It is not sufficient to
12

record the data and hope that the user will interpret its meaning appropriately. Visualization
methods should be chosen for their ability to communicate to the user in the most effective and
efficient manner. Context matters in this endeavor, requiring consideration of the intended
audience. Unfortunately, the current body of literature supports the use of static maps in most
settings.
Anderson designed his study to conduct an empirical consideration of static and dynamic
map representations depicting homicide patterns in Chicago. He selected small multiple map
displays for his static visualization method and time series, animated maps for the dynamic
representation in his research to take advantage of both tools’ ability to visualize the
chronological change. He then designed his research tool to assess user effectiveness, efficiency,
and preference through completing a series of choropleth map-based knowledge extraction tasks.
Anderson based effectiveness on the user’s ability to correctly answer questions and assessed
efficiency based on the amount of time the participant required. His findings indicated that users
interpreted the data more accurately and required less time to complete each task when using the
static products. Interestingly, they also tended to prefer the use of the static maps regardless of
whether or not they achieved better results while using them. (Anderson 2015)
Anderson’s research poses a direct challenge to pursuing GIS integration in secondary
school social studies curriculums. Based on his research, one could fairly conclude that students
are better served by continuing to use the static classroom products already in use. In addition to
the performance variables, Anderson’s user preference findings are particularly challenging.
Animated visualizations are commonly thought to be more visually appealing and desirable.
(Anderson 2015) However, Anderson’s research confirmed findings from previous studies that
argue that animated products may, in fact, be too complex and distracting for practical use.
13

Tversky, Morrison, and Betrancourt argue that in almost all cases where animated
products have out-performed their static counterparts, factors other than the dynamic animation
explained the variance. (2002) They concede that by the congruence principle, which states that
the content and format of the visualization method should match the concept conveyed, one
would expect animated products to excel at demonstrating change over time. However, in
practice, most studies demonstrate that static tools yield equivalent, if not higher learning results.
Tversky, Morrison, and Betrancourt go on to argue that in those cases where animated products
outperformed their static variants, the animated products provided additional detail or created
opportunities for interactive engagement.
The apprehension principle, which implies that visualization methods must be accurately
perceived and appropriately conceived, explains why animated products continuously fall short
despite their assumed advantage and visual appeal. Tversky, Morrison, and Betrancourt identify
that the disconnect between expectation and performance likely reflects the user’s perceptual and
cognitive limitations to rapidly process the animated product. This justification also explains
why incorporating interactivity into the design improves performance. Interactivity is a proven
instructional method that has a demonstrated record of improving learning. (Tversky, Morrison,
and Betrancourt 2002)
Lowe writes about two possible causes that prevent users from properly processing
information through animations. The first reason he articulates directly correlates to Tversky,
Morrison, and Betrancourt’s findings, the user is overwhelmed by excessive information
processing demands. The second reason lends further credence to why adding interactive
features to the dynamic visualization model improves learning. At the same time that users are
14

being overwhelmed by excessive information, they are being underwhelmed due to the passive
nature in which they engage the material. (Lowe 2003)
In reading these findings, one would be hard-pressed to conclude that dynamic products
could provide benefit to teachers and students in the classroom. However, context matters.
Definitions matter. This chapter began by defining dynamic visualization methods as those that
enable the user to manipulate the data and choose how they wish to visualize the information
presented. By this definition, an argument could be made that animated maps, as they have been
tested in the past, are in fact static. Once published, the user is forced to visualize the data as the
creator intended. That is the very definition of static. The fact that the image moves does not
necessarily mean that the product is any more malleable or interactive than a paper map. Much
as the planimetric map can be argued as dynamic due to its ability to facilitate layered analysis,
the animated map can be classified as static for its inability.
Because context matters, visualization research geared toward GIS integration in
secondary school curriculums must consider the intended audience. Children interpret data
differently than adults. Therefore, standard map practices appropriate for adults, may not
translate in a secondary classroom environment. While prior visualization research should not be
discarded completely, it should be considered in light of its context and setting. (Slocum et al.
2001) Unlike previously discussed examples, this study employed a classroom-based research
model. This decision created the opportunity to identify findings amongst the intended
population that further GIS integration in the classroom would impact most.
2.3 Opportunities for GIS Integration in History Classrooms
Mares and Moschek identify two aims that teachers should consider when teaching space
in history. The student should understand that space has evolved over time, and they should be
15

able to recognize and reflect on the way that their own views shape the way that they consider
historical space. In order to fully realize the temporal deviations of space, students must be able
to capture and critically analyze the impact of human activity on space and the way in which the
natural environment shaped human actions. (2013)
Mares and Moschek argue that GIS' dynamic nature creates unique teaching opportunities
for educators to help students become aware of their preconceived images of historical space that
the traditional use of static printed maps, pictures, and texts cannot duplicate. (2013) Story maps
are particularly well suited for these objectives. Put simply; students learn better through
increased interaction with data, and particularly through direct, gesture-based manipulations.
(Blaser, Sester, and Egenhofer 2000) The primary reason for this stems from the way that
humans record and recall information.
Unlike short-term memory, long-term memory has an infinite capacity to store
information. The challenge for the educator and the student is to get information to make the
transition to long-term memory. As a general rule, the transition depends on two factors; how
well the new material relates to previously learned ideas and the level and depth of processing
applied to the data as the student learns the lesson. Once transitioned, the student’s ability to
retrieve information from long-term memory depends on the number and strength of connections
formed between the new information and other concepts. (Heuer 1999) The interactive nature of
Story maps and their gesture driven manipulation increases the depth of processing that students
employ as they learn new lessons.
Mares and Moschek champion the use of GIS in the classroom to encourage student
participation and critical thought. However, they caution against giving students too much
leeway too quickly. (Mares and Moschek 2013, 67). In addition to the risk of frustration from
16

poorly understood tools and processes, students run the risk of getting "lost" in a virtually
limitless pool of data if proper measures are not implemented to constrain their environment.
This presents an opportunity for Story maps. As opposed to an open GIS where students have
complete control over the selection and representation of data, Story maps allow teachers the
opportunity to constrain the data available. Although they rely upon live, published web maps,
Story maps only relay the data that the author authorizes for dissemination. This increased level
of control reduces the risk of distraction and enables educators to design Story maps around their
lessons. Because the Story map already contains all of the layers of data the student requires to
complete the lesson, Students no longer need to understand how to manipulate GIS software to
reap the benefits of dynamic mapping. As a result, the medium should mitigate student
frustrations that stem from incoherent workflows.

17

Chapter 3 : Research Design and Methods
In order to test the study’s research questions, and assess the applicability of static and dynamic
maps amongst the intended audience, the study developed and executed a classroom research
experiment. The classroom experiment took place over a period of two weeks, with two distinct
iterations of the study conducted with each group. The first iteration of the study exposed
students directly to the research tool. The control group worked with static tools modeled after
what classroom teachers currently employ in their lesson plans, and the experimental group
explored the dynamic tool. The second iteration of the study occurred two weeks after the first
iteration and asked the students to recall the information that they learned from the tool to
complete the required tasks.
Between iterations, students retained access to the spatial tools and lesson content for
self-study in order to provide further opportunity for exposure. The control group (Static) took
the packet home with them, and additional copies of the study tool remained in the classroom in
case students lost access to their original materials. The experimental group (Dynamic) retained
access to the Story Map online, and could review the product using school computers, personal
computers, tablets, or smart phones as they desired. To ensure students remained focused
between iterations, the classroom teacher provided reminders periodically throughout the two
weeks. Figure 2, below, presents a graphic depiction of the study’s timeline for execution.

Figure 2 – Graphic depiction of the timeline for the classroom experiment
18

3.1 Research Design:
3.1.1. Securing and Protecting the Research Population

To gain access to the necessary student populations, the study required permission from
local high schools on Oahu to enter their classrooms and conduct the study. However, prior to
reaching out to local administrators, the study had to apply for an exemption for human subjects
testing through the institutional review board (IRB). The visualization study qualified for
exemption based on two factors. First, although the study meets the definition of research as
defined in 45 CFR 46.102, it does not meet the definition of human research. The study does not
seek to obtain information about the students themselves, nor does it require the collection of
personally identifiable information. Instead, the study employs risk mitigation strategies such as
using independently assigned identification numbers in place of student names. Second,
according to subpart D of 45 CFR 46.102 and 46.101 para (b) 1, the following conditions merit
exemption from human subject research when children are involved:
1. Research conducted in established or commonly accepted educational settings, involving
normal educational practices, such as (i) research on regular and special education
instructional strategies, or (ii) research on the effectiveness of or the comparison among
instructional techniques, curricula, or classroom management methods."

2. Research conducted using educational tests

The study’s methodologies fit both conditions. Although the study met both factors, the IRB
based their exemption decision primarily on subpart D, criteria 1 and 2.
In addition to securing an exemption for the study through the IRB, Hawaii State
Department of Education (HIDOE) needed to approve the data sharing agreement between the
researcher and the school, to grant the study access to student information and the classroom
environment. Due to the timelines required for obtaining a data sharing agreement, the study
19

narrowed its scope to a single high school. Mililani High School agreed to support the study and
provided access to a sufficient sample size of students.
A public school located in an established suburban neighborhood on the west side of
Oahu, Mililani is the only high school on the island to achieve a perfect 10 rating from
greatschool.org (an online resource that ranks schools based on student standardized test
performance). Mililani serves a diverse student population, from multiple backgrounds, and
provides access to learning for grades 9 - 12. Due to its proximity to the nearby United States
military garrisons at Schofield Barracks, Hunter Army Airfield, Joint Base Pearl Harbor-Hickam
Airfield, and Fort Shafter, Mililani hosts a large number of military families. From the study’s
perspective, military students add a unique variable to the classroom-based research thanks to
their varied educational experiences from multiple regions around the world.
The study presented the high school administrator with a presentation describing the
purpose of the study, along with its intended research design and methodologies. In line with the
findings from previous studies, the principal conditionally approved the study, but left the
ultimate decision to support the research in the hands of the classroom teacher. The study
provided the same materials to the classroom teacher, as well as conducted a phone interview to
further clarify the research goals and intended experiment design. Once approved, the study
provided two forms of written notification to student parents to provide them with an opportunity
to prevent their student from participating in the research. Digital copies of the IRB approved
consent/assent form went to the school, and paper copies went home with the students for their
parents to review and sign. The study then asked the students to return the forms directly to their
teacher to protect the student’s anonymity during the study.
20

The study provided the teacher with a spreadsheet of pre-defined participant
identification numbers. Upon receiving the student’s signed permission form, the teacher
recorded the student’s name adjacent to the next available number. From that point forward,
students identified themselves by their participant identification number. Students who did not
return a signed consent form did not receive participant identification numbers, and did not
participate in the study.
Initially, 116 students returned signed consent forms. However, the study required each
student to be present on both days of the study for their data to be included in the results. As a
result, the final study population totaled 101 students; 51 students assigned to the control group
(Static), and 50 students assigned to the experimental group (Dynamic). Table 1 on page 20,
describes the population demographics for the two groups. Due to privacy considerations for the
students participating, the study did not collect extensive demographic data relating to race,
nationality, income level, previous exam scores, or school performance. The study did capture
gender and age at the school’s request.
Table 1- Study Population Demographics
Control Group (Static) Experimental Group
(Dynamic)
Population Size 51 50
Gender:
Male 20 24
Female 31 26
Age (Range 14 – 18):
Mean Age 16 15
Mode Age 15 15

The study assigned students to the control group or experimental group by class. Both,
the school administrator and the classroom teacher, preferred that the study leverage the entire
21

student population assigned to the classroom teacher in order to mitigate the impact of the study
on teacher’s daily lesson plans. This worked in the study’s favor, as it made for a simple,
random, division of students. Both research groups included three full class periods of students.
The study assigned periods 1, 6, and 7 to the control group, and periods 2, 3, and 4 to the
experimental group. Using periods 2 through 4 for the experimental group facilitated the study’s
ability to secure computers. Since they were the first three periods of the day (Period 1 did not
hold class during the first day of the study), the study only required access to computers for half
of day. The study then switched to the static tool for the remaining class periods.
3.1.2. Lesson Plan Development
The study’s adherence to using a classroom environment defined the time available for
each portion of the experiment. Two iterations of the study were conducted, each within a
standard 55 minute class period. The first iteration of the study focused on measuring the
student’s ability to extract information using the given set of tools; standard static maps for the
control group, and a dynamic, Story Map for the experimental group (Dynamic). The second
iteration of the study measured the student’s ability to retain information, and the transition of
ideas from short term to long term memory. Both iterations afforded the opportunity to measure
critical thought and observe how the spatial tool influenced the way the students interacted with
and interpreted the data provided. Table 2, on page 22, illustrates the relationships between the
iteration of the study and the study’s research questions. Both iterations occurred over a two day
period in order to capture each class period; periods 2 through 4, 6, and 7, participated on the
first day of each iteration and period 1 participated on the second day.

22

Table 2 - Relationship Between Study Design and Research Questions
First Iteration Second Iteration
1. Are there measurable increases in student
performance in knowledge extraction?
Observed Not Observed
2. Does the use of story maps increase the
likelihood transitioning ideas from short
term to long term memory?
Not Observed Observed
3. Do story maps facilitate critical thought
through active participation with data as it is
presented?
Observed Observed

In order to make full use of the allotted time, the study pre-positioned all research tools at
the student’s desk prior to the beginning of the period. The control group received a printed copy
of the study maps and lesson materials, bound in report folders, while the experimental group
used the classroom’s computers to access the Story Map. The classroom teacher loaded the link
to the Story Map in the class share drive, and a written copy of the web address was projected on
the board for students to transcribe manually as required. Both groups received the same exam
material, pre-positioned next to their designated study tool, face-down.
The first iteration provided the students in both groups five minutes for familiarization
with the study. During this time, the students were reminded that participation in the research
was voluntary and afforded an opportunity to work on an alternative activity as provided by their
teacher. Students that refrained from returning their signed consent forms were separated from
the study population at this time as well by the classroom teacher. The remaining students
continued with the study.
Both research groups received the same lesson plan for the study in order to isolate the
tool. The lesson began with ten minutes of instruction provided by the researcher designed to
broadly introduce the topic and explain the rationale for selecting the subject material to the
students. During this period, the researched confirmed that no participants entered the study with
23

pre-existing knowledge of the lesson material. Following instruction, the students received
instruction to flip over their exam material and begin working on it, using their designated tool to
identify the answers to the questions. While working on their exams, the classroom teacher took
a copy of the study’s participant list around for the students to reference to record their
participant identification number on the exam packet. At the end of the class period, students
returned their exam to their teacher. Table 3 (below) describes the breakdown of time allotment
during the first iteration of the study.
Table 3 – Description of Time Allotment Used During the First Study Iteration
Time Activity
10 Minutes Before Class Pre-Stage Study Tools at Each Desk
Class Begins
5 Minutes - Introduce Study
- Separate Study Population from Non-
Participants
10 Minutes - Introduction to the Bascom Affair
- Explain the Rationale for Choosing the Topic
40 Minutes - Students Work on Exam Packet Using Their
Designated Tool
- Classroom Teacher Issues Participant IDs
- Classroom Teacher Observes and Records
Participation
Class Ends
5 Minutes Classroom Teacher Collects Exams and
Confirms Participant ID For Each Student

In addition to collecting data from the exam itself, the first iteration of the study also
observed student engagement with the material. While the students worked on their exam
packets, the classroom teacher observed each student to assess their level of engagement with the
study tool. The teacher recorded the initial assessment within the first ten minutes, and then re-
assessed each student at the midway mark, and ten minutes prior to the end of the class period.
24

The study only recorded successful engagement for students that remained on task throughout
the entire class period.
Similar to the first iteration of the study, the second iteration pre-staged the exam tool at
each student’s desk prior to the beginning of the class period. Once again, the first five minutes
of the period were used to segregate the research population from the students not participating
in the study and re-introduce the study to the participants. However, unlike the first iteration, the
participants received no further instruction following the initial re-introduction. The students
used the remaining time in the period to complete the exam; this time without the use of the
study tool. Students completed the second iteration of the study with the information that they
remembered from interacting with the tool. Table 4, below, provides the distribution of time used
for the second iteration of the study.
Table 4 – Description of Time Allotment Used During the Second Study Iteration
Time Activity
10 Minutes Before Class Pre-Stage Study Tools at Each Desk
Class Begins
5 Minutes - Introduce Study
- Separate Study Population from Non-
Participants
50 Minutes - Students Work on Exam Packet Without
Their Designated Tool
- Classroom Teacher Issues Participant IDs
Class Ends
5 Minutes Classroom Teacher Collects Exams and
Confirms Participant ID For Each Student

3.1.3. Exam Tool Data
The exam tool provided the study with the data required to assess student performance
and answer the study’s research questions. To avoid introducing a new variable to the study, both
groups used the exact same exam tool, and method of filling it out. Divided into five sections, the
25

exam consisted of a total of 24 questions. For the second iteration, the study introduced a sixth
section requesting information about how the student interacted with the cartographic tool during
the two week self-study period.
Each of the five sections employed a unique question format. Section one included ten
fill in the blank questions; section two provided two true or false statements; section three asked
the student to respond to two short answer questions; section four included six questions focused
on spatial analysis skills that required the student to directly interpret data from the study’s maps;
and section five gave the students an opportunity to respond to four essay questions to elaborate
on how they interpreted the lesson material. Most sections included a question that could be
approached and answered in multiple ways to assess critical thought. To confirm the student’s
thought process, the essay questions provided the student an opportunity to elaborate on similar
ideas queried in those questions. For a complete copy of the exam tool used in the study, refer to
Appendix E. Table 5 below further illustrates the data collected from the exam tool, and
identifies the questions that correlated in the study.
Table 5 – Description of Exam Tool Sections and Data Collected
Section Question Format Quantity Points Correlation
Section One Fill in the Blank 10 1 Each #10 to Essay 2&3
Section Two True or False 2 1 Each #11 to Essay 1
Section Three Short Answer 2 1 Each N/A
Section Four Spatial Analysis 6 1 Each #15-16 to Essay 4
Section Five Essay 4 5 Each N/A
Section Six
(Second Iteration
Only)
Study Assessment 6 0 Each N/A

26

3.2 Selection of Appropriate Lesson Material

Hawaii General Learning Objectives establish the opportunity for increased use of GIS in
their classrooms by focusing on developing student skillsets to use a variety of technology
effectively and ethically, and creating opportunities to improve complex critical thinking and
problem-solving skills (Hawaii Standards 2005, 2). The study design intentionally ties into
existing Hawaii state standards for secondary school social studies curriculums to test the
students’ ability to interpret data from the maps with ties to stated learning objectives appropriate
to their grade level. However, to assess the performance of each visualization method
appropriately, the study required that students enter the classroom without sufficient background
information on the topic of instruction. By the time students reach high school, they are familiar
with many standard narrative topics. Therefore, the study intentionally selected an obscure event
from United States history to mitigate the risk of pre-existing knowledge providing students with
an advantage on the examination materials.
In February 1861, the Bascom Affair triggered 25 years of war between the United States
government, and the Apache Indians. (Ball 1980, 25; Sweeny 2014, 13; and Mort 2013) Few
adults understand the events that occurred during United States’ westward expansion in the
nineteenth century, particularly those that occurred in 1861 in line with the outset of the
American Civil War. The associated conflicts with North America’s native populations are
broadly interpreted as the Indian Wars and receive very little attention in standard curriculums.
Fortunately, lack of attention does not equate to lack of value. The Bascom Affair provides the
opportunity for the study to tie into existing state standards without risking the integrity of its
findings.
27

Hawaiian students cover westward expansion and “Manifest Destiny” as part of their
standard 8
th
grade curriculum. Therefore, high school students in the state are familiar with the
period. Their prior familiarization enabled the study to meet several state standards by
leveraging the Bascom Affair to provide new information to an already existing knowledge base.
The study ties into Hawaii state standards 1, 2, 6, and 7, described in Table 6 on page 26, to
determine how static and dynamic visualization methods support student learning.
3.2.1. Sources of Historical Data
Because the Bascom Affair is not part of standard high school curriculums, the study had to
develop supporting lesson materials. The study relied upon a mix of primary and secondary
sources to develop the facts surrounding the event. The bibliography identifies the full list of
sources used in the study, but three works deserve focus here. First, Edwin Sweeney’s Cochise:
First Hand Accounts of the Chiricahua Apache Chief, provided access to valuable primary
sources on the Apache War, seamlessly stitched together to follow the timeline of events.
Sweeney’s notes cross-reference multiple primary sources to reveal the original author’s bias,
and add depth to traditional narratives through the inclusion of official military reports, eye-
witness accounts, newspaper articles, and interviews with Cochise himself. These interviews are
the closest thing to an autobiography on the great Apache leader and are crucial to understanding
the Apache road to war, and motivations for fighting the Americans.

28

Table 6 – Relationship between Hawaii Content and Performance Standards for Social Studies
and the Bascom Affair lesson plan used in the study (Hawaii Standards 2005)
Standard 1: Historical Understanding: Change, Continuity, and Causality – Understand change and/or
continuity and cause and/or effect in history
Hawaii Benchmark: No benchmarks identified Study Benchmark: Examine the events that led to
war between the United States and Apache in 1861
Sample Performance Assessment:
The student:
Identifies the relationship between the United States and the Apache in 1861 prior to Bascom Affair, and
articulates the connections between the kidnapping of John Ward’s son, and the actions taken by the
United States Army that led to war.
Standard 2: Historical Understanding: Inquiry, Empathy, and Perspective – Use the tools and methods
of inquiry, perspective, and empathy to explain historical events with multiple interpretations and judge
the past on its own terms
Hawaii Benchmark: No benchmarks identified Study Benchmark: Examine the Bascom Affair
from multiple perspectives: American and Apache
Sample Performance Assessment:
The student:
Identifies their own preconceived biases, articulates how the principal actors perceived the events leading
up to the conflict at Apache Pass, and critical differences in each narrative that shape the way the event is
viewed today.
Standard 6: Cultural Anthropology: Systems, Dynamics, and Inquiry – Understand culture as a system
of beliefs, knowledge, and practices shared by a group and understand how cultural systems change over
time
Hawaii Benchmark: No benchmarks identified Study Benchmark: Examine how failure to
understand and account for culture, exacerbated
conditions between the United States and Apache
Sample Performance Assessment:
The student:
Identifies cultural misunderstandings that escalated tensions to the point of war and is able to discuss
opportunities for intervention that both parties missed.
Standard 7: Geography: World in Spatial Terms – Use geographic representations to organize, analyze,
and present information on people, places, and environments and understand the nature and interaction of
geographic regions and societies around the world
Hawaii Benchmark:
SS.11.7.1 – Trace changing political boundaries
under the influence of European Imperialism
Study Benchmark: Trace changing political
boundaries under the influence of American
Western Expansion
Sample Performance Assessment:
The student:
Examines the new political boundaries created by American Western Expansion in the present-day
American Southwest.
Hawaii Benchmark:
SS 11.7.2 – Use tools and methods of geographers
to understand changing views of world regions
Study Benchmark: Use tools and methods of
geographers to understand changing views of the
present day American Southwest
Sample Performance Assessment:
The student:
Uses geographic visualization methods to understand changing conceptions of the present day American
Southwest.
29

Second, Dan Thrapp’s The Conquest of Apacheria, is known by many scholars of the
Apache Wars as the seminal work on the subject. The bibliography comprises 16 pages of
sources including manuscripts, unpublished documents, newspaper articles, military reports,
government documents, and numerous other primary and secondary sources. Thrapp’s research
is unparalleled in its exhaustive treatment of the source material. This study relies heavily on his
experience and relatively unbiased interpretation of events to ensure accurate treatment of the
lesson material.
Third, Terry Mort’s, The Wrath of Cochise: The Bascom Affair and the Origins of the
Apache Wars, provides a significantly different interpretation of the Bascom Affair. Mort’s focus
on the awkward reality of being a young second lieutenant charged for the first time with leading
soldiers to the romanticized version often written about that yearn for war and are born ready to
lead helps personify Lieutenant Bascom. Mort does the Bascom Affair justice through his deep
analysis of not only the event itself, but the political situation surrounding it, and the education
and development of the United States Army and Apache leaders involved. His narrative provides
excellent insight into the decision cycles that brought the two nations to war, and helps tie the
study’s lesson materials to Hawaii state standards.
3.2.2. Sources of Historical Maps, Photographs, and Materials

The study relied on two sources of information to produce its historical maps,
photographs, and lesson materials: books and open source, publically available, internet data.
Higher quality or more appropriate primary sources may exist, but they are not readily available
to the average person. The source materials selected to support this study reflect those that
teachers could expect to access for free, and without the requirement for travel. By restricting the
30

study in this manner, the findings reflect outcomes that teachers can realistically expect to
replicate in their classrooms.

3.3 GIS Datasets
As identified in Chapter 2, technology requirements to implement desktop GIS programs
consistently present a barrier to further implementation. However, thanks to President Obama’s
ConnectEd initiative and Esri’s gift of free ArcGIS online organizational accounts to K-12
educators, teachers now have access to the required tools. In line with the decision to use source
materials readily available to classroom teachers, the study decided to employ ArcGIS online to
the fullest extent possible to create its visualization tools; the lone exception being the creation of
the final published maps that study’s static tool employed. By maximizing the use of ArcGIS
online, the study remained consistent in its effort to employ replicable methods that educators
can develop for their classrooms.
Most of the data required by this study are readily available through ArcGIS online.
Table 2, on the next page, describes each layer of data and its availability. Esri provided base
maps meet industry metadata standards and required little manipulation for use. In addition to the
base maps provided by Esri, the United States Geological Survey provides digital elevation
models, and land use/land cover data directly through the search function on ArcGIS online.
Likewise, the David Rumsey historic map collection also hosted georectified historical maps for
quick overlay through the ArcGIS online search tool. However, since most of the locations in
question no longer exist, they had to be recreated from historical records and digitized onto the
map.

31

Table 7 – Spatial Data Used to Support the Study
Dataset Content Format Attributes Quality
Key Locations Location of critical
events in the
Bascom Affair
Vector - Points,
Lines, and
Polygons
Name, coordinate,
description (ranch,
military outpost,
town, village, etc.)
N/A
Availability: Created for the study; digitized from historical records (See Bibliography).
Historical Photos Photos of key
persons, locations,
and conditions
Raster Name, description
Availability: Located through online queries using the Google search engine.
Base layer Base Imagery Raster Cover SE AZ Provide 5m
resolution of
Sonoita and
Apache Pass
Availability: ArcGIS online base maps used for this study.
DEM Elevation Data for
SE AZ
Raster Covers SE AZ 10 - 30 m
Availability: USGS elevation sets; ArcGIS online utilized to conduct observer point/line-of-sight
analysis and viewshed analysis.
Land use/Land
Cover
Soil types,
vegetation
Raster Soil type,
vegetation type,
height, density,
color
30 m
Availability: ArcGIS online provided USGS LULC sets.
Hydrography Hydrographic
profiles for SE AZ
Vector: points,
Lines, and
Polygons
Type of water
feature, direction
of flow, and name

Availability: ArcGIS online provided USGS NHD data for SE AZ; feature layer depicting the
national park visitor trail at Apache Pass identified the natural spring location.
Historic Maps Digital renditions
of historic maps
from the Apache
War era
Raster Georeferenced for
use in ArcGIS

Availability: ArcGIS online provided access to the David Rumsey historic map collection.

3.4 Data Aggregation

The lesson, and accompanying maps used in the study focus on a 30 day period from
January 19, 1861, through February 19, 1861. The study placed special emphasis on two events;
the kidnapping of Felix Ward in late January, and the recovery efforts led by Lieutenant Bascom
in the first week of February. Both sets of maps, static and dynamic, reflect the overall area of
32

interest in present Southeast Arizona, as well as provide regional level detail for the areas around
Sonoita Creek and Apache Pass.
3.5 Visualization Methods:
3.5.1. Static Visualization Tools

The static tool resembled a book, bound in a report folder. The study created six maps
and incorporated an additional two open source maps, eight photographs, and a timeline of
critical events to visualize the lesson materials for the study. The tool integrated the open source
maps, photographs, and timeline into the text of the document to replicate as closely as possible
the format in which the dynamic tool portrayed the lesson to the students. The tool incorporated
the six maps created for the study at the end of the text, in chronological order. The Decision to
place the maps at the end of the tool was made to enable the students to easily separate them
from the lesson material and reference them throughout their interaction with the data. For a
complete copy of the static tool, in its original format, refer to Appendix A.
The first map created for the study (shown on page 33 in figure 3) demonstrated a macro,
state level view, of the Arizona and New Mexico territories. The map focused on showing the
student how, and when the United States acquired the territories through a series of annexations,
cessations, and purchases in the first half of the nineteenth century. The second map portrayed
the same land mass, but from the Apache point of view (Figure 4 on page 34). United States’
political boundaries are absent from the map and replaced by shadowed imagery, and polygons
depicting the approximate range of Western and Chiricahua Apache land claims. The map
purposefully included the title “Indeh” for the Apache peoples, to re-emphasize the portion of the
text that described how the Apache refer to themselves.
33

The study created the first two maps in line with Hawaii Content and Performance
Standards for Social Studies number 2 and number 7. (Hawaii Standards 2005) The maps
intended to demonstrate changing political boundaries at the time of the Bascom Affair, as well
as how both parties viewed the landscape. The study employed both maps in an attempt to
redefine the student’s preconceived ideas and enable the student to analyze the Bascom Affair in
the context of its historical space and time. (Mares and Moschek 2013)

Figure 3 – Map of the Arizona and New Mexico Territories; date and source of land acquisition
are portrayed in the map’s legend.
34

Figure 4 – Map of Western and Chiricahua Apache lands at the time of the Bascom Affair in
1861; polygons represent approximate ranges reconstructed from historical accounts and do not
include portions of Apache territories from other bands or south of the U.S. – Mexico border.
The third map (Figure 5 below) reflected all four state standards targeted in the lesson
plan for the study. It defined the location of the Arivaipa Group of the Western Apache and
included the kidnapping site at John Ward’s Ranch for reference. Of note on the third map, the
United States’ political boundaries are absent, and the Apache land claims are emphasized. In
addition to demonstrating where the kidnappers originated, Map 3 helped the student understand
which bands claimed the lands surrounding the kidnapping site and begin to understand the
faulty logic behind accusing the Chiricahua Band.
35

Figure 5 – Map depicting the Arivaipa Apache Group of the Western Apache Band
Finally, the fourth Map (figure 6 below) provided a macro view of the key locations
during the Bascom Affair, while maps five and six (figures 7 and 8 on page 35 and 36
respectively) focused on a micro level view of the incident site. Map four depicted the
36

kidnapping site, location of the United States Army garrison at Fort Buchanan, the route used by
Lieutenant Bascom to approach Apache Pass, and the location of the Bascom Affair; map five
portrayed the line of sight analysis looking from positions in Apache Pass into the valley to
observe Bascom’s approach; and map six provided a smaller scale view of the critical sites at
Apache Pass to provide additional detail not available in map four.

Figure 6 – Macro level view of the key sites and routes involved in the Bascom Affair
37

Figure 7 – Line of sight analysis portraying the approximate range at which Apache Scouts
identified Lieutenant Bascom’s approach into Apache Pass
The study used this series of maps to focus on Hawaii State Standards 1 and 2. The three
maps illustrate the cause and effect, linear nature, of the incident and enable the student to trace
the events spatially and temporally on the map. Specifically, the study relied on the micro level
maps to illustrate the Apache point of view going into the meeting with the United States troops.
They demonstrated the Chiricahua’s awareness of Bascom’s approach, and their willingness to
allow him to advance and make camp near their positions in the vicinity of the spring in Apache
Pass. (Hawaii Standards 2005)
38

Figure 8 – Micro level view of the United States positions in Apache Pass, and their proximity to
the spring used by the Chokonen Apache Group.
The study designed the static tool to replicate standard texts and maps that classroom
teachers presently use in their curriculums. Teachers employing static products already make use
of additional visual aids to enhance their lesson plans. Therefore, it made sense to use the same
sketch maps, historical and present day photographs of Apache Pass and key personalities, and
timelines to enhance the delivery of lesson materials for the students in the static tool that the
dynamic tool employed in its format. Similar techniques are presently employed regularly in
classrooms and are in line with standard teaching practices.
39

3.5.2. Dynamic Visualization Tool

The story map used in the study was created using the tabbed map series application
created by Esri, and hosted on ArcGIS online. Each of the six maps identified in section 3.5.1
resides within the tabs of the story map. However, in addition to the static variants, the Story
Map hosts a live web map version of the data. Students had the opportunity to remain on any
given tab for as long as required, enabling them to navigate freely between them at their pace.
Figures 9 through 16 on pages 37 through 41 depict screen captures of each of the dynamic
tool’s tabs used in the study. In screen captures with the web maps depicted, the legend and map
overview tabs have been maximized. However, students had the ability to minimize both tools at
will in order to view a greater portion of the web map during the study.

Figure 9 – Screen capture of the title tab that students saw when they first navigated to the Story
Map.
40

Figure 10 – Screen capture of tab 2 entitled, “Identifying our Space” in the Story Map; the web
map from tab 2 equates to Map 1 in the static tool.

Figure 11 – Screen capture of tab 3 entitled, “The Indeh Circa 1861” in the Story Map; the web
map from tab 3 equates to Map 2 in the static tool. The base map used in the web map was
lightened to make it easier to read on computer monitors.
41

Figure 12 – Screen capture of tab 4 entitled, “The Raid on John Ward’s Ranch”; Map 3 from the
static tool embedded in the narrative with the ability to maximize the map on screen by clicking
the top right corner of the image.

Figure 13 – Screen capture of tab 5 entitled, “Cultural Considerations” in the Story Map; no
maps included in this tab.
42

Figure 14 – Screen capture of tab 6 entitled, “The Approach to Apache Pass” in the Story Map;
web map equates to map 4 from the static tool. Map 5 from the static tool embedded in the
narrative with the ability to maximize the map on screen by clicking the top right corner of the
image.

Figure 15 – Screen capture of tab 7 entitled, “The Conflict Begins” in the Story Map; web map
equates to map 6 from the static tool, and is further embedded in the narrative on the left.
43

Figure 16 - Screen capture of tab 8 entitled, “What Went Wrong” in the Story Map; map 4 from
the static tool embedded in the narrative with the ability to maximize the map on screen by
clicking the top right corner of the image.
Unlike the control group (Static), additional visualization aids in the experimental group
(Dynamic) were built directly into the story map. The study embedded open source maps,
photographs, and timelines in the scrolling narrative on the left side of the map or as a substitute
for the web map in the main frame (tab 5 and tab 8). At times, the study also embedded maps
developed for the static study tool into the narrative panel driven by the web map’s lack of a
scale bar in the story map tool. Incorporating the static maps as an image in the dynamic tool
provided students in the experimental group the opportunity to answer the spatial analysis
questions in the exam tool while maintaining uniformity in the data presented to the two groups
in the study.

44

3.5.3. Data Uniformity
The study’s focus is to test the visualization methods. Therefore, in order to isolate the
two tools, students from the control group (Static) and experimental group (Dynamic) received
access to the same data in their lessons. The study ensured that all data presented in the story
map, also be included in the static variant in order to isolate the delivery mechanism. Toward
that end, the static tool even reflected the tab titles from the dynamic tool. However, the static
tool incorporated the tab titles as section breaks in the narrative packet. The only variation
between the control group and the experimental group was the student’s ability to manipulate the
data in the web map, and the medium of delivery.
3.6 Methodology for Analyzing Findings:
The study drew from Ben Anderson’s methodology to develop its methodology for
assessing knowledge extraction and increases in long-term memory. The study carried two of
Anderson’s factors, effectiveness and efficiency, to develop its findings. (Anderson 2015) To
assess critical thought, the study incorporated Bloom’s Taxonomy of Cognitive Learning Skills,
modelled after the research conducted by Liu Et Al. (2010) By incorporating methodologies
from prior visualization research in a new environment, the study adds to the body of current
knowledge. Specific methodologies employed by the study to answer each of its research
questions are discussed in greater detail in sections 3.6.1 – 3.6.3.
3.6.1. Determining student performance in knowledge extraction
Research question #1 asked, are there measurable increases in student performance in
knowledge extraction? To answer this, the study adopted Anderson’s criteria of effectiveness and
efficiency to measure the student’s ability to extract information from their given study tool to
respond to questions on the exam tool. The study measured effectiveness by recording the
45

number of points earned divided by the number of points possible in each section of the exam
tool for each student. The study then determined the mean scores achieved by each study group
on each section and compared them to determine which population performed knowledge
extraction tasks most effectively. Independent samples (two-tailed) t-tests compared the mean
scores from each section, from each study group, to determine statistical significance.
The study measured efficiency somewhat differently than Anderson’s model. Whereas
Anderson recorded the amount of time it took each participant to answer each question, this
study observed how many questions remained un-answered in the first iteration of the study. The
online format employed by Anderson in his research made it possible to record the exact amount
of time it took each participant to respond to each task. However, the classroom environment
employed in this study did not support this method. The student to researcher ratio in the
classroom made it impossible to monitor each student’s progress with that level of fidelity.
However, since the study monitored participation, it was possible to know whether each student
in the study worked continuously throughout the class period.
Very few students completed the exam tool in its entirety during the first iteration of the
study. Only four students from the control group (Static) and three students from the
experimental group (Dynamic) completed all twenty-four questions on the exam tool. Therefore,
the study measured efficiency based on the mean progress the students from each group made on
the exam tool. The study did not record completion percentage of students the classroom teacher
assessed as disengaged in order to avoid lack of effort from influencing the findings.
3.6.2. Determining measurable increases in student long term memory
Research question #2 questioned whether the use of story maps increased the likelihood
of transitioning ideas from short-term to long term memory. To answer this, the study used the
46

same criterion it employed to assess knowledge extraction. However, the way that the study
assessed the effectiveness and efficiency changed to reflect the different focus of the question.
To assess effectiveness, the study subtracted student scores from the second iteration of the
study, from their scores achieved while using the study tool in the first iteration of the study. The
study then recorded the delta between the two scores and calculated the mean difference for each
study group on each section of the exam tool to determine how much information the students
retained between iterations of the study. To assess efficiency, the study recorded the time of
completion for each student during the second iteration of the study and calculated the mean
completion time for each study group. The study then compared the mean completion time to
the mean difference in scores to assess whether a correlation existed between a faster response
time and greater transfer of ideas to long term memory.
3.6.3. Evaluating increases in critical thought
Research question #3 sought to determine if story maps facilitate critical thought through
active engagement with data as it is presented. To answer this question, the study focused on two
factors: class participation and critical thought. As previously discussed, the classroom teacher
observed and recorded participation for each student in the study. Students received credit for
participation if they remained on task throughout the entire experiment. The study developed
group participation scores based on the percentage of the population who received credit for
active participation. Table 8, on page 47, defines the thresholds for each participation category.
In addition to class participation, the study recorded student responses to whether they engaged
with the study tool during their two week, self-study period.
47

Table 8 – Description of Student Participation Scores
Student Participation Percentage
Full Participation > 90%
Partial Participation 75% - 90%
Low Participation 50% - 74%
Poor Participation < 50%

The study assessed critical thought using the five categories of Bloom’s Taxonomy of
Cognitive Learning Skills. Recall represented the students’ ability to remember important facts,
numbers, or events; the study judged understanding based on the students’ ability to explain
figures, tables, and concepts clearly outlined or provided in the lesson material; analyze reflected
a consideration of cause and effect; evaluate indicated that the student provided critical or
expansive comments beyond recall; and create applied to genuine or creative ideas developed by
the student that could not be identified explicitly in the lesson material. The researcher graded
each essay with a score from 1 to 5 based on the highest level of cognitive learning displayed in
the student’s written response (refer to Table 9 on page 48). The study conducted two rounds of
grading to verify the researcher’s initial assessment, and provide the most objective score
possible. For greater detail on how the study assessed student responses, refer to the grading
rubric provided in Appendix G. To assess critical thought, the study counted the total number of
responses in each group that met each skill. It also identified the mean score for each essay
question in each of the study groups. The study assessed both counts and averages to determine
the level of critical thought generated by each tool.

48

Table 9 – Scoring Criteria for Essay Questions on the Exam Tool
Bloom’s Cognitive Skill Score
Recall 1
Understanding 2
Analyze 3
Evaluate 4
Create 5

Finally, the study compared the level of participation and engagement with the tool to the
level of critical thought observed in student responses. To answer research question #3, the study
had to determine an increase in critical thought, and whether participation and engagement with
the tool were factors in higher scores. Table 10 (below) provides a description of each research
question and its corresponding hypothesis.
Table 10 – Description of the Relationship Between the Study’s Research Questions and Their
Corresponding Hypotheses.
Research Question Hypothesis Null Hypothesis Alternate Hypothesis
1. Are there
measurable increases
in student
performance in
knowledge
extraction?
higher levels of
performance in the
experimental group
(Dynamic)
no measurable
difference between
the two methods
higher levels of
performance in the
control group (Static)
2. Does the use of
story maps increase
the likelihood
transitioning ideas
from short term to
long term memory?
higher levels of
retention in the
experimental group
(Dynamic)
no measurable
difference between
the two methods
higher levels of
retention in the
control group (Static)
3. Do story maps
facilitate critical
thought through active
participation with data
as it is presented?
higher levels of
critical thought in the
experimental group
(Dynamic) and greater
levels of participation
no measurable
difference between
the two methods
higher levels of
critical thought in the
control group (Static)
and greater levels of
participation

49

Chapter 4 : Findings
Chapter 4 describes the results of the study’s classroom based, student performance experiment.
However, prior to proceeding with the findings, it merits re-emphasizing the scope of this study.
The findings recorded in this chapter reflect the outcomes from this particular population of high
school students over a two week period of research, consisting of two proctored iterations of the
study and a two-week self-study opportunity. Prior to beginning the experiment, the study
confirmed that the students had no prior knowledge of the Bascom Affair that would influence
their responses to the questions on the exam tool. Student responses reflect their understanding
of the subject based solely on the data provided in the research tool employed. Although the
study raises some thought-provoking insights, the findings are not conclusive, nor are they
intended to be applied universally. The study’s findings reflect an initial investment into
classroom based visualization studies, with the intent of inspiring additional research in the field.
The findings are reported in two categories; Quantitative findings based on the
methodology reported in chapter 3, and qualitative findings based on observations from the
researcher and classroom teacher during the conduct of the study iterations. Section 4.1 describes
student performance in knowledge extraction tasks; Section 4.2 describes student performance in
transitioning ideas to long term memory; and Section 4.3 reports the findings related to
generating critical thought. The chapter closes with a discussion of qualitative findings based on
observations in Section 4.4.
Statistically significant findings are identified within their tables in bold font. Significant
findings at 90 percent confidence are annotated with a single asterisk “*”; significance at 95
percent confidences are annotated by two asterisks “**”; and significance at 99 percent
50

confidence are annotated by three asterisks “***”. Unless required to denote the level of
significance, the study reflects P-scores rounded to the nearest hundredth.
4.1 Classroom Based Knowledge Extraction Performance:
4.1.1. Effectiveness
When looking at each research group as a whole, the study recorded similar performance
scores on the exam tool from both groups. Table 11 (below) captures the breakdown of student
scores in fill-in-the-blank, true-or-false, short answer, and spatial analysis format. The control
group (Static) students earned a higher average mean score on fill-in-the-blank questions,
outperforming the experimental group (Dynamic) by an average of 0.5 points (6.08 to 5.58).
They also earned higher marks for true-or-false (0.75 to 0.70), and spatial analysis questions
(0.88 to 0.82). The experimental group recorded a higher average score on short answer
questions (0.92 to 0.86). However, unpaired, two-tailed T-tests demonstrated that the differences
recorded between the two groups fall within expected ranges. When considering the full
population of each group, the study supports the null hypothesis that neither tool offers students a
significant advantage over the other in the classroom setting.
Table 11 - Student scores by question format
Question
Format
Control Group (Static) Experimental Group (Dynamic) P
Min Max Mode Mean SD Min Max Mode Mean SD Score
Fill
Blank
2 10 6 6.08 1.80 2 8 6 5.58 1.56 0.14
True or
False
0 1 1 0.75 0.44 0 2 1 0.70 0.57 0.62
Short
Answer
0 2 0 0.86 0.84 0 2 0 0.92 0.80 0.71
Spatial
Analysis
0 6 0 0.88 1.13 0 5 0 0.82 1.07 0.78

51

Isolation of the gender variable produced different results. Tables 12 (below) and 13
(page 52) describe the performance of male and female students in each group. Although the
study captured no statistically significant differences in the male population, it deserves mention
that the male population in the experimental group (Dynamic) scored higher in all four question
formats than the control group (Static). Perhaps more notable, when considered in line with the
performance of female students who used the dynamic tool, is that the male population in the
experimental group recorded higher scores than the full experimental population in each of the
four, question formats as well.
Table 12 - Student scores amongst male students by question format

Question
Format
Control Group (Static)
Males
Experimental Group (Dynamic)
Males

P
Count Min Max Mean SD Count Min Max Mean SD Score
Fill
Blank
20 3 10 5.70 1.82 24 2 8 5.83 1.55 0.80
True or
False
20 0 1 0.65 0.48 24 0 2 0.79 0.58 0.39
Short
Answer
20 0 2 0.95 0.81 24 0 2 1.04 0.61 0.68
Spatial
Analysis
20 0 2 1.05 0.74 24 0 5 1.13 1.30 0.81

Contrary to the findings identified so far, which failed to disprove the null hypothesis, the
performance of female students yielded statistically significant results supporting the use of static
tools in the classroom. Unlike their male counterparts who performed higher using the dynamic
tool, females from the control group (Static) outperformed or matched their peers in the
experimental group (Dynamic) in all four, question formats. Most notably, the T-test confirmed
the difference between the average mean scores in the fill-in-the-blank section (6.32 to 5.35) as
statistically significant with a probability score (P) of 0.03. If the null hypothesis were true, then
52

in a random sampling of similar size, future experiments should observe a difference between the
mean scores greater than the 0.97 points observed in this study in less than 3% of attempts. In
this instance, the study finds evidence to support the alternate hypothesis that static maps provide
female secondary school students an advantage in knowledge extraction tasks.
Table 13 - Student scores amongst female students by question format

Question
Format
Control Group (Static)
Females
Experimental Group (Dynamic)
Females

P
Count Min Score Mean SD Count Min Max Mean SD Score
Fill
Blank
31 2 20 6.32 1.75 26 2 8 5.35 1.54 0.03**
True or
False
31 0 1 0.81 0.40 26 0 2 0.62 0.56 0.14
Short
Answer
31 0 2 0.81 0.86 26 0 2 0.81 0.92 1.00
Spatial
Analysis
31 0 6 0.77 1.31 26 0 2 0.54 0.69 0.43
Note: ** represents a statistically significant difference at 95% confidence; intermediate values
used in calculations: t = 2.1756, t Critical (two-tail) = 2.0040, df = 55
Isolating age also produced statistically significant results in support of the alternate
hypothesis. While students in the age 14 to 15, and 16 years old groups reflected the findings of
the general population, older students (17 to 18 years) in the control group (Static) outperformed
their peers in the experimental group (Dynamic) by the largest margin in the study for
knowledge extraction. The older students using the static tools produced an average score of 6.73
on the fill-in-the-blank questions, a full 1.90 points higher than their peers in the experimental
group who earned an average of 4.83 points on the same questions. T-test confirmed the
difference as statistically significant at 90 percent confidence with a P-score of 0.0508. Tables 14
through 16 on page 53 provide the breakdown of student performance by age group on
knowledge extraction tasks.
53

Table 14 - Student scores amongst students age 14-15 by question format

Question
Format
Control Group (Static)
Age 14-15
Experimental Group (Dynamic)
Age 14-15

P
Count Min Max Mean SD Count Min Max Mean SD Score
Fill
Blank
27 2 10 6.00 1.75 34 3 8 5.71 1.31 0.46
True or
False
27 0 1 0.70 0.46 34 0 2 0.65 0.54 0.70
Short
Answer
27 0 2 0.67 0.81 34 0 2 0.97 0.82 0.16
Spatial
Analysis
27 0 4 0.74 0.89 34 0 5 0.79 1.16 0.85
Table 15 - Student scores amongst students age 16 by question format

Question
Format
Control Group (Static)
Age 16
Experimental Group (Dynamic)
Age 16

P
Count Min Max Mean SD Count Min Max Mean SD Score
Fill
Blank
13 3 8 5.70 1.59 10 2 8 5.60 2.01 0.90
True or
False
13 0 1 0.70 0.46 10 0 2 0.70 0.64 1.00
Short
Answer
13 0 2 0.85 0.77 10 0 2 0.70 0.64 0.62
Spatial
Analysis
13 0 3 0.62 0.92 10 0 1 0.40 0.49 0.50
Table 16 - Student scores amongst students age 17-18 by question format

Question
Format
Control Group (Static)
Age 17-18
Experimental Group (Dynamic)
Age 17-18

P
Count Min Max Mean SD Count Min Max Mean SD Score
Fill
Blank
11 4 10 6.73 1.76 6 2 7 4.83 1.77 0.05*
True or
False
11 0 1 0.91 0.29 6 0 2 1.00 0.58 0.67
Short
Answer
11 0 2 1.36 0.77 6 0 2 1.00 0.82 0.38
Spatial
Analysis
11 0 6 1.55 1.60 6 0 2 1.67 0.75 0.86
Note: * represents a statistically significant difference at 90% confidence; intermediate values
used in calculations: t = 1.9850, t Critical (two-tail) = 1.7531, df = 15
54

4.1.2. Efficiency
Efficiency scores for the general population support the alternate hypothesis. On average,
students in the control group (Static) responded to approximately two additional questions on the
exam tool during the first iteration of the study. The T-test confirmed the statistical significance
of the disparity between the groups at 90 percent confidence with a P-score of 0.06. Further
isolation of the gender variable revealed that female students in the control group answered
significantly more questions than those using the dynamic tool in the experimental group
(Dynamic). Control group females responded to an additional three questions on the exam tool
during their allotted time in the class period. The T-test confirmed the statistical significance of
the efficiency gap between the two groups with a P-score of 0.02 at 95 percent confidence. Table
17 (below) identifies the efficiency scores for each population.
Table 17 - Student efficiency scores

Population
Control Group (Static) Experimental Group (Dynamic) P
Count Min Max Mean SD Count Min Max Mean SD Score
Full 51 7 24 15.41 4.76 50 6 23 13.66 4.54 0.06*
Males 20 8 23 14.85 4.44 24 6 21 14.91 4.21 0.96
Females 31 7 24 15.77 4.92 26 6 23 12.50 4.52 0.01**
Age 14-15 27 7 24 14.63 4.72 34 6 21 13.18 4.20 0.20
Age 16 16 8 24 15.08 5.14 10 6 19 12.60 4.10 0.21
Age 17-18 11 11 23 17.73 3.49 6 11 23 18.17 4.49 0.83
Note: * represents a statistically significant difference at 90% confidence; intermediate values
used in calculations: t = 1.8734, t Critical (two-tail) = 1.6604, df = 99
** represents a statistically significant difference at 95% confidence; intermediate values used in
calculations: t = 2.5503, t Critical (two-tail) = 2.0040, df = 55
4.2 Transition to Long Term Memory:
4.2.1. Effectiveness
In terms of effectively transitioning ideas from short term to long term memory, the
study’s findings support the null hypothesis that neither tool provides students a statistically
55

significant advantage in the classroom. In the general population, the experimental group
(Dynamic) outperformed the control group (Static) in three of the question formats: Fill-in-the-
blank (3.36 to 3.41), true-or-false (-0.08 to 0.10), and spatial analysis (0.04 to 0.22). Students in
the control group demonstrated increased retention on the short answer format (0.57 to 0.64).
However, when run through the T-tests, the differences in average scores, failed to meet the
threshold for significance in any of the question formats. For a full description of average scores
for each population refer to tables 18 through 23 on pages 55 to 57.
Although not statistically significant, two trends stand out when isolating gender and age.
Female students from the experimental group (Dynamic) retained more information from the
dynamic visualization tool than their peers using the static tool in all four question formats. They
also outperformed their male counterparts in both groups and demonstrated greater transition of
ideas to long term memory than either group in the general population. Similarly, 17 to 18-year-
olds in the experimental group exhibited greater retention of ideas than their peers in the control
group (Static). However, unlike the female population, the older student scores do not stand up
when measured against the full population.
Table 18 - Difference in Scores Between Study Iterations; 0 reflects no change; negative
Numbers Reflect an Increase in Student Score From Test 1 on Test 2
Question
Format
Control Group (Static) Experimental Group (Dynamic) P
Min Max Mode Mean SD Min Max Mode Mean SD Score
Fill
Blank
0 8 3 3.41 1.75 -1 7 3 3.36 1.71 0.88
True or
False
-1 1 0 0.10 0.57 -2 2 0 -0.08 0.72 0.16
Short
Answer
-1 2 0 0.57 0.90 -1 2 1 0.64 0.93 0.70
Spatial
Analysis
-3 5 0 0.22 1.11 -3 4 0 0.04 1.06 0.41

56

Table 19 - Difference in Scores of Male Students Between Study Iterations; Negative Numbers
Reflect an Increase in Student Score From Test 1 on Test 2

Question
Format
Control Group (Static)
Males
Experimental Group (Dynamic)
Males

P
Count Min Max Mean SD Count Min Max Mean SD Score
Fill
Blank
20 0 8 3.15 2.13 24 1 7 3.63 1.41 0.38
True or
False
20 -1 1 0.20 0.60 24 -1 1 -0.04 0.61 0.20
Short
Answer
20 -1 2 0.55 0.81 24 0 2 0.83 0.62 0.20
Spatial
Analysis
20 -1 2 0.25 0.77 24 -3 4 0.25 1.36 1.00
Table 20 - Difference in Scores of Female Students Between Study Iterations; Negative Numbers
Reflect an Increase in Student Score From Test 1 on Test 2

Question
Format
Control Group (Static)
Females
Experimental Group (Dynamic)
Females

P
Count Min Max Mean SD Count Min Max Mean SD Score
Fill
Blank
31 1 6 3.58 1.43 26 -1 7 3.12 1.91 0.30
True or
False
31 -1 1 0.03 0.54 26 -2 2 -0.12 0.80 0.49
Short
Answer
31 -1 2 0.58 0.91 26 -1 2 0.46 1.12 0.66
Spatial
Analysis
31 -3 5 0.19 1.28 26 -1 2 -0.15 0.60 0.22
Table 21 - Difference in Scores of Students Age 14-15 Between Study Iterations; Negative
Numbers Reflect an Increase in Student Score From Test 1 on Test 2

Question
Format
Control Group (Static)
Age 14-15
Experimental Group (Dynamic)
Age 14-15

P
Count Min Max Mean SD Count Min Max Mean SD Score
Fill
Blank
27 0 8 3.70 2.02 34 -1 7 3.32 1.76 0.44
True or
False
27 -1 1 0.11 0.57 34 -2 2 -0.15 0.77 0.15
Short
Answer
27 -1 2 0.30 0.76 34 -1 2 0.62 0.97 0.17
Spatial
Analysis
27 -3 2 0.15 0.97 34 -3 4 0.18 1.12 0.91

57

Table 22 - Difference in Scores of Students Age 16 Between Study Iterations; Negative Numbers
Reflect an Increase in Student Score From Test 1 on Test 2

Question
Format
Control Group (Static)
Age 16
Experimental Group (Dynamic)
Age 16

P
Count Min Max Mean SD Count Min Max Mean SD Score
Fill
Blank
13 1 5 2.92 1.07 10 1 6 3.70 1.62 0.18
True or
False
13 -1 1 0.08 0.62 10 -1 1 0.10 0.54 0.94
Short
Answer
13 -1 2 0.62 0.84 10 -1 2 0.50 0.81 0.73
Spatial
Analysis
13 -2 1 -0.08 0.92 10 -1 0 -0.50 0.50 0.21
Table 23 - Difference in Scores of Students Age 17-18 Between Study Iterations; Negative
Numbers Reflect an Increase in Student Score From Test 1 on Test 2

Question
Format
Control Group (Static)
Age 17-18
Experimental Group (Dynamic)
Age 17-18

P
Count Min Max Mean SD Count Min Max Mean SD Score
Fill
Blank
11 1 6 3.27 1.54 6 1 5 3.00 1.41 0.73
True or
False
11 -1 1 0.09 0.51 6 -1 1 0.00 0.58 0.75
Short
Answer
11 0 2 1.18 0.83 6 0 2 1.00 0.82 0.67
Spatial
Analysis
11 0 5 0.73 1.42 6 -1 2 0.17 1.07 0.41

4.2.2. Efficiency
Efficiency scores for memory retention continue the trend established from scores
reported in the previous section. Like the scores related to the amount of time required for
students to complete knowledge extraction tasks from the first iteration, the general population
scores indicate that the control group recalled information more efficiently. The control group
(Static) completed their exam tool faster than the experimental group (Dynamic), with an
58

average score of 14.29 to 16.56 minutes. T-test confirmed the statistical significance of the
finding at 90 percent confidence with a reported a P-score of 0.07.
Unlike effectiveness, when isolating the age variable, the study’s findings reveal highly
statistically significant information that supports the alternate hypothesis reflecting that the static
tool outperformed the dynamic tool. In students age 14 to 15, the control group (Static)
outperformed the experimental group (Dynamic) by 5.38 minutes. T-test confirmed the statistical
significance of the efficiency score, returning a P-score of 0.0026. The findings indicate that in
students age 14 to 15, the static tool offers a significant advantage to students in terms of
transitioning ideas from short-term to long-term memory.
In contrast to the younger population group, older students appeared to enjoy greater
success at transitioning ideas from short-term to long-term memory using the dynamic tool.
Students age 17 to 18 in the experimental group (Dynamic) completed the exam tool 3.76
minutes faster, demonstrating more efficient and rapid recall of information. The T-test
confirmed the large gap in performance at 90 percent confidence with a P-score of 0.06. Table 24
(page 59) provides a full description of efficiency scores based on student completion times on
the second iteration of the study.

59

Table 24 - Student Completion Times on the Second Iteration of the Study; Min/Max Times are
Rounded to the Nearest Minute

Population
Control Group (Static) Experimental Group (Dynamic) P
Count Min Max Mean SD Count Min Max Mean SD Score
Full 51 7 26 14.29 5.07 50 7 60 16.56 7.26 0.07*
Males 20 8 26 14.45 5.45 24 7 24 15.88 3.75 0.31
Females 31 7 25 14.19 4.81 26 10 60 17.19 9.36 0.12
Age 14-15 27 8 19 12.00 3.22 34 7 60 17.38 8.40 0.0026***
Age 16 16 7 25 14.15 5.28 10 10 19 13.90 2.70 0.95
Age 17-18 11 15 26 20.09 3.80 6 13 23 16.33 3.35 0.06*
Note: * represents statistically significant differences at 90%; intermediate values used in
calculations for the full population: t = -1.8032, t Critical (two-tail) = 1.6604, df = 99; ;
intermediate values used in calculations for the age 17 - 18 population: t = 1.9064, t Critical
(two-tail) = 1.7531, df = 15
*** represents a very statistically significant difference at 99% confidence - Intermediate values
used in calculations: t = -3.0980, t Critical (two-tail) = 2.6618 df = 59
4.3 Assessing Critical Thought:
4.3.1. Effectiveness
The study primarily evaluated critical thought based on student responses to the exam
tool’s four essay questions in each of the two study iterations. Response rates to the essay
questions increased for both research groups between the first and second iteration. As
demonstrated by Table 25 (below), the control group (Static) saw a 29% increase and the
experimental group (Dynamic) more than doubled their rate of response from 30% on the first
test, to 76% on test two.
Table 25 - Percentage of Students That Attempted the Essay Questions on the Exam Tool
1
st
Test 2
nd
Test
Control Group (Static) 49% 78%
Experimental Group (Dynamic) 30% 76%

60

Based on mean scores as depicted in table 26 on page 57, the control group (Static)
outperformed the experimental group (Dynamic) on three of the four essay questions during the
first iteration. On essay question #1, which asked the students to describe where the Bascom
Affair took place, the control group recorded an average score of 1.20. The experimental group
scored an average of 0.52 on the same question using the dynamic tool. When assessed using
through the T-test, the difference in scores received a P-score of 0.02 at 95 percent confidence.
The experimental group (Dynamic) improved their performance on the second iteration
as their response rate improved. The control group (Static) continued to score higher on the first
essay question, but the difference between the scores no longer registered statistical significance.
As illustrated in Table 26 on page 61, the experimental group doubled their mean score from
0.52 on the first exam, to 1.04 on exam two. Meanwhile, despite an increased response rate in the
control group as well, their mean score only improved to 1.27, a difference of 0.07 hundredths of
a point.
Both research groups experienced statistically significant improvement between the two
iterations, as demonstrated in Table 27 on page 61. The control group (Static) improved from
0.82 to 1.39 (P-score of 0.0538) on question 2; from 0.27 to 0.90 on question 3 (P-score of
0.0059); and from 0.29 to 0.88 (P-score of 0.02) on question 4. The experimental group
(Dynamic) saw the largest improvement, going from a 0.52 to 1.04 on question 1(P-score of
0.03); a 0.040 to a 1.58 on question 2 (P-score of 0.0001); a 0.36 to a 0.86 on question 3(P-score
of 0.04), and from 0.18 to 1.12 on essay question 4 (P-score of 0.0004) on the second iteration.
Universal improvement in mean scores by both research groups continues to support the null
hypothesis. Following the second study iteration, neither group posted scores significantly
greater than the other.
61

Table 26 - Student Performance on Exam Tool Essay Questions

Question
Number
Control Group (Static) Experimental Group
(Dynamic)

P
Min Max Mean SD Min Max Mean SD Score
1
st
Test
Essay #1 0 5 1.20 1.69 0 5 0.52 1.04 0.02**
Essay #2 0 5 0.82 1.45 0 5 0.40 1.10 0.10
Essay #3 0 4 0.27 0.84 0 5 0.36 1.11 0.65
Essay #4 0 4 0.29 0.89 0 3 0.18 0.71 0.50

2
nd
Test
Essay #1 0 5 1.27 1.60 0 4 1.04 1.30 0.43
Essay #2 0 4 1.39 1.51 0 5 1.58 1.73 0.56
Essay #3 0 4 0.90 1.36 0 5 0.86 1.23 0.88
Essay #4 0 5 0.88 1.54 0 5 1.12 1.68 0.46
Note: ** represents a statistically significant difference at 95% confidence; intermediate values
used in calculations: t = 2.3870, t Critical (two-tail) = 1.9842, df = 99
Table 27 - Difference in Average Scores Between Study Iterations
Question # 1
st
Test 2
nd
Test Change P-Score
Control
Group
(Static)
(N=51)
Essay #1 1.20 1.27 +0.07 0.83
Essay #2 0.82 1.39 +0.57 0.05*
Essay #3 0.27 0.90 +0.63 0.0059***
Essay #4 0.29 0.88 +0.59 0.02**

Experimental
Group
(Dynamic)
(N=50)
Essay #1 0.52 1.04 +0.52 0.03**
Essay #2 0.40 1.58 +1.18 0.0001***
Essay #3 0.36 0.86 +0.50 0.04**
Essay #4 0.18 1.12 +0.94 0.0004***
Note: Control Group: * represents a statistically significant difference at 90% confidence;
intermediate values used in calculations: t = -1.9204, t Critical (two-tail) = 1.6602, df = 100
** represents a statistically significant difference at 95% confidence; intermediate values used in
calculations: t = -2.3346, t Critical (two-tail) = 1.9840, df = 100
*** represents a statistically significant difference at 99% confidence; intermediate values used
in calculations: t = -2.7718, t Critical (two-tail) = 2.6259, df = 100
Experimental Group: ** represent statistically significant differences at 95% confidence;
intermediate values used in calculations for essay #1: t = -2.1879, t Critical (two-tail) = 1.9845,
df = 98; intermediate values used in calculations for essay #3: t = -2.1103, t Critical (two-tail) =
1.9845, df = 98
*** represent statistically significant differences at 99% confidence; intermediate values used in
calculations for essay #2: t = -4.0287, t Critical (two-tail) = 2.6269, df = 98; intermediate values
used in calculations for essay #4: t = -3.6041, t Critical (two-tail) = 2.6269; df = 98
62

Assessing student responses by count provided the study additional insight. Table 28
(below) depicts the number of responses recorded from each research group organized by the
highest level of cognitive thought exhibited in the student’s response. The study considered
responses recorded as analyze, evaluate, or create as higher forms of cognitive thought, while
simple recall and understanding reflected basic knowledge of the subject. Looking at responses
by count allowed the study to negate the influence of missing responses by determining
percentages derived from actual answers provided by the students. By count, students in the
control group (Static) outperformed the experimental group (Dynamic) in both study iterations,
although the margin decreased between the first and second iteration of the study.
Table 28 - Student Performance By Count Reflecting Bloom’s Hierarchy of Cognitive Thought
Question
Number
Control Group(Static) Experimental Group (Dynamic)
Recall Understanding Analyze Evaluate Create Recall Understanding Analyze Evaluate Create
1
st
Test
Essay #1
4 7 0 7 3 7 5 0 1 1
Essay #2
7 2 4 1 3 1 2 2 1 1
Essay #3
2 1 2 1 0 1 2 1 0 2
Essay #4
2 0 3 1 0 0 0 3 0 0
Total
15 10 9 10 6 9 9 6 2 4

2
nd
Test
Essay #1
16 6 0 3 5 17 4 1 6 0
Essay #2
8 3 11 6 0 8 2 8 7 3
Essay #3
4 3 8 3 0 10 7 2 2 1
Essay #4
3 2 4 4 2 7 0 8 0 5
Total
31 14 23 16 7 42 11 19 15 9

On the first exam, students from the control group (Static) answered 25/50 (50%)
questions that the study scored as higher cognitive thought. The experimental group (Dynamic)
recorded 12/30 (40%) higher level responses on the same test. Similarly, on the second exam, the
control group recorded 46/91 (51%) higher level responses while the experimental group earned
43/96 (45%). These findings support the alternate hypothesis, indicating that the static tool
63

provided students an increased advantage in the classroom, and generated increased levels of
critical thought in the general student population.
Isolation of the gender variable continued to yield significant findings. Although
differences in mean scores across the male population of students failed to yield statistically
significant findings demonstrating a benefit from either tool, males in the experimental group
(Dynamic) continued to post higher average scores. As indicated in Table 29 (below), the
experimental population outperformed the control group (Static) on three out of four essay
questions in both iterations of the study. Both groups saw improvement between their scores on
the first test and the second test, but not to the same level witnessed in the general population.
Table 30 (on page 64) identifies significant findings in essay question 1 for the control group,
and question 2 for the experimental group.
Table 29- Male Student Performance on Exam Tool Essay Questions
Question
Number
Control Group Males (Static)
(N=20)
Experimental Group (Dynamic)
Males (N=24)
P
Min Max Mean SD Min Max Mean SD Score
1
st
Test
Essay #1 0 5 0.75 1.51 0 5 0.75 1.30 1.00
Essay #2 0 5 1.00 1.70 0 5 0.67 1.43 0.49
Essay #3 0 3 0.20 0.68 0 5 0.63 1.50 0.24
Essay #4 0 3 0.15 0.65 0 3 0.38 0.99 0.38

2
nd
Test
Essay #1 0 5 1.65 1.93 0 4 1.21 1.55 0.41
Essay #2 0 4 1.55 1.53 0 5 1.75 1.83 0.70
Essay #3 0 4 0.80 1.25 0 4 0.96 1.14 0.66
Essay #4 0 4 0.65 1.31 0 5 0.83 1.40 0.67

By count, the study’s findings continue to reinforce the null hypothesis amongst the male
population. In the first iteration, males in the control group (Static) received credit for 9/15
(60%) responses matching higher level cognitive values. The experimental group (Dynamic)
earned 12/21(57%) higher level responses. Greater parody occurred on the second exam where
64

the control group scored 18/36 (50%) higher level scores to the experimental group’s 23/45
(51%). Male student scores by count are captured below in Table 31.
Table 30 - Difference in Average Scores for Male Student Between Study Iterations
Question # 1
st
Test 2
nd
Test Change P-Score
Control
Group
(Static)
(N=20)
Essay #1 0.75 1.65 +0.90 0.12
Essay #2 1.00 1.55 +0.55 0.29
Essay #3 0.20 0.80 +0.60 0.07*
Essay #4 0.15 0.65 +0.50 0.14

Experimental
Group
(Dynamic)
(N=24)
Essay #1 0.75 1.21 +0.46 0.27
Essay #2 0.67 1.75 +1.08 0.03**
Essay #3 0.63 0.96 +0.33 0.39
Essay #4 0.38 0.83 +0.45 0.21
Note: Control Group: * represents a statistically significant difference at 90% confidence;
intermediate values used in calculations: t = -1.8401, t Critical (two-tail) = 1.6859, df = 38
Experimental Group: ** represent a statistically significant difference at 95% confidence;
intermediate values used in calculations: t = -2.2338, t Critical (two-tail) = 2.0129, df = 46
Table 31 - Male Student Performance By Count Reflecting Bloom’s Hierarchy of Cognitive
Thought
Question
Number
Control Group (Static) Males Experimental Group (Dynamic) Males
Recall Understanding Analyze Evaluate Create Recall Understanding Analyze Evaluate Create
1
st
Test
Essay #1
3 1 0 0 2 5 2 0 1 1
Essay #2
1 0 3 0 2 1 0 2 1 1
Essay #3
1 0 1 0 0 0 1 1 0 2
Essay #4
0 0 1 0 0 0 0 3 0 0
Total
5 1 5 0 4 6 3 6 2 4
2
nd
Test
Essay #1
5 2 0 1 4 5 2 0 5 0
Essay #2
3 2 4 3 0 2 0 6 3 2
Essay #3
2 2 2 1 0 4 6 1 1 0
Essay #4
2 0 1 2 0 3 0 4 0 1
Total
12 6 7 7 4 14 8 11 9 3

Table 32 on page 65 portrays the scores from the female population of students on the
essay questions from the exam tool. On the first essay question, females in the control group
65

(Static) recorded an average score of 1.48. The experimental group (Dynamic) earned an average
score of 0.31. The T-test returned a P-score of 0.0021, indicating strong support for the alternate
hypothesis. In fact, the disparity between the recorded average scores of female students in both
groups during the first iteration of the study was so large, that it influenced the statistically
significant finding in the general population on the same question.
Table 32- Female Student Performance on Exam Tool Essay Questions
Question
Number
Control Group (Static)
Females (N=31)
Experimental Group (Dynamic)
Females (N=26)
P
Min Max Mean SD Min Max Mean Std_Dev Score
1
st
Test
Essay #1 0 5 1.48 1.74 0 2 0.31 0.67 0.002***
Essay #2 0 5 0.71 1.25 0 2 0.15 0.53 0.04**
Essay #3 0 4 0.32 0.93 0 2 0.12 0.42 0.32
Essay #4 0 4 0.39 1.01 0 0 0 0 N/A

2
nd
Test
Essay #1 0 5 1.03 1.28 0 4 0.88 0.97 0.63
Essay #2 0 4 1.29 1.49 0 5 1.42 1.62 0.75
Essay #3 0 4 0.97 1.43 0 5 0.77 1.31 0.59
Essay #4 0 5 1.03 1.66 0 5 1.38 1.86 0.46
Note: ** represent a statistically significant difference at 95% confidence; intermediate values
used in calculations: t = -2.0756, t Critical (two-tail) = 2.0040, df = 55
*** represent statistically significant differences at 99% confidence; intermediate values used in
calculations: t = 3.1968, t Critical (two-tail) = 2.6682, df = 55
The findings also indicate that contrary to the relative parody recorded between the two
exams in the male population, female students demonstrated statistically significant increases in
performance at a much greater rate in the experimental group (Dynamic). (Table 33 on page 66
provides a full description of difference in scores recorded by female students in each research
group) During the second iteration, the experimental group recorded highly statistically
significant increases in score on every essay question, confirmed through T-tests. In contrast, the
average scores in the control group (Static) mirrored changes observed in the male population
and remained statistically consistent.
66

Table 33 - Difference in Average Scores for Female Student Between Study Iterations
Question # 1
st
Test 2
nd
Test Change P-Score
Control
Group
(Static)
(N=31)
Essay #1 1.48 1.03 -0.45 0.25
Essay #2 0.71 1.29 +0.58 0.10
Essay #3 0.32 0.97 +0.65 0.04**
Essay #4 0.39 1.03 +0.64 0.07*

Experimental
Group
(Dynamic)
(N=26)
Essay #1 0.31 0.88 +0.57 0.02**
Essay #2 0.15 1.42 +1.27 0.0004***
Essay #3 0.12 0.77 +0.65 0.02**
Essay #4 0 1.38 +1.38 N/A
Note: Control Group: * represents a statistically significant difference at 90% confidence;
intermediate values used in calculations: t = -1.8242, t Critical (two-tail) = 1.6706, df = 60
** represents a statistically significant difference at 95% confidence; intermediate values used in
calculations: t = -2.0769, t Critical (two-tail) = 2.0003, df = 60
Experimental Group: ** represent statistically significant differences at 95% confidence;
intermediate values used in calculations for essay #1: t = -2.4450, t Critical (two-tail) = 2.0086,
df = 50; intermediate values used in calculations for essay #3: t = -2.3749,
t Critical (two-tail) = 2.0086, df = 50
*** represents a statistically significant difference at 99% confidence; intermediate values used
in calculations: t = -3.7184, t Critical (two-tail) = 2.6778, df = 50
Observations on female performance on the exam tool’s essay questions by count also
reinforce the alternate hypothesis (Table 34 on page 67). The control group (Static) recorded
16/35 (46%) responses that matched the criteria for higher level cognitive function during the
first iteration. The experimental group (Dynamic) failed to provide a single response that the
study measured above basic understanding of the topic. Despite significant improvement in the
experimental population during the second iteration, they still only provided 20/53 (38%)
responses that met higher level parameters. The control group received credit for 28/55 (51%) of
their responses.

67

Table 34 - Female Student Performance By Count Reflecting Bloom’s Hierarchy of Cognitive
Thought
Question
Number
Control Group (Static) Females Experimental Group (Dynamic)
Females
Recall Understanding Analyze Evaluate Create Recall Understanding Analyze Evaluate Create
1
st
Test
Essay #1
1 6 0 7 1 2 3 0 0 0
Essay #2
6 2 1 1 1 0 2 0 0 0
Essay #3
1 1 1 1 0 1 1 0 0 0
Essay #4
2 0 2 1 0 0 0 0 0 0
Total
10 9 4 10 2 3 6 0 0 0
2
nd
Test
Essay #1
11 4 0 2 1 12 2 1 1 0
Essay #2
5 1 7 3 0 6 2 2 4 1
Essay #3
2 1 6 2 0 6 1 1 1 1
Essay #4
1 2 3 2 2 4 0 4 0 4
Total
19 8 16 9 3 28 5 8 6 6

The study also identified several significant findings through isolation of the age variable.
Despite having the largest sub-population group in the study, students in the experimental group
(Dynamic) for the 14 to 15-year-old age category produced the lowest critical thought scores in
the study on the first exam. During the first iteration, they only provided responses to the first
two essay questions. The control group (Static) outperformed them on the first essay question by
an average score of 1.44 to 0.26, and the second essay question by a score of 0.67 to 0.15.
Through T-tests, the study confirmed the statistical significance of the difference in mean scores,
producing P-scores of 0.0007 and 0.03 respectively.
However, as depicted in Table 35 and Table 36 on page 68, the experimental group
(Dynamic) demonstrated extremely statistically significant improvement between the two
iterations. Although none of the average differences in scores between the research groups met
the threshold for significance in the second iteration, the experimental group outperformed the
control group (Static) in three of the four questions by relatively large margins; 1.56 to 0.96 on
question 2; 1.31 to 0.41 on question 3; and 1.70 to 0.52 on question 4. T-test ran against scores
68

for essay questions 1 and 2 both returned P-scores of 0.0001 confirming the significance of the
change observed in performance between the two iterations of the study.
Table 35 - Performance of Students Age 14-15 on the Exam Tool Essay Questions
Question
Number
Control Group (Static)
Age 14-15 (N=27)
Experimental Group (Dynamic)
Age 14-15 (N=34)

P
Min Max Mean SD Min Max Mean SD Score
1
st
Test
Essay #1 0 5 1.44 1.81 0 2 0.26 0.61 0.0009***
Essay #2 0 5 0.67 1.22 0 3 0.15 0.60 0.04**
Essay #3 0 2 0.11 0.42 0 0 0 0 N/A
Essay #4 0 3 0.30 0.81 0 0 0 0 N/A

2
nd
Test
Essay #1 0 5 1.19 1.52 0 4 1.09 0.75 0.74
Essay #2 0 4 0.96 1.55 0 5 1.56 1.75 0.17
Essay #3 0 4 0.41 1.55 0 5 0.85 1.31 0.23
Essay #4 0 4 0.52 1.31 0 5 1.00 1.70 0.23
Note: ** represents a statistically significant difference at 95% confidence; intermediate values
used in calculations: t = 2.1417, t Critical (two-tail) = 2.0010, df = 59
*** represents a statistically significant difference at 99% confidence; intermediate values used
in calculations: t = 3.4930, t Critical (two-tail) = 2.6618, df = 59
Table 36 - Difference in Average Scores Between Study Iterations for Students Age 14-15
Question # 1
st
Test 2
nd
Test Change P-Score
Control
Group
(Static)
(N=27)
Essay #1 1.44 1.19 -0.25 0.58
Essay #2 0.67 0.96 +0.29 0.45
Essay #3 0.11 0.41 +0.30 0.33
Essay #4 0.30 0.52 +0.22 0.46

Experimental
Group
(Dynamic)
(N=34)
Essay #1 0.26 1.09 +0.83 0.0001***
Essay #2 0.15 1.56 +1.41 0.0001***
Essay #3 0 0.85 +0.85 N/A
Essay #4 0 1.00 +1.00 N/A
Note: *** represent statistically significant differences at 99% confidence; intermediate values
used in calculations for essay #1: t = -3.0967, t Critical (two-tail) = 2.6524, df = 66; intermediate
values used in calculations for essay #2: t = -4.3785, t Critical (two-tail) = 2.6524, df = 66
Assessment of student responses by count for the 14-15-year-old age category provided
additional insight reflected in Table 37 on page 69. As expected based on the number of
responses during the first iteration of the study, the control group (Static) outperformed the
69

experimental group (Dynamic). On the first exam, the control group recorded 12/28 (43%)
responses meeting the threshold for higher level cognitive function. The experimental group
recorded 1/8 (13%) responses. However, on the second exam, the control group only recorded
13/40 (33%) responses meeting the criteria for higher level thought. The experimental group
provided 29/61 (48%) higher level responses, providing evidence in support of the hypothesis
that the dynamic study tool leads to increased levels of critical thought in students.
Table 37 - Performance of Students Age 14-15 By Count Reflecting Bloom’s Hierarchy of
Cognitive Thought
Question
Number
Control Group (Static)
Age 14-15
Experimental Group (Dynamic)
Age 14-15
Recall Understanding Analyze Evaluate Create Recall Understanding Analyze Evaluate Create
1
st
Test
Essay #1
3 3 0 5 2 3 3 0 0 0
Essay #2
5 1 2 0 1 0 1 1 0 0
Essay #3
1 1 0 0 0 0 0 0 0 0
Essay #4
2 0 2 0 0 0 0 0 0 0
Total
11 5 4 5 3 3 4 1 0 0
2
nd
Test
Essay #1
10 2 0 2 2 10 2 1 5 0
Essay #2
6 3 2 2 0 4 2 5 5 2
Essay #3
2 1 1 1 0 4 5 2 1 1
Essay #4
2 1 2 1 0 5 0 3 0 4
Total
20 7 5 6 2 23 9 11 11 7

The study observed the opposite trend amongst 16 year old students, as reflected in Table
38 on page 70. Students in the experimental group (Dynamic) of the 16 year old population
earned higher mean scores on all four essay questions in the first iteration of the study. In the
second iteration, the control group (Static) earned higher scores on three of the four questions.
Based on the findings reported in Table 39 on page 71, neither group experienced a significant
change in scores between iterations suggesting that the variation observed likely occurred by
chance. When the study implemented T-tests, the resulting P-scores supported the null
70

hypothesis. Based on the recorded scores, the study lacks sufficient evidence to support either
tool offering students in the 16-year-old category a significant advantage over the other.
Table 38 - Performance of Students Age 16 on the Exam Tool Essay Questions
Question
Number
Control Group (Static)
Age 16 (N=13)
Experimental Group (Dynamic)
Age 16 (N=10)
P
Min Max Mean SD Min Max Mean SD Score
1
st
Test
Essay #1 0 5 1.08 1.64 0 5 1.10 1.76 0.98
Essay #2 0 4 1.00 1.41 0 5 1.20 1.89 0.77
Essay #3 0 4 0.77 1.42 0 5 1.60 1.96 0.25
Essay #4 0 4 0.54 1.28 0 3 0.90 1.38 0.52

2
nd
Test
Essay #1 0 5 1.00 1.36 0 2 0.80 0.75 0.68
Essay #2 0 4 1.54 1.55 0 4 1.40 1.36 0.82
Essay #3 0 4 1.38 1.55 0 4 1.00 1.18 0.53
Essay #4 0 4 0.77 1.31 0 3 1.20 1.47 0.47

Score assessments by count (Table 40 on page 69) reflect the same variation observed by
studying the mean scores of each research group. In the first iteration, the control group (Static)
earned credit for 9/15 (60%) higher level responses. The experimental group (Dynamic) turned
in 11/15 (73%) responses, earning the highest percentage of responses reflecting higher cognitive
function of any sub-population in the study in either iteration. During the second iteration, the
control group (Static) increased their performance level. They responded with 19/30 (63%)
responses matching the study’s criteria for higher level thought. The experimental group’s
performance decreased as the study recorded 8/23 (35%) upper-level responses from them on the
second exam.

71

Table 39 - Difference in Average Scores Between Study Iterations for Students Age 16
Question # 1
st
Test 2
nd
Test Change P-Score
Control
Group
(Static)
(N=13)
Essay #1 1.08 1.00 -0.08 0.89
Essay #2 1.00 1.54 +0.54 0.36
Essay #3 0.77 1.38 +0.61 0.31
Essay #4 0.54 0.77 +0.23 0.65

Experimental
Group
(Dynamic)
(N=10)
Essay #1 1.10 0.80 -0.30 0.63
Essay #2 1.20 1.40 +0.20 0.79
Essay #3 1.60 1.00 -0.60 0.42
Essay #4 0.90 1.20 +0.30 0.64
Table 40 - Performance of Students Age 16 By Count Reflecting Bloom’s Hierarchy of
Cognitive Thought
Question
Number
Control Group (Static) Age 16 Experimental Group (Dynamic) Age 16
Recall Understanding Analyze Evaluate Create Recall Understanding Analyze Evaluate Create
1
st
Test
Essay #1
1 2 0 1 1 2 0 0 1 1
Essay #2
1 1 2 1 0 0 0 1 1 1
Essay #3
0 0 2 1 0 1 1 1 0 2
Essay #4
0 0 1 1 0 0 0 3 0 0
Total
2 3 5 4 1 3 1 5 2 4
2
nd
Test
Essay #1
4 2 0 0 1 4 2 0 0 0
Essay #2
1 0 5 1 0 4 0 2 1 0
Essay #3
0 1 4 1 0 4 1 0 1 0
Essay #4
1 2 3 4 0 0 0 4 0 0
Total
6 5 12 6 1 12 3 6 2 0

The findings reported in Table 41, and Table 42 (page 72) indicate that neither tool
provided a significant advantage to students in the 17 to 18 age group. Although the
experimental group (Dynamic) outperformed the control group (Static) on two of the three
questions that they responded to during the first iteration of the study, the control group
outperformed the experimental group during the second iteration on all four questions. T-tests
indicate a lack of significant evidence to support a significant difference between the mean
scores of each group in either iteration, supporting the null hypothesis.
72

Table 41 - Performance of Students Age 17-18 on the Exam Tool Essay Questions
Question
Number
Control Group (Static)
Age 17-18 (N=11)
Experimental Group (Dynamic)
Age 17-18 (N=6)

P
Min Max Mean SD Min Max Mean SD Score
1
st
Test
Essay #1 0 4 0.73 1.29 0 2 1.00 0.82 0.65
Essay #2 0 5 1.00 1.91 0 2 0.50 0.76 0.55
Essay #3 0 1 0.09 0.29 0 2 0.33 0.75 0.35
Essay #4 0 0 0 0 0 0 0 0 N/A

2
nd
Test
Essay #1 0 5 1.82 1.90 0 4 1.17 1.34 0.47
Essay #2 0 4 2.27 1.60 0 5 2.00 2.08 0.77
Essay #3 0 4 1.55 1.44 0 2 0.67 0.75 0.19
Essay #4 0 5 1.91 2.15 0 5 1.67 1.80 0.82
Table 42 - Difference in Average Scores Between Study Iterations for Students Age 17-18
Question # 1
st
Test 2
nd
Test Change P-Score
Control
Group
(Static)
(N=11)
Essay #1 0.73 1.82 +1.09 0.13
Essay #2 1.00 2.27 +1.27 0.11
Essay #3 0.09 1.55 +1.46 0.005***
Essay #4 0 1.91 +1.91 N/A

Experimental
Group
(Dynamic)
(N=6)
Essay #1 1.00 1.17 +0.17 0.80
Essay #2 0.50 2.00 +1.50 0.13
Essay #3 0.33 0.67 +0.34 0.45
Essay #4 0 1.67 +1.67 N/A
Note: *** represents a statistically significant difference at 99% confidence; intermediate values
used in calculations: t = -3.1379, t Critical (two-tail) = 2.8453, df = 20
Unlike the findings based on mean scores, the study found that the assessment by count
indicated support for the alternate hypothesis. Although the experimental group (Dynamic)
earned higher mean scores in the first iteration, they failed to earn a single response registering
higher level thought while the control group (Static) earned 3/7 (42%) responses that
demonstrated advance cognitive function. The control group outperformed the experimental
group in the second iteration as well by recording 19/27 (70%) higher level responses. The
experimental population achieved 6/14 (43%) responses on the second exam that demonstrated
73

higher cognitive function. See table 43 below for a full description of student scores by count in
the 17-18 age group.
Table 43 - Performance of Students Age 17-18 By Count Reflecting Bloom’s Hierarchy of
Cognitive Thought
Question
Number
Control Group (Static)
Age 17-18
Experimental Group (Dynamic)
Age 17-18
Recall Understanding Analyze Evaluate Create Recall Understanding Analyze Evaluate Create
1
st
Test
Essay #1
0 2 0 1 0 2 2 0 0 0
Essay #2
1 0 0 0 2 1 1 0 0 0
Essay #3
1 0 0 0 0 0 1 0 0 0
Essay #4
0 0 0 0 0 0 0 0 0 0
Total
2 2 0 1 2 3 4 0 0 0
2
nd
Test
Essay #1
2 2 0 1 2 3 0 0 1 0
Essay #2
1 0 4 3 0 0 0 1 1 1
Essay #3
2 1 3 1 0 2 1 0 0 0
Essay #4
0 0 1 2 2 2 0 1 0 1
Total
5 3 8 7 4 7 1 2 2 2

4.3.2. Participation and Engagement
Both groups actively participated in the study. Table 44 on page 74 describes the level of
participation for each group during the first iteration of the study, as recorded by the classroom
teacher. The experimental group (Dynamic) received credit for full participation (100%) based
on their actions while the control group (Static) earned credit for partial participation (90%).
Further description of student participation is captured in qualitative findings in the following
sub-section of chapter 4.
The study observed low student engagement with the tool outside of the classroom in
both groups. Contrary to the hypothesis that the dynamic tool would generate student interest and
lead to additional engagement with the tool itself, the study found evidence that the students in
the control group (Static) accessed the study tool at a higher rate, and more frequently during the
74

two-week self-study period. The experimental group (Dynamic) students did engage with the
tool for longer periods of time when they accessed it. However, due to the frequency with which
the control group returned to the tool, they spent more time with their tool than the experimental
group students did between iterations.
Table 44 - Student Class Participation and Engagement with the Research Tool
Study Group Class
Participation
Individual
Access
Mean Access
Frequency
Mean Access
Hours
Access
Method
Control
(Static)
90% 20% 3.73 1.27 10 x Packet
Experimental
(Dynamic)
100% 14% 2.14 1.43
6 x Computer
1 x Phone

In both groups, instruction to review the study tool played the primary factor that led
students to study the materials between iterations. As reflected in Table 45 on page 75, only 6%
of students in the control group (Static), and 4% of students in the experimental group (Dynamic)
reported interest in the topic or tool itself as their primary reason for study. In fact, 66% of
students in the experimental group actually indicated a complete lack of interest as the primary
reason that they declined to participate in the two-week self-study period. The study found that a
majority of the control group students neglected to participate based on a lack of time due to
other course work requirements, and extra-curricular activities such as after-school sports and
clubs.

75

Table 45 - Student Motivation for Level of Engagement with the Research Tool During the Self
Study Period
Control Group (Static) Experimental Group (Dynamic)
Motivation Percentage Motivation Percentage
Map/Topic Interest 6% Map/Topic Interest 4%
Easy Access 2% Easy Access 4%
Instructed to Study 12% Instructed to Study 6%
No Time 41% No Time 20%
No Interest 39% No Interest 66%

The study found that students in both groups who accessed the study materials witnessed
an increase in their critical thought scores (Table 46 below). However, as previously discussed,
most students increased their score between iterations. The study lacks sufficient data to identify
access to the tool as the primary reason for this change.
Table 46 - Correlation Between Class Participation and Student Engagement with the Study
Tool, and Student Performance in Critical Thought
ID# Participation
in Class
Motivation
for
Engagement
Response to
#11
(1
st
/2
nd
Test)
1
st
Test
Score
(Essays)
2
nd
Test
Score
(Essays)
Change
Control
13589 Yes Topic Interest T/T 9 (4/5/0/0) 16 (4/3/4/5) + 5
13591 Yes Topic Interest T/T 0 1 (0/1/0/0) + 1
13593 Yes Instructed to T/T 0 4 (0/4/0/0) + 4
13605 Yes Instructed to T/T 0 10 (1/3/2/4) + 10
13611 Yes Easy Access T/T 0 9 (2/3/1/3) + 9
13765 Yes Instructed to T/T 4 (1/3/0/0) 4 (1/2/1/0) --
13771 Yes Topic Interest T/F 1 (1/0/0/0) 0 -1
13781 Yes Instructed to T/T 2 (2/0/0/0) 8 (0/3/3/2) + 8
13785 Yes Instructed to T/T 7 (2/1/1/3) 7 (1/3/0/3) --
13805 Yes Instructed to F/T 4 (4/0/0/0) 1 (1/0/0/0) - 3

Experimental

13627 Yes Easy Access T/T 0 2 (1/1/0/0) + 2
13631 Yes Map Interest T/T 0 0 --
13639 Yes Topic Interest T/T 0 10 (4/3/2/1) + 10
13659 Yes Instructed to T/T 0 16 (2/4/5/5) + 16
13661 Yes Instructed to T/T 0 1 (1/0/0/0) + 1
13667 Yes Instructed to T/T 0 2 (1/0/0/1) + 2
13689 Yes Easy Access T/T 0 14 (4/3/2/5) + 14
76

4.4 Qualitative Research Observations
4.4.1. Student Engagement
The study observed increased levels of distraction in the control group (Static), not
present in the iterations conducted with the experimental group (Dynamic). Although most
students remained on task, approximately 10 to 15 minutes into the class period, individuals
began losing focus due to frustration incurred while mining for data in the packet to answer a
question on the exam tool. As a result, students in the control group exhibited a higher likelihood
to work in pairs, or small groups at times as they turned to other students at their table to help
them make progress. This behavior was completely absent from the experimental group.
Students using the dynamic tool remained completely engaged and focused on the computer in
front of them for the duration of the experiment. Over the course of the three class periods that
comprised the experimental population, students remained on task and did not engage with other
students.
4.4.2. Student Interaction with the Study Tool
Due to their familiarity with the format, students from the control group (Static)
experienced little difficulty working with their static tool set. As supported by their efficiency
scores and essay attempts from the first iteration of the study, the students worked through the
packet at a much faster pace than their peers in the experimental population (Dynamic). They
also demonstrated a greater likelihood to create and implement measurement tools to use with
their maps for the purpose of judging distance on the spatial analysis questions. Whereas most
students from the experimental group judged distance primarily by estimation, students in the
control group frequently made tick marks on pieces of scratch paper or reached for a ruler.
77

Students in the experimental group (Dynamic) exhibited far more interest in the tool
itself, possibly to the point of distraction. The study had to remind the experimental group not to
linger on questions they couldn’t answer, and to continue to make forward progress through the
exam tool. As previously noted, they worked completely silently throughout the period.
Although the study presented the same material in both tools in the same order, students using
the story map went through the data in a less linear manner than those using the paper tool. They
tended to move freely between tabs and manipulate the maps and pop outs according to the
question they were trying to answer from the exam tool. The study also observed greater focus
on the maps themselves in the experimental group, whereas the control group (Static) focused
more heavily on the narrative.
Although the study threw out the student’s responses when developing the mean score for
the experimental group (Dynamic), it recorded an instance of web browser use to search for
additional information on the topic. The study purposely excluded hyperlinks from the dynamic
tool in order to maintain parody with the static tool. However, they provide a powerful resource
to teachers in the classroom. The ability to link sources directly into the lesson content increases
the likelihood that students will follow them. Although students from the control group (Static)
could have used their phone at any point to conduct similar searches, the study did not record a
single instance.
4.4.3. Financial Considerations
The study incurred approximately $500.00 in costs to develop the static tool for the
control group (Static) students. Because the maps required color, ink for printing accounted for
the majority of the cost. The study opted to use a home printer to create the static tools to save
money on printing. Had the booklets been prepared professionally, it would have costs $550.00
78

without binding materials. The study produced 60 packets for the control group, enough for each
student plus 10% to have extras on hand in the classroom in case students lost their materials
during the two-week self-study period.
The study incurred no costs to produce and implement the static tool. School computers
provided an adequate medium to explore the story map, so the study required no additional
hardware to implement the dynamic tool set. The school also provided access to wireless internet
in the classroom alleviating concerns over cloud hosting. Alternatively, most students in the
classroom owned smart phones that they could have used to manipulate the dynamic tool had the
computers been unavailable or access to the internet compromised.

79

Chapter 5 : Discussion
This study conducted a classroom based, empirical comparison of standard static visualization
tools and emerging dynamic Esri story map visualizations. The study focused on answering three
research questions intended to demonstrate whether dynamic story maps deserved additional
consideration as a viable teaching aid in the classroom, and to add to the existing body of
visualization literature: Are their measurable increases in student performance in knowledge
extraction tasks; Do story maps increase the likelihood of transitioning ideas from short-term to
long-term memory; and do story maps facilitate critical thought through active participation with
the data as it is presented? Although the study hypothesized that the dynamic tool would
outperform the static tool in all three research objectives, the study design implemented a two-
tailed T-test based on the null hypothesis that neither tool offered students a distinct advantage.
The following chapter provides a discussion of the findings from the classroom based
experiment in section 5.1, broken down further by research question in sections 5.1.1 through
5.1.3, and a general discussion of qualitative findings in section 5.1.4. Section 5.2 discusses the
overall strengths and weaknesses of the research design. Implications for future research are
highlighted in section 5.3. The chapter concludes with a summary of conclusions from the
research in section 5.4.
5.1 Discussion of Findings
5.1.1. Student Performance on Knowledge Extraction Tasks
The study found that when looking at the full population of students within each research
group, student performance in how effectively they answered the questions on the exam tool
supported the null hypothesis. Although the control group (Static) earned higher mean scores in
three of the four question formats, relatively high P-scores in all but the Fill-in-the-blank format
80

suggest that the differences between the groups are more likely a result of chance. The same
tendency repeated itself when the study focused solely on the male population. Although male
students in the experimental group (Dynamic) outperformed their peers in the control group in all
four formats, the differences between the scores were largely insignificant. They could have
easily been reported in reverse in another random sampling as confirmed by high P-scores across
the board. However, isolation of the female student population revealed statistically significant
findings in favor of the alternate hypothesis.
Although the difference in scores on the fill-in-the-blank format only reflected a 0.97
point difference between the control group (Static) and the experimental group (Dynamic), the
statistical significance of the finding was confirmed with a P-score of 0.03 at 95 percent
confidence. In fact, female performance in knowledge extraction favored the static tool so
heavily, that their scores on the Fill-in-the-blank questions pulled the general population P-score
to 0.14. The male population observed a P-score of 0.80 on the same questions.
Student efficiency scores in knowledge extraction further support these findings. On
average, students in the control group (Static) answered an additional two questions (1.75) more
than their peers in the experimental group (Dynamic) during the first study iteration. The T-test
confirmed the statistical significance of the finding at 90 percent with a P score equal to 0.06.
However, looking at the efficiency score for the full population does not provide a completely
accurate account. Approaching efficiency by gender explains why the general population scores
deserve scrutiny.
Male students in the experimental group (Dynamic) earned a slightly higher mean
efficiency score than the control group (Static), 14.91 to 14.85. It was the female students in the
control group answering an additional three questions more than the experimental group that
81

caused the general population mean to appear significant. The T-test confirmed the statistical
difference in efficiency scores between females in the control group and females in the
experimental group, producing a P-score of 0.01 at 95 percent confidence.
Based on the study’s findings, both tools produced similar results in the general
population. Analysis by age failed to return any statistically significant findings, but isolation of
the gender variable revealed that female students tended to perform knowledge extraction task
more effectively and efficiently when using static products. Interestingly, the study’s findings
showed the opposite for male students. Based on scores from this study, males performed
knowledge extraction more effectively and efficiently using the dynamic story map.
Unfortunately, their scores did not reveal statistically significant findings indicating the need for
further research to ascertain whether the trend holds or was merely a result of chance.
5.1.2. Impact on Transition to Long Term Memory
The study demonstrated that neither tool led to statistically significant increases in
transitioning ideas from short-term to long-term memory. However, students in the experimental
group (Dynamic) did demonstrate slightly lower scores in three of the four formats (including
spatial analysis), indicating slightly higher retention of concepts between the two study
iterations. Also worth noting, female students in the experimental group demonstrated greater
transition to long-term memory in all four formats. Whereas female students struggled to extract
information using the dynamic tool during the first iteration of the study, initial findings indicate
that their increased interaction with the data did lead to greater retention of the information that
they learned.
The study’s findings related to student efficiency scores on knowledge retention tasks can
be interpreted in two ways. The first trend which immediately pops out when viewing student
82

completion times reflects the amount of effort each group put into answering questions on the
research tool during the second iteration of the study. With the exception of students in the age
16, and age 17 to 18 populations, the experimental group (Dynamic) spent more time responding
to questions. In fact, students in the experimental group, age 14 to 15 population, used an
additional 5.38 minutes on the exam than their peers in the age 14 to 15 control group
population. T-test further confirmed the very statistically significant value of the recorded
discrepancy, producing a P-score of 0.0026 at 95 percent confidence.
The immediate impulse when viewing such a large discrepancy is to conclude that the
experimental group (Dynamic) took the second exam more seriously. However, when measured
against their effectiveness scores, this conclusion fails. Not only did students in the age 14 to 15
control group (Static) retain essentially the same amount of information between the two study
iterations as their peers in the experimental group, but they recalled the information using far less
time indicating a far stronger connection with the study material. The same argument holds for
the general population of each group. T-test demonstrated a statistically significant difference
between the two groups at 90 percent confidence (P = 0.07). However, older students
demonstrated the opposite tendency. Their mean effectiveness and efficiency scores reflect a
stronger tendency toward confirming the study’s hypothesis. However, the extremely small
population size of the 17 to 18-year-old students in the experimental group failed to influence the
findings in the general population.
5.1.3. Implications for Increasing Critical Thought in Students
The lack of responses attempted on the exam tool’s essay questions during the first
iteration speaks more to the efficiency with which each group employed their designated study
tool for knowledge extraction tasks than anything else. Many of the additional questions that the
83

control group (Static) received credit for attempting in the first iteration came from the essay
portion of the exam, reflected in their 19 percent higher response rate over the experimental
group (Dynamic). By the second iteration, both groups increased their percentages and
responded to the essay questions at about the same rate. The initial disparity in attempts drove
down mean scores in each group, but it particularly affected the outputs from the experimental
group on the mean critical thought scores. This explains why during the first iteration, the study
recorded a statistically significant difference between the control group’s responses to the first
essay question, and the experimental group’s, but failed to repeat the same finding during the
second iteration of the study. It also accounts for the extremely statistically significant findings
the study captured related to the increase in scores between iterations for both groups.
Despite the disparity of attempts between the two iterations identified in both groups, the
critical thought findings remained consistent when adjusted for the count. For example, when
reviewing responses by count, the control group nearly doubled their attempts, from 50 in the
first iteration to 91 in the second iteration. However, their higher level thought scores remained
fairly consistent at 50 percent and 51 percent respectively. Similarly, the experimental group
more than tripled their initial attempts at the essay responses in the second iteration going from
30 to 96, but despite the dramatic increase in response levels, their scores too remained
consistent at 40 percent and 45 percent higher level responses respectively. In both instances,
students in the control group recorded a higher percentage of increased levels of critical thought.
The consistency in scores recorded for each group, when only graded attempts were taken
into account, seems to confirm that the increase in performance demonstrated by the control
group (Static) and the experimental group (Dynamic) reflects an increase in attempts. It does not
indicate an increased average score as the raw numbers indicate. Therefore, it would be
84

inaccurate to conclude that students from either group improved their performance significantly
between iterations based on higher mean scores achieved by each group.
Again, as with the previous research questions, the study identified gender as a critical
factor influencing the way that students interacted with the study tools to demonstrate higher
orders of thought. Male students actually demonstrated equivalent levels of critical thought using
both tools. By mean scores, the males failed to produce any statistically significant differences in
scores between the tools that would indicate a preference in learning styles. The study confirmed
these findings by looking at the males’ scores by count. During the first iteration, the control
group only outperformed the experimental group by 3 percent. During the second iteration, the
experimental group reversed the results and outperformed the control group by 1 percent.
However, the female population reflects a much different result.
Female students exhibit a strong inclination toward static visualization tools. Beyond the
fact that female students in the control group (Static) outperformed the experimental group
(Dynamic) by statistically significant margins on both of the essay questions the experimental
group attempted during the first iteration, they also reflected significantly greater scores by
count. Using the static tools, female students in the control group consistently demonstrated
higher level thought in approximately 50 percent of their responses. The experimental group
barely approached 40 percent on the second iteration and failed to record a single higher level
response while working directly with the story map.
The large disparity reaffirms the study’s earlier findings that the female students in the
experimental group experienced difficulty with knowledge extraction tasks. Had it simply been a
matter of lower critical thought performance, the study should have recorded a higher level
thought percentage closer to 40 percent in the first iteration like they achieved during the second
85

iteration without the dynamic tool. At a minimum, the study should have recorded at least one
attempt out of those submitted that earned credit for a higher level response. The fact that the
female student population from the experimental group was the only population in the study to
fail to record a single higher level response during the first iteration of the study speaks volumes.
Rather than reflecting a lack of critical thought, which their performance during the second
iteration clearly demonstrated they possessed, their scores from the first iteration are more
indicative of a hurried response at the last minute as time ran out in the class period.
Age also appeared to play a factor in how the research groups responded to the different
tools. Students in the experimental group’s 14 to 15 age population reflected similar challenges
as the females in working with the dynamic tool set. Their minimal attempts on the essay
questions during the first iteration are clearly indicative of challenges with using the study’s
story map tool. Students age 14 to 15 in the experimental group responded at 29 percent of the
rate (8/28) as their peers in the control group during the first iteration. However, during the
second iteration, the same population outperformed their peers in the control group in both
number of responses (61 to 40) and quality of responses demonstrating higher level thought (48
percent to 33 percent). In fact, the students in the age 14 to 15 population of the experimental
group are the only population in the study’s experimental group to demonstrate significantly
greater levels of critical thought than their peers in the control group during the second iteration
of the study.
Although the study recorded full participation from students in the experimental group
(Dynamic), only seven of the students in the group actually accessed the material on their own
during their two week self-study period. The control group (Static) reported ten students who
took advantage of the opportunity. The lack of self-motivated study in both groups indicated that
86

neither tool inspired significant excitement in the students, but perhaps more telling were the
student responses as to why they ignored the tool in between iterations, a lack of interest and
time. It is fair to assume that neither group reflected on the study material between iterations,
making the increase in performance level witnessed in the female and age 14 to 15 populations in
the experimental group that much more significant. To answer the essay questions during the
second iteration, they needed to recall the information they learned while using the story map.
This would further indicate that they knew the answer to the questions, but ran out of time to
formulate coherent thoughts during the first iteration.
5.1.4. General Observations
In chapter 2, the study identified that visualization methods should be selected for their
ability to communicate to the user in the most effective and efficient manner. While it would
appear, based on the findings articulated in chapter 4, that the students in the control group
performed knowledge extraction tasks more efficiently than those in the experimental group,
what if the study looked at efficiency through a different lens? What if the study employed
financial burden on the school or classroom teacher as the measurement for efficiency?
When developing the research design, the study took financial costs associated with
developing static tools for granted. However, the costs incurred to develop the static tools
deserve serious consideration, and may be a previously unidentified justification for integrating
dynamic GIS tools into the classroom for social sciences. The study spent $500.00 on lesson
materials for less than half of the total student population belonging to the classroom teacher. To
develop materials for the entire student population, the costs would likely double. For one lesson,
not included in the standard curriculum textbook, the materials could wind up costing the school
87

or teacher well over $1,000.00 in materials to produce. In contrast, the story map costs absolutely
nothing to produce.
Another advantage to using story maps in the classroom is that the lessons can be
repeated indefinitely. Unlike custom static tools which must either be retrieved following the
lesson, or re-produced (incurring additional financial burden) for future classes, dynamic tools
enable students to carry the lesson with them without causing undo financial hardship. If they
lose the link to the data, the educator simply re-provides it, and the problem resolves itself. Story
maps provide educators with a cost effective means of professionally developing new content to
enhance their existing curriculum that would have previously been unavailable.
The study’s findings also support a similar discussion regarding the effectiveness of story
maps in expanding new lesson materials. Although the findings don’t support the original
hypothesis that story maps result in greater performance in knowledge extraction, improved
transition of ideas to long-term memory, or enhanced critical thought, overwhelming support for
the null hypothesis does indicate that they do not detract from any of those objectives either in
most instances. In some regard, the study stands as proof of the concept that story maps can
introduce new ideas in the classroom just as effectively as traditional methods. However, the cost
of producing static tools makes expanding lesson content impractical, particularly when using
color ink. Therefore, the study concludes that in terms of developing new lesson content to
enhance current curriculums, the story map outperforms static tools in terms of efficiency (cost)
and effectiveness (practicality of implementation).

88

5.2 Assessment of Study Design Strengths and Weaknesses
5.2.1. Strengths
Tversky, Morrison, and Betrancourt identified the primary weakness existing in most
visualization research where dynamic tools outperform static variants as a lack of parody in the
research tools. (2002) This study made it a point to isolate the methods of delivering information
to the students in the research groups by ensuring that both tools reflected the same information.
If anything, this study swung the pendulum too far toward parody, opting to remove several
features from the dynamic tool that would have increased student performance in the
experimental group (Dynamic). Specifically, the study chose a story map format that prevented
students from interactively layering data through the use of the swipe tool, or spyglass
techniques. It also refrained from introducing interactive layers to depict a line of sight analysis
and proximity, as neither would be available to the students using static tools in the control
group. Instead, the study included the static maps depicting both sets of analysis in the dynamic
tool as pop-ups that the students in the experimental group could control as required to answer
associated questions in the exam tool. Ultimately, the dynamic tool reflected the same
information as the static tool, with no additional details or opportunities for interactive
engagement that would skew the findings.
Along the same lines, selecting the Bascom Affair for the study proved to be a positive
decision. Students understood enough about United States history to place the event in the
context of western expansion. However, no one had previously learned about the event itself.
The students entered the study without pre-existing knowledge, ensuring that the research tool
provided their only source of knowledge for the study.
Finally, Isolation of gender and age variables in each research group also proved to be
strengths in the research design. The effectiveness of visualization methods varies by age,
89

gender, culture, and other characteristics that shape the way people view and interpret
information. (Slocum et al. 2001) Incorporating two such factors provided the study the
opportunity to explore variance in responses. Isolation of gender in particular produced
statistically significant findings that the study would have missed completely in the general
population.
5.2.2. Weaknesses
In chapter 2, the study argued that animated products could be interpreted as static maps
due to the inability of the user to alter the data beyond how the creator originally intended. The
story map used in this study is vulnerable to the same criticism. Although students in the
experimental group retained the ability to manipulate the web map, the study turned many of the
interactive features off. The consequence of focusing on parody between the tools is a neutered
dynamic tool that fails to accurately portray the tool’s inherent strengths. The study concedes the
importance of parody, hence the decision to enforce it in the research design. However, outside
of a research environment, none of the same constraints apply. At some point, research needs to
demonstrate the full capacity of dynamic tools. Story maps clearly provide educators an
advantage at introducing increased levels of student interactivity within new lesson content.
However, in the name of parody, the study purposely decreased the opportunities available to the
experimental group.
In addition to the techniques discussed under strengths that the study decided to forgo,
one of the major advantages inherent to dynamic visualization methods is the ability to hyperlink
additional resources directly into the lesson content. Shortening the distance between the student
and sources of external information by embedding links directly into the lesson content increases
the likelihood that they will pursue additional research on the topic. In fact, even without
90

hyperlinking external sources, the study still observed a student attempting to pursue additional
research. As previously discussed, the action invalidated the student’s results, but it also supports
the argument that dynamic products increase interaction with the data as it is presented.
Sample size also posed a challenge to the initial research design. Although sufficient
sample sizes participated in the general population for each research group, when the study
isolated age and gender variables the sample sizes decreased significantly. Small sample sizes
posed a particular challenge to students in the 16-year-old and 17 to 18-year-old age populations.
Although the study produced statistically significant findings in each group, the sample size calls
the validity of those findings into question in those populations.
Finally, time posed the most significant challenge to the study. With few exceptions,
students from both research groups failed to respond to all of the questions on the research tool
in both iterations. The study was restricted to a 55 minute class period, but ideally, students
would have as long as they required to respond to each question for the study to develop a true
estimate of the mean time required using each tool. Also, because the study leveraged a single
teaching opportunity, students from the control group enjoyed a distinct advantage. They used a
familiar format to answer the questions on the exam tool. The students in the experimental group
had never used a story map prior to the study, so they had to contend with learning to manipulate
the new technology.
Time on the day of the study was not the only challenge. A large percentage of students
from both groups reflected that their official studies and extra-curricular programs prevented
them from capitalizing on the two week self-study period, which was designed to increase their
exposure to their assigned research tool. Ultimately, the study’s findings are heavily influence by
91

the lack of time each group had to familiarize themselves with the tool. This particularly hurt the
experimental group (Dynamic) because they had to learn to use an unfamiliar format.
5.3 Implications for Future Research
Despite the weaknesses in this study, the research identified several statistically
significant findings related to how female students interact with visualization tools. Future
research should continue to incorporate gender as a variable for consideration in its design. In
particular, although male students failed to produce statistically significant differences with
either tool, their mean scores heavily favored the dynamic tool. Taken in context with the female
population’s demonstrated preference for learning with static tools, this may represent a
significant finding with implications for how each gender interprets spatial visualization
methods. Further research should focus on further developing how each gender interacts with the
different tool sets.
Access to a single campus restricted the availability of respondents. After removing
students from the study that could not participate due to a lack of parental permission or were
disqualified from the findings due to unsanctioned behavior (group work in the control group and
looking up external references in the experimental group), the study maintained a respectable
population size of 101 students. Future research should apply the framework developed in this
study to reach greater populations across a wider demographic spread. In particular, future
research should attempt to enter both public and private campuses across different geographic
regions, all-male and all-female schools, and Schools with a heavy emphasis on Science,
Technology, Engineering, and Math (STEM) to see how the different environments influence the
study’s findings.
92

Future research should also attempt to increase the length of the study. This study
focused on a single class period and lesson plan. However, developing the study to take place
over the length of a semester, or school year, as part of the students’ regular class load will
significantly increase the significance of findings by enabling the researcher to incorporate
multiple lesson plans across a variety of topics. A larger body of work, over an extended period
of time, should reduce the impact that students in the experimental group felt from learning to
use the new technology. Finally, incorporating the research into the student’s coursework will
reduce the level of distraction and force them to focus on the study as part of their regular
instruction.
A longer research design should also strive to incorporate the interactive features that
story maps support, and use multiple formats of the application to provide a more holistic view
of the technology’s capabilities in the classroom. Over the course of a semester, or school year,
maintaining parody between static and dynamic tools will likely be impractical. However, full
implementation of both tools in the classroom environment in an unconstrained manner will
provide a clearer picture of the true strengths and weaknesses of each tool.
5.4 Conclusions
In 2013, Lunen and Travis argued that future research needed to focus on determining
WHY historians should embrace GIS by demonstrating concrete examples of gains that the field
would incur through incorporating the technology. (Lunen and Travis 2013) In the same year,
Kerski, Demirci, and Milson recommended establishing a base of research for the same purpose
from a secondary school perspective, to empirically demonstrate WHY GIS makes a difference in
education. Rather than continuing to focus on HOW to implement the technology in the
93

classroom, the researchers recommended pursuing data that shows educators how GIS
technology is an improvement over their current teaching strategies.
Lisner’s findings in 2008 validated the shift in focus from demonstrating new teaching
methods, to identifying evidence of improvement in student performance that results from
adopting the new technology. As her research identified, the teachers who had already made the
transition did so overwhelmingly because they felt that GIS offered their students an increased
advantage over traditional classroom tools. (Lisner 2008) To justify the personal risk that
classroom teachers assume when transitioning to new technologies, research needs to
demonstrate that GIS implementation results in quantitative increases in student performance.
This study intended to demonstrate why GIS integration into secondary school social
science curriculums benefits students in the classroom. Although it failed to disprove the null
hypothesis, the study produced several findings of note to high school social science teachers.
First, it confirmed the findings from previous visualization research indicating that students
perform at the same level or better when using static visualization aids as opposed to dynamic
products. Despite the fact that the study employed a simplified dynamic tool in the form of a
story map, the new format still posed a challenge to students performing knowledge extraction
tasks. The study produced findings of particular interest to teachers working with freshmen and
sophomore populations, and those on single-gender campuses Females and younger students
(age 14 to 15) in the study exhibited the most difficulty with the dynamic tool set, producing
statistically significant findings that favored the use of static products in the classroom.
However, the study also identified insights that indicate student performance with the
dynamic tools may improve as familiarity increases. Poor performances from the experimental
group on the critical thought questions reflected a lack of time for thought, rather than a lack of
94

knowledge learned. Without studying between iterations, the female and younger student (14 to
15) populations significantly improved their performance on the second iteration. Female
students in the experimental group also demonstrated greater success at retaining information
between the iterations.
Finally, the study produced unexpected qualitative findings that support the viability of
using story maps to improve the depth of curriculums in secondary school social science
classrooms. Financially, schools and classroom teachers can’t afford to create new content
regularly with standard static products. Production costs are often prohibitive, reducing the
likelihood of implementing new content. Story maps provide an effective teaching aid that
matches the quality of current classroom tools at a price that teachers can realistically hope to
employ in their classrooms.

95

References

GIS in Education Sources:

Artvinli, Eyüp. 2010. The contribution of geographic information systems (GIS) to
geography education and secondary school students' attitudes related to GIS. Kuram ve
Uygulamada Egitim Bilimleri 10, (3): 1277

Bednarz, Sarah W., and Gail Ludwig. 1997. Ten things higher education needs to know about
GIS in primary and secondary education. Transactions in GIS 2, (2): 123-133

Chalmers, L. 2002. Developments in getting GIS technologies into classrooms. Geographical
Education 15, (2002): 22-27

Demirci, Ali, Ahmet Karaburun, and Mehmet Ünlü. 2013. Implementation and effectiveness of
GIS-based projects in secondary schools. The Journal of Geography 112, (5): 214

Kerski, Joseph. 2003. The implementation and effectiveness of geographic information
systems technology and methods in secondary education. Journal of Geography 102,
(3): 128-137

Kerski, Joseph J., Ali Demirci, and Andrew J. Milson. 2013. The global landscape of GIS in
secondary education. The Journal of Geography 112, (6): 232

Kerski, Joseph J., Andrew J. Milson, and Ali Demirci. 2012. International Perspectives on
Teaching and Learning with GIS in Secondary Schools. Dordrecht: Springer
Netherlands

Knowles, Anne. 2014. The contested nature of historical GIS. International Journal of
Geographical Information Science 28, (1): 206-211

Liu, Yan, Elisabeth Bui, Chew-Hung Chang, and Hans Lossman. 2010. PBL-GIS in
secondary geography education: Does it result in higher-order learning outcomes?
Journal of Geography 109, (4): 150-158

Lisner, Aryliss Lee. 2008. Characteristic beliefs of K--12 teachers that influence their
decision to implement GIS into classroom practice. Ph.D. diss., ProQuest Dissertations
Publishing

Lünen, Alexander von, 1971, and Charles Travis 1964. 2013; 2012. History and GIS:
Epistemologies, considerations and reflections. 1. Aufl.; 1 ed. Dordrecht ;New York:
Springer

96

Mares, Detlev, and Wolfgang Moschek. 2013. "Place in time: GIS and the spatial imagination in
teaching history." History and GIS, CH 5: 59 - 72. Dordrecht ;New York: Springer

Richardson, Doug and Michael Solem. 2014. “Supporting the Esri-ConnectED Initiative:
Crossing Borders.” ArcNews, Fall 2014. Accessed October 11, 2015. Online at:
http://www.esri.com/esrinews/arcnews/fall14articles/supportingtheesriconnectedinitiative

Visualization & Cartography Sources:

Anderson, Benjamin Green. 2015. Cartographic Approaches to the Visual Exploration of Violent
Crime Patterns in Space and Time: A User Performance Based Comparison of Methods.
MS. Thesis., University of Southern California.

Baldwin, Shawn M. 2014. Institutional Inscription in the Minorcan Quarter of Saint
Augustine, Florida. M.S. Thesis, University of Southern California.

Betrancourt, Mireille, Barbara Tversky, and Julie Bauer Morrison. 2002.
“Animation: Can it Facilitate?” International Journal of Human-Computer Studies. 57:
247-262.

Blaser, Andreas D., Monika Sester, and Max J. Egenhofer. 2000. “Visualization in an early
state of the problem-solving process in GIS” Compters & Geosciences. 26: 57-66.

Dodge, Martin, Mary McDerby, and Martin Turner. 2008. “The Power of Geographical
Visualizations.” In Geographic Visualization: Concepts, Tools and Applications, edited
by Martin Dodge, Mary McDerby, and Martin Turner, 1–10. Hoboken, NJ: John Wiley &
Sons, Ltd.

Hoffler, Tim N., and Detlev Leutner. 2007. “Instructional animation versus static pictures:
A meta-analysis.” Learning and Instruction. 17: 722-738.

Heuer, Richards .J. 1999. Psychology of Intelligence Analysis. Washington, D.C., Center
for the Study of Intelligence. On-line at http://www.scip.org/files/Resources/Heuer ‐
Psychology ‐of ‐Intelligence ‐Analysis.pdf

Irina Fabrikant, Sara, Stacy Rebich-Hespanha, Natalia Andrienko, Gennady
Andrienko, and Daniel R. Montello. 2008. “Novel method to measure inference
affordance in static small-multiple map displays representing dynamic processes.”
Cartographic Journal 45 (3): 201-15.

Lowe, R.K. 2003. “Animation and Learning: selective processing of information in
dynamic graphics.” Learning and Instruction. 13: 157-176.

97

Slocum, Terry A., Connie Block, Bin Jiang, Alexandra Koussoulakou, Daniel R. Montello,
Sven Fuhrmann, and Nicholas R. Hedley. 2001. “Cognitive and Usability Issues in
Geovisualization.” Cartography and Geographic Information Science. Vol. 28 No. 1: 61-
75.

98

BIBLIOGRAPHY

Alonso, G. (2015) Apacheria. online at: https://apacheria.es/

Digimap for schools: Scalable web mapping without the jargon: Richard spooner reflects on this
innovative online mapping service for teachers and pupils in primary and secondary
education, and discovers that even time-served GIS professionals can learn from its
customer focus. 2013. GEO: connexion 12, (1): 50

Esri. 2015. “Telling Your Story with Esri Story Maps” Esri Virtual Campus. Online at:
http://training.esri.com/gateway/index.cfm?fa=catalog.webCourseDetail&courseid=2924

Esri. 2015. “The Five Principles of Effective Storytelling”
online at: http://storymaps.arcgis.com/en/five-principles/

Esri. 2015. “How to Make a Story Map”
online at: http://storymaps.arcgis.com/en/how-to/

Esri GeoNet. 2015. Story Maps Blog.
Online at: https://geonet.esri.com/community/gis/web-gis/storymaps/content

Gregory, Ian N., editor, Alistair Geddes editor, and ebrary. 2014. Toward spatial humanities:
Historical GIS and spatial history. Bloomington, Indiana: Indiana University Press

Krathwohl, D. R. 2002. A revision of Bloom’s taxonomy: An overview. Theory into Practice 41
(4): 212–218.

Mitchell, S.A. 1846. A New Map of Texas, Oregon, and California with the Regions Adjoining.
online at: http://www.davidrumsey.com/luna/servlet/detail/RUMSEY~8~1~238~20003:
A-New-Map-of-Texas-Oregon-and-Calif

Roth, Robert E. 2013. “An Empirically-Derived Taxonomy of Interaction Primitives for
Interactive Cartography and Geovisualization.” IEEE Transactions on Visualization and
Computer Graphics, Vol. 19, No. 12. December 2013: 2356-2365.

Snyder, Jeffrey W., and Thomas C. Hammond. 2012. "so that's what the whiskey rebellion
was!": Teaching early U.S. history with GIS. The History Teacher 45, (3): 447-455

Szukalski, Bern. 2015. “Using Create Story to choose a Story map” ArcGIS Resources.
Online at: http://blogs.esri.com/esri/arcgis/2015/10/06/using-create-story-to-choose-a-
story-map/

99

History Sources
(These sources will provide the historical data required to develop the study's lesson plans and
maps.)

Primary Sources:

Ball, Eve. Indeh: An Apache Odyssey. Forward by Dan L. Thrapp. Norman, OK: University
of Oklahoma Press, 1980.

Cochise: Firsthand Accounts of the Chiricahua Apache Chief. Edited by Edwin R. Sweeney.
Norman, OK: University of Oklahoma Press, 2014.

Geronimo. Geronimo: His Own Story - The Autobiography of a Great Patriot Warrior.
Revised Edition. Edited by S.M. Barrett and Frederick Turner. New York, NY:
Penguin Books,1996.

Secondary Sources:

Josephy, Alvin M. The Civil War in the American West. New York, NY: Vintage Books, 1991.

Mort, Terry. The Wrath of Cochise: The Bascom Affair and the Origins of the Apache Wars.
New York, NY: Pegasus Books, 2013.

Thrapp, Dan L. The Conquest of Apacheria. Norman, OK: University of Oklahoma Press,
1967.

100

Appendix A: Institutional Review Board (IRB) Documentation

101

102

Appendix B: IRB Certified Youth Assent-Parental Permission Form

103

104

105

106

Appendix C: Hawaii Department of Education Data Sharing Agreement
(DSA) and Work Plan
107

108

109

Appendix D: Static Research Tool
THE BASCOM AFFAIR
“Young officers were often entrusted with important duties, the execution of which affected their
military standing more or less ever afterwards… The first paragraph in Army regulations
explains the manner in which orders shall be obeyed, and in this spirit, Lieutenant Bascom tried
to carry out his orders.”

-- SGT Daniel Robinson

Figure 17 - Plaque located in Apache Pass commemorating the Bascom Affair

IDENTIFYING OUR SPACE…

We use spatial skills to analyze and interpret data from maps relating to people, places, and
environments in an attempt to explain the interactions between geographic regions, and various
societies throughout history. However, in order to fully appreciate the relationship between
humans and geographic space, one must recognize and contend with the, “imaginative quality of
their own views.” (Mares and Moschek 2013) In other words, you must account for the ways that
personal prejudice and preconceived notions influence the manner in which you interpret
spatial-temporal data.

110

A simple question like, "Where are we?" can be answered in several ways depending on the way
you approach the question. Take for example the two maps depicting the southwestern United
States as it was in 1846 and 1861. Consult the 1861 map's legend to determine when Arizona
and New Mexico became United States Territories.

Figure 18 - Map of Texas, California, and Oregon from 1846, originally published by S.
Augustus Mitchell in Philadelphia, Pennsylvania (David Rumsey Collection Online 1998)
111

THE INDEH CIRCA 1861

"Who" is another perspective that is often forgotten. From the United States' perspective, they
acquired the lands in present day Arizona and New Mexico through a series of cessions,
annexations, and purchases from the Mexican government following the Mexican-American
War. However, the Indeh (as the Apache refer to themselves) perceived land ownership
somewhat differently... The Indeh inhabited the area on the map long before the Spanish,
Mexican, or American governments laid claim.

The Map on this tab depicts the approximate territorial boundaries of the primary Apache bands
involved in the Bascom Affair - the event that triggered 25 years of war between the United
States and Apache nation. Spend some time reviewing the maps you've been presented with so
far to familiarize yourself with the different perspectives of space in question. See if you can
identify your own bias before continuing with the lesson.

Figure 19 - Painting of the Apache by David Nordahl
In many respects, when Americans conceive of the Apache Indians they envision the Chiricahua
band. This statement rings as true today as in February, 1861 at the onset of the Apache War. In
reality, the Indeh included several bands of American Indians that spanned the southwest with a
combined population of around six thousand people. The Chiricahua settled largely in
Southeastern Arizona and portions of New Mexico, and Northern Sonora and Chihuahua.

Although confusion exists as to their true subdivisions, the Chokonen led by Cochise, the
Chihenne (Ojo Caliente/Hot Springs) led by Victorio, the Bedonkohe led by Mangas Coloradas,
and the Nednhi led by Juh are generally accepted as accurate inclusions. At the very least, strong
112

bonds existed between the four groups and they frequently lived together, raided, and went to
war as allies.

Figure 20 - Map by Alonso, depicting the approximate regions of each of the major Apache
bands. The Spanish referred to the Apache lands as Apacheria. See if you can spot Apacheria on
the map from 1846. (Accessed online at https://apacheria.es/)

THE RAID ON THE WARD RANCH

In January 1861, raiders from the Arivaipa group of the Western Apache band descended upon
John Ward’s ranch in present day Sonoita, Arizona. They made off with approximately twenty
head of cattle, and perhaps most importantly, Ward’s twelve year old stepson, Felix. During their
escape, the Arivaipa likely laid a false trail to the east to avoid suspicion before heading to their
homes along Arivaipa Creek to the North. In any event, Mr. Ward, who had not been present at
the attack itself, identified their spoor and immediately blamed Cochise for the incident. He
reported as such to Lieutenant Colonel Pitcairn Morrison at nearby Fort Buchanan, who
responded by ordering Lieutenant George N. Bascom to pursue the Apaches and use the force
under his command to recover the stolen property and Felix Ward.

At the outset, neither party expected the sequence of events that would follow. For Bascom,
recovering lost property captured during an Apache raid was a common task. Although Bascom
113

had never participated in direct actions with the Apache himself, by 1861 the Army grew
accustomed to policing the Southwest, and likely would have viewed the Ward incident as
routine.

CULTURAL CONSIDERATIONS

Apache culture revolved around the practice of raiding, which they depended upon for
sustenance to support their nomadic lifestyle. They learned early that established agricultural
communities and fixed rancherias meant extinction for their peoples. Not only did the terrain
make it difficult to sustain large communities, but fixed sites provided vulnerable targets for Ute
and Comanche war parties in the late seventeenth and early eighteenth centuries. The level of
violence and destruction visited upon the Apache by the Utes and Comanches in this period
dwarfed their losses to Mexican and American forces in the nineteenth century. Of the fourteen
Apache groups that ranged as far north as present day Nebraska, and well into central Texas,
only the Jicarilla survived the brutal onslaught and retained possession of a small portion of their
original land. The remaining Apache bands learned a valuable lesson; survival depended on
mobility, concealment, and resourcefulness.

Figure 21 - Example of a common Apache dwelling site established at Fort Bowie, Arizona for
visitors to observe.
In the three years after Cochise first met with the United States’ Apache agent, Michael Steck, he
and his Chokonen had become the most well known group in the region. Anytime violence
occurred, he generally received credit for the raid whether or not he played a role. There are
114

several explanations for this, but primarily, Cochise continued to raid after agreeing to let the
Butterfield stage line cross through Apache Pass.
The Apache had a difficult time comprehending that the various United States and Mexican
settlements shared common governance. Therefore, while Cochise had agreed upon peaceful
relations surrounding Apache Pass, he likely did not understand that residents outside of that
region fell under the same protections. Similarly, American settlers at the time could not grasp
that Apache bands operated autonomously. Cochise and his Chokonen often drew accusations
simply because his name carried the most notoriety and most settlers saw him as the Apaches’
leader.

The Apache criminal justice system provides yet another explanation for the Chokonen receiving
credit for so many depredations during the period when Cochise tried to live at peace with the
Americans. When Apaches committed crimes against their own people, the group exiled them.
These exiles could not seek refuge with other groups, so they often banded together to form their
own support structures. Frequently, the official bands received credit for raids committed by
their exiled members. At any rate, although John Ward lacked physical evidence of Cochise’s
involvement at the time, Lieutenant Colonel Morrison had ample reason to investigate his claim
and little reason to suspect that his orders would inadvertently trigger all-out war.

Figure 22 - Geronimo with three of his warriors in Canon de los Embudos; from left to right:
Yahnozha, Chappo, Fun, and Geronimo.
THE APPROACH TO APACHE PASS

Several factors indicate that Cochise did not view Bascom as a threat when he arrived at Apache
Pass on February 3, 1861 with John Ward in tow as his interpreter and fifty four soldiers of
115

Charlie Company, Seventh Infantry under his command. Rather than assuming a defensive
posture, the Chokonen continued to trade openly with United States soldiers in vicinity of
Apache Pass, and approach them without reservation.

Figure 23 - View approaching Apache Pass similar to what LT Bascom and his Soldiers would
have enjoyed.
When Cochise went to meet Bascom in his camp, he brought his family in tow. Cochise arrived
with his brother Coyuntura, two of his nephews, and most notably his wife Dos-teh-seh
(daughter of Mangus Coloradas), and two of their children.

THE CONFLICT BEGINS

Bascom and Cochise met on the morning of February 4, 1861 at Bascom’s camp in Apache Pass.
Bascom took Cochise and the adult males (Coyuntura, his nephews, and the unnamed warrior)
with him into his tent, and instructed his soldiers to form a security perimeter around the site.
Through Ward, Bascom accused Cochise of conducting the raid and required that he return Felix
Ward. Cochise denied any involvement, but offered to identify the guilty party and bring the boy
back if the lieutenant would grant him ten days to account for travel to and from the Black
Mountains where he thought the Ward boy to be. Bascom refused, and instead informed Cochise
that he would be held prisoner, along with his family, until he returned Felix Ward. At this point,
Cochise drew his knife, and cut his way out of the tent, accompanied by the un-named warrior.
The warrior fell victim to one of the guards’ bayonets, but Cochise managed to escape.

Over the next several weeks, conditions devolved rapidly as Cochise attempted first to intimidate
Bascom into releasing his family, then seeing that fail, looked for opportunities to collect
116

hostages as leverage to secure their release. When that failed too, Cochise sought vengeance.
Prior to breaking camp and removing the remaining Chiricahua to safety in preparation for war,
Cochise ordered the execution of his hostages. Events culminated on February 19, 1861. In
retaliation for Cochise’s actions, Bascom ordered his soldiers to hang the Chiricahua chief’s
brother and nephews, along with three additional Apache prisoners captured by his command in
the previous weeks. He then took Cochise’s wife and children back to Fort Buchanan with him,
where the Army eventually released them.

Figure 24 - Remnants of the old Butterfield Stage station at Apache Pass; Cochise collected
hostages from the Butterfield station after his initial attempt to intimidate LT Bascom into
releasing his family failed.

WHAT WENT WRONG

Why did a routine policing action devolve into to what Cochise later termed, “a very great
wrong” committed by Bascom and his soldiers that motivated Cochise and the Chiricahua
Apache to unite in war against the United States?

Foremost, Morrison and Bascom made a critical error in their choice of interpreters. John Ward
understood nothing of the Apache language or culture, and had poor control of the Spanish
language that the mission relied upon to communicate with Cochise. More importantly, Ward
approached the situation heavily invested in the outcome, and clearly biased as to who he
117

thought bore responsibility for the raid on his property. Bascom paid for his decision to use Ward
immediately, as Ward insulted Cochise the moment he stepped foot in the tent.

But what of LT Bascom himself? Immediate narratives developed placing the blame for the
event at his feet, and modern interpretations of the Bascom Affair continue to follow suit. But
was he properly prepared for the task at hand? If as SGT Robinson's quote implied, that young
officers were frequently entrusted with significant responsibilities, for which the consequences of
failure could potentially lead to war, what did the Army do to prepare their officers for these
missions? After all, if the argument can be made that a more seasoned officer would have reacted
differently, then it stands to reason that, that officer must have understood something more than
Bascom, which means that something in Bascom’s development may have been lacking.

Figure 25 - Timeline of major events leading up to the Bascom Affair

Bascom lacked the skills required to perform the mission that Lieutenant Colonel Morrison
ordered him to accomplish: Diplomacy, negotiation, and cultural awareness. West Point certainly
118

did not develop these skills in the officers that they commissioned. Their curriculum focused
heavily on math and science, neither of which proved much use to Bascom in Apache Pass.

If Bascom had arrived at Apache Pass fresh off of the trip from the Hudson, it would be fair to
lay the blame squarely on his military academy background, but Bascom didn’t graduate from
West Point in 1860. He graduated the Academy as part of the class of 1858, and had already
been an officer for three years when he confronted Cochise. In fact, he had been operating in the
Trans-Mississippi West since approximately May of 1859 and served under Morrison’s
command since the summer of 1860. Ultimately, the Army had three years to professionally
develop Bascom, to share with him the knowledge that would have allowed a more seasoned
officer to successfully negotiate with Cochise to recover John Ward’s property and son.
Unfortunately, none of his superior officers felt the endeavor merited consideration.

119

120

121

122

123

124

125

Appendix E: Dynamic Research Tool
The University of Southern California, Spatial Sciences Institute will continue to host the study’s
dynamic tool on their Esri organizational account through May 2019. To access the story map
used in the study go to the link provided below:

http://uscssi.maps.arcgis.com/apps/MapSeries/?appid=0dacfa88833b4c509fbafaed0ce22941

126

Appendix F: Study Examination Tool
THE BASCOM AFFAIR – Examination Participant ID:__________
Time Completed:________

Standards Assessed:
Based on Hawaii Content and Performance Standards III for Social Studies

 Standard 1: Historical Understanding: Change, Continuity, and Causality – Understand
change and/or continuity and cause and/or effect in history

 Standard 2: Historical Understanding: Inquiry, Empathy, and Perspective – Use the tools
and methods of inquiry, perspective, and empathy to explain historical events with multiple
interpretations and judge the past on its own terms

 Standard 6: Cultural Anthropology: Systems, Dynamics, and Inquiry – Understand
culture as a system of beliefs, knowledge, and practices shared by a group and understand
how cultural systems change over time

 Standard 7: Geography: World in Spatial Terms – Use geographic representations to
organize, analyze, and present information on people, places, and environments and
understand the nature and interaction of geographic regions and societies around the
world

Section One: Fill in the blank (10 questions – 1 pt each)

1) The Apache refer to themselves as the ________________.

2) John Ward’s ranch is near present day ________________, in the state of
________________.

3) Felix Ward was kidnapped by the ________________ Apache, belonging to the
________________ Apache Band.

4) John Ward blamed Cochise, chief of the ________________ Apache, for the raid on
his ranch.

5) Cochise was a member of the ________________ band of Apache.

6) Cochise spoke fluent ________________, and ________________.

7) Apache Pass is located in the ________________ Mountains.

8) Cochise permitted the ________________ to cross through Apache Pass.

127

9) Lieutenant Colonel Morrison was the commanding officer at ________________.

10) Lieutenant Bascom represented the________________ when he arrived at Apache
Pass on February 3, 1861.

Section Two – True or False (2 questions – 1 pt each)

11) The Bascom Affair took place in Arizona?

12) The Apache bands shared a common form of governance similar to the United
States model?

Section Three – Short Answer (2 questions – 1 pt each)

13) Which Apache band claimed the region in the vicinity of John Ward’s Ranch?

14) Which Apache groups did Cochise turn to for support following the capture of his
family at Apache Pass?

Section Four – Spatial Analysis (6 questions – 1 pt each)

15) Could Cochise see Lieutenant Bascom approaching Apache Pass with his company
of 54 soldiers?

16) If you answered yes to question 15, approximately how far away was Lieutenant
Bascom’s company when Cochise was able to identify their movement? If you
answered no, what prevented Cochise from identifying Lieutenant Bascom’s
approach?

17) How far did Lieutenant Bascom Travel to reach Apache Pass?

18) What was the distance between Apache Pass and the location where Felix Ward was
taken?

19) What was the nearest United States controlled city to Apache Pass?

20) What is the distance between the city identified in question 19 and Apache Pass?

Section Five – Essay (4 questions – 5 pts each)

128

1) Where did the Apache live in 1861?

129

2) In your own words, describe the events that led the United States and the Apache to
war in 1861?

130

3) Were Lieutenant Colonel Morrison and Lieutenant Bascom justified in their
assumption of Cochise’s involvement? Why or Why not?

131

4) Do you think that Cochise knew about the kidnapping before Lieutenant Bascom
confronted him at Apache Pass? What indicators support your argument?

132

Section Six – Study Assessment (0 pts each – research assessment only)

Did you review/access the research tools between sessions?

If yes, how often (how many times)? How many hours (approximately) did you spend using
the research tool outside of the classroom?

Why did you, or didn’t you choose to access the materials between sessions?

If you were in the experimental group, which medium(s) did you use to access the story
map (Computer internet browser, tablet, smart phone, or a combination of the three)?

Did you look for any outside resources about the Bascom Affair?

If yes, what kind of resources did you pursue?

Thank you for your participation. Please remember to record your participant ID on the Front of the
exam. If you forgot your ID, your teacher can provide it when you submit your exam.

133

Appendix G: Study Exam Tool Grading Rubric
This rubric defines the criteria used by the study to assess student responses to the exam tool’s
essay questions. The study targeted specific Hawaii Content and Performance Standards in each
question, and used the stated performance benchmarks to evaluate levels of critical thought based
on Bloom’s taxonomy as revised by Krathwohl in 2002.

Essay Question #1: Where did the Apache live in 1861?

Hawaii Content and Performance Standards Targeted:

Standard 2: Historical Understanding: Inquiry, Empathy, and Perspective – Use the tools and
methods of inquiry, perspective, and empathy to explain historical events with multiple
interpretations and judge the past on its own terms
Hawaii Benchmark: No benchmarks
identified
Study Benchmark: Examine the Bascom
Affair from multiple perspectives: American
and Apache
Sample Performance Assessment:
The student:
Identifies their own preconceived biases, articulates how the principal actors perceived the
events leading up to the conflict at Apache Pass, and critical differences in each narrative that
shape the way the event is viewed today.

Standard 7: Geography: World in Spatial Terms – Use geographic representations to organize,
analyze, and present information on people, places, and environments and understand the nature
and interaction of geographic regions and societies around the world
Hawaii Benchmark:
SS.11.7.1 – Trace changing political
boundaries under the influence of European
Imperialism
Study Benchmark: Trace changing political
boundaries under the influence of American
Western Expansion
Sample Performance Assessment:
The student:
Examines the new political boundaries created by American Western Expansion in the present
day American Southwest.
Hawaii Benchmark:
SS 11.7.2 – Use tools and methods of
geographers to understand changing views of
world regions
Study Benchmark: Use tools and methods of
geographers to understand changing views of
the present day American Southwest
Sample Performance Assessment:
The student:
Uses geographic visualization methods to understand changing conceptions of the present day
American Southwest.

134

Performance Assessment:

Recall: Student response indicates a basic understanding that the Apache people lived in Arizona
and New Mexico; however lack understanding of the changing perceptions of political
boundaries over time or account for the Apache point of view.

Understanding: Student provides detailed descriptions of the surrounding terrain and political
boundaries from the United States’ perspective; however, continue to lack temporal
understanding of the changing perception of political boundaries over time or account for the
Apache point of view.

Analyze: Student response indicates a basic sense of conflicting land claims between the United
States and Apache, but fails to demonstrate awareness of temporal change; student response
reflects current United States’ bias and fails to fully account for the Apache point of view.

Evaluate: Student demonstrates a clear understanding of changing conceptions of the present
day American Southwest, and accurately identifies the land as the Arizona and New Mexico
Territories; however, student response fails to account for the Apache view of the land and
continues to relay United States’ bias

Create: Student response indicates that the student accounts for the Apache perspective of the
land as they saw it in 1861; response provides evidence that the student overcame their pre-
conceived notions of space based on modern interpretations of political boundaries and land
ownership.

Essay Question #2: In your own words, describe the events that led the United States and the
Apache to war in 1861?

Hawaii Content and Performance Standards Targeted:

Standard 1: Historical Understanding: Change, Continuity, and Causality – Understand
change and/or continuity and cause and/or effect in history
Hawaii Benchmark: No benchmarks
identified
Study Benchmark: Examine the events that
led to war between the United States and
Apache in 1861
Sample Performance Assessment:
The student:
Identifies the relationship between the United States and the Apache in 1861 prior to Bascom
Affair, and articulates the connections between the kidnapping of John Ward’s son, and the
actions taken by the United States Army that led to war.

135

Standard 2: Historical Understanding: Inquiry, Empathy, and Perspective – Use the tools and
methods of inquiry, perspective, and empathy to explain historical events with multiple
interpretations and judge the past on its own terms
Hawaii Benchmark: No benchmarks
identified
Study Benchmark: Examine the Bascom
Affair from multiple perspectives: American
and Apache
Sample Performance Assessment:
The student:
Identifies their own preconceived biases, articulates how the principal actors perceived the
events leading up to the conflict at Apache Pass, and critical differences in each narrative that
shape the way the event is viewed today.

Standard 6: Cultural Anthropology: Systems, Dynamics, and Inquiry – Understand culture as
a system of beliefs, knowledge, and practices shared by a group and understand how cultural
systems change over time
Hawaii Benchmark: No benchmarks
identified
Study Benchmark: Examine how failure to
understand and account for culture
exacerbated conditions between the United
States and Apache
Sample Performance Assessment:
The student:
Identifies cultural misunderstandings that escalated tensions to the point of war and is able to
discuss opportunities for intervention that both parties missed.

Performance Assessment:

Recall: Student response provides a basic review of the main events leading to war; however
lacks understanding of the existing relationship between the U.S. and Apache, how the events
relate, and a broader sense of how the role of culture influenced the outcome at Apache Pass.

Understanding: Student response provides a detailed description the events leading to war;
however continues to lack understanding of the existing relationship between the U.S. and
Apache, how the events relate, and a broader sense of how the role of culture influenced the
outcome at Apache Pass.

Analyze: Student reflects a basic understanding of how the kidnapping of Felix Ward and the
false accusation against the Chokonen resulted in conflict; however student response indicates
confusion over how cultural misunderstandings influenced U.S. actions, and fails to account for
conditions between the U.S. and Apache prior to 1861.

Evaluate: Student articulates a detailed understanding of the multiple layers of actions that led to
war on both sides, and indicates how failing to understand the Apache culture caused the U.S to
miss opportunities for peaceful resolution; however, student response does not consider the event
in the broader context of the pre-existing relationship between the U.S. and Apache in 1861.

136

Create: Student response considers both points of view, and indicates a clear understanding of
the events that led the U.S. and Apache to war in 1861; student defines how failure to
comprehend Apache culture influenced the U.S. and places the events in the context of the
relationship between the U.S. and Apache in 1861.

Essay Question #3: Were Lieutenant Colonel Morrison and Lieutenant Bascom justified in
their assumption of Cochise’s involvement? Why or Why not?

Hawaii Content and Performance Standards Targeted:

Standard 1: Historical Understanding: Change, Continuity, and Causality – Understand
change and/or continuity and cause and/or effect in history
Hawaii Benchmark: No benchmarks
identified
Study Benchmark: Examine the events that
led to war between the United States and
Apache in 1861
Sample Performance Assessment:
The student:
Identifies the relationship between the United States and the Apache in 1861 prior to Bascom
Affair, and articulates the connections between the kidnapping of John Ward’s son, and the
actions taken by the United States Army that led to war.

Standard 2: Historical Understanding: Inquiry, Empathy, and Perspective – Use the tools and
methods of inquiry, perspective, and empathy to explain historical events with multiple
interpretations and judge the past on its own terms
Hawaii Benchmark: No benchmarks
identified
Study Benchmark: Examine the Bascom
Affair from multiple perspectives: American
and Apache
Sample Performance Assessment:
The student:
Identifies their own preconceived biases, articulates how the principal actors perceived the
events leading up to the conflict at Apache Pass, and critical differences in each narrative that
shape the way the event is viewed today.

Standard 6: Cultural Anthropology: Systems, Dynamics, and Inquiry – Understand culture as
a system of beliefs, knowledge, and practices shared by a group and understand how cultural
systems change over time
Hawaii Benchmark: No benchmarks
identified
Study Benchmark: Examine how failure to
understand and account for culture,
exacerbated conditions between the United
States and Apache
Sample Performance Assessment:
The student:
Identifies cultural misunderstandings that escalated tensions to the point of war and is able to
discuss opportunities for intervention that both parties missed.

137

Standard 7: Geography: World in Spatial Terms – Use geographic representations to organize,
analyze, and present information on people, places, and environments and understand the nature
and interaction of geographic regions and societies around the world
Hawaii Benchmark:
SS.11.7.1 – Trace changing political
boundaries under the influence of European
Imperialism
Study Benchmark: Trace changing political
boundaries under the influence of American
Western Expansion
Sample Performance Assessment:
The student:
Examines the new political boundaries created by American Western Expansion in the present
day American Southwest.
Hawaii Benchmark:
SS 11.7.2 – Use tools and methods of
geographers to understand changing views of
world regions
Study Benchmark: Use tools and methods of
geographers to understand changing views of
the present day American Southwest
Sample Performance Assessment:
The student:
Uses geographic visualization methods to understand changing conceptions of the present day
American Southwest.

Performance Assessment:

Recall: Student adopts a weak position; response confuses the relationship between events
leading up to the Bascom Affair, displays pre-existing bias that fails to account for the existing
conditions in 1861 and the role of culture, and fails to consider the implications of geographic
space.

Understanding: Student defends their position using evidence from the lesson, but evidence
lacks a clear connection to the argument; student response still lacks consideration of geographic
space, and a clear understanding of how events prior to 1861 shaped the U.S. response or how
failing to contend with Apache culture influenced U.S. assumptions of guilt.

Analyze: Student responds using evidence from the lesson that clearly supports their position;
however, response relies on current events and fails to overcome initial bias to consider the
historical context of the event or how U.S. actions were influenced by a lack of cultural
awareness.

Evaluate: Student response uses persuasive evidence that indicates a clear understanding of
missed opportunities for a peaceful resolution had the U.S. understood the Apache culture;
however, response still portrays clear evidence of student bias and fails to contend with the
events from the U.S. perspective in the historical context.

Create: Student response indicates that they overcame their personal bias to analyze the event
from Morrison and Bascom’s perspective; response considers the events in the context of the
historical setting and reflects an understanding of how failing to account for Apache culture
138

impacted their ability to assess how geographic distance between the Western and Chiricahua
Apache bands influenced Cochise’s knowledge of the kidnapping event.

Essay Question #4: Do you think that Cochise knew about the kidnapping before Lieutenant
Bascom confronted him at Apache Pass? What indicators support your argument?

Hawaii Content and Performance Standards Targeted:

Standard 1: Historical Understanding: Change, Continuity, and Causality – Understand
change and/or continuity and cause and/or effect in history
Hawaii Benchmark: No benchmarks
identified
Study Benchmark: Examine the events that
led to war between the United States and
Apache in 1861
Sample Performance Assessment:
The student:
Identifies the relationship between the United States and the Apache in 1861 prior to Bascom
Affair, and articulates the connections between the kidnapping of John Ward’s son, and the
actions taken by the United States Army that led to war.

Standard 2: Historical Understanding: Inquiry, Empathy, and Perspective – Use the tools and
methods of inquiry, perspective, and empathy to explain historical events with multiple
interpretations and judge the past on its own terms
Hawaii Benchmark: No benchmarks
identified
Study Benchmark: Examine the Bascom
Affair from multiple perspectives: American
and Apache
Sample Performance Assessment:
The student:
Identifies their own preconceived biases, articulates how the principal actors perceived the
events leading up to the conflict at Apache Pass, and critical differences in each narrative that
shape the way the event is viewed today.

Standard 6: Cultural Anthropology: Systems, Dynamics, and Inquiry – Understand culture as
a system of beliefs, knowledge, and practices shared by a group and understand how cultural
systems change over time
Hawaii Benchmark: No benchmarks
identified
Study Benchmark: Examine how failure to
understand and account for culture
exacerbated conditions between the United
States and Apache
Sample Performance Assessment:
The student:
Identifies cultural misunderstandings that escalated tensions to the point of war and is able to
discuss opportunities for intervention that both parties missed.

139

Standard 7: Geography: World in Spatial Terms – Use geographic representations to organize,
analyze, and present information on people, places, and environments and understand the nature
and interaction of geographic regions and societies around the world
Hawaii Benchmark:
SS.11.7.1 – Trace changing political
boundaries under the influence of European
Imperialism
Study Benchmark: Trace changing political
boundaries under the influence of American
Western Expansion
Sample Performance Assessment:
The student:
Examines the new political boundaries created by American Western Expansion in the present
day American Southwest.
Hawaii Benchmark:
SS 11.7.2 – Use tools and methods of
geographers to understand changing views of
world regions
Study Benchmark: Use tools and methods of
geographers to understand changing views of
the present day American Southwest
Sample Performance Assessment:
The student:
Uses geographic visualization methods to understand changing conceptions of the present day
American Southwest.

Performance Assessment:

Recall: Student adopts a weak position; response confuses the relationship between events
leading up to the Bascom Affair, displays pre-existing bias that fails to account for the political
subdivisions of the Apache in 1861, and fails to consider the implications of geographic space on
Cochise’s sphere of influence.

Understanding: Student defends their position using evidence from the lesson, but evidence
lacks a clear connection to the argument; student response still lacks consideration of geographic
space, and a clear understanding of how events prior to 1861 shaped the Apache response or how
Apache political divisions would have influenced Cochise’s knowledge of the event.

Analyze: Student responds using evidence from the lesson that clearly supports their position;
however, response relies on current events and fails to overcome initial bias to consider the
historical context of the event or how Apache political divisions influenced the knowledge
Cochise had access to prior to Bascom’s arrival.

Evaluate: Student response uses persuasive evidence that indicated a clear understanding of
Cochise’s frame of mind entering the meeting with Bascom; however, response still portrays
clear evidence of initial bias (Cochise as the chief of the Apache as opposed to chief of a small
sub-group of the Chiricahua band) and fails to contend with the implications of geographic
distance between the Western and Chiricahua Apache bands.

140

Create: Student response indicates that they overcame their initial bias to analyze the event from
Cochise’s perspective; response considers the events in the context of the historical setting and
reflects an understanding of how Apache political divisions and the geographic distance between
the Western and Chiricahua Apache bands influenced Cochise’s knowledge of the kidnapping
event.

Abstract (if available)

Abstract Spatial scientists spent the better part of the last three decades pushing for further integration of Geographic Information Science (GIS) technologies in K-12 curriculums. Their efforts to date are leading to moderate breakthroughs in geography and physical sciences, but social studies continue to neglect its use almost entirely. Unfortunately, little empirical evidence exists that suggests students realize quantifiable gains from its inclusion in the classroom. In fact, the findings from most research comparing visualization methods indicate that static mapping methods outperform dynamic methods when assessed by the user’s ability to extract information from the product. This study adds to existing literature by expanding upon current research into static versus dynamic visualization methods. In contrast to previous visualization studies that focus heavily on animation for their dynamic representations, this study tested static methods against story maps to determine whether they provide teachers an advantage in the classroom. ❧ To develop its findings, the study employed standard classroom instruction methods and examination materials to identify which visualization method most effectively communicated the material to students in secondary school history classrooms. The study divided students into a control group using standard classroom static visualization tools, and an experimental group using dynamic story maps. Written exams conducted immediately following initial instruction, and again two weeks later, provided the basis for evaluation. The study failed to demonstrate that dynamic products provide students a distinct advantage over traditional static products in a classroom environment. Its findings suggest that students can use both tools equally effectively, supporting the findings from previous research. Of note, this study suggests that among female students, dynamic products may yield decreased learning outcomes. This indicates the need for further research to identify how gender affects visualization strategies.

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

Surface representations of rainfall at small extents: a study of rainfall mapping based on volunteered geographic information in Kona, Hawaii

PDF

Integrating spatial visualization to improve public health understanding and communication

PDF

Peoples of Washington historical geographic information system: geocoding culture using archival standards

PDF

Mapping future population impacts caused by sea level rise in Huntington Beach and Newport Beach: comparing the cadastral-based dasymetric system to past dasymetric mapping methods

PDF

Cartographic approaches to the visual exploration of violent crime patterns in space and time: a user performance based comparison of methods

PDF

Visualizing email response data to improve marketing campaigns

PDF

Generating bicyclist counts using volunteered and professional geographic information through a mobile application

PDF

Harnessing GIST-enabled resources in the classroom: developing a Story Map for use with secondary students

PDF

Social media to locate urban displacement: assessing the risk of displacement using volunteered geographic information in the city of Los Angeles

PDF

Semi-automated visualization of spatial information in unstructured text

PDF

A cartographic exploration of census data on select housing challenges among California residents

PDF

Web GIS as a disease management workspace: enabling advocacy at multiple scales across multiple continents with the case of tungiasis

PDF

Identification and analysis of future land-use conflict in Mecklenburg County, North Carolina

PDF

Investigating bus route walkability: comparative case study in Orange County, California

PDF

Operational optimization model for Hungry Marketplace using geographic information systems

PDF

Finding food deserts: a study of food access measures in the Phoenix-Mesa urban area

PDF

Walkability study for school accessibility: case study of the San Juan, Puerto Rico elementary schools

PDF

Using Maxent modeling to predict habitat of mountain pine beetle in response to climate change

PDF

Geographic information systems and marketing: a transdisciplinary approach to curriculum development

PDF

A comparison of urban land cover change: a study of Pasadena and Inglewood, California, 1992‐2011

Asset Metadata

Creator Martos, Jason Edward (author)

Core Title Visualizing historic space through the integration of geographic information science in secondary school curriculums: a comparison of static versus dynamic methods

School College of Letters, Arts and Sciences

Degree Master of Science

Degree Program Geographic Information Science and Technology

Publication Date 06/17/2016

Defense Date 05/02/2016

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag classroom teaching strategy,Geographic Information Science,GIS,OAI-PMH Harvest,spatial science,visualization methods

Format application/pdf (imt)

Language English

Contributor Electronically uploaded by the author (provenance)

Advisor Ruddell, Darren (committee chair), Hasan, Angela (committee member), Oda, Kirk (committee member), Swift, Jennifer (committee member)

Creator Email martos@usc.edu,MartosJE@gmail.com

Permanent Link (DOI) https://doi.org/10.25549/usctheses-c40-252992

Unique identifier UC11280157

Identifier etd-MartosJaso-4440.pdf (filename),usctheses-c40-252992 (legacy record id)

Legacy Identifier etd-MartosJaso-4440.pdf

Dmrecord 252992

Document Type Thesis

Format application/pdf (imt)

Rights Martos, Jason Edward

Type texts

Source University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Access Conditions The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the a...

Repository Name University of Southern California Digital Library

Repository Location USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA