University of Southern California Dissertations and Theses

Psychophysiological assessment of cognitive and affective responses for prediction of performance in arousal inducing virtual environments

(USC Thesis Other)

Psychophysiological assessment of cognitive and affective responses for prediction of performance in arousal inducing virtual environments

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content

PSYCHOPHYSIOLOGICAL ASSESSMENT OF COGNITIVE AND AFFECTIVE
RESPONSES FOR PREDICTION OF PERFORMANCE IN AROUSAL INDUCING VIRTUAL
ENVIRONMENTS

by

Christopher Gaelan Courtney

A Dissertation Presented to the
FACULTY OF THE USC GRADUATE SCHOOL
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulfillment of the
Requirements for the Degree
DOCTOR OF PHILOSOPHY
(PSYCHOLOGY)

December 2012

Copyright 2012 Christopher Gaelan Courtney

ii

Acknowledgements
I owe a debt of gratitude to a number of people who have helped me along the way. I
would like to start by thanking all of my friends who have supported me throughout this process,
especially Teresa, who has had to endure the brunt of the rollercoaster of emotions that I have
experienced while completing this work and has done so with patience and caring that will never
go unappreciated in our many years ahead.
I would also like to thank my family for their love and support throughout my life. My
father meant so much to me and I know that he would be so proud today. My uncle Eric always
spoke to me with the same respect that he would give to an adult, even at a very young age, and
taught me how to examine everything in a logical and broadminded manner. Thank you for
helping to raise me to be a thoughtful person, I owe you so much. Finally, my mom, who has
given everything she has to bring me to this point. I know that this means as much to you as it
does to me, if not more, and I want you to know that I appreciate all of the sacrifices you have
made and the hardships you have had to endure to raise me. You taught me that I should always
look on the bright side of any situation, and that is one of the most important lessons I have
learned. I know that without your undying support I would not be who I am today and I will
always appreciate all that you have done for me.
I would like to thank my committee members who have helped guide me to this point:
Thomas Parsons, Anne Schell, Bosco Tjan, Stanley Huey, John Brekke, and Michael Dawson. I
would also like to thank former committee members Steven Read and Shrikanth Narayanan for
their much appreciated guidance. Thank you all very much. I would especially like to thank Anne
Schell for all that I have learned from her throughout the years.
A special thanks to Thomas Parsons and the rest of the team at the Institute for Creative
Technologies. This research would not have been possible without you. I have enjoyed and
iii

appreciated every minute that I have had the privilege to work with Dr. Parsons. I thank him for
introducing me to an exciting field of research and I look forward to maintaining a future
relationship as both colleagues and friends.
Finally, I would like to thank my mentor Michael Dawson. I could not have asked for a
better guide through my educational experience in graduate school. It has been an absolute
pleasure working with you and learning from you. I am honored to have been your student and I
cannot thank you enough for being an incredible mentor and a friend.

iv

Table of Contents
Acknowledgements ii

List of Tables vi

List of Figures vii

Abstract ix

Chapter One: Introduction 1
Response Patterns Associated with Exposure to Threat 3
Response Patterns Associated with Changes in Cognitive Workload 6
VR Simulations for Adaptive System Development 10
Navigation in VEs 11
Conditioning in VEs 13
Toward Adaptive System Development 15
Multi-layer Perceptrons & Multiple Linear Regression 17

Chapter Two: Methodological Overview 19
Participants 19
Materials 19
Procedural Overview 20

Chapter Three: Experiment 1 Overview 21
Purpose 21
Design 21
Participants 23
Procedures 23
Dependent Variables 30
Experiment 1 Hypotheses 32

Chapter Four: Experiment 1 Analytic Approach 35
Data Reduction 35
Analyses 38
MLR and ANN Analytic Approach: Experiment 1 39

Chapter Five: Results 44
Threat Manipulation Task Results 44
Navigation Task Results 50
MLR and BPN results 55

Chapter Six: Experiment 1 Discussion 61
Threat Level Manipulation Task Effects 61
v

Navigation Task Effects 64
Simple and Complex Conditioning Effects 66
Regression and Artificial Neural Network Model Comparisons 68

Chapter Seven: Experiment 2 Overview 71
Purpose 71
Design 71
Participants 71
Procedures 72
Dependent Variables 74
Experiment 2 Hypotheses 74

Chapter Eight: Experiment 2 Data Analytic Approach 77
Data Reduction 77
Analyses 77
MLR and ANN Analytic Approach: Experiment 2 78

Chapter Nine: Experiment 2 Results 80
Cognitive Workload Manipulation Task Results 80
Navigation Task Results 85
MLR and BPN Results 92

Chapter Ten: Experiment 2 Discussion 98
Cognitive Workload Manipulation Task Effects 98
Navigation Task Effects 100
Regression and Artificial Neural Network Model Comparisons 103

Chapter Eleven: General Discussion 104

References 108

vi

List of Tables

Table 1: Dependent variables associated with the threat level manipulation task 30

Table 2: Dependent variables associated with the navigation task 31

Table 3: Dependent variables associated with the simple conditioning task 31

Table 4: Dependent variables associated with the complex conditioning task 32

Table 5: Correlations between navigation task outcome measures 40

Table 6: Multiple linear regression model summary statistics 55

Table 7: Predictor regression coefficients – Predictors based on overall 55
differences between high and low threat zones

Table 8: Predictor regression coefficients – Predictors based on differences 56
between zone pair 1 and zone pair 3

Table 9: Descriptive statistics associated with BPN inputs and target 57
variable for Experiment 1

Table 10: BPN model parameters for Experiment 1 58

Table 11: Global sensitivity analysis for the BPN developed for Experiment 1 59

Table 12: Summary statistics for MLR and BPN comparison in Experiment 1 59

Table 13: Multiple linear regression model summary statistics for Experiment 2 93

Table 14: Predictor regression coefficients – Predictors based on overall 94
differences between high and low cognitive workload zones

Table 15: Predictor regression coefficients – Predictors based on differences 94
between zone pair 1 and zone pair 3

Table 16: BPN model parameters for Experiment 2 95

Table 17: Descriptive statistics associated with BPN inputs and target variable 95
for Experiment 2

Table 18: Global sensitivity analysis for the BPN developed for Experiment 2 96

Table 19: Summary statistics for MLR and BPN comparison in Experiment 2 97

vii

List of Figures

Figure 1: Example of a High Threat Zone 22
Figure 2. Example of a Low Threat Zone. 23
Figure 3: Example of a Red Zone Marker 27

Figure 4: Example of a Yellow Zone Marker 28

Figure 5: Experiment 1 BPN Structure Example 42

Figure 6: Threat Level Main Effects 44

Figure 7: Zone Order Main Effects 45

Figure 8: Threat Level by Zone Order Interaction 47

Figure 9: Zone Order by Navigation Awareness Interaction 48

Figure 10: Skin Conductance Responses in Anticipation of Conditioned Stimuli 50

Figure 11: Zone Order Effects in Navigation 52

Figure 12: Skin Conductance Levels During Navigation Task 53

Figure 13: Interbreath intervals and Navigation Performance 56

Figure 14: Experiment 2 BPN Structure Example 79
Figure 15: Cognitive Workload Main Effects 80
Figure 16: Zone Order Effects 81

Figure 17: Cognitive Workload Intensity by Zone Order Interaction 82
Figure 18: Cognitive Workload by Navigation Awareness Interactions 83
Figure 19: PASAT Performance 85

Figure 20: Psychophysiological zone order main effects 86

Figure 21: Navigation Zone Order Main Effects 87

Figure 22: Cognitive Workload by Zone Order Interactions 88

Figure 23: Behavioral Workload by Zone Order Interactions 89

viii

Figure 24: Zone Order by Navigation Awareness Interactions 91

Figure 25: HF Component Power and Navigation Performance 93

ix

Abstract
The current study sought to examine the psychophysiological response patterns
associated with varying levels of threat and cognitive workload in a highly immersive virtual
environment (VE) containing a route-learning and navigation task scenario. Participants were led
down a specific path through a virtual city by a group of virtual guides. Upon reaching the goal of
this initial tour through the city, participants were instructed to navigate back to the starting point
of the tour following the same path taken to reach the goal. Two separate experiments were
conducted to examine the effects of threat and cognitive workload variations in the environment
separately. Psychophysiological responses to threat in Experiment 1 and varying levels of
cognitive workload in Experiment 2 were then utilized to develop multiple linear regression
(MLR) and artificial neural network (ANN) models for prediction of performance on the
navigation task. Comparisons of predictive abilities between the developed models were
performed to determine optimal model parameters. Awareness of the navigation task was
manipulated such that half of the participants in each experiment were made aware of the
navigation task prior to the initial tour to allow for route-learning assessment, while the other half
were told only after tour completion to assess response patterns associated with threat and
cognitive workload in the absence of the added task of committing the route to memory.
Participants made aware of the navigation task evidenced increased efficiency on the return trip
through the city. Additionally, the threat level and cognitive workload manipulations were
successful in eliciting varying response patterns during areas of high and low intensity stimulus
presentations. Finally, ANN models were determined to better predict navigation performance
based on psychophysiological responses gleaned during the initial tour through the city. The
selected models were able to predict navigation performance with better than 80% accuracy in
x

both Experiments 1 and 2. Applications of the models toward improved human-computer
interaction and psychophysiologically-based adaptive systems are discussed.

1

Chapter One: Introduction
Recently, research involved in the improvement of human-computer interaction (HCI)
has implemented a greater degree of reliance on psychophysiological metrics to enhance
objective assessments of user-states for numerous applications. For example, psychophysiology
has played an integral role in the burgeoning field of neuroergonomics, which melds information
gleaned from the fields of neuroscience and human factors (for review see, Parasuraman & Rizzo,
2006). The aim of neuroergonomics is to develop an understanding of cognitive processes and
human performance characteristics, and to apply that knowledge to creating systems that improve
safety and efficiency for the human user. Psychophysiological computing, which uses human
physiological responses as a source of input to a computer system (Allanson & Fairclough, 2004),
provides a vital bridge between the human user and the computer, and can provide the computer
with information regarding the participant’s functional state to make real-time adaptations to a
system designed to improve efficiency, maintain vigilance, or reduce stress (Parasuraman &
Wilson, 2008).
Psychophysiological methodologies contain a number of distinct advantages compared to
behavioral measures for use in adaptive systems, and for improving human-computer interaction
(HCI) in general. For example, psychophysiological response measures involve covert signals
that do not require added tasks from the participant in order for a response to be recorded.
Additionally, the psychophysiological data trace is continuously available for monitoring, even
when no overt task-related behavior is required, whereas behavioral data tends only to be
available intermittently (Alanson & Fairclough, 2004). Thus, adaptive systems utilizing
psychophysiological computing techniques offer numerous possibilities for improving HCI and
related applications. One such application is adaptive automation, which uses
psychophysiological feedback from the user (e.g. airline pilots or military personnel) to assess
2

psychological states such as engagement and cognitive workload in order to provide assistance
when a lack of focus or an overload of task difficulty occurs (Allanson & Fairclough, 2004;
Byrne & Parasuraman, 1996; Middendorf et al., 2000). Additional applications include brain-
computer interfaces, relying mainly on the psychophysiological measure of
electroencephalography, that have been utilized to assist patients with motor disorders, providing
what has been called a “mental prosthesis” (Donchin et al., 2000; Farwell & Donchin, 1988).
Psychophysiological computing has also been used to vary task difficulty to improve training
scenarios and create intelligent tutoring systems (Berka et al., 2004; Chaudry et al., 1999; Coyne
et al., 2009; D’Mello et al., 2005).
Psychophysiological computing represents a means of creating for the computer system a
more empathic link to the human user. The goal is to allow for the computer to have an
understanding of the participant’s current state and to adjust and adapt to better address the
specific needs of that participant. Allanson (2002) notes that much of human-human interaction is
influenced by largely unconscious emotional cues, which are unavailable to typical computing
systems, but which can be tapped into through psychophysiological assessment to provide a
computer with some of the same knowledge that allows humans to intelligently interact with
other humans.
It should be noted that while the current research reported herein does not attempt to
create a fully functional psychophysiologically-based adaptive system, data collected and analytic
techniques performed will facilitate future adaptive system development. Adaptive virtual
environments (VEs) can be useful in training scenarios. For example, completing training tasks in
a VE containing high levels of threat may lead to performance decrements and hinder optimal
decision-making abilities when the perceived threat is too great. However, exposure to a
simulation environment that adapts to the current psychological state of the user can
3

incrementally raise the amount of threat the user can endure while maintaining the ability to
generate behaviors promoting optimal performance. While adaptive VEs based on
psychophysiological response patterns lend themselves well to a number of training applications,
military scenarios provide a concrete example of the utility of such systems and were the focus of
the research described herein. In order to work toward an effective adaptive system, researchers
draw upon numerous characteristics of the participant’s state. Two such state indices of critical
importance to a military-relevant training system are found in response patterns elicited by
varying levels of threat and cognitive workload. A thorough understanding of the
psychophysiological response patterns associated with these psychological states is important for
successful adaptive system design, which is a key incentive for these psychological states to be
manipulated and investigated. Thus, the current report is comprised of two separate experiments
carried out to investigate response patterns associated with varying levels of threat in the first and
cognitive workload in the second. As described in greater detail below, the responses elicited by
the variations in threat and cognitive workload in Experiments 1 and 2, respectively, were used to
predict an outcome measure related to participants’ performance navigating along a newly
learned route in a novel virtual city. The development of prediction models for performance
assessment represents a primary goal of the current research, with the added aim of facilitating
the advancement of future applications of psychophysiologically-based adaptive systems.
Response Patterns Associated with Exposure to Threat
Monitoring and recognizing response patterns associated with threat responses is crucial,
as performance is likely to wane if the participant’s arousal level is too high, causing stress, or too
low, leading to boredom. The goal of adaptive training simulations is to create a “flow” state.
According to Csikszentmihalyi (1990), “flow” is best understood as an optimal state of
consciousness characterized by a state of intense focus that causes one to become completely
4

absorbed within an activity. An understanding of the response patterns that predict optimal
performance is needed to develop systems with the ability to maintain a flow state for individuals
exposed to the training. The current study involves manipulation of the participant’s arousal with
varying levels of threat, which is likely to elicit psychophysiological responses associated with
stress and fear. Below, an account of typical responses associated with exposure to threatening
stimuli is presented. Response patterns related to each psychophysiological measure included in
the current research are discussed.
Electrodermal responses to threat. Changes in the activity of the eccrine sweat glands
are responsible for fluctuations in electrodermal activity. The eccrine sweat glands provide an
atypical and useful index of autonomic functioning as they are innervated exclusively by the
sympathetic nervous system. Other measures, such as cardiovascular responses, are influenced by
parasympathetic vagal tone as well. Thus, electrodermal responding can be utilized as a
peripheral measure of sympathetic activation (for review see Dawson, Schell, & Filion, 2007).
Electrodermal responses correlate linearly with arousal (Lang, 1995) and are influenced by both
emotional responses and cognitive activity (Boucsein, 2012).
Of particular interest in the present context is the way skin conductance responses are
affected by fear inducing or stressful stimuli. Öhman & Soares (1994) found that subjects who are
highly fearful of a certain biologically prepared stimulus (i.e. pictures of snakes or spiders) will
exhibit potentiated electrodermal responses when presented with pictures of their specifically
feared stimulus. Likewise, Globisch et al. (1999) found that fearful subjects showed increased
skin conductance responses to feared stimuli relative to neutral and positive stimuli. Another
study found that compared to neutral films, fear inducing films led to increases in skin
conductance level (SCL), in the rate of spontaneous fluctuations (SFs; also referred to as non-
specific skin conductance responses), and in skin conductance response (SCR) amplitudes when
5

subjects viewed the fear inducing films (Kreibig, Willhelm, Roth, & Gross, 2007). Amygdala
activation has also been demonstrated to co-occur with skin conductance responses when subjects
view fearful faces (Williams et al., 2001). Additionally, individuals with a fear of flying
evidenced a greater number of SFs and higher SCLs during flight than control participants
(Willhelm & Roth, 1998). These findings indicate that greater levels of fear responding result in
SCR potentiation and increased SCLs.
Electrocardiographic responses to threat. Heart rate responding during highly arousing
and fearful situations is generally associated with defensive responding, which results in
increased heart rate, as opposed to orienting responses which reduce heart rate (Fredrickson,
1981; Bernston et al., 1991). Van Oyen, Witvliet, and Vrana (1995) found the greatest increase in
heart rate acceleration when startle probes were presented during a high arousal mental imagery
task compared to low arousal imagery. Fearful subjects exposed to prolonged fear inducing
situations have also been evidenced to maintain a heightened heart rate compared to controls
during the exposure period (Willhelm & Roth, 1998). Additionally, numerous studies have found
that phobic subjects will exhibit an accelerated HR when viewing feared images, while control
subjects will exhibit an orienting response coupled with a HR deceleration to the same negative
images (e.g., Elsesser, Heuschen, Pundt, & Sartory, 2006). Thus, increased arousal elicited
specifically by fear inducing and not merely negative stimuli tends to result in heart rate
increases. Additionally, power spectral density analyses performed on heart rate variability
measures can provide a marker of sympathetic activation in the low frequency (LF) component of
the power spectrum (Malliani, Pagani, Lombardi, & Cerutti, 1991), which may provide a useful
index of emotional responding (see Appelhans & Luecken, 2006 for review). Accordingly,
cardiovascular responding can be an informative index of fear elicitation, which can be useful
when designing an adaptive system meant, for example, to train military personnel to exhibit
6

optimal cognitive performance in highly threatening situations (e.g., Wu et al., 2010). Patterns of
heart rate response related to fear induced arousal can also be useful when classifying fearful
subjects based on psychophysiological responses. One study was able to classify highly fearful
subjects compared to controls with 88% accuracy based on HR differences alone when
participants were exposed to a tunnel driving VR simulation (Mühlberger, Bülthoff, Wiedemann,
& Pauli, 2007).
Respiratory responses to threat. Respiration rate has consistently been shown to increase
in response to heightened levels of arousal associated with fear (for review, see Boiten, Frijda, &
Wientjes, 1994). Etzel et al. (2006) found that music clips that participants rated as fearful led to
significantly increased rates of respiration compared to sad music clips. Likewise, responses to
fear inducing film clips have been demonstrated to significantly increase respiration rates
compared to sad and neutral film clips (Kriebig et al., 2007). Mental imagery of fearful events has
also been shown to increase rates of respiration (Rainville et al., 2006). Participants anticipating
eminent shock will exhibit increased rates of respiration (Suess et al., 1980).
Pupil dilation and threat. Pupil dilation tends to operate in a similar fashion to
electrodermal activity in relation to fear induced arousal. Negative sounds have been associated
with increased pupil dilation, and subjective ratings of arousal were also positively correlated
with pupil dilation (Partala & Surakka, 2003). Indeed, electrical stimulation of the central nucleus
of the amygdala, resulting in a fear response, leads to pupil dilation (Davis, 1997).
Response Patterns Associated with Changes in Cognitive Workload
Similar to responses to threat, cognitive workload is an important participant state feature
to monitor as extremely high levels can lead to frustration, while again, low levels may cause
boredom. An optimal level of cognitive workload is sought by adaptive training systems; thus
7

responses to variations in cognitive workload were examined in Experiment 2. A brief review of
the same four response systems is presented below as they relate to cognitive workload.
Electrodermal activity and cognitive workload. Skin conductance will generally increase
as workload increases. During the Stroop task, incongruent stimuli, associated with a higher
degree of task difficulty than congruent stimuli, will elicit larger SCR amplitudes (Kobayashi et
al., 2007). Additionally, participants in the Stroop study evidenced larger SCRs when responding
to a stimulus incorrectly, and longer response times were associated with larger SCRs as well.
These findings lend credence to the notion that EDA provides a measure of overall activation, as
more difficult tasks created a greater cognitive workload and resulted in increases in the SCR.
Increased task difficulty using an n-back task also results in increased skin conductance levels
(Mehler et al., 2009). Another study examined the time interval between SFs as a measure of
workload assessment while participants drove a car and performed cognitive tasks (Verwey &
Veltman, 1996). SF rate increased significantly when participants performed the dual task of
driving while completing cognitive tasks compared to baseline responding and to driving with no
additional cognitive task. These findings suggest that SFs might be a useful measure of workload
assessment for adaptive simulation technologies. It is important to note, however, that it may be
wise to utilize EDA as a monitoring measure in conjunction with other measures of workload, as
EDA will increase in response to a number of stressors, and it is not always possible to attribute
activation to workload alone. For example, Gendolla and Krusken (2001) found that mood states
interacted with task difficulty to determine the amount of effort likely afforded to task
completion, and increased effort raised SCLs.
Electrocardiographic activity and cognitive workload. The Paced Auditory Serial
Addition Task (PASAT), which is the mental arithmetic task employed in Experiment 2, has been
found to result in increased heart rate at onset of the task, and elevated rates throughout task
8

completion compared to baseline (Ring et al., 1999). Additionally, numerous studies using
various cognitive tasks have evidenced increased heart rate associated with increased cognitive
workload (e.g., Carroll et al., 1986; Kennedy & Scholey, 2000; Mehler et al., 2009; Sloan,
Korten, & Myers, 1991). Participants in the previously mentioned Verwey and Veltman study
evidenced reduced interbeat intervals (IBIs), which equates to increased heart rate, when
performing the dual task of driving and completing cognitive tests. Fournier et al. (1999)
corroborated these findings as participants subjected to a challenging multi-task condition
exhibited increased heart rate compared to single-task conditions. Moreover, Fournier et al. found
that measures of heart rate variability (HRV) were also sensitive to increased cognitive workload
such that the high frequency (HF) component of the HRV power spectrum, which has been linked
to parasympathetic vagal tone (Malliani, Pagani, Lombardi, & Cerutti, 1991), was reduced in
multi-task conditions.
A recent study utilized heart rate as input to an adaptive system to create change in the
challenge level of a video game (Gilleade & Allanson, 2003). Decreases in heart rate were
interpreted as a lack of challenge, which could lead to boredom. Heart rate decreases would
trigger an increase the challenge level of the game. This is an example of how knowledge of the
response patterns associated with heart rate change can serve to inform the development of an
adaptive system.
Respiration and cognitive workload. An increase in respiratory rate has been consistently
associated with increased cognitive demand (e.g., Backs & Selijos, 1994; Brookings, Wilson, &
Swain, 1996; Mehler et al. 2009). Backs and Selijos (1994) found that rates of respiration
increased as task difficulty increased in a working memory task. During an air traffic control
simulation with three levels of task difficulty, air traffic controllers exhibited increased rates of
respiration as task difficulty increased (Brookings, Wilson, & Swain, 1996). Respiration rate also
9

increased when participants performed a difficult multi-task condition when compared to a
single-task condition (Fournier et al, 1999). In a study involving a driving simulation which
included both low and high levels of secondary cognitive task conditions, respiration rate was a
sensitive measure of workload intensity (Mehler et al., 2009). During the low level task difficulty,
respiration rate did not increase above baseline driving conditions, and performance on this task
was nearly perfect across participants. However, during the high difficulty task condition,
respiration rate increased and driving performance waned. These findings demonstrate the
effectiveness of respiration as a means of monitoring workload, and provide support for the
incorporation of respiration rate as an indicator of overload in future adaptive automation
research design.
Pupil dilation and cognitive workload. Pupil dilation is often utilized as an index of
cognitive workload and task difficulty. The Index of Cognitive Activity (ICA) was created to
measure the dilation reflex, which is a response caused by the presence of a cognitive stimulus, as
opposed to the light reflex, which is caused by changes in the light source (Boehm-Davis et al.,
2003). The two reflexes are controlled by different muscle groups. The dilation reflex results in
activation of the radial muscles and inhibition of the circular muscles, creating a burst of dilation
larger than either muscle group could produce alone. Numerous studies have reported increased
pupil dilation in response to increased cognitive demand (Beatty & Wagoner, 1978; Pomplun &
Sunkara, 2003; Porter, Troscianko, & Gilchrist, 2002; Schaefer et al., 1968).
To summarize, psychophysiological metrics provide a means of obtaining objective and
ongoing measures of user-state through noninvasive and non-conscious methods. Arousal and
cognitive workload are two aspects of participant-state that provide vital information for the
successful implementation of adaptive systems that can be applied to improve real-world
performance. However, testing such systems in real-world environments can be dangerous and
10

costly, especially when military-relevant scenarios are involved. Virtual reality (VR) scenarios
offer the potential for simulated environments to provide cogent and calculated response
approaches to real-time changes in user emotion, cognitive state, and motivational processes. The
value in using simulation technology to produce VEs targeting such processes has been
acknowledged by an encouraging body of research.
VR Simulations for Adaptive System Development
The incorporation of simulation technology into neuroergonomic and
psychophysiological research is advancing at a steady rate (see Parasuraman & Wilson, 2008).
The range and depth of these simulations cover a large domain, from simple low fidelity
environments to complex fully immersive simulators. All of these simulators rely on some type of
representation of the real world.
The determination of the appropriate level of scenario fidelity is relative to the questions
asked and the population studied. However, it is often important to consider the extent to which
level of fidelity impacts the participants’ experience of presence (Slater et al., 2009). Presence is
defined as the propensity of users to respond to virtually generated sensory stimuli as if they were
real (Sanchez-Vives & Slater, 2005). Researchers seek to quantify presence by measuring
responses evoked by stimuli in immersive VEs. A low fidelity virtual environment may be
preferable in studies where a maximal amount of control is desired because such environments
may increase psychometric rigor through limiting the number of sensory variables available.
However, high fidelity environments are preferable for studies desiring increased ecological
validity as they more closely resemble reality and better capture the participant’s performance as
it would occur in actuality. Leeb et al. (2006) reported that participants who were immersed in a
high fidelity virtual environment reported a greater sense of presence and exhibited higher
performance rates than when the same task was presented on a two-dimensional computer
11

monitor display. Effective adaptive systems for simulation applications are characterized by those
systems that facilitate high levels of presence, as this allows the participant to experience greater
levels of absorption in the simulation. The current study utilized a high fidelity, highly immersive
virtual environment, as increased applicability to real-world performance was the goal. The VR
system used was considered highly immersive because of the employment of high fidelity visual,
auditory, and haptic stimuli, which are described in greater detail below. Specifically, the VE
utilized in the current research was that of a virtual Iraqi city, which allowed for simulation of
route-learning and wayfinding, to assess landmark and route knowledge of the VE.
Navigation in VEs
Navigation learning in VEs is typically discussed in the same manner as is characteristic
of real-world navigation learning. In line with this notion, studies have conveyed the benefits of
ecologically valid simulated navigation tasks as predictors of real-world functioning (Nadolne &
Stringer, 2000; Waller, Hunt, & Knapp, 1998). Navigation abilities are customarily broken down
into three knowledge based components, each adding to the cognitive map developed by the
participant. The first is landmark knowledge, which involves learning to recognize landmarks or
salient features of the environment upon initial exploration of said environment (Golledge, 1991).
In the current study, zone markers indicating the entrance into a new zone were the key
landmarks involved (Fig. 1). The second component is referred to as procedural or route
knowledge and involves information gleaned from first-hand experience with a route. Procedural
knowledge of the environment differs from landmark knowledge in that landmark knowledge
simply refers to forming memories of the landmarks themselves, whereas procedural knowledge
allows the participant to form an idea of how these landmarks relate to each other spatially. This
provides the ability to create distance and orientation relationships enabling the identification of
routes connecting landmarks (Golledge, 1991; Thorndyke & Hayes-Roth, 1982). Real-world and
12

VR experiments suggest that active navigation, which was utilized in the current research, is more
effective for route-learning than passively being exposed to the environment (Hahm et al., 2006).
The third component of navigation ability is referred to as survey knowledge, which can be
described as having developed a “bird’s eye view” of the environment. Survey knowledge affords
the development of a cognitive map that provides associations between locations with increased
levels of exposure to the environment (Golledge, 1991; Ramloll & Mowat, 2002). Survey
knowledge is valuable as a means of finding shortcuts through the environment, but is not
necessarily useful in the present study, as participants were instructed to follow a specific route
without deviating. Thus, this study is primarily concerned with landmark and route knowledge.
The current research design afforded the opportunity to investigate the effects of
exposure to threat and cognitive workload on route-learning in Experiments 1 and 2, respectively.
To our knowledge, no study has examined the effects of varying levels of threat on route-
learning, making this a novel approach. The effects of a secondary cognitive workload task tend
to interfere with route-learning. Walker and Lindsay (2006) reported decreased efficiency in
wayfinding performance in a virtual city when a secondary speech discrimination task was
introduced. They postulate that this was due to the switch of attentional resources to the
completion of the secondary task. A similar result was found in a between-subjects study
involving examination of the effects three separate types of cognitively distracting tasks presented
during the route-learning phase compared to a no task condition. All groups that experienced
distracting tasks performed less efficiently on a wayfinding task than the group that was not
presented with any distracting task (Meilinger, Knauff, & Bülthoff, 2007). Knowledge of
psychophysiological states gleaned during the route-learning phase may serve as an indicator of
wayfinding abilities. For example, participants with lower psychophysiological response levels
during the route-learning phase may prove more efficient during the navigation phase.
13

A secondary benefit of inclusion of the navigation task in the current study was the
opportunity to investigate simple and complex conditioning with the threat level manipulation in
Experiment 1.
Conditioning in VEs
Single-cue, or simple, fear conditioning paradigms pair a previously neutral stimulus,
which becomes the conditioned stimulus (CS), with a naturally aversive stimulus, or
unconditioned stimulus (US). After repeated trials in which the CS and US are paired (the
acquisition phase), the CS alone becomes sufficient for eliciting a fear response (CR) in the
absence of the US. If the CS is repeatedly presented without the US (extinction phase), the fear
response elicited by the CS will subside. This is the basis for exposure therapy for treatment of
phobias and post-traumatic stress disorder. However, phobias and post-traumatic stress symptoms
are known to spontaneously return after the passage of time (Rachman & Lopatke, 1988),
suggesting that the extinction of the fear response may be context dependent. Indeed, animal
research supports this view. Rats that acquire a fear response to a CS in a specific context, and
then experience an extinction phase in a different context (i.e., a different cage), will show a
renewed fear response to the CS presented alone when returned to the cage in which acquisition
occurred (Bouton & Bolles, 1979). Studies of contextual effects in conditioning paradigms have
given rise to the notion that extinction involves the learning of a new competing memory rather
than the unlearning of the original fear memory created during acquisition (Alvarez et al., 2007).
Certain contexts and the passage of time can thus weaken the competing extinction memory,
leading to spontaneous recovery of the original fear memory. This line of research is of great
clinical importance for the understanding of anxiety and fear-related disorders, as it is suggested
that contextual conditioning responses are more closely related to a sustained anxiety response
than responses elicited during simple conditioning (Davis, 1998).
14

A possible underlying cause for contextual conditioning is that associations are learned
between complex arrays of stimuli associated with the environment in which the conditioned
responses are acquired. To investigate further, an examination of whether participants in the
current study were able to make associations between more complex stimuli was undertaken.
More specifically, tonic response levels elicited during areas of the virtual city that were
associated with either high or low threat stimuli were examined. The physical characteristics of
the high and low threat zones served as the complex CSs, whereas explosions and other high
threat stimuli present during acquisition (i.e., the guided tour task) functioned as complex USs.
Observations were then made as to whether participants were conditioned to the complex CSs
during the extinction phase (i.e., the navigation back task), in which high threat stimuli were no
longer present. It is possible that when individuals are more susceptible to complex conditioning,
they may be at greater risk for the development of posttraumatic stress symptoms. While the
present research is unable to conclude whether complex conditioning is linked to increased
symptomatology, varying levels of susceptibility to complex conditioning may help guide future
research toward prodromal signs of disorder.
A number of studies have been conducted to test the effects of complex conditioning in
humans, providing empirical evidence that such conditioning can occur. Baas et al. (2004)
presented a stimulus that was paired with shock (CS+) in one room (the shock room) of a virtual
environment, but was not paired with shocks in another room (the safe room). Participants
evidenced potentiated startle responses to the CS+ in both rooms, but when there were no CSs
presented during extinction, participants responded with greater potentiated startles in the shock
room, leading to the conclusion that contextual conditioning had occurred. These results have
been validated by numerous studies (e.g., Alvarez et al. 2007; Alvarez et al., 2008; Grillon et al.,
2006; Milad et al., 2005).
15

The complex conditioning paradigm employed in the current study provides insight into
the development of post-traumatic stress symptoms, due especially to the military-relevant
simulation utilized herein. Differences in response during acquisition to simple and complex
stimuli may exist in participants who exhibit increased psychophysiological responses during the
extinction phase (i.e., the navigation task) of the experiment, suggesting heightened complex-cue
related anxiety. These differences may offer indications of susceptibility toward development of a
stress or anxiety related disorder. This type of information would be highly relevant to the
development of an adaptive system aimed at providing aid to military service members who
experience high levels of fear and anxiety in combat situations.
Toward Adaptive System Development
The current research was concerned with informing psychophysiological computing
strategies for creation of VEs capable of adapting to the participant’s affective and cognitive state
to foster optimal performance. Psychophysiological computing represents an innovative mode of
HCI wherein system interaction is achieved by monitoring, analyzing and responding to covert
psychophysiological activity from the user in real-time (Parsons et al., 2009; Allanson &
Fairclough, 2004). Psychophysiological computing represents a means of creating for the
computer system a more empathic link to the user.
The strategy employed herein for creation of a psychophysiological computing system
initially required assessment of psychophysiological response patterns associated with varying
cognitive and affective states. The current research manipulated environmental threat and
cognitive workload to create response variability in order to perform such assessments. Data
analytic approaches designed for prediction were then compared and tested for effective
development of a psychophysiological computing system capable of predicting performance
outcomes. Namely, the efficacy of multiple linear regression (MLR) and artificial neural network
16

(ANN) models were compared for prediction of navigation performance based on
psychophysiological responses to threat and cognitive workload during the route-learning phase
of the experiment.
Threat and cognitive workload were manipulated in the current research for a number of
reasons. First, psychophysiological response patterns associated with responses to manipulations
of threat and workload can be used to inform adaptive systems concerned with the monitoring of
cognitive and affective user-states, which are two of the most widely researched for providing
adaptive automation (e.g., Allanson & Fairclough, 2004; Byrne & Parasuraman, 1996; Gilleade &
Allanson, 2003; Middendorf, McMillan, Calhoun, & Jones, 2000). Furthermore, responses to
varying levels of threat and cognitive workload generally lead to increased physiological
activation; however, separate examination of the response patterns related to each may expose
subtle differences that may prove informative when designing adaptive systems to promote
optimal performance. The current research investigated which response measures are more
predictive of navigation performance and developed models for prediction of performance based
on said responses. Future adaptive systems employing some of the methods developed here will
have the goal of facilitating response patterns associated with effective learning and optimal
performance outcomes. If performance outcomes can indeed be predicted based on
psychophysiological responses gleaned during the learning phase of a training simulation, proper
adjustments can be made to the simulation to foster a level of challenge suited for the individual
user. In the present studies, participants were exposed to varying levels of threat and cognitive
workload while concurrently completing a route-learning task, and the responses collected were
submitted to MLR and ANN models to predict performance on the subsequent test of route-
learning efficacy during a navigation task.

17

Multi-layer Perceptrons & Multiple Linear Regression
In MLR one or more additional variables are added to a simple linear regression that
may be useful as predictors. In the ANN literature, one finds a multi-layer perceptron as an
analogous, though nonlinear, version of the MLR (Sarle, 1994). The MLR fits a criterion as a
linear combination of multiple predictors by the method of least squares. Hence, the objective is
to find the best linear fit. However, there may be cases in which data points are not evenly
distributed around the regression line. Further, the data spread may be such that it would be better
represented by a curve than a straight regression line. The multi-layer perceptron allows the ANN
researcher to develop such a curve fitting due to the addition of a layer of hidden nodes that have
suitably curved activation, or transfer, functions. The most widely used activation functions are
sigmoid functions (see Erb, 1993), such as the hyperbolic tangent (tanh) function used in the
current study. The predominately utilized multi-layer perceptron is the feed-forward
backpropagation network, commonly referred to as a backpropagation network (BPN).
Backpropagation was introduced by Werbos (1974), but was popularized by Rumelhart and
colleagues (1986).
The multi-layer perceptron structure is an ANN with one or more layers of nodes
between the input and output nodes, called hidden layers. Inputs are analogous to predictor
variables in MLR, and output nodes represent the predicted values. The additional hidden layers
contain nodes that are directly connected to both the input and output nodes. While multiple
hidden layers may be employed, one hidden layer is typically sufficient for adequate mapping,
even for continuous outcome measures (Lippman, 1987). The perceptron offers the advantage of
using nonlinearities within nodes. Further, in cases where more than one output nodes and
sigmoidal nonlinearities are used, the backpropagation training algorithm can be used for gradient
training. The backpropagation algorithm uses the gradient descent technique (see Baldi, 1995;
18

Rumelhart, Hinton, & McClelland, 1986) to minimize an error function equal to the mean square
difference between the desired and the actual net outputs. When a gradient descent learning
algorithm is employed, random initial values are chosen for the model’s weight parameters. Next
the gradient of the error function is calculated with respect to each model parameter. The model
parameters are changed so that a short distance is moved in the direction of the greatest rate of
decrease of the error. This process is repeated until the gradient of the error function gets close to
zero. At this point, the algorithm has “converged”.
To simplify, the BPN finds a set of weights and values that minimizes the error across the
input and output data pairs. When the network undergoes training, differences between the actual
outputs and those predicted by the model are disseminated back through the network to create
adjustments in the weights on the output units. Finally, a backpropagation of output unit error
through the weights determines the hidden nodes and their weights are altered. This is a recursive
process that occurs until the error is at a low enough level (see Bishop, 1995; Hecht-Nielsen,
1989 for review). Following training a separate set of inputs and outputs, not used during training
of the network, are applied to test the generalizability of the model.
Herein, both MLR and BPN models were employed to predict navigation performance
based on responses elicited by variations in threat and cognitive workload experienced during the
route-learning phase of the Experiments 1 and 2, respectively. The design parameters of these
models are described below, as will the methods for comparison of predictive abilities associated
with each. Finally, results are discussed in terms of possible application to adaptive training
systems.

19

Chapter Two: Methodological Overview
Participants
A total of 53 participants took part in Experiment 1, whereas 50 participated in
Experiment 2. Each experiment contained two groups, those who were told in advance that they
would be navigating back through the city and those who were not told in advance. Group
assignment was semi-random, as gender proportions in each group required balancing.
Participants were recruited through the psychology subject pool at the University of
Southern California. Inclusion criteria included normal or corrected to normal vision, and English
fluency. Participants were between the ages of 18 and 35.
Materials
A virtual environment depicting an Iraqi city was presented to participants with use of an
eMagin Z800 head mounted display complete with head tracking capabilities to allow the
participant to explore the environment freely. The virtual environment was created using graphic
assets from Full Spectrum Warrior, using the Gamebryo graphics engine to create the
environment. A tactile transducer floor was utilized to enhance the ecological validity of the VE
by making explosions and other high threat stimuli feel more lifelike. Auditory stimuli were
presented with a Logitech surround sound system. Psychophysiological measures related to
electrodermal, respiratory, and electroencephalographic activity were collected using a Biopac
MP150 system. Pupil dilation responses were collected with a binocular Arrington ViewPoint
eyetracking system, made specifically for recording within an eMagin Z800 head mounted
display. Participants experienced the VE while residing in an acoustic dampening chamber, which
had the added benefit of creating a dark environment to remove any peripheral visual stimuli that
were not associated with the VE, resulting in increased immersive qualities of the simulation.

20

Procedural Overview
Upon arrival at the laboratory, following the consenting procedure, participants were
given the pre-test questionnaires, described in detail below. Next, participants were fitted with all
necessary psychophysiological recording equipment and the head mounted display device.
Following a baseline procedure, participants were exposed to either the threat
manipulation task or the cognitive load manipulation task. Each experimental task consisted of
following a guide through six zones that alternated between high and low levels of threat or
cognitive workload. All environmental stimuli were pre-scripted, allowing each participant to
experience exactly the same environmental stimuli at the same time to enhance experimental
control of stimulus presentation. The high and low zone presentation order was counterbalanced
across subjects as to which type of zone was experienced first. Participants in Experiment 1
experienced the threat manipulation task, while those in Experiment 2 were subjected to the
cognitive workload manipulation task. Both of the manipulation tasks were followed immediately
by the navigation task in which the participants were asked to return to the starting point of their
tour through the city.

21

Chapter Three: Experiment 1 Overview
Purpose
The primary purpose of Experiment 1 was to examine differences in psychophysiological
response patterns brought about by varying levels of threat in a VE, and to determine the effect
that varying levels of threat had on route-learning and wayfinding through the VE. The response
patterns caused by the exposure to threatening stimuli were then used to train an ANN to predict
navigation performance. Additionally, simple and complex conditioned responses were examined
to further elucidate the differences in the navigation performance outcome measures.
Design
The threat level manipulation in Experiment 1 employed a 2 (threat level; high vs. low)
by 3 (zone pair) by 2 (navigation awareness; aware vs. unaware) mixed design. Both threat level
and zone represented within-participant variables, while the navigation awareness variable was
between-participants. The zone variable was included to determine trial order effects. Participants
were told of the navigation portion of the experiment either before entering the VE or only after
the initial tour through the city upon completing the threat level manipulation portion of the
experiment. Those told before entering the VE were referred to as navigation aware (NA) and
those told only after the initial tour through the city was complete were referred to as navigation
unaware (NU). During the high threat zones (see Figure 1), participants experienced an ambush
situation in which bombs, gunfire, screams and other visual and auditory forms of threat were
present, whereas none of these stimuli were presented in the low threat zone (see Figure 2). Upon
the return trip through each zone during the navigation task, psychophysiological response levels
during each zone type were assessed to determine the effects of complex conditioning.

22

A secondary investigation pertaining to Experiment 1 related to the embedded simple
conditioning paradigm. A 2 (CS typ; CS+ vs. CS-) by 3 (zone pair) by 2 (navigation awareness)
mixed design was employed, where CS type represented a within-participant variable while
navigation awareness served as a between-participants variable. At the beginning of each high
threat zone, a red zone marker (CS+) was consistently paired (100% reinforcement) with an
aversive electrical stimulation delivered to the participant’s forearm (US). Upon arrival to each
low threat zone, a yellow zone marker (CS-) was passed through, and was not paired with any
electrical stimulation. Anticipatory responses to the CS+ and CS- were examined during both the
threat manipulation portion of the test and the navigation portion of the test.

Figure 1: Example of a High Threat Zone.
23

Figure 2. Example of a Low Threat Zone.
Participants
A total of 53 students (67.9% female; mean age = 19.79; age range = 18 to 22) from the
psychology subject pool at the University of Southern California served as participants. A high
threat zone was experienced first by 27 of the participants, while 26 of the participants were first
exposed to a low threat zone. The NA group consisted of 28 participants (67.9% female; mean
age = 20.04; age range = 18 to 22), while the NU group totaled 25 participants (68.0% female;
mean age = 19.52; age range = 18 to 22). There were no significant differences relating to gender
or age between groups. All participants signed an informed consent form approved by the
University of Southern California’s Institutional Review Board.
Procedures
Pre-test questionnaires. The first questionnaire administered was the Pre-Exposure State
Checklist which was used to determine the participant’s levels of discomfort and fatigue before
24

entrance into the VE (see Appendix A for complete questionnaire forms). This includes a list of
possible symptoms of fatigue and discomfort, and the participant is instructed to circle the option
that best describes the extent to which each symptom is affecting them right now. The options
include “none,” “slight,” “moderate,” and “severe.” It is important to assess whether the VE
caused any discomfort during the test, and knowledge of the participant’s symptoms relating to
fatigue and discomfort prior to exposure allows for assessment of the changes in symptomatology
following exposure.
Next, participants were asked to complete the Participant History Form. This form asked
for basic demographic information as well as information about the participant’s mental health
and medication regimen. The form also asked the participant to rate his or her familiarity with
computers on a 9 point Likert scale. Certain mental health disorders or use of certain medications
known to affect psychophysiological response would lead to post-hoc exclusion from analyses,
meaning that no participant was turned away from participating in the study based on responses
to this form. It should be noted that responses to this form did not actually lead to the exclusion of
any participants.
Psychophysiological equipment setup. The psychophysiological recording equipment
included two sensors attached to the volar surface of the distal phalanges of the index and middle
fingers of the non-dominant hand to measure skin conductance responses. Electrocardiographic
information was recorded using a Lead 1 electrode placement. An elastic band was placed around
the participant’s chest to measure rate of respiration. Eyetracking equipment, used here to
measure pupillary responses, was embedded in the head mounted display used to view the VE.
Electrical stimulation work-up. Participants had two electrodes attached to the left
forearm which were used to deliver an electrical stimulation at four points during the entire
experiment. One such stimulation occurred as the participant passed through the marker for the
25

high threat zone during the baseline run and the other three occurred as the subject passed through
the markers for the three high threat zones during the threat manipulation task. No electrical
stimulations were experienced during the navigation task. The experimenter assisted the
participants in setting their own acceptable level of stimulation. A stimulation work-up procedure
was employed in which participants began with a sub-threshold level of stimulation that was
increased in small increments until the participant indicated that the electrical stimulation was
“annoying, but not painful.”
Baseline. During the baseline procedure, the participants followed a virtual soldier (the
guide) down a street in the virtual Iraqi city not experienced during the experimental task. The
participants were instructed that they were to follow closely behind the guide and that they would
experience some areas that were safe and others which were more dangerous. The baseline
procedure was split into three zones. Each zone lasted for 30 seconds. The first zone was a low
threat zone that was used as an opportunity for habituation to allow the participant to adjust to the
novelty of being in a VE. The second zone was another low threat zone that was meant to assess
the participant’s lowest levels of psychophysiological response while immersed in the VE. The
third zone was the high threat zone, meant to assess the participant’s maximum levels of
psychophysiological response. Each zone was preceded by a zone marker, which was used to
denote entrance into a new zone. Low threat zones were preceded by green zone markers while
the high threat zone was preceded by an orange marker. When participants crossed the orange
high threat marker, an electrical stimulation was delivered, followed immediately by the high
threat stimuli which include, for example, bombs, missiles, gunfire and screams. The baseline
procedure was utilized to familiarize participants with the experience of being immersed in a VE
and to allow for practice of the task requirements.
26

Threat manipulation task. During the threat manipulation task, participants followed a
team of three soldiers, which served as the guides through the six zones. The zones alternated
between high and low levels of threat.
All participants were given the following instructions, made to sound like radio
communication, prior to starting the task: “Ok soldier, your task is to make it through this city
alive. You’ll be following a predetermined path that includes markers along the way, just like the
one you see in front of you now. Stay close to the soldiers in front of you, some of the areas
you’ll be passing through are extremely dangerous. It is imperative that you follow them closely
at all times. Now move out.”
At this point, participants in the NU group began their tour through the city. Participants
in the NA group were given these additional instructions, which were received just prior to the
command to move out: “After reaching your goal, you will be asked to return to this beginning
point where you currently stand, so you’d better pay attention to the route that you take. You will
have to pass back through the city, passing all of the zone markers in the reverse order.”
High threat zones were preceded by red zone markers (Figure 3), and participants
received an electrical stimulation when crossing these red markers. Low threat zones were
preceded by yellow zone markers (Figure 4) and were not paired with electrical stimulation. The
type of starting zone was counterbalanced across participants where half of the participants in
each group experienced a high threat zone first and the other half initially experienced a low
threat zone.
27

Figure 3: Example of a Red Zone Marker. This is indicative of the start of a high threat zone in
Experiment 1 or a high cognitive workload zone in Experiment 2. The red zone marker also
serves as the CS+ in the cue-specific conditioning portion of Experiment 1.
Red zone markers served as the cued CS+, as they were always paired with the aversive
electrical stimulation (US). The yellow zone markers served as the CS-, and were never paired
with the US. It should be noted that zone markers in the baseline run were colored differently
than the zone markers in the threat manipulation task. This was done to prevent participants from
learning the association between the red colored markers and the electrical stimulation as
conditioning was not assessed during baseline.
28

Figure 4: Example of a Yellow Zone Marker. This indicates the beginning of a low threat or low
cognitive workload zone in Experiments 1 and 2, respectively. The yellow zone marker
represents the CS- in the cue-specific conditioning portion of Experiment 1.
Participants followed the guides to the “goal” zone marker marking the end of the tour,
which was blue in color, where they were then instructed to follow the guides around a nearby
corner to be given the instructions to begin the navigation task.
Navigation Task. After reaching the goal zone marker in the threat level manipulation
task, participants were led by the guides past the marker and around the first corner, where they
were given the following instructions: “Excellent work soldier, you’ve made it this far. From here
on out, you’ll be on your own. Your task is to return to starting point, being sure to return the
same way that you came. You’ll pass marker 6 first, then 5 and so on, until you reach the start.
We have men positioned to keep an eye on you throughout your return. If you stray too far from
29

the path, I’ll let you know and point you in the right direction. You should go as quickly as you
can, we’re not sure how stable these areas are. Your task begins now.”
Following the instructions, participants began to navigate back through the city. They
were to pass through each zone in reverse order until reaching the original starting point. If the
participant strayed too far from the path, which was quantified as the distance it would take to
walk for 10 seconds in a perpendicular direction from the original path, an arrow appeared in the
corner of the screen that assisted the participant in finding his or her way back to the original
path. The navigation task ended when the participant crossed the zone 1 marker.
During the navigation task, there were no longer any threatening stimuli presented in the
high threat zones. However, physiological responses were still collected and assessed during the
previously high and low threat zones to examine whether complex conditioning had occurred.
Additionally, anticipatory responses to the red and yellow zone markers were examined to assess
simple conditioning that had taken place during the threat manipulation task. The CS+ was no
longer paired with the US during navigation back through the city, thus the navigation task was
utilized as the extinction phase, whereas the threat manipulation task served as the acquisition
phase.
Post-test questionnaire. Finally, participants completed the Post-Exposure State
Questionnaire. The content of this questionnaire was identical to the pre-exposure questionnaire
and was used to assess changes in fatigue and comfort level following exposure to the VE. In
addition, this questionnaire asked for information regarding whether the participant felt like he or
she experienced motion in the virtual environment, i.e., the degree to which they felt as though
they were actually moving. It also asked the participants to describe any unusual events
experienced in the VE and to rate their perceived performance in the VE on a scale of 1 (poor) to
10 (excellent).
30

Dependent Variables
The key dependent variables associated with the threat level manipulation task are the
psychophysiological responses exhibited during the high and low threat zones. See Table 1 for a
complete list of the psychophysiological features that were extracted during the threat
manipulation task.
Table 1
Dependent variables associated with the threat level manipulation task
Psychophysiological Response System Selected Features (Per Zone)

Electrodermal Activity

Skin conductance level
Number of spontaneous fluctuations
Electrocardiographic Activity Interbeat interval
Heart Rate Variability Low frequency component
High frequency component
Respiration Interbreath interval
Pupillary Activity Pupil diameter

During the navigation task the same psychophysiological responses were assessed in each
zone as participants returned to the starting point (Table 2). In addition, time spent in each zone
and the deviation from the original path was assessed to examine the speed and efficiency with
which each participant navigated through the city. The number of times an arrow pointing the
subject back to the original path was required was also used as a variable for assessing the
efficiency of the participants’ passage through each zone.

31

Table 2
Dependent variables associated with the navigation task
Response System Selected Features (Per Zone)

Electrodermal Activity

Skin conductance level
Number of Spontaneous Fluctuations
Electrocardiographic Activity Interbeat interval
Heart Rate Variability Low Frequency component
High Frequency component
Respiration Interbreath interval
Pupillary Activity Pupil diameter

Behavioral Measures Total time spent in navigation task
Distance deviated from the original path
Number of arrows needed for assistance

Dependent variables relating to the conditioning portion of the test included both phasic
and tonic psychophysiological responses. During the threat manipulation task, phasic anticipatory
responses to the sight of the CS+ and CS- were assessed. These responses are outlined in Table 3.
During the navigation task, the same anticipatory response variables were examined in addition to
tonic changes in mean physiological response levels during each zone. The difference in tonic
response levels during the high and low threat zones were used to assess the amount of complex
conditioning that had taken place (Table 4).
Table 3
Dependent variables associated with the simple conditioning task
Psychophysiological Response System Selected Features

Electrodermal Activity

Maximum amplitude skin conductance
response
Electrocardiographic Activity Interbeat interval
Respiration Interbreath interval
Pupillary Activity Maximum pupil dilation response

32

Table 4
Dependent variables associated with the complex conditioning task
Psychophysiological Response System Selected Features (Per Zone)

Electrodermal Activity

Skin conductance level
Number of Spontaneous Fluctuations
Electrocardiographic Activity Interbeat interval
Heart Rate Variability Low Frequency power spectral component
High Frequency power spectral component
Respiration Interbreath interval
Pupillary Activity Pupil diameter

Note. All data are taken from the extinction, or navigation, phase of the experiment.
Experiment 1 Hypotheses
Threat level manipulation task hypotheses. During the initial walkthrough, a general
increase in psychophysiological response during the high threat zones in comparison to the low
threat zones as a result of increased levels of arousal was hypothesized. This includes an increase
in heart rate (analyzed as a decrease in IBI), respiration rate (indicated by decreased interbreath
intervals), and SCL, as well as a greater number of SFs, and increased pupil dilation. The low
frequency (LF) component of HRV was expected to increase, while the high frequency (HF)
component would decrease in power.
It was also hypothesized that there would be between-subjects differences as a result of
the navigation awareness manipulation. Participants in the NA condition had the duel task of
following the guides and trying to remember their path as they walked through the city. Thus, a
greater level of cognitive load was present for those in the NA group. This added cognitive load
was expected to diminish the responses caused by the high threat zones, as less attention would
be given to the surrounding threatening stimuli. It was therefore expected that participants in the
NU group would exhibit a greater increase in psychophysiological response to the high threat
zones when compared to the low threat zones. This would represent an interaction between group
and threat level, as it was expected that the greater cognitive load would cause higher levels of
33

psychophysiological response for the NA group during the low threat zones, but lower response
levels during the high threat zones compared to responses from participants in the NU group.
Navigation task hypotheses. It was predicted that participants would spend more time in
the previously high threat zones and demonstrate a less efficient path through these zones. This
would include greater deviation from the original path, a greater amount time spent, and a greater
number of redirecting arrows needed in the previously high threat zones. It was anticipated that
visual and auditory stimuli associated with the high threat zones would serve as distracters,
causing participants to have a more difficult time navigating through these zones.
The navigation awareness manipulation was also predicted to result in a between-subjects
difference in navigation efficiency. Participants in the NA group would have greater procedural
knowledge of the path through the city because they were forewarned that they would need to
remember their path. It follows that this would lead to increased efficiency during the navigation
task overall. It was also hypothesized that there would be a greater disparity in navigation
efficiency between the high and low threat zones in the NA group leading to a group by threat
level interaction. NA participants would be more affected by the distracting stimuli than those in
the NU group because they were aware that they had a secondary task of remembering their route
through the city. Thus, the NU group would be less efficient regardless of the type of zone they
were experiencing compared to the NA group.
Conditioning hypotheses. The conditioning portion of Experiment 1 involved both simple
and complex conditioning hypotheses. Simple conditioning hypotheses in general related to
phasic changes in response due to the anticipation of the US caused by the CS. Contextual
conditioning was related to changes in tonic response levels caused by experiencing what had
been a high or low threat zone during the initial tour through the city. Simple conditioning
hypotheses thus dealt with responses during the threat manipulation task (i.e., the initial tour
34

through the city) and the navigation task, whereas complex conditioning could only be assessed
during extinction (i.e., the navigation task).
The red and yellow zone markers served as the CS+ and CS-, respectively, and were the
specific cues to which participants were conditioned. Conditioning was inferred if there was
differential responding to the CS+ and CS-, with increased responding to the CS+. It was
hypothesized that participants would have increased responses to the red zone markers compared
to the yellow zone markers.
During the navigation task (i.e., the extinction phase), it was anticipated that differential
CS responding would decrease across trials leading to an interaction of zone number and CS type
during this phase of the experiment. This interaction would be caused by the fact that the CS+
was no longer paired with the US, which would lead to rapid extinction of the anticipatory
response to the CS+. It was likely therefore, that significant differential responding would only be
found during the first pair of CSs experienced in the extinction phase.
Complex conditioning was assessed during the navigation task. Each high and low threat
zone as a whole was assessed for differential responding, as differences between response levels
for the varying zone types were considered the basis for contextual conditioning. Similar to the
simple conditioning hypotheses related to extinction, it was predicted that an interaction between
zone number and threat level would be present. The greatest differential responding would be
exhibited during the first pair of high and low threat zones, whereas the final high and low threat
zones would likely result in very little difference in psychophysiological responding because the
high threat stimuli were no longer paired with the high threat zones.

35

Chapter Four: Experiment 1 Analytic Approach
Data Reduction
Data were scored using an in-house custom designed Matlab scoring program. The
program includes graphical representations of each channel of psychophysiological data for
manual inspection of scoring accuracy. Data were standardized using a z-transformation in order
assess the occurrence of implausible values and outliers. An outlier was considered any value
with a z-score greater than three or less than negative three. Outliers were replaced with averages
of the nearest data points within the same condition.
All tonic measures were assessed during a “trimmed” version of each complete zone in
both the initial tour through the city and the navigation back. The zones were trimmed by
excluding data collected during the first and last 10 seconds of the zone in order to reduce the
influence of the anticipatory responses to the zone markers, and the responses to the electrical
stimulations following the high threat zone markers. The data collected during the initial tour
through the city was trimmed to about 40 seconds per zone, while the trimmed zones during the
navigation back task will vary based on the efficiency with which each participant moves through
the zone, but will not include the first and last 10 seconds of the zone.
Phasic measures were utilized for anticipatory responses to zone markers. These
responses were assessed during an 8 second window occurring at the end of each zone,
immediately preceding the crossing of the zone markers. External environmental stimuli were
reduced in each zone prior to the onset of the 8 second windows (i.e., threatening stimuli
gradually ceased in the high threat zones) allowing for responses to be attributed to anticipation
of zone markers, and to create a more realistic transition between zones than would be
experienced with a sudden cessation of environmental auditory and visual stimuli.
36

Electrodermal data scoring. The scoring program was used to partition response levels
into each zone, and then calculate the median skin conductance level (SCL) and the number of
spontaneous fluctuations (SFs) in each. The median SCL was chosen for analyses rather than the
mean because it is a more robust feature as it is less susceptible to influences of artifacts, which
will be especially useful in future adaptive applications. SFs, which were also scored during
trimmed zones, were quantified as any change in slope of the response curve resulting in a > 0.01
µS response, with a peak latency of 1 to 3 seconds following onset.
Phasic responses were assessed for the conditioning portion of Experiment 1.
Anticipatory SCRs in response to the CSs were assessed as the largest amplitude response during
the eight seconds leading up to the participant physically crossing each zone marker. Median
SCLs during the eight second windows were also scored and analyzed.
Electrocardiographic data scoring. ECG data were scored as interbeat intervals, which
were calculated as median values for each zone. Accuracy of the peak detection scoring program
was assessed manually, with visual inspection of all selected R-waves. Missed R-waves were
manually added to the calculation of zone medians. Anticipatory responses to the CSs in
Experiment 1 were calculated as median IBIs during the eight second window of the final
approach to each of the zone marker CSs.
Power spectral density analyses of HRV were also performed with use of a fast Fourier
Transform based algorithm. The algorithm was used to calculate the spectral power of the low
frequency (LF) component and the high frequency (HF) component of HRV associated with each
zone. The frequency range of the LF component is between 0.04 and 0.15 Hz, while the HF
component is between 0.15 and 0.4 Hz (Task Force of the European Society of Cardiology the
North American Society of Pacing Electrophysiology, 1996). LF and HF component values were
expressed in normalized units, which are calculated by dividing the absolute power of a given
37

component by the total power in the signal minus the very low frequency (VLF) component and
multiplying by 100 (Pagani et al., 1986; Malliani, Pagani, Lombardi, & Cerutti, 1991).
Respiratory data scoring. Respiration was scored in a similar fashion to the ECG data,
and reported as interbreath intervals. Peak detection of each positive deflecting curve in the
breathing cycle was manually reviewed in order to ensure accuracy of the scoring program, and
median intervals were calculated for each zone. Also analogous to the data reduction techniques
applied to the ECG data, anticipatory responses to the CSs were calculated as median interbreath
intervals during the eight second window of the final approach to each of the zone marker CSs.
Pupillary data scoring. Unfortunately, data collected with the eyetracking equipment
embedded within the head mounted display did not yield data that were fit for analysis. The
equipment had apparent difficulty tracking the pupil, leading to the collection of data that were
too noisy for inclusion consideration. Thus, results related to changes in pupil diameter will not
be reported herein.
Behavioral navigation data scoring. Each participant’s position in three-dimensional
space within the VE was logged every two seconds during exposure to the environment. From
these data, the total amount of deviation from the original path was calculated for each zone by
computing the distance from the closest point on the original path at each logged position and
adding those distances together across the entire zone. Three dimensional position coordinates
were logged every second, allowing for the distance from the known coordinates of the original
path to be calculated. The number of arrows needed for assistance to complete passage through
the zone were also logged and recorded, and the amount of time spent in each zone was
calculated.

38

Analyses
Threat level manipulation task analyses. Each dependent variable was analyzed
separately with use of a repeated measures analysis of variance (ANOVA). The ANOVA
employed included a 2 (threat level) by 3 (zone pair) by 2 (navigation awareness) mixed-model
design. This ANOVA was performed to determine the effects of the threat level manipulation,
how it varied across zones, and whether it affected the NA and NU groups differently. Zone pairs
will be referred to as zone pairs A, B, and C. The first zone pair experienced during the initial
tour through the city was labeled zone pair A, while the final pair of high and low threat zone
were labeled zone pair C. The same repeated-measures ANOVA design was also applied to the
phasic anticipatory responses during the eight second windows leading up to the zone markers in
order to determine the effects of simple conditioning during acquisition. All significant
interactions and main effects with greater than one degree of freedom were supplemented with
paired samples t-tests in order to identify the precise nature of these effects. A modified
Bonferroni correction was utilized to prevent inflation of type 1 error rates in all reported
significant t-test results (Rom, 1990). Additionally, a Greenhouse-Geisser correction was used for
all reported main effects and interactions with greater than one degree of freedom.
Navigation analyses. A separate set of ANOVAs was employed to assess responses
during the navigation task. The same 2 (threat level) by 3 (zone pair) by 2 (navigation awareness)
repeated measures ANOVA was applied to each dependent variable, which in this case also
included the set of behavioral variables associated with the navigation task (i.e., deviation,
number of arrows, and time spent in each zone). The zones are experienced in a reverse order
during the navigation phase, and thus the order of the zone pairs experienced during the
navigation task went from zone pair C to zone pair B and finishing after zone pair A. The zone
pair labels refer to the same physical zones during the initial tour and the navigation back, though
39

the order in which they are experienced is reversed. The navigation task data were subjected to a
separate set of ANOVAs in order to determine the effects of conditioning during the extinction
phase. Complex conditioning effects were assessed with comparisons using trimmed zone
variables, while extinction was examined for simple conditioning by analyzing anticipatory
responses when approaching zone markers. Paired samples t-tests were again used to examine the
nature of main effects and interactions, and the same corrections were employed to protect
against type 1 errors.
MLR and ANN Analytic Approach
The experimental conditions described herein are designed to provoke responses typical
of high and low extremes of experienced threat. The ultimate purpose of the proposed ANNs is to
develop a strategy for creating adaptive systems for future research and eventual real-world
applications including enhanced training scenarios and adaptive assistance for any individual who
must fulfill tasks that involve high levels of threat, stress, or cognitive effort. A backpropagated
algorithm was utilized to train the ANN models mainly because it can be thought of as a
specialized case of the general linear model that is capable of more effectively fitting curvilinear
data distributions than is possible with a linear regression model. Additionally, because the BPN
model can be thought of as a special type of regression, and provides similar output, results can
be compared directly to predictive results generated with the use of more standard and widely
used linear regression models. This sets the backpropagated algorithm apart from numerous
machine learning algorithms, such as support vector machines, which can lead to difficulties
when trying to compare causes for predictive differences with other algorithms. Thus, this study
compared the predictive qualities of the BPNs developed to those of the more standard approach,
especially in the social sciences, of MLR.
40

First, a MLR model that used the psychophysiological data gathered during the initial
tour through the city to predict the navigation performance was developed. As described above,
the navigation performance was quantified as the time needed to return to the starting point, the
deviation from the original path, and the number of arrows needed for assistance. Each
performance measure was significantly correlated (see Table 5), and it is likely that each measure
is assessing very similar wayfinding related phenomena. Logic would dictate that a participant
who takes more time to navigate back to the starting point would also likely deviate more from
the original path and would in turn require more arrows for assistance in maintaining the proper
path through the city. Additionally, results of the regression models were similar in regards to
their predictive abilities in relation to each of the outcome measures. Therefore, the time duration
to return to the starting point was the only outcome measure that was considered for the
regression and subsequent ANN models presented herein.
Table 5
Correlations between navigation task outcome measures

Time Deviation Arrows
Time Pearson r 1 .267
**
.318
**

Sig. (2-tailed)

.007 .001
Deviation Pearson r
.267
**

1
.461
**

Sig. (2-tailed) .007

.000
Arrows Pearson r
.318
**
.461
**

1
Sig. (2-tailed) .001 .000
**. Correlation is significant at the 0.01 level (2-tailed).

A set of seven psychophysiological predictors were utilized. Included in the analyses
were SCLs, SFs, IBIs, interbreath intervals, and the LF and HF components of the HRV measure.
Navigation awareness group assignment was also added as a dichotomous predictor variable. Due
to the relatively small sample size in this experiment, an attempt to condense the number of
predictors was made by calculating difference scores between the high and low threat zones for
41

each of the psychophysiological predictors. Difference scores were calculated in two ways. First,
the overall difference between all three high and low threat zones was calculated as a
representation of the response levels associated with the task as a whole. Next, a difference score
that would serve as an index of the habituation involved in the responses to the high threat zones
compared to the low threat zones was calculated. To accomplish this, difference scores between
the high and low threat zones in zone pairs A and C were calculated. The zone pair C difference
score was then subtracted from the zone pair A difference score. This threat habituation index
was calculated to account for the waning response levels during high threat zones present in a
number of response measures (see Figure 7). This resulted in a total of seven predictor variables
for each type of difference score. Separate backward-elimination stepwise regressions were
utilized for each set of difference score predictors. Standardized regression coefficients associated
with each predictor variable were inspected for significance. Results from both regression models
are presented below. The model that best predicts the outcome measure was compared to the
predictive ability of the BPN developed.
The BPN model was developed in a manner analogous to the above MLR model, such
that the predictor variables, or inputs, were the same in each model. The output node will again
represent the continuous navigation performance outcome measure of the time needed to return to
the starting point (refer to Figure 5 for an example of the BPN structure). The primary goal of the
BPN used herein is prediction. In order to increase the probability of generalization and to avoid
over-fitting of the observed sample, three data sets were considered, including the training set,
validation set, and the test set. According to Ripley (1996), each can be understood as follows: 1)
The training set is comprised of a set of data examples used for learning meant to fit the
parameters (i.e., weight estimates) of the classifier; 2) The validation set is a set of examples used
to tune the parameters of a classifier (e.g., selections of the number of hidden units in the neural
42

network) and to assess the predictive ability of the network on sample units that have not been
considered during training. The validation set is kept aside to evaluate the accuracy of the model
derived from the training procedures. In the validation phase, the model output is compared with
actual outputs using statistical measurements such as root mean squared error (RMSE) and the
coefficient of correlation (see Hagan et al., 1996; Haykin, 1999) The test set contains a third set
of examples that had not been previously considered during the training or validations phases,
which is used to calculate the global predictive ability of the network for generalizations to future
practical applications.

Figure 5: Experiment 1 BPN Structure Example. Representation of an example BPN which
includes the inputs associated with the threat level manipulation.
Following Kindermann and Linden (1990), a gradient descent technique was used in the
BPN development to minimize least squared error and avoid getting “trapped in local minima.”
To accomplish this, hidden layer nodes in the BPN were adjusted. To assure that the BPN gets as
43

close as possible to true (absolute) minimum error, Maghami and Sparks’ (2000) technique was
employed, stating that one should build a BPN with one hidden layer and continually increase the
number of nodes in the hidden layer until the error is no longer reduced. A tanh activation
function was applied to each hidden layer node. The BPN models began in a feed forward
fashion, and output error was then backpropagated to adjust the weights of the given hidden
nodes, in order to minimize output error.
After the development and implementation of the BPN, comparisons were made
(following Parsons et al., 2004) between its output and that of the general linear model’s
regression for the predicted outcome measure examined by performing the following tasks. First,
the criterion for the BPN was recorded. The predictor set and the criterion output from the BPN
was input into a new regression analysis. Next, the standard error of the estimate and R
2
were
computed from the BPN regression, and the results were compared with the results of the
straightforward regression models. The variance of the standard error of the estimates was noted
to determine if the differences are statistically significant, and the model with the smallest
standard error of the estimate was considered preferable. Additionally, a Fisher z transformation
was performed to directly compare the correlation coefficients of the competing models. This
calculates a value of z that can be applied to assess the significance of the difference between two
correlation coefficients (see Meng, Rosenthal, & Rubin, 1992 for review). Predictor-criterion
correlation coefficients extracted from both the MLR and BPN models were directly compared
following methods originally outlined by Dunn and Clark (1969) for comparing correlation
coefficients measured on the same sample population.

44

Chapter Five: Experiment 1 Results
Threat Manipulation Task Results
Threat level main effects. Participants evidenced increased levels of arousal while being
led through the high threat zones when compared to response levels in the low threat zones during
the initial tour through the city. SCLs were significantly higher in the high threat zones, F(1, 51)
= 26.89, p < 0.001. There were also an increased number of SFs exhibited during high threat
zones, F(1, 51) = 4.24, p < 0.05. Both interbreath intervals, F(1, 51) = 34.64, p < 0.001, and IBIs,
F(1, 51) = 5.71, p < 0.05, were shorter during high threat zones. Interval measures in this case
have an inverse relationship with rate measures, thus shorter IBIs indicate an increased heart rate.
See Figure 6 for a graphical representation of these findings.
a) b)

5
5.1
5.2
5.3
5.4
5.5
High Threat Low Threat
microsiemens (µS)
Skin Conductance Level
2.5
2.6
2.7
2.8
2.9
3
3.1
3.2
3.3
High Threat Low Threat
Number of SFs
Spontaneous Fluctuations
45

c) d)

Figure 6: Threat Level Main Effects. High threat and low threat response level comparisons for a)
SCL, b) SFs, c) respriation rate, and d) heart rate. Though respiratory and electrocardiographic
data were collected and analyzed as interbreath and interbeat intervals, respectively, both have
been converted here to cycles per minute for display purposes.
Zone order main effects. Psychophysiological response levels were sensitive to the order
in which zones within the city were experienced. The zone variable refers to the average of each
high and low zone pair, meaning that zone pair A is actually the average response level between
the first high and the first low zone. Each pair of high and low zones was experienced
sequentially. Figure 7 displays a summary of the significant results outlined below.
a) b)

15
15.5
16
16.5
17
17.5
18
18.5
19
High Threat Low Threat
breaths per minute
Respiration Rate
65
65.2
65.4
65.6
65.8
66
66.2
66.4
High Threat Low Threat
beats per minute
Heart Rate
5
5.1
5.2
5.3
5.4
5.5
5.6
Zone A Zone B Zone C
microsiemens (µS)
Skin Conductance Level
1.2
1.4
1.6
1.8
2
2.2
2.4
Zone A Zone B Zone C
Number of SFs
Spontaneous Fluctuations
46

c) d)

Figure 7: Zone Order Main Effects. Psychophysiological responses across zones for a) SCL, b)
SFs, c) respiration rate, and d) the LF component. Interbreath intervals were again converted to
breaths per minute for display purposes.
SCLs demonstrated a typical habituation curve across time resulting in a zone main
effect, F(2, 50) = 19.18, p < 0.001. SCLs in zone pair A were higher than those in zone pair B,
t(52) = 3.73, p < 0.001, and zone pair C, t(52) = 4.87, p < 0.001. SCLs in zone pair B were higher
than those in zone pair C as well, t(52) = 3.82, p < 0.001.
Similar results were found in the number of SFs exhibited in each zone pair, F(2, 50) =
5.66, p < 0.01. The greatest number of SFs were elicited during the first zone pair compared to
both the second zone, t(52) = 2.61, p < 0.05, and third zone pair, t(52) = 2.89, p < 0.01. There
were generally more SFs in zone pair B than zone pair C, though not significantly so.
A typical habituation curve was also evidenced by the interbreath interval measure, F(2,
50) = 18.54, p < 0.001. The shortest interbreath intervals were found during the first zone pair,
and were significantly shorter than those found in the second, t(52) = 2.91, p < 0.01, and third
zones, t(52) = 4.89, p < 0.001. Interbreath intervals recorded in zone pair B were also shorter than
those in zone pair C, t(52) = 3.61, p < 0.01.
The LF component of the HRV measure, related to sympathetic activation, also revealed
a pattern of response that was typical of habituation across zones. Zone pair A had higher levels
15
15.5
16
16.5
17
17.5
18
18.5
19
Zone A Zone B Zone C
breaths per minute
Respiration Rate
38
40
42
44
46
48
50
Zone A Zone B Zone C
Power (normal units)
Low Frequency Component
Power
47

of sympathetic activation than zone pair C, t(52) = 2.62, p < 0.05, or zone pair B, t(52) = 2.14, p
< 0.05. The final two zones did not differ significantly with respect to LF power.
Navigation awareness main effects. The only measure that was sensitive to the between
group manipulation related to navigation awareness was the interbreath interval. Those in the NA
group evidenced shorter interbreath intervals than those in the NU group, F(1, 51) = 5.61, p < .05.
Other response measures including SCLs, IBIs, SFs, and the LF component all demonstrated
increased activation in the NA group, though not significantly so.
Threat manipulation zone interactions. A significant interaction was revealed between
threat level and zone in the SCL measure, F(2, 50) = 16.14, p < 0.001 (Figure 8). The greatest
difference in response between high and low zones was found in the first zone pair, t(52) = 5.35,
p < 0.001. The high zone elicited increased SCLs in both the second zone pair, t(52) = 3.08, p <
0.01, and third zone pair, t(52) = 2.04, p < 0.05, as well.

Figure 8: Threat Level by Zone Order Interaction. Significant threat level by zone order
interaction for skin conductance level.
4.8
4.9
5
5.1
5.2
5.3
5.4
5.5
5.6
5.7
5.8
5.9
Zone A Zone B Zone C
microsiemens (µS)
Threat Level by Zone Order Interaction
Skin conductance Level
High Threat
Low Threat
48

A zone order by navigation awareness interaction was uncovered in regard to interbreath
intervals, F(2, 50) = 3.25, p < 0.05. The NA group evidenced shorter interbreath intervals
throughout the experiment, though the greatest and only significant difference in response level
between the two groups was found in the third zone as revealed by one-way ANOVA, F(1, 51) =
7.50, p < 0.01 (Figure 9).

Figure 9: Zone Order by Navigation Awareness Interaction. Depicted here is the significant zone
order by navigation awareness interaction for respiration rate, which was again converted from
interbreath intervals for display purposes.
Simple conditioning acquisition results. Anticipatory responses collected as the
participants approached the zone markers revealed that differential responding between CS+ and
CS- was exhibited in both skin conductance and interbreath interval measures. Larger SCR
amplitudes were elicited by the CS+ than the CS-, F(1, 51) = 7.12, p < 0.01. Additionally, SCLs
were higher during the approach to the CS+ zone markers, F(1, 51) = 69.37, p < 0.001.
Participants also evidenced shorter interbreath intervals in anticipation of the CS+, F(1, 51) =
22.65, p < 0.001. It should be noted that when approaching a CS+ participants have just passed
14
15
16
17
18
19
20
Zone A Zone B Zone C
breaths per minute
Zone Order by Navigation Awareness Interaction
Respiration Rate
NA Group
NU Group
49

through a low threat zone, meaning that increased levels during the eight second window prior to
reaching the CS+ are likely due to anticipation of the CS+ rather than lingering effects of the
environmental stimuli which may be present as a participant exits a high threat zone and
approaches a CS-.
Participants also responded differently across zones. A zone order main effect was found
for SCR amplitudes, F(2, 50) = 4.09, p < 0.05, such that responses in zone pair B were lower than
those in zone A, t(52) = 2.44, p < 0.05, and zone C, t(52) = 2.40, p < 0.05. It is possible that
heightened responses in the first zone were large due to the novelty of the situation, and responses
to CS+ declined and then grew larger during the third zone pair as the CS-US contingency was
learned. Though the interaction between CS type and zone order was not significant, differential
responding only occurred during the second and third zone pairs, while responses in the first zone
pair were high in response to both CS+ and CS- (See Figure 10). Likewise, analysis of IBIs
revealed a main effect of zone order, F(2, 50) = 8.90, p < 0.001, such that a pattern of responding
in which the highest levels of activation were found in zone pairs A and C was evident. IBIs were
shortest in zone pair C, but zone pair B was associated with longer IBIs than zone pair A, t(52) =
2.36, p < 0.05, or zone pair C, t(52) = 4.07, p < 0.001.
50

Figure 10: Skin Conductance Responses in Anticipation of Conditioned Stimuli. SCR amplitudes
in anticipation of the conditioned stimuli (i.e., red and yellow zone markers).
Participants’ SCLs tended to habituate across zones during anticipation of the CSs, F(2,
50) = 12.77, p < 0.001. Higher levels were evidenced in the first zone pair than either the second,
t(52) = 4.26, p < 0.001, or third zone pair, t(52) = 3.92, p < 0.001. The same pattern of response
was apparent in interbreath intervals, F(2, 50) = 4.56, p < 0.05, as breathing was more rapid, as
evidenced by shorter interbreath intervals, during the first zone compared to the second, t(52) =
2.43, p < 0.05, or third zone pairs, t(52) = 2.29, p < 0.05.
Interbreath intervals were again the only measure sensitive to the navigation awareness
factor. Shorter interbreath intervals were exhibited by the NA group during anticipation of the
CSs, F(1, 51) = 4.33, p < 0.05.
Navigation Task Results
Complex conditioning effects. The only measure sensitive to previously high and low fear
zones during the navigation task was the LF component of the HRV power spectrum, meaning
that there was increased sympathetic activation in previously high threat zones when compared to
0
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0.18
Zone A Zone B Zone C
Amplitude (µS)
Skin Conductance Responses in Anticipation of
Conditioned Stimuli
CS+
CS-
51

previously low threat zones, F(1, 51) = 6.73, p < 0.05. This finding represents the only
psychophysiological evidence for complex conditioning. As a reminder, the navigation task also
served as the extinction phase of the embedded conditioning paradigm. High threat zones
represented complex CS+’s while low threat zones were complex CS-’s. The high threat zones
were no longer paired with the threatening stimuli during the navigation task, creating an
extinction phase to test the effects of complex conditioning. Anticipatory responses to the single-
cued CSs (i.e., zone markers) appeared to extinguish rapidly, as differential responding to the CSs
was only evident in IBI responses in the first zone experienced during the return trip through the
city, t(52) = 2.17, p < 0.05.
The lingering effects of the threat level manipulation also had a behavioral effect, such
that participants spent more time navigating through the previously high threat zones than the low
threat zones, F(1, 51) = 6.09, p < 0.05. Post hoc analyses revealed that only the NA group spent
significantly more time in previously high threat zones, t(27) = 2.70, p < 0.05, while prior threat
level did not appear to affect the NU group during navigation, t(24) = 0.72, p = 0.48.
Zone order main effects. A significant zone main effect was found in interbreath
intervals, F(2, 50) = 6.21, p < 0.01. The response pattern was such that the shortest intervals were
found in zone pair A, and the longest in zone pair C. During the navigation task, zone pair C
refers to the first zones experienced during the navigation task. Thus, zone pair C in the
navigation task refers to the same physical zone pair as zone pair C of the initial tour through the
city, though in the navigation task it was experienced first. The only significant difference in
interbreath intervals was between zone pairs C and A, t(52) = 3.55, p < 0.01. There were also
behavioral zone main effects related to participants’ deviation from the original path, F(2, 50) =
5.75, p < 0.01, and the number of arrows needed to navigate back to the starting point, F(1, 51) =
4.61, p < 0.01. Participants deviated progressively more from the original path as they returned to
52

the starting point, deviating significantly more in zone pair A than in zone pair C, t(52) = 4.14, p
< 0.001. A similar behavioral pattern was apparent in the number of arrows needed, such that
more arrows were required in zone pair A than either C, t(52) = 5.53, p < 0.001, or zone pair B,
t(52) = 4.06, p < 0.001. The time spent in each zone also increased as participants advanced
through the navigation task; however, high levels of variability between participants prevented a
significant zone order main effect. Figure 11 provides a graphical representation of the zone order
effects.
a) b)

c) d)

Figure 11: Zone Order Effects in Navigation. Zone order main effects regarding a) time spent in
each zone, b) deviation from the original path, c) the number of arrows needed and d) respiration
rate. The zone order main effect was not significant in the time spent in each zone, though it is
presented here to demonstrate the similar pattern of results as found in the other behavioral
variables. Deviation units are arbitrary distance units within the VE.
70.5
71
71.5
72
72.5
73
73.5
Zone C Zone B Zone A
seconds
Time Spent in Each Zone
0
20
40
60
80
100
Zone C Zone B Zone A
distance units
Deviation from Original Path
0
5
10
15
20
Zone C Zone B Zone A
number of arrows
Arrows Needed
15
15.5
16
16.5
17
17.5
18
18.5
19
Zone C Zone B Zone A
breaths per minute
Respiration Rate
53

Navigation awareness effects. Interbreath intervals were again sensitive to between group
differences related to navigation awareness. As found during the threat level manipulation task as
participants took their initial tour through the virtual city, the NA group had shorter interbreath
intervals than the NU group, F(1, 51) = 6.84, p < 0.05. As expected, the NA group deviated from
the original path through the city less so than the NU group, F(1, 51) = 4.61, p < 0.01, and
required fewer arrows for assistance, F(1, 51) = 17.96, p < 0.001.
A zone by navigation awareness interaction was uncovered in reference to SCLs, F(2, 50)
= 3.69, p < 0.05. As seen in Figure 12, the NA group evidenced decreased SCLs from zone pair C
to zone pair A, while the NU group demonstrated increased response levels from the first to the
third zone pair during the return trip.

Figure 12: Skin Conductance Levels During Navigation Task. Graphical depiction of the
significant zone order by navigation awareness interaction found with regard to SCL.
MLR and BPN Results
Regression model comparisons. The regression model utilizing the overall high threat
versus low threat difference scores as predictors failed to explain a significant proportion of the
4.2
4.4
4.6
4.8
5
5.2
5.4
5.6
5.8
Zone C Zone B Zone A
Amplitude (µS)
Skin Conductance Levels During Navigation Task
NA Group
NU Group
54

variance in the navigation performance outcome measure (Table 6). The IBI measure was the
only significant predictor of total navigation duration (Table 7). Participants who had greater
differences in IBI response levels between high and low threat zones, such that shorter IBIs were
exhibited in high threat zones, tended to perform less efficiently on the navigation task.
When the difference scores calculated between the differences in the first and third zone
pairs were used as predictor variables, the model was able to explain a significant proportion of
the variance in navigation performance, R
2
= 0.27, F(7, 45) = 2.32, p < 0.05. Significant
predictors included SFs and interbreath intervals. The negative correlation coefficient related to
the SF measure (seen in Table 8) indicates that participants who had a greater difference between
the high and low threat zones during zone pair A than zone pair C took less time navigating back.
Participants whose threat response habituated from zone pair A to zone pair C performed more
efficiently on the navigation task. Those who were still exhibiting differences in response during
zone pair C similar to differences in zone pair A spent more time in each zone during navigation.
Though the correlation between interbreath intervals and navigation performance is positive, the
results are analogous to those of the SFs. Increased activation leads to an increased number of
SFs, however it also leads to decreased interbreath intervals, so the response patterns are
reversed. In regards to interbreath intervals, those who had a greater negative difference score
between high and low threat zones in zone pair A than zone pair C took less time on the
navigation task. Again, greater reduction in differential activation between high and low threat
zones during zone pair C resulted in more efficient navigation performance (Figure 13). Summary
statistics comparing both regression models are presented in Table 6. Additionally, coefficient
statistics for each predictor used in each model are presented in Tables 7 and 8. The model
utilizing the predictors calculated as a threat response habituation index was determined to be
55

preferable due to the increased variance explained by the model, as well as decreased error, and
was thus used in comparison to the performance of the BPN.
Table 6
Multiple linear regression model summary statistics
Predictor
Set
R R
2
Adj. R
2
Std. Error RMSE F p
Set 1
*
0.44 0.19 0.07 51558.1 227.1 1.51 0.19
Set 2
**
0.52 0.27 0.17 48538.0 220.3 2.32 <0.05

Note.
*
Set 1 = set of predictors calculated from the overall difference between high and low threat
zone responses. Predictors contained within the final equation include SFs and IBIs.
**
Set 2 = set
of predictors calculated from the difference between responses in zone pair A and zone pair C.
Predictors contained within the final equation include SFs, interbreath intervals, and the LF
component. RMSE = root mean squared error.
Table 7
Predictor regression coefficients – Predictors based on overall differences between high and low
threat zones
Measure b Std. Error β t p
SCL

-11441.3 22162.2 -0.08 -0.52 0.61
SFs

-18359.2 9774.3 -0.25 -1.88 0.066
Interbreath
Intervals
0.141 16.01 0.001 0.009 0.99
IBIs

-566.6 201.2 -0.38 -2.82 < 0.01

LF
component
-146.9 205.6 -0.10 -0.71 0.48
HF
component
-426.4 7909.0 -0.01 -0.05 0.96
Navigation
Awareness
12696.0 14046.8 0.12 0.90 0.37
Note. For each measure, statistics for the final step before elimination from the model are
reported for those predictors that were not included in the final equation.

56

Table 8
Predictor regression coefficients – Predictors based on differences between zone pair A and zone
pair C
Measure b Std. Error β t p
SCL

-6187.2 13068.4 0.07 0.47 0.64
SFs

-15972.7 5998.4 -0.34 -2.66 < 0.05
Interbreath
Intervals
19.0 8.9 0.28 2.13 < 0.05
IBIs

-23.1 132.2 -0.02 -0.18 0.86

LF
component
6232.3 3487.6 0.23 1.79 0.08
HF
component
-270.2 167.0 -0.26 -1.62 0.11
Navigation
Awareness
7235.0 14677.8 0.07 0.49 0.62
Note. For each measure, statistics for the final step before elimination from the model are
reported for those predictors that were not included in the final equation.

Figure 13: Interbreath intervals and Navigation Performance. Scatter plot depicting the positive
relationship between interbreath interval difference scores and increased time spent in the
navigation task.

BPN results. The BPN developed included the same seven predictor variables used in the
preferred MLR model (i.e., difference scores calculated as a habituation index), here entered as
-3000
-2500
-2000
-1500
-1000
-500
0
500
1000
50 60 70 80 90 100
Difference Scores
Time Per Zone (Seconds)
Interbreath Intervals and Navigation Performance
57

inputs to the system. In the preliminary tests to assure that the BPN achieved its optimal output,
the network model was developed with different numbers of nodes in the single hidden layer. The
hidden layer learns to provide a representation for the inputs through an alteration of the weights
associated with each node and then connects to the output layer. The experimental method
involved developing a hidden layer that contained a minimum of four nodes and a maximum of
twenty-four nodes. It was found that six hidden layer nodes resulted in optimal model
performance. A tanh activation function was applied to the hidden and output nodes, which is
recommended when the sum of squares error function is employed, as it was in this case.
Descriptive statistics associated with the training, validation, and test set samples are included in
Table 9.
Following network training, the test set was applied to the network to test the
generalizability of the model. It should be noted that the predictor values of the test set were not
involved in the training of the model, providing a “test” of the generalizability of the model to
new data. The BPN was able to predict the outcome measure with 76.0% accuracy. Table 10
presents a summary of the optimal BPN model parameters.
Table 9
Descriptive statistics associated with BPN inputs and target variable for Experiment 1
Inputs Target

SCL

SFs
Interbreath
Intervals

IBIs
LF
Comp.
HF
Comp.
Time
Spent
Training
Sample

Mean 0.544 0.515 -32.88 -7.73 2.77 -0.337 445.50
Std. Err. 0.081 0.201 59.00 11.28 11.66 0.314 6.99
Validation
Sample

Mean 0.234 -0.500 192.75 -11.00 10.30 -0.954 421.42
Std. Err. 0.141 0.117 150.45 6.85 8.02 0.438 7.25
Test
Sample

Mean 0.447 0.208 -48.16 -9.95 3.82 -0.482 435.39
Std. Err. 0.091 0.195 96.39 10.28 11.29 0.360 7.25
58

Note. All input values are calculated from the difference between response levels in zone pair A
and zone pair C. Time spent is reported as seconds to complete the navigation task.

Table 10
BPN model parameters for Experiment 1
Layers
Training
Performance
Test Set
Performance
Learning
Algorithm
Error
Function
Hidden
Activation
Output
Activation
7-6-1 0.938 0.871 Gradient
Descent
SOS Tanh Tanh
Note. The layers refer to the number of inputs – hidden nodes – and outputs. The performance
values provided are the correlation coefficients associated with the training and test sets. The
training performance describes the model’s fit to the training set, while the test set performance is
a measure of the generalizability of the model. SOS = sum of squares. Tanh = hyperbolic tangent
activation function.

A global sensitivity analysis was performed in order to determine the relative importance
of each input (i.e., predictor variable) to the successful prediction of the output. A sensitivity
analysis tests how the error rates would increase or decrease if each individual input value were
changed (see Saltelli, 2005 for review). More specifically, the data set is repeatedly submitted to
the network, and in turn each input variable is replaced with its mean value calculated from the
training sample, and the resulting network error is recorded. Important inputs cause for a large
increase in error, while the error increase was small for unimportant inputs. Thus, sensitivity
analysis allows for a rank order of the importance of the individual inputs (Chen & Kocaoglu,
2008; Winebrake & Creswick, 2003). Table 11 provides a list of the input variables in rank order,
with sensitivity values that represent the ratio of network error with the input omitted to the
network error with the input included. Ratio values less than 1 indicate that the network actual
performs better without inclusion of the associated input. As seen in Table 11, ratios associated
with all seven input variables are above 1, and are thus important contributors to the performance
of the model.

59

Table 11
Global sensitivity analysis for the BPN developed for Experiment 1
Rank Measure Ratio Value
1

SCL 5.22

2

IBI

4.83
3 SF

4.62
4

Interbreath
Interval

4.27
5 Navigation
Awareness
2.52
6 LF
component
2.50
7 HF
component
1.05

MLR and BPN model comparison. Summary statistics regarding both the selected MLR
model and the BPN are provided in Table 12. Examination of the squared correlation coefficient
associated with each model reveals that there is a 49.0% increase in prediction of navigation
performance when the BPN is employed. The drop in root mean squared error related to use of
the BPN in comparison the MLR model signifies that the neural network model better fits the
data.
Table 12
Summary statistics for MLR and BPN comparison in Experiment 1
Model
Sample
Size R R
2
Adjusted
R
2
Standard
Error RMSE
MLR 53 0.520 0.270 0.17 48538.0 220.30
BPN 53 0.871 0.760 0.71 42183.70 205.39
Note. RMSE = root mean squared error.

Direct comparison of correlation coefficients associated with each model with use of the
Fisher z transformation revealed that the BPN has significantly greater predictive ability than the
MLR model, z = 3.84, p < 0.001. Thus, the BPN was determined to be the preferable model due
60

to the increase in the squared correlation coefficient in addition to the decrease in root mean
squared error (RMSE).

61

Chapter Six: Experiment 1 Discussion
Threat Level Manipulation Task Effects

Effects of threat. A primary goal of Experiment 1 involved the examination of response
patterns associated with exposure to threat in a VE, and to investigate whether those responses
would predict navigation performance. As expected, participants evidenced increased
psychophysiological arousal during the high threat zones. Increased SCLs and a greater number
of SFs were exhibited, as well as decreased interbreath intervals and IBIs, indicative of increased
respiration and heart rates, respectively. These results indicate that the high threat zones were
sufficiently threatening to promote increased psychophysiological response levels, meaning that
the manipulation of threat was effective.
Zone order effects. Response levels during high threat zones were consistently enhanced
compared to those in low threat zones; however, response levels in general decreased from the
beginning of the tour through the city to the goal. This result is typical, as repeated exposure to a
stimulus or task condition may result in habituation of the psychophysiological response being
measured, meaning that the relative intensity of the responses elicited may decrease over time
(see Groves & Thompson, 1970). Indeed, the majority of psychophysiological response measures
analyzed yielded results indicative of a habituation curve. Response levels were consistently
greatest during the first pair of high and low threat zones experienced. Habituation effects may
have been particularly strong at the onset of the experiment due to the novelty related to
immersion in a VE, which may compel researchers to consider discarding psychophysiological
data collected at the beginning of VR exposure conditions. Habituation effects were robust in the
current study, even though a baseline procedure was employed with to goal of counteracting
some of the effects of habituation by reducing the novelty associated with initial VR exposure.
Post-hoc examination of IBIs, which actually failed to yield a significant zone order main effect,
62

revealed that participants nevertheless demonstrated decreased intervals in the first zone
compared to the second, t(52) = 2.01, p = 0.05. Thus, it is important to note that though increased
threat associated with half of the experienced zones successfully increased psychophysiological
arousal, the absolute level of arousal waned over time.
SCL results revealed that in addition to a zone order main effect, a significant interaction
between threat level and zone was also present. SCLs were lower in the low threat zones
throughout the initial tour through the city, and habituation effects were only evident in the high
threat zones. Low threat zones elicited low levels of response throughout (see Figure 8). This
highlights the notion that habituation may not occur at the same rate for different types of stimuli,
such as responses elicited by target stimuli compared to passively viewed stimuli (see Bradley,
2008). Moreover, the fact that only SCLs demonstrated this pattern of response supports the view
that responses collected with different psychophysiological measures may also habituate at
different rates to the same stimuli (e.g., Bradley, Lang, & Cuthbert, 1993). Adaptive system
design, in general, must consider habituation effects and provide capabilities to adapt not only to
changes in an individual’s current responses to relevant stimuli, but also to the changes in
response over time related to habituation. The presence of habituation calls for the development
of dynamic models of adaptation (e.g., Haarmon, Boucsein, & Schaefer, 2009; Parsons &
Reinbold, 2011).
Effects of navigation awareness. It was hypothesized that group differences in
psychophysiological responses would arise from the navigation awareness manipulation.
Participants in the NA group were expected to exhibit diminished responses to threat due to the
greater level of cognitive resources required to commit the path through the city to memory. In
other words, the participants in the NA group might feel less present in the high threat zones
because they were preoccupied with the secondary route-learning task. Friedman et al. (2007)
63

reported that concentration on a novel secondary task decreased levels of presence, and it has
been reported that presence is positively correlated with psychophysiological response levels
(Meehan, Insko, Whitton, & Brooks, 2002), meaning that lowered levels of presence result in
lowered physiological response levels. The results, however, did not support this notion. The only
psychophysiological response difference due to navigation awareness was found in shorter
interbreath intervals among the NA group. It is possible that the additional cognitive resources
required for route-learning had an additive effect on psychophysiological response, leading to
more rapid breathing. Other response measures followed this response pattern, lending marginal
support for this explanation, but results were not significant. Navigation awareness also interacted
with zone order, such that the NU group appeared to habituate to a greater degree than the NA
group, as evidenced by the only significant difference between the groups occurring in the third
zone. As Figure 10 demonstrates, responses habituated in both groups across zones, but the NA
group was perhaps more resistant to habituation than the NU group. This may provide further
evidence that the addition of the secondary route-learning task had an effect on participants such
that it increased levels of arousal compared to those who were not aware of the secondary task,
causing the NA group to maintain heightened arousal and habituate less rapidly across zones.
Generally speaking however, the navigation awareness manipulation appeared to have
little effect on psychophysiological response during the initial tour through the city. As discussed
below, behavioral differences between groups did arise during the navigation task, but little effect
on psychophysiological response levels during the route-learning portion of the task was
apparent. It is possible that the threatening stimuli experienced within the environment were
intense enough to render the manipulation ineffective for creating psychophysiological response
change. Results suggest that psychophysiological responses related to route-learning are
superseded by responses to high levels of threat within the environment, such that participants
64

who are unaware of the route-learning task will not respond differentially to threatening stimuli
from those who are aware of the secondary route-learning task.
Navigation Task Effects
A simulation task involving route-learning and a test of navigation abilities was selected
for the current study for a number of reasons. First, it lends itself well to a future training module
in a military setting. Enhancing route-learning and navigation abilities, especially during highly
threatening situations, has the potential to provide life-saving skills to military service members
confronted with similar stressors in combat situations. A number of researchers have also
postulated that there is little difference between the navigational experiences and skills gained in
simulated environments and those in natural environments (See Darken, 1996; Goldin &
Thorndike, 1982; cited in Chen & Stanney, 1999). Additionally, it affords the opportunity to
assess responses to single and dual-task conditions with the inclusion of the navigation awareness
manipulation. Participants who are told ahead of time that they must remember their path through
the city have the dual-task of following the guides through highly threatening areas and an active
route-learning task. Participants in the NU group did not experience the effects of the secondary
route-learning task. Any route-learning that did occur in the NU group was gathered in a passive
way. Thus, the NU group provided the ability to assess response patterns associated with varying
levels of threat alone, while the NA group allowed for assessment of the effects related to both
threat and a secondary cognitive task on psychophysiological responses in exactly the same
environment. Furthermore, to our knowledge, the effects of varying levels of threat on route-
learning and navigation performance has not been studied, making this a novel approach.
Navigation task zone order effects. The only psychophysiological zone order main effect
was found with the interbreath interval measure, such that participants evidenced shorter
interbreath intervals as they progressed further, indicating more rapid breathing rates across
65

zones. The reason for this finding becomes more apparent when the behavioral zone order effects
are considered. As participants ventured further in the return trip through the city, the efficiency
with which they navigated declined. Participants required more time to traverse the third zone
pair experienced on the way back (zone pair A), deviated more, and needed more arrows for
assistance locating the original path. Thus, difficulty with navigation was associated with
shortened interbreath intervals. These findings corroborate the results of a previous study
employing a driving simulation task, in which respiration rate increased as driving performance
waned during a high task difficulty condition (Mehler et al., 2009). In the current study, the
navigation task became more difficult as participants navigated further from the starting point of
the task, as evidenced by the decrements in navigation performance which were accompanied by
increased rates of respiration. This provides support for the effectiveness of respiration as a
means monitoring user frustration and perceived task difficulty related to navigation performance.
Navigation awareness effects. As expected, the NA group was able to navigate back
through the city in a more efficient manner, deviating less and requiring fewer arrows for
assistance. Additionally, interbreath intervals again differentiated the two groups, with the NA
group again exhibiting shorter interbreath intervals, lending credence to the notion that the NA
group simply included participants who tended to breathe more rapidly than the NU group.
Navigation awareness also interacted with zone order in SCL. The NU group exhibited increased
response levels across zones, while responses from the NA group declined across zones. It
appears as though the NU group may have experienced increased levels of arousal as they
navigated further through the virtual city, which is likely due to increased levels of stress or
frustration due to the decrements in navigation performance across zones, as noted above.
Participants also spent more time in previously high threat zones, though post-hoc
analyses revealed that this difference was only observed in NA participants. This result suggests
66

that the threatening stimuli presented in the high threat zones during the initial tour through the
city had a deleterious effect on route-learning. Participants in the NU group did not have
increased difficulty in previously high threat zones as they were not actively engaged in route-
learning during the initial tour.
Simple and Complex Conditioning Effects
Evidence for simple conditioning in response to the single-cued zone marker CSs was
found in electrodermal and respiratory responding. Anticipatory SCR amplitudes were larger and
SCLs higher during anticipation of the CS+ compared to CS-. Interbreath intervals were also
reduced during the approach to the CS+. It should be noted that when approaching a CS+,
participants had just passed through a low threat zone, meaning that increased levels during the
eight second window prior to reaching the CS+ are likely due to anticipation of the CS+ rather
than lingering effects of the environmental stimuli associated with the preceding low threat zone.
Participants approached the CS- immediately following a high threat zone, yet responses in
anticipation of CS+ were still augmented compared to CS- despite tonic response levels likely
being elevated during the zones immediately preceding the CS-.
For conditioning effects to manifest, learning of the CS-US contingency must first take
place (Dawson & Biferno, 1973; Dawson & Furedy, 1976; Lovibond & Shanks, 2002). This
explains why differential responding in anticipation of the CSs only occurred in the second and
third zones during acquisition. SCR amplitudes were higher in the first zone than the second
zone, though the responses in the first zone were high regardless of whether CS+ or CS- was
being approached. This was due to the novelty and uncertainty related to crossing the first pair of
zone markers.
Unexpectedly, anticipatory responses to the CSs extinguished immediately upon
beginning the navigation task with regard to nearly all psychophysiological response measures.
67

Also surprising was the fact that the only differential responding to the CSs was found in the IBI
measure, which did not evidence differential responding during acquisition. One possible
explanation for the lack of any differential responding during the extinction phase is that half of
the participants experienced a high threat zone last before reaching the goal zone marker at the
end of the initial tour through the city; thus, it quickly became evident that high threat stimuli
were no longer being presented before reaching the first zone marker on the return trip. To test
this, participants were split depending on which of the two zone orders were experienced and the
same set of extinction analyses were performed on both sets of participants. However, no
differential responding was evidenced in relation to electrodermal activity in either group. It is
also possible that participants were sufficiently focused on the navigation task to reduce any
differential anticipatory responses to the CSs. Previous research supports this notion, as increased
cognitive task demands tend to decrease the likelihood of conditioned responses (Carter,
Hofstötter, Tsuchiya, & Kock, 2003).
Response levels during the previously high and low threat zones were also investigated to
determine if complex conditioning had taken place. Areas of the city that were previously high
threat zones were considered complex CS+’s while previously low threat zones were complex
CS-‘s. The only evidence of differential responding to the complex CSs during extinction came
from the LF component measure of HRV, such that there was greater LF component power in
previously high threat zones. This suggests that greater sympathetic activation was associated
with the previously high threat zones during extinction. While posttraumatic stress symptoms
were not assessed in the current study, possible risk factors for symptomatology were sought with
use of the complex conditioning paradigm. It is of interest to note that past research has
evidenced increased LF component power in post-traumatic stress disordered patients compared
to controls (Cohen et al., 1997). Further testing of the reliability of the results described here is
68

needed, as well as tests including assessment of post-traumatic symptomatology, but the LF
component may provide insight into the development of complex learned associations between
fear inducing stimuli and environmental contexts in which they are experienced.
No other response measure evidenced differential responding to the complex CSs during
extinction. The length of the complex CS exposure may underlie these results. It was rapidly
made clear that high threat stimuli were no longer being presented when participants entered
previously high threat zones, but the participants still had to spend an extended amount time in
these zones to successfully navigate through to the subsequent zone. In past research, contextual
conditioning has often been assessed with a use of startle probe presented during the previously
threatening environment and responses were compared to those to the same probe presented in a
previously neutral environment (Alvarez et al., 2007; Baas et al., 2004; Grillon et al., 2006). It is
possible that potentiated startle responses may have been exhibited by participants while
navigating through previously high threat zones; however, this was not tested in the current
research. Additionally, participants were confronted with the added concern of performing the
navigation task, and the lack of threatening stimuli quickly allowed them to focus solely on this
task, which may have added to the lack of differential responses during extinction.
Regression and Artificial Neural Network Model Comparisons
After developing an understanding of the response patterns associated with varying levels
threat, route-learning, and navigation, the next step was to determine whether these response
patterns could effectively predict navigation performance. This was done first by comparing two
regression models that used two separate sets of predictor variables calculated from responses
derived from the same sample set. Next the same set of predictors that led to increased predictive
power in one regression model compared to the other, were used to train a BPN to investigate
69

whether a non-linear model would provide a better fit to the data and lead to more robust
predictive power.
It was determined that the set of predictors calculated from the difference in response
levels between high and low threat zones in zone pair A compared to zone pair C were preferable
to the predictors calculated from the overall difference between high and low threat zones. The
preferred predictors led to the development of a better fitting MLR model, which was able to
explain a greater proportion of the variance in navigation performance. This was likely due to the
strong habituation of threat responses evidenced in the threat level manipulation task across
zones. For example, as seen in Figure 8, differential SCLs in response to high and low threat
zones were greatest during the first zone pair, but little difference existed in the third zone pair.
By calculating differences between the first and third zone pairs, habituation effects were
involved in the calculation of the predictor variables. Additionally, interbreath intervals and SFs
were significant predictors of navigation performance using the preferred model. Participants who
displayed reduced differential responding between high and low threat zones in zone pair C
compared to zone pair A performed more efficiently on the navigation task. This suggests that
those who were able to habituate to the threatening stimuli were better able to learn the route
through the city. IBIs represented the only significant predictor using the predictors calculated
from the overall difference between high and low threat zones. It is worth noting that IBIs were
not sensitive to zone order effects, thus the predictor that was least influenced by habituation
effects was the only significant predictor using the overall difference scores.
The same set of preferred predictor variables was then used to develop a BPN to
investigate whether a neural network model or a more traditionally employed MLR model would
better predict navigation performance. Due to increased predictive power and a sizable decrease
in the error term, the BPN was established as the most effective model, leading to the conclusion
70

that the relationship between the predictors, or inputs, and the criterion is likely non-linear. The
BPN is a more robust prediction tool as it has the ability to fit non-linear data with use of sigmoid
transfer functions in the hidden layer units, which can create universal approximators (Cybenko,
1989). Additionally, the BPN is likely more generalizable to new data collected in this
simulation. The network was trained using data from 60% of the subjects in this study, an
additional 20% of the data points were utilized for fine tuning of the network weights in the
validation phase. Finally, the remaining 20% of the data was applied to the network to assess the
trained network’s generalizability and to ensure that over-fitting of the network to the training set
had not occurred. Moreover, as new data are collected and applied to the BPN, the network
weights can continually be updated to provide a sustained and likely improved fit to the data.
Possible future applications of this model are discussed in the general discussion section.

71

Chapter Seven: Experiment 2 Overview
Purpose
The primary purpose of Experiment 2 was to examine differences in psychophysiological
response patterns brought about by varying levels of cognitive workload in a VE, and to
determine the effect of varying levels of cognitive workload on wayfinding through the VE. The
response patterns caused by the varying levels of cognitive workload would then be used to train
an ANN to predict the behavioral outcome measure of the duration of time needed to return to the
starting point during the navigation task.
Design
The cognitive workload manipulation in Experiment 2 employed a 2 (cognitive workload
level; high vs. low) by 3 (zone pair) by 2 (navigation awareness) mixed design. The cognitive
workload level and zone pair variables were within-participants variables, while the navigation
awareness variable represented a between-participants variable. As in Experiment 1, participants
were either told of the navigation portion of the experiment before entering the VE (NA group),
or they were told only after the initial tour through the city upon completing the cognitive
workload manipulation portion of the experiment (NU group).
Participants
A total of 50 students (64.0% female; mean age = 19.62; age range = 18 to 25) from the
psychology subject pool at the University of Southern California served as participants. Exactly
half (n = 25) of the participants experienced a high cognitive workload zone first, while the other
half were initially exposed to a low cognitive workload zone. The NA group consisted of 26
participants (69.2% female; mean age = 19.84; age range = 18 to 25). The NU group had 24
members (58.3% female; mean age = 19.37; age range = 18 to 23). There were no significant
72

differences related to gender or age between the two groups. All participants signed an informed
consent form approved by the University of Southern California’s Institutional Review Board.
Procedures
Pre-test questionnaires. Pre-test questionnaires were identical to those employed in
Experiment 1.
Psychophysiological equipment setup. The same procedures were followed as were
outlined in Experiment 1.
Baseline. The baseline procedure was structured in the same manner as in Experiment 1,
with three 30 second zones. The first zone consisted of the low intensity cognitive workload
condition, in which the participant was not given any additional task aside from following the
guide. This initial zone was used as a habituation portion of the baseline procedure to allow the
participant to adjust to the novelty of being in a VE. The second zone experienced was again low
in cognitive workload. The second zone was meant to assess the participant’s lowest levels of
psychophysiological response while immersed in the VE. The third zone included the high
intensity cognitive workload task, meant to assess the participant’s maximum levels of
psychophysiological response caused by increased cognitive load. Each zone was again preceded
by a zone marker. During the high intensity cognitive workload zone, participants were asked to
complete a paced auditory serial addition task (PASAT), which is described in greater detail
below. As in Experiment 1, the baseline procedure was used to reduce the novelty of VE
exposure and to allow for practice of the task requirements.
Cognitive workload manipulation task. During the cognitive workload manipulation task,
the participant followed a group of 3 soldiers, which served as the guides through the 6 zones.
The zones alternated between high and low intensity cognitive workload. During the low
73

cognitive workload zones participants were not asked to perform a cognitive task other than
following the guides.
Participants were given the following instructions prior to starting the task: “Ok soldier.
Your task is to make it through this city alive. You’ll be following a predetermined path that
includes markers along the way, like the one you see in front of you. Stay close to your guide,
we’re not sure how safe this city is. At certain points you will be receiving radio communications
in the form of numbers. These numbers are a code that you don’t need to fully understand at this
point. Your task is to add together the last two numbers that you hear and say the sum aloud. If
you hear the numbers 4, 7, you will say 11. If the next number is 2, you will say 9, because the 7
and 2 add up to 9. The numbers that you speak give your company information about the
whereabouts of possible insurgent activity and could affect your path. Now move out.” All
participants were given a few practice trials to be sure they understand the cognitive task before
the orders were given to begin the task. At this point, participants in the NU group began the tour
through the city. Participants in the NA group were again given additional instructions related to
the navigation back task, which were presented just prior to the command to move out.
High cognitive workload zones were preceded by red zone markers, while low cognitive
workload zones were preceded by yellow zone markers. Low cognitive workload zones did not
include any specific cognitive task aside from following the guides, whereas the high cognitive
workload zones included the PASAT. The type of starting zone was counterbalanced across
participants, where half of the participants in each group experienced a high cognitive workload
zone first and the other half initially experienced a low cognitive workload zone.
The PASAT is a standardized and validated test that involves adding together only the
last two numbers heard and saying the sum aloud (see Tombaugh, 2006 for review). PASAT form
A was utilized in this experiment, with an inter-stimulus interval of 3 seconds. The PASAT was
74

chosen as the cognitive task due to its relative difficulty, which increased the variability in
response performance between participants. Each zone was 60 seconds long, allowing for 20
PASAT stimuli per zone. The PASAT includes 60 total stimuli, which in this study, were spread
across the three high cognitive workload zones. The participant responded aloud for each answer
and the experimenter checked a box next to each stimulus if the correct response is given.
Navigation task. After reaching the goal zone marker in the cognitive workload
manipulation task, participants were led by the guides past the marker and around the first corner,
where they were given instructions identical to those delivered in Experiment 1 to explain the
navigation back task.
Following the instructions, participants commenced navigating back through the city. No
high intensity workload stimuli (i.e., PASAT stimuli) were presented during the return trip. The
navigation task ended when the participant crossed the zone 1 marker.
Post-test questionnaires. Experiment 2 employed the same post-test questionnaire as that
which was used in Experiment 1.
Dependent Variables
The primary dependent variables associated with the cognitive workload manipulation
task were psychophysiological and behavioral responses during the high and low cognitive
workload zones. The psychophysiological responses measured are listed in Table 1. Behavioral
responses refer to the participants’ response accuracy during the PASAT.
During the navigation task, the same behavioral measures listed in Table 2 were used.
Additionally, all psychophysiological measures were again recorded and analyzed.
Experiment 2 Hypotheses
Cognitive workload manipulation hypotheses. Psychophysiological response increases in
high cognitive workload zones were expected compared to responses elicited by the low
75

workload zones. Heart rate was expected to increase with greater task demand for a novice
performer (Fairclough, 2008); thus participants in this experiment, who were expected to be
novice PASAT performers, would exhibit increased heart rate from low to high cognitive
workload zones. Respiration has been shown to increase with greater task difficulty (Brookings,
1996), which was anticipated to lead participants in the current study to display an increased
respiration rate in high cognitive workload zones. Likewise, pupil diameter was expected to
increase in high cognitive workload zones, which would be in accordance with past research
(Beatty, 1982; Fairclough, 2009). SCL was also predicted to increase during high cognitive
workload zones, as are the number of non-specific skin conductance responses. Kobayashi et al.
(2007) found that skin conductance increased during the interference portion of the Stroop task
compared to the less demanding word-reading and color-naming portions of the task.
Additionally, the increased skin conductance responses were associated with increased response
times, leading to the conclusion that increased cognitive workload leads to increased skin
conductance.
Group assignment was also expected to affect responses elicited by the varying cognitive
workload zones. Because the NA group had the dual task of performing the PASAT and
remembering the path of the guides, the cognitive workload of this group’s members was
increased compared to those in the NU group. This should have led to greater responding from
the NA group regardless of zone type, as the cognitive workload was greater even in the low
workload zones due to the added task of committing the path to memory.
It was also hypothesized that performance on the PASAT would vary depending on
group assignment. Participants in the NA group had the added task of remembering the route they
were proceeding along during the tour through the city, which required additional cognitive
resources and was expected to lead to a greater number of PASAT response errors during the high
76

cognitive workload zones. Participants in the NU group could commit a greater proportion of
cognitive resources to the PASAT, as they were not expected to concentrate on remembering the
route they were taking through the city.
Navigation task hypotheses. Due to the increased mental effort required in the high
cognitive workload zones, it was expected that a main effect of cognitive workload level would
be present during the navigation task such that participants would spend more time and use a less
efficient path in the previously high cognitive workload zones, which is in line with previous
research (Meilinger, Knauff, & Bülthoff, 2007; Walker & Lindsay, 2006).
Participants in the NA group were posited to navigate through each zone more efficiently
and more rapidly because they gathered a greater procedural knowledge of the path they had to
navigate, having been told to pay attention to the route prior to beginning the initial tour through
the city.

77

Chapter Eight: Experiment 2 Data Analytic Approach
Data Reduction
Data were scored using the same in-house custom designed Matlab scoring program that
was employed in Experiment 1. Likewise, each response measure was scored following the same
response parameters outlined in Experiment 1. In order to maintain continuity with Experiment 1,
all tonic measures were again assessed during a trimmed version of the complete zone, excluding
the initial and final ten seconds of each zone, during both the initial tour through the city and the
navigation back. Experiment 2 did not include a fear conditioning component, so phasic
responses in anticipation of zone markers were not analyzed.
Analyses
Cognitive workload manipulation analyses. Analyses related to the cognitive workload
manipulation of Experiment 2 utilized a similar repeated-measures ANOVA to that which was
employed in Experiment 1. The key difference is that the ANOVA used to analyze data gathered
in Experiment 2 included a 2 level cognitive workload within-participants variable rather than the
threat level variable. The ANOVA consisted of a 2 (cognitive workload; high vs. low) by 3 (zone
pair) by 2 (navigation awareness) mixed-model design. Again, separate ANOVAs were run for
each dependant variable. All significant interactions and main effects with greater than one
degree of freedom were supplemented with paired samples t-tests in order to identify the precise
nature of these effects. A modified Bonferroni correction was utilized to prevent inflation of type
1 error rates in all reported significant t-test results (Rom, 1990). Additionally, a Greenhouse-
Geisser correction was used for all reported main effects and interactions with greater than one
degree of freedom.
Additionally, performance on the PASAT was also analyzed with the use of a one-way
ANOVA to examine differences in performance between the NA and NU groups. It was
78

hypothesized that the NA group would exhibit performance decrements due to the increased
difficulty associated with the duel task of completing the PASAT and attempting to commit the
route through the city to memory, which the NU group was not faced with. Correlations between
PASAT performance and psychophysiological response levels were also investigated as a means
of elucidating further the effects of a cognitively challenging task on psychophysiological
response levels.
Navigation analyses. The same 2 (cognitive workload) by 3 (zone pair) by 2 (navigation
awareness) repeated measures ANOVAs were utilized for the analyses related to the navigation
task. Again, separate ANOVAs were used for each dependent variable, which in this case also
included the set of behavioral variables included in the navigation task. Paired samples t-tests
were again used to examine the nature of main effects and interactions, and the same corrections
were employed to protect against type 1 error. Correlations between PASAT performance and
navigation task performance were also investigated to determine the effects of the cognitive
workload task on route-learning.
MLR and ANN Analytic Approach: Experiment 2
MLR and BPN models were developed using methodologies analogous to those
employed in Experiment 1. Key differences between the regression models and BPNs developed
for this experiment included the added predictor variable of behavioral response accuracy on the
PASAT task experienced in the high cognitive workload zones. This variable was added as a
predictor to the MLR models and as an input to the BPN model. See Figure 14 for an example of
the architecture of the BPN developed in Experiment 2. The criterion measure was again the
amount of time needed to complete the navigation back task. Comparisons between the models
will also be carried out in the same manner as described in Experiment 1.

79

Figure 14: Experiment 2 BPN Structure Example.

80

Chapter Nine: Experiment 2 Results
Cognitive Workload Manipulation Task Results
Cognitive workload effects. Psychophysiological measures that were sensitive to changes
in cognitive workload included SCL and IBI. SCLs were significantly higher during high
workload zones, F(1, 48) = 10.31, p < 0.01, and IBIs were significantly shorter during high
workload zones, F(1, 48) = 8.91, p < 0.01. Both results are indicative of increased levels of
psychophysiological arousal associated with increased cognitive workload. Figure 15 illustrates
these findings.
a) b)

Figure 15: Cognitive Workload Main Effects. High threat and low cognitive workload response
level comparisons for a) SCL and b) heart rate. Though electrocardiographic data were collected
and analyzed as IBIs, it has been converted here to beats per minute for display purposes.
Zone order effects. Psychophysiological response levels were sensitive to the order in
which zones within the virtual city were experienced. As in Experiment 1, the zone variable
refers to the average of each high and low zone pair, meaning that zone pair A represents the
average response level between the first high and low zones experienced in the initial tour. Figure
16 displays a summary of the significant results reported below.

5.7
5.8
5.9
6
6.1
6.2
6.3
6.4
High Workload Low Workload
microsiemens (µS)
Skin Conductance Level
66.5
67
67.5
68
68.5
69
69.5
70
70.5
71
71.5
High Workload Low Workload
beats per minute
Heart Rate
81

a) b)

c)

Figure 16: Zone Order Effects. Psychophysiological responses across zones for a) SFs, b)
respiration rate, and c) heart rate. IBIs were again converted to beats per minute for display
purposes, and interbreath intervals were changed to respiration rate for the same reason.

The zone order had a significant effect on the number of SFs elicited during the initial
tour through the city, F(2, 47) = 3.84, p < 0.05. The greatest number of SFs were found in the
first zone pair, which represented a greater number than the second, t(49) = 2.43, p < 0.05, or
third zone pairs, t(49) = 2.29, p < 0.05. The number of SFs did not differ between zone pairs B
and C.
Interbreath intervals were also significantly affected by zone order, F(2, 47) = 8.72, p <
0.001. In this case, interbreath intervals did not differ in the first two zone pairs, but zone pair C
2.5
2.7
2.9
3.1
3.3
3.5
Zone A Zone B Zone C
Number of SFs
Spontaneous Fluctuations
15.6
15.8
16
16.2
16.4
16.6
16.8
Zone A Zone B Zone C
breaths per minute
Respiration Rate
68
68.5
69
69.5
70
70.5
71
Zone A Zone B Zone C
beats per minute
Heart Rate
82

was associated with longer interbreath intervals than either zone pair A, t(49) = 3.29, p < 0.01, or
zone pair B, t(49) = 3.49, p < 0.01.
A significant zone order effect was also revealed with the IBI measure, F(2, 47) = 7.46, p
< 0.01. The IBIs elicited by zone pair A were shorter than those in either the second, t(49) = 3.98,
p < 0.001, or the third zone pair, t(49) = 2.77, p < 0.01.
Cognitive workload task interactions. A significant interaction between cognitive
workload and zone pair was revealed in relation to SCL, F(2, 47) = 9.41, p < 0.01 (Figure 17).
This interaction was mainly the result of the greatest difference between high and low cognitive
workload zones which occurred in zone pair A, t(49) = 3.60, p < 0.01. While there were higher
SCLs in the high cognitive workload portions of the other two zone pairs as well, these
differences were not significant.

Figure 17: Cognitive Workload Intensity by Zone Order Interaction. Interaction between
cognitive workload and zone order for SCL. Response levels during high and low workload zones
only differ significantly in the zone pair A.
Interactions between cognitive workload and navigation awareness were present in
regards to interbreath intervals, F(2, 47) = 5.66, p < 0.05, and a trend in IBIs, F(2, 47) = 3.78, p =
5.4
5.6
5.8
6
6.2
6.4
6.6
Zone A Zone B Zone C
microsiemens (µS)
Cognitive Workload Intensity by Zone Order Interaction
Skin conductance Level
High Workload
Low Workload
83

0.058. Interbreath interval response differences between high and low cognitive workload were
not significant in the NA group, while the differences were nearly significant in the NU group,
t(23) = 2.05, p = 0.052 (Figure 18). Similar results were found in the IBI response levels, such
that the difference between high and low cognitive workload was only significant in the NU
group, t(23) = 3.49, p < 0.01. This suggests that the NA and NU groups differed only during the
high workload condition, when workload is at maximum for the NA group, but did not differ at
all during the low workload condition. To test whether the NA group simply had lower levels of
response when interbreath interval and IBI were concerned, baseline levels of both measures were
compared between the two groups, but no significant differences were revealed.
a)

15.2
15.4
15.6
15.8
16
16.2
16.4
16.6
16.8
17
17.2
17.4
High Workload Low Workload
breaths per minute
Respiration Rate
NA Group
NU Group
84

b)

Figure 18: Cognitive Workload by Navigation Awareness Interactions. With regard to both a)
respiration rate and b) heart rate. In both cases, differential responding during high and low
workload zones was only occurred in the NU group.
PASAT performance. As expected, participants in the NU group demonstrated superior
performance to those in the NA group, F(1, 48) = 5.66, p < 0.05. There was also a significant
practice effect related to zone order, F(2, 47) = 8.01, p < 0.001.Participants exhibited lower
scores in the first zone than either the second zone, t(49) = 4.19, p < 0.001, or the third zone, t(49)
= 3.38, p < 0.01 (see Figure 19). Interbreath interval was the only psychophysiological measure
that correlated with PASAT performance. The total PASAT score was inversely correlated with
the average interbreath interval across all high cognitive workload zones, r(48) = -0.37, p < 0.01,
meaning that those who were breathing faster (i.e., had shorter interbreath intervals) performed
better on the PASAT. However, this overall correlation was due entirely to the inverse correlation
between PASAT scores in the first zone and the associated interbreath intervals gathered during
that zone, r(48) = -0.52, p < 0.001. The correlation between PASAT performance and interbreath
interval was not significant in high workload zone B or zone C.
65
66
67
68
69
70
71
72
73
74
75
High Workload Low Workload
beats per minute
Heart Rate
NA Group
NU Group
85

Figure 19: PASAT Performance. This graph illustrates both the significant learning effect across
zones as well as the significant difference in performance between the NA and NU groups.
Navigation Task Results
Cognitive workload effects. The LF and HF components of HRV power spectrum were
sensitive to differences between formerly high and low cognitive workload zones. The LF
component was greater in formerly high cognitive workload zones, suggesting greater
sympathetic activation during these zones, F(1, 48) = 6.92, p < 0.05. Likewise, the HF component
was also greater during the previously high workload zones, F(1, 48) = 4.52, p < 0.05.
Behavioral measures associated with the navigation task also revealed differences in
performance pertaining to formerly high and low cognitive workload zones. Participants spent
more time navigating through formerly high workload zones, F(1, 48) = 9.07, p < 0.01. There
was also greater deviation from the original path through the city during previously high
workload zones, F(1, 48) = 4.88, p < 0.05, and a greater number of arrows were needed for
assistance in these zones, F(1, 48) = 5.19, p < 0.05.
Zone order effects. The LF and HF power components were also sensitive to zone order
effects. The main effect of zone order on the LF component, F(2, 47) = 18.21, p < 0.001, was due
to an increase in power as participants navigated back through the virtual city. There was
13.5
14
14.5
15
15.5
16
16.5
17
17.5
Zone A Zone B Zone C
Score
PASAT Performance
NA Group
NU Group
86

significantly less LF power during the first zone pair navigating back (zone pair C) when
compared to both the second (zone pair B), t(49) = 3.51, p < 0.01, and third zone pairs (zone pair
A), t(49) = 4.15, p < 0.001. A reverse pattern of results was apparent for the HF component, F(2,
47) = 16.70, p < 0.001, such that the HF power was lower zone pair A than in zone pair C, t(49) =
4.29, p < 0.001, or zone pair B, t(49) = 5.31, p < 0.001 (Figure 20).
a) b)

Figure 20: Psychophysiological zone order main effects. With regard to a) LF power and b) HF
power. As a reminder, the LF component is thought to be associated with sympathetic power,
while the HF component is more closely related to vagal tone.
There was also a zone order main effect associated with the amount of time needed to
navigate through each zone, F(2, 47) = 31.96, p < 0.001, the deviation from the original path, F(2,
47) = 7.62, p < 0.01, and the number of arrows needed for assistance, F(2, 47) = 6.28, p < 0.01.
Participants required less time to navigate through zone pair C than either zone pair B, t(49) =
9.20, p < 0.001, or zone pair A, t(49) = 6.32, p < 0.001. Likewise, participants deviated less
during the first zone pair than zone pair B, t(49) = 3.86, p < 0.001, or zone pair A, t(49) = 3.52, p
< 0.01. A different pattern was found in the number of arrows required. More arrows were
required for assistance in zone pair B than either zone pair C, t(49) = 3.26, p < 0.01, or zone pair
A, t(49) = 2.30, p < 0.05. The number of arrows needed in zone pairs A and C did not differ
(Figure 21).
0
20
40
60
80
100
Zone C Zone B Zone A
Power (normal units)
Low Frequency Component
Power
0
2
4
6
8
Zone C Zone B Zone A
Power (normal units)
High Frequency Component
Power
87

a) b)

c)

Figure 21: Navigation Zone Order Main Effects. Zone order main effects in behavioral
navigation task measures for a) time spent in each zone, b) deviation from the original path, and
c) the number of arrows needed for assistance.
Navigation task interactions. A cognitive workload by zone order interaction was
revealed in relation to interbreath intervals, F(2, 47) = 3.67, p < 0.05, and LF power, F(2, 47) =
4.42, p < 0.05 (see Figure 22). In both cases the only significant difference in response to
formerly high and low cognitive workload zones was present in the third zone pair experienced
during the navigation back (zone pair A). The formerly high cognitive workload portion of zone
pair A was associated with shorter interbreath intervals, t(49) = 2.95, p < 0.01, and increased LF
component power, t(49) = 2.75, p < 0.01.
60
65
70
75
80
85
Zone C Zone B Zone A
seconds
Time Spent in Each Zone
0
10
20
30
40
50
60
70
Zone C Zone B Zone A
distance units
Deviation from Original Path
0
2
4
6
8
10
12
14
16
Zone C Zone B Zone A
number of arrows
Number of Arrows Needed
88

a)

b)

Figure 22: Cognitive Workload by Zone Order Interactions. With regard to psychophysiological
responses including a) respiration rate and b) LF power. Differential responding to high and low
cognitive workload was significant only during the third zone pair of the navigation back task.
The cognitive workload by zone order interaction was also apparent in all three
navigation task behavioral performance measures (Figure 23). The significant interactions related
to time spent navigating through each zone, F(2, 47) = 3.43, p < 0.05, and deviation from the
original path, F(2, 47) = 3.47, p < 0.05, again evidenced the same pattern of results. The only
significant difference between previously high and low cognitive workload zones was found in
the third zone pair experienced during the navigation back (zone pair A) for both the time, t(49) =
14.5
15
15.5
16
16.5
17
Zone C Zone B Zone A
breaths per minute
Navigation Task Respiration Rates
High Workload
Low Workload
40
45
50
55
60
65
70
75
80
85
Zone C Zone B Zone A
Power (normal units)
Navigation Task Low Frequency Component Power
High Workload
Low Workload
89

3.30, p < 0.01, and deviation measures, t(49) = 2.60, p < 0.05. The interaction involving the
number of arrows required, F(2, 47) = 12.77, p < 0.001, was associated with a different pattern of
results. In this case, more arrows were needed for assistance in the previously high workload
zones of both zone pair C, t(49) = 3.04, p < 0.01, and zone pair A, t(49) = 3.49, p < 0.01.
However, during the zone pair B, more arrows were needed during the previously low workload
portion, t(49) = 2.46, p < 0.05.
a)

60000
65000
70000
75000
80000
85000
Zone C Zone B Zone A
seconds
Time Spent in Each Zone
High Workload
Low Workload
90

b)

c)

Figure 23: Behavioral Workload by Zone Order Interactions. With regard to behavioral
navigation task performance measures including a) time spent in each zone, b) deviation from the
original path, and c) the number of arrows needed for assistance.

There was also a pair of significant zone order by navigation awareness interactions.
There was a significant SCL interaction, F(2, 47) = 5.47, p < 0.05, such that while the NU group
0
10
20
30
40
50
60
70
80
90
100
Zone C Zone B Zone A
distance units
Deviation from Original Path
High Workload
Low Workload
0
2
4
6
8
10
12
14
16
18
20
Zone C Zone B Zone A
number of arrows
Arrows Needed
High Workload
Low Workload
91

tended to exhibit increased response levels from the beginning to the end of the navigation back,
the NA group demonstrated the greatest response levels during the first zone pair back (Figure
24). The only significant difference in SCL between zones was found in the decrease in response
level from zone pair C to zone pair B in the NA group, t(25) = 2.26, p < 0.05. IBIs revealed a
similar pattern of results, F(2, 47) = 4.08, p < 0.05, being that the NU group displayed shortened
IBIs as they passed from zone pair C to A, while the NA group responses were consistent with a
habituation curve.
PASAT performance scores did not significantly correlate with any navigation task
behavioral performance measures for either the NA or the NU group.
a)

5.2
5.3
5.4
5.5
5.6
5.7
5.8
5.9
6
6.1
Zone C Zone B Zone A
microsiemens (µS)
Skin Conductance Levels
NA Group
NU Group
92

b)

Figure 24: Zone Order by Navigation Awareness Interactions. With regard to a) SCLs and b)
heart rate.
MLR and BPN Results
Regression model comparisons. The regression model employing predictors based on
overall difference scores was able to explain a significant proportion of the variance in navigation
performance, R
2
= 0.20, F(3, 46) = 3.25, p < 0.05. The HF component was the only significant
predictor of total navigation task duration, though the final equation also included IBIs and SCLs.
As seen in Figure 25, participants who had greater levels of parasympathetic activation during the
high threat zones compared to the low threat zones tended to perform less efficiently on the
navigation task.
64
65
66
67
68
69
70
Zone C Zone B Zone A
beats per minute
Heart Rate
NA Group
NU Group
93

Figure 25: HF Component Power and Navigation Performance. Scatter plot depicting the
relationship between HF component difference scores between high and low threat zones and
navigation performance.
The second set of predictor variables, calculated from the difference between the
responses in zone pair A and zone pair C did not significantly predict navigation performance.
Summary statistics comparing both regression models are presented in Table 13. Additionally,
coefficient statistics for each set of predictors are presented in Tables 14 and 15. The model using
overall difference scores was preferable as it was able significantly explain the variance in
navigation performance and had a lower root mean squared error.
Table 13
Multiple linear regression model summary statistics for Experiment 2
Predictor
Set
R R
2
Adj. R
2
Std. Error RMSE F p
Set 1
*
0.45 0.20 0.12 43039.1 207.5 3.25 < 0.05
Set 2
**
0.38 0.15 0.02 46410.3 215.4 2.05 0.12

Note.
*
Set 1 = set of predictors calculated from the overall difference between high and low
cognitive workload zone responses.
**
Set 2 = set of predictors calculated from the difference
between responses in zone pair A and zone pair C. RMSE = root mean squared error.

-4
-2
0
2
4
6
8
55 65 75 85 95
Difference Scores
Time Per Zone (Seconds)
HF Component Power and Navigation
Performance
94

Table 14
Predictor regression coefficients – Predictors based on overall differences between high and low
cognitive workload zones
Measure b Std. Error β t p
SCL

17618.1 9296.8 0.32 1.90 0.06
SFs

-2976.4 3735.8 -0.13 -0.80 0.43
Interbreath
Intervals
-7.8 13.6 -0.08 -0.57 0.57
IBIs

150.3 80.7 0.32 1.86 0.07
LF
component
17.4 191.0 0.02 0.09 0.93
HF
component
7155.7 2898.6 0.34 2.47 < 0.05
Navigation
Awareness
-3230.2 14736.6 -0.04 -0.22 0.83
PASAT
Performance
-548.5 838.5 -0.10 -0.65 0.52
Note. For each measure, statistics for the final step before elimination from the model are
reported for those predictors that were not included in the final model equation.

Table 15
Predictor regression coefficients – Predictors based on differences between zone pair A and zone
pair C
Measure b Std. Error β t p
SCL

10303.8 5697.0 0.25 1.81 0.08
SFs

-1743.7 2817.7 -0.09 -0.62 0.54
Interbreath
Intervals
4.1 7.3 0.08 0.56 0.58
IBIs

21.5 113.9 0.03 0.19 0.85
LF
component
-119.2 62.7 -0.27 -1.90 0.06
HF
component
2054.9 2868.2 0.16 0.72 0.48
Navigation
Awareness
8208.3 13247.1 0.09 0.62 0.54
PASAT
Performance
-308.1 924.5 -0.06 -0.33 0.74
Note. For each measure, statistics for the final step before elimination from the model are
reported for those predictors that were not included in the final equation.

95

BPN results. The BPN developed included the same eight predictor variables used in the
better performing MLR model (i.e., the overall difference scores between high and low cognitive
workload zones) as inputs to the system. The same procedures used to determine the number
hidden layer nodes in Experiment 1 were again employed here. Optimal model performance was
reached with four hidden layer nodes. A tanh activation function was again applied to the hidden
and output neurons. Table 16 presents a summary of the optimal BPN model parameters.
Descriptive statistics associated with the training, validation, and test set samples are included in
Table 17.
Table 16
BPN model parameters for Experiment 2
Layers
Training
Performance
Test Set
Performance
Learning
Algorithm
Error
Function
Hidden
Activation
Output
Activation
8-4-1 0.801 0.897 Gradient
Descent
SOS Tanh Tanh
Note. The layers refer to the number of inputs – hidden nodes – and outputs. SOS = sum of
squares. Tanh = hyperbolic tangent activation function.

Table 17
Descriptive statistics associated with BPN inputs and target variable for Experiment 2
Inputs Target

SCL

SFs
Interbreath
Intervals

IBIs
LF
Comp.
HF
Comp.
PASAT
Score
Time
Spent
Training
Sample

Mean 0.468 0.463 -52.40 -43.45 5.34 0.770 46.86 438.97
Std. Err. 0.120 0.209 66.75 12.90 6.29 0.345 1.26 6.33
Validation
Sample

Mean 0.086 -0.667 -33.10 -6.85 5.35 -0.074 49.43 444.67
Std. Err. 0.116 0.338 56.08 18.11 4.77 0.143 0.94 6.91
Test
Sample

Mean 0.185 0.000 59.29 -45.92 11.39 0.585 45.43 428.30
Std. Err. 0.113 0.082 99.71 12.21 4.68 0.711 1.41 9.50
Note. All input values are calculated from the overall differences between response levels in the
high and low workload zones. Time spent is reported in seconds to complete the navigation task.

96

Following network training, the test set was applied to the network to test the
generalizability of the model. The BPN was able to predict navigation performance with an
accuracy rate of 80.5%. Following the procedures described in Experiment 1, a global sensitivity
analysis was performed for determination of the relative importance of each input variable to the
successful prediction of the output. Table 18 provides a list of the input variables in rank order.
As a reminder, ratio values less than 1 indicate that the network actual performs better without the
given input variable. However, all ratios associated with the eight input variables are greater than
1, meaning that they are important contributors to the performance of the model.
Table 18
Global sensitivity analysis for the BPN developed for Experiment 2
Rank Measure Ratio Value
1

PASAT
Performance
2.54

2

LF
Component
2.38
3 HF
Component
1.57
4

SFs

1.53
5 IBIs

1.37
6 SCL

1.29
7 Navigation
Awareness
1.24
8 Interbreath
Intervals
1.08

MLR and BPN model comparison. Summary statistics regarding both the selected MLR
model and the BPN are provided in Table 19. The BPN model outperformed the MLR with an
increase of 60.3% in predictive abilities related to navigation performance. The BPN also
provided a better model fit indicated by the smaller root mean squared error term.

97

Table 19
Summary statistics for MLR and BPN comparison in Experiment 2
Model
Sample
Size R R
2
Adjusted
R
2
Standard
Error RMSE
MLR 50 0.449 0.202 0.12 43039.1 207.5
BPN 50 0.897 0.805 0.77 29938.2 173.0
Note. RMSE = root mean squared error.

Direct comparison of correlation coefficients associated with each model with use of the
Fisher z transformation revealed that the BPN has significantly greater predictive ability than the
MLR model, z = 4.91, p < 0.001. Due to the significantly increased correlation coefficient of the
model and the reduction in error, the BPN was the better performing model.

98

Chapter Ten: Experiment 2 Discussion
Cognitive Workload Manipulation Task Effects
Effects of cognitive workload. Experiment 2 sought to investigate psychophysiological
response patterns associated with variations in cognitive workload and whether those responses
could predict navigation performance. It was hypothesized that response levels would increase
during high cognitive workload zones. Increased SCLs and reduced IBIs, indicative of increased
heart rate, provided support for this hypothesis. Increased effort related to completion of the
PASAT task in the high workload zones was responsible for increases in SCLs, which is
supported by past research (Gendolla & Krusken, 2001). Past research has indicated that tonic
heart rate measures tend to increase during cognitive workload zones, especially in novice
performers like those who participated in the current research (Fairclough, 2008). Respiration was
also expected to increase with greater task difficulty (Brookings, 1996), as was the number of SFs
(Verwey & Veltman, 1996), however, interbreath intervals and SFs were not sensitive measures
of varying levels of cognitive workload. Although there were shorter interbreath intervals and
more SFs in high workload zones compared to low zones, these differences were not significant.
Thus, the PASAT task appeared to generate a sufficient level of workload to engender increased
psychophysiological activation in the majority of response measures employed.
Effects of zone order. Similar to responses to threat in Experiment 1, responses to
cognitive workload changes tended to habituate across zones. However, the pattern with which
the habituation occurred was somewhat different and less pervasive across measures. SFs and
IBIs habituated quickly from the first zone pair to the second, and then leveled off. Interbreath
intervals on the other hand, were shorter in the first two zone pairs compared to zone pair C.
Increased activation during zone pair A may have been the result of learning effects related to the
PASAT. Participants performed more poorly on the PASAT during the initial zone compared to
99

the second and third high workload zones. As reviewed in the previous section related to the
effects of cognitive workload, increased task difficulty is associated with increased
psychophysiological activation. As the task was practiced across zones, performance levels
increased and psychophysiological response levels decreased. Berka et al. (2004) reported a
decrease in EEG-recorded vigilance measures due to practice effects. While the authors
concluded that practice lowered vigilance, another interpretation may suggest that habituation of
the response system employed occurred due to less task difficulty experienced due to practice
effects. It is not likely that the reduction in psychophysiological activation was due to decreases
in vigilance in the current study because PASAT performance increased while
psychophysiological responses habituated. Additionally, workload level interacted with zone
order in SCL responses such that the only differential responding between high and low workload
zones was exhibited during the first zone pair. Response levels did not differ across zones during
the low workload condition, but habituation occurred as performance increased across the high
workload zones.
Effects of navigation awareness. Navigation awareness interacted with cognitive
workload level in both the interbreath interval and IBI measures. It was hypothesized that the
dual-task associated with completing the PASAT in addition to route-learning in the NA group
would lead to increased response levels regardless of the workload condition, as even the low
workload zones required more cognitive resources for the NA group than the NU group due to the
added route-learning task. However, as illustrated in Figure 18, this did not appear to be the case.
Here, the NA group generally had lower levels of activation, and the only differential responding
between high and low workload zones was found in the NU group. It was possible that the NA
group simply had lower levels of response when interbreath intervals and IBIs were concerned. In
order to test that hypothesis, baseline interbreath intervals and IBIs were compared between the
100

two groups, but no significant differences were revealed. Alternatively, it is possible that the NU
group was more engaged in the PASAT task because a greater proportion of cognitive resources
could be devoted to this task. For example, Fairclough and Venables (2006) found that respiration
rate was the strongest predictor of task engagement amongst a number of psychophysiological
measures during a prolonged cognitive task, such that increased respiration rate coincided with
increased task engagement. Moreover, these results may be the consequence of overload
experienced by the NA group. In one study using psychophysiological responses to assess
overload, a number of psychophysiological measures evidenced decreased activation as overload
lead to disengagement (Fairclough, Gilleade, Ewing & Roberts, 2010). In support of this
explanation, the NA group did evidence significantly inferior performance on the PASAT task
compared to performance of the NU group. In this case, task difficulty associated with
performing the dual-task of PASAT completion and concurrent route-learning increased the level
of challenge in the environment beyond the skill level of the participants, resulting in
disengagement and performance decrements. These results emphasize the potential benefits that
an adaptive system could have on the performance of the NA group. If behavioral performance
and psychophysiological response levels began to wane during the PASAT task, the adaptive
system could react by decreasing the rate of PASAT stimulus presentation or include visual
stimuli to reduce task difficulty and maintain a flow state for the participants of the NA group,
allowing them to increase their skill level at their own rate.
Navigation Task Effects
Cognitive workload effects. The two measures of HRV analyzed were the only
psychophysiological responses that were sensitive to differential responding to previously high
and low workload zones, such that HRV increased in the previously high workload zones. Rowe,
Sibert, and Irwin (1998) reported that increased levels of HRV were consistent with increased
101

task difficulty beyond human capacity leading to disengagement from the task. Behavioral
navigation performance measures tend to support this view. Participants took significantly more
time in previously high workload zones, in addition to deviating more and requiring more arrows
for assistance. Previous research supports these results, as cognitively distracting tasks presented
during the route-learning phase have led to decreased efficiency during wayfinding tasks
(Meilinger, et al., 2007), likely due to a switch in attentional resources to the secondary
distracting task (Walker & Lindsay, 2006). The psychophysiological results reported here, in
conjunction with the behavioral performance measure outcomes, support Rowe et al.’s view that
HRV may be a sensitive index for task overload and disengagement and may lend itself well to
adaptive systems employed to maintain high levels of engagement, such as those used in air
traffic control simulations when vigilance must be maintained over long periods of time.
Zone order effects. HRV measures also provided an index of psychophysiological
activation across zones. The LF component increased across zones as participants navigated back
through the city, while spectral power associated with the HF component decreased across the
same zones. This would indicate that participants exhibited greater sympathetic activation as they
ventured further from the starting point of the navigation task (i.e., the goal zone marker of the
initial tour). Intuitively, these results seem sensible especially considering that behavioral
navigation performance declined across zones. The difficulty experienced in finding the original
route through the city increased across zones, likely leading to increased levels of frustration or
stress, which was also the case in Experiment 1.
Zone order also interacted with cognitive workload with regard to a number of
psychophysiological and behavioral measures. In each case, differential responding between
previously high and low workload zones was only apparent during the third zone pair experienced
in the navigation task. It may be of interest to note that the third zone pair in the navigation task
102

(zone pair A) is the same physical zone pair experienced first during the initial tour. The first
zone in the initial tour was associated with increased psychophysiological responding, and low
PASAT performance. Thus, in addition to being the furthest area of the city from the starting
point of the navigation task, zone pair A also represents the area of the city in which workload
task performance was the most inferior and psychophysiological activation was greatest. This
suggests that route-learning may have suffered during this zone due to task difficulties and
overstimulation associated with the novice level of experience within the VE at that point in the
learning phase. Difficulties in route-learning exacerbated by the high cognitive workload task led
to the worst navigation performance and highest psychophysiological activation during the third
previously high threat zone experienced in the navigation task (See Figures 21 and 22).
Navigation awareness effects. Navigation awareness interacted significantly with zone
order with regard to SCLs and IBIs. In both cases, members of the NA group tended to habituate
across zones, while members of the NU group exhibited increased activation across zones.
Similar to the results discussed in Experiment 1, the NU group appeared to experience increased
levels of arousal as they navigated further through the virtual city, which is likely due to
increased levels of stress or frustration due to the increased wayfinding difficulties experienced.
One study successfully used SCLs, IBIs, and other peripheral psychophysiological measures in an
adaptive game-playing scenario (Liu, Agrawal, Sarkar, & Chen, 2009). As SCL increased, the
HCI would adjust the intensity of the anxiety and frustration inducing aspects of the game in
order to optimize performance. Participants exhibited increased performance outcomes and
reported that the psychophysiological affect-based adaptations in the game made for a more
challenging and satisfying experience than did performance-based adaptations when the two were
compared.

103

Regression and Artificial Neural Network Model Comparisons
As described in Experiment 1, a pair of regression models using predictor variables
calculated to represent either the overall set of response levels across zones, or to create an index
of habituation effects were compared. Contrary to the results discussed in Experiment 1, overall
differences between high and low cognitive workload zones better predicted navigation
performance. It is possible that habituation had less of an effect during the cognitive workload
task. The PASAT task was challenging, and likely required high levels of engagement, as
evidenced by decreased performance in the NA group due to difficulties associated with dual-task
completion. PASAT performance was also included as a predictor in the model, which increased
with practice across zones. Fewer response measures collected throughout the cognitive workload
task were significantly influenced by habituation effects compared to responses collected during
the threat manipulation task.
The BPN developed for Experiment 2 utilized the same overall response difference
predictor variables employed in the preferred MLR model as inputs to the system. As was the
case in Experiment 1, increased predictive power and a decrease in the error term led to the
determination that the BPN was the most effective model for prediction of navigation
performance based on psychophysiological and behavioral responses to a cognitive workload
manipulation during route-learning. Potential applications of the developed model are discussed
below.

104

Chapter Eleven: General Discussion
The current research offers a number of beneficial design advances for potential use in
future training simulation technologies and adaptive systems in general. A VE was developed that
was capable of providing a route-learning scenario and the ability to test route-knowledge with
use of a navigation task. Manipulations embedded within the VE also afford the opportunity to
test the effects of varying levels of either threat in the environment or cognitive workload
associated with a distracting task. Psychophysiological response metrics recorded were able to
confirm the successful implementation of sufficient levels of threat and workload to produce
changes in psychophysiological response patterns specific to each. Additionally, the novel
approach of manipulating awareness of the fact that a navigation task was going to follow the tour
through the city allowed for differences in response patterns brought about by the added task of
route-learning in the NA group to be compared to the responses of participants of the NU group,
focused solely on threat or workload manipulations. Finally, models were designed to predict
navigation performance based on psychophysiological and behavioral response measures
collected during the route-learning phase. Evidence presented led to the conclusion that BPNs
were better able to predict performance outcomes, and were generalizable to previously unseen
data following training of the model. The goal of this study was to develop strategies for the
successful development of systems that utilize psychophysiological computing to adapt to the
individual in such a way that an optimal pace for training is achieved in order to foster ideal
learning settings. A number of findings reported in the current research provide informative
material for such adaptive system development.
Response patterns produced by variations in navigation awareness during the route-
learning phase differed with respect to threat and workload related stressors. Navigation
awareness appeared to have little effect on responses to threat. Participants in the NA and NU
105

groups in Experiment 1 differed only in their respiratory responses throughout the task. On the
other hand, navigation awareness interacted with cognitive workload in Experiment 2, such that
only the NU group responded differentially to high and low workload zones, and differences
between the two groups only occurred during the high workload conditions. Comparison of the
results of the two experiments suggest that while active route-learning has little effect on
psychophysiological responses to threat, it is possible that the cognitive resources required for
the dual-task of completing a challenging mental arithmetic task (i.e., the PASAT) while actively
engaging in route-learning, may lead to cognitive overload causing reduced response due to
disengagement. Indeed, participants in Experiment 2 who were engaged in route-learning also
exhibited decreased performance on the PASAT task. This information may be useful to adaptive
system design. If participants being monitored to assess variations in cognitive workload fail to
respond differentially between varying task difficulty conditions, it is possible that a state of
overload has been entered, and the system should adapt to lower task difficulty. Adaptive
automation systems generally utilize psychophysiological responses to assess user-states, such as
overload, in order to determine the necessity of automated assistance to facilitate optimal system
performance (Parasuraman & Wilson, 2008).
Wilson and Russel (2007) described the use of adaptive automation to enhance the
operation of an uninhabited air vehicle. Psychophysiological data were recorded and fed into an
ANN in real-time to determine high and low levels of cognitive workload. Task difficulty was
manipulated to create periods of high and low cognitive workload, and automated assistance was
initiated when participants surpassed a threshold for workload response measures. Automated
assistance based on psychophysiological response was found to improve performance compared
to randomly initiated automated assistance and to the no-aiding condition. Additionally, an
adaptive training model has been proposed (Coyne, Baldwin, Cole, Sibley, & Roberts, 2009),
106

which is based on the notion that when the difficulty level or working memory requirements of a
given task exceed the workload capacity of the trainee, training is impeded.
While psychophysiological responses to threatening stimuli may not be affected
significantly by route-learning, effects of habituation on threat responses led to differing sets of
predictor variables better suited for navigation performance prediction in Experiments 1 and 2. In
Experiment 1, responses to threat habituated almost universally throughout the task. Thus, a set of
predictors designed to account for habituation effects produced better prediction of navigation
performance. Habituation effects in response to a high cognitive workload task in Experiment 2
were not as robust, leading to better predictive abilities being associated with overall response
differences between high and low workload zones throughout the initial tour. This distinction can
be used to inform future adaptive system design in that thresholds for adaptations based on
responses to threatening stimuli must be concerned with the change in response levels with
repeated exposure to the stimuli allowing for dynamic adjustment to thresholds for change.
Finally, the current research provided encouraging support for the use of ANNs for
prediction of performance outcomes based on psychophysiological response measures. In both
Experiment 1 and 2, BPNs provided significantly enhanced predictive abilities compared to
traditional MLR models. This demonstrated that psychophysiological responses to varying levels
of threat and cognitive workload during a route-learning task could be used to predict
performance on a subsequent navigation task with better than 80% rates of accuracy. Recently,
researchers have begun applying advanced algorithms such as ANNs for data classification in
real-time. For example, a number of studies have utilized ANNs for the initiation of adaptive
assistance when features meet classification requirements for a state of overload (e.g. Gevins et
al., 1998; Wilson & Russel, 2003). These techniques are often used for assessment and
classification of nonlinear data (see Parsons, Rizzo, & Buckwalter, 2004). The models produced
107

in the current research lend themselves well to use in adaptive training simulations to enhance
route-learning abilities when confronted with threatening stimuli or increased task difficulty due
to a secondary task. An adaptive automation approach can be employed to training making use
the VE developed herein, such that psychophysiological responses gleaned during the route-
learning phase can be assessed for hyper- or sub-threshold criteria related to overload or fear, and
adaptive assistance may be provided during the navigation task to fit the needs of the individual
and promote optimal performance.

108

References
Allanson, J. (2002). Electro physiologically interactive computer systems. IEEE Computers, 35,
60–65.

Allanson, J., & Fairclough, S.H. (2004). A research agenda for physiological computing.
Interacting with Computers, 16, 857–878.

Alvarez, R.P., Johnson, L., & Grillon, C. (2007). Contextual-specificity of short-delay extinction
in humans: Renewal of fear-potentiated startle in a virtual environment. Learning and
Memory, 14, 247–253.

Alvarez, R.P., Biggs, A., Chen, G., Pine, D.S., & Grillon, C. (2008). Contextual fear conditioning
in humans: Cortical-hippocampal and amygdala contributions. Journal of Neuroscience,
28, 6211–6219.

Appelhans, B.M., & Luecken, L.J. (2006). Heart rate variability as an index of regulated
emotional responding. Review of General Psychology, 10, 229–240.

Baas, J.M., Nugent, M., Lissek, S., Pine, D.S., & Grillon, C. (2004). Fear conditioning in virtual
reality contexts: A new tool for the study of anxiety. Biological Psychiatry, 55, 1056–
1060.

Backs, R.W., & Seljos, K.A. (1994). Metabolic and cardiorespiratory measures of mental effort:
The effects of level of difficulty in a working memory task. International Journal of
Psychophysiology, 16, 57–68.

Baldi, P. (1995). Gradient descent learning algorithm overview: A general dynamical systems
perspective. IEEE Transactions on Neural Networks, 6, 182–195.

Beatty, J., (1982). Task-evoked papillary responses, processing load, and the structure of
processing resources. Psychological Bulletin, 91, 276–292.

Beatty, J., & Wagoner, B.L. (1978). Pupillometric signs of brain activation vary with level of
cognitive processing. Science, 199, 1216–1218.

Berka, C., Levendowski, D.J., Cvetinovic, M.M., Petrovic, M.M., Lumicao, M.N., Zivkovic,
V.T., Popovic, M.V., & Olmstead, R. (2004). Real-time analysis of EEG indexes of
alertness, cognition, and memory acquired with a wireless EEG headset. International
Journal of Human-Computer Interaction, 17, 151–170.

Berntson, G. G., Boyson, S. T., & Cacioppo, J. T. (1992). Cardiac orienting and defensive
responses: Potential origins in autonomic space. In B.A. Campbell, H. Hayne, & R.
Richardson (Eds.), Attention and information processing in infants and adults:
Perspectives from human and animal research (pp. 163–200). Hillsdale, NJ: Erlbaum.

Bishop, C.M. (1995). Neural Networks for Pattern Recognition. Oxford, Carendon Press.

109

Boehm-Davis, D.A., Gray, W.D., Adelman, L., Marshall, S., & Pozos, R. (2003). Understanding
and measuring cognitive workload: A coordinated multidisciplinary approach. Defense
Technical Information Center OAI-PMH Repository. 1–46.

Boiten, F.A., Frijda, N.H., & Wientjes, C.J.E. (1994). Emotions and respiratory patterns: Review
and critical analysis. International Journal of Psychophysiology, 17, 103–128.

Boucsein, W. (2012). Electrodermal activity. Plenum Press, New York.

Bouton, M.E., & Bolles, R.C. (1979). Contextual control of the extinction of conditioned fear.
Learning and Motivation, 10, 445–466.

Bradley, M.M. (2008). Natural selective attention: Orienting and emotion. Psychophysiology, 46,
1—11.

Bradley, M.M., Lang, P.J., & Cuthbert, B.N. (1993). Emotion, novelty, and the startle reflex:
Habituation in humans. Behavioral Neuroscience, 107, 970—980.

Brookings, J.B., Wilson, G.F., & Swain, C.R. (1996). Psychophysiological responses to changes
in workload during simulated air traffic control. Biological Psychology, 42, 361–377.

Byrne, E.A., & Parasuraman, R. (1996). Psychophysiology and adaptive automation. Biological
Psychology, 42, 249–268.

Carroll, D., Turner, J.R., & Hellawell, J.C. (1986). Heart rate and oxygen consumption during
active psychological challenge: The effects of level of difficulty. Psychophysiology, 23,
174–181.

Carter, R.M.K., Hofstötter, C., Tsuchiya, N., & Koch, C. (2003). Working memory and fear
conditioning. PNAS, 100, 1399–1404.

Chaudhry, A., Sutton, C., Wood, J., Stone, R., & McCloy, R. (1999). Learning rate for
laparoscopic surgical skills on MIST VR, a virtual reality simulator: Quality of human-
computer interface. Annuls of the Royal College of Surgeons of England, 81, 281–286.

Chen, H., & Kocauglu, D.F. (2003). A sensitivity analysis algorithm for hierarchical decision
models. European Journal of Operational Research, 185, 266–288.

Chen, J.L., & Stanney, K.M. (1999). A theoretical model of wayfinding in virtual environments:
Proposed strategies for navigational aiding. Presence, 8, 671–685.

Cohen, H., Kotler, M., Matar, M.A., Kaplan, Z., Miodownik, H., & Cassuto, Y. (1997). Power
spectral analysis of heart rate variability in posttraumatic stress disorder patients.
Biological Psychiatry, 41, 627–629.

Coyne, J.T., Baldwin, C., Cole, A., Sibley, C., & Roberts, D.M. (2009). Applying real time
physiological measures of cognitive load to improve training. Lecture Notes in Artificial
Intelligence, 5638, 469–478.
110

Csikszentmihalyi, M. (1990). Flow: The psychology of optimal experience. New York,
HarperCollins.

Cybenko, G. (1989). Approximations by superpositions of a sigmoidal function. Mathematics of
Control, Signals and Systems, 2, 303–314.

Darken, R. P. (1996). Wayfinding in large-scale virtual worlds. Unpublished doctoral
dissertation, George Washington University, Washington, DC.

Davis, M. (1997). Neurobiology of fear responses: The role of the amygdala. Journal of
Neuropsychiatry and Clinical Neuroscience, 9, 382–402.

Davis, M. (1998). Are different parts of the extended amygdala involved in fear versus anxiety?
Biological Psychiatry, 44, 1239–1247.

Dawson, M.E., & Biferno, M.A. (1973). Concurrent measurement of awareness and
electrodermal classical conditioning. Journal of Experimental Psychology, 101, 55–62.

Dawson, M.E., & Furedy, J.J. (1976). The role of awareness in human differential autonomic
classical conditioning: The necessary-gate hypothesis. Psychophysiology, 13, 50 -

Dawson, M.E., Schell, A.M., & Filion, D.L. (2000). The electrodermal system. In J.T. Cacioppo,
L.G. Tassinary, & G. Berston (Eds.), Handbook of Psychophysiology (pp.159–181). New
York, NY: Cambridge University Press.

D’Mello, S. K., Craig, S. D., Gholson, B., Franklin, S., Picard, R., & Graesser A. C. (2005).
Integrating affect sensors in an intelligent tutoring system. In Affective interactions: The
computer in the affective loop workshop at 2005 International Conference on intelligent
user interfaces (pp. 7–13). New York: AMC Press.

Donchin, E., Spencer, K.M., & Wijesinghe, R. (2000). The mental prosthesis: Assessing the
speed of a P300-based brain-computer interface. IEEE Transactions on Rehabilitation
Engineering, 8, 174–179.

Dunn, O.J., & Clark, V. (1969). Correlation coefficients measured on the same individuals.
Journal of the American Statistical Association, 64, 366–377.
Erb, R.J. (1993). Introduction to backpropagation neural network computation. Pharmaceutical
Research, 10, 165–170.

Etzel, J.A., Johnsen, E.L., Dickerson, J., Tranel, D., & Adolphs, R. (2006). Cardiovascular and
respiratory responses during musical mood induction. International Journal of
Psychophysiology, 61, 57–69.

Fairclough, S.H. (2009). Fundamentals of physiological computing. Interacting with Computers,
21, 133–145.

111

Fairclough, S.H., Gilleade, K., Ewing, K.C., & Roberts, J. (2010). Capturing user engagement via
psychophysiology: Measures and mechanisms for biocybernetic adaptation. International
Journal of Autonomous and Adaptive Communications Systems, 10, 1 —17.

Fairclough, S.H., & Venables, L. (2006). Prediction of the subjective states from
psychophysiology: A multivariate approach. Biological Psychology, 71, 100–106.

Farwell, L.A., & Donchin, E. (1988). Talking off the top of your head: Toward a mental
prosthesis utilizing event-related brain potentials. Electroencephalography and Clinical
Neurophysiology, 70, 510–523.

Fournier, L. R., Wilson, G. F., & Swain, C. R. (1999). Electrophysiological, behavioral, and
subjective indexes of workload when performing multiple tasks: manipulations of task
difficulty and training. International Journal of Psychophysiology, 31(2), 129-145.

Fredrickson, M. (1981). Orienting and defensive reactions to phobic and conditioned fear stimuli
in phobics and normals. Psychophysiology, 18, 456–465.

Gendolla, G. H. E., & Krüsken, J. (2001). The joint impact of mood state and task difficulty on
cardiovascular and electrodermal reactivity in active coping. Psychophysiology, 38(03),
548-556.

Gevins, A., Smith, M.E., Leong, H., McEvoy, L., Whitfield, S., Du, R., & Rush, G. (1998).
Monitoring working memory load during compter-based tasks with EEG pattern
recognition methods. Human Factors, 40, 79—91.

Gilleade, K., Allanson, J. (2003). A Toolkit for exploring affective interface adaptation in
videogames. Proceedings of the Human-Computer Interaction International Conference.
Crete, Greece, 2003.

Globisch, J., Hamm, A.O., Esteves, F., & Öhman, A. (1999). Fear appears fast: Temporal course
of startle reflex potentiation in animal fearful subjects. Psychophysiology, 36, 66–75.

Goldin, S. E., & Thorndyke, P. (1981). Spatial learning and reasoning skill (Technical Report R-
2805-ARMY). Santa Monica: Rand, Inc.

Golledge, R.G. (1991). Cognition of physical and built environments. In G. Garling, & G.W.
Evans (Eds.), Environment, Cognition and Action: An Integrated Approach (pp. 35–62).
NY: Oxford University Press.

Grillon, C., Baas, J.M.P., Cornwell, B., & Johnson, L. (2006). Context conditioning and
behavioral avoidance in a virtual reality environment: Effect of predictability. Biological
Psychiatry, 2006, 752–759.

Groves, P.M., & Thompson, R.F. (1970). Habituation: A dual-process theory. Psychological
Review, 77, 419–450.

112

Haarmann, A., Boucsein, W., & Schaefer, F. (2009). Combining Electrodermal responses and
cardiovascular measures for probing adaptive automation during simulated flight. Applied
Ergonomics, 40(6), 1026-1040.

Hagan, M. T., Demuth, H. B., & Beale, M. H.(1996). Neural Network Design. Boston: PWS
Publishing.

Hahm, J., Lee, K., Lim, S.L., Kim, S.Y., Kim, H.T., & Lee, J.H. (2006). A study of active
navigation and object recognition in virtual environments. Annual Review of
CyberTherapy and Telemedicine, 4, 67–72.

Haykin, S. (1999). Neural Networks, a Comprehensive Foundation. New Jersey: Prentice Hall.

Hecht-Nielsen, R. (1989). Theory of the backpropagation neural network. Proceedings of the
International Joint Conference on Neural Networks (pp. 1:593–I:608). San Diego.

Kennedy, D.O., & Scholey, A.B. (2000). Glucose administration, heart rate and cognitive
performance: Effects of increasing mental effort. Psychopharmacology, 149, 63–71.

Kindermann, J., & Linden, A. (1990). Inversion of neural networks by gradient descent. Parallel
Computing, 14, 277–286.

Kobayashi, N., Yoshino, A., Takahashi, Y., & Nomura, S. (2007). Autonomic arousal in
cognitive conflict resolution. Autonomic Neuroscience: Basic and Clinical, 132, 70–75.

Kreibig, S.D., Wilhelm, F.H., Roth, W.T., & Gross, J.J. (2007). Cardiovascular, electrodermal,
and respiratory response patterns to fear- and sadness-inducing films. Psychophysiology,
44, 787–806.

Leeb, R., Keinrath, C., Friedman, D., Guger, C., Scherer, R., Neuper, C., Garau, M., Antley, A.,
Steed, A., Slater, M., & Pfurtscheller, G. (2006). Walking by thinking: the brainwaves are
crucial, not the muscles! Presence: Teleoperators and Virtual Environments, 15(5), 500-
514.

Lovibond, P.F., & Shanks, D.R. (2002). The role of awareness in Pavlovian conditioning:
Empirical evidence and theoretical implications. Journal of Experimental Psychology:
Animal Behavior Processes, 28, 3–26.

Maghami, P.G., & Sparks, D.W. (2000). Design of neural networks for fast convergence and
accuracy: Dynamics and control. IEEE Transactions on Neural Networks, 11, 113–123.

Malliani, A., Pagani, M., Lombardi, F., & Cerutti, S. (1991). Cardiovascular neural regulation
explored in the frequency domain. Circulation, 84, 1482–1492.

Mehler, B., Reimer, B., Coughlin, J.F., & Dusek, J.A. (2009). Impact of incremental increases in
cognitive workload on physiological arousal and performance in young adult drivers.
Journal of the Transportation Research Board, 2138, 6–12.

113

Meilinger, T., Knauff, M., & Bulthoff, H.H. (2008). Working memory in wayfinding: A dual task
experiment in a virtual city. Cognitive Science, 32, 755–770.

Meng, X.L., Rosenthal, R., & Rubin, D.B. (1992). Comparing correlated correlation coefficients.
Psychological Bulletin, 111, 172–175.

Middendorf, M., McMillan, G., Calhoun, G., & Jones, K.S. (2000). Brain-computer interfaces
based on the steady-state visual-evoked response. IEEE Transactions on Rehabilitation
Engineering, 8, 211–214.

Milad, M.R., Orr, S.P., Pitman, R.K., & Rauch, S.L. (2005). Context modulation of memory for
fear extinction in humans. Psychophysiology, 42, 456–464.

Nadolne, M.J., & Stringer, A.Y. (2001). Ecologic validity in neuropsychological assessment:
Prediction of wayfinding. Journal of the International Neurophychological Society, 7,
675–682.

Öhman, A., & Soares, J.J.F. (1994). “Unconscious anxiety”: Phobic responses to masked stimuli.
Journal of Abnormal Psychology, 103, 231–240.

Pagani, M., Lombardi, F., Guzzetti, S., Rimoldi, O., Furlan, R., Pizzinelli, P., Sandrone, G.,
Malfatto, G., Dell'Orto, S., Piccaluga, E., Turiel, M., Baselli, G., Cerutti, S., & Malliani,
A. (1986). Power spectral analysis of heart rate and arterial pressure variabilities as a
marker of sympathovagal interaction in man and conscious dog. Circulation Research,
59, 178–193.
Parasuraman, R., & Rizzo, M. (2006). Neuroergonomics: The brain at work. NY: Oxford
University Press.

Parasuraman, R., & Wilson, G.F. (2008). Putting the brain to work: Neuroergonomics past,
present, and future. Human Factors, 50, 468–474.

Parsons, T.D., Iyer, A., Cosand, L., Courtney, C., & Rizzo, A.A. (2009). Neurocognitive and
psychophysiological analysis of human performance within virtual reality environments.
Studies in Health Technology and Informatics, 142, 247-252.

Parsons, T.D., Rizzo, A.A., & Buckwalter, J.G. (2004). Backpropagation and regression:
Comparative utility for neuropsychologists. Journal of Clinical and Experimental
Neuropsychology, 26, 95–104.

Parsons, T.D., & Reinbold, J.L. (2012). Adaptive virtual environments for neuropsychological
assessment in serious games. IEEE Transactions on Consumer Electronics, 58, 197–204.

Partala, T., & Surakka, V. (2003). Pupil size variation as an indication of affective processing.
International Journal of Human-Computer Studies, 59, 185–198.

Pomplun, M., & Sunkara, S. (2003). Pupil dilation as an indicator of cognitive workload in
human-computer interaction. In Proceedings of the 10th International Conference on
Human-Computer Interaction, V.D.D. Harris, M. Smith, & C. Stephanidis, Eds.
114

Porter, G., Troscianko, T., & Gilchrist, D. (2002). Pupil size as a measure of task difficulty in
vision. Perception, 31, 170–171.

Rachman, S., & Lopatka, C. (1988). Return of fear: Underlearning and overlearning. Behavioral
Research Therapy, 26, 99–104.

Rainville, P., Bechara, A., Naqvi, N., & Damasio, A.R. (2006). Basic emotions are associated
with distinct patterns of cardiorespiratory activity. International Journal of
Psychophysiology, 61, 5–18.

Ramloll, R., & Mowat, D. (2001). Wayfinding in virtual environments using an interactive spatial
cognitive map. In IV2001 Proceedings (pp. 574–583). London: IEEE Press.

Ring, C., Carroll, D., Willemsen, G., Cooke, J., Ferraro, A., Drayson, M. (1999). Secretory
immunoglobulin A and cardiovascular activity during mental arithmetic and paced
breathing. Psychophysiology, 36, 602–609.

Ripley, B.D. (1996) Pattern Recognition and Neural Networks, Cambridge: Cambridge
University Press.

Rom, D.M. (1990). A sequentially rejective test procedure based on a modified Bonferroni
inequality. Biometrika, 77, 663–665.

Rowe, D.W., Sibert, J., & Irwin, D. (1998). Heart rate variability: Indicator of user state as an aid
to human-computer interaction.

Rumelhart, D., Hinton, G., & McClelland, J. (1986). A general framework for parallel distributed
processing. In D. Rumelhart, & J. McClelland (Eds.), Parallel distributed processing,
(pp. 45-76). Cambridge, MA: Massachusetts Institute of Technology Press.

Saltelli, A. (2005). Global sensitivity analysis: An introduction. Proceedings of the 4th
International Conference on Sensitivity Analysis of Model Output, Santa Fe, New Mexico
(pp. 27–43).

Sanchez-Vives, M. V., & Slater, M. (2005). From presence to consciousness through virtual
reality. Nature Reviews Neuroscience, 6(4), 332-339.

Sarle, W.S., 1994. Neural networks and statistical models. In Proceedings of the Nineteenth
Annual SAS Users Group International Conference, (pp. 1538–1550). Cary, NC.

Schaefer, T., Ferguson, J.B., Klein, J.A., & Rawson, E.B. (1968). Pupillary responses during
mental activities. Psychonomic Science, 12, 137–138.

Slater, M., Khanna, P., Mortensen, J., & Yu, I. (2009). Visual realism enhances realistic response
in an immersive virtual environment. IEEE Computer Graphics and Applications, 29,
76–84.

115

Sloan, R.P., Korten, J.B., & Myers, M.M. (1991). Components of heart rate reactivity during
mental arithmetic with and without speaking. Physiology & Behavior, 50, 1039–1045.

Suess, W.M., Alexander, A.B., Smith, D.D., Sweeney, H.W., & Marion, R.J. (1980) The effects
of psychological stress on respiration: A preliminary study of anxiety and
hyperventilation. Psychophysiology, 77, 535—540.

Task Force of the European Society of Cardiology the North American Society of Pacing
Electrophysiology. (1996). Heart rate variability: Standards of measurement,
physiological interpretation and clinical use. Circulation, 93, 1043–1065.

Thorndyke, P.W., & Hayes-Roth, B. (1982). Differences in spatial knowledge acquired from
maps and navigation. Cognitive Psychology, 14, 560–589.

Tombaugh, T.N. (2006). A comprehensive review of the Paced Auditory Serial Addition Test
(PASAT). Archives of Clinical Neuropsychology, 21, 53–76.

Van Oyen Witvliet, C., & Vrana, S.R. (1995). Psychophysiological responses as indices of
affective dimensions. Psychophysiology, 32, 436–443.

Verwey, W. B., & Veltman, H. A. (1996). Detecting short periods of elevated workload: A
comparison of nine workload assessment techniques. Journal of Experimental
Psychology: Applied, 2(3), 270-285.

Walker, B.N., & Lindsay, J. (2006). The effect of a speech discrimination task on navigation in a
virtual environment. Proceedings of the Human Factors and Ergonomics Society 50
th

Annual Meeting. 1536–1541.

Waller, D., Hunt, E., & Knapp, D. (1998). The transfer of spatial knowledge in virtual
environment training. Presence, 7, 129–143.

Werbos, P.J. (1974). Beyond regression: New tools for prediction and analysis in behavioral
sciences. Ph.D. Thesis, Applied Mathematics, Harvard University, Cambridge, MA.

Wilhelm, F.H., & Roth, W.T. (1998). Taking the laboratory to the skies: Ambulatory assessment
of self-report, autonomic, and respiratory responses in flying phobia. Psychophysiology,
5, 596–606.

Wilson, G. F., & Russell, C. A. (2003b). Real-time assessment of mental workload using
psychophysiological measures and artificial neural networks. Human Factors, 45(4), 635.

Wilson, G.F., Russel, C.A. (2007). Performance enhancement in an uninhabited air vehicle task
using psychophysiologically determined adaptive aiding. Human Factors, 49, 1005—
1018.

Winebrake, J.J., & Creswick, B.P. (2003). The future of hydrogen fueling systems for
transportation: An application of perspective-based scenario analysis using the analytic
hierarchy process. Technological Forecasting and Social Chance, 70, 359–384.

Abstract (if available)

Abstract The current study sought to examine the psychophysiological response patterns associated with varying levels of threat and cognitive workload in a highly immersive virtual environment (VE) containing a route-learning and navigation task scenario. Participants were led down a specific path through a virtual city by a group of virtual guides. Upon reaching the goal of this initial tour through the city, participants were instructed to navigate back to the starting point of the tour following the same path taken to reach the goal. Two separate experiments were conducted to examine the effects of threat and cognitive workload variations in the environment separately. Psychophysiological responses to threat in Experiment 1 and varying levels of cognitive workload in Experiment 2 were then utilized to develop multiple linear regression (MLR) and artificial neural network (ANN) models for prediction of performance on the navigation task. Comparisons of predictive abilities between the developed models were performed to determine optimal model parameters. Awareness of the navigation task was manipulated such that half of the participants in each experiment were made aware of the navigation task prior to the initial tour to allow for route-learning assessment, while the other half were told only after tour completion to assess response patterns associated with threat and cognitive workload in the absence of the added task of committing the route to memory. Participants made aware of the navigation task evidenced increased efficiency on the return trip through the city. Additionally, the threat level and cognitive workload manipulations were successful in eliciting varying response patterns during areas of high and low intensity stimulus presentations. Finally, ANN models were determined to better predict navigation performance based on psychophysiological responses gleaned during the initial tour through the city. The selected models were able to predict navigation performance with better than 80% accuracy in both Experiments 1 and 2. Applications of the models toward improved human-computer interaction and psychophysiologically-based adaptive systems are discussed.

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

Classically conditioned responses to food cues among obese and normal weight individuals: conditioning as an explanatory mechanism for excessive eating

PDF

Effects of nicotine abstinence on orienting, executive function, arousal and vigilance

PDF

A functional use of response time data in cognitive assessment

PDF

A virtual reality exergaming system to enhance brain health in older adults at risk for Alzheimer’s disease

PDF

Individual differences in heart rate response and expressive behavior during social emotions: effects of resting cardiac vagal tone and culture, and relation to the default mode network

PDF

Behabioral and neural evidence of state-like variance in intertemporal decisions

Asset Metadata

Creator Courtney, Christopher Gaelan (author)

Core Title Psychophysiological assessment of cognitive and affective responses for prediction of performance in arousal inducing virtual environments

School College of Letters, Arts and Sciences

Degree Doctor of Philosophy

Degree Program Psychology

Publication Date 10/05/2012

Defense Date 08/30/2012

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag cognitive workload,navigation,OAI-PMH Harvest,psychophysiology,threat,virtual reality

Language English

Contributor Electronically uploaded by the author (provenance)

Advisor Dawson, Michael Edward (committee chair), Brekke, John S. (committee member), Huey, Stanley J., Jr. (committee member), Parsons, Thomas D. (committee member), Schell, Anne M. (committee member), Tjan, Bosco S. (committee member)

Creator Email cgcourtn@usc.edu,courtney@ict.usc.edu

Permanent Link (DOI) https://doi.org/10.25549/usctheses-c3-101676

Unique identifier UC11289170

Identifier usctheses-c3-101676 (legacy record id)

Legacy Identifier etd-CourtneyCh-1230.pdf

Dmrecord 101676

Document Type Dissertation

Rights Courtney, Christopher Gaelan

Type texts

Source University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Access Conditions The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the a...

Repository Name University of Southern California Digital Library

Repository Location USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA