University of Southern California Dissertations and Theses

Decoding information about human-agent negotiations from brain patterns

(USC Thesis Other)

Decoding information about human-agent negotiations from brain patterns

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content DECODING INFORMATION ABOUT HUMAN-AGENT NEGOTIATIONS
FROM BRAIN PATTERNS
by
Eunkyung Kim
A Dissertation Presented to the
FACULTY OF THE USC GRADUATE SCHOOL
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulllment of the
Requirements for the Degree
DOCTOR OF PHILOSOPHY
(COMPUTER SCIENCE)
December 2018
Copyright 2018 Eunkyung Kim
Acknowledgments
First of all, I would like to give my special thanks to my PhD advisor Morteza Dehghani.
Without his guidance, the research for this dissertation never would have taken place.
I want to express my heartfelt gratitude to him for patiently guiding me. I am also
grateful for his support as I started my family during the course of pursuing my PhD.
His understanding and consideration has made the toughest time of my life so much
smoother.
Next, I would like to thank other members of my thesis committee: Milind Tambe,
Jonathan Gratch, and Jonas Kaplan, for providing valuable feedback to my research and
pushing me to think deeper. I would also like to thank members of the computational
social science laboratory, particularly those that I have had the pleasure of spending my
PhD career with: Justin Garten, Reihane Boghrati, Aleksandra Litvinova, Niki Parmar,
and Joe Hoover.
Finally, I would like to thank my family for their unconditional love and support they
have provided me over the years. In particular, I would like to thank my parents and
parents-in-law for always supporting me and believing in me. Lastly, I would like to thank
my dear husband, David Inkyu Kim, for always being there for me through happiness
ii
and toughness, for being the best friend of my life, for supporting me to pursue my own
career, and for bringing Jane, the most precious gift ever, to my life.
iii
Table of Contents
Acknowledgments ii
List Of Tables vii
List of Figures viii
Abstract x
Chapter 1 Introduction 1
1.1 Motivation 1
1.2 Hypotheses 5
1.3 Contributions 7
1.4 Outline 8
Chapter 2 Background and Related Work 10
2.1 Human-Agent Interaction 10
2.2 Cognitive Neuroscience of Human-Agent Interaction 14
2.3 Objects Negotiation Task 17
Chapter 3 Decoding Negotiation Information in Human-Agent Negoti-
ations 22
3.1 Introduction 22
3.2 Experiment 25
3.2.1 Participants 25
3.2.2 Procedure 26
3.2.3 fMRI Data Acquisition 27
3.3 Predicting a Human Negotiator's Oer Types 28
3.3.1 Analysis 28
3.3.1.1 Data Labeling 28
3.3.1.2 General Linear Model Analysis 29
3.3.1.3 Multi-Voxel Pattern Analysis 30
3.3.2 Results 34
3.3.2.1 ROI MVPA: Left Dorsal Anterior Insula 34
3.3.2.2 MVPA with Searchlight as a Feature Selection Method 35
3.3.3 Discussion 36
iv
3.4 Decoding an Agent's Emotion and Strategy 38
3.4.1 Agent's Features 38
3.4.1.1 Agent's Emotional Expressions 38
3.4.1.2 Agent's Negotiation Strategies 39
3.4.2 Analysis 40
3.4.2.1 Region of Interest Multi-Voxel Pattern Analysis 41
3.4.3 Results 42
3.4.4 Discussion 45
3.5 General Discussion 47
Chapter 4 Decoding Partner Type in Human-Agent Negotiations 49
4.1 Introduction 49
4.2 Decoding Partner Type from Behavioral Data 51
4.2.1 Online Experiment 51
4.2.1.1 Participants 51
4.2.1.2 Modied Objects Negotiation Task 51
4.2.1.3 Negotiation Partners 55
4.2.1.4 Procedure 56
4.2.2 Analysis 57
4.2.3 Results 58
4.2.4 Discussion 59
4.3 Decoding Partner Type from Brain Data 60
4.3.1 fMRI Experiment 60
4.3.1.1 Participants 60
4.3.1.2 Modied Objects Negotiation Task 60
4.3.1.3 Negotiation Partners 61
4.3.1.4 Procedure 61
4.3.1.5 fMRI Data Acquisition 62
4.3.2 Analysis 62
4.3.3 Results 63
4.3.4 Discussion 64
4.4 General Discussion 65
Chapter 5 Effects of Moral Concerns in Human-Agent Negotiations 68
5.1 Introduction 68
5.2 Moral Foundations Theory 70
5.3 Experiment 72
5.3.1 Participants 73
5.3.2 Sacred-Objects Negotiation Task 73
5.3.3 Agent's Features 74
5.3.3.1 Agent's Emotional Expressions 74
5.3.3.2 Agent's Negotiation Strategies 75
5.3.4 Design 76
5.3.5 Procedure 77
5.4 Analysis and Results 79
v
5.4.1 Individualizing Foundations 80
5.4.2 Binding Foundations 82
5.5 Discussion 83
Chapter 6 Conclusions and Future Directions 86
6.1 Conclusions 86
6.2 Future Directions 89
Bibliography 93
vi
List Of Tables
3.1 Results of ROI MVPA with anterior insular regions. 34
3.2 Payo matrix for each negotiation party in negotiation information study 39
3.3 The negotiation strategy of the non-conceding agent and the conceding agent 39
3.4 Prediction accuracy on an agent's emotional expression and an agent's
strategy from ROI MVPA. One-tail binomial tests were performed for each
condition compared to the chance level. 45
4.1 Payo matrix for each negotiation party in negotiation partner type study 55
4.2 The negotiation strategy of the tough agent and the soft agent in negotia-
tion partner type study 56
5.1 The negotiation strategy of the tough agent and the soft agent in moral
concerns study. 75
vii
List of Figures
2.1 Objects Negotiation Task interface 18
2.2 Modied version of the Objects Negotiation Task interface 19
2.3 Timeline of the Objects Negotiation Task used in the fMRI experiment
discussed in this chapter. Here, I show the timeline of an example in
which the computer agent rejected the participant's oer (phase 3) and
then the participant accepted the computer agent's oer (phase 6). 20
3.1 Data labeling 29
3.2 All four anterior insular regions; right ventral anterior insula (blue), right
dorsal anterior insula (red), left ventral anterior insula (yellow), and left
dorsal anterior insula (green). 32
3.3 Combined t-maps that were used as masks for MVPA with searchlight as
a feature selection method. Each of seven t-maps were generated across
six participants' searchlight results, and then thresholded and binarized.
Areas marked with red indicate that the area is included for one t-map,
and areas with yellow indicate the area is included for all t-maps. Left
dorsal anterior insula is included in six t-maps (green boxes). 35
3.4 Comparison of ROI MVPA with left dorsal anterior insula and MVPA with
searchlight as a feature selection method results. Dashes denote the chance
level (33.3%) and error bars denote standard errors. 36
3.5 Agent's emotional expressions. Angry (left), neutral (middle), and sad
(right). 38
3.6 Payos for agents and participants across both agent strategies. 40
3.7 Frontal pole (green) 41
viii
3.8 (A) Accuracy rate of an agent's emotion prediction. (B) Accuracy rate of
an agent's strategy prediction. In both gures, results from ROI MVPA
with the left dorsal anterior insula and ROI MVPA with the frontal pole
are included. Dashes denote the chance level and error bars denote the
standard errors. 43
4.1 Modied version of the Objects Negotiation Task interface 52
4.2 Modied version of the Objects Negotiation Task timeline 53
4.3 Available emotions: happy, content, neutral, angry and sad. 54
4.4 Payos for agents and participants across both agent strategies. 56
4.5 Anthropomorphism Scores for human-labeled agents and computer-labeled
agents. Higher score means the agent is perceived as more human-like. The
error bar shows standard errors. 58
4.6 Concessions to human-labeled agents and computer-labeled agents. The
error bars show standard errors. 59
4.7 Frontal medial cortex from Harvard-Oxford atlas (left) and overlaid accu-
racy map for all participants (right). A part of frontal medial cortex was
included in the accuracy map (white dotted box). 64
5.1 Sacred-Objects Negotiation Task interface 74
5.2 Emotional expressions used by agents. 75
5.3 Number of sacred items oered for agents and participants across both
agent strategies. 76
5.4 Demand dierence for medicine in highHF cluster (left), in total population
(center), in lowHF cluster (right) 81
5.5 Demand dierence for medicine when perceived as sacred item in high/low
IAP cluster 82
ix
Abstract
In the past several decades, various aspects of human-agent negotiations have been ex-
tensively studied. However, most studies have focused on behavioral eects, and have
not examined the neural underpinnings of dierent behaviors witnessed in human-agent
negotiations. Investigating these neural factors underlying behaviors is important be-
cause it could help our understanding of the cognitive processes involved in human-agent
negotiations, and consequently would help us design more eective agents.
To explore neural substrates of human-agent negotiations, I designed experiments and
collected brain imaging data using functional magnetic resonance imaging. My disserta-
tion presents ndings using multi-voxel pattern analyses of brain imaging data acquired
during human-agent interactions.
My rst nding is that negotiation information can be decoded from brain patterns
during the course of the negotiation. Namely, I found that a human negotiator's oer
types (positively-changed oer, not-changed oer, or negatively-changed oer) can be
predicted using brain patterns shown in emotion-related brain regions. More specically,
the left dorsal anterior insula, which is known to be an emotion-related brain region,
was found to be the best predictor of a negotiator's oer types. Also, agent features,
including emotional expressions and negotiation strategies, can also be decoded from brain
x
patterns. An agent's emotional expressions were found to interact with brain patterns
in the emotion-related region (left dorsal anterior insula), and an agent's negotiation
strategies were found to interact with brain patterns in the decision-making-related region
(frontal pole).
My second nding is that a negotiation partner's type, either a computer program or
another human negotiator, can be decoded from brain patterns in the Theory of Mind
related brain regions including the frontal medial cortex. I also found that people perceive
human-labeled agents more human-like than computer-labeled agents, and the level of
concession in the negotiations is dependent on agent type.
Based on these ndings, I expanded my study to morally-charged negotiations and
explored the role of people's moral concerns on how they react to emotional expressions
and make concessions. My third nding is that people's moral foundations interact with
emotional expressions and concession-making during a morally charged negotiation. To
be specic, participants who had stronger concerns for the Individualizing foundations
(Harm and Fairness) were found to make greater concessions for sacred negotiation items
when faced with a sad opponent than an angry opponent. Also, I found that participants
who had high Binding foundations (In-group, Authority and Purity) are more sensitive
to social status, and make greater concessions in scenarios that involve agents in a higher
social status.
These ndings can potentially broaden the human-agent interaction research spectrum
as they can help us better understand the behavioral results shown by others. Also, by
knowing the neurological underpinnings of these features in more detailed interactions,
xi
we will be able to build more specic and eective agents that interact more naturally
with people.
xii
Chapter 1
Introduction
1.1 Motivation
We encounter many computer agents in our daily lives. From smartphones we can easily
use to execute voice commands through programs like Siri to automated web chat agents
that direct online customers to relevant information through one to one chat, more and
more articial agents are interacting with people. Along with this dramatic increase
of interactions with computer agents, researchers have started studying various agent
features that can enhance these interactions. For example, relevant lines of research
have investigated how to make users more engaged in the interactions with computer
agents (Castellano, Pereira, Leite, Paiva, and McOwan, 2009; Moreno, Mayer, Spires,
and Lester, 2001; Iacobelli and Cassell, 2007).
Negotiation platforms have been used quite often in these studies to examine human-
agent communication, because negotiations involve complex cognitive eort and estab-
lished social interaction techniques. Several studies have attempted to simulate human
1
behavior in human-agent negotiation. For example, Byde, Yearworth, Chen, and Bar-
tolini (2003) designed a system that automates price negotiation between buyers and
sellers over the price and quantity of a given product, and Lin, Kraus, Wilkenfeld, and
Barry (2008) developed an agent that is capable of negotiating with human counterparts
under conditions of incomplete information.
However, a major drawback of these studies is that they did not pay much attention
to examining the underlying cognitive processes involved in human-agent negotiations.
Rather, they mostly focus on replicating human-human negotiation results or trying to
achieve better negotiation outcomes. I believe that exploring neural factors is essential on
advancing our understanding of the fundamental cognitive processes involved in human-
agent negotiations.
The use of functional Magnetic Resonance Imaging (fMRI) allows the investigation
of the neural substrates of these human-agent negotiations. Previous fMRI studies have
showed that brain activities in certain brain regions are related to specic negotiation
settings, such as the negotiation partner's expressed emotion or the partner type (being
either another human or a computer) (Sanfey, Rilling, Aronson, Nystrom, and Cohen,
2003; Van't Wout, Kahn, Sanfey, and Aleman, 2006). One noticeable shortcoming is that
these studies generally used single-shot negotiations such as the Ultimatum game.
In this dissertation, I focus on exploring neural activities during a general multi-round
human-agent negotiation. A multi-round human-agent negotiation is more real-life, than
a single-shot negotiation which is more of an experimental setup. When people negotiate
with a partner over multiple rounds, they have more chance to think about the conse-
quences of their negotiation oer. Hence, more complicated decision-making processes
2
are involved during a multi-round negotiation. Expanding neuroimaging studies from a
single-shot human-agent negotiation to a general multi-round human-agent negotiation
is vital to understanding the way humans interact with computer agents. As far as I
know, my work is the rst study that uses brain imaging data to decode information of
general multi-round human-agent negotiations. In addition, my work is distinguishable
from previous fMRI studies that used conventional univariate analysis because I used the
multi-voxel pattern analysis (MVPA) (Norman, Polyn, Detre, and Haxby, 2006). MVPA
allows to detect dierences between conditions with a signicantly higher sensitivity than
the univariate analysis by examining the multidimensional relationship between patterns
of activity instead of examining the dierences at each voxel.
This dissertation studies brain data acquired during human-agent negotiations to
decode the information about the negotiations. The information includes human ne-
gotiator's oer types and negotiation partner's (agent) properties. To predict human
negotiator's oer types; whether the negotiator concedes, does not change, or asks for
more during the negotiation, I focused on analyzing brain patterns in emotion-related
brain regions. This is because emotion plays a key role in social interactions (e.g. Hess
and Bourgeois, 2010), therefore activity in emotion-related brain regions is believed to
aect human behaviors during various types of negotiations. Well-known emotion-related
brain regions include the insular cortex (Damasio, Grabowski, Bechara, Damasio, Ponto,
Parvizi, and Hichwa, 2000; Ruiz, Lee, Soekadar, Caria, Veit, Kircher, Birbaumer, and
Sitaram, 2013) as increased insula activation was reported when cooperators see their
partner defect (Rilling, Dagenais, Goldsmith, Glenn, and Pagnoni, 2008) and when peo-
ple are presented with unfair oers (Sanfey, Rilling, Aronson, Nystrom, and Cohen, 2003).
3
Using brain data, I also tried to decode negotiation partner agent's properties, such
as agent's expressed emotion and negotiation strategy. To decode this information, I
designed computer agents that display three dierent emotional expressions and apply two
xed negotiation strategies, for six combinations of emotional expression and negotiation
strategy. Then I focused on analyzing brain patterns in emotion-related brain region and
decision-making-related brain region, because each of them is closely related to agent's
expressed emotion or its negotiation strategy. Well-known emotion-related brain regions
include the insular cortex as stated previously, and well-known decision-making-related
brain regions include the frontal pole. The frontal pole has been reported to play a
signicant role in thinking about the future (Okuda, Fujii, Ohtake, Tsukiura, Tanji,
Suzuki, Kawashima, Fukuda, Itoh, and Yamadori, 2003), and people with frontal pole
impairment make disadvantageous decisions (Anderson, Bechara, Damasio, Tranel, and
Damasio, 1999). Therefore, I explored brain patterns in the insular cortex to nd its
relationship with agent's expressed emotion, and ones in the frontal pole to nd the
relationship with agent's negotiation strategy.
Next, this dissertation studies the dierences between human-human interactions and
human-agent interactions. One of the reasons why there are active research on human-
agent interactions is that human-human interactions are often quite dierent from human-
agent interactions (e.g. Gray, Gray, and Wegner, 2007; de Melo, Marsella, and Gratch,
2016), and many factors contribute to these dierences. In this dissertation, I explore
whether a computer agent introduced as another human was perceived more anthro-
pomorphic than one that was introduced as a computer program. I then investigated
4
whether agent type produced behavioral dierences, and whether one type of agent re-
sulted in more concessions compared to another. I also compare brain activity during
interactions with human-labeled and computer-labeled agents to determine whether these
perceptual dierences were also observable in brain patterns, and investigate whether
classiers could be trained to determine whether the participant was playing against a
human-labeled or computer-labeled agent.
Lastly, this dissertation studies the role of people's moral concerns on how they react to
emotional expressions and make concessions. Even though many studies have investigated
how emotional expression aects negotiation processes and outcomes (Ames and Johar,
2009; Choi, Melo, Woo, and Gratch, 2012), little research has paid attention to how moral
concerns impact reactions to emotional expressions and aect concession-making. A step
further from previous fMRI studies where I use common fruit as negotiation items, I
introduce sacred negotiation items to study the eects of moral concerns in the human-
agent negotiations. Adapting the Moral Foundations Theory (Haidt and Graham, 2007;
Haidt, Joseph, et al., 2007; Graham, 2013; Dehghani, Johnson, Hoover, Sagi, Garten,
Parmar, Vaisey, Iliev, and Graham, 2016), I examine the eects of two dierent types of
moral foundations in terms of individualism (i.e., Individualizing foundations and Binding
foundations) on concession-making.
1.2 Hypotheses
Hypothesis 1
In human-agent negotiations, negotiation information is decodable from brain data.
5
{ Emotion-related brain regions play a key role in decision-making paradigms
during human-agent negotiations. Emotions arising during human-agent in-
teraction are an indicator of what type of decision the person is going to make,
potentially predicting one's negotiation behavior.
{ Negotiation agents' properties is decodable from brain patterns in relevant
brain regions. Therefore, an agent's negotiation strategy could be predicted
based on the activity in the decision-making-related brain region, and an
agent's emotional expression could be predicted based on the activity in the
emotion-related brain region.
Hypothesis 2
Participants' negotiation behavior and brain activity are dierent during interac-
tions with human-labeled agents, compared to interactions with computer-labeled
agents, even though both agents used exactly the same strategies and emotional ex-
pressions. Therefore, whether the participant was interacting with a human-labeled
agent or a computer-labeled agent can be decoded from his/her behavior or brain
patterns during negotiations.
Hypothesis 3
In morally charged negotiation, people's moral concerns that focus on individual
rights interact with the agent's expressed emotion, and their moral concerns that
focus on binding individuals within groups interact with the social status of the
opponent.
6
1.3 Contributions
This section gives a brief overview of research contributions of this dissertation.
Predicting Negotiation Oers
This dissertation explores the relationship between brain patterns and oers that
human negotiators make during human-agent negotiations. Using a multi-round,
multi-object negotiation platform (instead of the more common single-shot negoti-
ation), I show that it is possible to predict whether the human negotiator concedes,
does not change, or asks for more during the negotiation from their brain patterns in
the emotion-related brain region. This work has been published in the thirty-eighth
annual meeting of the cognitive science society (Kim, Gimbel, Litvinova, Kaplan,
and Dehghani, 2016).
Decoding Negotiation Agent's Properties
This dissertation explores the relationship between brain patterns and a computer
agent's properties such as its expressed emotional expressions and its negotiation
strategies. With computer agents who show three dierent emotional expressions
and use two dierent types of negotiation strategies, I demonstrate that we can
reliably decode agents' emotional expressions based on the activity in the emotion-
related brain region, and also agents' strategies based on the activity in the decision-
making-related brain region. This work has been submitted to the thirty-ninth
annual meeting of the cognitive science society.
Decoding Negotiation Partner Types
This dissertation explores the relationship between brain patterns and negotiation
7
partner type, which is either another human or a computer program. I demonstrate
that people perceive human-labeled agents more human-like than computer-labeled
agents, and that these dierences can be captured in brain activation. I show
that parts of the Theory of Mind neural correlates are activated in human-labeled
agent conditions, but not in computer-labeled agent conditions. This work has been
submitted to the thirty-ninth annual meeting of the cognitive science society.
Eects of Moral Concerns in Sacred-Objects Negotiations
The last part of this dissertation explores the interplay between people's moral
concerns, emotional expressions and concession-making during a morally charged
negotiation. Adapting the Moral Foundations Theory, I show that Individualizing
foundations including Harm and Fairness interact with reactions to emotions, while
Binding foundations including In-group, Authority and Purity interact with oppo-
nent's social status rather than reactions to emotions. I discuss how these ndings
are important information when designing autonomous agents that are to operate
in morally sensitive negotiations. This work has been published in the thirty-sixth
annual meeting of the cognitive science society (Kim, Dehghani, Kim, Carnevale,
and Gratch, 2014).
1.4 Outline
The rest of the thesis is organized as follows.
8
Chapter 2 presents related works on human-agent negotiations and cognitive neuro-
science, and introduces the Objects Negotiation Task which was used as a framework
of my experiments.
Chapter 3 describes approaches to decode negotiation informations, such as human
negotiator's oer types and a computer agent's expressed emotion and negotiation
strategy.
Chapter 4 describes the work on decoding partner type, either another human or
a computer program, in human-agent negotiations using both behavioral and brain
data.
Chapter 5 explores more complicated human-agent negotiations, by introducing
the Sacred Negotiations Task. The interplay between people's moral concerns,
emotional expressions, and concession-making are explored in the sacred-objects
negotiations.
Chapter 6 gives concluding remarks and discusses interesting and promising future
research directions.
9
Chapter 2
Background and Related Work
In this chapter, I rst provide a brief background regarding human-agent interaction in
section 2.1. Next, I discuss the cognitive neuroscience of human-agent interaction in
section 2.2. Finally, in section 2.3, I describe the Objects Negotiation Task which is the
framework I used for experiments in this dissertation.
2.1 Human-Agent Interaction
As interactions between humans and agents become common, eort has been made to
study these interactions thoroughly. For example, research has examined the contributing
factors to user engagement (Bickmore, Schulman, and Yin, 2010; Castellano, Pereira,
Leite, Paiva, and McOwan, 2009; Iacobelli and Cassell, 2007) and establishment of bonds
with computer agents (Cassell and Thorisson, 1999; Wang and Gratch, 2009; Gratch,
Wang, Gerten, Fast, and Duy, 2007). Bickmore, Schulman, and Yin (2010) showed
that people engage more with life-like computer agents, such as a relational agent who
remembers past history and relates to that history when communicating. Moreover, Wang
and Gratch (2009) demonstrated that computer agents who give non-verbal immediacy
10
feedback, such as eye contact and gestures, are found to establish rapport with human
partners.
Related to my work, various lines of research have investigated the eects of emotion
and strategies in the interactions with agents (Maldonado, Lee, Brave, Nass, Nakajima,
Yamada, Iwamura, and Morishima, 2005; Van Kleef, De Dreu, and Manstead, 2004a;
Das, Hanson, Kephart, and Tesauro, 2001). These factors are particularly important
because they are central in providing clues about the internal states and the intentions of
the counterpart in any type of interactions (Jurafsky, Ranganath, and McFarland, 2009).
Knowing one's internal states and intentions could greatly contribute to an eective
interaction.
Previous studies on the eect of emotion include the work of Maldonado, Lee, Brave,
Nass, Nakajima, Yamada, Iwamura, and Morishima (2005) where they demonstrated
that people who interact with an emotional agent perform better on a test than those
who interact with an emotionless agent in a web-based learning environment. Van Kleef,
De Dreu, and Manstead (2004a) argue that automated agents who express emotions, such
as anger or happiness, elicit dierent levels of concessions based on the type of expressed
emotions, and de Melo, Carnevale, and Gratch (2011a) replicated these eects with virtual
agents who use verbal or non-verbal expression. Sinaceur and Tiedens (2006) further
reveal that the eect of anger on concession works only when anger recipients have poor
alternatives, and Allred (1999) suggests that emotion plays an important role as a signal
(e.g., anger indicates a negotiator's dissatisfaction with his opponent's oer). de Melo,
Carnevale, and Gratch (2011b) also showed that participants cooperate signicantly more
with the agent showing emotions that are consistent with the goal of reaching mutual
11
cooperation, than the one showing emotions representing the goal of maximizing its own
rewards.
The eects of dierent types of negotiation strategies during human-agent interac-
tions have been examined by several researchers. For example, Das, Hanson, Kephart,
and Tesauro (2001) demonstrated that the agreed trade prices from human-agent inter-
actions are dierent for two types of agent strategies; one strategy is to maximize its
expected surplus using trade history and the other strategy is to make small random
adjustments to the trade price continuously. Similarly, Williams, Robu, Gerding, and
Jennings (2011) demonstrated that the particular agent strategy that predicts the op-
ponent's future behavior based on the Gaussian processes elicit more concessions during
negotiations. These have helped the eld establish sets of features that in
uence the
quality of human-agent interaction, resulting in a more enhanced and realistic experience
for the human user.
Along with numerous other studies, the dierences between human-human interac-
tions and human-agent interactions have been widely explored. Researchers have contin-
uously tried to identify what these disparities are and why they occur, with the hopes
to bridge the gap between human-human and human-robot/agent encounters. A robot's
appearance has been found to be paramount in the interaction style of the human sub-
jects. For example, when people interface with robots that have mechanical, nonhuman
like features, even when the robot performs human-like actions, they are often unable to
overlook these traits (Hegel, Krach, Kircher, Wrede, and Sagerer, 2008). Thus, robots
designed to have eyes similar to humans (Banh, Rea, Young, and Sharlin, 2015), or baby
12
face-like heads (Powers and Kiesler, 2006), were found to be more eective in evoking a
more human-like interaction.
Interestingly, it has been also found that people's behaviors during interaction with
a human partner is signicantly dierent from ones with a computer partner (Houser,
Schunk, and Winter, 2006; Van't Wout, Kahn, Sanfey, and Aleman, 2006). Houser,
Schunk, and Winter (2006) compared participants' oers in human interaction treat-
ment with one in computer interaction treatment using a trust game. The participant
knew his/her counterpart was human in the human interaction treatment, while he/she
knew the counterpart was a computer program that make return decisions according to
a known random process in computer interaction treatment. The distribution of risky
decisions with computer counterparts is found to be unimodal and bell-shaped distri-
bution, although the human-counterpart distribution is bimodal, with clustering at the
extremes of investing everything and investing nothing. There has also been research
into physiological responses to both types of partners, for example examining skin con-
ductance activity during the interaction. Van't Wout, Kahn, Sanfey, and Aleman (2006)
showed that skin conductance activity was higher for unfair oers proposed by human
conspecics, but not for unfair oers generated by computers.
The majority of these previous research work has focused on comparing one's behav-
ior during human-human interaction with human-agent interactions. Our understanding
about the cognitive processes involved in human-agent negotiations could be advanced by
examining the neural factors underlying behavior. To examine the neural underpinnings
of human-agent interactions, it is important to understand the functional organization
and neurological processes in the human brain involved in such interactions. The use
13
of functional Magnetic Resonance Imaging (fMRI) allows the investigation of the neural
substrates of these human-agent interactions (Ogawa, Lee, Kay, and Tank, 1990). There-
fore, I review cognitive neuroscience of human-agent interaction which has been greatly
prospered with the development of fMRI in the next section.
2.2 Cognitive Neuroscience of Human-Agent Interaction
Cognitive neuroscientists have studied brain activities in depth to locate brain regions
that are closely related to certain functions. The use of fMRI has allowed this inves-
tigation of the neural substrates in various research domains. Emotion and decision-
making are among the most extensively studied functions because of their importance
in social interactions (LeDoux, 2003; Heekeren, Marrett, Bandettini, and Ungerleider,
2004). It is worth noting that these functions play key roles in human-agent interactions
as well (Cowie, Douglas-Cowie, Tsapatsoulis, Votsis, Kollias, Fellenz, and Taylor, 2001;
Van Kleef, De Dreu, and Manstead, 2004a).
Relevant lines of research have located the emotion-related brain regions which play a
signicant role in generating and processing emotions (Wager and Barrett, 2004; Kober,
Barrett, Joseph, Bliss-Moreau, Lindquist, and Wager, 2008), as well as the decision-
making-related brain regions which are crucial for prospective thoughts about the fu-
ture (Okuda, Fujii, Ohtake, Tsukiura, Tanji, Suzuki, Kawashima, Fukuda, Itoh, and
Yamadori, 2003). Well-known emotion-related brain regions include the anterior in-
sula (Lamm and Singer, 2010; Ruiz, Lee, Soekadar, Caria, Veit, Kircher, Birbaumer,
14
and Sitaram, 2013) and the amygdala (LeDoux, 2003), and well-known decision-making-
related brain regions include the frontal pole (Bechara and Van Der Linden, 2005; Bechara
and Damasio, 2005).
A few studies on emotion-related brain regions have been conducted using a human-
agent interaction framework. Haruno and Frith (2010) showed that fairness of the ne-
gotiation outcome can be predicted based on patterns of activity in the insula and the
amygdala. Rilling, Dagenais, Goldsmith, Glenn, and Pagnoni (2008) reported increased
anterior insula activation when cooperators see their partner defect, and Sanfey, Rilling,
Aronson, Nystrom, and Cohen (2003) reported the same when people are presented with
unfair oers during the Ultimatum Game. Activity in these brain regions is believed to
aect human behaviors during various types of negotiations as emotion plays a key role in
social interactions that are vital during negotiations (Hess and Bourgeois, 2010; Olekalns,
2002). For example, it has been shown that getting an unfair oer or rejecting an oer in
the Ultimatum game triggers negative emotions, such as anger (Pillutla and Murnighan,
1996).
On the other hand, not many studies on decision-making-related brain regions have
used human-agent interaction framework. Rather, studies have been conducted with
brain lesion patients to show clear dierences in decision-making between patients and
controlled group. Brain lesion studies have demonstrated that people with damage in the
frontal pole exhibit a remarkable dissociation between normal intellectual abilities and
severely disturbed decision-making and judgement (Damasio, Everitt, and Bishop, 1996;
Bechara, Tranel, Damasio, and Damasio, 1996). Also, Okuda, Fujii, Ohtake, Tsukiura,
Tanji, Suzuki, Kawashima, Fukuda, Itoh, and Yamadori (2003) revealed that the frontal
15
pole plays an important role in the cognitive processing of future prospects by showing
that frontal pole is activated during tasks in which normal subjects simply think and
orally report their future prospects.
In addition, brain regions associated with Theory of Mind (ToM) have been exten-
sively studied due to their importance in social interactions. ToM refers to the ability of
one person to reason about another person's mental states, including their intentions and
beliefs (Premack and Woodru, 1978). Several neuroimaging studies have attempted to
elucidate the neural substrates related to this distinctive human ability, and found that
brain areas including medial prefrontal cortex and temporoparietal junction are consis-
tently activated during ToM tasks such as the False Belief Task (Gallagher and Frith,
2003; Amodio and Frith, 2006).
These ToM brain areas have been also studied to compare human-human interaction
with human-computer interaction because dierent level of activation were observed in
these areas for the two types of interactions. Previous fMRI studies have demonstrated
that cortical activity in the neural structures related to ToM tend to be more active when
participants were told they were facing a human partner compared to a computer pro-
gram (Gallagher, Jack, Roepstor, and Frith, 2002; Kircher, Bl umel, Marjoram, Lataster,
Krabbendam, Weber, van Os, and Krach, 2009). For example, Gallagher, Jack, Roep-
stor, and Frith (2002) showed that the medial prefrontal cortex, part of the ToM-related
neural structures, is activated while playing rock-paper-scissors with a human player but
not activated when playing the same game with a known pre-dened computer algorithm.
Research also demonstrated that activity in these same regions scaled according to the
human-likeness of their interaction partner when using computer-animated characters
16
or nonhuman agents (Chaminade, Hodgins, and Kawato, 2007; Krach, Hegel, Wrede,
Sagerer, Binkofski, and Kircher, 2008).
2.3 Objects Negotiation Task
Negotiation platforms have been used in many previous studies to examine human-agent
communication, because negotiations involve complex cognitive eort and established
social interaction techniques. Dehghani, Carnevale, and Gratch (2014) introduced the
Objects Negotiation Task which is a web-based multi-round negotiation task. In this
task, a participant and a computer agent can make proposals in turn to negotiate about
the distribution of dierent types of items. I used this task through my dissertation
because aspects of this task are easily congurable and can be used for imagining exper-
iments. This task allows researchers to use diverse negotiation items that have dierent
payos, and various types of negotiation partner agents that employ dierent negotiation
strategies.
Figure 2.1 shows the interface of the Objects Negotiation Task. The top portion
belongs to a computer agent, and the bottom portion belongs to the participant. Items
are initially positioned in the middle portion indicating that they are up for grabs and
need to be fully distributed between the negotiators, without leaving any items in the
middle portion. The participant moves items one by one to his/her own side or to the
partner's side, using a mouse. A computer agent's predened emotional expressions are
shown on the top right, and the participant can choose his/her emotional expressions
on the bottom right side anytime during the negotiation. Emotional expressions that
17
Figure 2.1: Objects Negotiation Task interface
participant can choose include happy, content, neutral, angry and sad. Buttons placed
between both parties' emotional expressions indicate the participant's available actions,
for example, `Make Initial Proposal' button is shown when the participant nish relocating
all negotiation items on the rst round.
Because this task was originally designed for behavioral studies, some changes were
made to optimize the task for use in the fMRI scanner. Figure 2.2 shows the interface of
the modied Objects Negotiation Task. In the modied version of the task, common fruits
were used as negotiation items and less number of negotiation items were used for shorter
item relocations. Participant's emotion selection box was removed so that they can focus
more on negotiation partner's emotional expressions and oers they received, and the
extra wait time is introduced to separate the brain activity of each phase from brain
activity of other phases. Also, the payo for each item for both players were explicitly
specied on the screen. In order to make sure all participants had the same goal during
18
Figure 2.2: Modied version of the Objects Negotiation Task interface
negotiations, they were asked to focus on maximizing their total payos. To ease the
calculation of total payos, the system automatically calculated the player's total payo
as well as the agent's total payo, and displayed whenever the items are redistributed.
The modied Objects Negotiation Task used in this fMRI experiment is composed of
six phases. Figure 2.3 shows an example of the timeline of the Objects Negotiation Task.
1. The participant proposes an oer.
2. The participant waits for the computer agent to accept or reject the oer.
3. If accepted, negotiation ends. If rejected, the participant waits for the computer
agent to propose an oer.
4. The participant reviews the computer agent's oer for ve seconds.
5. The participant decides whether to accept or reject the computer agent's oer.
6. If accepted, negotiation ends. If rejected, the participant waits for ve seconds and
then is redirected to the rst phase.
19
Figure 2.3: Timeline of the Objects Negotiation Task used in the fMRI experiment dis-
cussed in this chapter. Here, I show the timeline of an example in which the computer
agent rejected the participant's oer (phase 3) and then the participant accepted the
computer agent's oer (phase 6).
When the negotiation starts, items are placed in the middle row indicating that they
do not belong to anyone. After the participant distributes all the items, the `Go!' button
on the right bottom corner is enabled and the participant can propose his/her oer
by clicking the button. The participant is then asked to wait until the virtual agent
accepts or rejects the oer. If accepted, both parties get the proposed items and the
negotiation ends. If rejected, the participant is asked to wait for the virtual agent to
propose a counteroer. Next, the virtual agent's oer is shown and the participant
reviews the oer for ve seconds. During this time, the participant simply observes
the counteroer and cannot relocate the items. For the fMRI version of the task, this
review time was introduced to make sure that we can separate brain activity between the
oer-making period and the non-oer-making periods. After the review, the participant
20
decides whether to accept or reject the virtual agent's oer. If he/she accepts the virtual
agent's oer, the negotiation ends. If rejected, the participant is again asked to wait for
ve seconds and then is redirected to the rst step.
Each negotiation can be as long as six rounds. On the sixth round, a message is
displayed to the participant to indicate that it is the last round: `This is the last round of
the negotiation. After seeing the other player's proposal, you can either decide to accept
the other player's proposal or if you don't agree with the proposal, you can decide to toss
a coin. If it comes up `heads', you will get two of each item, if it comes up `tails', you will
get none of the items.' As stated in the message, the participant then can either accept
the computer agent's oer or toss a coin. If a participant chooses to toss a coin, he/she
gets two of each item if the coin lands on heads or nothing if the coin lands on tails.
21
Chapter 3
Decoding Negotiation Information in Human-Agent
Negotiations
3.1 Introduction
There is now considerable amount of research showing that various emotions expressed by
agents can either enhance or hinder interactions (Beale and Creed, 2009; Andr e, Klesen,
Gebhard, Allen, Rist, et al., 2000; Dehn and Van Mulken, 2000; Kim, Dehghani, Kim,
Carnevale, and Gratch, 2014). One example of enhanced interactions is demonstrated by
Brave, Nass, and Hutchinson (2005), who showed that agents showing emphatic emotion
(e.g., showing a happy face when the user won the game and showing a sad face when
the user lost the game) resulted in greater positive responses from users compared to
when agents showed self-oriented emotion. Also, Van Kleef, De Dreu, and Manstead
(2004a) showed that people tend to concede more to an angry counterpart than to a
happy counterpart, and de Melo, Carnevale, and Gratch (2012) suggested that displays
of guilt are eective in eliciting concessions in human-agent negotiation.
22
The majority of these research have focused on replicating human-human negotiation
results, or achieving better outcomes in human-agent negotiations. However, through ex-
amining the neural factors underlying behavior, we could advance research in understand-
ing cognitive processes involved in negotiations. To examine the neural underpinnings
of human-agent negotiations, it is important to understand the functional organization
and neurological processes in the human brain involved in such interactions. The use
of functional Magnetic Resonance Imaging (fMRI) allows the investigation of the neural
substrates of these human-agent negotiations.
Previous fMRI studies have identied specic emotion-related brain regions that play
a signicant role in social functions: the insular cortex (Damasio, Grabowski, Bechara,
Damasio, Ponto, Parvizi, and Hichwa, 2000; Ruiz, Lee, Soekadar, Caria, Veit, Kircher,
Birbaumer, and Sitaram, 2013) and the amygdala (LeDoux, 2003). Increased insula
activation was reported when cooperators see their partner defect (Rilling, Dagenais,
Goldsmith, Glenn, and Pagnoni, 2008) or when people are presented with unfair of-
fers (Sanfey, Rilling, Aronson, Nystrom, and Cohen, 2003). Haruno and Frith (2010)
showed that fairness of the negotiation outcome can be predicted based on patterns of
activity in amygdala. Activity in these brain regions is believed to aect human behav-
iors during various types of negotiations, as emotion plays a key role in social interactions
that are vital during negotiations (Hess and Bourgeois, 2010). For example, it has been
shown that getting an unfair oer or rejecting an oer in the Ultimatum game triggers
negative emotions, such as anger (Pillutla and Murnighan, 1996). However, these neu-
rological studies were mostly limited to single-shot negotiations such as the Ultimatum
game (Sanfey, Rilling, Aronson, Nystrom, and Cohen, 2003).
23
In this dissertation, I build on and extend this line of research by studying the rela-
tionship between brain patterns and decision-making during general multi-round human-
agent negotiations, which are more similar to real-world negotiations than single-shot
negotiations. Investigating brain activity during general multi-round human-agent nego-
tiation is vital to understanding how humans react in interactions with agents. More-
over, investigating the neural systems that are active during dierent types of interactions
would allow us to better understand the underlying cognitive processes responsible for
the resultant behavior.
The contributions of this chapter include ndings on the relationship between brain
patterns and oers that human negotiators make during negotiation with agents. As it
has been shown that emotion plays a key role in social interactions that are vital during
negotiations (Hess and Bourgeois, 2010; Olekalns, 2002), I hypothesized that activity in
emotion-related brain regions would aect oer types during human-agent negotiation.
Emotions arising during human-agent interaction could be an indicator of what type of
decision the person is going to make, potentially predicting one's negotiation behavior.
To test my hypothesis, I recruited human participants and collected brain imaging data
during their negotiations with computer agents.
The contributions of this chapter also include ndings on the relationship between
brain patterns and a computer agent's features. More specically, I studied an agent's
emotional expressions and negotiation strategies. Relevant lines of research have shown
that the anterior insula is activated when processing emotions (Kober, Barrett, Joseph,
Bliss-Moreau, Lindquist, and Wager, 2008; Lamm and Singer, 2010), and the frontal
pole is activated when making a decision that aects the future (Okuda, Fujii, Ohtake,
24
Tsukiura, Tanji, Suzuki, Kawashima, Fukuda, Itoh, and Yamadori, 2003). I therefore
hypothesized that an agent's emotional expression can be predicted based on the human
negotiator's brain activity in the anterior insula, and an agent's negotiation strategy can
be predicted based on the human negotiator's brain activity in the frontal pole. To test
my hypothesis, I chose the anterior insula and the frontal pole as my regions of interest
(ROIs), and performed ROI MVPA analyses on the same dataset I used to predict a
human negotiator's oer types.
In the following sections, I describe the details of my experiment including modied
version of the Objects Negotiation Task which is the framework I used for the experiment.
Next, I explain details about my data analysis on predicting a human negotiator's oer
types and decoding an agent's properties. Lastly, I show my results and discuss my
ndings.
3.2 Experiment
3.2.1 Participants
Ten participants (seven female), recruited from the online campus bulletin board at the
University of Southern California, took part in the experiment. Participants' mean age
was 27.5 years ( 4.93) and all participants had normal or corrected to normal vision.
Each participant was provided with a written informed consent according to the guide-
lines of the USC Institutional Review Board. All participants were screened to rule out
medication use, head trauma, history of neurological or psychiatric disorders, substance
abuse, and other serious medical conditions.
25
3.2.2 Procedure
Participants were instructed to read the hypothetical scenario shown below.
You are a restaurant owner in a small town. There has been a major re in the
market providing the necessary fruits for your restaurant and as a result only a
limited number of fruits are available. Because of this you have to split the avail-
able fruits with another restaurant owner. You and the other owner value each fruit
dierently. In order to run your restaurant you need to get as many fruits as possible.
In the task that follows, you will negotiate about how to distribute the fruits
between you and the other restaurant owner.
Participants learned the rules of the Objects Negotiation Task (as discussed in chap-
ter 2.3) and performed a practice task. The practice negotiation task had an identical
interface to the task performed in the scanner, but with a dierent set of negotiation
items and payos. During the practice negotiation task, participants were provided with
a trackball mouse similar to the one used in the scanner in order to got used to operating
it and moving items around the task interface. After completing the practice negotiation
task, participants were checked with a metal detector to ensure that they were safe to go
inside the fMRI scanner.
Participants viewed a screen positioned behind their head via a mirror mounted about
their eyes. Participants performed a total of six negotiations in the scanner, each including
up to six rounds. Dierent sets of negotiation items were used in each negotiation.
After the fMRI scan, the participants received a short survey to acquire background
26
information including gender, age, and handedness. All participants were paid $30 for
their participation.
Participants were not told that they would be playing with articial agents. To
simulate playing against other humans, I added randomized delays when the computer
agent proposed an oer.
3.2.3 fMRI Data Acquisition
fMRI scans were performed at the USC Dana & David Dornsife Cognitive Neuroscience
Imaging center. Images were acquired using a 3-Tesla Siemens PRISMA MRI scanner
with a 20-channel matrix head coil. Two sets of high-resolution anatomical images were
acquired for registration purposes. A T1-weighted high-resolution image was acquired
using a three-dimensional magnetization-prepared rapid acquisition gradient (MPRAGE)
sequence (TR = 2530ms, TE = 3.09ms,
ip angle = 10

, 256256 matrix). 208 coronal
slices covering the entire brain were acquired with a voxel resolution of 111 mm. I
also acquired a T2-weighted anatomical scan (TR = 10,000ms, TE = 88ms,
ip angle =
120

, 256256 matrix) with 40 transverse slices with a voxel resolution of 0.820.823.5
mm.
Six sets of echo-planar images (EPI), one set for each negotiation, were acquired
continuously with the following parameters: TR = 2,000ms, TE = 25ms,
ip angle =
90

, 6464 matrix, one shot per repetition, in-plane resolution 33 mm
2
, 41 transverse
slices, each 3mm thick, covering the whole brain. Total scan time for each participant
was approximately 45 minutes.
27
3.3 Predicting a Human Negotiator's Oer Types
3.3.1 Analysis
In my analyses, I aimed to nd the best predictor of participants' oer types based on
their brain activities. To achieve this goal, I labeled my data rst, and then performed
general linear model (GLM) analysis. Lastly, I used the GLM analysis results as input
to multi-voxel pattern analyses (MVPA) using two types of feature selection methods.
3.3.1.1 Data Labeling
All of the participants' oers were labeled with one of three categories based on their pay-
o changes from the previous oer: (1) positively-changed oer, (2) not-changed oer,
and (3) negatively-changed oer. Participants' oers on the rst rounds of each negotia-
tion were labeled `not-changed oer,' and their oers on all other rounds were compared
with their oer on the previous round based on payos and then labeled according to the
comparison. For example, if the participant's payo in the rst round was 20, his/her
payo in the second round was 23, and in the third round 21, the rst round would be
labeled as a `not-changed oer,' the second round as a `positively-changed oer' (because
his/her payo was increased by 3), and the third round as a `negatively-changed oer'
(because his/her payo was decreased by 2). Figure 3.1 shows this example in graph.
Here, `positively-changed oer' means that the participant concedes more to the other
player, `not-changed oer' means that the participant holds, i.e., he/she retains the oer
on the previous round, and `negatively-changed oer' means that the participant asks for
more. In my dataset, all participants had at least ve oers in each category.
28
Figure 3.1: Data labeling
Before analyzing the fMRI data, I had to exclude data from three participants. The
rst exclusion was due to incomplete fMRI scan data. In the second case the participant
made less than ve oers in each oer category, leading to a too small number of oers
in each oer category. In the third case I excluded the data because of the participant's
left-handedness, since more than 70% of left-handed people have dierent functional brain
structure compared to right-handed people such as for language processing (Warrington
and Pratt, 1973).
3.3.1.2 General Linear Model Analysis
To compare brain activity between the oer-making period and the non-oer-making pe-
riod, I ran a general linear model (GLM) analysis using tools from the FMRIB's Software
Library (FSL) (Smith, Jenkinson, Woolrich, Beckmann, Behrens, Johansen-Berg, Ban-
nister, De Luca, Drobnjak, Flitney, et al., 2004). Data pre-processing for GLM analysis
included following steps. First, all participants' fMRI data were motion-corrected using
FSL's MCFLIRT tool (Jenkinson, Bannister, Brady, and Smith, 2002) to x head motion
artifacts during scans. Then, non-brain such as a scalp was removed from the data using
29
FSL's Brain Extraction Tool (Smith, 2002). Next, spatial smoothing using a 5mm full
width at half maximum Gaussian kernel was applied to increase statistical power by im-
proving the signal to noise ratio. Also, slice timing correction for interleaved acquisitions
was used to compensate for timing dierence between slices of functional images. Finally,
high-pass temporal ltering was performed to let high frequencies (containing activities
relevant to decision-making) pass and to remove low frequencies such as signal drifts.
After completing data pre-processing, I modeled brain activity during oer-making
with a double gamma hemodynamic response function. Brain activity during all other
time points were considered baseline.
3.3.1.3 Multi-Voxel Pattern Analysis
To analyze brain patterns in decision-making during negotiations, I used multi-voxel
pattern analysis (MVPA) (Norman, Polyn, Detre, and Haxby, 2006), a machine learning
approach for investigating various patterns of brain voxels. Instead of analyzing each voxel
separately, MVPA takes multiple voxels into account together. This is useful because
activity in one voxel cannot be separated from neighboring voxels.
I used GLM analysis results as input to MVPA. As data pre-processing steps for
MVPA, I linearly de-trended the data to remove any bias resulting from scanner drift
over the acquisition time. Then, I converted the data to z-scores by scan to normalize
the range of each voxel.
Z =
mean signal dierence
standard deviation
30
MVPA was run using the PyMVPA (Hanke, Halchenko, Sederberg, Olivetti, Fr und,
Rieger, Herrmann, Haxby, Hanson, and Pollmann, 2009) software package version 2.3.1. I
used leave-one-participant-out cross validation for MVPA, in which a classier is trained
on six participants' data and then tested with the last participant's data. I repeated
this seven times, leaving each participant out once, then averaged the results to calculate
prediction accuracy.
A balancer was also included to keep the chance level the same (33%) throughout
my analyses because every participant has a dierent set of oers. For instance, if a
participant made ve `positively-changed oers,' eight `not-changed oers,' and seven
`negatively-changed oers' during a negotiation, the balancer randomly chose ve oers
from each category. Since a balancer chooses a new set of oers in each category whenever
it runs, I ran MVPA ve times and averaged the results. The prediction accuracy was
calculated as:
number of correctly classied oers
number of total oers
100(%)
Lastly, I applied feature selection methods for choosing the voxels used in my anal-
ysis. Feature selection is a common approach to reduce the number of features (voxels)
by selecting only relevant features as input to a classier. Classication performance
improves with feature selection because it picks features that vary signicantly between
categories (Guyon and Elissee, 2003). To validate my hypothesis that brain activities in
emotion-related regions are closely related to decision-making during negotiations, I used
two dierent feature selection methods for MVPA. First, I performed region of interest
(ROI) analysis using only voxels from within the anterior insula, which is known to be an
31
Figure 3.2: All four anterior insular regions; right ventral anterior insula (blue), right
dorsal anterior insula (red), left ventral anterior insula (yellow), and left dorsal anterior
insula (green).
emotion-related brain region. Second, I used searchlight as a feature selection method to
conrm that brain activities on the anterior insula are correlated with decision-making
during negotiations, and to nd out if other areas also show signicantly dierent brain
patterns between oer types. For both approaches, I used a linear Support Vector Ma-
chine (SVM) classier to perform classication. In the following sections, I explain each
feature selection method used for MVPA.
Region of Interest Analysis: Anterior Insula In the region of interest (ROI) anal-
ysis, I attempted to predict the oer categories based only on the voxels in the anterior
insula, which is known as an emotion-related brain region. The anterior insula on each
side of the brain can be divided into two subregions with distinct patterns of connectiv-
ity: dorsal anterior insula, connected with dorsal anterior cingulate cortex; and ventral
anterior insula, connected with pregenual anterior cingulate cortex (Deen, Pitskel, and
Pelphrey, 2011) (gure 3.2).
I trained the classier using voxels from each of the four regions separately with
feature selection. In my analyses, I used GLM analysis results to compute F-score per
32
each voxel, and then used an analysis of variance (ANOVA) measure to select the top
10% of features with the highest F-scores.
Each participant's brain was rst transformed into standard MNI space (Montreal
Neurological Institute (Evans, Collins, Mills, Brown, Kelly, and Peters, 1993)) to minimize
dierences from individual brains. After performing this process for all participants,
individual-level analyses were combined for a group-level analysis.
Searchlight as a Feature Selection Method In the searchlight analysis (Kriegesko-
rte, Goebel, and Bandettini, 2006), a map of classication accuracies is generated by
measuring the information in small spheres (radius = 5 voxels) centered on every voxel
in brain. As activities on one voxel are inevitably in
uenced by activities on neighboring
voxels, searchlight analysis is a preferable approach to capture local spatial areas that
show signicantly dierent activity patterns on each experimental condition. To conrm
the strong correlation between brain activities on emotion-related regions and negotiation
oer types, and to nd other brain regions that show notable pattern dierences on each
oer type, I used searchlight analysis as a feature selection method for MVPA.
Steps for using searchlight analysis as a feature selection method are as follows. I rst
generated searchlight maps from each participant's brain. Then I transformed individual
searchlight maps into the standard MNI space, and merged six of them to generate
seven merged searchlight maps in total. Next, I generated t-maps using t-tests across
six participants versus chance, and thresholded the top 5% of the t-maps and binarized
them. After that, I transformed all participants' functional images (EPIs) into individual
33
Table 3.1: Results of ROI MVPA with anterior insular regions.
Anterior Insular Region
Prediction
Accuracy
Chance
Level
Left
Ventral Anterior 35.23%
33.33%
Dorsal Anterior 43.88%
Right
Ventral Anterior 26.97%
Dorsal Anterior 28.60%
EPI space and ran MVPA using thresholded and binarized t-maps as masks for each
participant. Finally, I averaged MVPA results to calculate overall prediction accuracy.
3.3.2 Results
3.3.2.1 ROI MVPA: Left Dorsal Anterior Insula
I hypothesized that activity in the anterior insula is highly correlated with dierent
types of oers. I performed ROI MVPA with each of the four anterior insula regions
(left/right ventral anterior insula and left/right dorsal anterior insula), and found that
the left dorsal anterior insula to be the best predictor of oer types (positively-changed
oer, not-changed oer, or negatively-changed oer). The prediction accuracy of ROI
MVPA using voxels from the left dorsal anterior insula is 43.88%, with a standard error
1.30%. The results of ROI MVPA with all four anterior insular regions are shown on
table 3.1. Interestingly, anterior insular regions other than the left dorsal anterior insula
show chance level performance.
A binomial test shows that the performance of ROI MVPA with the left dorsal anterior
insula is signicantly higher than chance level (p = 0.0058), indicating that activity in
the left dorsal anterior insula predicts oer types in negotiations.
34
Figure 3.3: Combined t-maps that were used as masks for MVPA with searchlight as a
feature selection method. Each of seven t-maps were generated across six participants'
searchlight results, and then thresholded and binarized. Areas marked with red indicate
that the area is included for one t-map, and areas with yellow indicate the area is included
for all t-maps. Left dorsal anterior insula is included in six t-maps (green boxes).
3.3.2.2 MVPA with Searchlight as a Feature Selection Method
To map the spatial distribution of information, I performed MVPA with searchlight as
a feature selection method. Figure 3.3 shows the overlaid accuracy map for all seven
participants. Before overlaying accuracy maps, I thresholded the top 5% of seven t-maps
each across six participants and binarized them. Complementing the nding from the
ROI analysis, I nd that the left dorsal anterior insula is included in six t-maps.
Figure 3.4 shows MVPA with searchlight as a feature selection method results, ROI
MVPA with the left dorsal anterior insula results, and the chance level. The prediction
accuracy for MVPA with searchlight as a feature selection method is 47.45% with a
standard error 0.017%. Compared to ROI MVPA, MVPA with searchlight as a feature
selection method has the prediction accuracy improved by 3.57% and has much smaller
standard error. Two sample t-test results revealed that there is a signicant dierence in
prediction accuracy between MVPA with searchlight as a feature selection method and
ROI MVPA with the left dorsal anterior insula, t(23.124) = 1.6272, p = 0.058.
35
Figure 3.4: Comparison of ROI MVPA with left dorsal anterior insula and MVPA with
searchlight as a feature selection method results. Dashes denote the chance level (33.3%)
and error bars denote standard errors.
Results from MVPA with searchlight as a feature selection method also indicate that
there are other brain regions that show signicantly dierent patterns on each oer type.
For example, right posterior supramarginal gyrus, which plays a central role in controlling
one's empathy towards other people (Silani, Lamm, Ru, and Singer, 2013), was included
in all seven t-maps. Right hippocampus, which plays important roles in formation of
spatial memory (Smith and Milner, 1981), was also included in all seven t-maps.
3.3.3 Discussion
In this analysis, I investigated the neural correlates of decision-making in negotiations.
I classied negotiation oers into three categories: conceding, holding, or asking for
more. Then I analyzed brain imaging data by oer category to see if I could predict
the category of negotiation based on the brain activation. I used functional MRI to
capture participants' brain activity during negotiations, and used MVPA to analyze the
fMRI data. ROI MVPA and MVPA with searchlight as feature selection methods were
used and both methods resulted in signicantly better prediction accuracies than the
36
chance level of 33%. Most notably, MVPA with searchlight as a feature selection method
yielded the higher prediction accuracy of 47.45%, indicating the importance of analyzing
neighboring voxel clusters together.
My results reveal that there are distinct brain patterns across participants for each
type of oer. More specically, activity in the left dorsal anterior insula, which is a well-
known emotion-related brain region, was found to play a key role in distinguishing oer
types. This is in line with my hypothesis that activations in emotion-related brain regions
would be closely related to decision-making. Emotions provoked while the participant is
interacting with the computer agent during negotiations mediate the participant's nego-
tiation behaviors, and these processes are captured in fMRI data as a form of increased
blood
ow in emotion-related brain regions. Thus, this conrms not only the importance
of the role of emotion in human-agent interaction, but also the possibility of interpreting
the underlying processes during negotiations with fMRI data.
Furthermore, the results indicate that negotiation oers can be predicted based on
brain activities. Oer type prediction results from MVPAs with two types of feature
selection methods support the feasibility of successful predictions of negotiation behav-
iors. This has implications for human-agent negotiation research, allowing us to perform
more detailed predictions of negotiation behavior based on fMRI data as compared to
predictions based on behavioral research.
37
Figure 3.5: Agent's emotional expressions. Angry (left), neutral (middle), and sad (right).
3.4 Decoding an Agent's Emotion and Strategy
3.4.1 Agent's Features
In this study, I used six types of agents characterized by the three types of emotions they
expressed and the two types of oer sets representing their negotiation strategies. More
details about these features are described in the following two sections.
3.4.1.1 Agent's Emotional Expressions
The role of emotional displays in negotiation has been extensively documented (Lerner,
Li, Valdesolo, and Kassam, 2015). To nd the neural mechanisms involved in processing
dierent emotional displays in human-agent interactions, I used three types of facial
expressions to express agents' emotions; angry, neutral (no emotion) and sad. Figure 3.5
shows agents' emotional expressions that were used in the experiment. In angry and
sad conditions, the virtual agent's face starts as neutral and changes to the emotional
expression for ve seconds on the rst, third, and fth rounds of negotiation. In the
neutral condition, the agent's face starts as neutral and does not change.
38
Table 3.2: Payo matrix for each negotiation party in negotiation information study
Item 1 Item 2 Item 3
Computer agent 2 3 4
Participant 3 4 2
Table 3.3: The negotiation strategy of the non-conceding agent and the conceding agent
Round 1 Round 2 Round 3 Round 4 Round 5 Round 6
Non-conceding [0, 0, 0] [0, 0, 1] [1, 0, 0] [2, 0, 0] [0, 2, 0] [1, 1, 1]
Conceding [2, 0, 1] [2, 0, 2] [2, 1, 1] [0, 2, 2] [2, 2, 0] [2, 1, 2]
3.4.1.2 Agent's Negotiation Strategies
I used two sets of pre-programmed agent oer strategies: Non-conceder and Conceder.
In the non-conceder strategy, the agent starts with no concession and continues with
gradually increased concession. In the conceder strategy, the agent starts with some
concessions and keeps conceding further in the next rounds. During the experiment, we
wanted participants not to notice that they were repeatedly negotiating with the agent
that uses the same strategy. Therefore, we assigned dierent payo values on each item for
each negotiation party. For example, an apple has a value of 2 for a computer agent and
3 for a participant. This payo combination is maintained through all six negotiations,
in other words, the item that has a value of 2 for a computer agent always has a value of
3 for a participant.
Table 3.2 shows the payo matrix for each negotiation party and table 3.3 shows
the computer agent's oers. And gure 3.6 shows sum of payos for the agent and the
participant when the agent uses non-conceding or conceding strategies.
When the virtual agent decides whether to accept or reject a participant's oer, the
agent calculates the summed payos and compares it with its next oer. If the summed
39
Figure 3.6: Payos for agents and participants across both agent strategies.
payos are larger than the summed payos of the next oer, the agent accepts the oer.
Otherwise, the agent rejects the oer and proposes a new oer.
The same payo matrix was used across all six negotiation tasks so that I could
control for the potential eect of varying payo values. However, I randomized the order
of items shown on the screen in each task to give participants the impression that they
were playing a new negotiation task every time.
3.4.2 Analysis
I hypothesized that the agent's negotiation strategy can be predicted based on the par-
ticipant's brain activity in the frontal pole, and the agent's emotional expression can be
predicted based on the participant's brain activity in the anterior insula. To test my
hypothesis, I rst performed GLM analysis which is described in chapter 3.3.1, and used
the GLM analysis results as input to ROI MVPA. The anterior insula and the frontal pole
was chosen as my regions of interest (ROIs). The analysis was done after excluding data
40
Figure 3.7: Frontal pole (green)
from one participant because of incomplete fMRI scan data. In the following section, I
explain the ROI MVPA approach.
3.4.2.1 Region of Interest Multi-Voxel Pattern Analysis
To nd the relationship between an agent's expressed emotion and brain activity as well
as an agent's negotiation strategy and brain activity, I performed region of interest (ROI)
MVPA analyses with both the anterior insula and the frontal pole.
As the anterior insula can be divided into four subregions with distinct patterns of
connectivity (Deen, Pitskel, and Pelphrey, 2011), I ran ROI MVPA analyses for all four
anterior insular regions separately (left/right ventral anterior insula and left/right dorsal
anterior insula, gure 3.2). The frontal pole does not have widely accepted subregions
on the contrary (Moayedi, Salomons, Dunlop, Downar, and Davis, 2014), so I ran ROI
analysis for the whole frontal pole labeled by the Harvard Center for Morphometric
Analysis (Desikan, S egonne, Fischl, Quinn, Dickerson, Blacker, Buckner, Dale, Maguire,
Hyman, et al., 2006) (Figure 3.7).
To make sure brain activity in the anterior insula or the frontal pole is responsible
either for agent's emotional expressions or negotiation strategies, I ran ROI analyses for
41
both conditions, i.e., I calculated the prediction accuracy of agent's negotiation strategies
using both the anterior insula and the frontal pole as ROIs. I assumed that the prediction
accuracy with their expected ROI would be signicantly higher than the chance level,
but the prediction accuracy with their unexpected ROI would be indistinguishable from
chance.
I trained a linear Support Vector Machine (SVM) classier using voxels from each
of my ROIs separately using feature selection. More details on the ROI MVPA analysis
can be found in section 3.3.2.1. I would like to note that searchlight was not used as
a feature selection method for decoding an agent's features. This is because each EPI
had its own set of agent features, e.g., one EPI was acquired in the angry non-conceder
condition and another EPI was acquired in the sad conceder condition. As participants'
posture slightly changes whenever a new EPI is acquired, it is hard to trust results from
a within-participant analysis when the experimental conditions are divided by EPIs. As
a within-subject analysis is required to use searchlight as a feature selection method, I
did not use searchlight as a feature selection method. Instead, I focused on ROI MVPA
analysis.
3.4.3 Results
As discussed previously, the anterior insula is a brain region known to respond to emo-
tional expressions, and it can be divided into two subregions with distinct patterns of
connectivity. Therefore, I rst ran ROI MVPA analyses for all four anterior insular re-
gions (left/right ventral anterior insula and left/right dorsal anterior insula) separately.
42
Figure 3.8: (A) Accuracy rate of an agent's emotion prediction. (B) Accuracy rate of an
agent's strategy prediction. In both gures, results from ROI MVPA with the left dorsal
anterior insula and ROI MVPA with the frontal pole are included. Dashes denote the
chance level and error bars denote the standard errors.
The prediction accuracies of emotional expression using my ROI MVPA with four ante-
rior insular regions indicate that ROI MVPA with the left dorsal anterior insula has the
best prediction accuracy (38.40%), while the prediction accuracy of other anterior insular
regions are indistinguishable from the chance level of 33%. A binomial test revealed that
the prediction accuracy of the ROI MVPA with the left dorsal anterior insula is signi-
cantly above chance (p = 0.0566). Therefore, I focus my analysis on the results of ROI
MVPA with the left dorsal anterior insula.
43
Figure 3.8A displays the prediction accuracy for decoding an agent's emotional ex-
pressions in the left dorsal anterior insula and the frontal pole. The accuracy is higher
for the left dorsal anterior insula decoder (38.4%, left bar on gure 3.8A), compared to
33.87% for the frontal pole (p = 0.0960) or chance (p = 0.0566). The prediction accuracy
using the frontal pole ROI MVPA was not dierent from chance (p = 0.4266). This
supports my hypothesis that an agent's emotional expression can be reliably predicted
using brain activity in the anterior insula which is an emotion-related brain region, but
not with information in the frontal pole.
Similarly, gure 3.8B displays the accuracies for predicting an agent's negotiation
strategy using the ROI MVPA with the left dorsal anterior insula and with the frontal
pole. This prediction accuracy in the frontal pole (right bar on gure 3.8B) was 58.96%.
A binomial test conrmed that this performance is signicantly higher than the chance
level of 50% (p = 0.0085). The prediction performance was almost at chance (52.71%)
for the left dorsal anterior insula (p = 0.2581). This result validates my hypothesis that
counterpart's negotiation strategy can be predicted based on brain activity in the frontal
pole, which is activated when people do active decision-making, but not with the insula,
which is involved in emotion processing.
In order to examine these results in more detail, I broke down the predictions. Speci-
cally, I analyzed the prediction accuracy for each agent's emotional expression and negoti-
ation strategy from ROI MVPAs with anterior insular regions and frontal pole (Table 3.4).
My results indicate that brain patterns in the left dorsal anterior insula can predict an-
gry and sad conditions but not neutral agent facial expressions. Also, brain patterns in
44
the frontal pole can predict the non-conceder negotiation strategy but not the conceder
strategy.
Overall, my results conrm that negotiating with dierent types of agents results in
activity in dierent brain regions, and these activity patterns can be used to further
decode the specic type of interaction agent.
Table 3.4: Prediction accuracy on an agent's emotional expression and an agent's strategy
from ROI MVPA. One-tail binomial tests were performed for each condition compared
to the chance level.
Region of Interest (ROI) Condition
Prediction
Accuracy
Binomial Test
Result
Chance
Level
Left Dorsal Anterior Insula
All Emotions 38.40% p = 0.0566
33.33%
Angry 45.07% p = 0.0001
Neutral 29.60% p = 0.8935
Sad 40.53% p = 0.0115
Frontal Pole
All Strategies 58.96% p = 0.0085
50.00% Conceder 47.50% p = 0.7863
Non-conceder 70.42% p = 0.0001
3.4.4 Discussion
To answer the question of how an agent's features interact with underlying neural mech-
anisms, I investigated brain activity during human-agent interactions. More specically,
participants engaged with virtual agents who showed three dierent emotional expressions
(angry, neutral and sad) and used two dierent types of negotiation strategies (conceding
and non-conceding).
I hypothesized that an agent's emotional expression could be predicted based on
patterns in emotion-related brain regions, and an agent's negotiation strategy could be
predicted based on patterns in decision-making-related brain regions. Therefore, I focused
my analyses on the anterior insula and the frontal pole, as previous studies have shown
45
that anterior insula is activated when people engage in emotional tasks, and the frontal
pole is activated when people perform active decision-making tasks.
My ROI MVPA results support my hypothesis; prediction accuracy of an agent's
emotional expression based on brain patterns in the left dorsal anterior insula, and that
of agent's negotiation strategy based on brain patterns in the frontal pole are well above
the chance level. These results indicate that dierent features are likely processed in
dierent brain regions. Finding which information is processed in certain brain regions
would allow us to reliably decode the feature of the agent from users' brain activity.
More detailed analyses revealed that brain patterns in the left dorsal anterior insula
could be used to predict angry (45.07%) and sad (40.53%) conditions, but not the neutral
condition (29.60%). This indicates that there are clear dierences in brain patterns in
the left dorsal anterior insula between angry and sad conditions. I hypothesize that the
reason why the patterns in this region failed to predict the neutral condition is that the
neutral facial expression is the default expression throughout the experiment. The facial
expression of the agent only changes when it morphs into sad or angry. I plan to tackle
this problem by only showing the face of the agent during the decision making phase.
With regard to agent's negotiation strategies, predictions using brain patterns from
the frontal pole showed signicantly higher accuracy compared to the chance level (50%)
for the non-conceding condition (70.42%), but not for the conceding condition (47.50%).
I assume that this is because participants expected to deal with a counterpart that acted
like a conceding and fair agent, i.e. an agent who might start with a slightly unfair oer
but over time it makes adjustments toward a fair oer. It is possible that the distinct
patterns in the frontal pole witnessed during negotiations with the non-conceding agent
46
is because this agent acts in a very greedy and tough way that is not typical in social
interactions. This could result in unique patterns of activity in the frontal pole.
While my sample size could be considered small, I would like to note that sample
size tends to be small in fMRI studies. Also, it is worth mentioning that the probability
of nding the same eect as one found in the original experiment is not dependent on
sample size, but dependent on p value (Killeen, 2005). This is because large eect sizes
produce signicant results, even with small sample size.
3.5 General Discussion
In this study, I explored the relationship between the information of negotiations and
brain patterns from specic brain regions. Using a human-agent negotiation platform,
participants interacted with computer agents in an fMRI scanner, and their brain activity
during the interaction was recorded. Then their brain patterns in each negotiation phase
were analyzed using MVPA analysis.
First, my results suggest that it is possible to predict whether the human negotiator
concedes, does not change, or asks for more during the human-agent negotiation using
brain patterns from emotion-related brain region. Most importantly, I demonstrated
that the left dorsal anterior insula, which is known to be an emotion-related brain region,
shows a dierent pattern of activity for each of the three oer types and therefore can
be used to predict the oer type. This implies that emotion plays a key role in people's
decision during negotiations.
47
Second, my results indicate that dierent brain patterns are observed for various types
of virtual agents; consequently, we can decode the strategy and emotional display of the
agent based on the counterpart's brain activity. Using fMRI data, I analyzed partici-
pants' brain activity during negotiations with agents who show three dierent emotional
expressions and use two dierent types of negotiation strategies. I demonstrated that,
using MVPA, we can reliably decode agents' emotional expressions based on the activity
in the left dorsal anterior insula, and also agents' strategies based on the activity in the
frontal pole.
In conclusion, my results show that there are links between various negotiation in-
formation and brain activity in specic brain regions. Further, brain patterns in these
regions can be used to decode negotiation information, such as human negotiator's of-
fer types or agent's emotional expressions. Even though the results are preliminary, my
work sheds light on the links between certain brain regions and dierent negotiation
information.
48
Chapter 4
Decoding Partner Type in Human-Agent Negotiations
4.1 Introduction
Whether computer agents can act as substitute for human beings during the course of an
interaction or not has been a popular topic in sci- movies for decades. Even though some
computer agents including ones that were thought to exist only in movies a few decades
ago are now widely used in daily life, human-agent interactions are often quite dierent
from human-human interactions (Gray, Gray, and Wegner, 2007; de Melo, Marsella, and
Gratch, 2016). These dierences have been extensively studied both behaviorally and
neurologically to understand why they occur and to gure out how we can ll these gap.
Using the Object Negotiation Task, I explored whether a computer agent introduced
as another human is perceived more anthropomorphically than one that is introduced
as a computer program. I then investigated whether agent type produced behavioral
dierences, and whether one type of agent resulted in more concessions compared to
the other. In a follow up experiment, I compared brain activity during interactions
with human-labeled and computer-labeled agents to determine whether these perceptual
49
dierences were also observable in brain patterns. Following collection of fMRI data, I
investigated whether classiers could be trained to determine whether the participant
was playing against a human-labeled or computer-labeled agent.
I hypothesized that participant behavior and brain activity would be dierent during
interactions with human-labeled agents, compared to interactions with computer-labeled
agents, even though both agents used exactly the same strategies and emotions. My
initial experiment consisted of an online negotiation task intended to explore perceptual
and behavioral dierences pertaining to anthropomorphic characteristics in human and
computer-labeled agents. Next, I adapted the negotiation framework into an fMRI ex-
periment, attempting to nd neural dierences for the two distinct partner conditions.
In addition to these studies, I also ran a prediction algorithm and multi-voxel pattern
analysis (Norman, Polyn, Detre, and Haxby, 2006) based on the fMRI data.
This work is distinct from previous studies due to the use of an identical computer
agent, regardless of what partner type was specied. The majority of previous studies
employed computer-animated characters or robots that had diering levels of anthropo-
morphism. Even though Burnham, McCabe, and Smith (2000) ran a similar experiment
where a counterpart is labeled either a `partner' or an `opponent', their studies were
done for one-shot human-human interactions and each participant interacted with a dif-
ferent counterpart. In this work, I demonstrate that even though the same computer
agent is used, perceptual dierences during human-agent interactions were captured in
behavioral and brain data. I believe that natural interactions take place over multi-
ple rounds/sessions, and it is therefore important to investigate perception dierences
through multi-round negotiations.
50
4.2 Decoding Partner Type from Behavioral Data
4.2.1 Online Experiment
I designed an online experiment to determine whether people perceive human-labeled
agent dierently from computer-labeled agents during negotiations, as well as to nd
behavioral dierences in terms of concession-making between interactions with human-
labeled agents and ones with computer-labeled agents.
4.2.1.1 Participants
420 subjects (237 male and 183 female; mean age = 33.5) living in the United States
were recruited via Amazon Mechanical Turk (MTurk). I set the following qualication
requirement in the MTurk; to avoid novice MTurk workers, I limited participants to
ones having greater than or equal to 100 approved HIT. To avoid excessively professional
survey-takers, I also limited participants to ones having less than 10,000 HIT.
4.2.1.2 Modied Objects Negotiation Task
I used the modied version of the Objects Negotiation Task (Dehghani, Carnevale, and
Gratch, 2014) in this study. Figure 4.1 shows the interface of the modied version of this
task. In addition to changes that were made for my previous fMRI study (introduced
in section 2.3), I added a partner introduction phase and an emotion-reporting phase.
The partner introduction phase allows participants to receive a notication specifying
whether their partner type was another participant or the computer program before the
negotiation began. Instead of showing emotional expressions on the trading board, the
51
Figure 4.1: Modied version of the Objects Negotiation Task interface
emotion-reporting phase was added to allow participants focus solely on the negotiation
partner's emotional expressions.
The modied Objects Negotiation Task used in this behavioral experiment is com-
posed of nine phases. Figure 4.2 shows an example of the timeline of the modied version
of this task after the partner introduction phase.
1. The participant receive a notication about their partner type.
2. The participant proposes an oer.
3. The participant conrms computer agent's emotional expression about the oer.
4. The participant waits for the computer agent to accept or reject the oer.
5. If accepted, negotiation ends. If rejected, the participant waits for the computer
agent to propose an oer.
6. The participant reviews the computer agent's oer for ve seconds.
7. The participant reports his/her emotion about the received oer.
52
Figure 4.2: Modied version of the Objects Negotiation Task timeline
8. The participant decides whether to accept or reject the computer agent's oer.
9. If accepted, negotiation ends. If rejected, the participant waits for ve seconds and
then is redirected to the second phase.
When the task begins, the negotiation partner type is displayed. Specically, in the
human-labeled agent condition, the message shown to the participant is `In this task, you
will be negotiating with the other participant.' In a computer-labeled agent condition, the
same message is shown but `the other participant' is changed to `a computer program'.
Next, a `connection establishment' message for the human condition and a `program setup'
message for the computer condition appear on screen, to persuade participants of their
partner setting. Throughout the negotiation, the partner type is constantly included on
screen so the participant clearly recognizes his/her partner type. The partner is labeled
as `the other participant' or `the computer program.'
In the rst negotiation round, items are positioned in the middle row, indicating that
those items belong to neither player. The participant is asked to propose an initial oer
53
Figure 4.3: Available emotions: happy, content, neutral, angry and sad.
by moving items into his/her own set of boxes (bottom row) or their partner's set of
boxes (top row). Once the initial oer is made, the partner (agent) chooses an emotion
pertaining to the oer which is then displayed to the participant. Available emotions
include: happy, content, neutral, angry and sad (gure 4.3). The partner only shows the
predened emotion for each round. After the emotion is displayed, the partner decides
whether to accept or reject the oer. This decision is based on a predened oer value;
when the payos of the predened oer are more than the participant's current oer, the
partner rejects, when the payo is less, the partner accepts.
If the participant's oer is accepted, the items are distributed as proposed and the
participant is notied. If the participant's oer is rejected, the partner then proposes a
counteroer. When the counteroer is received, the participant has 5-seconds for review.
During this time, the participant can only observe; no items can be transferred. The
review time was specically introduced for optimal brain activation, as I wanted to record
an active decision-making process. For the same reason, my analysis was focused on data
collected during this review phase. After the review phase, the participant reports his/her
emotion about the proposed oer by choosing from the following descriptive options:
54
Table 4.1: Payo matrix for each negotiation party in negotiation partner type study
Item 1 Item 2 Item 3
Computer agent 3 4 5
Participant 4 5 3
happy, content, neutral, angry, or sad. The participant also decides whether to accept or
reject the oer. If the participant rejects the oer, a new round begins, and all phases
are repeated. The negotiation can last for a maximum of six rounds. If no agreement is
made in six rounds, neither party receives anything.
4.2.1.3 Negotiation Partners
Two sets of strategies and two types of emotions were used for the agents. Agent strategies
included tough and soft. A tough strategy starts with a greedy oer, and a soft strategy
starts with a relatively generous oer. A payo matrix for a computer agent was dierent
from one for a participant so that the participant cannot easily notify his/her negotiation
partner uses the same strategy over and over again. Table 4.1 shows a payo matrix for
each negotiation party.
Table 4.2 shows oers of the computer agent that uses a tough strategy or a soft
strategy. Numbers in brackets in table 4.2 indicate how many items are oered by an
agent. For example, agent that uses a tough strategy makes an oer of [1, 1, 0] in round
1, which means that it oers one of item 1, one of item 2, and none of item 3 to a
participant. It also means that the agent oers to keep two of item 1, two of item 2, and
three of item 3 for itself because three of each item are available initially. Hence, total
payo of the tough agent's oer in the rst round is 9 for a participant and 29 for the
55
Table 4.2: The negotiation strategy of the tough agent and the soft agent in negotiation
partner type study
Round 1 Round 2 Round 3 Round 4 Round 5 Round 6
Tough strategy [1, 1, 0] [0, 2, 0] [3, 0, 0] [1, 2, 0] [3, 0, 1] [1, 2, 1]
Soft strategy [2, 1, 0] [0, 3, 0] [2, 2, 0] [1, 3, 0] [3, 1, 1] [2, 2, 1]
Figure 4.4: Payos for agents and participants across both agent strategies.
agent. Figure 4.4 shows total payos for the agent and the participant when the agent
uses tough or soft strategies.
Agent emotions included anger and neutral (no emotion). Anger was chosen because
it was found to be the most eective emotion in yielding concessions during negotiation
tasks (Van Kleef, De Dreu, and Manstead, 2004a). For the anger condition, the agent
displayed an angry face in rounds 2, 4, and 6, and a neutral face in rounds 1, 3, and 5.
For the neutral condition, the agent reported a neutral face in every round.
4.2.1.4 Procedure
Each participant was asked to read a hypothetical scenario in which they acted as a
restaurant owner, and negotiated for fruit with another restaurant representative due
to a fruit shortage as a result of a recent re in a local market. Each subject was
56
then told to negotiate with either a computer program or another (hypothetical) MTurk
player. Regardless of type label, the negotiation partner was always a pre-programmed
computer agent. After completing all negotiations, subjects were asked to ll out an
anthropomorphism questionnaire (Bartneck, Kuli c, Croft, and Zoghbi, 2009) about their
partner, as well as a demographic questionnaire. In the anthropomorphism questionnaire,
participants rated their impression of their partner using a scale from 1 to 7, where 7
means human-like and 1 means machine-like. Subjects were also given a simple attention-
check question, implemented to make sure the participants were paying attention; it
merely asked what type of partner they were assigned during the task. Each participant
was compensated $1.
4.2.2 Analysis
I excluded subjects who had participated in my previous negotiation studies or failed
to give the correct answer to the attention-check question. After exclusion, I had data
from 329 subjects. Scores from each condition were calculated for the anthropomorphism
questionnaire to verify whether participants perceived human-likeness dierently between
agents. In addition, I calculated concessions across partner type in each condition to
analyze behavioral dierences. Concession was calculated by subtracting payos of agreed
oers from payos of initial oers. A three-way between-subjects analysis of variance
(ANOVA) was used to nd the interaction between partner type, partner strategy, and
partner emotion during concession.
57
Figure 4.5: Anthropomorphism Scores for human-labeled agents and computer-labeled
agents. Higher score means the agent is perceived as more human-like. The error bar
shows standard errors.
4.2.3 Results
The anthropomorphism scores of the human-labeled agents and the computer-labeled
agents are shown in Figure 4.5. One-way ANOVA results show that people consistently
thought their partner to be more human-like when told their partner was a human player,
no matter what the negotiation strategy or what the expressed emotion (F(1, 321) =
16.537, p < 0.001). The results also show that people thought their partner to be more
human-like when their partner express emotion (F(1, 321) = 28.950, p < 0.001), and
when their partner uses a soft strategy (F(1, 321) = 21.541, p < 0.001). However, no
interactions between agent type, agent emotion, and agent strategy were found for the
anthropomorphism scores.
Concessions to human-labeled agents and computer-labeled agents during negotiations
are shown in gure 4.6. ANOVA results show that there is a 2 2 2 interaction
between agent type (human/computer) agent emotion (angry/neutral) agent strategy
(tough/soft) for concession (F(1, 321) = 3.387, p = 0.066). I also ran a 2 2 ANOVA
58
Figure 4.6: Concessions to human-labeled agents and computer-labeled agents. The error
bars show standard errors.
after dropping each strategy. A two-way interaction was found for tough strategy (F(1,
164) = 4.699, p = 0.031).
4.2.4 Discussion
My ndings from anthropomorphism scores suggest that there are perception dierences
in the interactions between human-labeled agents and computer-labeled agents. Also, the
concession results suggest that there is an interaction between agent type agent emotion
agent strategy for concession. This indicates that not only are people's perceptions of
the two agents distinct, but their behaviors also vary depending on agent type. To study
whether these behavioral dierences have neural correlates, I designed the following fMRI
experiment. Because the largest concession dierences were found in the tough conditions,
implying the tough strategy was best suited to observe those dierences in behavior, I
mainly employed tough agents in the following experiment.
59
4.3 Decoding Partner Type from Brain Data
4.3.1 fMRI Experiment
I hypothesized that perceptual and behavioral dierences could be captured in brain
activity, especially in ToM related brain regions, as they were found to be correlated
with human-likeness of physically existing human-like robots (Chaminade, Hodgins, and
Kawato, 2007; Krach, Hegel, Wrede, Sagerer, Binkofski, and Kircher, 2008). Each subject
performed the negotiation task with both types of agent in order to compare brain ac-
tivity from human-labeled vs. computer-labeled agent interaction. To make interactions
with human-labeled agents more realistic, I introduced a confederate into the study, so
participants believed they would be competing against another human player.
4.3.1.1 Participants
Twenty healthy American subjects (ten male and ten female), recruited via the University
of Southern California online bulletin board, took part in this study. Subjects were 21.4
years old on average (SD = 2.58). All participants were right-handed and had no history
of neurological or psychiatric disorders.
4.3.1.2 Modied Objects Negotiation Task
The modied version of the Objects Negotiation Task introduced in section 4.2.1.2 was
used again for this fMRI study. However, I have shortened wait times between each phase
from 810 seconds to 35 seconds, so that participants can nish six negotiations in one
hour.
60
4.3.1.3 Negotiation Partners
Although I were only interested in tough agent type, I used two types of soft agents on top
of four types of tough agents (human/computer-labeled angry/neutral agents). This
is modication was implemented because subjects participated in a series of consecutive
negotiations, unlike my online experiment where each subject only played in a single
negotiation. Including soft agents ensured that participants did not play with the same
agent over and over again. Each subject negotiated with six types of agents. While
every subject negotiated with four types of tough agents, ten subjects (ve male and ve
female) negotiated with two types of emotion-neutral soft agents, while the remaining
ten subjects negotiated with two types of emotion-angry soft agents. Agent order was
randomized.
4.3.1.4 Procedure
Each participant was greeted by an experimenter and introduced to the confederate as
the competing player. The participant and the confederate were guided to a preparation
room where they lled out an informed consent form, incidental ndings form, and safety
screening form. After forms were completed, the confederate was guided to a separate
MRI room for \setting up". The participant was given the instructions and rules regarding
the negotiation task, and played a trial negotiation against a computer program before
starting the experiment. During the trial, a trackball mouse similar to one used in the
scanner environment was provided, so that the participant became familiarized with it's
operation.
61
The participant was then guided to the actual MRI room and was told that while in
the scanner he/she would run through three negotiation tasks with the participant in the
other MRI room, and three negotiation tasks with the computer program. The task was
back projected on a screen, seen through a mirror attached to the head coil, and operated
a trackball mouse to navigate negotiations. In each task, a dierent set of negotiation
items were used, and payos for these items varied with position, in order to give an
impression that each negotiation was unique.
Participants answered a shortened version of the anthropomorphism questionnaire at
the completion of each round. After a maximum of six negotiation rounds, participants
lled out a handedness and demographic questionnaire. Before leaving, subjects were
debriefed and compensated $30.
4.3.1.5 fMRI Data Acquisition
fMRI scans were again performed at the USC Dana & David Dornsife Cognitive Neuro-
science Imaging center. I acquired two sets of high-resolution anatomical images and six
sets of echo-planar images (EPI) using the same settings described in section 3.2.3. Total
scan time for each participant was approximately 50 minutes.
4.3.2 Analysis
I conducted a general linear model (GLM) analysis and then used the results as input for
multi-voxel pattern analysis (MVPA). GLM analysis was performed using FMRIB's Soft-
ware Library (FSL) (Smith, Jenkinson, Woolrich, Beckmann, Behrens, Johansen-Berg,
Bannister, De Luca, Drobnjak, Flitney, et al., 2004) to locate brain regions activated
62
during proposal review phases. As mentioned earlier, this phase was specically targeted
due to the likelihood of collecting data pertaining to decision making for subsequent nego-
tiation rounds. For GLM analysis, data pre-processing steps included motion correction,
brain extraction, spatial smoothing, slice timing correction, and high-pass temporal l-
tering. After completing data pre-processing, I modeled brain activity during proposal
review phases using a double gamma hemodynamic response function. Data collection
from all other time points were used as baseline. I then performed MVPA to nd brain
regions that illustrated dierent patterns across agent type. In MVPA, neural representa-
tions were decoded by applying pattern-classication algorithms on fMRI data (Norman,
Polyn, Detre, and Haxby, 2006). I used detrended and z-scored GLM analysis results as
inputs for MVPA, and trained a linear Support Vector Machine (SVM) classier using
feature selection, introduced to improve classication performance by picking the most
relevant features as inputs for the classier (Guyon and Elissee, 2003). Searchlight
analysis (Kriegeskorte, Goebel, and Bandettini, 2006) was used as the feature selection
method to analyze contents multivariately at each location in the brain. I implemented
a leave-one-participant-out cross validation as balance for MVPA. More details on fMRI
data analysis can be found in section 3.3.1.2 and 3.3.1.3, as the same analysis methods
were used.
4.3.3 Results
The right side of gure 4.7 shows an overlaid accuracy map for all participants. Accuracy
maps were acquired from the top 5% of all t-maps, which were generated using t-tests
versus chance. Interestingly, medial prefrontal cortex, one of the ToM brain regions, was
63
Figure 4.7: Frontal medial cortex from Harvard-Oxford atlas (left) and overlaid accuracy
map for all participants (right). A part of frontal medial cortex was included in the
accuracy map (white dotted box).
included in the accuracy map, suggesting that people indeed perceived the human-labeled
agent to have more human-like qualities.
MVPA, with searchlight as a feature selection method, revealed that agent type (hu-
man/computer) can be predicted based on brain activity during proposal-review phases.
Prediction accuracy for agent type was 58.41%, with a standard error 0.01%, where chance
level is 50%. The improvement was found to have statistical signicance (Two-tailed t-
test: p < 0.001).
4.3.4 Discussion
The results of fMRI experiment demonstrates that dierences in how we perceive the
`humanness' of an agent can be captured using fMRI. Specically, my results show that
parts of the ToM neural correlates are activated in human-labeled agent conditions, but
not in computer-labeled agent conditions. This nding is consistent with previous studies
that reported increased brain activity in ToM brain regions corresponding to human-
likeness of interacting partners (Chaminade, Hodgins, and Kawato, 2007; Krach, Hegel,
Wrede, Sagerer, Binkofski, and Kircher, 2008). My MVPA analysis further revealed
64
that these dierences are great enough that classiers can be trained that can reliably
distinguish brain activity between the two types of agents.
4.4 General Discussion
My goal was to investigate dierences in behavior and brain activity during human-agent
negotiations. Focusing on partner type, I hypothesized that both parameters would be
distinct when comparing computer-labeled and human-labeled interactions.
Results from my online experiment indicate that people perceive human-labeled agents
more human-like than computer-labeled ones, even though both used parallel strategies
and emotions. This suggests that people's attitudes towards computer partners are dis-
tinguishable from those towards human partners. Furthermore, a three-way interaction
between agent type agent emotion agent strategy with main eect of agent strategy
was found for concession indicating that there is a two-way interaction between agent type
agent emotion that varies according to agent strategy. Interestingly, a two-way inter-
action between agent type agent emotion was also found which implies people tend to
concede more to a computer-labeled agent than to a human-labeled agent when the agent
express anger, but concede more to a human-labeled agent than to a computer-labeled
agent when the agent express neutral for the agent that uses a tough strategy.
Results from my fMRI experiment suggest that brain patterns observed during inter-
actions with human-labeled agents are dierent from ones with computer-labeled agents.
More specically, the medial prefrontal cortex, part of the ToM-related neural structures,
65
was found to be included in accuracy maps, indicating neural activity during interac-
tions with human-labeled agents are distinct from ones with computer-labeled agents.
This is in line with a previous nding, where the medial prefrontal cortex was found to
be activated while playing rock-paper-scissors with a human player, but not activated
when playing the same game with a known, pre-dened computer algorithm (Gallagher,
Jack, Roepstor, and Frith, 2002). Using a negotiation paradigm, a more complicated
task than rock-paper-scissors due to the inclusion of multiple decision-making rounds, I
found that the same eect exists with dierently labeled agents. Negotiations require a
more substantial cognitive eort than a game like rock-paper-scissors; there are a larger
amount of possible actions to consider, an increased opportunities for loss, and a greater
obligation to compete, or cooperate and come to some sort of agreement. The negotia-
tion tasks used in the aforementioned experiments are able to advantageously measure
interactions that require high levels of cognitive energy, and are therefore more useful
when attempting to explore human-robot interactions.
My ndings are also consistent with previous studies that found ToM-related neural
structures to be more activated when interacting with agents that had more human-like
characteristics, regardless of whether those agents happened to be robots or computer-
generated characters (Chaminade, Hodgins, and Kawato, 2007; Krach, Hegel, Wrede,
Sagerer, Binkofski, and Kircher, 2008). These ndings imply perceptual variance between
interactions with human-like and nonhuman-like agents, and leads us to believe that
human participants do engage in greater mentalising eorts when faced with human-
labeled or human-like robots.
66
In conclusion, I examined the relationship between negotiation partner type and be-
havioral and neural measures regarding individual's perceptions of human-agent inter-
actions. My study suggests that when either by labeling agents as other humans or as
computer programs signicantly impacts one's perception of the situation; this is ulti-
mately demonstrated through negotiation behavior and brain activity. The results give
us further insight into the counter-play between emotional and cognitive processes, lead-
ing us to believe that our emotions may have greater impact on decision making than
which we are consciously aware. Ultimately, these results inform us that there is a no-
ticeable and consistent dierence between the perceptual and emotional reactions that
humans have towards other humans when compared to those same reactions with com-
puter agents. Further research needs to be executed to more thoroughly understand why
these dierences in interaction occur, but this study has illustrated that computers and
technology do indeed impact the way humans interact with the world, and each other.
This is important to consider as computers continue to be increasingly implemented in
everyday life and society.
67
Chapter 5
Effects of Moral Concerns in Human-Agent
Negotiations
5.1 Introduction
The previous sections focused on brain patterns during general multi-round human-agent
negotiations. I used common fruits as negotiation items in these studies so that the
negotiation can be as simple as possible. However, our real-life negotiations often involve
sophisticated issues and sometimes people's negotiation behaviors are completely changed
due to the values assigned to these issues. Sacred values are a good example; several
researchers have shown that oering material incentives to compromise backres when
people treat contested issues as sacred values (Ginges, Atran, Medin, and Shikaki, 2007).
In this chapter, I introduce sacred values in human-agent negotiation and explore the
interplay between people's moral concerns, emotional expressions and concession-making.
With the growing interest in understanding the role of emotional expressions in ne-
gotiation (Barry, Fulmer, and Goates, 2006; Van Kleef, De Dreu, and Manstead, 2010),
many studies have investigated how emotional expression aects negotiation processes
68
and outcomes (Ames and Johar, 2009; Choi, Melo, Woo, and Gratch, 2012; de Melo,
Carnevale, Read, and Gratch, 2014). For instance, the past works on emotional expres-
sion suggest that emotion plays an important role as a signal (Allred, 1999) and several
researchers have shown that negotiators concede more when their opponent expresses
anger instead of happiness (Van Kleef, De Dreu, Manstead, et al., 2004b; Van Kleef,
De Dreu, and Manstead, 2004a). Thus, it is important to understand what moderates a
negotiator's reaction to emotional expression.
Although some studies have tried to understand how people's personality interacts
with their emotion during negotiation games (Bolton, Katok, and Zwick, 1998; Bat-
son and Moran, 1999), little research has paid attention to how moral concerns im-
pact reactions to emotional expressions and aect concession-making. Recent research
demonstrates that emotional expressions can potentially shift moral concerns during a
negotiation, such that displays of anger would backre if the negotiator associates moral
signicance to the objects of the negotiation, whereas displays of sadness promote higher
concession-making (Dehghani, Gratch, and Carnevale, 2012). Because morality signi-
cantly in
uences decision-making (e.g., Sj oberg and Winroth, 1986; Gintis, Bowles, Boyd,
and Fehr, 2003), I aim to examine the role of people's moral concerns on how they react
to emotional expressions and make concessions in this chapter.
It has been reported that morality signicantly in
uences decision-making (Sj oberg
and Winroth, 1986; Gintis, Bowles, Boyd, and Fehr, 2003). Adapting the Moral Foun-
dations Theory (Haidt and Graham, 2007; Haidt, Joseph, et al., 2007; Graham, 2013),
I explore the eects of two dierent types of foundations (i.e., Individualizing founda-
tions and Binding foundations) on concession-making. I predict that people who have
69
stronger Individualizing foundations would react more to emotional expressions because
the Individualizing foundations indicate the tendency to care about other people's emo-
tions (whether others are emotionally or physically suering, or being treated fairly) and
therefore, that would eect their concession-making. On the other hand, I predict that
people with stronger Binding foundations be more sensitive to their negotiation partner's
social status because Binding foundations indicate concern about other people's roles in
the group (whether negotiation partner is their boss or co-worker).
Understanding the interaction between moral concerns and emotion are crucial in de-
signing autonomous decision-making agents that operate in morally sensitive domains.
Progress in agent research has enabled us to work closely with software agents in morally
sensitive situations where agents' actions may lead to signicant results, such as loss
of life (Tambe, 2011; Dehghani, Immordino-Yang, Graham, Marsella, Forbus, Ginges,
Tambe, and Maheswaran, 2013). Therefore, it is important to better understand the
interactions between people's moral concerns, emotion and agent decision-making strate-
gies. My results suggest that incorporating understanding of people's moral concerns and
their reactions to emotional expressions are crucial in designing such agents.
5.2 Moral Foundations Theory
The Moral Foundations Theory (Haidt and Graham, 2007; Haidt, Joseph, et al., 2007;
Graham, 2013) posits that there are ve moral sensitivities that dierent cultures build
upon to dierent degrees. These ve foundations are:
Harm/Care: A concern for caring for and protecting individuals from harm.
70
Fairness/Reciprocity: A concern for justice and fairness.
In-group/Loyalty: A concern with issues of loyalty and self-sacrice for ones in-
group.
Authority/Subversion: A concern with issues associated with showing respect and
obedience to authority.
Purity/Sanctity: A concern for purity and sanctity.
The Moral Foundations Theory argues that each of these ve foundations serve dis-
tinct but related social functions and the degree of emphasis on these foundations varies
across cultures. This theory has been used to investigate political cultures (e.g., liber-
als and conservatives) and judgments about cultural issues (e.g., abortion, immigration,
same-sex marriage) (Koleva, Graham, Iyer, Ditto, and Haidt, 2012). Specically, Gra-
ham, Haidt, and Nosek (2009) demonstrate that liberals place more signicance on the
Harm/Care and Fairness/Reciprocity foundations relative to the other three foundations,
whereas conservatives place a relatively equally focus on all ve foundations. The 32-item
Moral Foundations Questionnaire (MFQ) measures the degree to which people value each
of ve foundations (Graham, Nosek, Haidt, Iyer, Koleva, and Ditto, 2011). MFQ is com-
posed of two dierent sections: moral relevance and moral judgment. Sample MFQ for
each domain is as follows:
Part 1. Moral relevance: When you decide whether something is right or wrong, to
what extent are the following considerations relevant to your thinking?
Harm/Care: Whether or not someone suered emotionally.
71
Fairness/Reciprocity: Whether or not some people treated dierently from
others.
Part 2. Moral Judgments: Please read the following sentences and indicate your agree-
ment or disagreement.
In-group/Loyalty: I am proud of my country's history.
Authority/Subversion: Men and women each have dierent roles to play in
society.
Purity/Sanctity: People should not do things that are disgusting, even if no
one is harmed.
Among ve foundations, Graham, Haidt, and Nosek (2009) refer to Harm/Care and
Fairness/Reciprocity as Individualizing foundations because they provide a foundation for
the liberal philosophical tradition in which the rights and welfare of individuals are empha-
sized. On the other hand, In-group/Loyalty, Authority/Subversion and Purity/Sanctity
are referred as Binding foundations because they provide a foundation for conservative
and religious moralities in which group-binding loyalty, duty, and self-control are empha-
sized. I hypothesize that Individual foundations would interact with the agent's expressed
emotion and Binding foundations would interact with the social status of the opponent.
5.3 Experiment
In the following experiment, I investigate the interplay of moral concerns and the inter-
personal eects of emotion in the Sacred-Objects Negotiation Task described above. As
72
discussed previously, my rst hypothesis is that strong preference on Individual founda-
tions (Harm and Fairness) would yield larger concessions to sad compared to angry agent
on items that are of moral importance to participants. Secondly, I hypothesize Bind-
ing foundations (In-group, Authority and Purity) interacts with opponent's social status
(boss vs. co-worker). I suggest that people with high Binding foundations concede more
to their boss than their co-worker, while people with low Binding foundations behave
reversely.
5.3.1 Participants
153 American MTurk workers (70 male and 83 female; mean age = 35.1) were paid $1
each to participate in my study. I used the same qualication requirement I used for my
previous behavioral experiment described in 4.2.1.1. On average it took each participant
15 minutes to complete my task.
5.3.2 Sacred-Objects Negotiation Task
I again used the Objects Negotiation Task introduced by Dehghani, Gratch, and Carnevale
(2012) for the experiment. To apply moral concerns on the negotiations, I edited negoti-
ation scenarios so that one of negotiation item is considered sacred. All participants were
asked to read the scenario before negotiation. Because of importance of the scenario,
`Review scenario' button was added on the top of the interface. During the game-play,
participants could review the scenario anytime by pressing the button. The Sacred-
Objects Negotiation Task interface is shown in the Figure 5.1.
73
Figure 5.1: Sacred-Objects Negotiation Task interface
5.3.3 Agent's Features
In this study, I used four types of agents characterized by the two types of emotions they
expressed and the two types of oer sets representing their negotiation strategies. More
details about these features are described in the following two sections.
5.3.3.1 Agent's Emotional Expressions
Agents follow one of two possible facial display policies depending on the condition: anger
or sadness. Regardless of a participant's oer, the angry agent always displays anger on
rounds 1, 3, and 5, and returning to a neutral face after ve seconds. The sad agent acts
in the same way but displays sadness instead of anger. In all other rounds, both agents
display a neutral face (Figure 5.2).
74
Figure 5.2: Emotional expressions used by agents.
Table 5.1: The negotiation strategy of the tough agent and the soft agent in moral
concerns study.
Round 1 Round 2 Round 3 Round 4 Round 5 Round 6
Tough [0, 0, 0, 0] [0, 1, 2, 2] [1, 0, 1, 2] [1, 1, 3, 2] [1, 2, 4, 4] [2, 1, 2, 2]
Soft [2, 1, 1, 1] [2, 1, 2, 2] [2, 2, 2, 3] [3, 1, 1, 1] [3, 2, 1, 2] [3, 3, 2, 2]
5.3.3.2 Agent's Negotiation Strategies
All agents in this study follow one of the following two strategies. In the rst strategy,
the agent starts with making no concessions at all (non-conceder agent) and concedes
little through the negotiation. In the second strategy, the agent starts with some conces-
sion (conceder) and gradually increases its concession. Both strategies involve more and
more concession over time. There are four dierent groups of items in the negotiations
([medicine packages, water bottles, food cans, hand sanitizers]), with ve items per group.
The negotiation strategy of non-conceder and conceder agent is shown in table 5.1 where
the numbers in the brackets represent how many times in each group the agent chooses
to give to the participant.
To decide whether to accept or reject a participant's oer, the agent uses hidden payo
values for each group of items; the value of medicine package, a water bottle, a food can,
and a hand sanitizer are estimated as 50, 10, 5, and 1 respectively. The agent accepts
75
Figure 5.3: Number of sacred items oered for agents and participants across both agent
strategies.
the oer only when the received oer has greater or equal values to the one the agent is
about to make. Otherwise, it rejects the oer and proposes a new oer. As this study
is designed to focus on a sacred item, much higher value was assigned on a sacred item
(medicine) than other items (a water bottle, a food can, and a hand sanitizer). Figure 5.3
shows the number of sacred items oered for agents and participants on each round across
both agent strategies.
5.3.4 Design
The experiment follows a 2 2 2 between-subject factorial design with the following
independent variables: Agent's expressed emotion (anger vs. sadness), experimental sce-
nario (boss vs. co-worker), and moral concern cluster (high Individualizing foundations
vs. low Individualizing foundations). The main dependent variable in my experiment is
demand dierence of medicine, which is calculated by subtracting demand of medicine
in round one from demand of medicine in the last round of negotiation. Higher demand
76
dierence of medicine indicates higher concession on medicine in the last round of nego-
tiation compared to the rst round. The maximum demand dierence is ve, and the
minimum demand dierence is zero in my setting.
5.3.5 Procedure
After agreeing on the consent form, each participant was rst given the 32-item MFQ. In
the MFQ, I inserted two questions with clear correct answer to ensure that the participants
were lling out the questionnaire in good faith. Participants who missed the lter question
were not allowed to participate in the experiment. Participants then received one of the
two scenarios described below. I dierentiate the hierarchy of the opponent in two levels,
boss and co-worker.
There has been an earthquake in the town you live in and many have been injured.
All roads to your town have been blocked and as a result aid is coming in very slowly.
Because of this every family has to split packages of aid sent using helicopters with
your [A: boss] [B: co-worker]'s family.
You and the family that have to split the aids with each other, both have babies
who have been injured and have developed infections. The only way to control the
spread of infection, which if not stopped will become lethal, is to use penicillin. You
are also running low on food, but have enough clean water that would last you for
several days. You have enough soaps, so you might not need any hand sanitizers.
77
Given the circumstances, you know that no other aid package will be received for
another week.
In the task, to review, you will negotiate with the other family over the aid
packages that include
1. Medicine (penicillin)
2. Water bottles
3. Canned food
4. Hand sanitizer
You have to negotiate how these items have to be split between your family and
your [A: boss] [B: co-worker]'s family. You do not know how much food and water
the other family has.
The negotiation is done in a sequence of alternating oers. You will make the
rst oer. The other negotiator may or may not accept your oer. If it does not
accept it, that is, if it rejects your oer, it will send you a new oer. You can either
accept or reject its oer. If you accept it, you will get to keep the items that you did
not give them. If you reject their oer, you can make another oer and submit it to
them. Both families will have a chance to make 6 oers in this negotiation.
After reading one of the scenarios, participants were asked to take a quiz, which
was designed to check their understanding of the experimental scenario. There were ve
questions (e.g., Which item in your possession is running low?; What is the only way
78
to control the spread of infection in your baby?). If they missed any of the questions,
they were asked to read the scenario again until they gave correct answers to all ve
questions in the quiz. This was to ensure that all participants completely understood the
experimental scenario and would provide reliable data.
I then assessed participants' values regarding the medicine package using Baron and
Spranca's measure of sacred value (Baron and Spranca, 1997). In line with this measure,
I asked participants `How do you feel about giving up the medicine package?' and they
received the following four choices:
a. I think this denitely needs to happen.
b. I do not object to this.
c. This is acceptable only if the benets of trading the medicine are great enough.
d. This should not be done no matter how great the benets.
I categorized participants who selected `d' as having a sacred value for the medicine
package. Participants then played the Sacred-Objects Negotiation Task with one of the
agents described above (conceder vs. non-conceder, sad vs. angry). Participants were
not told they would play with articial agents. After completing the Sacred-Objects
Negotiation Task, I asked participants to ll out a short demographic questionnaire.
5.4 Analysis and Results
Participants who dropped out of the negotiation before the third round were excluded
from the analysis (N = 20) because they made only one or two oers and were exposed
to the emotional displays of the agent only once. There was no eect of strategy, so I
79
collapsed across agent strategy conditions. Among a total of 133 participants, 96 par-
ticipants (72.2%) regarded the medicine package as a sacred item, hence I compared
participants who perceived medicine as sacred items compared to those who did not.
Extending the previous research, I hypothesized that an agent's anger and sadness
expression would interact with a negotiator's moral concerns. I clustered participants
based on their concerns on Individualizing (Harm and Fairness) and Binding (In-group,
Authority and Purity) foundations (measured using the MFQ).
5.4.1 Individualizing Foundations
Individualizing foundations (Harm and Fairness) concentrate on the individual as the
locus of moral value (Haidt and Graham, 2007). I hypothesized there would be behavioral
dierences in how participants with high or low Individualizing foundations negotiate in
a moral domain. I added participants' scores on the Individualizing foundations, and
performed a median split on my data using these scores. I named the people who had
high Harm and Fairness values as highHF and low Harm and Fairness value as lowHF.
The data was analyzed using a two way ANOVA with three between-subject fac-
tors, the agent's expressed emotion (anger vs. sadness), participants' perception to the
medicine package (sacred value vs. no sacred value) and moral foundation cluster (high
HF vs. low HF). There was a signicant interaction between agent emotion and sa-
cred value (F(1,83) = 4.248, p = 0.04). This is a replication of Dehghani, Gratch, and
Carnevale (2012) where they demonstrate that people conceded more to a sad agent than
an angry agent when they perceived a negotiation item as sacred, but act in an opposite
way when perceiving an item as non-sacred one (Figure 5.4, center). There was also
80
Figure 5.4: Demand dierence for medicine in highHF cluster (left), in total population
(center), in lowHF cluster (right)
a marginal three-way interaction between agent emotion, sacred values and HF cluster
(F(1,83) = 3.189, p = 0.07). To further investigate this three-way interaction, I analyzed
the data separately for the two HF clusters (highHF vs. lowHF). In the highHF clus-
ter, ANOVA results indicate a signicant interaction between emotion and sacred values
(F(1,39) = 6.545, p = 0.01). Following up, T-tests show that highHF participants con-
ceded more on morally signicant items to sad agents compared to angry agents (t(26)
= 2.598, p = 0.01). However, the interaction between emotion and sacred values is
insignicant in lowHF cluster (F(1,44) = 0.077, p = 0.78).
I also analyzed participants' frequency of expressed emotions. Our results show that
participants who have higher Individualizing concerns expressed emotions more frequently
than those who have low Individualizing concerns (t(131) = 2.003, p = 0.04).
81
Figure 5.5: Demand dierence for medicine when perceived as sacred item in high/low
IAP cluster
5.4.2 Binding Foundations
Binding foundations (In-group, Authority and Purity) concentrate on the group as the
locus of moral value (Haidt and Graham, 2007). I did the same clustering for Binding
foundations. I added participants' In-group, Authority and Purity values and divided all
the participants in two groups using the median. I referred to participants who had high
Binding foundation values as highIAP and low Binding foundations as lowIAP.
As my two scenarios included two dierent levels of hierarchy of the opponent (boss
vs. co-worker), I expected that participant's level of Binding concerns would aect their
concession rate. I performed a 2 2 ANOVA, where the rst factor was the scenario
(boss vs. co-worker) and the second factor was IAP cluster. As expected, there was a
signicant interaction between the scenario and the IAP cluster for demand dierence of
medicine (F(1,67) = 5.162, p = 0.02) (Figure 5.5).
82
5.5 Discussion
In this experiment, I demonstrated that emotions in morally charged negotiations interact
with moral concerns. Specically, the moral concerns a person considers most important
are directly correlated with their behavior in negotiations. The main ndings include that
Individualizing foundations (Harm and Fairness) interact with reactions to emotions,
while Binding foundations (In-group, Authority and Purity) interact with opponent's
social status rather than reactions to emotions.
People with high Individualizing foundations show greater concessions to sad com-
pared to angry agents on items that they associate a moral signicance to, while they
show greater concessions to angry compared to sad agents on items which they do not
consider as morally signicant. In other words, for people who value Harm and Fairness
strongly, when sacred value items are at stake in a negotiation, anger produces a counter-
productive eect; that is, concessions on morally signicant items become larger in the
case of seeing the other player's sad face and feeling that the other player is weak. The
nature of Individualizing foundations can explain this interplay of emotions and moral-
ity: this is due to valuing fairness and well-being of individuals over other factors. The
assumption made is that the sad player has suered some sort of injustice, or that the
sad player needs to be taken care of. Thus, people with high Individualizing foundations
make more concession on morally signicant items to sad agents in negotiations.
Another nding of this paper is that high Binding foundations are correlated with
the opponent's social status rather than opponent's emotional expressions. People with
higher Binding foundations concede more to an opponent with high social status than
83
one with low social status. This is because people with high Binding concerns tend to
care more about people's roles in the group, rather than an individual's emotional status.
Thus, a player with high group standing will be more respected, feared, or admired and
a person with high Bindings concerns will concede more to that player. This tendency
is reversed in people with lower Binding foundations, who care less about the authority
of the other side. They rather feel empathy for their co-workers, so they make more
concessions to their co-workers.
These ndings are important to understand what aects a negotiator's reaction to
emotional expression, hence they should be considered while designing autonomous agents
that are to operate in morally sensitive negotiations. Considering that decision-making in
morally sensitive missions (e.g., military operations) often results in life or death outcomes
for humans (Dehghani, Immordino-Yang, Graham, Marsella, Forbus, Ginges, Tambe, and
Maheswaran, 2013), accurate prediction of people's concession-making would be an im-
portant factor. For example, if a robot is trying to save people from a disaster, based on
that culture's moral concerns, it might be more eective for it to express sadness to per-
suade them to leave their morally signicant possessions and escape from the dangerous
area.
Related to my ndings, a possible direction for future work would be exploring brain
patterns during morally-charged human-agent negotiations. Lewis, Kanai, Bates, and
Rees (2012) have shown that each Individualizing and Binding foundations are associ-
ated with specic brain structures. The left dorsomedial prefrontal cortex were found to
be positively associated with Individualizing foundations, and positive association with
bilateral subcallosal gyrus was found for Binding foundations. Therefore, it would be
84
interesting to study brain patterns on these regions when participants who have dier-
ent level of Individualizing/Binding foundations interact with agents that express anger
or sad, and that have high or low group standing. Studying dierences in neural pat-
terns that are related to one's moral concerns would provide a clue nding how these
information are processed.
85
Chapter 6
Conclusions and Future Directions
6.1 Conclusions
This dissertation described four connected research topics that converge to information
decoding in human-agent negotiations. The rst topic was on predicting human negotia-
tor's oer types and the second topic was on decoding agent's properties. Next topic was
on decoding partner type, whether the partner is another human or a computer program.
The last topic was on the eect of moral concerns on concession-making. To sum up, I
readdress the contributions of this dissertation.
1. Predicting Human Negotiator's Oer Types from Brain Patterns
This dissertation demonstrated that human negotiator's oer types can be predicted
from brain patterns. Participants negotiated with a computer agent over multiple
rounds inside an fMRI machine, and their brain activity during negotiations were
collected. Participants' oer types were then classied in three categories; whether
the negotiator concedes, does not change, or asks for more. During dierent types
of oers, distinct brain patterns were found in the left dorsal anterior insula which
86
is one of well-known emotion-related brain regions. This nding is supported by
previous studies that have shown the importance of the dorsal anterior insula in
empathy for the suering of others (Bernhardt, Klimecki, Leiberg, and Singer, 2013)
and enhanced activity in the left anterior insula during the processing of fearful
faces (Morris, Friston, B uchel, Frith, Young, Calder, and Dolan, 1998).
2. Decoding an Agent's Properties from Brain Patterns
This dissertation explored the relationship between brain patterns and a computer
agent's properties including its emotional expressions and its negotiation strategies.
Participants took part in negotiations with a computer agent that uses two dierent
negotiation strategies and three dierent emotional expressions, and their brain
activity during negotiations were recorded. Participants' brain patterns were then
analyzed using MVPA. It was found that agents' emotional expressions can be
decoded based on brain patterns from the left dorsal anterior insula, and agent's
negotiation strategies from the frontal pole. The left dorsal anterior insula is one
of well-known emotion-related brain regions (Wager and Barrett, 2004) and the
frontal pole is one of well-known decision-making-related brain regions (Okuda,
Fujii, Ohtake, Tsukiura, Tanji, Suzuki, Kawashima, Fukuda, Itoh, and Yamadori,
2003).
3. Decoding Negotiation Partner's Type from Brain Patterns
This dissertation investigated the dierences in people's behavior and brain activity
during the course of a negotiation with agents labeled as either `another partici-
pant' or a `computer program'. It was found that people perceive human-labeled
87
agents more human-like than computer-labeled agents, and the level of concession
in the negotiations is dependent on agent type. By showing that parts of the
ToM neural correlates are activated in human-labeled agent conditions but not in
computer-labeled agent conditions, this dissertation demonstrated that dierences
in negotiation partner type can be captured in brain activation. Further, the brain
activity was found to be able to predict whether the negotiation agent was intro-
duced as a competing human player or a computer program. This suggests that
labeling an interaction partner as either another human or a computer program
leads to signicant impacts on one's decision making.
4. Exploring the Eects of Moral Concerns
This dissertation explored the interplay between people's moral concerns, emotional
expressions and concession-making during a morally charged negotiation. Sacred
values were introduced in the negotiations to study the relationship between ones'
moral foundations (Haidt and Graham, 2007) and their negotiation behavior. The
results demonstrated that participants who has stronger concerns for the Individu-
alizing foundations including Harm and Fairness make greater concessions for sacred
negotiation items when faced with a sad opponent than an angry opponent. Also, it
was found that participants who has high Binding foundations including In-group,
Authority and Purity are more sensitive to social status, and make greater conces-
sions in scenarios that involve agents in a higher social status.
These ndings can help developing eective negotiation agents as they provide a clue
to the dierences of fundamental cognitive processes underlying negotiation behavior.
88
For example, my results suggest that people think more about the agent's intention when
they interact with a human-labeled agent than with a computer-labeled agent. Therefore,
we can choose to label a negotiation partner either as another human or a computer based
on the purpose of the negotiation. Also, these ndings can help designing online adaptive
agents. Even though fMRI analyses were done after the experiment in my studies, the
analyses can be done in real time for an agent to decide necessary actions. To make this
possible, more studies would be needed so that brain regions to be analyzed for various
types of decisions can be specied.
6.2 Future Directions
The work presented in this dissertation points to a series of possible future directions
that would be interesting to take as follow-up research.
Decode Further Information about Human-Agent Negotiations
An interesting follow-up research work would be to decode more information about
human-agent negotiations from brain patterns. For instance, participants' emotions
or their stance towards the partner during the negotiation could be the decodable
information. In the modied Objects Negotiation Task I used in chapter 4, I added
a participant's emotion report phase where the participant reports his/her emotion
about the agent's oer. Hence, brain patterns on emotion-related brain regions dur-
ing oer review phase can be investigated to nd its correlation with the reported
emotion. On the other hand, participants' stance towards their negotiation partner
89
can be studied by manipulating the negotiating partner's aliation. More speci-
cally, brain patterns in cases where the negotiating partner's group is the same as
the negotiator's (in-group) and where that is not the same as the negotiator's (out-
group) can be compared. In-group bias, which refers to the tendency to evaluate
one's own group members more favorably than other groups, has been consistently
reported in social sciences (Tajfel, Billig, Bundy, and Flament, 1971; Hewstone,
Rubin, and Willis, 2002). By manipulating the negotiating partner's aliation, I
expect to nd dierences in brain patterns between negotiations with an in-group
member and one with an out-group member.
Explore Brain Patterns during Sacred-Objects Negotiations
Another interesting direction would be to explore brain patterns during human-
agent negotiations where sacred issues are at stake. Previous studies have reported
the importance of sacred values in negotiations, by showing that oering material
incentives to compromise backres when negotiators posit sacred values on the ne-
gotiation item (Ginges, Atran, Medin, and Shikaki, 2007; Dehghani, Iliev, Sachdeva,
Atran, Ginges, and Medin, 2009). These suggest that there are dierences between
the way people negotiate sacred items and normal items. Therefore, comparing
brain patterns of people who posit sacred values on the negotiation item with ones
of people who do not would provide a clue on where the sacredness information are
processed in the brain and ultimately why these dierences occur. Also, compar-
ing brain patterns of people with high Individualizing foundations to one with low
Individualizing foundations would be interesting since I found the Individualizing
90
foundations interact with reactions to emotions and Binding foundations interact
with opponent's social status. Comparing those brain patterns would tell us which
brain regions are closely related to one's specic moral concerns.
Expand Beyond the Object Negotiation Task
Even though the Objects Negotiation Task is much closer to a real-life negotiation
than a single-shot negotiation such as the ultimatum game, there is still a room
for improvement. For example, in the Object Negotiation Task, an agent interacts
with a participant only through negotiation oers and emotional expressions. To
make the negotiation task more realistic, a chatroom which allows an agent verbally
interact with a participant could be added (Gratch et al., 2013). A chat-based ne-
gotiation agent proposed by Rosenfeld, Zuckerman, Segal-Halevi, Drein, and Kraus
(2014) could be an example that shows how an agent can interact with a participant
during negotiations via a chatroom. Further, some of our ndings can be explored
outside of the negotiation context all together, and can be used to study interaction
during educational games (Kr amer et al., 2016). In addition, emotional expressions
that has smooth transition from one to another emotion could be used instead of
current discrete images. These improvements would help us to study negotiations
that are closer to real-life ones.
Apply Deep Learning Techniques
Deep learning techniques have been found to construct better representations in
other domains such as object recognition and natural language processing (Bengio,
Courville, and Vincent, 2013). Thus, applying deep learning techniques on top of
91
the current fMRI analysis is expected to result in powerful representation learning,
leading to higher accuracies in prediction and decoding of human-agent negotiation
features. Even though several studies have tried to use deep learning techniques
to explore brain imaging data (Plis, Hjelm, Salakhutdinov, Allen, Bockholt, Long,
Johnson, Paulsen, Turner, and Calhoun, 2014; Hatakeyama, Yoshida, Kataoka, and
Okuhara, 2014), they were limited to structural MRI data instead of fMRI data or
simple three-layer networks. To analyze fMRI data, a deeper architecture such as
Convolutional Neural Networks (CNNs) can be used. CNNs is a particular type
of deep architecture that has achieved the state-of-the-art large-scale visual recog-
nition (Russakovsky, Deng, Su, Krause, Satheesh, Ma, Huang, Karpathy, Khosla,
Bernstein, et al., 2014). Since fMRI data is a collection of 3D images of voxel
intensities, fMRI data could be treated as image les and therefore CNNs can be
used to analyze them. Although more brain data need to be collected rst because
large-scale data is required to apply deep learning techniques, the usage of deep ar-
chitecture would greatly improve accuracies in prediction and decoding negotiation
information.
92
Bibliography
K. G. Allred. Anger and retaliation: Toward an understanding of impassioned con
ict in
organizations. Research on negotiation in organizations, 7:27{58, 1999.
D. R. Ames and G. V. Johar. I'll know what you're like when i see how you feel: How and
when aective displays in
uence behavior-based impressions. Psychological Science, 20
(5):586{593, 2009.
D. M. Amodio and C. D. Frith. Meeting of minds: the medial frontal cortex and social
cognition. Nature Reviews Neuroscience, 7(4):268{277, 2006.
S. W. Anderson, A. Bechara, H. Damasio, D. Tranel, and A. R. Damasio. Impairment of
social and moral behavior related to early damage in human prefrontal cortex. Nature
neuroscience, 2(11):1032{1037, 1999.
E. Andr e, M. Klesen, P. Gebhard, S. Allen, T. Rist, et al. Exploiting models of person-
ality and emotions to control the behavior of animated interactive agents. In Fourth
International Conference on Autonomous Agents, pages 3{7, 2000.
A. Banh, D. J. Rea, J. E. Young, and E. Sharlin. Inspector baxter: The social aspects
of integrating a robot as a quality inspector in an assembly line. In Proceedings of the
3rd International Conference on Human-Agent Interaction, pages 19{26. ACM, 2015.
J. Baron and M. Spranca. Protected values. Organizational behavior and human decision
processes, 70(1):1{16, 1997.
B. Barry, I. S. Fulmer, and N. Goates. Bargaining with feeling: Emotionality in and
around negotiation. Negotiation theory and research, pages 99{127, 2006.
C. Bartneck, D. Kuli c, E. Croft, and S. Zoghbi. Measurement instruments for the an-
thropomorphism, animacy, likeability, perceived intelligence, and perceived safety of
robots. International journal of social robotics, pages 71{81, 2009.
C. D. Batson and T. Moran. Empathy-induced altruism in a prisoner's dilemma. European
Journal of Social Psychology, 29(7):909{924, 1999.
R. Beale and C. Creed. Aective interaction: How emotional agents aect users. Inter-
national Journal of Human-Computer Studies, 67(9):755{776, 2009.
A. Bechara and A. R. Damasio. The somatic marker hypothesis: A neural theory of
economic decision. Games and economic behavior, 52(2):336{372, 2005.
93
A. Bechara and M. Van Der Linden. Decision-making and impulse control after frontal
lobe injuries. Current opinion in neurology, 18(6):734{739, 2005.
A. Bechara, D. Tranel, H. Damasio, and A. R. Damasio. Failure to respond autonomically
to anticipated future outcomes following damage to prefrontal cortex. Cerebral cortex,
6(2):215{225, 1996.
Y. Bengio, A. Courville, and P. Vincent. Representation learning: A review and new
perspectives. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 35
(8):1798{1828, 2013.
B. C. Bernhardt, O. M. Klimecki, S. Leiberg, and T. Singer. Structural covariance
networks of the dorsal anterior insula predict females' individual dierences in empathic
responding. Cerebral Cortex, page bht072, 2013.
T. Bickmore, D. Schulman, and L. Yin. Maintaining engagement in long-term interven-
tions with relational agents. Applied Articial Intelligence, 24(6):648{666, 2010.
G. E. Bolton, E. Katok, and R. Zwick. Dictator game giving: Rules of fairness versus
acts of kindness. International journal of game theory, 27(2):269{299, 1998.
S. Brave, C. Nass, and K. Hutchinson. Computers that care: investigating the eects of
orientation of emotion exhibited by an embodied computer agent. International journal
of human-computer studies, 62(2):161{178, 2005.
T. Burnham, K. McCabe, and V. L. Smith. Friend-or-foe intentionality priming in an
extensive form trust game. Journal of Economic Behavior & Organization, 43(1):57{73,
2000.
A. Byde, M. Yearworth, K.-Y. Chen, and C. Bartolini. Autona: A system for auto-
mated multiple 1-1 negotiation. In E-Commerce, 2003. CEC 2003. IEEE International
Conference on, pages 59{67. IEEE, 2003.
J. Cassell and K. R. Thorisson. The power of a nod and a glance: Envelope vs. emotional
feedback in animated conversational agents. Applied Articial Intelligence, 13(4-5):
519{538, 1999.
G. Castellano, A. Pereira, I. Leite, A. Paiva, and P. W. McOwan. Detecting user en-
gagement with a robot companion using task and social interaction-based features.
In Proceedings of the 2009 international conference on Multimodal interfaces, pages
119{126. ACM, 2009.
T. Chaminade, J. Hodgins, and M. Kawato. Anthropomorphism in
uences perception of
computer-animated characters' actions. Social cognitive and aective neuroscience, 2
(3):206{216, 2007.
A. Choi, C. D. Melo, W. Woo, and J. Gratch. Aective engagement to emotional facial
expressions of embodied social agents in a decision-making game. Computer Animation
and Virtual Worlds, 23(3-4):331{342, 2012.
94
R. Cowie, E. Douglas-Cowie, N. Tsapatsoulis, G. Votsis, S. Kollias, W. Fellenz, and
J. G. Taylor. Emotion recognition in human-computer interaction. Signal Processing
Magazine, IEEE, 18(1):32{80, 2001.
A. R. Damasio, B. Everitt, and D. Bishop. The somatic marker hypothesis and the
possible functions of the prefrontal cortex [and discussion]. Philosophical Transactions
of the Royal Society B: Biological Sciences, 351(1346):1413{1420, 1996.
A. R. Damasio, T. J. Grabowski, A. Bechara, H. Damasio, L. L. Ponto, J. Parvizi, and
R. D. Hichwa. Subcortical and cortical brain activity during the feeling of self-generated
emotions. Nature neuroscience, 3(10):1049{1056, 2000.
R. Das, J. E. Hanson, J. O. Kephart, and G. Tesauro. Agent-human interactions in the
continuous double auction. In International Joint Conference on Articial Intelligence,
volume 17, pages 1169{1178. Lawrence Erlbaum Associates Ltd, 2001.
C. de Melo, S. Marsella, and J. Gratch. People do not feel guilty about exploiting
machines. ACM Transactions on Computer-Human Interaction (TOCHI), 23(2):8,
2016.
C. M. de Melo, P. Carnevale, and J. Gratch. The eect of expression of anger and
happiness in computer agents on negotiations with humans. In The 10th International
Conference on Autonomous Agents and Multiagent Systems-Volume 3, pages 937{944.
International Foundation for Autonomous Agents and Multiagent Systems, 2011a.
C. M. de Melo, P. Carnevale, and J. Gratch. The impact of emotion displays in embodied
agents on emergence of cooperation with people. Presence: teleoperators and virtual
environments, 20(5):449{465, 2011b.
C. M. de Melo, P. Carnevale, and J. Gratch. The eect of virtual agents' emotion displays
and appraisals on people's decision making in negotiation. In Intelligent Virtual Agents,
pages 53{66. Springer, 2012.
C. M. de Melo, P. J. Carnevale, S. J. Read, and J. Gratch. Reading people?s minds
from emotion expressions in interdependent decision making. Journal of personality
and social psychology, 106(1):73, 2014.
B. Deen, N. B. Pitskel, and K. A. Pelphrey. Three systems of insular functional connec-
tivity identied with cluster analysis. Cerebral Cortex, 21(7):1498{1506, 2011.
M. Dehghani, R. Iliev, S. Sachdeva, S. Atran, J. Ginges, and D. Medin. Emerging sacred
values: Iran's nuclear program. Judgment and Decision Making, 4(7):530, 2009.
M. Dehghani, J. Gratch, and P. J. Carnevale. Interpersonal eects of emotions in morally-
charged negotiations. In Proceedings of the 34th Annual Meeting of the Cognitive Sci-
ence Society, 2012.
95
M. Dehghani, M. H. Immordino-Yang, J. Graham, S. Marsella, K. Forbus, J. Ginges,
M. Tambe, and R. Maheswaran. Computational models of moral perception, con
ict
and elevation. Proceedings of the International Association for Computing and Philos-
ophy (IACAP), 2013.
M. Dehghani, P. J. Carnevale, and J. Gratch. Interpersonal eects of expressed anger and
sorrow in morally charged negotiation. Judgment and Decision Making, 9(2):104{113,
2014.
M. Dehghani, K. Johnson, J. Hoover, E. Sagi, J. Garten, N. J. Parmar, S. Vaisey, R. Iliev,
and J. Graham. Purity homophily in social networks. Journal of Experimental Psy-
chology: General, 145(3):366, 2016.
D. M. Dehn and S. Van Mulken. The impact of animated interface agents: a review of
empirical research. International journal of human-computer studies, 52(1):1{22, 2000.
R. S. Desikan, F. S egonne, B. Fischl, B. T. Quinn, B. C. Dickerson, D. Blacker, R. L.
Buckner, A. M. Dale, R. P. Maguire, B. T. Hyman, et al. An automated labeling
system for subdividing the human cerebral cortex on mri scans into gyral based regions
of interest. Neuroimage, 31(3):968{980, 2006.
A. C. Evans, D. L. Collins, S. Mills, E. Brown, R. Kelly, and T. M. Peters. 3d statistical
neuroanatomical models from 305 mri volumes. In Nuclear Science Symposium and
Medical Imaging Conference, 1993., 1993 IEEE Conference Record., pages 1813{1817.
IEEE, 1993.
H. L. Gallagher and C. D. Frith. Functional imaging of theory of mind. Trends in cognitive
sciences, 7(2):77{83, 2003.
H. L. Gallagher, A. I. Jack, A. Roepstor, and C. D. Frith. Imaging the intentional stance
in a competitive game. Neuroimage, 16(3):814{821, 2002.
J. Ginges, S. Atran, D. Medin, and K. Shikaki. Sacred bounds on rational resolution of
violent political con
ict. Proceedings of the National Academy of Sciences, 104(18):
7357{7360, 2007.
H. Gintis, S. Bowles, R. Boyd, and E. Fehr. Explaining altruistic behavior in humans.
Evolution and human Behavior, 24(3):153{172, 2003.
J. Graham. Mapping the moral maps: From alternate taxonomies to competing predic-
tions. Personality and Social Psychology Review, 17(3):237{241, 2013.
J. Graham, J. Haidt, and B. A. Nosek. Liberals and conservatives rely on dierent sets
of moral foundations. Journal of personality and social psychology, 96(5):1029, 2009.
J. Graham, B. A. Nosek, J. Haidt, R. Iyer, S. Koleva, and P. H. Ditto. Mapping the
moral domain. Journal of personality and social psychology, 101(2):366, 2011.
J. Gratch, N. Wang, J. Gerten, E. Fast, and R. Duy. Creating rapport with virtual
agents. In Intelligent Virtual Agents, pages 125{138. Springer, 2007.
96
J. Gratch, A. Hartholt, M. Dehghani, and S. Marsella. Virtual humans: a new toolkit
for cognitive science research. Applied Articial Intelligence, 19:215{233, 2013.
H. M. Gray, K. Gray, and D. M. Wegner. Dimensions of mind perception. Science, 315
(5812):619{619, 2007.
I. Guyon and A. Elissee. An introduction to variable and feature selection. The Journal
of Machine Learning Research, 3:1157{1182, 2003.
J. Haidt and J. Graham. When morality opposes justice: Conservatives have moral
intuitions that liberals may not recognize. Social Justice Research, 20(1):98{116, 2007.
J. Haidt, C. Joseph, et al. The moral mind: How ve sets of innate intuitions guide the
development of many culture-specic virtues, and perhaps even modules. The innate
mind, 3:367{391, 2007.
M. Hanke, Y. O. Halchenko, P. B. Sederberg, E. Olivetti, I. Fr und, J. W. Rieger, C. S.
Herrmann, J. V. Haxby, S. J. Hanson, and S. Pollmann. Pymvpa: a unifying approach
to the analysis of neuroscientic data. Frontiers in neuroinformatics, 3, 2009.
M. Haruno and C. D. Frith. Activity in the amygdala elicited by unfair divisions predicts
social value orientation. Nature neuroscience, 13(2):160{161, 2010.
Y. Hatakeyama, S. Yoshida, H. Kataoka, and Y. Okuhara. Multi-voxel pattern analysis
of fmri based on deep learning methods. In Soft Computing in Big Data Processing,
pages 29{38. Springer, 2014.
H. R. Heekeren, S. Marrett, P. Bandettini, and L. Ungerleider. A general mechanism for
perceptual decision-making in the human brain. Nature, 431(7010):859{862, 2004.
F. Hegel, S. Krach, T. Kircher, B. Wrede, and G. Sagerer. Understanding social robots:
A user study on anthropomorphism. In Robot and Human Interactive Communication,
2008. RO-MAN 2008. The 17th IEEE International Symposium on, pages 574{579.
IEEE, 2008.
U. Hess and P. Bourgeois. You smile{i smile: Emotion expression in social interaction.
Biological psychology, 84(3):514{520, 2010.
M. Hewstone, M. Rubin, and H. Willis. Intergroup bias. Annual review of psychology, 53
(1):575{604, 2002.
D. Houser, D. Schunk, and J. Winter. Trust games measure trust. Technical report,
Munich Discussion Paper, 2006.
F. Iacobelli and J. Cassell. Ethnic identity and engagement in embodied conversational
agents. In Intelligent virtual agents, pages 57{63. Springer, 2007.
M. Jenkinson, P. Bannister, M. Brady, and S. Smith. Improved optimization for the robust
and accurate linear registration and motion correction of brain images. Neuroimage,
17(2):825{841, 2002.
97
D. Jurafsky, R. Ranganath, and D. McFarland. Extracting social meaning: Identifying
interactional style in spoken conversation. In Proceedings of Human Language Tech-
nologies: The 2009 Annual Conference of the North American Chapter of the Asso-
ciation for Computational Linguistics, pages 638{646. Association for Computational
Linguistics, 2009.
P. R. Killeen. An alternative to null-hypothesis signicance tests. Psychological science,
16(5):345{353, 2005.
E. Kim, M. Dehghani, Y. K. Kim, P. J. Carnevale, and J. Gratch. Eects of moral
concerns on negotiations. Proceedings of the 36th Annual Meeting of the Cognitive
Science Society, 2014.
E. Kim, S. I. Gimbel, A. Litvinova, J. T. Kaplan, and M. Dehghani. Predicting decision in
human-agent negotiation using functional mri. Proceedings of the 38th Annual Meeting
of the Cognitive Science Society, 2016.
T. Kircher, I. Bl umel, D. Marjoram, T. Lataster, L. Krabbendam, J. Weber, J. van
Os, and S. Krach. Online mentalising investigated with functional mri. Neuroscience
letters, 454(3):176{181, 2009.
H. Kober, L. F. Barrett, J. Joseph, E. Bliss-Moreau, K. Lindquist, and T. D. Wager.
Functional grouping and cortical{subcortical interactions in emotion: a meta-analysis
of neuroimaging studies. Neuroimage, 42(2):998{1031, 2008.
S. P. Koleva, J. Graham, R. Iyer, P. H. Ditto, and J. Haidt. Tracing the threads: How
ve moral concerns (especially purity) help explain culture war attitudes. Journal of
Research in Personality, 46(2):184{194, 2012.
S. Krach, F. Hegel, B. Wrede, G. Sagerer, F. Binkofski, and T. Kircher. Can machines
think? interaction and perspective taking with robots investigated via fmri. PloS one,
3(7):e2597, 2008.
N. C. Kr amer, B. Karacora, G. Lucas, M. Dehghani, G. R uther, and J. Gratch. Closing
the gender gap in stem with friendly male instructors? on the eects of rapport behavior
and gender of a virtual agent in an instructional interaction. Computers & Education,
99:1{13, 2016.
N. Kriegeskorte, R. Goebel, and P. Bandettini. Information-based functional brain map-
ping. Proceedings of the National Academy of Sciences of the United States of America,
103(10):3863{3868, 2006.
C. Lamm and T. Singer. The role of anterior insular cortex in social emotions. Brain
Structure and Function, 214(5-6):579{591, 2010.
J. LeDoux. The emotional brain, fear, and the amygdala. Cellular and molecular neuro-
biology, 23(4-5):727{738, 2003.
98
J. S. Lerner, Y. Li, P. Valdesolo, and K. S. Kassam. Emotion and decision making.
Annual Review of Psychology, 66:799{823, 2015.
G. J. Lewis, R. Kanai, T. C. Bates, and G. Rees. Moral values are associated with
individual dierences in regional brain volume. Journal of cognitive neuroscience, 24
(8):1657{1663, 2012.
R. Lin, S. Kraus, J. Wilkenfeld, and J. Barry. Negotiating with bounded rational agents
in environments with incomplete information using an automated agent. Articial
Intelligence, 172(6):823{851, 2008.
H. Maldonado, J.-E. R. Lee, S. Brave, C. Nass, H. Nakajima, R. Yamada, K. Iwamura,
and Y. Morishima. We learn better together: enhancing elearning with emotional
characters. In Proceedings of th 2005 conference on Computer support for collaborative
learning: learning 2005: the next 10 years!, pages 408{417. International Society of the
Learning Sciences, 2005.
M. Moayedi, T. V. Salomons, K. A. Dunlop, J. Downar, and K. D. Davis. Connectivity-
based parcellation of the human frontal polar cortex. Brain Structure and Function,
pages 1{14, 2014.
R. Moreno, R. E. Mayer, H. A. Spires, and J. C. Lester. The case for social agency
in computer-based teaching: Do students learn more deeply when they interact with
animated pedagogical agents? Cognition and Instruction, 19(2):177{213, 2001.
J. S. Morris, K. J. Friston, C. B uchel, C. D. Frith, A. W. Young, A. J. Calder, and R. J.
Dolan. A neuromodulatory role for the human amygdala in processing emotional facial
expressions. Brain, 121(1):47{57, 1998.
K. A. Norman, S. M. Polyn, G. J. Detre, and J. V. Haxby. Beyond mind-reading: multi-
voxel pattern analysis of fmri data. Trends in cognitive sciences, 10(9):424{430, 2006.
S. Ogawa, T.-M. Lee, A. R. Kay, and D. W. Tank. Brain magnetic resonance imaging
with contrast dependent on blood oxygenation. Proceedings of the National Academy
of Sciences, 87(24):9868{9872, 1990.
J. Okuda, T. Fujii, H. Ohtake, T. Tsukiura, K. Tanji, K. Suzuki, R. Kawashima,
H. Fukuda, M. Itoh, and A. Yamadori. Thinking of the future and past: The roles
of the frontal pole and the medial temporal lobes. Neuroimage, 19(4):1369{1380, 2003.
M. Olekalns. Negotiation as social interaction. Australian Journal of Management, 27(1
suppl):39{46, 2002.
M. M. Pillutla and J. K. Murnighan. Unfairness, anger, and spite: Emotional rejections
of ultimatum oers. Organizational behavior and human decision processes, 68(3):208{
224, 1996.
99
S. M. Plis, D. R. Hjelm, R. Salakhutdinov, E. A. Allen, H. J. Bockholt, J. D. Long,
H. J. Johnson, J. S. Paulsen, J. A. Turner, and V. D. Calhoun. Deep learning for
neuroimaging: a validation study. Frontiers in neuroscience, 8, 2014.
A. Powers and S. Kiesler. The advisor robot: tracing people's mental model from a robot's
physical attributes. In Proceedings of the 1st ACM SIGCHI/SIGART conference on
Human-robot interaction, pages 218{225. ACM, 2006.
D. Premack and G. Woodru. Does the chimpanzee have a theory of mind? Behavioral
and brain sciences, 1(04):515{526, 1978.
J. K. Rilling, J. E. Dagenais, D. R. Goldsmith, A. L. Glenn, and G. Pagnoni. Social
cognitive neural networks during in-group and out-group interactions. Neuroimage, 41
(4):1447{1461, 2008.
A. Rosenfeld, I. Zuckerman, E. Segal-Halevi, O. Drein, and S. Kraus. Negochat: a
chat-based negotiation agent. In Proceedings of the 2014 international conference on
Autonomous agents and multi-agent systems, pages 525{532. International Foundation
for Autonomous Agents and Multiagent Systems, 2014.
S. Ruiz, S. Lee, S. R. Soekadar, A. Caria, R. Veit, T. Kircher, N. Birbaumer, and
R. Sitaram. Acquired self-control of insula cortex modulates emotion recognition and
brain network connectivity in schizophrenia. Human brain mapping, 34(1):200{212,
2013.
O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy,
A. Khosla, M. Bernstein, et al. Imagenet large scale visual recognition challenge.
International Journal of Computer Vision, pages 1{42, 2014.
A. G. Sanfey, J. K. Rilling, J. A. Aronson, L. E. Nystrom, and J. D. Cohen. The neural
basis of economic decision-making in the ultimatum game. Science, 300(5626):1755{
1758, 2003.
G. Silani, C. Lamm, C. C. Ru, and T. Singer. Right supramarginal gyrus is crucial
to overcome emotional egocentricity bias in social judgments. The Journal of Neuro-
science, 33(39):15466{15476, 2013.
M. Sinaceur and L. Z. Tiedens. Get mad and get more than even: When and why anger
expression is eective in negotiations. Journal of Experimental Social Psychology, 42
(3):314{322, 2006.
L. Sj oberg and E. Winroth. Risk, moral value of actions, and mood. Scandinavian Journal
of Psychology, 27(1):191{208, 1986.
M. L. Smith and B. Milner. The role of the right hippocampus in the recall of spatial
location. Neuropsychologia, 19(6):781{793, 1981.
S. M. Smith. Fast robust automated brain extraction. Human brain mapping, 17(3):
143{155, 2002.
100
S. M. Smith, M. Jenkinson, M. W. Woolrich, C. F. Beckmann, T. E. Behrens,
H. Johansen-Berg, P. R. Bannister, M. De Luca, I. Drobnjak, D. E. Flitney, et al.
Advances in functional and structural mr image analysis and implementation as fsl.
Neuroimage, 23:S208{S219, 2004.
H. Tajfel, M. G. Billig, R. P. Bundy, and C. Flament. Social categorization and intergroup
behaviour. European journal of social psychology, 1(2):149{178, 1971.
M. Tambe. Security and game theory: algorithms, deployed systems, lessons learned.
Cambridge University Press, 2011.
G. A. Van Kleef, C. K. De Dreu, and A. S. Manstead. The interpersonal eects of
emotions in negotiations: a motivated information processing approach. Journal of
personality and social psychology, 87(4):510, 2004a.
G. A. Van Kleef, C. K. De Dreu, A. S. Manstead, et al. The interpersonal eects of
anger and happiness in negotiations. Journal of personality and social psychology, 86
(1):57{76, 2004b.
G. A. Van Kleef, C. K. De Dreu, and A. S. Manstead. An interpersonal approach to
emotion in social decision making: The emotions as social information model. Advances
in experimental social psychology, 42:45{96, 2010.
M. Van't Wout, R. S. Kahn, A. G. Sanfey, and A. Aleman. Aective state and decision-
making in the ultimatum game. Experimental brain research, 169(4):564{568, 2006.
T. D. Wager and L. F. Barrett. From aect to control: Functional specialization of the
insula in motivation and regulation. Published online at PsycExtra, 2004.
N. Wang and J. Gratch. Can virtual human build rapport and promote learning? In
AIED, pages 737{739, 2009.
E. K. Warrington and R. Pratt. Language laterality in left-handers assessed by unilateral
ect. Neuropsychologia, 11(4):423{428, 1973.
C. R. Williams, V. Robu, E. H. Gerding, and N. R. Jennings. Using gaussian processes
to optimise concession in complex negotiations against unknown opponents. 2011.
101

Abstract (if available)

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

A framework for research in human-agent negotiation

PDF

The interpersonal effect of emotion in decision-making and social dilemmas

PDF

Automated negotiation with humans

PDF

Planning with continuous resources in agent systems

PDF

The human element: addressing human adversaries in security domains

PDF

Decoding situational perspective: incorporating contextual influences into facial expression perception modeling

PDF

Spatial and temporal patterns of brain activity associated with emotions in music

PDF

Computational foundations for mixed-motive human-machine dialogue

PDF

An intelligent tutoring system’s approach for negotiation training

PDF

Modeling emotional effects on decision-making by agents in game-based simulations

PDF

Behavior understanding from speech under constrained conditions: exploring sparse networks, transfer and unsupervised learning

PDF

Parasocial consensus sampling: modeling human nonverbal behaviors from multiple perspectives

PDF

Towards social virtual listeners: computational models of human nonverbal behaviors

PDF

Human adversaries in security games: integrating models of bounded rationality and fast algorithms

PDF

Hierarchical planning in security games: a game theoretic approach to strategic, tactical and operational decision making

PDF

Game theoretic deception and threat screening for cyber security

PDF

An investigation of fully interactive multi-role dialogue agents

PDF

Improving language understanding and summarization by leveraging auxiliary information through self-supervised or unsupervised learning

PDF

Computational modeling of human behavior in negotiation and persuasion: the challenges of micro-level behavior annotations and multimodal modeling

PDF

Predicting and modeling human behavioral changes using digital traces

Asset Metadata

Creator Kim, Eunkyung (author)

Core Title Decoding information about human-agent negotiations from brain patterns

School Viterbi School of Engineering

Degree Doctor of Philosophy

Degree Program Computer Science

Publication Date 10/24/2018

Defense Date 04/24/2017

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag decision-making,emotion,fMRI,human-agent interaction,negotiation,OAI-PMH Harvest

Format application/pdf (imt)

Language English

Contributor Electronically uploaded by the author (provenance)

Advisor Dehghani, Morteza (committee chair), Gratch, Jonathan (committee member), Kaplan, Jonas (committee member), Tambe, Milind (committee member)

Creator Email eungaeng@gmail.com,eunkyung@usc.edu

Permanent Link (DOI) https://doi.org/10.25549/usctheses-c89-98640

Unique identifier UC11675706

Identifier etd-KimEunkyun-6907.pdf (filename),usctheses-c89-98640 (legacy record id)

Legacy Identifier etd-KimEunkyun-6907.pdf

Dmrecord 98640

Document Type Dissertation

Format application/pdf (imt)

Rights Kim, Eunkyung

Type texts

Source University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Access Conditions The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the a...

Repository Name University of Southern California Digital Library

Repository Location USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA