Close
About
FAQ
Home
Collections
Login
USC Login
Register
0
Selected
Invert selection
Deselect all
Deselect all
Click here to refresh results
Click here to refresh results
USC
/
Digital Library
/
University of Southern California Dissertations and Theses
/
Automated negotiation with humans
(USC Thesis Other)
Automated negotiation with humans
PDF
Download
Share
Open document
Flip pages
Contact Us
Contact Us
Copy asset link
Request this asset
Transcript (if available)
Content
Automated Negotiation with Humans by Zahra Nazari A Dissertation Presented to the FACULTY OF THE GRADUATE SCHOOL UNIVERSITY OF SOUTHERN CALIFORNIA In Partial Fulllment of the Requirements for the Degree DOCTOR OF PHILOSOPHY (Computer Science) December 2017 Copyright 2018 Zahra Nazari Dedication To my beloved parents, Shahin Azadbakht and Ghodrat Nazari for their unconditional support and encouragement to pursue my dreams. To my dearest siblings: Mahdi, Fatemeh, Zeinab, and Mohammad Amin whom love has always kept my heart warm from thousands of miles away. To my best friend, Zahra Vaseqi, who has always been there for me through my ups and downs, and to my beloved Mohammad Mahdian, for his unconditional love and support. ii Acknowledgements I would like to acknowledge the role of luck in my life. It has provided me with the opportunity to pursue my interests and dreams. I am always grateful for it. I would like to express my gratitude to my supervisor, Dr. Jonathan Gratch, for his kindness and continuous support throughout my Ph.D. program. His supervision makes research an even more enjoyable experience. I would also like to thank Dr. Gale Lucas, for being a great teacher and colleague. I have spent ve beautiful years of my life at the Institute for Creative Technologies at University of Southern California, where I had the pleasure of getting to know and work with many great researchers. Chung-cheng Chiu, Rens Hoegen, Eli Pincus, Christopher Wienberg, Melissa Roemmele and many other friends made my stay at ICT even more enjoyable. I would like to thank them all. Last, but certainly not least, I would like to thank Reihaneh Boghrati for being the best roommate I could ask for; her friendship means the world to me. iii Table of Contents Dedication ii Acknowledgements iii List Of Tables vi List Of Figures vii Abstract ix Chapter 1: Introduction 1 Chapter 2: Preliminaries and Previous Work 5 2.1 Multi-issue Bargaining . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 2.1.1 Preference Weights . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 2.1.2 BATNA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 2.1.3 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 2.2 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 Chapter 3: The Problem: Multi-Agent Negotiation6= Human Negotiation 13 3.1 Agent Behavior on the Antiques Negotiation Task . . . . . . . . . . . . . . . . . . 14 3.2 Human Behavior on the Antiques Negotiation Task . . . . . . . . . . . . . . . . . 16 3.3 Dierences between Agent and Human Negotiations . . . . . . . . . . . . . . . . . 17 Chapter 4: Framework and Guide to Thesis 23 Chapter 5: Opponent Modeling 28 5.1 Interest Modeling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 5.1.1 Accuracy Measures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 5.1.2 Interest Modeling Heuristics . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 5.1.3 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 5.1.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 5.2 Character Modeling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 5.2.1 Honesty Perception . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 5.2.2 Machiavellian Perception . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 5.2.2.1 Data Set . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 5.2.2.2 Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 5.2.2.3 Statistical Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . 40 5.2.2.4 Prediction Models . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 5.2.2.5 Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 5.2.2.6 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 iv 5.2.2.7 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 Chapter 6: In uencing the Opponent 45 6.1 Misrepresentation Game . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 6.1.1 Denition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 6.1.2 Solving the Game . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52 6.1.2.1 Known Opponent's Preferences . . . . . . . . . . . . . . . . . . . . 52 6.1.2.2 Unknown Opponent's Preferences . . . . . . . . . . . . . . . . . . 56 6.1.2.3 Simulation Study . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 6.1.3 User Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62 6.1.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 6.2 Model Validation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70 6.2.1 Measures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71 6.2.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73 Chapter 7: Agent Implementation and Experiments 76 7.1 Framework and Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77 7.1.1 IAGO Platform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77 7.1.2 Agent Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79 7.1.3 Negotiation Task . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81 7.2 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82 7.2.1 Study 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82 7.2.2 Study 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83 7.2.3 Measures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83 7.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84 7.3.1 Study 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84 7.3.2 Study 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87 7.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88 Chapter 8: Conclusion 90 Reference List 94 v List Of Tables 2.1 Example 3-issue bargaining problem . . . . . . . . . . . . . . . . . . . . . . . . . . 6 3.1 Payos for a distributive and integrative version of the Antiques Dealer Negotiation. 14 6.1 Experimental Results ( p 0:05; + p 0:10) . . . . . . . . . . . . . . . . . . . . . 63 7.1 tables show the space of possible utility weights for all issues under negotiation. . . 80 7.2 Preference prole for the considered negotiation task . . . . . . . . . . . . . . . . . 82 vi List Of Figures 2.1 Summary of key bargaining concepts . . . . . . . . . . . . . . . . . . . . . . . . . . 8 3.1 Illustration of two participants engaged in the negotiation task . . . . . . . . . . . 17 3.2 Negotiation outcomes for human and multi-agent settings . . . . . . . . . . . . . . 18 3.3 The distribution of all deals discovered by the human and automated agents. Each circle represents the joint utility of the nal agreement (i.e., <utility of side A, utility of side B>) and the size of the circle represents the relative number of deals that achieved that joint utility. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 3.4 Concession rates: y-axis is the relative amount of total value claimed by each oer by round. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 4.1 A framework for any entity aiming to negotiate with humans. . . . . . . . . . . . . 24 4.2 Automated negotiation agents implement a special case of the IRPC framework . . 25 4.3 Automated negotiation agents implement a special case of the IRPC framework . . 26 4.4 The focus of research and guide to thesis contributions . . . . . . . . . . . . . . . . 27 5.1 Performance of various models based on two measures, Rank Distance of the deals, and Max-regret [Note that the smaller our measures are the better that prediction is] 34 5.2 Boxplots for some of the statistically signicant features: Money, Question, Facial Expressivity, Pitch SD and Honesty . . . . . . . . . . . . . . . . . . . . . . . . . . 40 5.3 Accuracy of chance, unimodal and multimodal classiers . . . . . . . . . . . . . . . 44 6.1 Bargaining without (top) and with (bottom) lying . . . . . . . . . . . . . . . . . . 47 6.2 The space of bargaining solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50 6.3 Relative advantage of lying for all possible 5- issue negotiations . . . . . . . . . . . 56 6.4 Example of preference elicitation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58 vii 6.5 Benet of lying when the opponent's preferences are unknown. Results contrast the returns from two misrepresentation heuristics with being honest . . . . . . . . 62 6.6 An illustration of the \truthful" condition . . . . . . . . . . . . . . . . . . . . . . . 64 6.7 Divergence between the issue-ratio and issue-sentiment models. Liars: (M = 0:32, SD = 0:19), Non-Liars: (M = 0:22, SD = 0:12), t(144) = 3:54, p = 0:001 . . . . . 75 7.1 IAGO's User Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 7.2 The number of oers made by users in each condition . . . . . . . . . . . . . . . . 85 7.3 Self-assessed rating on how satised the users were at the end of negotiation . . . . 86 7.4 How honest (left) and strategic (right) agents were perceived by the user (Study 1) 86 7.5 How strategic agents were perceived by the users (Study 2) . . . . . . . . . . . . . 88 viii Abstract Negotiation is a crucial skill in personal and organizational interactions. In the last two decades, there has been a growing interest to create agents that can autonomously negotiating with other agents. The focus of this thesis, however, is on creating agents that can negotiate with human opponents. Besides improving on articial social intelligence, such agents could be used for the purpose of training or assisting human negotiators. A central challenge is to handle the com- plexity of actual human behavior. When compared with idealized game-theoretic models, human negotiations are far richer, both in terms of the nature of information exchanged and the number of factors that inform their decision-making. We consider a negotiation task that is simple, yet general enough to drive agent-human re- search, and analyze an extensive data set of transcribed human negotiation on such tasks. Based on human behavior in this task, and the previous research on human negotiations, we propose a new framework to structure the design of agents that negotiate with people. We address two main decision problems inspired by this framework: modeling and in uencing the opponent. Three techniques are proposed to model an opponent's preferences and character (e.g. honesty and per- sonality traits) and a misrepresentation technique is then used to in uence the opponent and gain better prot. The proposed techniques are then implemented in automatic web-based agents. We ran a number of negotiations between these agents and humans recruited on Amazon Mechanical Turk. The resulting data show that the agents can perform these strategies successfully when negotiating with human counterparts and give us valuable insight about the behavior of humans when negotiating with an agent. ix Chapter 1 Introduction When a family divides up the household chores, when a customer calls a customer service line, when a divorcing couple divides up their property, or when a seller sells her items in an auction, they are all conducting negotiations. Even though negotiations are so pervasive in our everyday lives, people tend to avoid them due to lack of skill or fear of confrontation. This has contributed to income inequality, political gridlock and social injustice [28]. Therefore, understanding nego- tiations and designing methods for automating them can have important practical and scientic benets. In practical terms, automated negotiators can help teach interpersonal skills [56, 21, 11]. More broadly, the capabilities needed to successfully negotiate can inform the design of a wide range of machines that socially interact with people. From a scientic perspective, the act of cre- ating an automated negotiator can serve to advance theories of human cognition. This can occur through the act of concertizing social theories into working artifacts [79], but also because virtual agents enable a level of experimental control unobtainable in most social science research [8]. The prior literature on automated negotiation mostly centers on agents that are capable of negotiating with other agents but not other humans. However, years of research in psychology and behavioral game theory tells us that humans are very dierent from automated agents (for a review, see [15]). Humans are not rational utility maximizers. They are cooperative and selsh at the same time; they value social norms such as honesty, reciprocity, and politeness; 1 and they make judgment errors. Also, they do not follow rigid instructions when interacting with other humans. In short, the assumptions that allow agents to eectively negotiate with other agents are simply inappropriate when humans are involved. Thus, it should not be a surprise that negotiations among humans unfold very dierently from those of multi-agents. In contrast to multi-agent negotiations that follow a predened protocol and therefore are highly structured, human negotiations tend to be mostly unstructured. For example, humans, unlike typical agents, do not limit their communications to oer exchange [13]. They rely heavily on arbitrary information exchange on various objectives and considerations, through verbal and even non-verbal communication. The vast dierence between the behavior of agents and humans demonstrates the inadequacy of relying on the results in the multi-agent negotiations literature when the negotiating partner is a human, and motivates a new line of research on human-agent negotiation. Even though human-agent negotiation has not been formed as an independent research eld, several attempts were made to create negotiation-involved interactions between humans and agent. These pro- posed agents posses one or more of three fundamental limitations. First, they mostly reduce the communication level into oer exchange, similar to what we see in multi-agent settings; second, they are proposed for specic domains and are not trivial to generalize across domains; and third, a learning phase is needed when confronted with new negotiation tasks. In the next chapter, we rst describe the multi-issue bargaining problem, a formulation of negotiation that has been adopted by the social science, education and the multi-agent communities. Next we survey the existing agents and address their shortcomings when faced with new domains that need real-time decision making without prior information. In this thesis, we make several contributions to research on human-agent negotiations. First, we illustrate important dierences between how humans and machines solve negotiations and that human negotiators violate many of the assumptions that underlie the majority of computational methods (Chapter 3). 2 Second, we introduce a general framework for structuring research on agent-human nego- tiations and highlighting the important problems that any solution must address. This helps organize the space of possible solutions, including traditional automated methods (Chapter 4). A well-known theory introduced by Ury, Brett, and Goldberg [83] proposes that human negotiations are based on three considerations of interests, rights, power. We will loosely adapt this theory as the basis of our framework of the structure of negotiations with humans. This framework high- lights two fundamental challenges that confront human-agent negotiations. The rst challenge is opponent modeling (i.e., inferring the private state of a human opponent through their visible behavior). The second challenge is in uencing the opponent (i.e., how to in uence the opponent's understanding of the agent's private state). We address these challenges in Chapters 5 and 6. Third, we introduce novel methods for inferring opponent models in the context of human- agent negotiations (Chapter 5). We show how to infer the priority an opponent assign to dierent issues involved in the negotiation (Section 5.1); Next, we show how to infer aspects of the op- ponent's character that are relevant for predicting their negotiation decisions. Specically, we propose methods for inferring if an opponent is honest and if they exhibit a Machiavellian per- sonality (Section 5.2). Fourth, we introduce novel methods for in uencing a human opponent's understanding of the agent's private state. Specically, we discuss a game-theoretical formulation of communication, called the misrepresentation game, which captures the problem of how an agent should deceptively shape a human opponent's beliefs in order to maximize its own rewards (Section 6.1). In Sec- tion 6.2, we illustrate that these models have value in predicting and explaining human behavior. Specically, we show that when human negotiators follow the approach outlined in Section 6.1 they outperform their opponents. Finally, building on the techniques in Chapter 5, we show how agents can automatically detect when a negotiator is attempting this deceptive strategy. Finally, we conduct two experiments to study the eectiveness of our opponent in uencing tactics when used in practice against human opponents recruited on Amazon's Mechanical Turk 3 (Chapter 7). In the rst experiment, we show that our agents can use the \xed-pie lie" strategy (a technique introduced in Chapter 6) to gain more value while leaving their opponents happier at the end. In the second experiment, we investigate how far the exploitative agents can push for more gain and examine their eect on people's behavior. 1 1 While I am the primary contributor of the techniques described in this thesis, I developed the ideas in collabo- ration with my advisor, Jonathan Gratch. Additionally, some aspects of the work were developed in collaboration with other students and faculty at USC. The annotation scheme presented in Chapter 3 was developed by Dr. David DeVault. The Opponent In uence techniques in Chapter 6 involved collaboration with Emmanuel Johnson (who contributed ideas on preference elicitation) and Thanh Nguyen (who contributed the mathematical formalization of the misrepresentation game [37]). 4 Chapter 2 Preliminaries and Previous Work Negotiations are dialogues aimed at reaching an agreement between parties when there is a per- ceived divergence of interests, beliefs, or in ways to achieve joint ends [16]. Although this denition is broad, researchers have sought to abstract essential elements of negotiations into more struc- tured formalisms that are suitable for both teaching and scientic inquiry. In this thesis, we focus on one useful and common abstraction known as the multi-issue bargaining task [33], which has become a de facto standard for both teaching and research on negotiation in both the social and computer sciences (e.g., see [60, 65, 85]). Multi-issue bargaining generalizes simpler games developed in game theory, such as the ultima- tum game, and more closely approximates many of the challenges found in real-life negotiations. This task has received so much attention amongst educators and researchers because, with only a small number of mathematical parameters, one can evoke a wide range of psychologically-distinct decision tasks. This has made multi-issue bargaining a central model for teaching and studying a wide range of negotiation concepts. 2.1 Multi-issue Bargaining In its basic form, multi-issue bargaining requires parties (typically 2) to nd agreement over a set of issues. Each issue consists of a set of levels and players must jointly decide on a level for each 5 issue (levels might correspond to the amount of a product one player wishes to buy, or it might represent attributes of a single object, such as the price or the warranty of a car). Each party receives some payo for each possible agreement and each player's payo is usually not known to the other party. The payo is often assumed to be additive (i.e., a player's total payo is the sum of the value obtained for each issue) and presented to players through a payo matrix. For example, Table 2.1 illustrates the two payo matrices for a hypothetical negotiation over items in an antique store. In this case, players must divide up three crates of records, two lamps and one painting, and each party assigns dierent value to these items. Side A Payo Record Crates Lamps Painting Level Value Level Value Level Value 0 $0 0 $0 0 $0 1 $20 1 $10 1 $100 2 $40 2 $20 3 $60 Side B Payo Record Crates Lamps Painting Level Value Level Value Level Value 0 $0 0 $0 0 $0 1 $10 1 $30 1 $0 2 $20 2 $60 3 $30 Table 2.1: Example 3-issue bargaining problem 2.1.1 Preference Weights The weight each party assigns to the issues denes one class of parameters for creating qualitatively dierent classes of negotiation. The payo structure in Table 2.1 denes an integrative (or win- win) negotiation. For example, as player A receives the most value from the painting and records, whereas player B receives the most value from the lamps, and the joint payo is maximized when player B gets all the lamps and player A gets the rest (this is also a Pareto ecient solution). A distributive (or zero-sum) negotiation arises when both parties have con icting preferences. For example, if both parties had the same payo as side A, any gain in value to one side would result in an equal loss to the other side. The painting represents a special type of issue known as a compatible issue as one party does not incur a cost if the other party receives their preferred level. Compatible issues create an opportunity for misrepresentation. Specically, if player B, claims 6 that the painting has value to them, they can oer this `invented' value in exchange for other items they want [65]. 2.1.2 BATNA The second important class of parameters is the Best Alternative to a Negotiated Agreement (BATNA) for each player. This represents how much a party would receive if the negotiation fails. For example, if player A already has a tentative deal with another player that aords him $150, there is no reason to accept a deal worth less than $150 from player B (e.g., 2 records and a painting). The BATNA represents the player's bargaining power, and as with preference weights, these are typically unknown to the other player. If player B's BATNA is only $20, then player A has more potential power in the negotiation, although whether this translates into a better outcome depends on how each party shapes the other party's perceptions and how carefully they attend to the structure of the negotiation. Figure 2.1 summarizes several basic negotiation concepts. The graph shows all 24 possible agreements dened in Table 2.1 in terms of the value each player receives. The Pareto frontier is dened as the set of Pareto ecient agreements, i.e., agreements where it is not possible to improve the payo of one party without harming another. Rational negotiators should not accept any deal below this frontier as inecient solutions can always be improved for one party without harming the other (thus increasing joint value), although unskilled negotiators often fail to discover ecient solutions. The BATNAs dene a zone of agreement. Any deal outside this zone should be rejected by one player as it is below their BATNA, although, again, unskilled negotiators often fail to follow this principle. The fact that the Pareto frontier is strictly convex means there is integrative potential: players can improve on a 50-50 split by understanding each other's preferences and allocating each player their most important issue. Inexpert negotiators often assume negotiations are distributive (a `xed-pie' bias) and fail to realize integrative potential. 7 Figure 2.1: Summary of key bargaining concepts It is important that preference weights and BATNA are typically unknown to the other party and must be estimated through language and/or the pattern of oers. Much of the skill of negotia- tion comes from learning when to reveal truthful information or how to elicit truthful information from the other party. A player that reveals too much information without reciprocation could be exploited, creating a tension between cooperation and competition. However, even when all information is public, players often fail to nd ecient agreements. Preference weights and BATNA dene the basic mathematics of the multi-issue bargaining task. There are several other factors which are irrelevant from the perspective of classical ra- tionality, but varying them can have a profound impact on human decision-making, especially for unskilled negotiators. For example, preferences can be presented as losses or gains. Issues can carry moral signicance [25]. Parties can negotiate for themselves or as representatives of their organization or as part of a team [64]. The amount of information available (e.g., the other player's preferences and/or BATNA) can also be varied. All of these and other factors have been shown to in uence the negotiation process, especially for unskilled negotiators. 8 2.1.3 Applications Multi-issue bargaining allows researchers to create a vast number of psychologically-distinct ne- gotiation problems with only a small number of mathematical parameters and a bit of textual framing. Thus, multi-issue bargaining has proven an especially rich tool for the studying and teaching of human social skills, as well as a tool for advancing articially intelligent agents. For example, in emotion research, bargaining tasks are used to examine how signaled or induced emo- tion shapes joint outcomes [85]. In con ict-resolution research it is used to study various social processes involved in resolving disputes [16]. In social neuroscience, it is used to examine specic brain regions associated with social cognition [7]. In game theory, it is used to advance ratio- nal models of multi-party decision-making [62]. In articial intelligence, it serves as a standard challenge problem for advancing automated models of social decision-making [2]. Finally, this ex- tensive body of research on multi-issue bargaining provides a rm theoretical basis for informing pedagogy in educational settings, as bargaining games are commonly used to teach a wide range of interpersonal skills including negotiation, con ict-resolution, teamwork, emotional intelligence and inter-cultural uency (e.g., see the leadership exercises at the Northwestern Dispute Reso- lution Research Center at negotiationexercises.com). Therefore, virtual agents that can perform multi-issue bargaining tasks with humans in a general way will have broad applications in science and education. 2.2 Related Work Multi-agent negotiation. The majority of research in automated negotiation centers around multi-agent negotiations. Two important characteristics of automated agents (as opposed to hu- mans) are that they follow instructions and they have well-dened goals that they rationally try to achieve. As a result, the main areas of research in the multi-agent automated negotiation literature include negotiation protocols, negotiation objectives, and optimal decision making. Negotiation 9 protocols are protocols that the agents follow to submit oers or other pieces of information to the other party in order to reach an agreement. This is essentially an extension of the mechanism de- sign literature [53, 59]. Negotiation objectives determine the range of objects that are considered in the negotiation, which in multi-issue bargaining translate to the issues under negotiation. De- cision making involves modeling the opponent (i.e., understanding the objective of the opponent and the strategy the opponent is following to reach their objective), and optimizing agent's own objective given the model for the opponent. The outcome of a negotiation is commonly analyzed using game theory [67]. In particular, notions of equilibria in non-cooperative games are often used to model settings where the agents are purely self-interested. A prominent example here is the model of alternating oers introduced by Rubinstein [76] that is analyzed using the no- tions of Nash equilibria and subgame-perfect equilibria. Settings where the agents are not purely self-interested and are willing to cooperate to reach an acceptable outcome are modeled using cooperative game theory. A typical result in cooperative game theory lists a number of desirable properties the solution should satisfy (e.g., fairness or individual rationality), and characterizes solutions that satisfy these properties. A prominent result in this eld is the celebrated Nash bargaining game [62]. See [66] for a survey of the game theory literature on bargaining. Agent-human negotiation. Research on automated agents that negotiate with humans has been in uenced by the multi-agent research literature and has yet to become an independent line of research. We believe that all of the previously proposed agents are limited in one or more of three main aspects. Here, we discuss these limitations and explain how we address each of them in our work. Limited Communication and Restrictive protocols: Most of the the currently proposed agents limit their communication channel to oer exchange, similar to what we see in multi-agent re- search. Therefore, they fail to address one important characteristic of human behavior: that 10 humans talk and express their preferences. Even in their oer channel, these agents use restric- tive protocols such as complete oer exchange, which is far from the natural behavior of human negotiators. Cli Agent [54], Colored trail agent [32] and QOAgent [55] are examples of such attempts. Cli Agent competes repeatedly in one-shot interactions each time against a dierent human opponent and over a single issue, then uses reinforcement learning to t a better model after each interaction. The Colored-trail agent uses collected negotiation data to mimic human behavior in a colored-trail game environment. Despite being a general purpose agent, QOAgent has its limitations in communication channel and opponent modeling. It considers oer exchange as its sole communication channel, and for opponent modeling, QOAgent considers a nite set of opponent types and calculates the likelihood of each opponent type, thus failing to generate an accurate model of the opponent's preference prole. In our research, we consider preference expressions as an indisputable part of human-involved communications, and introduce new mod- els aiming to utilize this channel's rich source of information. We also relax the oer exchange protocols to include partial and uneven oers that resemble human's natural behavior. Specialized Domains: Some of the proposed agents are designed to operate in specialized domains and it is not a trivial task to generalize them across domains. Diplomat [50] was the rst attempt at implementing agent behavior involving negotiation with humans. This agent randomly picks one of its ve implemented personality traits to win the human counterpart in a diplomacy game. Diplomat could only negotiate in a diplomacy game and due to the complex nature of this game, it is nontrivial to use this agent for other domains. Autona [12] is another agent proposed for a specic domain. It performs multiple negotiations between buyers and sellers over price and quantity of a given product. Autona uses history of past interactions to model the opponent and oers proper deals. In our research, we address a large family of domains using multi-issue bargaining task (described in the previous section), which has become the standard formalism for teaching and scientic inquiry in negotiations. 11 Need Oine Training Phase: Most of the agents that do address multi-issue negotiation task need a training phase to train their models of the opponent. For example, KBAgent [68] is essentially QOAgent which is trained oine by a database of past negotiations to improve its models of the opponent types. Using this database, KBAgent estimates the probability of an opponent accepting or rejecting an oer, by calculating the expected average utility of each deal for the other party. Negochat [73] uses several main components of KBAgent and introduces Aspiration Adaptation Theory to improve its performance. Using previous data, Negochat determines which issues are usually addressed rst by assigning a typical urgency to the them. In addition to Aspiration Adaptation theory, Negochat is the rst agent to extend the communication channel by adding a natural language processing module. Negochat-A [74] adds an anchoring component to further improve Negochat. It starts o by an initial oer as an anchor, and then facilitates incremental agreements throughout the negotiation. We believe that the main limitation with the more recent proposed agents is their costly training phase when used in new domains. Actual human negotiation data is rare and expensive to acquire, thus making it even more important to have real time agents that could negotiate in new domains without prior knowledge. Our focus in this thesis is to propose new models that reduce reliance on previous data. We use theories and principles of human behavior to facilitate the usage of such agents in new domains. 12 Chapter 3 The Problem: Multi-Agent Negotiation6= Human Negotiation In this chapter, we perform a comparative analysis of how negotiations unfold in automated agent settings compared to the human-human settings. In doing so, we focus on a multi-issue bargaining task (described in Section 2.1), and use a few of the previously proposed successful agents to perform a series of round-robin tournaments on this task. Next, we bring people to the lab and ask them to perform the same negotiation task. We analyze these two populations, and demonstrate fundamental dierences in how they approach negotiations, both in terms of the outcome and their behavior throughout the negotiation. As discussed in Section 2.1, multi-issue bargaining is the primary formalism used in automated negotiation research, and is not only an important domain in its own right, but also a model that captures many of the essential elements of negotiations in general. For our analysis, we consider a 3-issue bargaining task, in which two antiques dealers must negotiate over how to divide the contents of an abandoned storage locker. The locker contains six antique items: three crates of LP records, two art deco lamps, and one art deco painting. Each negotiator assigns some private value to each item and is unaware of how much their opponent values each item. Due to the important distinction between distributive and integrative negotiations and their signicant eect 13 Task Player Records Lamps Painting BATNA Distributive Player A High(30) Medium (15) Low (5) 30 Player B High (30) Medium (15) Low(0) 30 Integrative Player A High(20) Medium (10) Low (5) 20 Player B Medium(10) High(30) Low(0) 30 Table 3.1: Payos for a distributive and integrative version of the Antiques Dealer Negotiation. on how the negotiation unfolds, we consider two versions for this task that yield qualitatively dierent negotiations. Table 3.1 summarizes the pay-o matrix for the two versions. In the distributive version, the two players have the same preference over the items. They both want the records the most, then the lamps, and then the painting. This means that bargaining has a purely competitive nature: negotiators have to divide a xed pie between them, and one side's gain means the other side's loss. The integrative version, however, is not purely competitive. It also require cooperation between the two sides to arrive at a better solution, thereby expanding the size of the pie. For example, in the integrative payo matrix in Table 3.1, one side wants the records the most and the other side wants the lamps the most, and so giving each side her favorite item is a win-win solution. We next contrast how automated agents and human negotiators approach the introduced multi-issue bargaining problem. 3.1 Agent Behavior on the Antiques Negotiation Task To examine how agents approach this problem, we introduced the antiques negotiation task into the Automated Negotiation Agents Competition (ANAC). ANAC is a yearly competition held in conjunction with the International Conference on Autonomous Agents and Multiagent Systems (AAMAS) and was introduced to spur advances in automated negotiation agent technology. To achieve this goal, ANAC releases all agents and the competition software to the public, mak- ing it possible to compare previous agents with novel opponents and domains. Here, we run a 14 competition involving previous ANAC contestants on the Antiques Dealer Negotiation task and consider their performance. We also examine the performance of agents across multiple years to gain insight into how these systems are evolving. Competitions at ANAC are run using the General Environment for Negotiation with Intelligent multi-purpose Usage Simulation, or Genius [40]. Genius is an open architecture for developing and evaluating the performance of negotiation agents, and allows running round-robin competitions between agents. It also includes all previous contestants of the ANAC competition, organized by year. For the purpose of this comparison, we consider agents form 2011, 2012 and 2013. Agents prior to 2011 were considered obsolete and agents after 2013 focused on aspect of negotiation that are irrelevant to this comparison (2014 focused on learning across negotiations and 2015 and 2016 involved multiparty negotiations). Agents before 2012 did not incorporate a BATNA, but the BATNA for the Antiques Dealer is close to zero so we felt it was appropriate to include agents from 2011. A series of round-robin tournaments were run with all agents that participated in ANAC. Competitions were broken down by year and by negotiation type (integrative or distributive). Using the default settings, agents were allowed three minutes to nd an agreement, which is far more than necessary for such agents but approximates the average negotiation time of human negotiators. We compute the following basic statistics: The oers exchanged measures the average number of oers agents needed to reach an agreement. The impasse rate indicates what percent of negotiations ended in failure (i.e., the negotiators failed to reach an agreement). When agents did reach an agreement, the dierence between what each agent received is a measure of fairness (with zero dierence being truly fair). Thus, the average dierence between oers is reported. Finally, the average social welfare of agreements is reported. For this, we report a measure proposed by Nash [62] (specically, the product of each agent's utility) which captures both the eciency and equity of outcomes. 15 3.2 Human Behavior on the Antiques Negotiation Task To examine how people approach this problem we recruited a large number of human dyads to solve the task. Participants were brought into the lab in pairs, given instructions about the task and their private payos, and allowed to talk privately for a predetermined amount of time to reach an agreement. Participants received one of the two payo combinations illustrated in Table 3.1. To motivate participants to follow this payo matrix, negotiators received tickets for a $100 cash lottery where their chances for winning re ected their skill at negotiating. Participants received a number of lottery tickets corresponding to the value, for them, of the items they obtained. Higher-value items received more lottery tickets. Negotiators had 15 minutes to talk and reach an agreement. If they failed to reach agreement, they received only the number of lottery tickets that they would have received for one of their highest value items (corresponding to their BATNA). All negotiations were recorded and transcribed for analysis. Participants: We recruited 226 participants (113 same-sex dyads; 66% male) from craigslist.com. In addition to the lottery, participants were compensated $30 for completing the study. Of the 226 participants, 8 participants did not comply with the procedure and 28 failed to correctly report their own preferences at the end of the study (implying that they did not understand the task) and were excluded from our analysis, resulting in 190 participants. Participants also provided subjective ratings on how honest their opponent seemed. Annotations: To gain insight into how human negotiators are reaching agreements, we anno- tated the negotiators' speech (these annotations are also used in later chapters of this thesis). All dialogues were manually transcribed and annotated with dialog acts. Oers correspond to state- ments about a division of items. For example, the statement \How about if I take two records and you take both lamps?" is a partial oer. It is partial in two distinct senses as a) fails to mention the painting, and b) it requests two records while leaving the third record unspecied. Preference- assertions correspond to statements about an individual issue such as \I like records" or \I don't 16 Figure 3.1: Illustration of two participants engaged in the negotiation task like paintings." Preference-assertions are also annotated for their truthfulness. For example, if a participant asserts \I like the painting the best" but his payo matrix states that painting is not his most valuable item, this would be annotated as a false statement. False statements do not necessarily mean the participant is lying (they could be confused or have misspoken); however, it was clear that many participants attempted to misrepresent their true preferences. Other dialog acts included statements related to information exchange on various issues, quality of a proposed deal (this-proposal-is-fair, this-proposal-is-unfair), and other negotiation related dialogues (see [26] for a complete description). 3.3 Dierences between Agent and Human Negotiations Dierences in outcomes: Figure 3.2 illustrates the statistics dened in Section 3.1 for the automated negotiation agents (listed by each year of the competition) and contrasts these with human performance. Results are displayed separately for the integrative and distributive versions of the negotiation task. Figure 3.2 summarizes several statistics: the number of oers exchanged on average; the percent of negotiations that failed to reach an agreement (i.e., the impasse rate); the average disparity between oers; and the Nash social welfare (i.e. the product of utility 17 Figure 3.2: Negotiation outcomes for human and multi-agent settings received by each side). The last two measures index the level of fairness and eciency of the negotiations. Several things can be observed from these results. As can be seen in Figure 3.2, agents exchanged far more oers than human participants (which should allow them to better infer their opponent's preferences), yet this did not translate into better outcomes. Agents were less likely than human negotiators to reach an agreement, and this tendency became more pronounced in more recent years of the competition. Negotiated agreements between agents were less fair than human agreements and resulted in less social welfare. Again, this tendency became more pronounced with more modern agents. To give greater insight into these dierences, Figure 3.3 illustrates the distribution of all deals discovered by the human and automated agents. Each circle represents the joint utility of the nal agreement (i.e., <utility of side A, utility of side B>) and the size of the circle represents the relative number of deals that achieved that joint utility (because the issues in the Antiques 18 Figure 3.3: The distribution of all deals discovered by the human and automated agents. Each circle represents the joint utility of the nal agreement (i.e., <utility of side A, utility of side B>) and the size of the circle represents the relative number of deals that achieved that joint utility. Negotiation are discrete, only a small number of distinct deals were possible). The solid line represents the set of valid solutions. Note that the one circle outside this region in the human- integrative negotiations corresponds to a person that received less than their BATNA. Note also that agents in the 2011 competition treated the BATNA as if they received zero utility. Again, Figure 3.3 emphasizes that agent negotiations are less fair and less ecient (as many end in failure). For example, in the distributive negotiations, the 2011 agents settled on a wide range of deals seemingly without regard for fairness concerns whereas human deals are clustered about the 50/50 split. In later years, most agents largely failed to reach an agreement and only received their BATNA. The situation is somewhat better in the integrative negotiations, yet again, many agent negotiations ended without agreement. Essentially, agent and human behavior represent dierent equilibria in how to approach the social dilemma of cooperation versus competition: agents tend to choose multiple defection whereas humans tend towards mutual cooperation. The consequence 19 is that humans achieve greater individual utility through cooperation (as most humans achieve the ecient fair solution, whereas most agents only receive their BATNA). Dierences in process: An analysis of the dialog acts of human negotiators illustrates several dierences in how participants reached these outcomes. We divide these dierences into dierences in protocol, communication, and decision making. Protocol: Most automated negotiation agents follow the Rubinstein alternating-oers pro- tocol, meaning that agents take turns exchanging compete oers, whereas complete oer would be a full partition of items between the participants (e.g., a complete oer would be for side A to ask for all records in exchange for the other items). People diverge from this in several ways: { People prefer partial oers to complete oers. Of the 522 oers in the produced by participants, 370 (70%) were partial. Oers were partial in two ways. Some oers referred to a subset of issues (e.g., attempting to allocate one issue while leaving the other issues unspecied). Some oers were partial within an issue. For example, a participant might oer one record to each side while leaving the remaining record unspecied. { People do not evenly alternate their oers. In fact, we observed many negotiations in which only one party proposed all the oers, and the opponent only accepted or rejected. { People make very few oers. In contrast with automated agents that exchange thou- sands of oers in a typical negotiation, participants exchanged 5.8 oers on average. Communication: Most automated negotiation agents restrict their communication to ex- change of oers. In contrast, oers made up a small percentage of information people exchanged: 20 { On average 40 dialog acts where stated by a negotiator, and only 5.8 of them (14%) where oer-related statements. { People made many explicit statements about their preferences. On average, they used 9.9 statements about their preference during a negotiation (24%), which is almost twice as many as the average number of exchanged oers. Most of this information was truthful and only about 18% were lies. { People also try to in uence their opponent's preferences. For example, they might try to convince their opponent that they should prefer lamps to records. On average, each negotiator asserted 1.7 statements about his opponent's preferences. { People talk about rights and norms (primarily the norm of fairness). On average 1.94 times a statement related to fairness was mentioned in a negotiation. Decision making: Most automated negotiation agents begin with very aggressive oers (e.g., \give me everything") and gradually concede (over thousands of oer exchanges) to a point of mutual agreement. Human participants followed a very dierent strategy: { Participants did not start high and concede over time, as observed in many automated agents' behavior. Rather, they began near the perceived middle. For example, Figure 3.4 shows the average aggressiveness of oers (utility value for self) by round and by the type of negotiation. A value of 1.0 would indicate that the participant made the high-utility oer they could obtain. Instead, they only asked for about half of this on the rst round. { Participants seemed to highly weigh the norm of fairness. Most nal oers were close to a 50-50 split. These behaviors would tend to confuse standard agent algorithms. For example, Multi-agent opponent modeling techniques usually assume that the opponent asks for the highest possible utility and concedes towards the end. Observed human behavior does not conform with this assumption. 21 Figure 3.4: Concession rates: y-axis is the relative amount of total value claimed by each oer by round. In this Chapter, we demonstrated the fundamental dierences in behavior of humans compared to agents in negotiations. These observed dierences motivated us to propose a framework that could address the characteristics of human behavior in negotiations, and structure the research in human-agent negotiation. We introduce this framework in the next chapter. 22 Chapter 4 Framework and Guide to Thesis The dierences in how negotiations unfold between humans compared to automated agents em- phasize the need for designing a new framework that models human natural behavior. In this Section, we present a very general framework, the Interests, Rights, Power and Character (IRPC) framework, for structuring research on human-agent negotiation. Illustrated in Figure 4.1, IRPC captures the main characteristics of human negotiations and helps organize a broad space of po- tential solutions. This framework is used to structure the thesis contributions and as a guide for designing our agents. Negotiations usually start by each party holding private information about their goals and aspirations, and partial or no information about their opponent. Through negotiation, parties communicate what constitutes an acceptable deal for them, try to understand their opponent's goals and aspirations, and strive to reach a mutually acceptable agreement. Researchers of human negotiation often organize such communication into broad categories. In this thesis, we adapt the terminology of Ury, Brett, and Goldberg [83] which classies communicative acts into whether they address interests, rights, or power. Interests typically refer to each party's material goals for the negotiation (e.g., the monetary value each party assigns to dierent issues or deals). Rights refer to the norms and standards that parties believe should be applied to the negotiation (e.g., the solution should be fair; communications should be honest). Finally, power refers to the relative 23 Figure 4.1: A framework for any entity aiming to negotiate with humans. strength of each bargaining position (e.g., how much do they need this deal to succeed or do they have good alternatives in the case of failure). In addition to explicit statements of interests, rights, and power, negotiators also communicate a variety of other sources of information that can impact negotiated outcomes. For example, one's opponent might seem trustworthy or manipulative, condent or fearful, and gregario us or shy. These impressions can be shaped by explicit statements (e.g., \trust me") or inferred indirectly through a variety of channels including non-verbal communication or consistency/inconsistency between statements. We loosely combine these \softer" factors into a fourth class of commu- nicative acts that we call character. Character includes stable individual dierences, such as personality traits, but also more transitory states such as emotion. Thus, in this thesis, we adopt a four-way classication of communicative acts in terms of interests, rights, power, and character. Each party needs to build a model of their opponent based on these communicative acts (shown as Opponent Modeling in Figure 4.1) and tries to shape their opponent's model of themselves (shown as In uencing the Opponent in Figure 4.1). All of this is in service of informing and shaping each party's decision-making. 24 IRPC is a very general framework and dierent agent approaches can be seen as implementing special cases of this framework. For example, Figure 4.2 illustrates the typical architecture of automated negotiation agents. These systems represent a notion of interests (typically a utility function that maps total oers to some point or monetary value) and a very specic notion of power (represented as a BATNA). Although they represent these factors internally, agents are unable to communicate these factors directly. Rather, communication is limited strictly to an exchange of oers. Agents often engage in opponent modeling (inferring the opponent's interests and BATNA by tracking the pattern of concessions made by their opponent [40]. Similarly, the only in uence an agent has over its opponent is through the pattern of oers it generates. Figure 4.2: Automated negotiation agents implement a special case of the IRPC framework IRPC is also a general framework in the sense that each component in Figure 4.1 can be instantiated in a number of ways (see Figure 4.3). For example, research on automated negotiation agents has advanced, primarily, by exploring increasingly complex representations of an agent's interests. Early work focused on simple linear utility functions (where the utility of a deal was a linear combination of the priority assigned to each issue). In contrast, the most recent negotiation 25 competitions focus on nonlinear utility functions [26]. Similarly, most negotiation agents represent power solely through a BATNA, but other forms of power, such as coercive power (ability to administer rewards and punishments), could be incorporated into the negotiation. For example, argumentation-based negotiation systems incorporate communication of threats to maximize their power in a negotiation (see [51]). Human negotiators often emphasize fairness but there are many notions of fairness and parties might bring very dierent notions to the table (see [86]). For example, it is common to assume that an equal split is fair (the equality principle), others feel that the weak deserve a larger slice of the pie (the need principle), and yet others feel the powerful deserve more (the equity principle). Figure 4.3: Automated negotiation agents implement a special case of the IRPC framework The IRPC framework denes a large space of possible research challenges, but in this theses we focused on the subset of challenges illustrated in Figure 4.4, leaving the rest for future research. With regard to Interests, we restricted consideration to situations where interests can be repre- sented by a linear utility function (a common assumption in the human negotiation literature). 26 Figure 4.4: The focus of research and guide to thesis contributions Concerning Rights, we focus exclusively on the notion of fairness, and more specically, adopt the commonly accepted equality principle [86, 62], meaning that equal splits are viewed as fair. The equality principle is most likely to hold in situations where parties have relatively equal power (see [43]), therefore we focus on situations where each party's power (i.e., BATNA) is approximately equal. Therefore, no modeling techniques are proposed for the Rights and Power elements of the IRPC framework. Finally, concerning character, we focus on predicting the human opponent's honesty and Machiavellian personality (Chapter 5.2). Honesty plays a crucial role in the extent to which an agent should rely on its gained information from its counterpart and Machiavellianism represents the counterpart's propensity towards dishonesty and manipulation. 27 Chapter 5 Opponent Modeling Inspired by the IRPC framework (introduced in Chapter 4), we address opponent modeling as the rst challenge for designing automated agents who want to negotiate with humans. Based on this framework, a successful negotiator must take into account the interests, rights, power, and characteristics of its opponent to reach an agreement that is acceptable to both. In this chapter, we propose techniques for inferring the opponent's interests and character. In the context of the multi-issue negotiation task, interests stand for the utility weights that an opponent carries for each of the issues. Three techniques are proposed in Section 5.1 to infer the opponent's interests. The agent should further adjust its model based on the perception of the opponent's character. Here, we propose techniques to infer the opponent's honesty and Machiavellian personalty traits. Honesty perception can help the agent evaluate the extent to which it should rely on the information revealed by the opponent. Machiavellian personality perception can help the agent determine the opponent's tendency in using manipulative strategies to gain advantage. We investigate character modeling in Section 5.2. 5.1 Interest Modeling An important concept in negotiation is the notion of Pareto eciency which is a measure for the quality of a negotiated agreement. A deal is inecient if it is possible to improve the other 28 party's position without harming one's own. Negotiators must oer Pareto ecient deals as this maximizes joint gains and increases the chance of a benecial agreement for oneself. The incomplete information about the opponent's interests makes it dicult for the parties to calculate the eciency of a deal. The solution to this conundrum is to estimate the interests of one's opponent. Having a good estimate of the opponent's interests helps the negotiators make oers that are closer to Pareto optimal and are more likely to be accepted by their opponent. Several algorithms have been proposed to estimate the opponent's utility function (see [4] for a recent review). These models have been proposed for multi-agent settings and therefore, only use the pattern of oers for their estimations. These algorithms fall into two main types: Bayesian models and frequency models. Bayesian models generate a set of candidate preference proles rst, and then use certain assumptions about the opponent's concessions to update their models. One of the main assumptions in this set of models is that the opponent starts with asking for maximum possible utility and then gradually concedes towards lower utilities. Among frequency models, N.A.S.H Frequency [4] learns the issue weights based on how often the best value for each issue is oered. Hardheaded [52] learns the issue weights based on how often the level of an issue changes, hypothesizing that when a party changes the asked level for an issue frequently, they must assign lower utility for that issue. In Chapter 3, we showed that none of these assumptions apply in human negotiations and therefore agents need new methods to build accurate models of the opponent. In this section, we propose three new heuristics to estimate an opponent's interests (Section 5.1.2). But rst, we dene two measures to evaluate the accuracy of opponent models (Section 5.1.1). We will then evaluate the accuracy of our proposed techniques, and compare them to the existing ones in the multi-agent negotiation literature and show that our models perform signicantly better when applied to human negotiators (Section 5.1.3). 29 5.1.1 Accuracy Measures In negotiation research, various metrics have been proposed to assess the quality of an opponent interest modeling approach. In this thesis, we use a standard accuracy measure in negotiation contexts called rank distance of the deals [41] (also known as the Kendall tau rank distance [49]). This metric compares the utility of all possible deals in the outcome space ( ), given the estimated (u 0 op ) and the actual weights (u op ), and calculates the average number of con icts in how deals are ranked using the estimated vs. actual utility function: d r (u op ;u 0 op ) = 1 j j 2 X !2 ;! 0 2 c <u;<u 0(!;! 0 ) (5.1) The function c, the con ict indicator, takes any pair of deals (! and ! 0 ) and returns 1 if the ranking between the deals changes when calculated by the actual vs. estimated weights; otherwise it returns 0. An opponent interest modeling approach is considered to be more accurate if it produces a smaller rank distance than another approach. One limitation of rank distance is that it considers the average number of errors that can arise from the estimated weights, but not the severity of these errors. An alternative strategy, known as minmax regret, is to minimize the worst-case negative consequences that could arise from using the estimated weights. Minmax regret is considered a more conservative and robust procedure for selecting amongst models. To capture this intuition in the opponent interest modeling context, we proposed the following measure. Max-regret nds the maximum absolute dierence between the utility of a deal given actual weights (u op (!)) and its utility given estimated weights (u 0 op (!)) over all possible deals !2 : d M (u op ;u 0 op ) = max !2 ju op (!)u 0 op (!)j (5.2) 30 We assume utility has been normalized to the range [0; 1]. Thus, a max-regret of 0.5 indicates that the estimate may be o by half of its maximum possible value. For example, if our opponent could receive $1000 for his best deal, with a max-regret of 0.5 we might estimate his payout at only $500, leading us to think we are getting a great deal when we could have extracted greater concessions. We will use both rank distance and max-regret to assess the value of dierent opponent interest modeling techniques. 5.1.2 Interest Modeling Heuristics To infer opponent's interests, we propose three simple techniques that could be used to preform an online evaluation of the opponent's utility function: The Issue-ratio heuristic uses the sequence of oers to estimate the preference prole; the Issue-sentiment heuristic derives an estimate from explicit preference assertions; and the Oer/Sentiment heuristic simply uses the mean of these two models to integrate information in both oers and preference assertions. Issue-ratio heuristic: Rather than looking at concession rates, this heuristic examines each issue separately and assumes 1) if an issue is important, the participant will oer a greater percentage of the possible value to themselves, and 2) if an issue is important, they will include it more often in their partial oers. We realize these two intuitions in the following metric. If an issue (i) is discussed in an oer (k), it is assumed to have two parts: which level of that issue was claimed for self (l k ) and which level was assigned to the opponent (l 0 k ). The heuristic estimates each issue weight (w i ) for a participant by comparing the average level claimed for self ( l k ) to the average level oered to the opponent ( l 0 k ) across all oers made by that participant: w i = l k l 0 k (5.3) Issue-Sentiment heuristic: As we demonstrated in Chapter 3, participants make a large num- ber of explicit statements about their preferences (e.g., \I like the records the most"). Although 31 the trustworthiness of these expressions remains unknown, they could be considered as a valuable source of information for our preference models. We proposed a very simple heuristic based merely on counting the number of times a preference is expressed towards an issue. More precisely, every time a positive preference is asserted towards an item (\I like the painting"), we added one to a weight associated with the issue. Every time a negative preference is expressed (\I don't really care for the painting"), we subtracted one from the weight associated with the issue. All weights are normalized to compute a set of weights that are comparable to the weights derived from oers: w i =jP i jjN i j; (5.4) where P i is the set of all positive statements a participant asserted about issue i, and N i is the set of all negative ones. Oer/Sentiment heuristic: As negotiators reveal preferences both through their oers and through explicit preference statements, a heuristic that incorporates both sources of information might perform best. The simplest way to combine these two factors is to simply average the weights that arise from these two estimates. Our Oer/Sentiment heuristic accomplishes just this. If no information is given by one source (e.g., the participant makes oers but does not make any preference assertions), we used just one estimator. If both estimators produce valid weights, we simply took the mean of the two estimates. 5.1.3 Evaluation Here, we contrast the performance of our heuristics against the state-of-the art in opponent modeling techniques. We compare the performance of all heuristics on the human negotiation data set discussed in Chapter 3 and report their accuracy in terms of rank-distance and max- regret. 32 We chose HardHeaded and N.A.S.H. Frequency as representatives of the state-of-the-art in opponent interest modeling. HardHeaded won the 2011 Automated Negotiating Agents Compe- tition and had one of the highest accuracies in modeling opponents (see [4]). N.A.S.H. Frequency did not perform as well in practice but represents the standard Bayesian view on how to model opponents. Hardheaded is designed to solve a more general problem than was faced by our human partici- pants, so we also created a Hardheaded Modied to allow a more direct comparison. In the human negotiation task, participants were given knowledge about the ranking of levels within an issue (e.g., 2 lamps have more value than 1 lamp which has more value than no lamps). HardHeaded estimates this ranking from the pattern of oers. Thus, to create a fair comparison, we created a version of the algorithm with these parameters xed to their true values. In this way, both humans and the algorithms started the negotiation with the same knowledge about the task. Each model was given the sequence of oers and preference assertions that were produced by each participant in the corpus (preference assertions were only used by the Issue-Sentiment model and the hybrid Oer/Sentiment model). For each participant, we calculate the rank distance of the deals, and Max-regret between the estimated weights produced by the model and the true weights provided to the participant. We also include a model that produces random weights as a point of comparison. Results are shown in Figure 5.1. There were signicant dierences between models (Rank distance: F (4; 176) = 85:242, p < 0:001, Max-regret : F (4; 176) = 44:148, p < 0:001). As predicted, the existing models did not fare well on human data and their performance was close to random. Issue-sentiment and issue-ratio heuristics, however, made signicantly better estimations in terms of both rank distance of the deals, and Max-regret measures. The composite Oer/Sentiment heuristic made the best estimation for the participants' preferences. These dier- ences are signicant ((Oer/Sentiment) / issue-ratio, Rank distance: t(1; 179) = 5:006,p< 0:001), Max-regret: t(1; 179) = 3:195, p = 0:002) ((Oer/Sentiment) / Issue-sentiment, Rank distance: t(1; 179) = 7:656, p< 0:001), Max-regret: t(1; 179) = 7:21, p< 0:001). 33 Figure 5.1: Performance of various models based on two measures, Rank Distance of the deals, and Max-regret [Note that the smaller our measures are the better that prediction is] 5.1.4 Discussion Opponent interest modeling is an important skill for an agent who intends to have ecient ne- gotiations with humans or engage with them as a trainer. Despite the wide attention this area of research has received in agent-agent frameworks, negligible work was done for human involved settings. In Chapter 3, we used a corpus of human-human negotiations, to address the main dier- ences between human and agent negotiators. Human negotiators violated many of the assumptions of automated negotiation agents, rendering these existing models useless for our purpose. Hu- man participants made frequent use of partial oers, whereas agents exchanged complete oers; humans started with fair oers, whereas agents assume initial oers will be tough. Most impor- tantly, humans use language to communicate their preferences whereas agents only focus on the pattern of oers. As a result of these dierences, state-of-the-art opponent modeling methods perform close to chance on human data. We proposed a new set of simple methods to estimate opponent preferences in human negotiations. Our rst heuristic used this pattern of behavior to capture people's preferences and showed a signicantly better performance than other existing 34 models. Our second heuristic only used participants' preference assertions and performed as good as the oer heuristic. We used the average between oer based and the assertion based weight estimations as our third model performed the best of all. These heuristics have several potential uses for an agent aiming to negotiate with humans. By having a more accurate estimate of the human's preferences, the agent can make more ecient oers. The agent can also use the heuristic on its own oers to estimate how its language or oers might in uence the human's belief about the agent's own preferences. This can guide dialog strategies and even personality dierences. For example, a cooperative agent could select speech acts that make its own preferences transparent, whereas a Machiavellian agent might misrepresent its own preferences in order to gain strategic advantage. Similarly, discrepancies between the human's words and oers could serve to detect when the human is attempting to exploit the agent. Finally, these heuristics can serve as metrics for quantifying the information people are exchanging in negotiations. This could have potential use in psychological research on emotion to give insight into the negotiation process. It could also have value for pedagogical agents for assessing the performance of student negotiators. 5.2 Character Modeling Among the factors attributed to character, we address honesty and Machiavellianism, as two individual traits with potentially large impact on the negotiation process. An agent that is aware of an opponent who is lying, could use this information to correct its model of the opponent's interests, and attempt to use dierent techniques to counteract such behavior. Among personality dierences, Machiavellianism has received more attention in negotiation studies. Machiavellianism is a personality trait that refers to the tendency to be strategic, opportunistic and goal seeking. Theoretically, people with higher levels of Machiavellianism are expected to utilize better strategies 35 and achieve better outcomes in negotiations. Being aware of this personality trait could help the agent to utilize more eective strategies and correct its assumptions about the opponent. 5.2.1 Honesty Perception The two main channels people use to communicate their interests (oers and expressions) give the potential to our algorithms to detect inconsistencies. In the previous section, we explained how to gain a quantitative model of the opponent's interests using oers (Issue-ratio) and senti- ment expressions (Issue-sentiment). As the next step (on the same data set), we calculated the discrepancy of the weights using the rank distance of the deals between the weights estimated by each of these two models (Oer/Sentiment discrepancy). The Oer/Sentiment discrepancy was signicantly positively correlated with the number of false preference assertions (Lies) made by each participant (r = 0:174;p = 0:03). This could serve as a measure of honesty, and the smaller it is the more honest an opponent is. 5.2.2 Machiavellian Perception Among personality dierences, Machiavellianism has received more attention in negotiation stud- ies. Machiavellianism is a personality trait that refers to the tendency to be strategic, opportunistic and goal seeking. Theoretically, people with higher levels of Machiavellianism are expected to uti- lize better strategies and achieve better outcomes in negotiations. Being aware of this personality trait could help the agent to utilize more eective strategies and correct its assumptions about the opponent. We use textual, visual, acoustic, and behavioral cues to automatically predict High vs. Low Machiavellian personalities. These learned models have good accuracy when compared with other personality-recognition methods, and we provide evidence that the automatically-learned models are consistent with existing literature on this antisocial trait, giving evidence that these results could be used in a broader domain as well. 36 5.2.2.1 Data Set We used the face-to-face data set described in Chapter 3 to train and validate our predictive models. In addition to the annotation schema described in Chapter 3, a questionnaire was used to evaluate Machiavellianism as a personality measure. Measure of Machiavellianism Prior to the negotiation, all participants completed Christie and Geis' 20-item Mach-IV scale [19]. Respondent's rate dierent statements on a scale of 1 (strongly disagree) to 7 (strongly agree). Example questions include items positively associated with Machiavellianism \The best way to handle people is to tell them what they want to hear" and negatively associated with Machiavellianism \Honesty is the best policy." Negative items are reverse coded: a score of seven (i.e., strongly agree) is scored as one. Each participant receives a Mach score that corresponds to the mean of all twenty items (reversing the scores of negative items). Among all participants mean was 2.71 (SD = .41). We used a common procedure of dividing participants into two classes for prediction. A median split was used to divide participants into High Machs and Low Machs. 90 participants fell in each category. 5.2.2.2 Features Textual Features: We used three sets of features as our textual cues, LIWC based categories, the manually annotated Generic-Dialog-Acts, and the oer-ratio. LIWC (Linguistic Inquiry and Word Count) is a well-known approach in psychology towards automatic analysis of verbal behav- ior [69]. It counts the words and puts them into psychologically meaningful categories including social, aective, and cognitive processes. The default LIWC dictionary includes 4,500 words which are used to dene its 80 language categories. LIWC uses a hierarchical categorization of words: Linguistic processes include pronouns such as me, you, us, and them. Psychological Processes 37 Social processes include concepts about social partners such as family, friends or, more generally, humans. Aective processes qualify positive emotions (such as love, and nice) and negative emotions (such as sad and angry). Cognitive processes characterize aspects related to thoughts such as insight, discrepancy and inhibition. Perceptual processes pertaining to the basic senses such as seeing, hearing and feeling. Biological processes are described by words related to body, health, and sexuality. Relativity includes words such as space and time. Personal concerns include issues such as achievements, money, and work. Generic-Dialog-Acts were used as the second set of the textual features. 45 dialog acts fall into this category and were normalized by the total number of dialog acts. The last textual feature, Oer-Ratio compares the number of oers a participant has relative to their opponent throughout the negotiation. It is calculated by dividing an individual's oer count to the number of total oers in the negotiation. Visual Features: Using FACET, a facial expression recognition and analysis tool, seven expres- sions of primary emotions were extracted. FACET provides two sets of classication categories: Positive and Negative, and the basic emotions individually as Joy, Anger, Sadness, Surprise, Fear, Contempt, Disgust and Neutral. Here, we also propose a new feature by adding all the values from the seven primary emotions. We name this feature as facial expressivity and along with 10 other features we were provided with 11 total visual features. Acoustic Features: We used COVAREP (v 1.2.0) , an open source Matlab toolbox that pro- vides an extensive selection of open-source robust and tested speech processing algorithms enabling 38 comparative and cooperative research within the speech community. Following the standard pro- cedure, we extracted the pitch (f0) in addition to six features that are often used to segment voice qualities on breath and tense dimensions, which are considered to be on opposite ends of the voice quality spectrum. Fundamental frequency (Pitch or f0) The fundamental frequency, or pitch was tracked by using residual harmonics, as it works particularly well in conditions with extraneous audio inputs [27]. Normalized Amplitude Quotient (NAQ) The ratio between the maximum of the glottal ow and the minimum of its derivative, after being normalized by the fundamental frequency is given by the NAQ [1]. Quasi-Open Quotient (QOQ) The QOQ is measured by detecting the duration during which the glottal ow is 50% above the minimum ow; it is normalized by the local glottal period [77]. H1-H2 ratio The fundamental frequency relative to the second harmonic is given by the H1- H2 parameters. The H1-H2 ratio is best thought of as a descriptor of the open quotient [39]. Maxima Dispersion Quotient (MDQ) The extent of the dispersion of the maxima derived from the wavelet decomposition is represented by the MDQ parameter [46]. Peak Slope (PS) The Peak Slope is found by decomposing the speech signal into octave bands that are then regressed on the maximum amplitudes. Voice breathiness is measured by the slope coecient [45]. ANN Glottal Open Quotient (OQ) A method that uses spectral features as input to articial neural networks (ANNs) in order learn the mapping from spectral measurements to the time domain OQ values [47]. 39 Figure 5.2: Boxplots for some of the statistically signicant features: Money, Question, Facial Expressivity, Pitch SD and Honesty Behavioral Feature (Honesty Perception): Using the technique proposed in previous sec- tion, we calculated a metric that predicts if an individual is expressing untruthful information about their intentions. Oer/sentiment divergence takes two steps to measure trustworthiness of a negotiator. First, it uses pattern of oers and preference-assertions as two channels to build a preference model based on each. Then it calculates the diversion between the two learned models using rank distance of the deals. The rst model estimates a set of weights for the issues using positive and negative assertions stated by a participant and the second model learns the issue weights using an individual's pattern of oers. In other words, oer/sentiment divergence uses the divergence between what people utter they like and what they show they like in their pattern oers as a measure of trustworthiness. An advantage of this technique is that it does not require the ground truth information for a participant's real preferences to evaluate their trustworthiness. 5.2.2.3 Statistical Analysis An independent-samplest-test was performed to explore the most predictive features. Results are presented in the following. Among the LIWC categories, High Machs talked less about money (M = 0:54, SD = 0:42) compared to Low Machs (M = 0:70,SD = 0:6),t(90) =1:973 ,p = 0:05. High Machs used less 40 future tense (e.g., \will", \gonna") (M = 0:81, SD = 0:42) compared to Low Machs (M = 0:98, SD = 0:52), t(90) =2:387, p = 0:02. High Machs used somewhat less third person plural (e.g., \they", \their", \they'd") (M = 0:15, SD = 0:24) than Low Machs (M = 0:23, SD = 0:32), t(90) =1:848, p = 0:66, High Machs also used less oers compared to their partner (M = 0:44, SD = 0:28) versus Low Machs (M = 0:55, SD = 0:28), t(90) =2:38, p = 0:018. Examining the dialog acts, High Machs asked signicantly more questions (M = 0:08, SD = 0:06) than Low Machs (M = 0:07,SD = 0:05),t(90) = 1:94,p = 0:053. For visual features, High Machs showed a higher facial expressivity (M = 1:65, SD = 0:35) than Low Machs (M = 1:53, SD = 0:34), t(90) = 2:20, p = 0:029. Fear intensity was also larger for High Machs (M = 0:017, SD = 0:03) compared to Low Machs (M = 0:00, SD = 0:01), t(90) = 1:95, p = 0:052. One acoustic feature that was marginally signicant between High and Low Machs was pitch (f0)'s standard deviation. High Machs: (M = 24:87, SD = 10:89), Low Machs: (M = 22:20, SD = 9:58), t(90) = 1:74, p = 0:08. Among behavioral features, misrepresentation was higher for High Machs (M = 0:15,SD = 0:09) compared to Low Machs (Mean = 0:12,SD = 0:08),t(90) = 2:27, p = 0:024 and High Machs gained a higher outcome utility (M = 62:63, SD = 12:90) than Low Machs (M = 57:16, SD = 15:73), t(90) = 2:55, p = 0:012. Six of the statistically signicant features are shown in Figure 5.2. 5.2.2.4 Prediction Models The six classication models that are used for automatic prediction of High vs. Low Machiavellians are: Chance Baseline Since median split of all Mach scores was used to divide the population into High vs. Low Machs, assigning everyone to one group resulted in 50% accuracy in classication. Unimodal classiers (Textual, Visual, Acoustic, or Behavioral) We used a Support Vec- tor machine (SVM) classier for each of the modalities to evaluate their performance individually. 41 Textual + Visual + Acoustic+ Behavioral To fuse information from dierent modalities, we used an early fusion scheme by stacking the features from all modalities as an input to a new SVM classier. 5.2.2.5 Methodology We used one-person-leave-out test scheme for all of our classiers to assure generalization across participants. Using LibSVM [46] package we trained and validated each classier by performing 5 fold cross validation to nd the optimal C for the third order polynomial kernel SVM on the training set. Due to the large number of textual features, we used an automatic feature selection to help with interpretation and performance. Our original set of textual features were 126 features, and using t-test we selected the features with a p-value threshold of 0.1, where the two populations correspond to High vs Low Machs. This feature selection resulted in 20 textual features along with the 10 visual, 14 acoustic, and 2 behavioral features providing us with a total of 46 features for the multimodal classier. 5.2.2.6 Results The classication results are shown in Figure 5.3. Accuracy presents the percent of individuals correctly classied as High vs. Low Machs performed by each classier. Chance baseline shows 50% accuracy, and textual, visual, acoustic, and behavioral unimodal classiers resulted in 56%, 56%, 54%, and 55% accuracy respectively. The model stacking all the features from the four modalities, resulted in the highest accuracy (70%) and was signicantly better than any of the modalities alone (p< 0:001). 42 5.2.2.7 Discussion In this section, we discuss our ndings and compare them with the existing literature on Machi- avellianism. From our feature set, High Machs used fewer words from the LIWC's money category, asked more questions and proposed less oers. We interpret this as Machiavellians' strategic be- havior by revealing less information about their intentions through statements or pattern of oers. By asking more questions, instead they try to nd out more about their opponents' preferences. This nding is consistent with the literature on Machiavellianism that suggests they may use cog- nitive strategies, including hiding or misrepresenting their true intentions, and looking to exploit others by coaxing opponents to reveal information [30, 42]. The misrepresentation measure introduced in the previous section was signicantly larger for High Machs, suggesting that untruthful or vague information was communicated by Machiavel- lians. This is consistent with the studies that show Machiavellians are more comfortable using deception to advance their goals. Expecting this deceptive behavior, we could interpret our nd- ing in acoustic features too; deceptive speech is known to contain higher pitch variance [72]. In our study, we observed a higher pitch variance for the High Machs throughout the negotiation. We also replicated the ndings that Machiavellians use more words from the anger and negative emotion categories [80], given our nding of marginally higher usage of anger and negative emo- tions words as indexed by LIWC among higher Machiavellians. While previous research found less use of the word \we" among Machiavellians, we did not observe any eects for the word \we"but did nd the word \they" was used signicantly less often. While previous researchers interpreted decreased use of the word \we" as evidence that Machiavellians less often referenced others, we believe that less frequent use of the word \they" could be an even better indicator for Machiavel- lians not directly referring to others. In terms of visual features, facial expressivity was higher for Machiavellians. We interpret this as consistent with the literature that Machiavellians use 43 Figure 5.3: Accuracy of chance, unimodal and multimodal classiers emotion displays to gain advantage, such as feigning emotion for strategic gain. Such emotional displays may allow the machine annotation to capture more facial expressivity. Lastly, in terms of the negotiation outcomes, we observed High Machs winning more lottery values. This again is consistent with prior studies reporting Machiavellians outperforming their opponents in most short term interactions [88]. 44 Chapter 6 In uencing the Opponent In the previous chapter, we addressed opponent modeling as the rst challenge for automated agents when confronted with human negotiators. Next important challenge for these agents is making decisions on what information they aim to communicate to the opponent. A selsh agent could choose to communicate a sort of information that serves to its own benet. In this chapter, we introduce misrepresentation games as a way to formulate such intentions in order to gain strategic advantage. After introducing how misrepresentation game works in Section 6.1, we validate the eectiveness of this strategy when used by humans against each other (Section 6.2). 6.1 Misrepresentation Game Here, we describes a rational analysis of misrepresentation in negotiations. Our analysis diers from the classical game-theoretic analysis of communication between rational negotiators (such as [17, 23, 90]) and from behavioral game-theoretic analysis of how human negotiators behave towards each other (see [14]). Rather, we examine how a rational self-interested actor should act towards a typical human negotiator. Such an actor must reason about, and sometimes act in accordance with human social conventions, without being bound by them. Unfortunately, some human actors behave in this fashion. Indeed, this is the denition of the Machiavellian personality [34]. 45 The literature on human negotiation has documented the myriad ways negotiators can lie. One approach is to feign a position of power. For example, when negotiating for a car, people often lie about their alternative oers (\I got a much better oer from the last dealer" or \this car is so popular we can't give discounts"). This can be an eective strategy, but often leaves the other party feeling abused and can undermine the possibility of getting deals from this party in the future [24]. A more eective approach would be to lie to one's opponent in such a way that they feel like they got a fair deal. O'Connor and Carnevale [65] present a simple but surprisingly eective technique for doing just that. They point out that negotiations often involve issues where both parties' interests align but they often believe they are in opposition. For example, consider a negotiation where both parties want to divide a basket of fruit (see Figure 6.1). Parties might assume that each side has the same interests, where in fact one side wants all fruit equally, but the other side only wants apples. The apples are referred to as a common-value issue because both parties' preferences are aligned. The presence of common-value issues creates the opportunity for an eective negotiation tactic. If one party realizes an issue is of common value before the other, they can feign opposed interest. This allows them to \claim value" over the undesired issue and trade this value o against something they want. For example, in the bottom half of Figure 6.1, the woman has misrepresented her interests in such a way that she wins the negotiation yet the man believes he received a fair deal. In this chapter, we present an analysis of how people (or agents interacting with people) might optimally lie. By this, we consider how they might misrepresent their own interests in such a way to maximally benet themselves while creating an illusion of fairness towards the other party. Our analysis builds on a game theoretic perspective and presents a simple game { the misrepresentation game { that formalizes this notion. We also build on techniques from the preference-elicitation literature to show how people (or agents interacting with people) can eciently discover their opponent's interests without revealing much about themselves. 46 Figure 6.1: Bargaining without (top) and with (bottom) lying 47 Although we build on rational techniques like game theory, our goal is not to nd a rational solution to the misrepresentation game in the classical sense, but rather to identify strategies that a purely self-interested actor could use against typical human opponents. This analysis is in service of several more fundamental goals. First, we seek better understanding of communica- tion processes in human negotiation and how communication can be misused for malicious ends. Second, we aim to produce general models of negotiation that can guide behavior of agents that negotiate with people. In that people lie, it is important for agents to understand, detect and potentially defend against such malicious strategies. Finally, we seek to use these models to teach negotiation skills. For example, by practicing with malicious agents, human students can better prepare themselves to deal with malicious behavior that, unfortunately far too often, arises in the real world (e.g., see [36]). We start by dening the misrepresentation game in Section 6.1.1. We then give a solution for this problem, rst in the simplied version of the game wherein the preferences of the opponent are assumed to be known in advance (Section 6.1.2.1) and then in the case where the opponent's preferences are unknown in advance and must be inferred through information exchange (Sec- tion 6.1.2.2). Finally, we present a user study that demonstrates that people are more satised with a deal from a lying agent than the same deal from an honest one. We conclude with a discussion of limitations and extensions to the current approach. 6.1.1 Denition In this section, we dene the misrepresentation game in the context of a multi-issue bargaining task, as dened and explained in Chapters 2 and 3. Recall that in a multi-issue bargaining task, the two parties bargain overn issues. We focus on the canonical case where each issue corresponds to one resource to be split between the two parties. There areL i levels for issuei, corresponding to how much of this resource we have available. We assume the parties have linear utility functions, 48 i.e., if they receive x i on issue i, and their preference for this issue is a i , their overall utility will be P i a i x i . Our denition is motivated by the following observations about the typical behavior of human negotiators. Human negotiation behavior. In previous chapters we demonstrated important and system- atic dierences between rational predictions and actual human behavior. In particular, people rarely act out of pure self-interest. Rather, they exhibit \other-regarding" preferences (such as the desire for fairness and reciprocity). Further, they often feel bound by social and conver- sational conventions, such as honesty. Finally, people have limited cognitive abilities and have diculty, in particular, with the recursive theory-of-mind reasoning required by game theoretic solutions. In solving the misrepresentation game, we assume human negotiators will exhibit the following empirically-supported characteristics (we consider ways to relax these assumptions in the conclusion): Fairness: Following Rawls [70], we assume negotiators aim for an equitable solution. There have been many attempts to formalize principles of fairness [86]. Here, we adopt Kalai's principle of max-min fairness [44] (i.e., negotiators will strive to minimize the dierence in payout between the parties in a negotiation). There is good empirical support that human negotiators tend towards equal distributions across a wide variety of contexts [31, 75]. Eciency: Following Nash [62], we assume negotiators prefer deals that are Pareto ecient. This means that if a deal can be improved for one player without harming the other, the improvement will be preferred by both players. For example, Figure 6.2 shows the space of all possible deals for the negotiation illustrated in Figure 6.1. The x-axis represents the value of deals that the man could receive, whereas the y-axis represents the value of deals for the woman. The Pareto frontier represents the set of ecient solutions and any deal below this can be ignored. Note, however, that negotiators can only calculated the frontier if they share their preferences, but in 49 Figure 6.2: The space of bargaining solutions most negotiations, this information is unknown to one or both negotiators. This leads to our next assumption. The cooperative principle of language: Following Grice [38], we adopt the perspective that to communicate at all, negotiators must adhere to communicative conventions. Grice argues that speakers will exchange honest, relevant, clear and sucient information to complete the task at hand. Clearly, some negotiators are tempted to violate these maxims. In particular, negotiators are reluctant to reveal their true preferences (violating Grice's maxim of quantity) as this makes negotiators vulnerable to exploitation. Yet to nd ecient solutions, negotiators must exchange honest information. One solution to this dilemma is the principle of reciprocal information ex- change: reveal information if and only if the other party reciprocates [89]. By engaging in such reciprocity, parties develop a mutual understanding of each other's preferences and can discover Pareto ecient solutions [82]. Thus, we assume that parties are honest, relevant and clear, but will only reveal preference information if comparable information is oered in exchange. 50 Fixed-pie bias: Finally, following Bazerman [6], we assume that cognitive limitations lead to departures from rational predictions. In particular, in what is known as the xed-pie bias, nego- tiators often assume that the other party's interests are completely opposed to their own. In the absence of information exchange, this bias explains why negotiators often fail to discover ecient solutions [82]. It also implies they will be predisposed to believe lies that are consistent with this bias [65]. We assume human players hold this bias. From the discussion above, most human negotiators will strive for fair and ecient solutions, and will discover these to the extent they engage in reciprocal information exchange. However, this creates the opportunity for malicious negotiators to misrepresent their preferences for strategic gain. For example, in the bottom of Figure 6.1, the woman is lying about her true preferences (by saying she likes all fruit equally when she truly prefers apples). Assuming the male negotiator takes this lie at face value, he would perceive a dierent (and incorrect) Pareto frontier (illustrated by the dashed diagonal line in Figure 6.2) wherein the deal giving away all three apples appears fair and ecient. Given these preliminaries, we dene the misrepresentation game as the problem of: gaining an information advantage by learning the opponent's preferences without revealing one's own preferences, and identifying a false set of preferences (to be communicated to the opponent) and a proposed agreement that (a) maximize one's own payo in the proposed agreement, (b) subject to the constraint that the other party believes the proposed agreement to be fair and ecient with respect to the communicated preferences. For the remainder of this section we will formalize and analyze the properties of a special case of this game, subject to certain assumptions about the form of each player's preferences. 51 6.1.2 Solving the Game In this section, we present a simple and ecient solution to the misrepresentation game. In Section 6.1.2.1, we present a solution when the opponent's preferences are known in advance. Section 6.1.2.2 extends this solution the case where the opponent's preferences are unknown. Finally, Section 6.1.2.3 presents a simulation study that assesses the eectiveness of this solution. As explained in the next section, we will make an additional assumption about the form of players' utility functions (the ranking assumption). Although our solution to the misrepresentation game is well-dened and reasonable even without this assumption, this assumption will simplify the problem of preference elicitation (in the case of unknown opponent preferences) and also our evaluation of the solution. Furthermore, as Gratch and Nguyen [37] showed, this assumption makes it possible to theoretically analyze the solution and prove its optimality. 6.1.2.1 Known Opponent's Preferences The misrepresentation game can be greatly simplied if we know our other party's preferences in advance of the negotiation. This might occur if the other party rst truthfully reveals their preferences, or the liar has done sucient research to accurately estimate them. When the oppo- nent's preferences are known, the game can be cast as an optimization problem: for all possible preference weights on each issue, identify the set of weights that maximizes the value to self while seeming fair. The misrepresentation game requires the deceiver to claim a false set of preferences such that a personally-advantageous solution appears, to the opponent, to be a fair and ecient solution. We formalize the problem as follows. Let a i be the deceiver's true preferences for each issue i, b i the true preference of the opponent, andx i andy i represent the agreed upon levels for each player (e.g., salary or number of apples). The objective is to nd the optimal \false" preference a i that maximizes the deceiver's utility, given that the negotiated solution appears to be ecient and fair 52 if a i is believed. This can be formalized as an optimization problem (under the assumption that issues are independent) as: max a n X i=1 a i x i (6.1) where x i is the optimal solution of max x;y n X i=1 a i x i (6.2) such that: n X i=1 a i x i n X i=1 b i y i (6.3) 8i :x i +y i L i (6.4) 8i : 0x i ;y i L i (6.5) where L i is the total number of levels (in the discrete case) or amount of the resource (in the continuous case). Equations (6.2) to (6.5) attempt to nd the optimal ecient solution, which satises the fairness constraint (6.3) and the common sense axiom that an agreement deal should be worth more than the status quo for both players. Note that (6.3) represents the right element in the introduced framework in chapter 4, which we instantiated with equal values. This formu- lation allows discrete levels (where equal splits may not be possible). Although not obvious, this optimization problem can be reformulated and solved via mixed integer-linear programming [48]. Ranking assumption: Note that in our formulation, we already restricted utilities to be linear, i.e., the utility of a party is the sum of the utilities she obtains from dierent issues (resources), and her utility on an issue is the product of how much of this resource she gets (x i or y i in the above formulation) and a quantity that describes how important this issue is for her (a i , a i , or b i in the above formulation). Utility expressions in Equations (6.1) and (6.3) are based on this utility model. Here, we further restrict the players to holding a qualitative ranking over the set 53 of issues, i.e.,fa i g,f a i g, andfb i g in the above formulation are assumed to be permutations off1;:::;ng. This restriction is not uncommon in multi-issue problems as players often have diculty articulating their preferences as a numeric utility function. Rather, people are more comfortable communicating qualitatively their ordinal preference over the issues (e.g., they are good at saying they like A more than B but poor at quantifying how much they prefer A to B). These assumptions dene a standard utility function known as the Borda Score [5] often used by AI preference-elicitation techniques (e.g., see [18]). The ranking assumption simplies the problem of nding the optimal lie. It also simplies the problem, introduced in the next subsection, of eliciting preferences from the opponent (as many common preference elicitation techniques use the Borda Score). A consequence, however, is it precludes utility functions where preferences are non-linear or interdependent. It also limits the type of lies a negotiator can make. They can lie about the relative ranking of their preferences but not the absolute value of dierent issues. We return to these limitations in Section 6.1.4. The optimization problem could be generalized. Constraints (6.1) and (6.3) could be relaxed, but may preclude ecient solutions. Constraint (6.3) could be replaced with other fairness criteria, such as Nash's bargaining solution [62]. The ranking assumption could also be relaxed but will require additional constraints on lying to preclude trivial or degenerate solutions. The Fixed-Pie Lie: The xed-pie-lie heuristic is to feign a preference structure identical to one's opponent. That is, a i :=b i in the formulation (6.1){(6.5) above. We propose this heuristic as a solution to the misrepresentation game, partly motivated by the xed-pie bias introduced in Section 6.1.1, and partly by the fact that this is the optimal solution of the misrepresentation game in many cases. Our formulation of the misrepresentation game consists of two levels of optimization which is not straightforward to analyze theoretically. However, using the ranking assumption and the 54 additional assumption that all levels are equal (i.e., L i =L j for all i;j), Nguyen proved that the xed-pie-lie heuristic is indeed optimal (see [37]). The psychological advantage of the xed-pie lie heuristic is that aligns nicely with the xed- pie bias. According to this bias, people often come to a negotiation assuming their interests are opposed and that they will only be able to achieve half of their value (see [81]). In Figure 6.1, the total \pie" appears to be worth $30 to the man on the left, thus it is plausible to believe that $15 is a fair deal. In fact, as parties have dierent preferences, the fair deal is $20. With the xed-pie heuristic, a malicious opponent recognizes the pie can be grown but keeps this additional value for themselves. Lying in this way can aord considerable benet and comes at no cost (at least when the opponent's preferences are known). This is illustrated in Figure 6.3. This gure contrasts the value obtained by lying with the value obtained by being honest for a large set of possible negotiation problems. Specically, given n = 5 issues, each player could have one of n! possible rankings (i.e., all permutations of the n issues). As each player could have a dierent ranking, this creates a space of (n!) 2 possible joint rankings. A negotiation problem must also specify the number of levels per issue. Figure 6.3 illustrates all 14,400 possible combinations of player preferences for two extreme cases: (1) continuous { where each issue can be continuously partitioned; and (2) 2-level discrete { wherein each issue consists of a single indivisible quantity that must be allocated to one party or the other. In each case, L i equals one for all levels i. Figure 6.3 shows it never hurts to lie. In the continuous case (the blue circles), lying always oers improvement except when the negotiation truly has a xed-pie structure (in this case, the solution is to tell the truth, which aords each party 7.5). In the discrete case (red circles), lying sometimes aords no benet over truth, but also doesn't hurt. As the number of ways to partition each issue increases, these two sets of solutions will converge. It should also be noted that the xed-pie lie is especially nasty to the opponent. At least for the continuous case, the opponent always receives half of the diminished \fake pie." For the 55 Figure 6.3: Relative advantage of lying for all possible 5- issue negotiations 5-issue problem, this corresponds to 7.5 (the opponent can sometimes do better than this in the discrete case, but always no better than the liar). 6.1.2.2 Unknown Opponent's Preferences The previous analysis suggests the xed-pie-lie heuristic is optimal when the opponent's pref- erences are known. Unfortunately, negotiators do not have perfect access to their opponent's preferences but must infer them through an exchange of information. In agent-agent negotia- tions, opponent preferences (if they are considered at all) are typically inferred by the exchanging oers and counter-proposals [3]. In human negotiations, preferences are most commonly inferred from explicit statements of preferences [63, 82]. We present a solution to this problem, subject to assumptions about how the opponent communicates. Unlike the perfect-information game, our solution is heuristic and we make no claims of optimality. First, as discussed in Section 6.1.1, the cooperative principle of language suggests that hu- man negotiators feel bound by social norms of honesty [38] and engage in reciprocal information 56 exchange [9] (i.e., if preference information is provided by one party, the other party should recip- rocate). This implies that a negotiator cannot elicit their opponent's preferences without giving up some information in return. Of course, an agent can give up fake information, but it is unclear how to lie without having perfect information about the opponent. Further, premature lies can limit the liar's options. For example, an agent might pretend they like apples best (when truth- fully they like them least), but if they subsequently learn the opponent likes them least as well, they are trapped in their own lie: they can either agree to accept the undesired apples or admit to the lie. Our solution relies on the assumption that people follow the cooperative principles of language and liars cannot admit to lies. Second, as discussed in Section 6.1.2.1, people have diculty expressing numeric preferences but they nd it easy to make pairwise comparisons of the relative importance of dierent issues. Thus, we assume negotiators can only ask or assert relative pairwise comparison statements (e.g., \Do you like apples better than oranges?"). Obviously, more complex preference statements can be constructed from sequences of these primitives. Finally, following [17], we analyze this problem as a \take it or leave it" bargaining game where parties rst engage in several rounds of information exchange and then one party (the agent) makes a single oer which the other party (the human) must accept or reject. We assume the human opponent will accept this oer if and only if it appears fair under the possible models they have of the agent's preferences. Admittedly, this assumption imposes strong constraints on the liar's behavior: they are forced to make an initial oer and then stick to this oer at all costs. This may result in sub-optimal solutions but greatly simplies decision-making. From these assumptions, it is clear that misrepresenting agents should (a) ask questions about their opponent's preferences that gain the maximal information while (b) reciprocating by oering information that allows the most freedom to misrepresent and then (c) make an oer when the costs of (b) exceed the benets of (a). 57 Figure 6.4: Example of preference elicitation Eliciting preferences: The rst problem (a) is to determine which questions to ask, and in what sequence, to quickly and eciently minimize uncertainty about the opponent's ranking. This problem has already been addressed in the preference-elicitation literature. Here we adopt the solution introduced by Lu and Boutilier [57]. We illustrate this algorithm with the aid of Figure 6.4. This example imagines a 3-issue negotiation over pears, bananas and apples. Each negotiator could hold one of 3! Possible rankings. The deceptive negotiator holds the ranking at the top (pears > bananas > apples). The other negotiator reveals that they prefer apples to bananas. This information rules out some possible rankings, but three are consistent with this statement. The gure also illustrates the optimal \fair" oer for each of these possible rankings. For example, if the opponent truly preferred apples > bananas> pears, then the liar should claim the same, oer one apple to their opponent, and keep the rest. Under the Borda-count utility function, this appears to aord each agent a utility of 3 (3 for apples versus 2 for bananas plus 1 for pears) but, in truth, gains the liar 5 (3 for pears and 2 for bananas. Note that the same oer is optimal for the second ranking but a very dierent oer holds for the third ranking. 58 Lu and Boutilier use the principle of minimax regret to guide preference elicitation. With incomplete information, a lying negotiator must make their best guess concerning their opponent's preference. Regret is the loss (or regret) associated with guessing wrong. In Figure 6.4, the liar might propose oering a single apple as this yields the greatest expected return across all possible rankings. However, if the true ranking is ranking 3, this oer will be rejected (as it would appear unfair to the recipient). Lu and Boutilier propose a method of asking questions that minimizes the maximum possible regret. We refer the reader to their paper [57] for details. Under reciprocal information exchange, a misrepresenting agent faces a dilemma. The more information it asks, the more information it must provide (possibly constraining their freedom to misrepresent). Fortunately, the elicitation approach is incremental. At the start of a negotiation, an agent has no information about the opponent's preferences and all preference rankings are possible. This set reduces following each question. The algorithm repeatedly queries the opponent to determine their preference until regret falls below a certain threshold. In order to nd the opponent's exact preference, this algorithm would have to be repeated until only one ranking is consistent. However, the algorithm can be terminated early. At this point, the opponent might have one of a number of preferences but the regret at picking one of these arbitrarily is guaranteed to fall below this threshold (assuming the opponent responded honestly). If the elicitation algorithm is allowed to run to completion, the opponent's preferences are completely known and the agent can generate an oer as outlined in Section 6.1.2.1. If the preferences are incompletely determined, the agent can simply make an oer with the highest expected return over all possible consistent preferences. Oering preferences: Assuming an agent must oer some information in return after each preference that it elicits, a misrepresenting should oer information that gives their opponent the least strategic advantage. In general, we should consider that the opponent might use so- phisticated elicitation strategies as well, but in this thesis, we adopt a further simplication: the 59 misrepresenting agent simply oers a preference statement of its own choosing after asking a pref- erence question (e.g., Agent: \Do you like apples better than oranges?"; Human: \Yes?"; Agent: \I like apples better than oranges as well."). This minimally satises the principle of reciprocal information exchange while simplifying analysis (we return to this in the nal section). Given these further restrictions, we propose two possible misrepresentation approaches and, in the next section, contrast their eectiveness with an honest control condition: One misrepresentation approach (max-max regret) is essentially the dual of min-max regret. The idea is to oer information, at each round, that maximizes the possible regret of the opponent. For example, if the opponent asks a very informative question (\Do you like A>B?") the agent could respond with a dierent, less informative response (\Well, I like C >D"). A second misrepresentation approach is motivated by the xed-pie-lie heuristic. Namely, at each round, if the opponent acknowledges that A>B, the agent should simply respond in kind (\A>B for me too") Finally, an honest agent would oer preference statements that are maximally helpful to the opponent (i.e., that minimize the opponent's max regret). 6.1.2.3 Simulation Study The previous subsection identied two possible heuristics for solving the misrepresentation prob- lem while satisfying the constraint of reciprocal information exchange. Both approaches use min-max regret to elicit questions from the opponent but dier in how they oer information in return (max-max regret or xed-pie lie). Here we run a simulation experiment to contrast these approaches. In that there is a potential tradeo between the cost and benets of exchanging information, we empirically explore this tradeo. A misrepresenting agent could simply keep asking questions 60 until the opponent's preferences are fully known. This yields more information about the opponent but forces the agent to reveal its own information in return (possibly limiting opportunities to misrepresent). Alternatively, an agent could terminate questions early and make its best guess at a good oer. We compared max-max regret and xed-pie-lie against an agent that was honest and maxi- mally helpful. We compared these heuristics on a 5-issue negotiation task where there is a good opportunity to misrepresent: parties' preferences are complementary (A > B > C > D > E versus A < B < C < D < E). For each approach (max-max regret, xed-pie-lie and honest), we ran a simulated negotiation of these strategies against a truthful opponent. This opponent only accepted oers that appeared fair given what the agent revealed about its preferences. We varied how many rounds the agent could ask questions before making an oer from 1 to 10 (10 rounds of questions are suciently to completely determine the opponents preferences for a 5-issue negotiation). The results are shown in Figure 6.5. They illustrate the complex relationship between the costs and benets of asking questions. Both max-max regret and xed-pie lie led to greater returns for small numbers of questions. Fixed-pie lie performed best when the opponent's preferences were fully revealed. At some points, honesty appears to be the best policy. One might expect max-max regret to out-perform xed-pie-lie because it is doing a more sys- tematic exploration of the space of possible preference models and picking the one that maximizes maximum expected value. The problem is that, at early points in the elicitation phase there is insucient information to determine the value of dierent lies, so the method can commit itself to statements that constrain its options to lie in later rounds. Overall, the xed-pie-lie heuristic is simple and works surprisingly well. 61 Figure 6.5: Benet of lying when the opponent's preferences are unknown. Results contrast the returns from two misrepresentation heuristics with being honest 6.1.3 User Evaluation Up to this point, we have made theoretical arguments and evaluated methods against simulated human opponents. But would actual human negotiators be fooled by such simple techniques? Here we report the results of a study that answers this question. Specically, we test two hypotheses: H1 (perceived fairness of outcome): We hypothesize that people would be more willing to accept an oer (H1a), nd it more fair (H1b) and satisfying (H1c), and place greater trust in their opponent (H1d) if the opponent misrepresents their preferences as a xed pie, compared with an oer of equal value where the true preferences were honestly conveyed. H2 (perceived fairness of communication): We hypothesize that people need their pref- erence statements to be reciprocated and would be more willing to accept an oer (H2a), nd it more fair (H2b) and satisfying (H2c), and place greater trust in their opponent (H2d) if the 62 % Accept Satised Trust in partner Fairness of oer Fixed-pie Lie 0.88 3.73 + 3.15 3.73 Honest 0.58 3.12 3.04 2.81 No information 0.56 3.26 3.19 3.04 Table 6.1: Experimental Results ( p 0:05; + p 0:10) opponent provides reciprocal but false preference statements, compared with an oer of equal value without reciprocal information provided. Together, these hypotheses imply a corollary: opponent models matter. In other words, people base their decisions, not only on their absolute benet, but also their perceptions of the other party's interests and how this relates to norms of distributional and communicative fairness. To test these hypotheses, we conducted an online experiment using seventy-ve participants recruited on Amazon Mechanical Turk. Participants were randomly assigned to one of three conditions. In all conditions, participants were asked to imagine they were in a negotiation. They were presented a dialog between them and a hypothetical opponent concerning how to partition a basket of fruit (see Figure 6.6). They were asked if they would accept the oer made by their opponent (yes/no), and to rate its fairness, their satisfaction with the oer, and whether they felt the other party seemed honest (all items using 5-point Likert scale). The only dierence between conditions was the preference statements made by the opponent. In the misrepresentation condition, the opponent produces preference statements according to the xed-pie-lie heuristic and the participant was oered half the apparent value. In the honest condition (illustrated in Figure 6.6), the opponent produces honest preference statements (clearly indicating that the pie could be grown), but the participant received the same oer as in the misrepresentation condition. Finally, in the non-reciprocal condition, the opponent asks questions but does not provide any information in return. Again, the identical oer was proposed across conditions. Table 6.1 summarizes the results. Our rst hypotheses shows good support. People were more willing to accept an oer (H1a) and found it more fair (H1b) if their opponent misrepresented their preferences as a xed pie and 63 Figure 6.6: An illustration of the \truthful" condition 64 claimed half the pie. People accepted more oers from the xed-pie-lie condition (M = 88%) compared to the honest condition (M = 58%); 2 (1;N = 52) = 6:26, p = 0:027. They rated the oer from the lying opponent as more fair (M = 3:73, SD = 0:33) than the truthful opponent (M = 2:81, SD = 0:51); t(25) = 3:00, p = 0:006. There was a near-signicant trend (p = 0:065) in perceived satisfaction (H1c) and no dierence in trust (H1d) in the partner. Our second hypothesis also shows good support. People accepted more oers (H2a) and found them more fair (H2b) when provided reciprocal but deceptive feedback about their opponent's preferences than no feedback at all. They accepted more from the xed-pie-lie condition (M = 88%) than the no-information condition (56%); 2 (1;N = 53) = 7:07,p = 0:014. People rated the lying opponent as more fair (M = 3:73,SD = 0:33) than the no-information condition (M = 3:04, SD = 1:18); t(25) = 2:18, p = 0:039. There was a non-signicant trend (p = 0:091) in perceived satisfaction (H2c) and no signicance dierence in trust (H1d) in the partner. For completeness, we compared dierences in acceptance rate and subjective impressions be- tween the honest and no-information conditions but found no signicant dierences. In summary, people were much more willing to accept an oer when the opponent misrepresents their preferences then when truthful. This is because the truthful oer was seen as considerably less fair (people recognized the pie had grown but the opponent was keeping a disproportionate amount). People were also less willing to accept oers when the agent failed to reveal its own preferences, giving support for the principle of reciprocal information exchange. Again, such oers were considered less fair. From this we can conclude that malicious agents must attend to two concerns: relative fairness of the oer and relative fairness of information exchange. 6.1.4 Discussion We presented a method that allows negotiators to reap unequal rewards over opponents while maintaining the illusion of fairness. Our analysis and results demonstrate there can be consider- able strategic benets from this form of lying. The simulation results in Section 6.1.2.2 illustrate 65 that many negotiations can yield strategic gain and sometimes these rewards can be considerable (Figure 6.3). Section 3.4 illustrates that these rewards can be achieved even when an opponent's preferences are unknown (via deceptive reciprocal information exchange). Finally, the experimen- tal results in Section 6.1.3 illustrate that people nd these deceptive strategies to be credible and even preferred to honesty (or no information at all). Apparently in negotiation, honesty is not always the best policy. There are several ways to extend our analysis. Improving the analysis: We know the validity of the xed-pie lie when the opponent's pref- erences are known in advance [37] but rely on simulation and empirical data for the more typical situation where this information is unknown. We suspect it should be straightforward to prove the validity of the xed-pie-lie heuristic under fairly general assumptions. Relaxing assumptions about the nature of the opponent: Our results illustrate the po- tential benets of lying, but opponents might adopt a variety of countermeasures that could undermine this potential. In the current analysis, we assumed the opponent was truthful, and in indeed in many laboratory studies of negotiation participants tell the truth much more than would be predicted by rational models, but this tendency is hardly universal. One straightforward way to approach mutual deception is to extend the analysis in Sec- tion 6.1.2.1 to consider a Stackelberg formulation of the misrepresentation game. In Stackelberg games, one player (the defender) moves rst, and the second player responds. In a Stackelberg for- mulation of the misrepresentation game, one negotiator might publicize a false set of information about their preferences (e.g., the MSRP of dierent automobile packages) in order to minimize the risk of exploitation or maximize their own gain. Our current analysis also assumed the human opponent passively accepted reciprocal prefer- ence statements, however we should relax this assumption to allow opponents to ask arbitrary 66 preference questions, and perhaps to refuse to directly answer. If the human is free to ask arbi- trary preference questions, the misrepresenting agent is left the dilemma of whether to address these questions directly or to defect them in some way. For example, if the human asks \do you like apples more than oranges," the agent might de ect the response by answering a dierent question (\Well, I like bananas better than pears") or to hedge the response (\I might like apples better"). If the agent is allowed to de ect, we must assume the human opponent can as well, and such contingencies must be incorporated into the analysis. One approach is to assume de ections come at some cost and model these in the analysis of what questions to ask and answer. Finally, we only considered a negotiation protocol where parties exchange information about preferences before making a take-it-or-leave it oer. Human negotiations typically involve a mixture of information and oer exchanges and this sort of exibility needs to be incorporated into our models. Indeed, discrepancies between what one's opponent says and what they oer can be a good indicator of deception [63]. Countermeasures: The previous discussion has focused on how to increase the generality and robustness of strategic misrepresentation but our ultimate aim is to use such models to help people (and agents) to avoid being deceived. Understanding and modeling a malicious technique is often a necessary the rst step to proposing countermeasures (e.g., [61]). In our own work, we are developing these techniques to inform the design of a negotiation tutoring system [36]. Specically, students would be able to practice negotiation tactics, including how to detect and avoid deceptive behavior, with computer generate opponents. One such tactic is to oer one's opponent a counteroer that would, assuming they are truthful, have equivalent value. For example, in the situation at the bottom of Figure 6.1, the man could counter by oering the reverse (\How about I take all apples and you take all oranges?"). Taking her false statements at face value, she should be indierent to these two deals. In reality, this reverse deal has negative value to her. 67 The opportunity to make counteroers highlights the potential riskiness of misrepresentation. For example, if the man insisted on oering oranges, the woman would be caught in her lie. She would either be forced to accept a disadvantageous oer or backtrack on her preference statements (see [78]. The former approach would harm her gains within the negotiation but the latter would harm her reputation for honesty and possibly undermine long term rewards. The idea of repeated interactions presents other complications. For example, if a company uses deception in their negotiations but lies in dierent ways to dierent customers, customers could compare notes and uncover the deception. Characterizing the disadvantages of deception: Here we focus on potential benets of deception but it can also have obvious costs, especially if both parties are being deceptive. In negotiation research, several studies have illustrated that honest information exchange can lead to better outcomes than deceptive communication or no communication, and achieve greater gains that might seem possible from a purely rational analysis (e.g., [82, 84]). In fact, it might turn out that, under more relaxed assumptions, that honest is the best policy after all. Indeed, Figure 6.5 illustrates that honesty pays at least under certain circumstances (where intermediate-levels of information are exchanged). There may also be more intelligent strategies that switch between lying and honesty depending on the situation (see [78]). For example, although the xed-pie lie exceeded the performance of the max-max regret heuristic in our simulation study (Figure 6.5), a disadvantage of the xed-pie lie is that it commits a player to lying from the point of the very rst question. Max-max regret strives to be honest but minimally informative. An advantage of this latter approach is that it doesn't committed a player to deception. Rather, the max-max regret heuristic allows a player to revert to honesty if this turns out to aord the greatest reward. Contrasting human and rational behavior: The immediate next steps in our research, beyond relaxing the assumptions listed above, is to use these models to analyze the behavior of 68 human negotiators. The present analysis suggests that people will tend to win at negotiations if they ask a lot of questions about what their opponents want, are vague about their own interests, and then invent a set of lies that makes self-advantageous deals seem fair. In other work, we have collected a large corpus of human-human negotiations where people clearly lie (see [26]) but we have yet to characterize the tactics they use and the success or failure of these lies. The rational analysis we present here can provide a valuable yardstick with which to measure and give insight into human negotiation processes and ultimately aid in the design that can cope with humans behaving badly. One next step is to use the analysis in this section to examine if human liars are following our proposed solution. The ethics of misrepresentation: Our analysis suggests that people, and (potentially) au- tonomous agents acting on behalf of people, can perform better in negotiations by misrepresenting their preferences. But should such agents ever be constructed. One source of guidance to this question is the rich literature on business and legal ethics, wherein negotiation is a central topic. Interestingly, misrepresentation is often argued to be legal and even ethical. For example, the American Bar Association's code of ethical conduct prohibits lawyers from knowingly misrepre- senting facts, but then goes on to say that statements concerning price or value in a negotiation are not considered facts (e.g., see [22]). The rational is that a certain amount of \pung" is to be expected. In contrast, others point out that such lies cause harm to the opponent and there- fore are unethical, even if not illegal [87]. Others claim that it may be unethical to misrepresent through verbal statements but that misrepresenting the same information via emotional expres- sions is acceptable [35]. The ethics of such emotional manipulation has been considered within the eld of aective computing (e.g., see [10]). In conclusion, we presented an analysis of how people, or agents interacting with people, can win at negotiations while seeming fair. Our ndings demonstrate clear benets a type of lying (misrepresenting the relative importance of dierent issues) and suggest that negotiators should 69 adopt a \xed-pie lie" (i.e., pretend that their preferences are identical to their opponent). Future research should extend this analysis to more ecologically-valid situations and a broader class of misrepresentation techniques. 6.2 Model Validation From the solution to the misrepresentation game, one can derive several predictions about how deceptive negotiators should behave and what should be expected in terms of the negotiation outcome. Here, we test if people actually follow these predictions. If so, the model is a reasonable approximation of human behavior and could be used against human negotiators successfully. First, the formal analysis indicates that malicious negotiators will benet by claiming the negotiation is distributive (xed-pie), regardless of the actual structure of the task. Any deviation from this policy should undermine the potential for prot. From this we can derive our rst hypothesis: H1 (xed-pie communication): Prots will increase to the extent that negotiators convey distributive preferences. Second, the solution to the misrepresentation problem indicates that liars only gain when the negotiation has integrative potential. If the task has distributive structure, lies only undermine the opponent's belief that the negotiation is xed-pie. From this we can derive a two-part hypothesis: H2 (xed-pie lies): Lying will enhance prot i the negotiation has integrative potential (H2a) and the lies convey distributive preferences (H2b). In Chapter 5.2 we explained how liars exhibit inconsistency in their behavior. The solution to the misrepresentation problem should also force a discrepancy between their words and deeds. When a negotiator is in an integrative negotiation and uses xed pie lies, they must make an oer that creates integrative value, but keep this integrative value for themselves. This means their 70 nal oer will appear somewhat integrative. Figure 6.6 illustrates this. Here the liar oers their opponent more of what their opponent wants (apples), while keeping a larger amount of lower value items (where, in truth, these low-priority issues are of high value to the liar). Because of the lies, this appears to be a Nash bargaining solution ($12 each), however there is another equivalent solution that splits the pie down the middle: each receives two apples, two bananas and two pears (also $12 each). Although the deals are equally fair in terms of payout, the former might be seen as less consistent with xed-pie structure. This leads to our third hypothesis: H3 (evidence of lies): Liars should exhibit a discrepancy between their oers and stated preferences; truthful negotiators should not exhibit this discrepancy. Finally, a clever opponent might recognize this discrepancy and suspect lying. If people are good at detecting this sort of deception, it will be harder for the agent to succeed at the misrep- resentation game. On the other hand, negotiators often assume by default that negotiations are distributive { an eect known as a \xed-pie bias" [81] { which could make it especially hard to suspect xed-pie lies. From this we derive a research question: RQ1 (lie detection): Do lies impact perceptions of honesty? 6.2.1 Measures Lies: As a measure of explicit lies, we manually annotated all preference statements for their veracity. For each of the 342 preference statements, we recorded it as a lie if it was inconsistent with the participant's private preference. For example, if a participant stated they liked lamps most, but in truth they like records more than lamps, this would be recorded as a lie. A lie was classied as a \xed-pie lie" if it matched the opponent's preferences. For example, if 1) a participant liked records most, and 2) their opponent liked lamps most, and 3) the participant stated they liked lamps most, this statement was noted as a xed-pie lie. 71 These annotations do not capture all possible lies, but only misrepresentation of relative im- portance of preferences. Negotiators could lie about other things, such as their payout if the negotiation fails (we return to this at the conclusion). Using the opponent models introduced in Chapter 5, we can derive a measure of whether a person is communicating a distributive structure. RDFP-Words: We compare the distance between the preference weight estimation discovered by the issue-sentiment heuristic and the weights the party should have communicated if they wanted to convey a xed-pie ranking. For example, if one party talks most about lamps and least about the painting, this would imply the ranking: Lamps> Records> Painting. If their opponent has, in truth, this same ranking, then the distance between these two ranks would be zero. In other words, a small distance implies xed-pie communication. From this, we dene a measure of the \rank-distance from xed-pie" (or RDFP-Words) that captures the distance between the weights estimated by issue-sentiment heuristic and the weights a negotiator should have communicated based on the xed-pie lie tactic. Low values of RDFP-Words imply the negotiator is claiming distributive preferences. RDFP-Deeds: The issue-ratio heuristic is used to infer a negotiator's preferences from the oers (DIVs) they make throughout the negotiation. From this we derive a measure of the \rank- distance from xed-pie" (or RDFP-Deeds) to see how far an opponent's oers diverge from what they should oer in a distributive (xed-pie) negotiation. Honesty Perceptions: To address the research question, we need to measure the perceived honesty a negotiator has of their opponent. For this, we use the self-reported honesty ratings include in the corpus (a 7-point Likert scale). 72 6.2.2 Results H1: The rst hypothesis states that prots will increase to the extent that negotiators convey distributive preferences (ignoring if they are lying or honest). To test this, we examined the overall sentiment people communicated (RDFP-Words) which captures both explicit and implicit preference statements. We found a signicant negative correlation between RDFP-Words and the number of lottery tickets earned across the entire corpus (r(146) =0:247, p = 0:003). In other words, the closer their verbally-communicated preference was to a xed pie, the better they did. To see how this eect interacts with the structure of the negotiation, we performed a moderated regression, predicting earnings using both RDFP-Words and structure (dummy coded: 0 = integrative, 1 = distributive) in a rst step, followed by the RDFP-Words structure interaction term in a second step. This analyses showed a signicant main eect of RDFP-Words, as predicted by the correlation analysis ( =0:20, t(143) =2:43, p = 0:02), as well as signicant main eect of structure ( = 0:16, t(143) = 1:96, p = 0:05). In other words, participants win more money in distributive negotiations. The interaction of RDFP-Words structure, however, did not reach signicance ( = 0:17,t(143) = 1:56,p = 0:12), suggesting that RDFP-words predicts prot regardless of the negotiation structure. This supports our rst hypothesis. H2: The second hypothesis asserts that lying will only enhance prot if (H2a) the negotiation has integrative potential and (H2b) if the lies are in a xed-pie direction. For H2a, we conducted two separate t-tests for integrative and distributive negotiations. In the integrative case, liars earned signicantly more lottery value (M = 63:33, SD = 9:19) than non-liars (M = 55:12, SD = 13:91);t(71) = 2:159,p = 0:02. In the distributive case, however, the dierence in earnings between liars (M = 64:33, SD = 18:60) and non-liars (M = 63:19, SD = 15:57) as insignicant; t(71) = 0:24, p = 0:30. This result supports hypothesis H2a. Next, we examined the type of lies told in the integrative negotiations. Of the lies 25 lies elicited in the integrative task, virtual all (88%) were xed-pie lies. (Note that there cannot be 73 xed-pie lies in the distribution task. As the task, in truth, has a xed-pie structure, any lie would necessarily claim something other than a xed-pie.) Together, these two ndings conrm hypothesis H2. As second way to examine H2 is to look at the explicit and implicit communication made by negotiators. For this, we examined the overall sentiment people communicated (RDFP-Words) separately for integrative and distributive negotiations. We found a signicant correlation between xed-pie communications and prots for integrative negotiations (r(73) =0:365,p = 0:001), but no correlation in the distributive negotiations (r(73) = 0:045, p = 0:704). In other words, the closer their verbally-communicated preference was to a xed pie, the better they did, but only or integrative negotiations. Again, this supports hypothesis H2. H3: Our third hypothesis asserts that liars give o indications of lying. Specically, deceivers should be more distributive in their words but more integrative in their oers. To test this, we divided participants into liars and non-liars and calculated the rank distance between sentiment- ratio and issue-ratio to see if there is a signicant dierence between the liar and non-liar groups. A t-test shows that indeed liars have bigger dierence between their words and deeds than the non-liars group (see Figure 6.7). This supports hypothesis H3. Research question: Support for H3 suggests it is theoretically possible to detect malicious ac- tors. But do participants recognize this in practice? Our nal research questions ask if participants found liars to be any less honest than truth-tellers. To test this, we again divided participants into liars and non-liars and performed t-tests to compare perceived honesty of their opponent (a single-item 7-pt Likert scale). Liars were considered signicantly less honest in the distributive condition: liars (M = 5:33, SD = 1:67) vs. non-liars (M = 6:24, SD = 0:93), t(67) =2:75, p = 0:008. In the integrative condition, liars were actually considered more honest than honest participants { liars (M = 6:21, SD = 1:18) vs. Non-Liars (M = 5:89, SD = 1:42), t(65) = 0:79, p = 0:432 { but this dierence was not signicant. Recall that in the integrative condition, lies 74 Figure 6.7: Divergence between the issue-ratio and issue-sentiment models. Liars: (M = 0:32, SD = 0:19), Non-Liars: (M = 0:22, SD = 0:12), t(144) = 3:54, p = 0:001 conform with the common preconception that negotiations are xed-pie (i.e., the \xed-pie bias" [81]). But in the distribution condition (which truthfully is a xed pie), any lies are in con ict with the xed-pie bias. This suggests that xed-pie lies are especially dicult for negotiators to detect. 75 Chapter 7 Agent Implementation and Experiments In the previous chapter, we introduced the misrepresentation game, a formulation of how a decep- tive party could misrepresent its preferences to gain benet, while seeming fair and honest. We also introduced a particular communication strategy, called the xed-pie lie and provided evidence that this technique was eective against human participants. The evidence, however, relied on assumptions that are unlikely to hold in real-world situations. Thus, it is unclear if this technique is eective against people in more realistic scenarios, and how people actually use deception in practice. In this chapter, we explore the misrepresentation game in a more realistic context. Building on a platform that supports human-agent negotiation [58], we investigate our solution to the misrepresentation game under far less restrictive conditions. We show that the xed-pie lie is surprisingly eective in practice. These results give insight into how people use deceptive com- munication. They also highlight ways that people (and potentially automated agents) can exploit na ve negotiators. Ultimately, such models facilitate techniques to recognize and protect against such practices. 76 7.1 Framework and Implementation In Chapter 6, we provided evidence for eectiveness of the xed-pie lie strategy in a simplied setting. The agent started by asking questions about preferences of the user in a one-way infor- mation exchange and ended with an ultimatum-type take-it-or-leave-it complete oer. In more realistic negotiations, however, parties are free to ask any questions they like, make tentative oers, and make counter-proposals to oers they receive. Also, both parties are free to lie. Thus, it is unclear if an agent can use this strategy in more realistic settings. Here, we aim to answer this question using a publicly available platform called IAGO [58] that provides a realistic environment for negotiations between agents and humans. A few characteristics that make IAGO a great choice for our purpose are: 1. It supports partial oers; 2. Oer exchange is not limited to alternative oer protocol; 3. Visual representation of emotional signals are provided; and 4. Communication of preferences are made possible. This platform is used in human-agent competitions in the IJCAI conference and is a powerful framework to conduct human-agent negotiations. 7.1.1 IAGO Platform IAGO is a web-based platform proposed for conducting negotiations between agents and humans. It consists of a user interface that human users can interact with and an API that agent designers can use for their agent implementation. IAGO provides a web-based interface that could be displayed on a web browser via a link. The user then can interact with the implemented agent using the environment shown in Figure 7.1. The window on right shows the conversation between the agent and the user. Negotiation partners 77 Figure 7.1: IAGO's User Interface 78 can exchange full or partial oers (e.g., I'll oer you 1 bars of iron and take 3), ask and exchange information about preferences (e.g., \what do you like best? I like iron more than copper") and exchange emoji's that to convey emotional state. 7.1.2 Agent Design To successfully apply and examine \xed-pie lie" strategy and compare it to control conditions, our agents must be able to handle three critical requirements: 1) They must make an accurate model of the opponent's preference prole (User Modeling), 2) communicate an intended preference prole (User In uence) and 3) evaluate the utility of deals to come up with desired oers, or accept/reject received oers (Decision Making). This design is inspired by the IRPC framework introduced in Chapter 4. In this section, we brie y present the mechanisms by which our agents implement each of these three modules. User Modeling. The agent starts with no information about users' preference prole and attempts to acquire maximal information through communication with them. The agent explains to the user about the importance of communication and asks questions about user's preferences. At each point in the negotiation the agent saves a space of possible preferences for the user based on the information it has gathered up to that point. For example, in the beginning and for a three issues negotiation over bars of gold, silver, and iron, this space is constructed of six possible preferences (Table 7.1a). If the user expresses that she likes gold best, then only two possible preference proles remain (Table 7.1b), and after user's answer to if she likes silver or iron more (Table 7.1c for iron), the agent has a solid estimation of the user preference prole. When asked about preferences, users may provide contradictory information. This could be due to laziness, lying or just unintentional errors. To detect such incidents and keep the user expressions consistent, we implemented a \contradiction check" element that is called after each preference expression made by the user. If the new expression was consistent with the previous 79 Gold Silver Iron 1 2 3 1 3 2 2 1 3 2 3 1 3 1 2 3 2 1 (a) Gold Silver Iron 3 1 2 3 2 1 (b) Gold Silver Iron 3 1 2 (c) Table 7.1: tables show the space of possible utility weights for all issues under negotiation. ones, the knowledge base gets updated; otherwise, the user is informed about the inconsistency and asked for correction. The following is a description of how contradiction check works in our agents. Contradiction Check. To verify that a user's preference expression is consistent with her previous statements, the agent keeps a graph representation of preferences expressed by the user. Each node in this graph represents an issue, and each edge represents a relative connection. The method will add a new expression (and the corresponding edges) only if it does not create a loop in the graph. Otherwise, the contradiction check fails and the user is notied. For example, in a negotiation over three issues of gold, iron, and silver, the graph has three nodes, one for each issue. If a user states \I like gold the best", edges are is added from all other issues to gold (here, iron and silver). If the user then expresses \I like silver more than gold", adding this new edge could create a loop in the graph (from gold to silver, and silver to gold); The agent will inform the user about the inconsistency and asks for correction. Shaping user's model. During the negotiation, the agent provides the user with information about the agent's preferences. This information could be in the form of voluntary information exchange in return for users' similar behavior, or by answering questions asked by the user. A variable \Honesty" species how to shape the user's model. A truthful agent will communicate 80 its actual preference prole; however, a liar agent would communicate the same preference prole as the user. Decision Making. Once an agent has a proper model of the user's preferences, along with its own preference prole, the agent can calculate how much each deal is worth for itself and how much for the user. The percentage of the joint value that the agent receives is the criterion evaluating each deal. For example, if a deal is worth 22 points for the agent and 22 points for the user, a fair agent will accept it, but an exploitative one will reject it. The same module is used to pick an oer to propose from the agent side. 7.1.3 Negotiation Task Misrepresentation game illustrates how a negotiator can use the integrative potential in a multi- issue negotiation task to gain advantage while seeming fair. Even though this \xed-pie lie" strategy works best when the task is fully integrative, we use a partial integrative task to demon- strate the generalizability of the strategy. The negotiation was framed as a \Resource Exchange Game" where opponents negotiated over four issues of gold, silver, copper and iron. In this task, negotiators should distribute four bars from each issue (overall 16 bars) between themselves. The preference proles for the agent and the user (shown in Table 7.2) are used as weights to calculate the gained utility value for each party. For example, if an agent receives four bars of iron, two bars of silver and two bars of copper, the utility value will be calculated as: (44)+(23)+(22) = 26. User receives the remaining bars (two bars of silver, two bars of copper and four bars of gold) which gets her (4 4) + (2 2) + (2 3) = 26 points. If they fail to reach an agreement, each party receives 4 points, which makes this their BATNA (Best Alternative To Negotiated Agreement) value. 81 Gold Silver Copper Iron User 4 3 2 1 Agent 1 3 2 4 Table 7.2: Preference prole for the considered negotiation task 7.2 Methods 7.2.1 Study 1 We used a 2 (Agent Fairness: Fair versus Exploitative)2 (Agent Honesty: Truthful versus Liar) design to examine the eects of the agent's fairness and honesty on the participant's behavior in our designed negotiation task. Fairness (fair versus exploitative) species the value threshold of the deals that are acceptable to the agent. Fair agents would only accept deals that give the agent 50% of the total joint value, and exploitative agents would only settle for 60% or more of the overall joint value. Honesty (truthful versus liar) species the type of information the agent will provide to the participant. A truthful agent would provide truthful information about its own preferences and a liar will pretend the same preferences as the opponent (xed-pie lie). We hypothesized that H1: Fair agents should require that users make fewer oers to reach agreement com- pared to exploitative ones. H2: Liar agents should require that users make fewer oers to reach agreement com- pared to truthful ones. We recruited 223 participants using Amazon's Mechanical Turk (MTurk) service as subjects for our study. Participants were presented with a demographic survey followed by instructions on the negotiation task, their preferences, BATNA, and how to interact with the IAGO agent. Next, they were given 10 minutes to negotiate with the agent and come up with an agreement or walk 82 out of the negotiation. Following the negotiation, they were asked to rate how satised they were with the results, as well as rate the agent's level of honesty, fairness and strategy. To ensure that participants understood their priorities in the negotiation, they also completed a manipulation check. Specically, they were asked to report which items were the most and least valuable for them. From the original 223 participants, 11 failed the manipulation check. These participants were excluded, leaving 212 for analysis. 7.2.2 Study 2 In this study, we investigated the eectiveness of more exploitative agents. We used a 2 (Agent Fairness: More Exploitative vs Most Exploitative) x 2 (Agent Honesty: Truthful vs Liar). Honesty was manipulated as in study1. Fairness, however, was operationalized in this study as more exploitative (asked for 69% of the overall joint value) and most exploitative (asked for 76% of the overall joint value). In addition to testing H2 again, we also extent hypothesis H1 to predict that H3: The more exploitative agent should require that users make fewer oers to reach agreement compared to most exploitative one. Study ow was the same as the rst study and 114 participants were recruited using Amazon's Mechanical Turk (MTurk) service as subjects for this study. From the original 114 participants, 1 failed the manipulation check. This participant was excluded, leaving 113 for analysis. 7.2.3 Measures Two main types of measures were considered for evaluating eectiveness of the agent when nego- tiating with humans: behavioral measures and self-report measures. Behavioral Measures. The low value of alternative to no agreement (BATNA) in our negoti- ation task encourages participants to agree on some deal rather than leaving the negotiation with 83 no agreement. However, these agents in only propose (and ultimately accept) deals that gain them equal or more than a specic threshold value (depending on fairness condition). To the extent that participants nd this value acceptable, they will propose fewer counter oers. Therefore, the number of counter oers proposed by the participants indicates their dissatisfaction with the agent's proposed value. Self-report measures. We used self-report measures to assess satisfaction and perceptions of the agent's level of honesty, fairness and strategy. After the negotiation, participants reported their satisfaction with the outcome using a 7-point scale that ranged from 1 (very dissatised) to 7 (very satised). Then participants rated the agent on honesty, fairness and strategy using 1-7 scales that ranged from 1 (dishonest, unfair, or not-strategic) to 7 (honest, fair, or strategic, respectively). 7.3 Results 7.3.1 Study 1 Out of 212 analyzed negotiations, only 6 failed to reach an agreement and settled for their BATNA value. A 2 (Agent Fairness: More Exploitative vs Most Exploitative) 2 (Agent Honesty: Truth- ful vs Liar) ANOVA shows that, while there were signicant main eects of agent fairness and honesty on user oers (F (1; 208) = 6:82, p = 0:01 and F (1; 208) = 3:92, p = 0:049, respectively), these were qualied by a signicant interaction (F (1; 208) = 6:68, p = 0:01). As depicted in Fig- ure 7.2, both main eects were driven by users' increased frustration with the honest exploitative agent compared to the other conditions. Likewise, the main eects of agent fairness and honesty on satisfaction were either signicant (F (1; 208) = 17:61,p< 0:001) or at least trend-like (F (1; 208) = 2:26,p = 0:13) respectively), but again they were qualied by a signicant interaction (F (1; 208) = 14:88, p< 0:001). As depicted 84 Figure 7.2: The number of oers made by users in each condition in Figure 7.3, both main eects were again driven by users being less satised with the honest exploitative agent compared to the other conditions. In contrast, users' perceptions of the agent's honesty were unaected by actual honesty (F (1; 208) = 1:55, p = 0:22) or fairness (F (1; 208) = 0:86, p = 0:36). However, there was a signicant interaction (F (1; 208) = 19:61,p< 0:001); as depicted in Figure 7.4, the liar fair agent and the honest exploitative agent were both perceived as less honest than the other conditions. Perhaps either appearing to give too much or too little can reduce perceived honesty. Similarly, users' perceptions of the agent's strategy were unaected by honesty (F (1; 208) = 0:39, p = 0:53) or fairness (F (1; 208) = 0:15, p = 0:70), but there was a signicant interaction (F (1; 208) = 7:37, p = 0:007). As depicted in Figure 7.4, the liar fair agent and the honest exploitative agent were also both perceived as less strategic than the other conditions, again perhaps because users thought that either giving too much or too little was not particularly strategic. 85 Figure 7.3: Self-assessed rating on how satised the users were at the end of negotiation Figure 7.4: How honest (left) and strategic (right) agents were perceived by the user (Study 1) 86 7.3.2 Study 2 Out of 109 analyzed negotiations, 18 failed to reach an agreement and settled for their BATNA value. A 2 (Agent Fairness: More Exploitative vs Most Exploitative) 2 (Agent Honesty: Truthful vs Liar) ANOVA revealed a signicant eect of agent honesty on user oers (F (1; 109) = 7:89, p = 0:006), such that users made fewer oers to agents that lied (M = 1:90, SD = 1:90) than those that told the truth (M = 3:22, SD = 2:94). The main eect of agent fairness and the interaction eect both failed to approach signicance (Fs< 0:14, ps> 0:71). However, in this study, participants were no more likely to report feeling satised based on the honesty of the agent (F (1; 109) = 0:55, p = 0:46). Instead, there was a trend for participants to report greater satisfaction with the less exploitative of the two agents in this study (M = 4:41, SD = 2:12 versus M = 3:77, SD = 2:00; F (1; 109) = 2:73, p = 0:10). The interaction eect did not approach signicance (F (1; 109) = 0:33, p = 0:57). Likewise, users' perceptions of the agent's honesty were unaected by actual honesty of the agent (F (1; 109) = 1:05, p = 0:31). Instead, participants perceived the less exploitative of the two agents in this study as marginally more honest (M = 4:67, SD = 1:86 versus M = 4:08, SD = 1:71;F (1; 109) = 3:08,p = 0:08). Again, the interaction eect did not approach signicance (F (1; 109) = 0:85, p = 0:36). In contrast, while there were also trends for main eects of agent fairness and honesty on perceptions of how strategic the agent was (F (1; 109) = 2:03, p = 0:16 and F (1; 109) = 2:56, p = 0:11, respectively), these were qualied by a signicant interaction (F (1; 109) = 7:83, p = 0:006). As depicted in Figure 7.5, both main eects were driven by users perceiving the truthful agent that is relatively less exploitative as more strategic than the other conditions. 87 Figure 7.5: How strategic agents were perceived by the users (Study 2) 7.4 Discussion As an opponent in uencing technique, we presented an analysis of how people (or agents interact- ing with people) might optimally lie (maximally benet themselves) while maintaining the illusion of fairness towards the other party (Misrepresentation Game). In doing so, we built on concepts from game theory and the preference-elicitation literature, but apply these to human, not rational, behavior. Our ndings demonstrated clear benets to lying and provided empirical support for a heuristic (the xed-pie lie) that substantially enhances the eciency of such deceptive algorithms. In previous chapters, we proved the optimality of this strategy in theory, however, the feasibility of utilizing it by agents in a realistic environment was never tested. Here, we used a publicly available negotiation platform, IAGO, to test this strategy in realistic negotiations with humans. In the rst study, we examined the eect of honesty and fairness on user's behavior. We hy- pothesized that fair agents would require users to make fewer oers to reach agreement compared to the exploitative agent (H1). This hypothesis was conrmed. However, the eect was speci- cally driven by the truthful exploitative agent, with which users had the hardest time reaching agreement. 88 Our next hypothesis was that liar agents would require users to make fewer oers to reach agreement compared to truthful agents (H2). Although this hypothesis was conrmed in exploita- tive condition, participants treated the liar and truthful agents in the fair condition similarly. In exploitative condition, participants also found the liar agent more honest and fairer than the truthful agent, suggesting that there might be a \sweet spot" for deceptive agents. That is, the liar exploitative agent gained the most value, while appearing fair and honest. This result conrms the eectiveness of \xed-pie lie," particularly in a partially integrative task. In fair condition, however, we found an interesting eect. Even though the liar agent appears to give up a larger share of the overall value, there was a trend for users to be less satised with the outcome and nd the agent less strategic and less honest. People do not seem to trust an agent who gives up too much or too little, emphasizing the importance of appearing fair (but not too fair) when using \xed-pie lie" strategy. Study 1 conrmed the eectiveness of \xed-pie lie" in partially integrative situations. How- ever, because users showed high levels of satisfaction across the board, we conducted a second study that pushed the boundaries of fairness by using more exploitative agents. We expected the more exploitative agents to require fewer oers from users to reach agreement compared to the most exploitative agents (H3). However, we did not nd such eect. It is possible that pushing the boundaries of fairness in this way led to general dissatisfaction. However, Study 2 replicated the nding that liar agents require users to make fewer oers to reach agreement compared to the truthful agent. Although agents in Study 2 achieved higher gain (only 18 participants walked out of the ne- gotiation out of 109, even though the agents were selshly asking for 69% or 76% of the overall value), this is probably unrealistic for how agent would do in practice. Since in real-world nego- tiations users might refuse to negotiate with such agents after this initial exchange, because they were generally dissatised, these gains would likely be short-term. In contrast, the liar agent who appears to be fair makes people feel good while gaining more value. 89 Chapter 8 Conclusion Negotiation is an indispensable skill for any social creature. Within the multi-agent systems community, negotiation between autonomous agents has been a central focus of research, both as a means for advancing electronic commerce and as a challenge problem to enhance the capability of socially intelligent systems. In this thesis, however, we focused on designing automated agents that can negotiate with humans. Such agents can serve as virtual role players to help teach negotiation skills, and more broadly, they can serve as decision-support tools. Adapting a common formulation of negotiation known as the multi-issue bargaining task, we designed a negotiation domain to illustrate the vast dierences between how humans perform negotiations compared to automated agents. In the rst experiment of two, winners of the multi- agent negotiation competitions were put to negotiate in round-robin tournaments on a multi-issue bargaining task. In the second experiment, 226 people were recruited to negotiate with each other on the same negotiation tasks. The comparison of the two experiments demonstrated fundamental dierences between human and agent negotiators both in process and outcome. Based on the observed dierences and using a con ict resolution theory, we introduced a general framework (IRPC) that could structure the design of agents for negotiating with humans. IRPC suggests that two challenges should be addressed in design of agents who want to negotiate with humans: opponent modeling and in uencing the opponent. 90 We tackled the opponent modeling challenge by proposing new techniques to infer the op- ponent's interest and character. Interest modeling attempts to capture the opponent's utility function over the issues under negotiation, and character modeling addresses the individual dif- ferences in people's behavior in negotiations. We demonstrated that our three proposed techniques performed signicantly better than the previously introduced models in multi-agent community. To address the challenge of in uencing the opponent, we introduced the misrepresentation game, a mathematical formulation of how a deceptive party could misrepresent its preferences to gain benet, while seeming fair and honest. We proposed a particular communication strategy, called the xed-pie-lie, as a solution to the misrepresentation game and provided evidence that this technique is eective both in theory and simulation, and also against human participants. This evidence, however, was in a restricted environment unlike real-world human negotiations. In order to test the feasibility of utilizing this technique by agents in a realistic environment, we implemented our algorithms on a publicly available negotiation platform, IAGO, and conducted two experiments. In the rst experiment, we showed that the exploitative agent indeed wins more points while appearing fair and honest to its opponent. In the second experiment, we investigated how far the exploitative agents could push for more gain and examined their eect on the behavior of the human opponent. This study showed that even though overly exploitative agents gain high value in short-term, their long-term success remains questionable as they left their opponents unhappy and unsatised. Future work can proceed on several fronts. The current heuristics for interest modeling are very simple and we suspect that substantial improvements are possible, both in how weights can be estimated from issues and from the language separately, and also in how these two sources of information can be combined. Our work on character modeling was focused on recognition of character traits. This could be extended to perception and synthesis of this personality trait. Such models could give agents more tools to operate human-like.. 91 In opponent in uence, our assumptions about the structure of players' utilities greatly sim- plies our analysis but limits the generality of the approach. Much of the innovative work in negotiation agents explores less restrictive assumptions such as non-linear utilities [29], incom- plete information about levels and even non-convexity of the utility space [20]. More work is needed in generalizing our solution to these contexts. More importantly for the goals of this work, these assumptions preclude some opportunities for misrepresentation that have been observed in the human negotiation literature. For example, negotiations typically include a \disagreement payo" (the utility received by each player if a negotiation fails). One common tactic is for a negotiator to feign a higher disagreement payo than they actually hold. This could be explored by adding this payo to the game. Our assumption of an additive preference function also precludes certain misrepresentation tactics. For example, a negotiator might claim a nonlinear relationship over the levels of an issue (i.e., \I get a bonus if I get all three apples"). Including a broader space of preference functions would enrich our analysis. More generally, the concept of fairness itself might be open to negotiation and misrepresenta- tion. In this thesis, we adopted the equality principle which states that everyone in a group should share benets equally. However, Welsh [86] argues that dierent people use dierent fairness crite- ria, and even the same person might use dierent criteria in dierent circumstances. For example, the need principle states that those who are in more desperate need of a resource deserve a greater share, whereas the equity principle argues the opposite (arguing that gains should be partitioned relative to each parties' power/resources). Our ranking assumption obscures the dierences be- tween these principles because players discuss their ranking but not their need. Without this assumption, liars could be free to claim they receive less for each issue and thus \need" a greater share (e.g., see Roth's ndings of focal points [75]). Lastly, Our ndings in Chapter 7 emphasized there are clear incentives to creating deceptive agents. An obvious next question is how to avoid or defend against the proliferation of such 92 methods. One often-discussed approach is to create ethical guidelines to prevent their creation, yet this is not so simple as it may appear. For example, the type of deception explored in this work is not actually considered lying by the ethical guidelines that govern human negotiations (see the discussion at the end of Section 6.1.4, under the ethics of misrepresentation). Further, the value a negotiator assigns to specic issues is often subjective and dicult to objectively verify. As a result, some negotiation ethicists argue that ethical proscriptions will certainly fail and the only realistic approach is to give people the skills required to detect and defend against such techniques [71]. Indeed, one approach to defeat the xed-pie lie is to give the opponent what they claim they want in order to force them to reveal the deception or take a loss to save face. Unfortunately, few of our human participants employed this tactic, which suggests an opportunity for using this technology for teaching better negotiation skills. In this thesis, we demonstrated that research in human-agent negotiations demands researchers' attention beyond multi-agent settings. Humans' complex behavior introduces new challenges to the domain, and insights from social science ndings can greatly help computer scientists achieve more appropriate and realistic models. In this work, we proposed several new techniques based on the insights from humans' actual behavior. However, research on human-agent negotiation is far from reaching its potential and there remain challenges that pose interesting research questions which must be pursued. 93 Reference List [1] Paavo Alku, Tom B ackstr om, and Erkki Vilkman. Normalized amplitude quotient for parametrization of the glottal ow. The Journal of the Acoustical Society of America, 112(May):701{710, 2002. [2] Tim Baarslag, Katsuhide Fujita, Enrico H. Gerding, Koen Hindriks, Takayuki Ito, Nicholas R. Jennings, Catholijn Jonker, Sarit Kraus, Raz Lin, Valentin Robu, and Colin R. Williams. Evaluating practical negotiating agents: Results and analysis of the 2011 international com- petition. Articial Intelligence, 198:73{103, 2013. [3] Tim Baarslag, Mark Hendrikx, Koen Hindriks, and Catholijn Jonker. Measuring the perfor- mance of online opponent models in automated bilateral negotiation. In Australasian Joint Conference on Articial Intelligence, pages 1{14. Springer, 2012. [4] Tim Baarslag, Mark Hendrikx, Koen Hindriks, and Catholijn Jonker. Predicting the perfor- mance of opponent models in automated negotiation. Proceedings - 2013 IEEE/WIC/ACM International Conference on Intelligent Agent Technology, IAT 2013, 2:59{66, 2013. [5] Eyal Baharad and Shmuel Nitzan. The borda rule, condorcet consistency and condorcet stability. Economic Theory, 22(3):685{688, 2003. [6] Max H Bazerman and John S Carroll. Negotiator cognition. Research in organizational behavior, 1987. [7] Meghana A. Bhatta, Terry Lohrenz, Colin F. Camerer, and P. Read Montague. Neu- ral signatures of strategic types in a two-person bargaining game. Proc. Natl. Acad. Sci., 107(46):19720{19725, 2010. [8] Jim Blascovich, Jack Loomis, Andrew C Beall, Kimberly R Swinth, Crystal L Hoyt, and Jeremy N Bailenson. Immersive virtual environment technology as a methodological tool for social psychology. Psychological inquiry, 13(2):103{124, 2002. [9] J. M. Brett and M. Gelfand. At your service - teaching notes. Negotiation, teamwork and decision-making exercises, Evenston, IL: Dispute Resolution Research Center, 2009. [10] Selmer Bringsjord and Micah Henry Clark. Red-pill robots only, please. IEEE Transactions on Aective Computing, 3(4):394{397, 2012. [11] Joost Broekens, Maaike Harbers, Willem Paul Brinkman, Catholijn M. Jonker, Karel Van Den Bosch, and John Jules Meyer. Virtual reality negotiation training increases negotiation knowledge and skill. Lecture Notes in Computer Science (including subseries Lecture Notes in Articial Intelligence and Lecture Notes in Bioinformatics), 7502 LNAI:218{230, 2012. [12] Andrew Byde, Michael Yearworth, Kay-Yut Chen, and Claudio Bartolini. AutONA: A sys- tem for automated multiple 1-1 negotiation. In E-Commerce, 2003. CEC 2003. IEEE Inter- national Conference on, pages 59{67. IEEE, 2003. 94 [13] C Camerer, Gideon Nave, and Alec Smith. Dynamic unstructured bargaining with private information and deadlines: theory and experiment. Technical report, Caltech HSS working paper, 2015. [14] Colin Camerer. Behavioral game theory: Experiments in strategic interaction. Princeton University Press, 2003. [15] Colin Camerer. Behavioral game theory. New Age International, 2010. [16] Peter J. Carnevale and Dean G. Pruitt. Negotiation And Mediation. Annual Review of Psychology, 43:531{582, 1992. [17] Archishman Chakraborty and Rick Harbaugh. Cheap talk comparisons in multi-issue bar- gaining. Economics Letters, 78(3):357{363, 2003. [18] Li Chen and Pearl Pu. Survey of preference elicitation methods, 2004. [19] R Christie and F. L. Geis. Studies in Machiavellianism. Academic Press, New York, 1970. [20] John P Conley and Simon Wilkie. An extension of the nash bargaining solution to nonconvex problems. Games and Economic behavior, 13(1):26{38, 1996. [21] Mark Core, David Traum, H Chad Lane, William Swartout, Jonathan Gratch, Michael Van Lent, and Stacy Marsella. Teaching negotiation skills through practice and re ection with virtual humans. Simulation, 82(11):685{701, 2006. [22] Charles B Craver. Negotiation ethics: How to be deceptive without being dishonest/how to be assertive without being oensive. S. Tex. L. Rev., 38:713, 1997. [23] Vincent P Crawford and Joel Sobel. Strategic information transmission. Econometrica: Journal of the Econometric Society, pages 1431{1451, 1982. [24] Jared R Curhan, Hillary Anger Elfenbein, and Heng Xu. What do people value when they negotiate? mapping the domain of subjective value in negotiation. Journal of personality and social psychology, 91(3):493, 2006. [25] Morteza Dehghani, Scott Atran, Rumen Iliev, Sonya Sachdeva, Douglas Medin, and Jeremy Ginges. Sacred values and con ict over Iran's nuclear program. Judgment and Decision Making, 5(7):540{546, 2010. [26] David DeVault, Johnathan Mell, and Jonathan Gratch. Toward natural turn-taking in a vir- tual human negotiation agent. In AAAI Spring Symposium on Turn-taking and Coordination in Human-Machine Interaction. AAAI Press, Stanford, CA, 2015. [27] Thomas Drugman and Abeer Alwan. Joint robust voicing detection and pitch estimation based on residual harmonics. Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH, pages 1973{1976, 2011. [28] Theodore Eisenberg and Charlotte Lanvers. What is the settlement rate and why should we care? Journal of Empirical Legal Studies, 6(1):111{146, 2009. [29] Shaheen Fatima, Michael Wooldridge, and Nicholas R Jennings. An analysis of feasible solutions for multi-issue negotiation involving nonlinear utility functions. In Proceedings of The 8th International Conference on Autonomous Agents and Multiagent Systems-Volume 2, pages 1041{1048. International Foundation for Autonomous Agents and Multiagent Systems, 2009. 95 [30] Beverley Fehr, Deborah Samsom, and Delroy L. Paulhus. The Construct of Machiavellianism: Twenty Years Later. Advances in personality assessment, page 77116, 1992. [31] Ernst Fehr and Klaus M Schmidt. A theory of fairness, competition, and cooperation. The quarterly journal of economics, 114(3):817{868, 1999. [32] Sevan G Ficici and Avi Pfeer. Modeling how humans reason about others with partial information. In Proceedings of the 7th international joint conference on Autonomous agents and multiagent systems-Volume 1, pages 315{322. International Foundation for Autonomous Agents and Multiagent Systems, 2008. [33] Kenneth E Friend, James D Laing, and Richard J Morrison. Bargaining Processes and Coalition Outcomes An Integration. Journal of Con ict Resolution, 21(2):267{298, 1977. [34] William Rick Fry. The eect of dyad machiavellianism and visual access on integrative bargaining outcomes. Personality and Social Psychology Bulletin, 11(1):51{62, 1985. [35] Ingrid Smithey Fulmer, Bruce Barry, and D Adam Long. Lying and smiling: Informational and emotional deception in negotiation. Journal of Business Ethics, 88(4):691{709, 2009. [36] Jonathan Gratch, David DeVault, Gale M Lucas, and Stacy Marsella. Negotiation as a challenge problem for virtual humans. In International Conference on Intelligent Virtual Agents, pages 201{215. Springer, 2015. [37] Jonathan Gratch and T Nguyen. Misrepresentation Negotiation Games, 2016. [38] H. P. Grice. Logic and conversation. In P. Cole and J. Morgan, editors, Syntax and semantics, volume 3: Speech acts. Academic Press, New York, 1975. [39] H M Hanson and E S Chuang. Glottal characteristics of male speakers: acoustic corre- lates and comparison with female data. The Journal of the Acoustical Society of America, 106(2):1064{1077, 1999. [40] Koen Hindriks, Catholijn M Jonker, Sarit Kraus, Raz Lin, and Dmytro Tykhonov. Genius: negotiation environment for heterogeneous agents. In Proceedings of The 8th International Conference on Autonomous Agents and Multiagent Systems-Volume 2, pages 1397{1398. In- ternational Foundation for Autonomous Agents and Multiagent Systems, 2009. [41] Koen V. Hindriks and Dmytro Tykhonov. Towards a quality assessment method for learning preference proles in negotiation. Lecture Notes in Business Information Processing, 44 LNBIP:46{59, 2010. [42] Warren H. Jones, Ted W. Nickel, and Alan Schmidt. Machiavellianism and Self-Disclosure. Journal of Psychology, 102(1):p33 9p., 1979. [43] John H Kagel, Alvin E Roth, and John D Hey. The handbook of experimental economics. Princeton university press Princeton, 1995. [44] Ehud Kalai. Proportional solutions to bargaining situations: interpersonal utility compar- isons. Econometrica: Journal of the Econometric Society, pages 1623{1630, 1977. [45] John Kane and Christer Gobl. Identifying regions of non-modal phonation using features of the wavelet transform. In Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH, pages 177{180, August 2011. 96 [46] John Kane and Christer Gobl. Wavelet maxima dispersion for breathy to tense voice discrim- ination. IEEE Transactions on Audio, Speech and Language Processing, 21(6):1170{1179, 2013. [47] John Kane, Stefan Scherer, Louis-Philippe Morency, and Christer Gobl. A comparative study of glottal open quotient estimation techniques. In Annual Conference of the International Speech Communication Association (INTERSPEECH), pages 1658{1662, August 2013. [48] Ron Katz and Sarit Kraus. Ecient agents for cli-edge environments with a large set of decision options. In Proceedings of the fth international joint conference on Autonomous agents and multiagent systems, pages 697{704. ACM, 2006. [49] Maurice George Kendall. Rank correlation methods. Grin, 1948. [50] Sarit Kraus and Daniel Lehmann. Designing and building a negotiating automated agent. Computational Intelligence, 11(1):132{171, 1995. [51] Sarit Kraus, Katia Sycara, and Amir Evenchik. Reaching agreements through argumentation: a logical model and implementation. Articial Intelligence, 104(1):1{69, 1998. [52] Thijs Van Krimpen, Daphne Looije, and Siamak Hajizadeh. \Hardheaded," in Complex Auto- mated Negotiations: Theories, Models, and Software Competitions. Springer Berlin Heidelberg, pages 223{227, 2013. [53] Vijay Krishna. Auction theory. Academic press, 2009. [54] Raz Lin and Sarit Kraus. Can automated agents prociently negotiate with humans? Com- munications of the ACM, 53:78, 2010. [55] Raz Lin, Sarit Kraus, Jonathan Wilkenfeld, and James Barry. Negotiating with bounded rational agents in environments with incomplete information using an automated agent. Ar- ticial Intelligence, 172(6-7):823{851, 2008. [56] Raz Lin, Yinon Oshrat, and Sarit Kraus. Investigating the benets of automated nego- tiations in enhancing people's negotiation skills. In Proceedings of The 8th International Conference on Autonomous Agents and Multiagent Systems-Volume 1, pages 345{352. Inter- national Foundation for Autonomous Agents and Multiagent Systems, 2009. [57] Tyler Lu and Craig Boutilier. Robust approximation and incremental elicitation in voting protocols. In IJCAI, volume 1, pages 287{293, 2011. [58] Johnathan Mell and Jonathan Gratch. IAGO: Interactive Arbitration Guide Online. In Pro- ceedings of the 2016 International Conference on Autonomous Agents & Multiagent Systems, pages 1510{1512. International Foundation for Autonomous Agents and Multiagent Systems, 2016. [59] Paul Robert Milgrom. Putting auction theory to work. Cambridge University Press, 2004. [60] Hal Movius. The eectiveness of negotiation training. Negotiation Journal, 24(4):509{531, 2008. [61] Lihi Naamani-Dery, Svetlana Obraztsova, Zinovi Rabinovich, and Meir Kalech. Lie on the y: iterative voting center with manipulative voters. In Proceedings of the 24th International Conference on Articial Intelligence, pages 2033{2039. AAAI Press, 2015. [62] John F Nash Jr. The bargaining problem. Econometrica: Journal of the Econometric Society, pages 155{162, 1950. 97 [63] Zahra Nazari, Gale Lucas, and Jonathan Gratch. Opponent Modeling for Virtual Human Negotiators. In Intelligent Virtual Agents, 2015. [64] K. M. O'Connor. Groups and solos in context: the eects of accountability on team negoti- ation. Organ. Behav. Hum. Decis. Process, 72(3):384{407, 1997. [65] K. M. O'Connor and P. J. Carnevale. A Nasty but Eective Negotiation Strategy: Misrepre- sentation of a Common-Value Issue. Personality and Social Psychology Bulletin, 23(5):504{ 515, 5 1997. [66] Martin J Osborne and Ariel Rubinstein. Bargaining and markets. Academic press, 1990. [67] Martin J Osborne and Ariel Rubinstein. A course in game theory. MIT press, 1994. [68] Yinon Oshrat, Raz Lin, and Sarit Kraus. Facing the challenge of human-agent negotiations via eective general opponent modeling. In Proceedings of The 8th International Confer- ence on Autonomous Agents and Multiagent Systems-Volume 1, pages 377{384. International Foundation for Autonomous Agents and Multiagent Systems, 2009. [69] James W Pennebaker, Roger J Booth, and Martha E Francis. Operator's Manual: Linguistic Inquiry and Word Count - LIWC2007, 2007. [70] John Rawls. A theory of justice. Harvard university press, 2009. [71] Peter Reilly. Was Machiavelli Right - Lying in Negotiation and the Art of Defensive Self-Help. Ohio St. J. on Disp. Resol., 24:481, 2009. [72] Patricia Rockwell, David B. Buller, and Judee K. Burgoon. Measurement of deceptive voices: Comparing acoustic and perceptual data. Applied Psycholinguistics, 18(04):471{484, 1997. [73] Avi Rosenfeld, Inon Zuckerman, Erel Segal-Halevi, Osnat Drein, and Sarit Kraus. Negochat: a chat-based negotiation agent. In Proceedings of the 2014 international conference on Au- tonomous agents and multi-agent systems, pages 525{532. International Foundation for Au- tonomous Agents and Multiagent Systems, 2014. [74] Avi Rosenfeld, Inon Zuckerman, Erel Segal-Halevi, Osnat Drein, and Sarit Kraus. Negochat- a: a chat-based negotiation agent with bounded rationality. Autonomous Agents and Multi- Agent Systems, 30(1):60{81, 2016. [75] Alvin E Roth. Bargaining experiments, 1995. [76] Ariel Rubinstein. Perfect equilibrium in a bargaining model. Econometrica: Journal of the Econometric Society, pages 97{109, 1982. [77] Stefan Scherer, Louis-philippe Morency, and Jerey F Cohn. Dyadic Behavior Analysis in Depression Severity Assessment Interviews. In Proceedings of the16th International Confer- ence on Multimodal Interaction, pp., pages 112{119. ACM, 2014. [78] Maurice E Schweitzer, Susan E Brodt, and Rachel TA Croson. Seeing and believing: Visual access and the strategic use of deception. International Journal of Con ict Management, 13(3):258{375, 2002. [79] Herbert A Simon. The sciences of the articial. MIT press, 1996. [80] Chris Sumner, Alison Byers, Rachel Boochever, and Gregory J. Park. Predicting dark triad personality traits from twitter usage and a linguistic analysis of tweets. Proceedings - 2012 11th International Conference on Machine Learning and Applications, ICMLA 2012, 2:386{ 393, 2012. 98 [81] Leigh Thompson and Reid Hastie. Social perception in negotiation. Organizational Behavior and Human Decision Processes, 47(1):98{123, 1990. [82] Leigh L Thompson. Information exchange in negotiation. Journal of Experimental Social Psychology, 27(2):161{179, 1991. [83] William L. Ury, Jeanne M. Brett, and Stephen B. Goldberg. Getting disputes resolved: Designing systems to cut the costs of con ict. Jossey-Bass, 1988. [84] Kathleen Valley, Leigh Thompson, Robert Gibbons, and Max H Bazerman. How communica- tion improves eciency in bargaining games. Games and Economic Behavior, 38(1):127{155, 2002. [85] Gerben A Van Kleef, Carsten K W De Dreu, and Antony S R Manstead. The interpersonal eects of anger and happiness in negotiations. Journal of personality and social psychology, 86(1):57, 2004. [86] Nancy A Welsh. Perceptions of fairness in negotiation. Marq. L. Rev., 87:753, 2003. [87] Gerald B Wetlaufer. The ethics of lying in negotiations. Iowa L. Rev., 75:1219, 1989. [88] D S Wilson, D Near, and R R Miller. Machiavellianism: a synthesis of the evolutionary and psychological literatures. Psychological bulletin, 119(2):285{299, 1996. [89] Steven R Wilson and Linda L Putnam. Interaction goals in negotiation. Annals of the International Communication Association, 13(1):374{406, 1990. [90] Gilad Zlotkin and Jerey S Rosenschein. Incomplete information and deception in multi- agent negotiation. In IJCAI, volume 91, pages 225{231, 1991. 99
Abstract (if available)
Abstract
Negotiation is a crucial skill in personal and organizational interactions. In the last two decades, there has been a growing interest to create agents that can autonomously negotiate with other agents. The focus of this thesis, however, is on creating agents that can negotiate with human opponents. Besides improving on artificial social intelligence, such agents could be used for the purpose of training or assisting human negotiators. A central challenge is to handle the complexity of actual human behavior. When compared with idealized game-theoretic models, human negotiations are far richer, both in terms of the nature of information exchanged and the number of factors that inform their decision-making. ❧ We consider a negotiation task that is simple, yet general enough to drive agent-human research, and analyze an extensive data set of transcribed human negotiation on such tasks. Based on human behavior in this task, and the previous research on human negotiations, we propose a new framework to structure the design of agents that negotiate with people. We address two main decision problems inspired by this framework: modeling and influencing the opponent. Three techniques are proposed to model an opponent's preferences and character (e.g. honesty and personality traits) and a misrepresentation technique are then used to influence the opponent and gain better profit. The proposed techniques are then implemented in automatic web-based agents. We ran a number of negotiations between these agents and humans recruited on Amazon Mechanical Turk. The resulting data show that the agents can perform these strategies successfully when negotiating with human counterparts and give us valuable insight into the behavior of humans when negotiating with an agent.
Linked assets
University of Southern California Dissertations and Theses
Conceptually similar
PDF
Decoding information about human-agent negotiations from brain patterns
PDF
The human element: addressing human adversaries in security domains
PDF
Real-world evaluation and deployment of wildlife crime prediction models
PDF
Computational modeling of human behavior in negotiation and persuasion: the challenges of micro-level behavior annotations and multimodal modeling
PDF
Modeling human bounded rationality in opportunistic security games
PDF
Human adversaries in security games: integrating models of bounded rationality and fast algorithms
PDF
A framework for research in human-agent negotiation
PDF
Context dependent utility: an appraisal-based approach to modeling context, framing, and decisions
PDF
Handling attacker’s preference in security domains: robust optimization and learning approaches
PDF
Combating adversaries under uncertainties in real-world security problems: advanced game-theoretic behavioral models and robust algorithms
PDF
Planning with continuous resources in agent systems
PDF
When AI helps wildlife conservation: learning adversary behavior in green security games
PDF
The interpersonal effect of emotion in decision-making and social dilemmas
PDF
Parasocial consensus sampling: modeling human nonverbal behaviors from multiple perspectives
PDF
Computational foundations for mixed-motive human-machine dialogue
PDF
An intelligent tutoring system’s approach for negotiation training
PDF
Hierarchical planning in security games: a game theoretic approach to strategic, tactical and operational decision making
PDF
Game theoretic deception and threat screening for cyber security
PDF
Toward human-multiagent teams
PDF
Virtual extras: conversational behavior simulation for background virtual humans
Asset Metadata
Creator
Nazari, Zahra
(author)
Core Title
Automated negotiation with humans
School
Viterbi School of Engineering
Degree
Doctor of Philosophy
Degree Program
Computer Science
Publication Date
12/13/2017
Defense Date
06/15/2017
Publisher
University of Southern California
(original),
University of Southern California. Libraries
(digital)
Tag
game theory,human behavior modeling,misrepresentation,negotiation,OAI-PMH Harvest,opponent modeling,preference modeling
Language
English
Contributor
Electronically uploaded by the author
(provenance)
Advisor
Gratch, Jonathan (
committee chair
), Kim, Peter (
committee member
), Tambe, Milind (
committee member
)
Creator Email
zahra.nazary@gmail.com,zahran@spotify.com
Permanent Link (DOI)
https://doi.org/10.25549/usctheses-c40-461175
Unique identifier
UC11267092
Identifier
etd-NazariZahr-5953.pdf (filename),usctheses-c40-461175 (legacy record id)
Legacy Identifier
etd-NazariZahr-5953.pdf
Dmrecord
461175
Document Type
Thesis
Rights
Nazari, Zahra
Type
texts
Source
University of Southern California
(contributing entity),
University of Southern California Dissertations and Theses
(collection)
Access Conditions
The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the a...
Repository Name
University of Southern California Digital Library
Repository Location
USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA
Tags
game theory
human behavior modeling
misrepresentation
opponent modeling
preference modeling