University of Southern California Dissertations and Theses

Modeling information operations and diffusion on social media networks

(USC Thesis Other)

Modeling information operations and diffusion on social media networks

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content MODELING INFORMATION OPERATIONS AND DIFFUSION ON SOCIAL MEDIA
NETWORKS
by
Ashok Kumar Deb
A Dissertation Presented to the
FACULTY OF THE USC GRADUATE SCHOOL
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulllment of the
Requirements for the Degree
DOCTOR OF PHILOSOPHY
COMPUTER SCIENCE
May 2021
Copyright 2021 Ashok Kumar Deb
c
2021 Ashok Kumar Deb
TABLE OF CONTENTS
Page
Acknowledgements iv
List of Tables vi
List of Figures viii
Abstract xii
Introduction 1
1 Analysis of Ads Campaign in US Elections 8
1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.2 Objective and Plan of Action . . . . . . . . . . . . . . . . . . . . . . . . . . 10
1.3 Related Literature . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
1.4 The Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
1.5 Data Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
1.6 Research Framework and Research Questions . . . . . . . . . . . . . . . . . . 16
1.7 Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
1.7.1 Features of Eective Ads . . . . . . . . . . . . . . . . . . . . . . . . . 18
1.7.2 Campaign-Level Analysis . . . . . . . . . . . . . . . . . . . . . . . . . 20
1.7.3 Party Clustering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
1.8 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
1.8.1 Ad Eectiveness in Aggregate . . . . . . . . . . . . . . . . . . . . . . 23
1.8.2 Campaign-wise Analysis . . . . . . . . . . . . . . . . . . . . . . . . . 26
1.8.3 Party-wise Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
1.9 Conclusions and Future work . . . . . . . . . . . . . . . . . . . . . . . . . . 32
2 Perils and Challenges of Social Media and Elections 34
2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
2.1.1 Contributions of this work . . . . . . . . . . . . . . . . . . . . . . . . 36
2.2 Objective and Plan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
2.3 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
2.4 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
2.4.1 Voting Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
2.4.2 Political Manipulation . . . . . . . . . . . . . . . . . . . . . . . . . . 39
ii
2.4.3 Bias . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
2.5 Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
2.5.1 Data Pre-processing . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
2.6 Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
2.6.1 State Identication . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
2.6.2 Bot Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
2.6.3 Statistical Vote Comparison . . . . . . . . . . . . . . . . . . . . . . . 46
2.6.4 Political Ideology Inference . . . . . . . . . . . . . . . . . . . . . . . . 48
2.7 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
2.7.1 #ivoted (DS1) Statistical Analysis . . . . . . . . . . . . . . . . . . . 50
2.7.2 General Midterm (DS2&DS3) Statistical Analysis . . . . . . . . . . . 51
2.7.3 Bot Sensitivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
2.7.4 Political Ideology Analysis . . . . . . . . . . . . . . . . . . . . . . . . 52
2.7.5 Voting Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
2.8 Discussion & Recommendations . . . . . . . . . . . . . . . . . . . . . . . . . 55
2.9 Conclusion and Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
3 Red Bots Do It Better 64
3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
3.2 Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
3.3 Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
3.3.1 Bot Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
3.3.2 Political Ideology Inference . . . . . . . . . . . . . . . . . . . . . . . . 69
3.3.3 Bot Activity Eectiveness . . . . . . . . . . . . . . . . . . . . . . . . 70
3.4 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
3.4.1 RQ1: Bot Political Leaning . . . . . . . . . . . . . . . . . . . . . . . 72
3.4.2 RQ2: Bot Activity and Strategies . . . . . . . . . . . . . . . . . . . . 75
3.4.3 RQ3: Bot Eectiveness . . . . . . . . . . . . . . . . . . . . . . . . . . 77
3.5 Conclusions & Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
4 Evolution of Bot and Human Behavior 80
4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
4.2 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
4.3 Data Collection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
4.4 Data Processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
4.4.1 Bot Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
4.4.2 Political Ideology Inference . . . . . . . . . . . . . . . . . . . . . . . . 85
4.4.3 Sentiment Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
4.4.4 Bot Eectiveness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
4.4.5 Granger Causality . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
4.5 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
4.6 Discussion and Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
iii
5 Gullible and Skeptic Agents 99
5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
5.2 Objective and Plan of Action . . . . . . . . . . . . . . . . . . . . . . . . . . 102
5.3 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
5.3.1 Gullible Agents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
5.3.2 Stubborn Agents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
5.4 Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
5.4.1 KK-Greedy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
5.4.2 Cost Ecient, Lazy Forward . . . . . . . . . . . . . . . . . . . . . . . 105
5.4.3 Cost Ecient, Lazy Forward + + . . . . . . . . . . . . . . . . . . . . 106
5.4.4 Upper Bound, Lazy Forward . . . . . . . . . . . . . . . . . . . . . . . 106
5.5 Diusion Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
5.5.1 Independent Cascade Model . . . . . . . . . . . . . . . . . . . . . . . 107
5.5.2 Linear Threshold Model . . . . . . . . . . . . . . . . . . . . . . . . . 108
5.6 The Proposed Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
5.6.1 Gullible v. Skeptic Agents . . . . . . . . . . . . . . . . . . . . . . . . 110
5.6.2 Conforming v. Non-conforming . . . . . . . . . . . . . . . . . . . . . 111
5.7 Benchmark Graph . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
5.8 Evaluation Metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
5.9 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
5.10 Conclusion & Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
6 Summary of Contributions 116
6.1 Analysis of Ads in US Elections . . . . . . . . . . . . . . . . . . . . . . . . . 116
6.2 Evolution of Bot and Human Behavior . . . . . . . . . . . . . . . . . . . . . 116
6.3 Gullible and Skeptic Agents . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
Bibliography 117
iv
ACKNOWLEDGMENTS
I would like to thank my parents, the late Sergeant First Class (Retired) Alok Deb and Mrs.
Ada Deb. My dog, Caesar, who passed while I was in the program. My advisor Emilio
Ferrara as well as the rest of my committee Gaurav Sukhatme, Cyrus Shahabi and Eric
Rice. The other extended members of Information Sciences Institute's Articial Intelligence
Division{Kristina Lehman, Aram Galstyan, Greg Ver Steeg, Fred Morstatter and Nanyun
Peng. As well as my future boss Palash Goyal, oce mates Sahil Garg and Robert Brekel-
mans. The other members of my immediate lab, Adam Badawy, YiLei Zeng, Akira Matsui,
Hsien-Te Kao, Shen Yan, Emily Chen, Alex Spangher and Julie Jiang. Fellow researchers
Nathan Bartley, Nazgol Tavabi, Luca Luceri, Anna Sapienza, Homa Hosseinmardi, K.S.M.
Tozammel, Leo Huang, Ritam Dutt and Desheng Hu.
Active duty military students Sean Eskew, Christina Acojedo, Marc Eskew, Lucas Haravitch,
Kalman Lonai, Timothy Rhodes and Kristina Richardson.
Other faculty Sarah Mojarad, Pete Khooshabeh, Jon-Patrick Allem, Cliord Neuman, Pre-
cious McClendon, Farzin Samadadi and Shahla Fatimi.
Other people of note include Mina Son, Hannah Penninger, Tammy O'Conner-Lockett,
Alexandra Rossi, Melissa Jeerson, Devika Dhir, Abigail Chan, Shawnette Rochelle, Rod-
dran Grimes, Maura Pocasangre and Didi Orbay.
v
LIST OF TABLES
Page
1.1 Summary Statistics of Russian Facebook Ad Dataset . . . . . . . . . . . . . 12
1.2 Some frequent landing pages . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
1.3 Common interests of gender-directed ads . . . . . . . . . . . . . . . . . . . . 15
1.4 Campaigns identied in the dataset and parties associated with them. . . . . 23
1.5 Average values between more eective and less eective. Signicance of
the feature as denoted by *,**,***,**** correspond to p-values less than
0.05,0.1,0.001 and 0.0001 respectively. . . . . . . . . . . . . . . . . . . . . . . 25
1.6 Statistics of the campaign arranged in decreased order of eectiveness. . . . 26
1.7 Performance of the two parties. . . . . . . . . . . . . . . . . . . . . . . . . . 28
2.1 US Senate Seats Up for Election in 2018 . . . . . . . . . . . . . . . . . . . . 58
2.2 Datasets Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
2.3 Top 10 hashtags: liberals, conservatives, humans, bots . . . . . . . . . . . . . 60
vi
2.4 General Midterms DS3: bot and human population by State (sorted by
percent-wise bot prevalence). . . . . . . . . . . . . . . . . . . . . . . . . . . 63
3.1 Dataset statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
3.2 Users and tweets statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
3.3 Top 20 hashtags generated by liberal and conservative bots. Hashtags in bold
are not present in the top 50 hashtags used by the corresponding human group. 73
3.4 Average network centrality measures . . . . . . . . . . . . . . . . . . . . . . 76
3.5 Bot eectiveness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
4.1 Bots Eectiveness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
5.1 Diusion Evaluation IC or Baseline . . . . . . . . . . . . . . . . . . . . . . . 113
5.2 Diusion Evaluation LT or Modied . . . . . . . . . . . . . . . . . . . . . . 113
vii
LIST OF FIGURES
Page
1 Internet Research Agency . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.1 Distribution by Impressions, Clicks and Target Age Group . . . . . . . . . . 12
(a) Impressions vs Ad . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
(b) Clicks vs Ad . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
(c) Target Age Group vs Ads . . . . . . . . . . . . . . . . . . . . . . . . 12
1.2 Distribution of Clicks and Impressions . . . . . . . . . . . . . . . . . . . . . 13
(a) Impressions vs Ad . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
(b) Clicks vs Ad . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
(c) Correlation between clicks and impressions . . . . . . . . . . . . . . 13
1.3 Age-wise and timeline distribution of the advertisements . . . . . . . . . . . 13
(a) Age group targeted by the ads. . . . . . . . . . . . . . . . . . . . . . 13
(b) Timeline of the advertisements . . . . . . . . . . . . . . . . . . . . . 13
viii
1.4 Money spent on purchasing the Russian ads . . . . . . . . . . . . . . . . . . 16
(a) Frequency Distribution of money spent . . . . . . . . . . . . . . . . . 16
(b) Temporal distribution of money spent . . . . . . . . . . . . . . . . . 16
1.5 Emotion word counts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
1.6 Best performing ads for each party . . . . . . . . . . . . . . . . . . . . . . . 30
(a) Democratic highest clicks (56K) and impressions (968K) . . . . . . . 30
(b) Democratic highest CTR (84.42%) . . . . . . . . . . . . . . . . . . . 30
(c) Democratic highest cost ($1,200) . . . . . . . . . . . . . . . . . . . . 30
(d) Republican highest clicks (73K) and impressions (1.33M) . . . . . . . 30
(e) Republican highest CTR(28.16%) . . . . . . . . . . . . . . . . . . . . 30
(f) Republican highest cost ($5,317) . . . . . . . . . . . . . . . . . . . . . 30
1.7 Distribution by Impressions, Clicks and Target Age Group . . . . . . . . . . 31
(a) Ad Distribution by Cost . . . . . . . . . . . . . . . . . . . . . . . . . 31
(b) Ad Distribution by Count . . . . . . . . . . . . . . . . . . . . . . . . 31
1.8 Timeline of Campaigns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
2.1 Screen shot of the United States trends on election day showing the #ivoted
hashtag trending with 200K tweets. . . . . . . . . . . . . . . . . . . . . . . . 42
2.2 Bot Score Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
ix
2.3 Political ideology dierence, in terms of percentage of liberals vs. conserva-
tives, between DS5 and DS3 . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
2.5 #ivoted tweet from New York . . . . . . . . . . . . . . . . . . . . . . . . . . 61
2.6 #ivoted tweet from Florida . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
3.1 Bot score distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
3.2 Political discussion over (a) the 10-core, and (b) the 25-core decomposition
of the retweet network. Each node represents a user, while links represent
retweets. Links with weight (i.e., frequency of occurrence) less than 2 are
hidden to minimize visual clutter. Blue nodes represent liberal accounts,
while red nodes indicate conservative users. Darker tones (blue and red)
depict bots, while lighter tones (cyan and pink) relate to humans, and the few
green nodes represent unclassied accounts. The link takes the same color of
the source node (author of the retweet), whereas node size is proportional to
the in-degree of the user. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
3.3 Interactions according to political ideology . . . . . . . . . . . . . . . . . . . 76
3.4 k-core decomposition, liberal vs. conservative users . . . . . . . . . . . . . . 78
4.1 Users tweet activity in 2016 (left) and 2018 (right). (A): Consecutive tweet
inter-time for bots and humans (B): Cumulative sentiment over time of human
and bot tweets (C): Cumulative sentiment over time of human and bot tweets
according to their political leaning. . . . . . . . . . . . . . . . . . . . . . . . 89
x
4.2 Human and bots interplay. (A): Retweets (a), replies (b), and mentions (c)
between bots and humans in 2016 and 2018. Percentages represent the amount
of interaction between each class of users. (B): Human-Bot retweet interaction
according to their political leaning in 2016 (a) and 2018 (b). Percentages
indicate the group propensity to endorse (retweet) other groups. . . . . . . . 91
4.3 Volume of Bot-Human, Human-Bot, and Bot-Bot retweet interactions in 2018 93
4.4 Bots eectiveness in 2016 and 2018. (Left): RTP and RR distributions.
(Right): Information spread of bot generated content. . . . . . . . . . . . . . 94
5.1 In
uence Maximization Task . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
5.2 Two Traditional Information Diusion Models . . . . . . . . . . . . . . . . . 108
5.3 Gullibility and Conformability . . . . . . . . . . . . . . . . . . . . . . . . . . 110
5.4 This is how a small LFR network looks like . . . . . . . . . . . . . . . . . . . 112
5.5 Diusion vs. Budget . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
5.6 Diusion vs. Time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
5.7 Convergence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115
xi
Abstract of Dissertation
Modern society is a very complex system of people, institutions and ideas where individuals
and organizations can become polarized into subgroups with con
icting ideology. With the
widespread use of social media today, people's perception of reality can be in
uenced by
the messaging content to which they are exposed and the source from whom it comes. To
that end, I developed a framework for modeling information operations and diusion dy-
namics on social media networks. We present the context of Information Operations in the
Multi-Domain Battle (MDB) paradigm which includes land, sea, air, space and cyberspace.
As part of cyberspace, social media platforms have been used to spread false or misleading
information eectively, particularly because of how easy and how fast it is to reach a consen-
sus of users. After analyzing the 3,500 Russian Facebook ads purchased during the 2016 US
Presidential campaign, we found that the liberal and conservative ads were just as eective
and knowing that these ads came from the same group, we determined that the purpose was
to sow chaos. However, with more sophisticated approaches, platforms and researchers have
targeted the automated, inorganic user accounts. Therefore, the algorithmically controlled
account users will have to adapt, and this research is one of the rst attempts at targeting
their adaptation. We studied 245K accounts that posted during the 2016 and 2018 elections,
and bot behavior in 2016 was mechanical with a lot of retweets, but in 2018 bots aligned with
human activity trends which might indicate bots are evolving. Likewise on the human side,
there was a noticeable shift from retweets to replies which might indicate a desire to discuss
their ideas instead of just retweeting other user's content. We contend that the organized
campaigns could benet from an approach in In
uence Maximization (IM) using algorithms
that have theoretical guarantees over social media networks using various network diusion
models. This task is combinatorial complex, both approximation algorithms for the IM task
and Monte Carlo simulations for the diusion models must be utilized. Under this frame-
work, we addressing gullible and stubborn agents by incorporating user information from
xii
a content-based and network-based approach could provide increased speed and reduced
computations, while only marginally sacricing the spread eectiveness.
xiii
Introduction
The photo in Figure 1 is of the Internet Research Agency located at 55 Savushkina Street
in Saint Petersburg, Russia. A state-sponsored troll farm for manipulation campaign during
the last US Presidential election. The sign in red says \for lease" in Russian. Founded in
the mid 2013 and referred to as Glavset, they engaged in in
uence operations and started
to advocate for then-President-elect Trump as early as December 2015" as attempting to
in
uence the 2016 United States presidential election. More than 1,000 employees reportedly
worked in a single building of the agency in 2015.
16 February 2018, a United States grand jury indicted 13 Russian nationals and 3 Russian
entities, including the Internet Research Agency, on charges of violating criminal laws with
the intent to interfere "with U.S. elections and political processes", according to the Justice
Department.
The Internet Research Agency was indicted by a United States grand jury on charges of vio-
lating criminal laws with the intent to interfere "with U.S. elections and political processes"
[1]. We corroborated these ndings with the three empirical studies. First, after analysing
the 3,500 Russian Facebook ads purchased during the 2016 US Presidential campaign, we
found that the liberal and conservative ads were just as eective and knowing that these
ads came from the same group, we determined that the purpose was to sow chaos [2]. In
a follow-up study of Twitter, we determined that during the 2018 US Midterm Elections,
conservative bots where 10 percentage points more eective than liberal bots across a num-
ber of metrics and more importantly humans retweeted bot tweets over one-third of time
[3,4]. Finally, we studied 245K accounts that posted during the 2016 and 2018 elections, and
bot behavior in 2016 was mechanical with a lot of retweets, but in 2018 bots aligned with
human activity trends which might indicated bots are evolving. Likewise on the human side,
there was noticeable shift from retweets to replies which might indicate a desire to discuss
their ideas instead of just retweeting other user's content [5]. Therefore, there is a need
1
Figure 1: Internet Research Agency
to secure the US elections which is now on the Department of Homeland Security Critical
Infrastructure list.
Multi-Domain Battle
Since World War I, the main domains where war has been waged and peace has been kept
were air, land and sea. Space as a domain was rst formulated as a command in 1985, with
US Space Force becoming a separate service in 2019. With the formation of United States
Cyber Command (USCYBERCOM) in 2009, there is now the addition of cyberspace as a
domain. USCYBERCOM was elevated to a unied combatant command in 2018 making it
the 11th of such combatant commands. So there are the 5 joint domains. The U.S. and their
Allies are also contested in the electromagnetic spectrum, the information environment, and
the cognitive dimension of warfare. At the same time, advances in computational power has
created a growing evolution for the ability to collect, process and make decisions using vasts
amounts of data. In this chapter we will cover the ve domains which comprise the Multi-
Domain Battle (MDB) concept as well as the three tenants of MDB. We will only address
brie
y about the historical air, land and sea while mentioning space; however, we will explore
2
cyberspace in greater depth. From a machine learning and game theory perspective, we will
provide an broad overview of the eld We present will then propose a framework for modeling
adversarial games in a multi-round setting with applications to cyber security.
Each service such as the Army, Air Force, Navy, Marine Corps and the Coast Guard has
tried to approach the challenges independently across the ve operational domains. How-
ever, as advancements in cyber, which are specically cross-cutting in nature will require
whole of government solutions. This is because problems in the cyber domain need to have
cybersecurity woven in and not have cybersecurity as an afterthought. The Multi-Domain
Battle is a concept for a new world where the US's adversaries have studied the battleeld
success of the past and seek to disrupt that advantage. This concept was the brainchild of
the Army's Training and Doctrine Command (TRADOC) and the Air Force's Air Combat
Command (ACC), which are collaboratively integrated and converging of solutions [63].
The keystone of the layered stando problem is the rapid and continuous integration of all
domains of warfare. Army forces will need to engage across all domains to include space
and cyberspace and additionally the electromagnetic spectrum and information environment.
For this chapter, the cyberspace domain will be the focus in the context of the information
environment with direction by the Army Network, one of the six Army Materiel Modern-
ization Priorities. Even outside of the Army network, the information warfare engaged by
revisionist adversaries include cyber reconnaissance and strike actions that support other
reconnaissance [132].
Multi-Domain Battle: Evolution of Combined-arms Operations for the 21st Century [131] is
focused on a 15-year time window between 2025 and 2040. The Forward, dated December
2017 by General Perkins states that the operating environment has become more contested
and more lethal in the last 20 years. While he penned the Forward while serving as the
Commanding General of the Army Training and Doctrine Command (TRADOC), he would
have rst hand experience as the then-Brigade Commander of the rst Brigade to reach
3
Baghdad, Iraq during the US invasion in 2003 [149].
The former Army Operating Concept which Multi-Domain Battle replaced, had the inability
to deal with adversaries demonstrating asymmetric capabilities improvement, in part due to
the ease of which the commercial sector, not government is driving technological advances.
Therefore, there is a need for a threat base analytic approach that considers not only the
US military but also partner and host Nations. This is the concept of multi-domain battle.
The purpose of the multi-domain battle concept is to initiate the changes and designs for
a future Army to counter adversary eorts. Multi-domain battle must exploit all domains
across a vast operational framework as it must include whole-of-government approaches.
Again multi-domain battle extends the battle space particularly with respect to space and
cyberspace as the most recent additions to the original domains.
There's three main components for the solution that collectively will be known as the multi-
domain battle rst is calibrate Force posture second is employee resilient formations in third
is to cover capabilities. the calibrating Force posture is to design to defeat the hybrid War buy
hybrid War remain where there are military and non-military elements involved. for example
North Korea attacking the US base company Sony in releasing damaging emails in response
to video that was disparaging of their leader. Resilient formations mean semi-independent
and their operation. Could easily become semi-autonomous within their domain. More
importantly across the domains and across the government and allies reducing the number
of decisions that have to be made. and so either by cascading the outcome we can initiate
a sequence of events so that they are
National Defense Strategy
The US Department of Defense's (DoD) mission was to deter war and should deterrence fail,
the DoD is prepared to ght and win. However, as June 2018, President Donald Trump's ad-
ministration has been updated from \deter war" to \sustain American in
uences abroad."
4
The National Defense Strategy (NDS) is an annual classied document that provides the
DoD's perspective based on the National Security Strategy (NSS) produced by the US Pres-
ident. The NDS is from the perspective of the Secretary of Defense and there is an unclassied
summary of the classied 2018 National Defense Strategy which addresses countering adver-
saries in every operating domain. This document is not to be confused with the National
Military Strategy which is from the perspective of the Joints Chiefs of Sta for which there
is no unclassied summary. To that end, revisionist powers and rogue regimes are leveraging
military modernization and conducting in
uence operations. As such, the long enjoyed US
military advantage over near-peers to generally deploy forces uninhibited across the domains
of air, land, sea, space and cyberspace is now contested. New commercial technologies are
needed to address this problem and will require changes to the National Security Innovation
Base (NSIB). More specically, with space and cyberspace as warghting domains, there is
a need to invest in cyber defenses while continuing to integrate cyber capabilities across the
full spectrum of military operations. [90]
Multi-Domain Operations
Converging capabilities is a new idea introduced in Multi-Domain Battle as an evolution of
combined arms. Convergence is the act of applying a combination of capabilities (lethal and
nonlethal, whether within a domain or cross-domain) in time and space for a single purpose.
In competition, these adversaries will also employ sophisticated combinations of combined
arms that include the use of space and cyberspace operations, economic in
uence, political
shaping, information warfare, and lawfare to control the escalation and de-escalation of crises
in ways that undermine U.S. in
uence and delay U.S. reaction times. Lawfare is dened as a
strategy of using|or misusing|law as a substitute for traditional military means to achieve
an operational objective.
Convergence in competition, however, also includes the ability for an adversary to immedi-
ately turn globally common or friendly sovereign territory into \denied" areas. Positions of
5
advantage also exist in the non-physical areas of information, cyberspace, and the cognitive
dimension of warfare. Examples of this include human reconnaissance or resistance networks
built, over time, in likely operating areas for key enemy assets; openings detected, but not
yet exploited, in enemy networks; or a timely information narrative built on credible action
[131].
Background
Since the 2016 US Presidential election, there has been a big spotlight on the sovereignty of
the US election system. The article The Rise of Social Bots [39] brought awareness to the
issue of social bots in social media platforms. In [12], Bessi & Ferrara focused on social bots
detection within the online discussion related to the 2016 presidential election. Other than
characterizing the behavioral dierences between humans and bots, there was not an in-depth
analysis of any malicious intent. As a follow up, Badaway et al. [6] investigated Russian trolls
on Twitter to analyzed their in
uence campaign which Im et al. [67] provides indications
that the troll accounts remain. Since then, social media networks have been trying to ght
malicious actors to maintain an healthy conversation on their platform. However, nefarious
activity on social media does not show any sign of decline. Our aim here is to investigate
the evolution of social media actors through the lens of the last two US nation wide political
elections. Social media mass manipulation of public opinion is based on disinformation
campaigns [8, 16, 37, 57, 64, 118, 120, 136] and social media bots [12, 15, 99, 108, 119, 134,
141, 142]. Initial analysis from 2018 US Midterm elections contends that number of bots
have not reduced nor have tried to stop impersonating humans [31, 86].
Facebook.com was started by Mark Zuckerberg in late 2003. The company had its Initial
Public Oering in May 17 of 2012 for $38 a share. As of this week as of this writing there
was 2.5 billion monthly active users (MAUs) and the 52 week high share price was close
to $250. On February 4, 2000 for Mark Zuckerberg ocially launched the Facebook at the
Facebook.com. The success is despite still being banned in China since 2009.
6
On October 31, 2017 Facebook as well as Google and Twitter's legal counsel gave testimony
to the Senate Judiciary subcommittee. Collins Stephens, vice president and general counsel
for Facebook, received the brunt of outrage from Al Franklin the Senator from Minnesota
November 1, 2017. In total 29 million people or serve content from the 80,000 post from the
Internet research agency in the two year time span between 2015 and 2017.
7
Chapter 1
Analysis of Ads Campaign in US
Elections
1.1 Introduction
One of the key aspects of the United States democracy is free and fair elections, unhindered
by foreign in
uence, that allow for a peaceful transfer of power from one President to the
next. Campaign Finance laws forbid foreign governments or individuals from participating
or in
uencing the election. The 2016 US presidential election stands out not only due to
its political outsider winner, Donald J. Trump, but also due to suspected foreign in
uence
before and during the election. It is alleged that the Russian Federation operated the Main
Intelligence Directorate of the General Sta, a military intelligence agency. This agency
is suspected of in
uencing the election with resources allocated towards social media on a
variety of forums.
Corporate leadership and council from Google, Twitter, and Facebook testied on November
1, 2017 to the Senate Intelligence Committee concerning social media in
uence on their
8
platforms. While Facebook's General Council suggested that it would be dicult to verify
that every ad purchased on their platform adheres to US campaign nance laws, intuitively
ads purchased in Russian Rubles would be highly suspicious. There were approximately
3,500 ads identied by Facebook that met such criteria totaling close to $100K and purchased
between June 2015 and August 2017. The surfacing of these ads contributed to Facebook's
CEO Mark Zuckerberg's testimony of 10-11 April 2018 to a number of Senate and House
Committees.
1
Rep. Adam Schi, the ranking member of the House Intelligence Committee, had voiced his
opinion that the Russians had launched an `independent expenditure campaign on Trump's
behalf, regardless of his involvement.
2
However, as emphatically stated by Rob Goldman,
3
the vice president of ads at Facebook, the over-arching aim of the advertisements was to
bring about discord among dierent communities in the United States. In Goldman's words,
\(the ads) sought to sow division and discord" in the political proceedings before, during, and
after the 2016 US elections by leveraging the freedom of free speech and pervasive nature of
social media. This statement is contradictory to the claim that the primary objective of the
advertisements was to in
uence the eect of the 2016 elections and sway it in favor of Trump
or to vilify Clinton. Under the direction of Democrats on the House Intelligence Committee,
the Russian Facebook ads were released to the public on May 10, 2018. The main objective
of this work is to apply language analysis and machine learning techniques to understand
the intent of those ads by exploring their eectiveness from a campaign perspective.
1
https://www.judiciary.senate.gov/meetings/facebook-social-media-privacy-and-the-use-and-abuse-of-
data
2
http://www.businessinsider.com/russian-facebook-ads-2016-election-trump-clinton-bernie-2017-11
3
https://www.cnbc.com/2018/02/17/facebooks-vp-of-ads-says-russian-meddling-aimed-to-divide-
us.html
9
1.2 Objective and Plan of Action
By taking a data-driven perspective, we seek to understand the goals and tactics enacted
by our adversary to manipulate social media conversation. We will focus on charting the
landscape of narratives they pushed and quantify their ecacy with the targeted population.
We will study the Russian Internet Research Agency (IRA) ads campaign on Facebook, whose
data was released in the context of the ocial investigations.
1.3 Related Literature
Since the early 2000s, there has been increasing research in the new domain of computational
social science [81]. Most of the literature has focused on networked in
uence, information
(or misinformation) diusion, and social media association with real-world events [58]. As
it concerns our research eorts, related work focuses on social media use in politics as well
as campaign detection.
Politics in Social Media: Concerning divisive issue campaigns on Facebook, ongoing
work has explored the organizations and objectives behind the Russian ads from a political
communication standpoint. Kim [74] stated that suspicious groups which could include
foreign entities are behind many of the divisive campaigns. Additionally, approximately
18% of the suspicious groups were Russian. The authors asserts that there are shortcomings
in federal regulations and aspects of digital media that allow for anonymous groups to sow
division [74]. While Kim approaches the issue from a policy perspective, we focus more
on the eectiveness and organization of the ads themselves. While the data we used in
this research is specic to only the Russian Facebook ads, we present a methodology that
could be extended to automatically sort any ads into their divisive campaigns. Previous
work established that social media platforms were exploited during the 2016 US Presidential
10
Election [12, 2, 141, 7], as well as numerous other elections [109, 93, 65, 37, 32, 123] and
other real-world events [36, 38], by using tools like bots and trolls [39, 30, 133].
Campaign Detection: In order to combat misinformation, it is necessary to understand
the characteristics that allow it to be eective [91]. In addition to misinformation, divisive
information which creates polarized groups is counter to what the political system or a
democratic nation needs to thrive [125]. Previous campaign detection has been focused on
spam [33] and malware [117, 77] in order to protect computer information systems. The
most relevant work for campaign detection on social media is by Varol and collaborators
[135, 40]. They use supervised learning to categorize Twitter memes from millions of tweets
across a series of hashtags. In comparison to that work, we focus at the microscopic level
on paid Facebook ads determined to be from the same source. In addition to looking at the
Russian campaign messaging and content, we are able to factor cost and eectiveness into
our analysis.
1.4 The Data
The dataset comprises 3,516 advertisements with 22 variables as released by the Data for
Democracy organization in csv format.
4
The data was released under the direction of the
Democrats on the House Intelligence Committee. The ads were released to the public on May
10, 2018. The ads were purchased in Russian Rubles during the 2016 US Presidential election
and beyond from June 2015 to August 2017. In analyzing eectiveness, we only considered
ads which were viewed by at least one person (impressions greater than zero). In analyzing
campaigns, all ads with non-zero impressions or those which were purchased in Rubles were
considered. Our dataset consists of the Russian Facebook paid ads totaling $93K. Again,
the data was initially provided by Facebook, so there is no way to independently verify its
4
https://data.world/scottcame/us-house-psci-social-media-ads
11
completeness and ads purchased in Rubles would be a lower bound to all ads purchased on
behalf of the broader operation. Summary statistics of the data are shown in Table 1.1.
Table 1.1: Summary Statistics of Russian Facebook Ad Dataset
Criteria Count Value Analysis
All ads 3,516 $100K Individual
Ads with at least 1 impression 2,600 $93.0K Eectivness
Ads with at least 1 impression AND paid in RUB 2,539 $92.8K Campaign & Party
Clicks and Impression counts: Clicks and impressions are important metrics to under-
stand the outreach and ecacy of the advertisement. Clicks, or link clicks, quantify the
number of people who have clicked on the ad and was redirected to the particular landing-
page. Impressions refer to the total number of times the advertisement has been shown. It
diers from Reach which re
ects instead the number of individual people who have seen the
ad. We present the distribution of impressions and clicks for the FB ads in 1.1a and 1.2,
respectively. It is clearly evident that a majority of ads have attained sucient outreach
and popularity. We observe that a huge fraction of the ads are targeted to the younger age
group as seen in 1.1c.
0
1
10
100
1000
10000
100000
Impressions
0.00
0.05
0.10
0.15
0.20
0.25
No of Ads
(a) Impressions vs Ad
0
1
10
100
1000
10000
Clicks
0.00
0.05
0.10
0.15
0.20
0.25
0.30
No of Ads
(b) Clicks vs Ad
13
18
23
28
33
38
43
48
53
58
63
Targeted Age
0.00
0.02
0.04
0.06
0.08
0.10
0.12
Fractional Distribution
(c) Target Age Group vs Ads
Figure 1.1: Distribution by Impressions, Clicks and Target Age Group
1.5 Data Analysis
Clicks and Impression counts: Clicks and impressions are important metrics to under-
stand the outreach and ecacy of the advertisement. Clicks, or link clicks, quantify the
12
number of people who have clicked on the ad and was redirected to the particular landing-
page. Impressions refer to the total number of times the advertisement has been shown. It
diers from Reach which re
ects instead the number of individual people who have seen the
ad.
(a) Impressions vs Ad (b) Clicks vs Ad (c) Correlation between clicks and
impressions
Figure 1.2: Distribution of Clicks and Impressions
We present the distribution of impressions and clicks for the FB ads in 1.2(a) and 1.2 (b)
respectively. It is clearly evident that a majority of ads have attained sucient outreach and
popularity. We also see there exists a strong correlation (r=0.8927) between the impressions
and clicks as seen in the Figure 1.2(c).
Target Age groups: We observe that a huge fraction of the ads are targeted to the younger
age group as seen in 1.3 (a).
(a) Age group targeted by the ads. (b) Timeline of the advertisements
Figure 1.3: Age-wise and timeline distribution of the advertisements
Time-line analysis: We also observe the specic time-line when the advertisements were
posted as shown in 1.3(b). Although several advertisements were
oated during the election
13
Landing Page #Ads Landing Page #Ads
https://www.facebook.com/blackmattersus/ 291 https://www.facebook.com/LGBT-United/ 95
https://www.facebook.com/Black-Matters-/ 260 https://www.facebook.com/patriototus 83
https://www.facebook.com/WilliamsandKalvin/ 233 https://www.facebook.com/Woke-Blacks-/ 81
https://www.facebook.com/brownunitedfront/ 204 https://www.facebook.com/savethe2a/ 68
https://www.facebook.com/blacktivists/ 159 https://www.facebook.com/Stop-Al/ 63
https://www.facebook.com/Memopolis/ 154 https://www.facebook.com/PanAfrootsmove/ 60
https://www.facebook.com/Blacktivist/ 132 https://www.facebook.com/MuslimAmerica/ 54
https://www.facebook.com/Dont-Shoot/ 109 https://www.facebook.com/stoptherefugees/ 50
https://musicfb.info/ 107 https://www.facebook.com/StopAllInvaders 49
Table 1.2: Some frequent landing pages
period of 2016, a substantial portion was posted both prior to and after the elections.
Language Distribution A majority of the advertisements (54.29%) did not have a valid
language eld. To circumvent this predicament, we leveraged the language detector library
of TextBlob
5
, which is powered by the Google Translate API to identify the language of the
advertisement text.
6
. Unsurprisingly, almost all the advertisements (98.349%) are conveyed
in English, with small fractions in Arabic (0.341%), Spanish (0.284%) and other languages
(0.142%, comprising Italian, Hindi, Portugese, etc). There are also 32 ADs whose language
could not be identied due to absence of any text in the advertisement. Overall, it is evident
that most of the ads were targeted towards the English speaking population, which comprises
91.4% of the American population.
7
Landing Page Analysis We present here an overview of the common landing pages that
had posted these advertisements. Surprisingly, we found several pages which had posted
advertisements on more than one occasion. We illustrate some popular pages in the table
1.2.
Gender-wise Distribution A majority of the ads(95.762%) did not have a valid gender
eld, while the ads directed towards Males and Females accounted for 3.100% and 1.137%
respectively. The frequent interests associated with the advertisements which are directed
towards a particular gender are shown in Table 1.3. We observe that the ads directed exclu-
5
http://textblob.readthedocs.io/en/dev/
6
https://cloud.google.com/translate/docs/
7
http://blog.languageline.com/limited-english-procient-census
14
sively towards males talks about gun rights and the second amendment, the ones directed
exclusively towards women had light-hearted topics like music and funny pictures, while the
rest referred to serious issues that aected both namely racism, police accountability.
Males Females Both
gun rights music african-american history
anything about guns funny pictures african-american civil
rights movement (1954-68)
guns and patriots funny pics martin luther king jr
guns lol malcolm(x)
free music free software la raza
national ri
e association funny photo's cop block
2nd amendment black economic empowerment chicano movement
interest expansion
Table 1.3: Common interests of gender-directed ads
Currency Analysis Facebook
8
enables businesses and entrepreneurs to market their ideas
and specify the corresponding target audience for a requisite price. Higher the cost of
the advertisements, greater is the outreach. Consequently, it is not surprising to observe
that a majority of these advertisements (69.94%) were paid. Apart from 4 isolated ads,
the currency for facilitating this transaction was in Russian Rubles (RUB). The total cost
involved/ expended was a staggering 4,912,799.70 rubles, which amounts to $78,604 USD.
We represent the frequency distribution of the money spent on the ads in Figure 1.4 (a)
and acknowledge that a signicant portion (43.714%) of them have paid between 100-1000
rubles. Similarly, in Figure 1.4 (b), we observe that even after the completion of the 2016
elections, money still poured into the ads, indicating that the primary objective of the ads
might not have been to in
uence the electoral results.
Since the aim of targeted promotion is to popularize ads and increase its outreach, it is
unsurprising to observe a decent correlation between the number of clicks / impressions and
the money spent on the ad (r=0.615 and r=0.592 respectively).
Textual and Image Analysis A signicant proportion of the advertisements have textual
and image information embedded in them (99.08% and 88.51% respectively). We lever-
8
https://www.facebook.com/business/products/ads
15
(a) Frequency Distribution of money spent (b) Temporal distribution of money spent
Figure 1.4: Money spent on purchasing the Russian ads
age this information to analyze the dierences in eectiveness between the high and low
performance ads, which we outline in the next section.
Other Metadata Some of the advertisements have other metadata information associated
with them like, political leaning (1.649%), behaviors(9.812%), custom audience (0.853%),
people who match(12.116%) etc. Although essential, such information is present only in small
proportions. The percentages indicate the proportion of ads which had the corresponding
valid metadata information. Consequently, we disregarded this corresponding information.
1.6 Research Framework and Research Questions
In this paper, we dene our research framework tackling the following problem:
Research Question: What features are associated with the engagement of the Russian
Facebook ads and what was their impact (i.e., how eective were they) at a campaign-wise
(operational), and on a party-wise (strategic) basis?
To operationalize this question we split it into three sub-parts:
16
1. What features are associated with the engagement of the Russian Facebook
ads. Denition of engagement: To quantify engagement, we estimate how likely a
person would respond to an ad when it is shown to them. The metric we use is Click-
Through Rate (CTR). We approach the problem by classifying ads which have non-zero
impressions in two groups, namely more eective and less eective ads. The classication is
done using a decision rule where the median value of the CTR across all ads is the threshold.
We consider non-zero impressions only since we cannot evaluate the eectiveness of an ad
that was not seen.
We then analyze the stylistic and textual features of the Ads between the two categories,
using dierent natural language processing techniques. The features include sentiment, emo-
tion, structural content, parts of speech distribution, named entity distribution, and linguistic
categories. We note those features which show signicant dierences across categories.
2. What was the Ads impact (eectiveness) at a campaign-wise (operational)
level? Denition of eectiveness: At the campaign level we dene eectiveness as the
audience reach ecacy. The metric we use is Cost Per Thousand Impressions (CPM) and
Cost Per Click (CPC) (explained in methodology). We approach this question by clustering
the ads into the various campaigns and using CPM to determine the most and least eective
campaigns as well as any insights from the associated features mentioned in the rst sub-part.
3. What was the Ads impact (eectiveness) at a party-wise (strategic) level?
Denition of eectiveness: We dene the eectiveness at the party level by observing
signicant dierences in terms of CTR, CPM and total cost between the parties. We create
parties by manually labeling the ads into Democratic (Blue), Republican (Red) and Neutral
(Green). We exclude the Neutral campaigns and assess the eectiveness of the Blue and Red
parties and report any signicant ndings from a feature-wise perspective.
17
It is notable to mention our assumption that all of these ads within the campaigns and
parties were generated by the same alleged organization in Russia.
1.7 Methodology
1.7.1 Features of Eective Ads
The eectiveness of ads at the individual level is measured using CTR.
Click-Through Rate (CTR). CTR of a particular advertisement is the ratio of clicks to
impression for the ad expressed as a percentage. CTR re
ects the creativity and compelling
nature of the advertisement [28].
CTR =
#Clicks
#Impressions
100(%) (1.1)
The stylistic and textual features associated with the ads we analyzed are:
Sentiment Analysis: Sentiment analysis helps to identify the attitude of the text and
gauge whether it is more positive, negative or neutral. Based on the comparative analysis of
in [111], we utilized 2 methods to determine sentiment on the Ad text to obtain the overall
compounded sentiment score of the Ad. VADER: Valence Aware Dictionary for Sentiment
Reasoning [66] is a rule-based sentiment model that has both a dictionary and associated
intensity measures. Its dictionary has been tuned for micro-blog like contexts. We also
observe the categories corresponding to positive and negative emotions by performing LIWC
[103] analysis on the Ad Text.
Emotion Analysis: We leverage the NRC lexicon of [98] to calculate the average number
of words corresponding to an emotion per advertisement. The associated 8 emotions include
18
Trust
Fear
Sadness
Anticipation
Anger
Joy
Surprise
Disgust
0.00
0.25
0.50
0.75
1.00
1.25
1.50
1.75
2.00
Mean count per ad
Figure 1.5: Emotion word counts
anger, anticipation, joy, fear, trust, disgust, sadness and surprise.
Structural Content: The structural content of the text refers to the distribution of sen-
tences and words per advertisement. An ad's eciency often correlates with the amount of
textual content [113].
POS-TAG distribution: We employed the inbuilt Part-of-Speech (POS) tagger of NLTK
[130] and the Penn Tree Bank
9
to observe the distribution of dierent POS (Parts of Speech)
TAGS in the advertisement texts.
Named Entity Recognition (NER) distribution: The high proportion of proper nouns
from POS TAG analysis signies that the ads cater more to real-world events. Consequently,
we also inspected the distribution of dierent named entities using the Perceptron-based
NER of Stanford CoreNLP [89] pertaining to "PERSON", "ORGANIZATION" and "LO-
CATION".
Linguistic Inquiry and Word Count (LIWC) Analysis: LIWC [103] computes the
proportion of words in psychologically meaningful categories for the analyzed text which we
leverage to discover dierent linguistic and cognitive dimensions.
9
https://www.ling.upenn.edu/courses/Fall 2003/ling001/penn treebank pos.html
19
1.7.2 Campaign-Level Analysis
We leverage dierent methods to cluster Ads into non-overlapping campaigns:
LDA Topic extraction: We implemented LDA [13] (Latent Dirichlet Allocation) using
the in-built gensim model of [110] on the corpus of advertisement text to obtain a list of
50 topics. However these topics had several overlapping words and dealt with racism, gun-
control or police accountability. It also failed to capture broad topics like homosexuality or
immigration.
Key-word/ Key-phrase extraction: We also employ RAKE (Rapid Automatic Keyword
Extraction) [115], an unsupervised, domain-independent and language-independent tech-
nique to extract keywords from the advertisement texts. This methodology captures niche
topics since it observes each document individually.
However, the above methods suer two shortcomings. Firstly, they did not take into account
the Ad's images which serves the purpose of propagating ideas mentioned in the advertise-
ment. Secondly, these techniques do not incorporate the context associated outside of the
text. For example, the ad text, "The blue gang is free to do whatever they want" clearly
refers to police brutality, often misdirected at African-Americans, but it is impossible to deci-
pher from the text alone. Hence we resort to a semi-automated network clustering technique
to identify campaigns as described below.
Network-based Clustering: Some of the advertisements have a meta-data eld labeled
Interests which corresponds to topics. We represent each topic as a vector, obtained by
the FastText technique of [95]. We compute similarity between the topics using the given
20
equation:
sim(T
i
;T
j
) =(
~
T
i

~
T
j
) + (1)
jAds(T
i
)\Ads(T
j
)j
min(jAds(T
i
)j;jAds(T
j
)j)
(1.2)
wherein,T
i
andT
j
represent two arbitrary topics,
~
T
i
and
~
T
j
denote their vector representation
and Ads(T
i
) enlists the ads which have T
i
as a topic. The rst part of the equation (
~
T
i

~
T
j
) simply computes the cosine similarity score of the topic vectors, while the second part
calculates the overlap coecient similarity between two topics. While [0,1] determines
the trade-o between the two similarity scores.
Each topic is then represented as a node in an undirected graph with edges representing the
similarity between two topics. We binarize the graph by retaining only the edges above a
certain threshold and cluster it. We experimented with dierent values of, and dierent
algorithms and experimentally veried that the Louvain algorithm [14] with a threshold of
0.9 for both and gave the best results with 9 non-overlapping campaigns. A change
in and values drastically altered the number of communities, ranging from 2-3 on one
extreme to 40-50 in another. Likewise, ML based unsupervised clustering techniques like
KMeans or Spectral Clustering were unable to incorporate the overlap coecient similarity
and hence showed poorer performance.
Thus each topic belongs to one of the initial 9 campaigns. Since an ad can contain several
topics, they can belong to dierent campaigns, we assign them to the campaign that with
the most number of topics, breaking ties arbitrarily.
We then manually inspected the rest of the ads and assigned those which did not have the
Interests eld to one of the 9 campaigns. Sometimes, we had to create new campaigns since
the particular Ad did not conform with any of the previous ones. It was necessary to break
21
up large clusters which had similar notions (police brutality, racism and Black Lives Matter)
into dierent campaigns. Eventually, that yielded the nal 21 campaigns as demonstrated
in Table 1.4.
The eectiveness of ads at the campaign level is measured via CPM and CPC.
Cost Per Thousand Impressions (CPM): CPM for an ad is simply the amount of Rubles
spent to reach a mile (thousand impressions). CPM is primarily determined by the target
audience [10].
CPM =
AdCost(RUB)
#Impressions
1000 (1.3)
Cost Per Click (CPC): CPC for an Ad is the amount of Rubles required to receive a click.
CPC re
ects the trac generated by the ad to the landing page [10].
CPC =
AdCost(RUB)
#Clicks
(1.4)
A campaign's eectiveness is usually measured by a low CPC value because it implies that
the amount of Rubles required to get an audience's response is also less. However a low
CPM is sometimes essential if one wishes to target a particular audience and optimize the
overall cost of the campaign. If an ad itself has a high CTR, purchasing Ads using CPM
may be a better strategy.
The stylistic features analyzed are consistent with those outlined in Section 5.1.
22
1.7.3 Party Clustering
The campaigns are manually assigned to parties as stated in Table 1.4.
Campaign Denition Party
Police Brutality Injustice meted out to the Blacks by the Police Democrat
Entertainment Multi-media sources of entertainment (memes, songs, videos) None
Prison Prison reforms against mandatory sentences, prison privatization Democrat
Racism Acts of racism harbored against any racial minority in America Democrat
LGBT Rights and dignities for the LGBT people Democrat
Black Lives Matter Incarceration, shooting or other acts of cruelty against Blacks Democrat
Conservative Ideals of patriotism, preserving heritage and Republican advocacy Republican
Anti-immigration Preventing illegal immigration across the US borders Republican
Veterans Support for the hapless/ crippled veterans of war Republican
2nd Amendment Supporting the right to bear arms and guns Republican
All Lives Matter Counter to the Black Lives Matter Republican
Anti-war Opposition of wars and acts of aggression against the Middle East Democrat
Texas A medley of Conservative Ads specically leaned towards Texas. Republican
Islam Against Islamophobia and support for the Muslims in the US Democrat
Immigration Support the immigration of other nationalities into America Democrat
Liberalism In support of the various liberal reforms by the Blue part Democrat
Religious Support for the conservative Christians in the US Republican
Hispanic Support for the Hispanic/ Latino community in the US Democrat
Anti-Islam Messaging against the acceptance of Muslims in US Republican
Native Support for the Native American Indians and their community Democrat
Self-Defense Focused on martial arts training for anti-police brutality Democrat
Table 1.4: Campaigns identied in the dataset and parties associated with them.
The eectiveness of ads at the party level is measured using CPC and CPM, in a fashion
similar to the campaign-wise analysis.
1.8 Results
1.8.1 Ad Eectiveness in Aggregate
The calculated median CTR value of the advertisements is 10.24. Consequently, we cat-
egorize the ads as more or less eective if the CTR value is greater or lesser than 10.243
respectively. We present the signicant semantic and textual features here. In all cases,
23
signicant dierence refers to a p-value of 0:001.
Sentiment Analysis: We observe that the overall compounded score is signicantly lower
for the more eective ads than those in the less eective ads, implying that the former
ads tend to be less positive. Surprisingly, there is no signicant dierence between the
distribution of negative sentiments.
Emotion Analysis: None of the 8 emotions showed any signicance dierence across the
two categories, except surprise which demonstrated mild signicance (p-value 0:05).
Structural content: The distribution of sentences and words per advertisement do not
vary signicantly across the two categories.
POS-TAG distribution:We observe that adverbs (RB) and past tense verbs (VBD) occur
more frequently in the more eective ads. This implies that more eective ads tend to refer
to past events more frequently while the pronounced usage of adverbs implies that actions
are explained in detail. However, the proportion of nouns across advertisements is very high,
with NN (common nouns, singular) and NNP (proper nouns, singular) accounting for 6.32
and 5.89 words per advertisement respectively.
NER distribution: The NER analysis revealed that the category "PERSON" occurred in
signicantly higher proportion among the more eective ads.
LIWC Analysis: Only the most signicant LIWC categories have been taken to account
here.
Personalization: Categories belonging to SheHe and Ipron (personal pronouns) are higher
in more eective ads, while those belonging to We and Friends are lower in the more
eective category. This indicates the more eective ads are more personalized or cater to
the individuals rather than the communities.
24
Religion and Money: Religion and Money occur in lower proportions in the more eective
ads than the less eective ones. This shows that religious or nancial divide are not as
successful to ensure engagement.
We now present the dierences in Table 1.5. The columns corresponding to Less Eec-
tive(Mean) and More Eective(Mean) specify the mean value of the distribution for the
categories. The Mean Di column is simply computed
MeanDiff =
High(Mean)Low(Mean)
Low(Mean)
100% (1.5)
The stars beside a category name correspond to the level of signicance as indicated by the
p-value.
Category Less Eective More Eective Mean Di T-value
Compounded sentiment*** 0.139 0.048 -65.699 3.687
Positive sentiment**** 0.19 0.158 -16.61 4.414
Negative sentiment 0.097 0.089 -8.243 -6.281
Surprise** 0.503 0.64 27.4 -2.836
Anger 1.061 1.174 10.648 -1.37
#Sentences 3.768 3.733 -0.939 0.232
#Words 48.008 52.648 9.666 -1.873
RB (Adverb)**** 1.454 2.016 38.698 -4.85
VBD (Verb, past)**** 0.896 1.414 57.834 -4.927
NN (Common nouns) 6.296 6.555 4.111 -0.776
NNP (Proper nouns) 6.361 6.044 -4.983 0.93
PERSON*** 0.017 0.028 61.658 -3.209
LOCATION* 0.012 0.009 -21.195 2.127
Ipron**** 0.028 0.04 43.807 -5.973
We**** 0.033 0.017 -48.568 8.157
SheHe**** 0.005 0.013 148.432 -6.904
Friends**** 0.004 0.001 -64.348 4.588
Money**** 0.01 0.005 -48.167 5.423
Religion**** 0.01 0.004 -61.087 4.424
Table 1.5: Average values between more eective and less eective. Signicance of the
feature as denoted by *,**,***,**** correspond to p-values less than 0.05,0.1,0.001 and
0.0001 respectively.
25
Topics Cost in RUB Cost in USD Frequency Impressions Clicks CPM CPC CTR
Hispanic 164,146.40 2,628.05 186 5,943,904 713,804 27.62 0.23 12.01
Immigration 2,971.30 47.76 10 74,344 10,762 39.97 0.28 14.48
All Lives Matter 150,372.36 2,368.50 11 1,890,020 82,779 79.56 1.82 4.38
Black Lives Matter 1,807,407.97 28,631.85 1206 19,273,576 1,856,476 93.78 0.97 9.63
Entertainment 90,188.75 1,407.42 159 885,273 87,956 101.88 1.03 9.94
Racism 237,900.47 3,677.33 125 1,364,627 82,168 174.33 2.9 6.02
Native 9,397.14 160.94 12 47,428 5,355 198.13 1.75 11.29
Religious 212,647.46 3,543.32 21 1,032,898 78,669 205.87 2.7 7.62
2nd Amendment 234,324.96 3,833.16 50 1,119,281 87,986 209.35 2.66 7.86
Police Brutality 563,945.02 8,873.97 194 2,535,621 207,233 222.41 2.72 8.17
Veteran 220,615.91 3,468.31 97 794,826 59,925 277.57 3.68 7.54
Conservative 831,223.67 13,600.98 116 2,773,169 213,894 299.74 3.89 7.71
Anti-Islam 4,385.58 69.64 3 13,949 2,725 314.4 1.61 19.54
LGBT 303,738.01 4,796.96 95 887,058 82,217 342.41 3.69 9.27
Anti-war 27,469.85 444.45 15 75,517 6,980 363.76 3.94 9.24
Islam 271,567.36 4,271.96 56 581,392 22,033 467.1 12.33 3.79
Liberalism 87,405.43 1,387.71 33 177,089 15,542 493.57 5.62 8.78
Texas 295,043.68 4,698.09 35 589,409 51,400 500.58 5.74 8.72
Prison 13,552.58 215.30 19 25,954 1,981 522.18 6.84 7.63
Self-defense 30,982.02 518.22 25 53,712 2,136 576.82 14.5 3.98
Anti-Immigration 289,898.95 4,432.61 71 419,380 57,865 691.26 5.01 13.8
Table 1.6: Statistics of the campaign arranged in decreased order of eectiveness.
1.8.2 Campaign-wise Analysis
We present the statistics of the dierent campaigns in Table 1.6 which are arranged in
decreasing order of their eectiveness and thus in increasing order of CPM. We demarcate
the campaigns into more and less eective based on the median value of the CPM (A more
eective campaign has a CPM score less than 277.57).
We note the following stylistic dierences between the more eective and less eective cam-
paigns.
Sentiment Analysis: The compounded sentiment score is signicantly lower for the more
eective campaigns since those campaigns involve serious topics like police brutality, racism,
etc.
Emotion Analysis: All 8 emotions, barring surprise, are observed to be signicantly pro-
nounced in the more eective campaigns. We hypothesize that ads evoking emotions are
26
likely to be shared more and hence the impressions increase for the ad, thereby decreasing
the potential CPM.
Structural Analysis: Surprisingly, we note that ads in the more eective campaigns tend
to be of shorter length, i.e more concise.
POS-TAG distribution: POS corresponding to adverbs (RB), plural nouns (NNS, NNPS)
and verbs (VB) occur more frequently in the less eective campaigns, the signicance of which
is unknown.
Named Entity distribution: Named entity mentions corresponding to 'PERSON' is sig-
nicantly higher in the more eective campaigns while 'LOCATION' is higher in the less
eective ones. This nding is attributed to disproportionate large mentions of victims of
racial prejudice in the more eective campaigns. Likewise, the less eective campaigns in-
clude Texas, Anti-Immigration to US, Veterans, etc which directly reference America.
LIWC Analysis: In the category of Religion, the less eective campaigns have a higher
proportion of ads associated with Islam. This conforms the analysis at the individual ad level
that religious ads are less eective. As for Associativity, the more eective campaigns are
also individualistic/personal as opposed to community-driven. This nding is substantiated
by the signicantly high frequency of I and We categories respectively in the more and less
eective campaigns.
We also observe that the individual CPM and cost spent on an ad is signicantly lower for
the more eective campaigns than the less eective ones. Likewise, the number of clicks
and CTR of an individual Ad is signicantly higher for the more eective campaigns. Thus,
more eective ads do contribute to eective campaigns, although the eectiveness metrics
themselves are dierent for the parties and campaigns.
27
1.8.3 Party-wise Analysis
We present a statistical overview of the ads of the two parties in Table 1.7.
Party # Ads Cost Cost Clicks Impressions CPM(RUBs) CPC(RUBs) CTR
Democrat 1,976 3.5MRUB $55.6K 2,995K 31.0M 113.42 1.17 9.69
Republican 404 2.2MRUB $36.0K 647K 8.7M 259.30 3.52 7.36
Table 1.7: Performance of the two parties.
Although there is no signicant dierence in the distribution of clicks and impressions be-
tween the Ads of the two parties, the Democratic party had signicantly higher CTR and
lower CPM values. This implies that the Democratic party was more eective amongst the
two parties.
However, there was also an active involvement in propagating the Republican Ads as well.
This is evident from Table 1.7 which highlights that the disproportionate high amount spent
for the Republican Ads (38.87%) despite their low frequency (17.10%). Moreover, adjudging
from the campaign's time-line in Figure 1.8 Republican Ads occurred for a longer duration.
Finally, the campaigns of the two parties mostly dealt with con
icting or contradictory ideals
(Anti-Islam/Islam, Anti-Immigration/Immigration, All Lives Matter/Black Lives Matter).
This strongly suggests desire to sow discord.
We now present the semantic and textual dierences between the two parties.
Sentiment Analysis: The compounded sentiment score is signicantly lower for the Demo-
cratic party since a majority of the Democratic ads pertain to serious topics like police
brutality, racial tension, anti-war, etc.
Emotion Analysis: The emotion corresponding to sadness is signicantly more pronounced
in the Democratic ads due to the above reason.
Structural Analysis: There was no signicant dierence in the average distribution of
words and sentences between the two parties.
28
POS-TAG distribution: Surprisingly, plural nouns (both common and proper nouns) oc-
cur more frequently in the ads of the Republican party. Adverbs and comparative adjectives
are also more prevalent in the Republican ads.
Named Entity distribution: The fraction of named entities corresponding to Person is
higher in Democratic ads while those corresponding to Location is higher in Republican ads.
This happens since the Democratic ads mention the names of victims of racial prejudice like
Tamir Rice and Eric Garner. Republican ads of patriotism, veterans, and 2nd Amendment
indirectly referenced America.
LIWC Analysis: The category We is more signicantly pronounced in the Republican
party than the Democratic party which might indicate a closer community or inclusiveness.
This may be appealing to the target's sense of belonging.
29
(a)Democratic highest clicks (56K) and
impressions (968K)
(b) Democratic highest CTR (84.42%)
(c) Democratic highest cost ($1,200) (d)Republican highest clicks (73K) and
impressions (1.33M)
(e) Republican highest CTR(28.16%) (f) Republican highest cost ($5,317)
Figure 1.6: Best performing ads for each party
30
black_lives_matter
30.9%
anti-islam
0.1%
conservative
14.2%
native
0.2%
police_brutality
9.6%
anti-immigration
5.0%
anti-war
0.5%
texas
5.0% entertainment
1.5%
islam
4.6%
self-defense
0.5%
lgbt
5.2%
liberalism 1.5%
veteran
3.8%
immigration
0.1%
2nd_amendment
4.0%
prison
0.2%
religious
3.6%
all_lives_matter
2.6%
hispanic
2.8%
racism
4.1%
(a) Ad Distribution by Cost
anti-islam
0.1%
black_lives_matter
47.5%
immigration
0.4%
police_brutality
7.6%
all_lives_matter
0.4%
hispanic
7.3%
native
0.5%
entertainment
6.3%
anti-war
0.6%
racism
4.9%
prison
0.7%
conservative
4.6%
religious
0.8%
veteran
3.8%
self-defense
1.0%
lgbt
3.7%
liberalism
1.3%
anti-immigration
2.8%
texas
1.4%
2nd_amendment
2.0%
islam
2.2%
(b) Ad Distribution by Count
Figure 1.7: Distribution by Impressions, Clicks and Target Age Group
31
Figure 1.8: Timeline of Campaigns
1.9 Conclusions and Future work
In this paper we characterized the Russian Facebook in
uence operation that occurred be-
fore, during, and after the 2016 US presidential election. We focused on 3,500 ads allegedly
purchased by the Russian government, highlighting their features and studying their eec-
tiveness. The most eective ads tend to have less positive sentiment, focused on past events
and are more specic and personalized in nature. A similar observation holds true for the
more eective campaigns. The ads were predominately biased toward the Democratic party
as opposed to the Republican party in terms of frequency and eectiveness. Nevertheless
the campaigns' duration and promotion of the Republican Ads do hint at the eorts of the
32
Russians to cause divide along racial, religious and political ideologies. Areas for future work
include exploring other platforms and similar operations carried therein. For example, we
would like to investigate the connection to Russian troll accounts identied on Twitter, and
conduct campaign analysis to determine the eectiveness of such operations across various
platforms.
33
Chapter 2
Perils and Challenges of Social Media
and Elections
2.1 Introduction
Inherent bias of drawing conclusions from political polls stretch back to the famous headline
of "Dewey Defeats Truman" in the 1948 US Presidential election [101]. Confounding factors
that led to false conclusions in the 1948 election included telephone surveys which did not use
robust statistical methods and an under-sampling of Truman supporters. Likewise, in 2016,
many political pundits underestimated the likelihood that Donald Trump would be elected
as President of the United States. The research community demonstrated a strong interest
in studying social media to get a better understanding of how the 2016 events unfolded.
Numerous studies concluded that social media can be a vehicle for political manipulation,
citing factors such as the eect of fake news and disinformation [106, 64, 120, 136, 8, 59, 16,
118, 57], bots [12, 141, 134, 99, 15, 119, 142], polarization [9, 5], etc.
Research also suggests that social media data comes with signicant biases that limit the
34
ability to forecast oine events, e.g., the outcomes of political elections [94, 45, 49, 46, 47, 48],
or public health issues [80, 3, 140]. Despite these well documented issues and challenges,
social media are frequently relied upon and referred to as a trusted source of information to
speculate about, or try to explain, oine events. One such example is the recent 2018 US
Midterm elections where widespread claims of voter fraud and voter suppression appeared
in the news, often based on social media reports & accounts.
In this paper, we seek to understand whether it is possible to use Twitter as a sensor to
estimate the expected amount of votes generated by each state. We propose an undertaking
in which we use the tweets with the hashtag #ivoted on the election day as a proxy for
actual votes. At rst, this seemed like a promising research direction, as tweet volumes and
vote counts correlated well for 47 of the 50 states in America. We also considered if this
would be a useful approach to detecting voting issues like fraud or suppression, for example
by isolating statistical anomalies in estimated and observed volumes. To get a sense of
expected tweet volume, we carried out the same analysis against general keywords related to
the midterm election from a month before election day through two weeks after the election.
We also considered how bots may have had an in
uence on election manipulation narratives
by measuring their activity in the social media discourse. We nally applied a political
ideology inference technique and tested it to see how well it compared to an external source
of polls data.
The conclusions from our analysis are complex, and this work is meant as a note of caution
about the risks of using social media analysis to infer political election manipulation such as
voter fraud and voter suppression.
35
2.1.1 Contributions of this work
After exploring multiple Twitter data sets and two external sources (vote counts and Gallup),
we came to the following contributions:
We explored how social media analysis carries a lot of risks involved mainly with
population bias, data collection bias, lack of location-specic data, separation of bots
(and organizations) from humans, information verication and fact-checking, and lastly
assigning political ideology.
We saw a signicant dierence in the removal of retweets in our analysis as compared
with including them. However, the eect was isolated to one particular state, Texas,
indicating that the sensitivity of this eect could be a factor of location.
There is a signicant dierence between people's reported political ideologies using a
source like Gallup versus that can be inferred on social media. It is not possible to know
if this is due to limitations of political inference algorithms, confounders, population
representation biases, or else.
In the two states (NY & TX) where there was a statistically signicant discrepancy
between vote counts and instances of self-reported voting via #ivoted hashtags, we
found only limited anecdotal evidence of tweets reporting issues of voter fraud or
suppression. The divergence can possibly be explained by confounding factors, locality
and selection bias, or social in
uence of particular candidates in those states (e.g.,
Alexandria Ocasio-Cortez in NY and Beto O'Rourke in TX).
36
2.2 Objective and Plan
We aim at detecting whether our adversary employ automation tools (bots) to spread divisive
or malicious content on social media. We specically study Twitter to characterize how the
behavior of automated accounts controlled by our adversaries has changed between 2016
and 2018 (US Midterm elections), and in turn how this re
ects on human behavior. We
will leverage state-of-the-art bot detection techniques on a massive corpus of Twitter data
collected up to November 2018. We will then employ network analysis and sentiment analysis
to characterize the evolution of bot-driven narratives and their resonance with human users
targeted by such campaigns.
2.3 Background
The US Midterm elections were held on 6 November, 2018. They are referred to as mid-term
elections because they occur in the middle of a presidential term. Senators serve for 6 years,
thus, every 2 years, nearly a third of the Senators are up for re-election. The Senate is
divided into 3 classes, depending on which year they were elected. Class I was elected in
2012 and are up for re-election in 2018.
For 2018, 35 Senators out of a total of 100 senators in the 115th Congress will be up for
re-election. Of the 35 senators up for election, 33 are in Senate Class I and two are Senators
who vacated, whereas 15 are in what is to be considered contentious races. The 33 Class I
are 30 (23 Democrats (D), 5 Republicans (R), 2 Independents (I)) up for re-election and 3
Republicans (R) who are retiring. Details on the Senate seats up for re-election are in Table
2.1. Additionally, all 535 House of Representative seats are up for re-election every 2 years.
Excluded from our analysis are the non-voting delegates for DC and the US Territories.
37
2.4 Related Work
Since the 2016 US Presidential election, there has been a big spotlight on the sovereignty
of the US election system. The Bot Disclosure and Accountability Act of 2018
1
gave clear
guidelines for what has to be disclosed by social media companies. The article The Rise of
Social Bots [39] brought awareness to the issue of social bots in social media platforms. In
[12], Bessi & Ferrara focused on social bots detection within the online discussion related to
the 2016 presidential election. Other than characterizing the behavioral dierences between
humans and bots, there was not an in-depth analysis of any malicious intent. In this paper,
we address the potential malicious activity in online political discussion along the lines of
voter fraud, voter suppression, political misinformation, and then report on the biases we
found.
2.4.1 Voting Issues
Concerns related to voter fraud took center stage after the 2000 US Presidential election,
where it was argued that the candidate with the most votes lost and the Supreme Court
decided the winner [96]. Since then, a host of public debate, congressional testimony, and
several new laws passed, such as the Help America Vote Act [73], which surprisingly needed
to happened after the National Voter Registration Act of 1993 (NVRA).
2
The eects of the
NVRA were researched by (author?) [62], who concluded that provisions in the NVRA
would increase voter turnout by 4.7%-8.7% and that purging voter rolls of those who had
not voted in the last two years would have a 2% eect. Lastly, they identied the two most
vulnerable non-voting groups to be those under the age of 30 and those who moved within
2 years of an election [62].
1
https://www.congress.gov/bill/115th-congress/senate-bill/3127/text
2
https://www.justice.gov/crt/about-national-voter-registration-act
38
Moreover, it has been argued that the current US voter registration has a minimal impact
on registration and that there is marginal value in any updated laws [61]. Therefore, the
main concern argued by both parties is voter suppression [138]. Specically, due to recent
voter identication laws, there is an increased chance of voter suppression [60]. However, in
this work we seek to nd instances of voter suppression from an online social media analysis.
To our knowledge, this has not been done before.
2.4.2 Political Manipulation
Social media serve as convenient platforms for people to connect and to exchange ideas.
However, social media networks like Twitter and Facebook can be used for malicious purposes
[36]. Especially in the context of political discussion, there is a signicant risk of mass
manipulation of public opinion. Concerning the ongoing investigation of Russian meddling in
the 2016 US Presidential election, (author?) [6] studied political manipulation by analyzing
the released Russian troll accounts on Twitter. After using label propagation to assign
political ideology, they found that Conservatives retweeted Russian trolls over 30 times
more than Liberals and produced 36 times more tweets. More recently, (author?) [123]
highlighted how bots can play signicant roles in targeting in
uential humans to manipulate
online discussion thus increasing in-ghting. Especially for the spread of fake news, various
studies showed how political leaning [2], age [57], and education [118] can greatly aect fake
news spread, alongside with other mechanisms that leverage emotions [42, 41] and cognitive
limits [104, 105]. Additionally, (author?) [35] showed how foreign actors can more so than
just backing one candidate or the other, often manipulate social media for the purpose of
sowing discord.
39
2.4.3 Bias
Besides manipulation, other potential problems may aect data originating from online so-
cial systems. Selection bias is one such example. Concisely, this bias yields a statistically
non-representative sample of the true population. A main concern outlined by (author?)
[116], and to a lesser degree by (author?) [88], is that social media samples are not rep-
resentative of the whole voting population because users self-select to participate on the
platform and in specic online discussions. Each social media platform has its own set of
biases. (author?) [97] looked specically at the Twitter population from a location, gender,
and ethnicity viewpoint. From a location perspective, they found underrepresented counties
in the Mid-West and over-represented counties in highly dense urban areas [97]. Biases in
the representation of gender [107], ethnicity [18], and other sources of distortions [27] can
also potentially aect the inference of political ideology.
2.5 Data
In this study, we examine dierent data sources to investigate and explore the risk of using
social media in the context of political election manipulation.
We used Twitter as a sensor to estimate the expected amount of votes generated by each
state. For this purpose, we carried out two data collections. In the rst one, we gathered the
tweets with the hashtag #ivoted on election day. The second collection aimed to enlarge the
spectrum to a longer period of time exploiting a variety of general keywords, related to the
midterm election, to collect the tweets. As a basis for comparison, we employ two external
sources. The United States Election Project is used to unveil the amount of voters in each
state, while Gallup to have an estimate of the political polarization both at the country level
and at the state level. By means of these three data sources, we assembled ve data sets
40
(DS1-DS5), which will be analyzed in turn in the following subsections.
DS1: #ivoted Dataset
The #ivoted Dataset (DS1) gathers the tweets with the hashtag #ivoted generated on the
day of the election, November 6, 2018. It should be noticed that #ivoted was promoted by
Twitter and Instagram|which typically aects the hashtag spread [40, 135]|to encourage
citizens to participate in the midterm elections and increase the voter turnout. We used
the Python module Twyton to collect tweets through the Twitter Streaming API
3
during
election day. The data collection time window ranged from 6 a.m. EST on November 6
(when the rst polling station opened) to 1 a.m. HST on November 7 (2 hours after the last
polling station closed). Overall, we collected 249,106 tweets. As a sanity check, we queried
the OSoMe API provided by Indiana University [29]. OSoMe tracks the Twitter Decahose, a
pseudo-random 10% sample of the stream, and therefore can provide an estimate of the total
volume: OSoME contains 29.7K tweets with the #ivoted hashtag posted by 27.2K users|it
is worth noting that trending topics are typically slightly over-represented in the Twitter
Decahose [29, 100]|by extrapolation, this would suggest an estimated upper bound of the
total volume at around 300K tweets. In addition, on election day, Twitter reported that the
hashtag #ivoted was trending with over 200K tweets (cf. Fig. 2.1). Having collected 249K
such tweets, we can conclude that we have at our disposal a nearly complete #ivoted sample
dataset.
DS2 & DS3: General Midterm Dataset
In the General Midterm Dataset, we collect tweets on a broader set of keywords. Further,
we consider two dierent time windows for the data collection. The rationale behind these
3
Please note that we utilize the same approach for every Twitter data collection discussed in this work.
41
Figure 2.1: Screen shot of the United States trends on election day showing the #ivoted
hashtag trending with 200K tweets.
choices is to evaluate the sensitivity of our study against a dierent, but correlated, set of
data. In other words, the main purpose is to detect whether any divergence arose with the
#ivoted Dataset analysis or, on the other hand, to inspect the consistency of the results in
dierent settings.
Tweets were collected by using the following keywords as a lter: 2018midtermelections,
2018midterms, elections, midterm, and midtermelections. We distinguish two data sets ac-
cording to their temporal extent. In DS2, we consider only tweets generated on the election
day with exactly the same time window used for DS1. The third data set (DS3) provides a
42
view of the political discussion from a wide-angle lens. It includes tweets from the month
prior (October 6, 2018) to two weeks after (November 19, 2018) the day of the election. We
kept the collection running after the election day as several races remained unresolved. As
a result, DS3 consists of 2.7 million tweets, whose IDs are publicly available for download.
4
DS4: Actual Voting Data
The rst external data source used as a basis of comparison is made available by the United
States Election Project. They report on their website
5
the expected voter turnout per state,
along with the (ocial or certied) information source and other statistics about voters.
The data (DS4) we use in this work was assessed on November 18, 2018, and re
ects a voter
turnout of 116,241,100 citizens, which is aligned with other reported counts.
DS5: Party Aliation Data
To have an assessment of the political party aliation across the country, we make use of an
evaluation provided by Gallup, through the Gallup Daily tracking survey, a system which
continuously monitors Americans' attitudes and behaviors.
6
The data set (DS5), collected
on January 22, 2019, depicts the political leaning over a sample size of 180,106 citizens.
In particular, the data shows the percentage of Democratic and Republican population in
each state and over the entire country. Gallup's evaluation shows that, at the national level,
there exists a democratic advantage (7%), as 45% of the population is assessed as democratic
leaning while 38% is estimated as republican.
4
https://github.com/A-Deb/midterms
5
http://www.electproject.org/2018g
6
https://www.gallup.com/174155/gallup-daily-tracking-methodology.aspx
43
2.5.1 Data Pre-processing
Data pre-processing involved only Twitter data sets and consisted of three main steps. First,
we removed any duplicate tweet, which may have been captured by accidental duplicate
queries to the Twitter API. Then, we excluded from our analysis all the tweets not written
in English language. Despite the majority of the tweets were in English, and to a very lesser
degree in Spanish (3,177 tweets), we identied about 59 languages in the collected tweets.
Finally, we inspected tweets from other countries and removed them as they were out of the
context of this study. In particular, we ltered out tweets related to the Cameroon election
(October 7, 2018), to the Democratic Republic of the Congo presidential election (December
23, 2018), to the Biafra call for Independence (#biafra, #IPOB), to democracy in Kenya
(#democracyKE), to the two major political parties in India (BJP and UPA), and to college
midterm exams.
Overall, we count for almost 3 millions tweets distributed over the three Twitter data sets
(DS1-DS3). In Table 3.1, we report some aggregate statistics. It should be noticed that the
number of authors is lower than the number of users, which in turn also includes accounts
that got a retweet (or reply) of a tweet that was not captured in our collection and, thus,
they do not appear as authors.
2.6 Methodology
2.6.1 State Identication
The usage of geo-tagged tweets to assign a state to each user has been shown to not be
eective, being the fraction of geo-tagged tweets around 0.5% [23]. The location of the data
is of utmost importance, especially at the state and local level. However, less than 1% of the
44
collected tweets have been geo-tagged. Nevertheless, we aim to map as many users as possible
to a US state, to conduct a state by state comparison. For this purpose, we leveraged tweet
metadata, which may include the self-reported user prole location. The location entry is a
user-generated string (up to 100 characters), and it is pulled from the user prole metadata
for every tweet. From this eld, we rst search for the two-letter capitalized state codes,
followed by the full name of the state. Our analysis does not include Washington, D.C., so
we have to ensure anything initially labeled Washington does not include any variant of DC.
Using this string-search method, we managed to assign a state to approximately 50% of the
tweets and 30% of the users. Some users had multiple states over their tweet history, thus,
we only used the most common reported state. A few users often switched their location
from a state name to something else: for example, one user went from New York, NY to
Vote Blue!|for such users, we kept the valid state location.
2.6.2 Bot Detection
Bot detection has received ample attention [39] and increasingly sophisticated techniques
keep emerging [78]. In this study, we restrict our bot detection analysis to the use of the
widely popular Botometer,
7
developed by Indiana University. The underpinnings of the
system were rst published in [30, 134] and further revised in [142]. Botometer is based on
an ensemble classier [17] fed by over 1,000 features related to the Twitter account under
analysis and extracted through the Twitter API. Botometer aims to provide an indicator,
namely bot score, that is used to classify an account either as a bot or as a human. The lower
the bot score, the higher the probability that the user is not an automated and/or controlled
account. In this study we use version v3 of Botometer, which brings some innovations and
important detailed in [142]|e.g., the bot scores are now rescaled and not centered around
0.5 anymore.
7
https://botometer.iuni.iu.edu/
45
In Figure 3.1, we depict the bot score distribution of the 1,131,540 distinct users in our
datasets. The distribution exhibits a right skew: most of the probability mass is in the range
[0, 0.2] and some peaks can be noticed around 0.3. Prior studies used the 0.5 threshold to
separate humans from bots. However, according to the re-calibration introduced in the latest
version of Botometer [142], along with the emergence of increasingly more sophisticated bots,
we here lower the bot score threshold to 0.3 (i.e., a user is labeled as a bot if the bot score is
above 0.3). This threshold corresponds to the same level of sensitivity setting of 0.5 in prior
versions of Botometer (cf. Fig 5 in [142]). In both DS1 and DS3, 21.1% of the users have
been classied as bots, while in DS2 the percentage achieves the 22.9% of the users. Finally,
19.5% of the 295,352 users for which a State was identied have been scored as bots.
Overall, Botometer did not return a score for 42,904 accounts, which corresponds to 3.8%
of the users. To further examine this subset of users, we make use of the Twitter API.
Interestingly, 99% of these accounts were suspended by Twitter, whereas the remaining 1%
were protected (by privacy settings). For the users with an assigned location, only 1,033
accounts did not get a Botometer score. For those users, we assume that the accounts
suspended (1,019) are bots and the private accounts (14) are humans.
2.6.3 Statistical Vote Comparison
Once the states have been identied and the bots detected, we compared the distribution of
our various Twitter datasets (DS1, DS2, and DS3) with our control data in DS4 and DS5.
To do this, we start by counting the number of tweets per state and dividing it by the total
number of tweets across all states. We denote this fractional share in terms of tweets as
46
Figure 2.2: Bot Score Distribution
State Tweet Rate (STR), for each state i as
STR(i) =
no. tweets from State i
P
50
j
no. tweets from State j
(2.1)
For the actual voter data (DS4), we perform a similar metric to determine the State Vote
Rate (SVR) of each state i as
SVR(i) =
no. votes from State i
P
50
j
no. votes from State j
(2.2)
47
We then calculate the dierence (i) for each state i. Here it is important to note that any
positive value indicates more tweets than votes, as a percentage, and vice versa:
(i) =STR(i)SVR(i) (2.3)
Lastly, we convert the dierence into standard deviations s(i) (stdevs) by dividing (i) by
the standard deviation of all dierences:
s(i) =
(i)
q
P
((i))
50
(2.4)
being the average dierence over all states. We then inspect the results for any anomalous
statei whose standard deviationjs(i)j 2. States beyond two standard deviations are worth
further inspection.
2.6.4 Political Ideology Inference
We classify users by their ideology based on the political leaning of the media outlets they
share. We use lists of partisan media outlets compiled by third-party organizations, such
as AllSides
8
and Media Bias/Fact Check.
9
We combine liberal and liberal-center media
outlets into one list and conservative and conservative-center into another. The combined
list includes 641 liberal and 398 conservative outlets. However, in order to cross reference
these media URLs with the URLs in the Twitter dataset, we need to get the expanded URLs
for most of the links in the dataset, since most of them are shortened. As this process is
quite time-consuming, we get the top 5,000 URLs by popularity and then retrieve the long
version for those. These top 5,000 URLs account for more than 254K, or more than 1/3 of
8
https://www.allsides.com/media-bias/media-bias-ratings
9
https://mediabiasfactcheck.com/
48
all the URLs in the dataset. After cross-referencing the 5,000 long URLs with the media
URLs, we observe that 32,115 tweets in the dataset contain a URL that points to one of the
liberal media outlets and 25,273 tweets with a URL pointing to one of the conservative media
outlets. We use a polarity rule to label Twitter users as liberal or conservative depending
on the number of tweets they produce with links to liberal or conservative sources. In other
words, if a user has more tweets with URLs to liberal sources, he/she is labeled as liberal
and vice versa. Although the overwhelming majority of users include URLs that are either
liberal or conservative, we remove any user that has equal number of tweets from each side.
Our nal set of labeled users includes 38,920 users.
To classify the remaining accounts as liberal or conservative, we use label propagation, similar
to prior work [6]. For this purpose, we construct a retweet network, containing nodes (Twitter
users) with a direct link between them if one user retweet a post of another. To validate
results of the label propagation algorithm, we apply stratied cross (5-fold) validation to
a set of more than 38,920 seeds. We train the algorithm on 4/5 of the seed list and see
how it performs on the remaining 1/5. Both precision and recall scores are around 0.89.
Since we combine liberal and liberal-center into one list (same for conservatives), we can
see that the algorithm is not only labeling the far liberal or conservative correctly, which is
a relatively easier task, but it is performing well on the liberal/conservative center as well.
Overall, we nd that the liberal users population is almost three times larger the conservative
counterpart (73% vs. 27%).
49
2.7 Results
2.7.1 #ivoted (DS1) Statistical Analysis
There were 249,106 tweets in the #ivoted data set, of those we could map a state location
for 78,162 unique authors. Once we remove the 15,856 bots (using a bot threshold score of
0.3), we have 62,306 remaining authors of tweets and retweets. After applying the method
described in Statistical Vote Comparison section, we see that three states show an anomalous
behavior from the remaining 47 states. Figure 2.4a shows how New York is 5.8 standard
deviations greater than the mean dierence between the #ivoted percentage and the actual
voting percentage. Furthermore, both California and Texas have a stdev 2.2 greater than
the mean. This would lead to believe that if there was voter suppression, it would most
likely be in these three states, as they exhibit signicantly more self-reported voting tweets
than vote counts.
However, since our data set has both tweets and retweets, to check the sensitivity of our nd-
ings, we repeated our analysis without the retweets. Once removed, the 34,754 remaining
tweets, again without bots, we noticed something interesting. Not only did Texas drop from
2.2 stdevs to 0.4 stdevs, but New York increased from 5.8 stdevs to 6.3 stdevs. This high-
lights the sensitivity our this type of analysis to location-specic factors such as state, and
information dynamic factors such as retweet ltering. Further inspection showed that 62.2%
of the tweet activity in Texas (in the #ivoted data set) was based on retweets, highlighting
how this class of tweet can produce dierent results for some populations, and similar ones
for others, since the average across the states stayed at 0 (e.g., see Figure 2.4b).
50
2.7.2 General Midterm (DS2&DS3) Statistical Analysis
We carried out the same analysis against the general keywords data set both on election day
(DS2) and for a month before to two weeks after the election (DS3).
In DS2, we have 72,022 users, from which we ltered out 16,859 bots (using a bot threshold
of 0.3). From the remaining 55,163 authors, we were able to map a state for 26,081 users.
Performing the same comparative analysis from before, we found the same anomalies in the
same three states: CA (1.6 stdev), TX (2.8 stdev), and NY (5.6 stdev). Visually, this can
be appreciated in Figure 2.4c. Expanding the analysis to DS3, we removed 206,831 users,
as classied as bots, from the set of 977,966 authors. This left us with 771,135 users from
which we could identify a state for 295,705 of them. The statistical analysis revealed the
same outliers also in this data set: CA (2.8 stdev), TX (3.1 stdev), and NY (4.7 stdev), as
can been seen in Figure 2.4d.
2.7.3 Bot Sensitivity
Next, we investigate whether discarding malicious accounts, such as social bots, from the set
of users may have aected the ndings above. Table 2.4 shows the number (and percentage)
of bots and humans per state in DS3. The list of states is sorted (in descending order)
according to the percentage of bots, while the horizontal line separates the states with
a bots percentage above and below the average (20.3%). Note in particular that all the
three outliers (in bold) have values below the average. However, the distribution of bot
prevalence per state varies greatly and it should be analyzed taking into account both the
state population size and the number of Twitter users per state. Highly populated states like
California, Texas, and New York, have large sheer numbers of bots but low proportional bot
percentage. This should be taken into account when drawing conclusions from this analysis.
On the other side, this topic opens the way to further discussions about bots association
51
with a given state. One could make the argument that if the account was identied as a
bot, there is no point to assigning it to a state. However, the fact that automated accounts
declare a location in their prole can be viewed as a malicious strategy to embed in the
social system thus, it should be prudently examined.
For these reasons, we repeated our analysis including social bots in the users set. Results with
or without bots are substantially unchanged. In the interest of space, we do not duplicate
the maps shown in Figure 2.4, but the same anomalies are revealed if bots are retained.
It should be noticed that also for the #ivoted dataset (DS1), the percentage of bots in the
three outlier states are below the average (21.0%), NY (16.0%), CA (19.4%) and TX (20.2%),
respectively.
2.7.4 Political Ideology Analysis
Next we examine what topics talk about and how they address politically charged topics.
Table 2.3 shows the top 10 hashtags discussed respectively by humans and bots, for both
liberal and conservative ideologies. The hashtags have been colored to show the common
topics between bots and humans for each political wing. The amount of overlap between
bots and humans hashtags is noticeable. This is likely the reason why the removal of bots
from the analyzed accounts did not have any signicant impact on our outcome. To carefully
interpret this table, it should be noticed that the liberal group is almost three times larger
than the conservative one, as we stated in Political Ideology section.
Additionally, we took our political ideology labels by state and compared with DS5, the
Gallup poll survey. As mentioned before, the political ideology inference assigned 73% liberal
labels and 27% conservative labels to the nation at a whole. That compares with Gallup
reporting of 45% to 38% for the Nation as a whole. At the state level, we ran a comparison
to see the dierence in our assessment of political leaning of a state versus Gallup's. For
52
example, Alabama is 35% liberal and 50% conservative, according to Gallup, giving the
state a marked Republican advantage. However, in Twitter we observed 42% Liberal and
31% Conservative user labels, which may suggest the opposite trend. Figure 2.3 shows the
dierence between the Gallup poll and our analysis. For Alabama going from a Republican
advantage of 15% (Gallup) to a Democratic advantage of 11% (Twitter) would imply a shift
of 26 percent points toward the liberal side. Overall, every state showed movement toward
the left, as low as a few percent points and as high as over 60% dierence. This corroborates
the suspect that left-leaning users are over-represented in our data.
2.7.5 Voting Issues
New York was the state that exhibited the strongest statistical anomaly. Thus, we conducted
a manual inspection reading all tweets originating from there. We found no red
ags, but we
isolated a few tweets of interest. The rst one is in Figure 2.5 and it is from a user who was
classied as a human and from inspection of the account shown to live in New York. The
user mentions some important issues: at 11:20 am on the day of the election, they found out
they are the victim of voter fraud. There is no information to suggests this was resolved in
any meaningful way or if the accusation is substantiated.
A second example of potential voter issue was found after a manual inspection of the tweets
in New York. The tweet thread in Figure 2.6 is heavily redacted, but it shows an ongoing
conversation through replies and it shows multiple people presenting multiple sides. The
original tweet was actually posted on 5 November, 2018 and by the time of our viewing had
received a signicant number of retweets. It is from this original tweet that we see a reply
where the user is complaining that they can not get to the voting booth without a photo ID.
User 3 then asks for the name and number of the community and then User 4 provides an
election hotline number. This indicates that many people today are willing to speculate on
53
Twitter, but nothing seems to indicate that they also were going to the ocial Department
of Justice website to le a complaint.
From our inspection other tweets that are noteworthy include:
1. "First time voter in my family registered over a month ago on DMV website online
not realizing it's not automated. . . she could not vote. Not right."
2. "More voter fraud in Ohio. Why is it that all the errors are always the Democrats??
Because the only way they can win is if they cheat!! This madness needs to stop."
3. What we did see in our Twitter collection is early skepticism that there would be false
claims of voter fraud. A user tweeted "a little over 24 hours from now the Racist in
Chief will start Tweeting about rigged elections, voter fraud and illegal aliens voting
en mass...".
4. Shortly afterwards, many people started to retweet a user that stated "Massive voter
fraud in Texas Georgia Florida and others" and also indicating that MSM (main stream
media) are putting out fake polls. The Washington Post @washingtonpost tweeted
"without evidence, Trump and Sessions warn of voter fraud" which was retweeted
throughout election day.
5. There was a user who tweeted about voting machine malfunctions which mapped to
a story/blog from the Atlanta Journal Constitution (https://t.com/riCGdbwQ6R)
about machines being down; people left and were encouraged to come back. There
was an oer for casting a paper provisional ballot, but many said they did not trust
the paper ballot and wanted to vote on a machine.
54
2.8 Discussion & Recommendations
Our results have highlighted the challenges of using social media in election manipulation
analysis. A supercial interpretation of anomalies in online activity compared to real world
data can lead to misleading or false conclusions. In our case, we wanted to determine
the feasibility of using social media as a sensor to detect election manipulation such as
widespread voter suppression or voter fraud. While we did not nd widespread or systematic
manipulation, we learned a few lessons worthy of a discussion:
Data biases of online platforms can drastically aect the feasibility of a study. In our
case, we were looking for a representative sample of actual voters who are not bots
and whose political ideology and location could be known. Despite troves of data were
collected and analyzed, various encountered biases could not be adjusted for.
The second main issue is consistency in the analysis: the sensitivity to choices made
when carrying out data cleaning, parameter settings of inference algorithms, etc. yield
a so-called garden of forking paths [50]: some results can signicantly vary in function
of such choices (for example, location bias and the removal or retention of retweets
played a role in determining whether Texas exhibited a statistical anomaly in terms of
expected versus cast votes).
Political ideologies reported by Gallup signicantly vary with respect to that can be
inferred on social media. We were unable to determine if this is due to limitations
of the employed political inference tool, population biases, or other factors. This is
an open problem in social media analysis and a necessary one to tackle before social
media can be used to robustly replace polling.
The actual voting numbers reported by ocial sources correlated very closely to what
we inferred from our analysis on Twitter for 47 of 50 states. As such, the approach
55
seemed promising to identify voter suppression or fraud. However, the results show
a more complex picture: no evidence of fraud or suppression beyond anecdotal was
found in the three anomalous states under scrutiny. Yet, we suggest that prior and
during elections there should be an online social media presence for the Department
of Justice to engage with people who have a potential voting issue.
2.9 Conclusion and Future Work
In this work, we conducted an investigation to analyze social media during the 2018 US
Midterm election. In addition to studying bots and the political ideology of users, we stud-
ied the correlation between people talking about voting and actual voter data. We then
highlighted a few issues that could lead to inaccurate conclusions. In particular, removing
or retaining the bots didn't change the outcome of our results. This was not the case in
prior studies. However, in our case, removing retweets did make a signicant dierence for
one state, Texas, suggesting a dependency, or bias, on location.
The challenges we faced can all be expanded upon in future work. We only mapped a state to
44.7% of DS1 and 30.2% to DS2/DS3. If we can evaluate a user timeline to better recognize
what state they may be from that would enhance future location based studies. Our political
ideology inference started with the labeling of 38K users leveraging any link they posted,
and then labels were propagated on the retweet network. We could potentially identify the
users with high centrality and evaluate their timeline for party aliation and approach the
inference problem from a dierent angle. We could also focus on separating not just human
from bot accounts, but also human from corporate accounts. Some of the users that were
classied as human could be operating as part of a collective body, that while not necessarily
malicious, may insert an inorganic bias.
56
Ultimately, one of the goals of this work was to explore the feasibility of using social media
as a sensor to detect possible election manipulation at scale: despite our initial eort did
not produce the expected results, we highlighted some useful lessons that will illuminate on
future endeavors to use such data for social good.
57
Table 2.1: US Senate Seats Up for Election in 2018
Incumbent State Party Status
Tammy Baldwin WI D Contested
John Barraso WY R Safe
Sherrod Brown OH D Contested
Maria Cantrell WA D Safe
Ben Cardin MD D Safe
Tom Carper DE D Safe
Bob Casey PA D Safe
Bob Corker TN R Retiring
Ted Cruz TX R Contested
Joe Donnelly IN D Contested
Dianne Feinstein CA D Safe
Deb Fischer NE R Safe
Je Flake AZ R Retiring
Kirsten Gillibrand NY D Safe
Orrin Hatch UT R Retiring
Martin Heinrich NM D Safe
Heidi Heitkamp ND D Contested
Dean Heller NV R Contested
Mazie Hirono HI D Safe
Cindy Hyde-Smith MS R Contested
Tim Kaine VA D Safe
Angus King ME I Safe
Amy Klobuchar MN D Safe
Joe Manchin WV D Contested
Claire McCaskill MO D Contested
Bob Menendez NJ D Contested
Chris Murphy CT D Safe
Bill Nelson FL D Contested
Bernie Sanders VT I Safe
Tina Smith MN D Contested
Debbie Stabenow MI D Safe
Jon Tester MT D Contested
Elizabeth Warren MA D Safe
Sheldon Whitehouse RI D Safe
Roger Wicker MS R Safe
Table 2.2: Datasets Statistics
Statistic DS1 DS2 DS3
# of Tweets 90,763 20,450 452,288
# of Retweets 146,546 54,866 1,869,313
# of Replies 11,797 6,730 267,973
# of Authors 174,854 72,022 977,996
# of Users 178,503 77,749 997,406
58
Figure 2.3: Political ideology dierence, in terms of percentage of liberals vs. conservatives,
between DS5 and DS3
59
(a) #ivoted vs. Actual Votes (b) #ivoted (w/o RTs) vs. Actual Votes
(c) General (election) vs. Actual Votes (d) General (overall w/o RTs vs. Actual Votes
Figure 2.4: Various datasets versus Actual Votes (DS4) all without bots
Top 10 Hashtags
Liberal Conservative
Bots
#BlueWave #BrowardCounty
#VoteBlue #MAGA
#MAGA #Broward
#NovemberisComing #RedWave
#TheResistance #VoteRedToSaveAmerica
#Democrats #StopTheSteal
#Trump #VoteRed
#vote #Democrats
#Florida #Redwavepolls
#GOTV #WednesdayWisdom
Humans
#NovemberisComing #BrowardCounty
#VoteBlue #Broward
#BlueWave #MAGA
#vote #IranRegime
#txlege #Tehran
#electionday #StopTheSteal
#Russia #RedWave
#unhackthevote #PalmBeachCounty
#AMJoy #Redwavepolls
#Trump #Florida
Table 2.3: Top 10 hashtags: liberals, conservatives, humans, bots
60
Figure 2.5: #ivoted tweet from New York
61
Figure 2.6: #ivoted tweet from Florida
62
Table 2.4: General Midterms DS3: bot and human population by State (sorted by percent-
wise bot prevalence).
State # of bots # of humans
WY 97 (27.2%) 246 (68.9%)
ID 258 (23.8%) 791 (73.0%)
ND 289 (22.9%) 931 (73.9%)
AZ 1,514 (22.5%) 4,997 (74.2%)
NV 711 (22.4%) 2,377 (74.8%)
UT 420 (22.2%) 1,425 (75.4%)
DE 170 (22.1%) 575 (74.9%)
NM 325 (22.1%) 1,100 (74.7%)
NH 283 (22.1%) 968 (75.5%)
RI 402 (21.9%) 1,382 (75.3%)
WV 246 (21.7%) 854 (75.2%)
FL 4,696 (21.5%) 16,583 (75.9%)
MO 932 (21.4%) 3,336 (76.5%)
AL 697 (21.3%) 2,466 (75.4%)
TN 1,209 (21.3%) 4,369 (76.9%)
WI 808 (21.3%) 2,900 (76.4%)
MT 202 (21.0%) 730 (75.9%)
CO 1,144 (20.9%) 4,178 (76.4%)
NJ 1,311 (20.8%) 4,838 (76.8%)
MS 336 (20.6%) 1,239 (75.9%)
ME 290 (20.6%) 1,093 (77.6%)
CT 571 (20.4%) 2,141 (76.6%)
SC 769 (20.3%) 2,933 (77.5%)
OK 552 (20.2%) 2,098 (76.8%)
KS 661 (20.2%) 2,526 (77.2%)
GA 1,962 (20.2%) 7,489 (76.9%)
WA 1,561 (19.9%) 6,143 (78.2%)
NE 323 (19.9%) 1,253 (77.1%)
AK 160 (19.8%) 622 (77.1%)
HI 230 (19.8%) 895 (77.1%)
PA 1,898 (19.7%) 7,460 (77.6%)
MI 1,441 (19.6%) 5,714 (77.7%)
IA 414 (19.6%) 1,654 (78.2%)
VA 1,487 (19.6%) 5,931 (78.1%)
MA 1,372 (19.4%) 5,553 (78.4%)
NC 1,685 (19.3%) 6,872 (78.5%)
IL 1,702 (19.2%) 6,926 (78.0%)
IN 885 (19.1%) 3,593 (77.6%)
AR 199 (19.1%) 814 (78.0%)
KY 548 (19.0%) 2,270 (78.9%)
MN 866 (19.0%) 3,622 (79.6%)
OR 1,067 (18.9%) 4,416 (78.3%)
TX 5,550 (18.7%) 23,448 (79.1%)
OH 1,722 (18.6%) 7,271 (78.7%)
CA 7,073 (18.2%) 30,429 (78.5%)
VT 113 (17.9%) 505 (80.2%)
MD 887 (17.8%) 3,963 (79.7%)
NY 4,798 (17.5%) 21,896 (79.9%)
LA 708 (15.6%) 3,708 (81.7%)
SD 82 (15.4%) 439 (82.4%)
63
Chapter 3
Red Bots Do It Better
3.1 Introduction
During the last decade, social media have become the conventional communication channel to
socialize, share opinions, and access the news. Accuracy, truthfulness, and authenticity of the
shared content are necessary ingredients to maintain a healthy online discussion. However,
in recent times, social media have been dealing with a considerable growth of false content
and fake accounts. The resulting wave of misinformation (and disinformation) highlights
the pitfalls of social media networks and their potential harms to several constituents of our
society, ranging from politics to public health.
In fact, social media networks have been used for malicious purposes to a great extent [36].
Various studies raised awareness about the risk of mass manipulation of public opinion,
especially in the context of political discussion. Disinformation campaigns [106, 64, 120, 37,
136, 8, 59, 16, 118, 57] and social bots [12, 141, 134, 99, 108, 15, 119, 142] have been indicated
as factors contributing to social media manipulation.
64
The 2016 US Presidential election represents a prime example of the signicant perils of
mass manipulation of political discourse. (author?) [6] studied the Russian interference
in the election and the activity of Russian trolls on Twitter. (author?) [67] suggested
that troll accounts are still active to these days. The presence of social bots does not show
any sign of decline [142, 31] despite the attempts from social network providers to suspend
suspected, malicious accounts. Various research eorts have been focusing on the analysis,
detection, and countermeasures development against social bots. (author?) [39] highlighted
the consequences associated with bot activity in social media. The online conversation
related to the 2016 US presidential election was further examined [12] to quantify the extent
of social bots activity. More recently, (author?) [123] discussed bots' strategy of targeting
in
uential humans to manipulate online conversation during the Catalan referendum for
independence, whereas (author?) [119] analyzed the role of social bots in spreading articles
from low credibility sources. (author?) [31] focused on the 2018 US Midterms elections
with the objective to nd instances of voter suppression.
In this work, we investigate social bots behavior by analyzing their activity, strategy, and in-
teractions with humans. We aim to answer the following research questions (RQs) regarding
social bots behavior during the 2018 US Midterms election.
RQ1: Do social bots lean and behave according to a political ideology? We investigate whether
social bots can be classied based on their political inclination into liberal or conserva-
tive leaning. Further, we explore to what extent they act similarly to the corresponding
human counterparts.
RQ2: Can we observe dierent strategies among liberal and conservative bots? We examine
the dierences between social bot strategies to mimic humans and inltrate political
discussion. For this purpose, we measure bot activity in terms of volume and frequency
of posts, interactions with humans, and embeddedness in the social network.
65
RQ3: Are bot strategies eective? We introduce four metrics to estimate the eectiveness of
bot strategies and to evaluate the degree of human interplay with social bots.
We leverage Twitter to capture the political discourse during the 2018 US midterm elections.
We collected 2.6 million tweets for 42 days around election day from nearly 1 million users.
We then explore collected data and attain the following ndings:
We show that social bots are embedded in each political side and behave accordingly.
Conservative bots abide by the topic discussed by the human counterpart more than
liberal bots, which in turn exhibit a more provocative attitude.
We examined bots' interactions with humans and observed dierent strategies. Conser-
vative bots stand in a more central social network position, and divide their interactions
between humans and other conservative bots, whereas liberal bots focused mainly on
the interplay with the human counterparts.
We measured the eectiveness of these strategies and recognized the strategy of con-
servative bots as the most eective in terms of in
uence exerted on human users.
3.2 Data
In this study, we use Twitter to investigate the partisan behavior of malicious accounts during
the 2018 US midterm elections. For this purpose, we carried out a data collection from the
month prior (October 6, 2018) to two weeks after (November 19, 2018) the day of the election.
We kept the collection running after the election day as several races remained unresolved.
We employed the Python module Twyton to collect tweets through the Twitter Streaming
API using the following keywords as a lter: 2018midtermelections, 2018midterms, elections,
midterm, and midtermelections. As a result, we gathered 2.7 million tweets, whose IDs are
66
Table 3.1: Dataset statistics
Statistic Count
# of Tweets 452,288
# of Retweets 1,869,313
# of Replies 267,973
# of Users 997,406
publicly available for download.
1
From this set, we rst removed any duplicate tweet, which
may have been captured by accidental redundant queries to the Twitter API. Then, we
excluded all the tweets not written in English language. Despite the majority of the tweets
were in English, and to a lesser degree in Spanish (3,177 tweets), we identied 59 languages
in the collected data. Thus, we inspected tweets from other countries and removed them as
they were out of the context of this study. In particular, we ltered out tweets related to
the Cameroon election, the Democratic Republic of the Congo election, the Biafra call for
Independence, democracy in Kenya (#democracyKE), to the two major political parties in
India (BJP and UPA), and college midterm exams. Overall, we retain nearly 2.6 millions
tweets, whose aggregate statistics are reported in Table 3.1.
3.3 Methodology
3.3.1 Bot Detection
Nowadays, bot detection is a fundamental asset for understanding social media manipulation
and, more specically, to reveal malicious accounts. In the last few years, the problem of
detecting automated accounts gathered both attention and concern [39], also bringing a
wide variety of approaches to the table [124, 78, 19, 22]. While increasingly sophisticated
techniques keep emerging [78], in this study, we employ the widely used Botometer.
2
1
https://github.com/A-Deb/midterms
2
https://botometer.iuni.iu.edu/
67
Botometer is a machine learning-based tool developed by Indiana University [30, 134] to
detect social bots in Twitter. It is based on an ensemble classier [17] that aims to provide
an indicator, namely bot score, used to classify an account either as a bot or as a human. To
feed the classier, the Botometer API extracts about 1,200 features related to the Twitter
account under analysis. These features fall in six broad categories and characterize the ac-
count's prole, friends, social network, temporal activity patterns, language, and sentiment.
Botometer outputs a bot score: the lower the score, the higher the probability that the user
is human. In this study we use version v3 of Botometer, which brings some innovations,
as detailed in [142]. Most importantly, the bot scores are now rescaled (and not centered
around 0.5 anymore) through a non-linear re-calibration of the model.
In Figure 3.1, we depict the bot score distribution of the 997,406 distinct users in our datasets.
The distribution exhibits a right skew: most of the probability mass is in the range [0, 0.2]
and some peaks can be noticed around 0.3. Prior studies used the 0.5 threshold to separate
humans from bots. However, according to the re-calibration introduced in Botometer v3
[142], along with the emergence of increasingly more sophisticated bots, we here lower the
bot score threshold to 0.3 (i.e., a user is labeled as a bot if the score is above 0.3). This
threshold corresponds to the same level of sensitivity setting of 0.5 in prior versions of
Botometer (cf. Fig 5 from [142]).
According to this choice, we classied 21.1% of the accounts as bots, which in turn generated
30.6% of the tweets in our data set. Overall, Botometer did not return a score for 35,029 users
that corresponds to 3.5% of the accounts. We used the Twitter API to further inspect them.
Interestingly, 99.4% of these accounts were suspended by Twitter, whereas the remaining
percentage of users protected their tweets turning on the privacy settings of their accounts.
68
Figure 3.1: Bot score distribution
3.3.2 Political Ideology Inference
In parallel to the bot detection analysis, we examine the political leaning of both bots
and humans in our dataset. To classify users based on their political ideology, we rely on
the political leaning of the media outlets they share. We make use of a list of partisan
media outlets released by third-party organizations, such as AllSides
3
and Media Bias/Fact
Check.
4
We combine liberal and liberal-center media outlets into one list (composed of 641
outlets) and conservative and conservative-center into another (composed of 398 outlets).
To cross reference these media URLs with the URLs in the Twitter dataset, we need to get
the expanded URLs for most of the links in the dataset, as most of them are shortened.
However, this process is quite time-consuming, thus, we decided to rank the top 5,000 URLs
by popularity and retrieve the long version only for those. These top 5,000 URLs accounts
for more than 254K, or more than 1/3 of all the URLs in the dataset. After cross-referencing
3
https://www.allsides.com/media-bias/media-bias-ratings
4
https://mediabiasfactcheck.com/
69
the 5,000 extended URLs with the media URLs, we observe that 32,115 tweets in the dataset
contain a URL that points to one of the liberal media outlets and 25,273 tweets with a URL
pointing to one of the conservative media outlets.
To label Twitter accounts as liberal or conservative, we use a polarity rule based on the
number of tweets they produce with links to liberal or conservative sources. Thereby, if an
account has more tweets with URLs pointing to liberal sources, it is labeled as liberal and
vice versa. Although the overwhelming majority of accounts include URLs that are either
liberal or conservative, we remove any account that has equal number of tweets from each
side. Our nal set of labeled accounts includes 38,920 users.
Finally, we use label propagation to classify the remaining accounts in a similar way to
previous work (cf. [6]). For this purpose, we construct a social network based on the
retweets exchanged between users. The nodes of the retweet network are the users, which
are connected by a direct link if one user retweeted a post of another user. To validate
results of the label propagation algorithm, we apply a stratied cross (5-fold) validation to
a set composed of 38,920 seed accounts. We train the algorithm using 80% of the seeds
and we evaluate the performance on the remaining 20%. Finally, we compute precision and
recall by reiterating the validation of the 5-folds. Both precision and recall scores show value
around 0.89 and validate the proposed approach. Moreover, since we combine liberal and
liberal-center into one list (same for conservatives), we can see that the algorithm is not only
labeling the far liberal or conservative correctly, which is a relatively easier task, but it is
performing well on the liberal/conservative center as well.
3.3.3 Bot Activity Eectiveness
We next introduce four metrics to estimate bot eectiveness and, at the same time, measure
to what extent humans rely upon, and interact with the content generated by social bots.
70
Thereby, we propose the following metrics:
Retweet Pervasiveness (RTP ) measures the intrusiveness of bot-generated content in
human-generated retweets:
RTP =
no. of human retweets from bot tweets
no. of human retweets
(3.1)
Reply Rate (RR) measures the percentage of replies given by humans to social bots:
RR =
no. of human replies to bot tweets
no. of human replies
(3.2)
Human to Bot Rate (H2BR) quanties human interaction with bots over all the human
activities in the social network:
H2BR =
no. of humans interaction with bots
no. of humans activity
, (3.3)
where the numerator counts for human replies/retweets to/of bots generated content,
while the denominator is the sum of the number of human tweets, retweets, and replies.
Tweet Success Rate (TSR) is the percentage of tweets generated by bots that obtained
at least one retweet by a human:
TSR =
no. of tweet retweeted at least once by a human
no. of bots tweets
(3.4)
3.4 Results
Next, we address the research questions discussed in the Introduction. We examine social
bot partisanship and, accordingly, we analyze bots' strategies and measure the eectiveness
71
Table 3.2: Users and tweets statistics
Liberal Conservative
Humans 386,391 (38.7%) 122,761 (12.3%)
Bots 82,118 (8.2%) 49,488 (4.9%)
(a) Number (percentage) of users per group
Liberal Conservative
Humans 957,726 (37.0%) 476,231 (18.4%)
Bots 288,659 (11.1%) 364,727 (14.1%)
(b) Number (percentage) of tweets per group
of their actions.
3.4.1 RQ1: Bot Political Leaning
(b) 25-core decomposition
(a) 10-core decomposition
Figure 3.2: Political discussion over (a) the 10-core, and (b) the 25-core decomposition of
the retweet network. Each node represents a user, while links represent retweets. Links with
weight (i.e., frequency of occurrence) less than 2 are hidden to minimize visual clutter. Blue
nodes represent liberal accounts, while red nodes indicate conservative users. Darker tones
(blue and red) depict bots, while lighter tones (cyan and pink) relate to humans, and the
few green nodes represent unclassied accounts. The link takes the same color of the source
node (author of the retweet), whereas node size is proportional to the in-degree of the user.
The combination of the outcome from the bot detection algorithm and the political ideology
inference allowed us to identify four groups of users, namely Liberal Humans, Conservative
Humans, Liberal Bots, and Conservative Bots. In Table 3.2a, we show the percentage of
users per group. Note that percentages do not sum up to 100 as either the political ideology
72
Table 3.3: Top 20 hashtags generated by liberal and conservative bots. Hashtags in bold are
not present in the top 50 hashtags used by the corresponding human group.
Liberal Bots Conservative Bots
#MAGA #BrowardCounty
#NovemberisComing #MAGA
#TheResistance #StopTheSteal
#GOTV #WalkAway
#Florida #WednesdayWisdom
#ImpeachTrump #PalmBeachCounty
#Russia #Florida
#VoteThemOut #QAnon
#unhackthevote #KAG
#FlipTheHouse #IranRegime
#RegisterToVote #Tehran
#Resist #WWG1WGA
#ImpeachKavanaugh #Louisiana
#GOP #BayCounty
#MeToo #AmericaFirst
#AMJoy #DemocratsAreDangerous
#txlege #StopTheCaravan
#FlipTheSenate #Blexit
#CultureOfCorruption #VoteDemsOut
#TrumpTrain #VoterFraud
inference was not able to classify every user, or Botometer did not return a score, as we
previously mentioned. In particular, we were able to assign a political leaning to 63% of bots
and 67% of humans. We nd that the liberal user population is almost three times larger
than the conservative counterpart. This discrepancy is also present, but less evident, for the
bot accounts, which exhibit an unbalance in favor of liberal bots. Further, we investigate
the suspended accounts to inspect the consistency of this result. The inference algorithm
attributed a political ideology to 63% of these accounts, which in turn show once again the
liberal advantage over the conservative faction (45% vs. 18%).
Figure 3.2 shows two k-core decomposition graphs of the retweet network. In a k-core, each
node is connected with at least k other nodes. Figures 3.2a and 3.2b capture the 10-core and
25-core decomposition, respectively. Here, nodes represent Twitter users and link represent
retweets among them. We indicate as source the user that retweeted the tweet of a target
73
user. Colors represent the political ideology, with darker colors (red and blue) being bots
and lighter colors (cyan and pink) being human users; size represents the in-degree. The
graph is visualized using a force-directed layout [68], where nodes repulse each other, while
edges attract their nodes. In our setting, this means that users are spatially distributed
according to the amount of retweets between each other. The result is a network naturally
split into two communities, where each side is almost entirely populated by users with the
same political ideology. This polarization is also re
ected by bots, which are embedded, with
humans, in each political side. Two facts are worth noting: (i) as k increases, the left k-core
appears to disrupt, while the right k-core remains well connected; and, (ii) as k increases,
bots appear to outnumber humans, suggesting that bots may populate areas of the retweet
network that are more central and better connected.
Next, we examine the topics discussed by social bots and compare them with the human
counterparts. Table 3.3 shows the top 20 hashtags utilized by liberal and conservative bots.
We highlight (in bold) the hashtags that are not present in the top 50 hashtags used by the
corresponding human group to point out the similarities and dierences among the groups.
In this table, we do not take into account general hashtags (such as #elections, #midterms,
#democrats, #liberals, #VoteRed(or Blue)ToSaveAmerica, and #Trump) as (i) the overlap
between bot and human hashtags is noticeable when these terms are considered, and (ii)
we aim to narrow the analysis to specic topics and in
ammatory content, inspired by
[123]. Moreover, we used an enlarged subset of hashtags for the human groups to further
strengthen the dierences and, at the same time, to better understand the objective of social
bots. Although bots and humans share the majority of hashtags, two main dierences can be
noticed. First, conservative bots abide by the corresponding human counterpart more than
the liberal bots. Second, liberal bots focus on more in
ammatory and provocative content
(e.g., #ImpeachTrump, #ImpeachKavanaugh, #FlipTheSenate) w.r.t. conservative bots.
74
3.4.2 RQ2: Bot Activity and Strategies
In this Section, we investigate social bot activity based on their political leaning. We explore
their strategies in interacting with humans and the degree of embeddedness in the social
network.
Table 3.2b depicts the number (and percentage) of tweets generated by each group. Despite
the group composed of conservative bots is the smallest in terms of number of accounts, it
produced more tweets than liberal bots and closely approaches the number of tweets gener-
ated by the human counterpart. The resulting tweet per user ratio shows that conservative
bots produce 7.4 tweets per account, which is more than twice the ratio related to the liberal
bots (3.5), almost the double of the human counterpart (3.9), and nearly three times the
ratio of liberal humans (2.5).
To investigate the interplay between bots and humans, we consider the previously described
retweet network. Figure 3.3 shows the interaction among the four groups. We maintain the
same color mapping described before, with darker color (on the bottom) representing bots
and lighter color (on top) indicating humans. Node size is proportional to the percentage
of accounts in each group, while edge size is proportional to the percentage of interactions
between each group. In Figure 3.3a, this percentage is computed considering all the in-
teractions in the retweet network, while in Figure 3.3b we consider each group separately,
therefore, the edge size gives a measure of the group propensity to interact with the other
groups. Consistently with Figure 3.2, we observe that there is a limited amount of interac-
tion between the two political sides. The majority of interactions are either intra-group or
between groups of the same political leaning. From Figure 3.3b, we can observe that the two
bot factions adopted dierent strategies. Conservative bots balanced their interactions by
retweeting group members 43% of the time, and the human counterpart 52% of the time. On
the other hand, liberal bots mainly retweeted liberal humans (71% of the time) and limited
75
Bots
Humans
(a) Overall interactions (b) Group-based interactions
Figure 3.3: Interactions according to political ideology
Table 3.4: Average network centrality measures
Liberal Conservative
Humans 2.6610
6
4.1410
6
Bots 3.7010
6
7.8110
6
(a) Out-degree centrality
Liberal Conservative
Humans 2.5210
6
4.2410
6
Bots 2.5310
6
6.2210
6
(b) In-degree centrality
the intra-group interactions to the 22% of their retweet activity. Interestingly, conservative
humans interacted with the conservative bots (28% of the time) much more than the liberal
counterpart (16%) with the liberal bots. To better understand these results and to measure
the eectiveness of both the strategies, in the next Section we evaluate the four metrics
introduced earlier in this paper.
Finally, we examine the degree of embeddedness of both humans and bots within the retweet
network. For this purpose, we rst compute dierent network centrality measures, and then
we adopt the k-core decomposition technique to identify the most central nodes in the graph.
In Table 3.4, we show the average out- and in-degree centrality for each group of users. Out-
degree centrality measures the quantity of outgoing links, while in-degree centrality considers
76
Table 3.5: Bot eectiveness
Metric Liberal Bots Conservative Bots
RTP 14.1% 25.6%
RR 4.5% 15.5%
H2BR 12.3% 23.2%
TSR 35.3% 35.0%
the number of of incoming links. Both of these measures are normalized by the maximum
possible degree of the graph. Overall, conservative groups have higher centrality measures
than the liberal ones. We can notice that conservative bots achieve the highest values both
for the out- and in-degree centrality. To further investigate bots embeddedness in the social
network, we use the k-core decomposition. The objective of this technique is to determine
the set of nodes deeply embedded in a graph. The k-core is a subgraph of the original graph
in which every node has a degree equal to or greater than a given value k. We extracted
the k-cores from the retweet network by varying k in the range between 0 and 30. Figure
3.4 depicts the percentage of liberal and conservative users as a function of k. We can
notice that, as k grows, the fraction of conservative bots increases, while the percentage of
liberal bots remains almost stationary. On the human side, the liberal fraction drops with
k, whereas the conservative percentage remains approximately steady. Overall, conservative
bots sit in a more central position in the social network and are more deeply connected if
compared to the liberal counterpart.
3.4.3 RQ3: Bot Eectiveness
In this Section, we aim to estimate the eectiveness of bot strategies and measure to what
extent humans rely upon, and interact with the content generated by social bots. We examine
the eect of bot activities by means of the four metrics described in Section Bot Activity
Eectiveness. We evaluate each political side separately, thus, we compare the interaction
between bots and humans with the same leaning. In Table 4.1, we depict the results for each
77
Figure 3.4: k-core decomposition, liberal vs. conservative users
group of bots. Diverse aspects are worthy of consideration. We can observe that conservative
bots are signicantly more eective than the liberal counterpart. Although the TSRs of the
red and blue bots are comparable, the gap between the two groups, with respect to the
other metrics, is signicant. To carefully interpret this result, it should also be noticed
that (i) the TSR is inversely proportional to the number of tweets generated by bots, and
(ii) conservative bots tweeted more than the liberal counterpart, as depicted in Table 3.2b.
Overall, conservative bots received a larger degree of interaction with (and likely trust from)
human users. In fact, conservative humans interacted with the bot counterpart almost twice
with retweets (RTP ), and more than three times with replies (RR) if compared to the liberal
group. Finally, the H2BR highlights a remarkable amount of human activities that involve
social bots: almost one in four actions performed by conservative humans goes towards red
bots.
78
3.5 Conclusions & Future Work
In this work, we conducted an investigation to analyze social bots activity during the 2018
US Midterm election. We showed that social bots are embedded in each political wing and
behave accordingly. We observed dierent strategies between conservative and liberal bots.
Specically, conservative bots stand in a more central position in the social network and
abide by the topic discussed by the human counterpart more than the liberal bots, which in
turn exhibit an in
ammatory attitude. Further, conservative bots balanced their interaction
with humans and bots of the red wing, whereas liberal bots focused mainly on the interplay
with the human counterpart. Finally, we inspected the eectiveness of these strategies and
recognized the strategy of the conservative bots as the most eective. However, these results
open the door to further interpretation and discussion. Are conservative bots more eective
because of their strategy or because of the human ineptitude to distinguish their nature?
This, and related analysis, will be expanded in future work.
79
Chapter 4
Evolution of Bot and Human
Behavior
4.1 Introduction
Social media networks have been dealing with considerable growth of fake and automated
accounts, namely social bots, extensively used to manipulate people's belief and, in the
political context, to aect their voting behavior. We monitored the evolution of bot and
human users during the last two US voting events for understanding their evolution and
interplay. We show that humans have become more skeptical in re-sharing others' content
and more inclined to discuss and argue their ideas. Bots mainly targeted humans in their
interaction, but they were less eective in involving them in the discussion.
Our results, although look auspicious for the next elections, highlight the mutable nature of
bots and open further challenges to forecast their evolution and to enhance their detection.
In particular, social bots represent an inorganic element within online political discussions
and have been indicated as one of the main factor contributing to social media manipulation.
80
compromising the integrity of the US elections. We compared 280K accounts that tweeted
during 2016 US Presidential and 2018 US Midterm elections. From 2016 to 2018, despite
changing their strategy, the bots became less eective. Furthermore, humans have become
more aware of the presence of bot accounts by the statistically signicant decrease in retweets
and statistically signicant increase in replies. Additionally, there was no causality between
the interactions in 2016; however, there was causality in 2018. We found that Bot to Human
interaction was causally predictive of both Bot to Bot and Human to Bot activity. Our
results indicate that eorts to counter bot eectiveness are working.
4.2 Background
In the last decade, political discussion and propaganda have been migrated from the real-
world to virtual platforms constituted by social media networks. The online ecosystem,
however, does not only include human beings but has given room to an increasing number of
fake and automated accounts, referred to as social bots. Although social network providers
attempted to purge their platforms, these accounts persisted and acted undisturbed for
years. Monitoring a set of accounts active on Twitter during the last two US voting events,
we characterize their activity, interplay, and evolution. We show that both bot and human
users changed their behavior over the years. Bots better mimicked humans activity and
targeted human accounts more than in the past. On the other hand, humans also mutated
their online conduct: they retweeted less and replied more with respect to their prior attitude.
Overall, we found a decreasing level of social endorsement received by bots from human users.
While this nding represents an encouraging result towards the next elections, it poses further
challenges in the understanding of social media players interaction and evolution. Online
discussion centered around politics becomes a critical topic every election cycle and it would
expect to be the case for the upcoming 2020 US Presidential election. More importantly,
81
there has been a signicant amount of work on bot detection, the 2016 US Presidential
election and the 2018 Midterm election centered on bots and humans as well as political
ideology of both. Analyzing the users and tweets from both the 2016 data set and the 2018
data set, we focus on the 280K users common to both time periods and determine that 13%
are bots. We show that while bot strategies have increased, their eectiveness has decreased
based on human engagement metrics. Furthermore, we show this via Granger causality
particularly in 2018. Our ndings illustrate proof toward a healthier conversation online and
steps we can make 2020 even better.
The purpose of this chapter is to study the Human and Social Bots behavior evolution from
2016 to 2018. Studying the evolution of humans and bots from the 2016 US Presidential
Election to the 2018 US Midterm Elections with the objective of characterizing their strategy,
activity, and interaction. We nd that Bots were more eective in involving humans in 2016
with respect to those in 2018. Bots adopted dierent strategy in their activity and the
interaction with humans, in terms of volume and sentiment in tweet, retweets, and replies.
Both in 2016 and 2018, bots sit in a more central position in the social network if compared
to humans. Furthermore, as it relates to Suspended vs. Not Suspended Accounts, the goal
would be to examine suspended accounts both in 2016 and 2018, study their activity (textual
and temporal), and classify their behavior as abusive or spammy.
Nowadays, social media networks embody the complex societal architecture in an online in-
frastructure. Accessing the news, sharing opinions, and bonding connections are just a few
of the various actions that individuals regularly perform online. Along with those activities,
political discussion and propaganda have also been relocated from the oine scene to the
virtual communication medium. However, dierently from the concrete (oine) human in-
terplay, the online conversation presents diverse and still largely unknown pitfalls, which in
turn can seriously impact real-world dynamics. Malicious actors, e.g., fake accounts, foreign
agents, and scammers, are embedded in our online social systems interacting with social
82
network users with the objective of deceiving the mass and manipulating the public opinion.
In the political context, the 2016 US Presidential election represents the rst remarkable
example of social media manipulation [12, 2, 141, 7]. Since then, social media networks
have been trying to suspend malicious actors to maintain a healthy conversation on their
platform. However, nefarious activity on social media does not show any sign of decline:
Social bots, automated and software-controlled accounts, and trolls, human operators asso-
ciated with foreign agencies, are still active on online platforms [67, 86] playing a crucial role
in information spreading and disinformation campaigns [8, 16, 37, 57, 64, 118, 120, 136] in
global events [109, 93, 65, 32, 123]. In particular, social media bots, due to their tremen-
dously scalable nature, represent the major concern in the ght against media manipulation
[15, 99, 108, 119, 134, 142], as further demonstrated by the recent purge of (millions of) fake
accounts by Facebook and Twitter.
12
Understanding how human users deal with this massive wave of fake accounts and manip-
ulation attempts is of extreme importance. On the other hand, detecting and keeping the
pace of increasingly sophisticated malicious accounts is needed to build and adapt ecient
countermeasures. Here, we show how both social bots and humans behavior evolved over
the last two US voting events: the 2016 Presidential election and the 2018 midterms. To this
end, we captured the online discussion on Twitter and monitored a set of 280k accounts that
were substantially active in both the events. We recognized 35k bots and discovered that
their strategy has changed. They better emulated humans tweet activity rate, they focused
more in targeting human population, and they shared more negative content. Humans also
mutated their conduct becoming more skeptical: they retweeted signicantly less content
but incremented the usage of replies to participate in the discussion. Overall, we found that,
in the 2018 midterms, bots received fewer social endorsement from humans and were less
eective in involving them in their conversation with respect to the 2016 election.
1
https://edition.cnn.com/2019/05/23/tech/facebook-transparency-report/index.html
2
https://www.bbc.com/news/technology-44682354
83
4.3 Data Collection
We capture the political discussion on Twitter by gathering election related posts (tweets)
during the two election periods. We employed the Python module Twython to collect tweets
through the Twitter Streaming API using a set of keywords as a lter. Keywords were
selected ad hoc per each election, as described in the following.
For the 2016 US Presidential Election, 23 keywords were used to collect tweets from Septem-
ber 16, 2016 to October 21, 2016. Details on the keywords are available in [12]. Overall,
42.1 million tweets generated from 5.9 million users were gathered.
For the 2018 US Midterms, tweets were collected from October 6, 2018 to November 19, 2018
using the following keywords: 2018midtermelections, 2018midterms, elections, midterm, and
midtermelections. As a result, we obtained 2.6 million tweets, which involved 997,406 users.
4.4 Data Processing
In this analysis, we take into account only the users present in both the election discus-
sions. Thereby, we consider the 278,181 accounts that tweeted both in 2016 and 2018. This
subset of users represent a continuum between the two election conversations, other than
the core of the online discussion. In fact, these users were involved (as authors of a tweet,
or as retweeted/replied users) in 54% and 65% of the tweets collected in 2016 and 2018,
respectively.
To examine the same time window for the two voting events, data from the two datasets
have been ltered considering only tweets ranging from the month prior to the day following
the election. Finally, this ltering results in 8,383,611 tweets from 2016 and 660,296 from
2018.
84
4.4.1 Bot Detection
To label the accounts as bots or humans, we rely on the widely used Botometer.
3
Botometer is
a machine learning-based tool developed by Indiana University [30, 134] to detect social bots
on Twitter. It is based on an ensemble classier [17] that aims to provide an indicator, namely
bot score, used to classify an account either as a bot or as a human. To feed the classier, the
Botometer API extracts about 1,200 features related to the Twitter account under analysis.
These features fall in six broad categories and characterize the account's prole, friends,
social network, temporal activity patterns, language, and sentiment. Botometer outputs a
bot score: the lower the score, the higher the probability that the user is human. Prior
studies used the 0.5 threshold to separate humans from bots. However, according to the
re-calibration introduced in Botometer v3 [142], along with the emergence of increasingly
more sophisticated bots, we here lower the bot score threshold to 0.3 (i.e., a user is labeled
as a bot if the score is above 0.3). This threshold corresponds to the same level of sensitivity
setting of 0.5 in prior versions of Botometer (cf. Fig 5 from [142]).
As a result, over the 278,181 accounts, 12.6% were classied as bots, 86.1% as humans, while
the remaining 1.3% of the accounts were not found on Twitter (indicating users that have
canceled their account or have been suspended for violation of the Twitter rules).
4.4.2 Political Ideology Inference
To classify users based on their political ideology, we rely on the political leaning of the
media outlets they share. We make use of a list of partisan media outlets released by third-
party organizations, such as AllSides
4
and Media Bias/Fact Check.
5
We combine liberal
and liberal-center media outlets into one list (composed of 641 outlets) and conservative and
3
https://botometer.iuni.iu.edu/
4
https://www.allsides.com/media-bias/media-bias-ratings
5
https://mediabiasfactcheck.com/
85
conservative-center into another (composed of 398 outlets). To label Twitter accounts as
liberal or conservative, we use a polarity rule based on the number of tweets they produce
with links to liberal or conservative sources. Thereby, if an account has more tweets with
URLs pointing to liberal sources, it is labeled as liberal and vice versa. Although the
overwhelming majority of accounts include URLs that are either liberal or conservative, we
remove any account that has equal number of tweets from each side.
Finally, we use label propagation to classify the remaining accounts from the seed of labeled
users, in a similar way to previous work (cf. [6]). For this purpose, we construct a social
network based on the retweets exchanged between users. The nodes of the retweet network
are the users, which are connected by a direct link if one user retweeted a post of another
user. To validate results of the label propagation algorithm, we apply a stratied cross (5-
fold) validation to the set composed of the seed accounts. We train the algorithm using 80%
of the seeds and we evaluate the performance on the remaining 20%. Finally, we compute
precision and recall by reiterating the validation of the 5-folds. Both precision and recall
scores show value around 0.89.
4.4.3 Sentiment Analysis
The choice of a sentiment analysis is highly dependent on the data and application, therefore
you need to take into account prediction performance and coverage. There is no single
method that always achieves a consistent rank position for dierent datasets. Therefore,
in this paper we test multiple methods for sentiment analysis. Most languages themselves
are biased positive and if a lexicon is built on data, the positive bias that data can lead
to a bias in the lexicon. This is why most methods are better at classifying positive than
neutral or negative methods meaning that they are biased, neutral are the hardest to detect
[111]. To map each tweet to the sentiment it express, we make use of a sentiment analysis
86
algorithm. More specically, we employ SentiStrength [1], a lexicon-based approach that
is particularly suited for social media text analysis. Lexicon-based algorithms are based
on sentiment lexicons, dictionary of emotions where words are attributed a given sentiment
strength. SentiStrength attributes a numerical score of sentiment intensity. In particular, it
returns separately both a positive and negative score, which range from 1 to 5 (with 5 being
the greatest strength). Overall, we are interested in the total sentiment, thus, we subtract
the negative sentiment from the positive one. Thereby, the nal score ranges from -4 (most
negative) to 4 (most positive). Finally, in Fig. 4.2B-C, we compute the cumulative sentiment
over time and we normalize by the number of tweets.
Additionally, positive sentiment strength and negative sentiment strength is scored sepa-
rately. Each is scored from 1 to 5, with 5 being the greatest strength. For our purposes,
we seek overall sentiment so we subtract the negative sentiment from the positive sentiment
so that strongly positive (5,1) becomes 4, neutral (1,1) becomes 0 and strongly negative
(1,5) becomes -4. Therefore, SentiStrength scores range from -4 (most negative) to 4 (most
positive). SentiStrength is designed to do better with social media; however, it can't exploit
indirect indicators of sentiment. It is also weaker for positive sentiment in news-related
discussions.
4.4.4 Bot Eectiveness
To estimate the eectiveness of bot actions in involving humans and, at the same time,
measure to what extent humans rely upon, and interact with the content generated by social
bots, we rely on the metrics introduced in [86] and detailed as follows:
Retweet Pervasiveness (RTP ) measures the intrusiveness of bot-generated content in
87
human-generated retweets:
RTP =
no. of human retweets from bot tweets
no. of human retweets
(4.1)
Reply Rate (RR) measures the percentage of replies given by humans to social bots:
RR =
no. of human replies to bot tweets
no. of human replies
(4.2)
Human to Bot Rate (H2BR) quanties human interaction with bots over all the human
activities in the social network:
H2BR =
no. of humans interaction with bots
no. of humans activity
, (4.3)
where the numerator counts for human replies/retweets to/of bots generated content,
while the denominator is the sum of the number of human tweets, retweets, and replies.
Tweet Success Rate (TSR) is the percentage of tweets generated by bots that obtained
at least one retweet by a human:
TSR =
no. of tweet retweeted at least once by a human
no. of bots tweets
(4.4)
4.4.5 Granger Causality
We use the Granger causality test [54, 55, 56] to determine whether the time series X can be
used to predict the time series Y. Clive Granger proposed that the the variable X Granger-
causes the variable Y if the predictions of future values of Y based on the combination of
the past values of X with the past values of Y are better than the predictions of Y based
only on the past values of Y. This holds true unless also the reverse (Y Granger-causes X) is
88
Oct 11 2016
Oct 18 2016
Oct 25 2016
Nov 01 2016
Nov 08 2016
−0.5
−0.4
−0.3
−0.2
−0.1
0.0
Cumulative Sentiment
Oct 10 2018
Oct 17 2018
Oct 24 2018
Oct 31 2018
Nov 07 2018
−0.3
−0.2
−0.1
0.0
0.1
0.2
Liberal Bots
Conservative Bots
Liberal Humans
Conservative Humans
Oct 10 2018
Oct 17 2018
Oct 24 2018
Oct 31 2018
Nov 07 2018
−0.25
−0.20
−0.15
−0.10
−0.05
0.00
0.05
Oct 11 2016
Oct 18 2016
Oct 25 2016
Nov 01 2016
Nov 08 2016
−0.35
−0.30
−0.25
−0.20
−0.15
−0.10
−0.05
Cumulative Sentiment
0 50 100 150 200
Tweet inter-time [minute]
0.000
0.002
0.004
0.006
0.008
0.010
10
0
10
1
10
2
10
3
10
4
Tweet inter-time [10 minutes]
10
-5
10
-4
10
-3
10
-2
10
-1
0 50 100 150 200
Tweet inter-time [minute]
0.000
0.002
0.004
0.006
0.008
0.010
Frequency
Bots
Humans
10
0
10
1
10
2
10
3
10
4
Tweet inter-time [10 minutes]
10
-6
10
-5
10
-4
10
-3
10
-2
10
-1
A
B
C
2016 2018
Figure 4.1: Users tweet activity in 2016 (left) and 2018 (right). (A): Consecutive tweet inter-
time for bots and humans (B): Cumulative sentiment over time of human and bot tweets
(C): Cumulative sentiment over time of human and bot tweets according to their political
leaning.
veried. In such a case, no conclusion can be drawn. We further apply a dierentiation to
remove seasonal eects and, then, we tested the stationary time series. The autoregression
of Y is augmented by lagged values of X and those individually signicant (t-statistic) that
increase the explanatory power of the regression (F-test).
89
4.5 Results
We explore bots and humans dynamics by measuring the time lag between consecutive shar-
ing activities (tweets). Fig. 4.1A displays the inter-time tweet distribution comparing bot
and human users in the 2016 and 2018 elections. Notably, in the 2016 election, the distri-
bution of bots tweet activity largely diers from the humans distribution. The discrepancy
is particularly relevant in the time range within 10 minutes and 3 hours, similarly to the
nding in [108], where bots shared content at a higher rate with respect to human users. On
the other side, in the recent midterm election (2018), inter-time distributions are similar,
suggesting that bots have been rened to better emulate human activity and to avoid detec-
tion. To shed light on the emotional content generated by bots and humans, we perform a
sentiment analysis (Material and Methods) of their tweets. Fig. 4.1B shows the cumulative
sentiment over time and reveals interesting trends between humans and bots emotion.
Although both in the Presidential election and in the midterm election bots shared more
negative contents with respect to humans, the gap is more noticeable in 2018. It should also
be noticed that, while in 2016 the gap decreases as the election day (red bar in Fig. 4.1) ap-
proaches, in 2018 the dierence is remarkable from a few weeks before the vote, highlighting
a more aggressive and negative behavior of bots during the last election. To better character-
ize and dierentiate the sentiment of each political wing, we assign a political leaning (liberal
vs. conservative) to each user following the procedure (Material and Methods) in [6]. In Fig.
4.1C, we show that conservative bots were more aligned with the sentiment of their human
counterpart both in 2016 and 2018 if compared to liberal bots, as it was also highlighted in
the analysis of the 2018 midterms in [86]. In fact, in both elections, liberal bots shared more
negative original contents with respect to liberal humans. More specically, in 2016 liberal
bots were more negative than any other group, whereas in 2018 liberal humans shared more
positive content if compared to the other users.
90
A
B
Figure 4.2: Human and bots interplay. (A): Retweets (a), replies (b), and mentions (c)
between bots and humans in 2016 and 2018. Percentages represent the amount of interaction
between each class of users. (B): Human-Bot retweet interaction according to their political
leaning in 2016 (a) and 2018 (b). Percentages indicate the group propensity to endorse
(retweet) other groups.
To examine humans and bots interaction, we measure the volume of retweet (re-sharing
the original content generated by other users), reply (responding to a post), and mention
(involving other users in a post) exchanged during the election periods. Each of these forms
of online interaction encodes a social meaning [122]: Retweeting is considered as a proxy
of social endorsement [92, 4] as users tend to share content they agree with, whereas reply
and mention are suitable means to accommodate a discussion, either complying or con
icting
with the topic of the post. Fig. 4.2A shows how the interplay (overall amount of retweets (a),
replies (b), and mentions (c)) between humans and bots has changed over the years. While
reply and mention distributions (Fig. 4.2A (b)-(c)) do not show broad dierences between
the two elections, diverse aspects are worth of consideration in the retweet dynamics. In the
2016 election, bots retweet activity was balanced between humans and bots (21% and 16%,
respectively). Dierently, in the 2018 midterms, bots were more inclined to engage with
humans by retweeting their posts more than three times the ones created by their similar
(19% vs. 6%), indicating the rise of increasingly sophisticated bots capable of adapting their
strategy and targeting users [123] in the social network. On the other side, humans endorsed
much less bot generated contents in 2018 with respect to 2016. More specically, during the
91
2016 election, humans retweeted bots almost one third of their time (19% vs. 44%), whereas
during the 2018 midterms the gap was more pronounced (7% vs. 68%).
To better characterize and comprehend the diminished level of endorsement received by
social bots, we consider the political leaning of both bot and human users. By measuring
the amount of retweet between the four groups (liberal humans, conservative humans, liberal
bots, conservative bots), in Fig. 4.2B, we quantify the group propensity to interact with the
other groups. To avoid visual clutter, we do not represent the interaction between the two
political sides, also because the majority of retweets are between groups of the same political
leaning. We observe that both in 2016 and 2018 liberal bots mainly targeted humans in their
interaction, while conservative bots interactions are more equalized between group members
and the human counterpart, especially in 2016. Further, in 2018, both bot factions increased
the number of retweets towards their human counterparts and, interestingly, received less
social endorsement from them with respect to the 2016 election. The augmented percentage
of bot retweets of human content is complementary to the reduced amount of retweets
between bots, which has been proven to elicit the illusion of a public consensus [39]. It is,
however, intricate to understand whether the reduced amount of endorsement is due to the
increasing level of interplay towards humans or to the minor echo provided by bots.
To further investigate the rationale of this mutated interplay, we evaluate the causality of bot
and human interactions. More specically, we measure the Granger causality [54] between
the daily volume of retweets between humans and bots. Results, also displayed in Fig.
4.3, show that, in the 2018 midterms, retweets from bots to humans (i.e., bots retweeting
human generated contents) were Granger-cause (5 days lag) of the retweets from humans
to bots, while there was no signal of causality in the 2016 election. This nding reveals
diverse aspects. It should be noticed that, while in 2018 humans were more conscious about
the presence of malicious actors in social media networks, in the 2016 Presidential election,
users allegedly re-shared contents disregarding the authenticity of the information and its
92
Figure 4.3: Volume of Bot-Human, Human-Bot, and Bot-Bot retweet interactions in 2018
93
Figure 4.4: Bots eectiveness in 2016 and 2018. (Left): RTP and RR distributions. (Right):
Information spread of bot generated content.
source. On the other side, in 2018 humans likely engaged with bots as a consequence of
bots prior interaction with the human population. Further, in 2018 midterms, the volume of
retweets from bots to humans was also Granger-cause (5 days lag) of the retweets from bots
to bots, suggesting that bots strategically distribute and organize their interactions with
other bots. Overall, results indicate that the bot-human interaction represents a proxy of
both human-bot and bot-bot interplay.
Relying on the metrics (Material and Methods) introduced in [86], we aim to further char-
acterize bots eectiveness in involving humans in the discussion. We found that bots were
more eective at engaging humans in 2016 than in the 2018 midterms. Results can be appre-
ciated in Table 4.1. The amount of humans retweets from bots generated tweets (RTP [4.1])
was three time larger in 2016 with respect to the 2018 midterms (29.4% vs. 9.3%), whereas
the number of human replies to bots generated tweets (RR [4.2]) was slightly higher in 2016
(5.8% vs. 4.4%). Overall, humans interaction with bots (H2BR [4.3]) was less noticeable in
2018 (19.5% vs. 6.5%) and bot tweets received more attention (TSR [4.4]) in 2016 (26.7% vs.
94
13.9%). Disaggregating at the user-level the metrics related to retweets (RTP) and replies
(RR), we further validate the reduced endorsement received by social bots in 2018 and the
moderate dierence in the reply interaction between the two election periods, as shown in
Fig. 4.4 (Left).
Additionally, in order to quantify the spread of bots generated content across the social
network, we propose to reconstruct the information cascade originated from a bot original
post. However, given that Twitter metadata does not provide information about intermedi-
ary retweets, we measure and indicate as cascade size the total number of retweets received
by each post, similarly to [122]. Results, depicted in Fig. 4.4 (Right), show that bots' infor-
mation was more spread in 2016 than in 2018 (average cascade size of 7.5 retweets in 2016
and 6.6 in 2018), further conrming bots larger pervasiveness in the Presidential election.
Although the gap in the social endorsement received by bots over the two election periods is
considerable, other forms of interaction, such as reply and mention, do not exhibit the same
discrepancy, as conrmed by previous ndings. To comprehend whether humans adapted
their behavior over the years, we investigate to what extent they relied on and make use of
the dierent forms of interplay. For this purpose, we measure the amount of human retweets,
replies, and mentions over all human interactions in the two voting events. Interestingly, the
amount of retweets decreased from 2016 to 2018 (from 85.5% in 2016 to 83.5% in 2018, Z
score = 29.5) along with the number of mentions (from 8.5% in 2016 to 7.3% in 2018, Z
score = 21.2), whereas the usage of replies signicantly increased in the 2018 midterms (from
6.0% in 2016 to 9.2% in 2018, Z score = -67.3). These results indicate a behavioral evolution
of humans in the usage of social networks. Despite retweets still represent the most used
form of interaction, human users appear to be more skeptical in endorsing other accounts.
This dubious attitude is also re
ected by the growth in the usage of replies, which indicate
humans propensity to discuss and argue their ideas instead of only sharing others' content.
This nding acquires even more signicance considering the cost related to each form of
95
Table 4.1: Bots Eectiveness
Metric US Presidential 2016 US Midterms 2018
RTP 29.4% 9.3%
RR 5.8% 4.4%
H2BR 19.5% 6.5%
TSR 26.7% 13.9%
interaction. While retweeting is a one-click operation, with a relatively small (or null) cost
in terms of time and eort, a reply requires a larger undertaking. Narrowing the spectrum
to the human interaction with bots, we found the same trend for each form of interaction,
indicating that humans behavioral evolution interested any kind of account.
Granger Causality at the daily level
No causality, 0way. Between H-B(2016) and B-B(2016)
No causality, 2way. Between H-B(2016) and B-H(2016) 1day
No causality, 2way. Between B-B(2016) and B-H(2016) 1day
Causality, 1way. Between B-H(2018) and B-B(2018) 5days
No causality, 0way. Between B-B(2018) and H-B(2018)
Causality, 1way. Between B-H(2018) and H-B(2018) 5days
Granger causality at the hourly level
No causality, 2way. Between H-B(2016) and B-B(2016)
No causality, 2way. Between H-B(2016) and B-H(2016)
No causality, 2way. Between B-B(2016) and B-H(2016)
No causality, 2way. Between B-H(2018) and B-B(2018)
96
No causality, 2way. Between B-B(2018) and H-B(2018)
No causality, 2way. Between B-H(2018) and H-B(2018)
Granger causality at the 12 hour level
No causality, 2way. Between B-H(2018) and H-B(2018)
No causality, 0way. Between B-B(2018) and H-B(2018)
No causality, 0way. Between B-H(2018) and B-B(2018)
Granger causality at day level for replies
No causality, 0way. Between B-H(2016) and H-B(2016)
No causality, 0way. Between B-B(2016) and H-B(2016)
Causality, 1way. Between B-H(2016) and B-B(2016) 3 days
No causality, 0way. Between B-H(2018) and H-B(2018)
No causality, 2way. Between B-B(2018) and H-B(2018)
No causality, 2way. Between B-H(2018) and B-B(2018)
4.6 Discussion and Conclusion
By analyzing the online discussion on Twitter during the last two US voting events, we found
strong evidence of bots and humans behavioral evolution on social media. Bots strategically
focused mainly on the interaction with humans and shared more negative content in the
recent midterms with respect to the 2016 Presidential election. On the other side, humans
97
also mutated their behavior becoming more skeptical in endorsing other accounts. They
retweeted signicantly less with respect to the 2016 election but were more inclined to discuss
through the usage of replies.
Although bots became more sophisticated and better emulated humans tweet activity in
the recent midterms, they were less eective at engaging humans in their conversation with
respect to the 2016 presidential election. In particular, in 2018, humans endorsed bots to
a less extent. While this nding represents an encouraging result towards a healthy online
conversation in the 2020 election, it should be contextualized considering social media history
and development over the last years. In fact, in 2016, the majority of social media users did
not acknowledge the existence of any malicious and/or software controlled players, whereas
an increasing level of awareness about nefarious actors in social media networks has grown
over the last few years. Humans metamorphosis represents an optimistic perspective that,
in turn, is bonded with bots evolution. Thereby, it remains dicult to distinguish whether
humans reinforced their level of attention on social media or bots strategy have been counter-
productive. This open problem initiates the way to further research and motivates the need
for a better understanding of social media players behavior, strategy, and evolution over
time.
98
Chapter 5
Gullible and Skeptic Agents
5.1 Introduction
Today, online social networks are instrumental to diusion of information at wide scale.
An active area of research from information diusion modeling to what this paper cov-
ers|identifying the seed users in an in
uence maximization setting. In
uence maximization
is an active area of research and many companies and organizations would like to have a
more realistic solution. While the in
uence maximization problem is NP-hard there are a
variety of approximation algorithms that give us some guarantees that are ecient as well.
In order to determine the diusion of the network, which is NP-complete, we leverage the
Monte Carlo sampling with 10,000 iterations. A social network is a group of individuals
and a listing of their relationships or interactions upon which information can be shared or
propagated.
Network diusion processes have historically been studied in social sciences with "word-
of-mouth" eect viral marketing eect in the success of new products adoption of various
strategies in various game theoretic settings [71]. The foundation of viral marketing is the
99
in
uence members can be initially targeted the a spread of in
uence will occur. Domingos
and Richardson proposed the underlying problem with specic applications to marketing
[34, 112]. Domingos to nd the in
uential nodes or seeds did this by considering a proba-
bilistic model of interaction. Kempe [71] this problem is addressed as a discrete optimization
problem which is NP-hard for most models presented. Given the general model optimization
problem in [34] cannot be approximated within a reasonable factor, we turn our attention
to the polynomial-time solvable model in [112]. Another seminal work on diusion is [11].
We focus on the target set selection problem of the in
uence maximization problem which
dened in [72] choose a set of nodes to initially activate such that the nal spread will be
maximized in expectation. Numerous studies concluded that social media can be a vehicle for
political manipulation, citing factors such as the eect of fake news and disinformation [106,
64, 120, 136, 8, 59, 16, 118, 57], bots [12, 141, 134, 99, 15, 119, 142], polarization [9, 5], etc.
Research also suggests that social media data comes with signicant biases that limit the
ability to forecast oine events, e.g., the outcomes of political elections [94, 45, 49, 46, 47, 48],
or public health issues [80, 3, 140]. There is signicant impact to in
uence maximization
across viral marketing [87, 34, 52, 112, 83, 76], healthcare communities [137, 25, 44] and
In
uence warfare [43, 24, 145] among many others. As noted in [85], people interact with
others in various multiplex networks where cooperation and competition abound. Here we
seek to incorporate that where the latent variable from our real-world evaluation will have
evidence of that embedded. Mathematical models could be applied to other phenomenon
like the social contagion eect in information diusion as presented in [82].
The focus of this paper is to determine how does incorporating the agent attribute score of
and in the modeling of diusion aect the contagion eect. More importantly how sensitive
is the spread under these models related to the original task of in
uence maximization seed
selection? Do we choose the most gullible and most stubborn agents by default? We propose
a model and test it with large synthetic networks. We then validate it with empirical data
100
from a collection from a large real-world network on Twitter with 6.5 million nodes and 100
million edges of people that posted concerning the 2020 US Presidential Candidates. Gullible
agents are nodes where there is a continues spectrum that goes from 0 to 1 and is a measure
of how likely a person is to share false content that has been debunked before. This 0 to 1
interval is a gullibility score where on one end at 0 we have a fact-checker and the other end
at 1 we have a gullible agent. The second aspect we want to incorporate is the stubbornness
metric. On one end at 0 we have
exible and at the other end at 1 we have stubborn.
This gullibility score is the likelihood that an agent will not change their beliefs on a given
topic. By incorporating heterogeneous, agent-level gullibility and stubbornness scores we
can show signicant improvement over the diusion task and the in
uence maximization
tasks. The goal for a new model of social in
uence is to achieve more realistic results with
faster computing speed and using less memory.
We develop a content-based and a network-based model to incorporate impressionability and
conformity metrics, respectfully. We modied four of the algorithms for the seed selection
tasks with the impressionability metric. We determined that depending on what your aim
is, you can target a subset of users without having to calculate the in
uence spread for
the excluded nodes saving precious time and memory. Likewise for the diusion models
where we consider the linear threshold and the independent cascade, we modify both with a
conformity score to make the diusion more realistic. Given a social network and budget k,
we use an IM algorithm KK-Greedy, CELF, CELF++ and UBLF with a diusion model of
LT and IC in order to establish a baseline then update the model with a impressionability
score and a conformity score and compare. As our results show on our synthetic networks,
the in
uence maximization task can be done in equal or less time, while getting the similar
or more realistic diusion spreads. Again, this would be helpful for in number of contexts
and from a number of dierent perspectives.
101
Figure 5.1: In
uence Maximization Task
5.2 Objective and Plan of Action
We have characterized dierent strategies and attempts of our adversary to manipulation
social media discussion. The next question is, how can we mitigate these issues? Beyond
detection and suspension of malicious accounts, we can imagine to spread counter messages
aimed at correcting or unveiling inaccurate information. To do so, the typical strategy is
to identify in
uential nodes in a network that could be used to eectively share accurate
information. In the following, we introduce a new model of in
uence maximization that aims
at capturing two novel \ingredients": (1) gullible and skeptic agents; and (2) conformability
to an agent's neighbors' norms and beliefs.
5.3 Related Work
5.3.1 Gullible Agents
Tambuscio et al. in [127] incorporated gullible and skeptic users as they looked at the factors
for which factual verication were eective at containing the spread of misinformation?
Their main nding was that a more segregated network propagates a hoax only with low
102
forgetting rates, but has no relationship when agents forget at faster rates. They were about
to illustrate this via simulation and mean-eld analysis. The four parameters of the model
are spreading rate, gullibility, probability to verify a hoax, probability to forget one's current
belief. The threshold value for the fact-checking probability that guarantees the complete
removal of the hoax from the network does not depend on the spreading rate, but only on
the gullibility and forgetting probability [127, 126]. The model reveals a threshold for the
fact-checking probability that guarantees the complete removal of a hoax from the network.
The threshold value does not depend on the spreading rate, but only on the gullibility
and forgetting probability. The model has not been tested in the framework of in
uence
maximization. However, this work diers from their work is that one, we incorporate skeptic
as a metric and we approach the problem from a microscopic view where we are interested
in the agents not just the overall dynamics.
5.3.2 Stubborn Agents
Stubborn agents have the potential to transform a stable society into an unstable society
based on perturbations, but this work was done independent of the gullibility work [51]
[143, 144]. They used the well known voter model that contents that an individual has
one of two opinions. Society is modeled as a static social network and at each time step,
randomly selected nodes have the chance. to communicate with their neighbors and update
their opinions. In this work, they introduce the concept of stubborn agents who have a
constant opinion, but can in
uence others. In short, they can continuously in
uence their
neighbors without ever changing their opinion. Social media serve as convenient platforms
for people to connect and to exchange ideas. However, social media networks like Twitter
and Facebook can be used for malicious purposes [36]. Especially in the context of political
discussion, there is a signicant risk of mass manipulation of public opinion. Concerning the
ongoing investigation of Russian meddling in the 2016 US Presidential election, [6] studied
103
political manipulation by analyzing the released Russian troll accounts on Twitter. After
using label propagation to assign political ideology, they found that Conservatives retweeted
Russian trolls over 30 times more than Liberals and produced 36 times more tweets. More
recently, [123] highlighted how bots can play signicant roles in targeting in
uential humans
to manipulate online discussion thus increasing in-ghting. Especially for the spread of fake
news, various studies showed how political leaning [2], age [57], and education [118] can
greatly aect fake news spread, alongside with other mechanisms that leverage emotions
[42, 41] and cognitive limits [104, 105]. Additionally, [35] showed how foreign actors can
more so than just backing one candidate or the other, often manipulate social media for the
purpose of sowing discord.
5.4 Algorithms
Since the goal of IM is to have a close to optimal solution, while keeping running time mini-
mized and keeping in mind memory usage, most of the implementations are either a version
of the greedy algorithm or a heuristic algorithm. However, given the intent of this work is
to is do a better job on seed selection with better being more spread on average, we will
focus only on the greedy algorithms. This is inline with current research eorts to improve
accuracy. There are various heuristic algorithms like ShortestPath [75], DegreeDiscount [21]
and others [139, 20, 69, 70, 114, 129, 128, 121] which are notably faster, but minus the
theoretical guarantees, very well could perform arbitrarily worse in real-world settings. For
a summary of the time complexity of the aforementioned heuristics see Table 1 in [85].
104
5.4.1 KK-Greedy
In [71], Kempe et al. propose a greedy algorithm which selects nodes with greatest marginal
gain and includes that node in the seed set, starting from the empty set. Furthermore,
the authors proved that their implementation of the greedy referred to in this paper as
KK-Greedy achieves (1 1=e) for any > 0. Therefore, the performance guarantee
is greater than 63%. In [71], in addition to proposing the in
uence maximization problem
as NP-hard they also proposed the KK-Greedy that uses the hill climbing approximation
algorithm. The hill climbing approach leverages the theory of submodular functions [26, 102].
Basically, start with an empty set for the active nodes, compute the spread for each of the
nodes, then include the highest and now compute the spread for the remaining nodes and
include that in the seed set. Repeat until the seed set is sizek. This is based on the intuition
that the spread function (S) is submodular so the node with maximum marginal gain for
each node
u
(S) at each round will be the best choice to include in the active set. Starting
with an initialization step where the maximum marginal spread must be computed for each
node. Once the maximum is found, that node is added to the active set and we then proceed
with testing every node with the current active set in the next round.
5.4.2 Cost Ecient, Lazy Forward
Leskovec et al. proposed an update to the KK-Greedy algorithm in [84] which is the Cost
Ecient Lazy Forward (CELF) algorithm. It is still based on KK-Greedy algorithm since the
rst iteration is the same, meaning the marginal gain for each node needs to be calculated.
However, for subsequent iterations, we just leverage the intuition that as the node selection
grows, the marginal increments or gain from adding a new node can never increase. In other
words, the marginal gain of a potential node in the current iteration cannot be greater than
its marginal gain in previous iterations. So instead of recomputing the marginal gain of the
105
current seed list with every other potential node, they do a lazy forward by going through
the decreasing order of each non-included node's marginal gain from the rst calculation.
From the decreasing ordered list of marginal gains, we need only to check at most kjAj
nodes in the next round.
5.4.3 Cost Ecient, Lazy Forward + +
In [53], they leverage o the above mentioned CELF which the marginal gain of a node in the
the current iteration cannot be better than its marginal gain in the previous iteration. CELF
maintains a sorted table of marginal gains in descending order. The new marginal gain is re-
evaluated for the top node at a time and if needed, the table is resorted. On the other hand,
CELF++ maintains a heap where there is a tuple which stores the following 4 values: 1)
marginal gain of u w.r.t the current seed set, 2) the node that had the previous best marginal
exclusive of the node u, 3) marginal gain of u w.r.t the current seed set and the previous best
node, 4) a
ag for when the last time 1) was updated. Algorithm should be implemented in
such a way that both
u
(S) and
u
(S
S
prev best) are evaluated simultaneously in a single
iteration of MC simulation.
5.4.4 Upper Bound, Lazy Forward
As compared to the previously mentioned algorithms, Upper Bound Lazy Forward (UBLF)
does not require the initial step of calculating the spread for all of the nodes. Instead they
propose to leverage the theoretical upper bound on the maximal marginal gain from each
node which they derived in [147] for the Independent Cascade case and [146] for the Linear
Threshold case. In doing so, the nodes all rank ordered by their upper bound and the spread
is calculated sequentially until the average spread of one node is greater than the upper
bound on the remaining nodes. This approach is summarized in [148].
106
5.5 Diusion Models
Social diusion models are used to solved a particular problem in the eld of social network
in
uence analysis. The problem we focus on in this paper is in
uence maximization. In
u-
ence maximization formally dened as the optimization problem to nd the most in
uential
nodes on a social network as dened in [71] which states given a social network of directed
interaction between nodes and a set budget which can be translated into maximum number
of seed nodes, nd the k nodes for which the in
uence propagation would be maximized with
respect to some diusion model. For the IC and LT models, this is a discrete optimization
problem for which optimal solution is non-polynomial time hard to calculate. Other prob-
lems in this space include but are not limited to in
uence minimization,
ow of in
uence
and individual in
uence.
5.5.1 Independent Cascade Model
The Independent Cascade (IC) model is a diusion model. Given a social network graph,
G = (V;E), composed of nodes (users on the platform), V and edges (directed information
sharing activity from one user to another), E. Also, given a budget that allows you to
initially in
uence or activate a set seedsS where (SV ) which is subset of all nodes in the
network. At each time step, t + 1, each activated node v
i
2 S
t
has the chance to activate
its out-neighbors v
j
2VnS
i
with an independent probability p
ij
. The stopping criterion is
when no additional nodes can be activated. Each node only gets one chance to activate is
out-neighbors. Lastly, once activated a node will remain active [71, 72].
107
5.5.2 Linear Threshold Model
The Linear Threshold (LT) model is dierent diusion model where we have the same social
network graphG = (V;E) as constructed above. In this case, each node has its on threshold
value
i
, randomly chosen from a uniform distribution between 0 and 1. Node v is in
uence
by each neighborw according to a weightb
vw
such that
P
w neighbor of v
b
v;w
1 to determine
if the sum of its neighbors are in
uenced enough for the node to be activated. If the thresh-
old is less than the sum of the in
uence weights of the neighbors, the node is activated,
P
w active neighbor of v
b
v;w

v
. Each node gets multiple chances to active is out-neighbors
by updating the sum in additional rounds. The stopping criteria is when no node's sum
of in
uence weights from neighbors changes and once activated a node will remain active
[71, 72].
Figure 5.2: Two Traditional Information Diusion Models
5.6 The Proposed Model
Below we diagram our concept for how the content-based and network-based model would
be. Gullible: Indicates a state where a node is very willing to share content as long as it ts
her current belief system without regard to how true the content might be..
108
Skeptic: Indicates a state where a node is willing to share content that ts her belief system
but must has some reasonable notion that it is factual. If they do not feel that it is factual,
they would hesitate to share. Worse case, they would leverage either a non-partisan site for
corroborating evidence (Wikipedia
1
, Snopes
2
, FactCheck.org
3
or Politifact
4
) or a news-
based site for approval.
Non-Conforming: Indicates a state where despite states of the local neighborhood, these
nodes have a stronger preference for their internal beliefs.
Conforming: Indicates a state where the current local neighbors will more likely than not
dictate the nal state of the node in question.
Now that we have reviewed the current IM algorithms and diusion models, here we will
introduce our updated IM algorithms for gullible/skeptic agents and our updated diusion
models for conformability. Individual node attributes will have a signicant impact on the
results, so hear we consider a network-based and content-based metric.
We dene two dimensions. Along the X-axis Gullibility: the likelihood that an agent will
accept false information (e.g., hoax). Along the Y-axis Conformability: the likelihood that an
agent will be in
uence by a neighbor. Based on \mixtures" of gullibility and conformability,
an agent can sit in one quadrant:
Quadrant 1: Gullible & Conforming
Quadrant 2: Skeptic & Conforming
Quadrant 3: Skeptic & Non-conforming
Quadrant 4: Gullible & Non-conforming
1
https://wikipedia.org
2
https://www.snopes.com
3
https://www.factcheck.org
4
https://www.politifact.com
109
Figure 5.3: Gullibility and Conformability
5.6.1 Gullible v. Skeptic Agents
The gullible skeptic score can be used depending on the type of information you want to
push. For example if you want to counter a misinformation campaign like the World Health
Organization for anti-vaccines then you would like the most in
uential skeptic agents. In
which case you could use our framework to sort the nodes before doing the IM to get the
in
uential skeptic agents or the neutral agents. However, if you wanted to do advertising
or spread a rumor / hoax, you would want to sort the users before doing the IM to get the
most in
uential gullible agents or neutral agents.
110
5.6.2 Conforming v. Non-conforming
In a social network graph, G(V;E) where V = 1;:::;n represents the set of nodes and E
i;j
represents a set of directed edges from node i to node j. For any node i2 V , we dene
the neighbor set asN
i
=jj(i;j)2E. There is a continuous real-valued entry for each node.
Which for can be divided into two nonempty disjoint sets V
0
;V
1
V . Unlike the work [144]
where they characterize the placement of stubborn nodes in the underlying graph and it's
aect on the asymptotic behavior of the node opinions, here we seek to address given a
conformity score how that could be implemented in the diusion model.
In the Modied Linear Threshold Model for information diusion, for each node, cal-
culate the ratio of 1-hop neighbors that are of the same political ideology. Further
update the threshold values by multiplying such that more conforming will retain a
higher threshold than non-conforming.
In the Independent Cascade Model for information diusion, for each node, calculate
the ratio of 1-hop neighbors that are of the same political ideology. Further update
the probability values by multiplying such that more conforming will retain a higher
probability than non-conforming.
5.7 Benchmark Graph
We will use synthetic networks for which we can control their structure. We will employ the
LFR (Lancichinetti, Fortunato, Radicchi) [79] generative model since it allows for hetero-
geneity in communities based on node degrees and size of communities, providing a realistic
setup for information spread modeling. For the conguration for the benchmark graphs we
used n = 500,1000, 5000. We also used the following settings for all runs: tau1 = 3, tau2 =
1.5, mu = 0.1, avg deg = 5, min community = 75. Explanation of the parameters:
111
tau1 = exponent of the power law distribution for degree distribution
tau2 = exponent of the power law distribution for community size
mu = community mixing coecient (how many links between nodes of a community
vs among nodes of dierent communities)
Figure 5.4: This is how a small LFR network looks like
.
5.8 Evaluation Metrics
Seed Size: number of initial active nodes (e.g., 100 seed nodes, or budget)
Spread: number of nodes reached at the termination of the diusion process
Time: execution time of the dierent models, measured in seconds (less is better, i.e.,
faster algorithm)
Iterations: number of diusion model propagation iterations
5.9 Results
The introduction of gullibility and conformability allows for signicantly less spread of false
information (see blue lines in Figure 5.5 and Figure 5.6), both as a function of seed size
112
Table 5.1: Diusion Evaluation IC or Baseline
Model Run MC Expected Robust Scale
Time Calls Spread
IC + KK Greedy 90 9,900 98 0.8 0.7
LT + KK Greedy 146 8,000 94 .70 .65
IC + CELF 90 9,900 98 0.8 0.7
LT + CELF 146 8,000 94 .70 .65
IC + CELF++ 90 9,900 98 0.8 0.7
LT + CELF++ 146 8,000 94 .70 .65
IC + UB 90 9,900 98 0.8 0.7
LT + UB 146 8,000 94 .70 .65
Table 5.2: Diusion Evaluation LT or Modied
Model Run MC Expected Robust Scale
Time Calls Spread
IC + KK Greedy 90 9,900 98 0.8 0.7
LT + KK Greedy 146 8,000 94 .70 .65
IC + CELF 90 9,900 98 0.8 0.7
LT + CELF 146 8,000 94 .70 .65
IC + CELF++ 90 9,900 98 0.8 0.7
LT + CELF++ 146 8,000 94 .70 .65
IC + UB 90 9,900 98 0.8 0.7
LT + UB 146 8,000 94 .70 .65
(budget) and as a function of number of iterations, and signicantly faster convergence (see
blue lines vs red lines in Figure ??).
5.10 Conclusion & Future Work
By incorporating heterogeneous, agent-level impressionability and stubbornness scores we
can show signicant improvement in the realism of the in
uence maximization task and
the diusion of information task. We focused our attention on algorithms with provable
guarantees instead of heuristics which while fast, could in the situation of real world perform
very poorly. We leveraged a content-based impressionability score from the user as a node
attribute, which allowed for use to focus on a subset of those nodes which in context concerns
the promoter. If the goal is the have a counter-campaign to put out a correction, you can
113
Figure 5.5: Diusion vs. Budget
Figure 5.6: Diusion vs. Time
focus on the skeptics and if you want to spread a rumor such as with fake news, you can focus
on the gullible. W then included a conformity score which is network-based and implemented
into the diusion models. For the more conrming nodes, we apply a linear threshold booster
to the threshold scores in LT and for the IC we increase the number of activation attempts.
Both applied to synthetic networks and evaluated on a Twitter 2020 Presidential retweet
network, these methods showed increased time because of the few MC calls and average
spread increases for realistic nodes.
Further work would be aligned along two main eorts. The rst is to continue to rene the
model with more data and testing. The second would be to add an adversarial, game theoretic
component to the model. There the goal would be to model the situation as a multiplayer
114
Figure 5.7: Convergence
game and develop optimal strategies for each class of player. In
uence operations are already
happening on the social media platforms, and these insights will prove of value to all parties
involved.
115
Chapter 6
Summary of Contributions
6.1 Analysis of Ads in US Elections
Veried that the Russian disinformation campaign aimed to sow discord
Charted the landscape of narratives used in the disinformation ads
6.2 Evolution of Bot and Human Behavior
Extracted insights about persistent bots and measured their change between 2016 and
2018 in terms of emotional dynamics as well as in terms of their engagement and
interactions with humans
6.3 Gullible and Skeptic Agents
Proposed a novel framework to perform in
uence maximization incorporating two new
`ingredients', namely individual gullibility and conformability dynamics
116
Bibliography
[1] Heart and soul: sentiment strength detection in the social web with sentistrength. In
Journal of Language and Social Psychology, pages 24{54, 2010.
[2] H. Allcott and M. Gentzkow. Social media and fake news in the 2016 election. Journal
of Economic Perspectives, 31(2):211{36, 2017.
[3] J.-P. Allem, E. Ferrara, S. P. Uppu, T. B. Cruz, and J. B. Unger. E-cigarette surveil-
lance with social media data: social bots, emerging topics, and trends. JMIR public
health and surveillance, 2017.
[4] S. Aral and D. Walker. Identifying in
uential and susceptible members of social net-
works. Science, 337(6092):337{341, 2012.
[5] M. Azzimonti and M. Fernandes. Social media networks, fake news, and polarization.
Technical report, National Bureau of Economic Research, 2018.
[6] A. Badawy, E. Ferrara, and K. Lerman. Analyzing the digital traces of political ma-
nipulation: The 2016 russian interference twitter campaign. In Int. Conference on
Advances in Social Networks Analysis and Mining, pages 258{265, 2018.
[7] A. Badawy, E. Ferrara, and K. Lerman. Analyzing the digital traces of political ma-
nipulation: The 2016 russian interference twitter campaign. In Proceedings of the
2018 IEEE/ACM ASONAM International Conference on Advances in Social Networks
Analysis and Mining, 2018.
[8] A. Badawy, K. Lerman, and E. Ferrara. Who falls for online political manipulation?
arXiv preprint arXiv:1808.03281, 2018.
[9] C. A. Bail, L. P. Argyle, T. W. Brown, J. P. Bumpus, H. Chen, M. F. Hunzaker, J. Lee,
M. Mann, F. Merhout, and A. Volfovsky. Exposure to opposing views on social media
can increase political polarization. PNAS, 115(37):9216{9221, 2018.
[10] G. Baltas. Determinants of internet advertising eectiveness: an empirical study.
International Journal of Market Research, 45(4):1{9, 2003.
[11] F. M. Bass. A new product growth for model consumer durables. Management science,
15(5):215{227, 1969.
117
[12] A. Bessi and E. Ferrara. Social bots distort the 2016 us presidential election online
discussion. First Monday, 21(11), 2016.
[13] D. M. Blei, A. Y. Ng, and M. I. Jordan. Latent dirichlet allocation. J. Mach. Learn.
Res., 3:993{1022, Mar. 2003.
[14] V. D. Blondel, J.-L. Guillaume, R. Lambiotte, and E. Lefebvre. Fast unfolding of com-
munities in large networks. Journal of statistical mechanics: theory and experiment,
2008(10):P10008, 2008.
[15] O. Boichak, S. Jackson, J. Hemsley, and S. Tanupabrungsun. Automated diusion?
bots and their in
uence during the 2016 us presidential election. In International
Conference on Information, pages 17{26. Springer, 2018.
[16] A. Bovet and H. A. Makse. In
uence of fake news in twitter during the 2016 us
presidential election. Nature communications, 10(1):7, 2019.
[17] L. Breiman. Random forests. Machine learning, 45(1):5{32, 2001.
[18] J. Chang, I. Rosenn, L. Backstrom, and C. Marlow. epluribus: Ethnicity on social
networks. ICWSM, 10:18{25, 2010.
[19] N. Chavoshi, H. Hamooni, and A. Mueen. Debot: Twitter bot detection via warped
correlation. In ICDM, pages 817{822, 2016.
[20] W. Chen, C. Wang, and Y. Wang. Scalable in
uence maximization for prevalent viral
marketing in large-scale social networks. In Proceedings of the 16th ACM SIGKDD
international conference on Knowledge discovery and data mining, pages 1029{1038.
ACM, 2010.
[21] W. Chen, Y. Wang, and S. Yang. Ecient in
uence maximization in social networks.
In Proceedings of the 15th ACM SIGKDD international conference on Knowledge dis-
covery and data mining, pages 199{208. ACM, 2009.
[22] Z. Chen and D. Subramanian. An unsupervised approach to detect spam campaigns
that use botnets on twitter. arXiv preprint arXiv:1804.05232, 2018.
[23] Z. Cheng, J. Caverlee, and K. Lee. You are where you tweet: a content-based approach
to geo-locating twitter users. In CIKM, pages 759{768, 2010.
[24] C. S. Chivvis. Understanding russian hybrid warfare. Rand Corporation, 2017.
[25] N. A. Christakis and J. H. Fowler. The spread of obesity in a large social network over
32 years. New England journal of medicine, 357(4):370{379, 2007.
[26] G. Cornuejols, M. L. Fisher, and G. L. Nemhauser. Exceptional paper|location of
bank accounts to optimize
oat: An analytic study of exact and approximate algo-
rithms. Management science, 23(8):789{810, 1977.
118
[27] A. Culotta. Reducing sampling bias in social media data for county health inference.
In Joint Statistical Meetings Proceedings, pages 1{12, 2014.
[28] K. S. Dave and V. Varma. Learning the click-through rate for rare/new ads from similar
ads. In Proceedings of the 33rd international ACM SIGIR conference on Research and
development in information retrieval, pages 897{898. ACM, 2010.
[29] C. A. Davis, G. L. Ciampaglia, L. M. Aiello, K. Chung, M. D. Conover, E. Ferrara,
A. Flammini, G. C. Fox, X. Gao, B. Gon calves, et al. Osome: the iuni observatory on
social media. PeerJ Computer Science, 2:e87, 2016.
[30] C. A. Davis, O. Varol, E. Ferrara, A. Flammini, and F. Menczer. Botornot: A system to
evaluate social bots. In Proceedings of the 25th International Conference Companion on
World Wide Web, pages 273{274. International World Wide Web Conferences Steering
Committee, 2016.
[31] A. Deb, L. Luceri, A. Badawy, and E. Ferrara. Perils and challenges of social
media and election manipulation analysis: The 2018 us midterms. arXiv preprint
arXiv:1902.00043, 2019.
[32] M. Del Vicario, F. Zollo, G. Caldarelli, A. Scala, and W. Quattrociocchi. Mapping
social dynamics on facebook: The brexit debate. Social Networks, 50:6{16, 2017.
[33] S. Dinh, T. Azeb, F. Fortin, D. Mouheb, and M. Debbabi. Spam campaign detection,
analysis, and investigation. Digital Investigation, 12:S12{S21, 2015.
[34] P. Domingos and M. Richardson. Mining the network value of customers. In Proceed-
ings of the seventh ACM SIGKDD international conference on Knowledge discovery
and data mining, pages 57{66. ACM, 2001.
[35] R. Dutt, A. Deb, and E. Ferrara. \senator, we sell ads": Analysis of the 2016 rus-
sian facebook ads campaign. In International Conference on Intelligent Information
Technologies, pages 151{168. Springer, 2018.
[36] E. Ferrara. Manipulation and abuse on social media. ACM SIGWEB Newsletter,
(Spring):4, 2015.
[37] E. Ferrara. Disinformation and social bot operations in the run up to the 2017 french
presidential election. First Monday, 22(8), 2017.
[38] E. Ferrara. Measuring social spam and the eect of bots on information diusion in
social media. In Complex Spreading Phenomena in Social Systems, pages 229{255.
Springer, 2018.
[39] E. Ferrara, O. Varol, C. Davis, F. Menczer, and A. Flammini. The rise of social bots.
Communications of the ACM, 59(7):96{104, 2016.
[40] E. Ferrara, O. Varol, F. Menczer, and A. Flammini. Detection of promoted social
media campaigns. In ICWSM, pages 563{566, 2016.
119
[41] E. Ferrara and Z. Yang. Measuring emotional contagion in social media. PloS one,
10(11):e0142390, 2015.
[42] E. Ferrara and Z. Yang. Quantifying the eect of sentiment on information diusion
in social media. PeerJ Computer Science, 1:e26, 2015.
[43] J. J. Forest. In
uence Warfare: How Terrorists and Governments Fight to Shape
Perceptions in a War of Ideas: How Terrorists and Governments Fight to Shape Per-
ceptions in a War of Ideas. ABC-CLIO, 2009.
[44] J. H. Fowler and N. A. Christakis. Dynamic spread of happiness in a large social
network: longitudinal analysis over 20 years in the framingham heart study. Bmj,
337:a2338, 2008.
[45] D. Gayo-Avello. Don't turn social media into another'literary digest'poll. Communi-
cations of the ACM, 54(10):121{128, 2011.
[46] D. Gayo-Avello. " i wanted to predict elections with twitter and all i got was this lousy
paper"{a balanced survey on election prediction using twitter data. arXiv preprint
arXiv:1204.6441, 2012.
[47] D. Gayo-Avello. No, you cannot predict elections with twitter. IEEE Internet Com-
puting, 16(6):91{94, 2012.
[48] D. Gayo-Avello. A meta-analysis of state-of-the-art electoral prediction from twitter
data. Social Science Computer Review, 31(6):649{679, 2013.
[49] D. Gayo Avello, P. T. Metaxas, and E. Mustafaraj. Limits of electoral predictions
using twitter. In ICWSM, 2011.
[50] A. Gelman and E. Loken. The garden of forking paths: Why multiple comparisons
can be a problem, even when there is no "shing expedition" or "p-hacking" and the
research hypothesis was posited ahead of time. Department of Statistics, Columbia
University, 2013.
[51] J. Ghaderi and R. Srikant. Opinion dynamics in social networks: A local interaction
game with stubborn agents. In 2013 American Control Conference, pages 1982{1987.
IEEE, 2013.
[52] J. Goldenberg, B. Libai, and E. Muller. Talk of the network: A complex systems look
at the underlying process of word-of-mouth. Marketing letters, 12(3):211{223, 2001.
[53] A. Goyal, W. Lu, and L. V. Lakshmanan. Celf++: optimizing the greedy algorithm
for in
uence maximization in social networks. In Proceedings of the 20th international
conference companion on World wide web, pages 47{48. ACM, 2011.
[54] C. W. Granger. Investigating causal relations by econometric models and cross-spectral
methods. Econometrica: Journal of the Econometric Society, pages 424{438, 1969.
120
[55] C. W. Granger. Testing for causality: a personal viewpoint. Journal of Economic
Dynamics and control, 2:329{352, 1980.
[56] C. W. Granger. Some recent development in a concept of causality. Journal of econo-
metrics, 39(1-2):199{211, 1988.
[57] N. Grinberg, K. Joseph, L. Friedland, B. Swire-Thompson, and D. Lazer. Fake news on
Twitter during the 2016 U.S. presidential election. Science, 363(6425):374{378, 2019.
[58] A. Gruzd, J. Jacobson, B. Wellman, and P. H. Mai. Social media and society: Intro-
duction to the special issue, 2017.
[59] A. Guess, J. Nagler, and J. Tucker. Less than you think: Prevalence and predictors of
fake news dissemination on facebook. Science Advances, 5(1):eaau4586, 2019.
[60] Z. Hajnal, N. Lajevardi, and L. Nielson. Voter identication laws and the suppression
of minority votes. The Journal of Politics, 79(2), 2017.
[61] B. Highton. Voter registration and turnout in the united states. Perspectives on
Politics, 2(3):507{515, 2004.
[62] B. Highton and R. E. Wolnger. Estimating the eects of the national voter registration
act of 1993. Political Behavior, 20(2):79{104, 1998.
[63] J. M. Holmes and D. G. Perkins. Multi-domain battle: Converging concepts toward a
joint solution. Joint Force Quarterly.
[64] P. N. Howard, G. Bolsover, B. Kollanyi, S. Bradshaw, and L.-M. Neudert. Junk news
and bots during the us election: What were michigan voters sharing over twitter.
CompProp, OII, Data Memo, 2017.
[65] P. N. Howard and B. Kollanyi. Bots,# strongerin, and# brexit: computational pro-
paganda during the uk-eu referendum. 2016.
[66] C. J. Hutto and E. Gilbert. Vader: A parsimonious rule-based model for sentiment
analysis of social media text. In ICWSM, 2014.
[67] J. Im, E. Chandrasekharan, J. Sargent, P. Lighthammer, T. Denby, A. Bhargava,
L. Hemphill, D. Jurgens, and E. Gilbert. Still out there: Modeling and identifying
russian troll accounts on twitter. arXiv preprint arXiv:1901.11162, 2019.
[68] M. Jacomy, T. Venturini, S. Heymann, and M. Bastian. Forceatlas2, a continuous graph
layout algorithm for handy network visualization designed for the gephi software. PloS
one, 9(6):e98679, 2014.
[69] Q. Jiang, G. Song, C. Gao, Y. Wang, W. Si, and K. Xie. Simulated annealing based in-

uence maximization in social networks. In Twenty-fth AAAI conference on articial
intelligence, 2011.
121
[70] K. Jung, W. Heo, and W. Chen. Irie: Scalable and robust in
uence maximization in
social networks. In 2012 IEEE 12th International Conference on Data Mining, pages
918{923. IEEE, 2012.
[71] D. Kempe, J. Kleinberg, and

E. Tardos. Maximizing the spread of in
uence through
a social network. In Proceedings of the ninth ACM SIGKDD international conference
on Knowledge discovery and data mining, pages 137{146. ACM, 2003.
[72] D. Kempe, J. Kleinberg, and

E. Tardos. In
uential nodes in a diusion model for social
networks. In International Colloquium on Automata, Languages, and Programming,
pages 1127{1138. Springer, 2005.
[73] B. Kim. Help america vote act, 2003.
[74] Y. M. Kim. The stealth media? groups and targets behind divisive issue campaigns
on facebook. 2018.
[75] M. Kimura and K. Saito. Tractable models for information diusion in social networks.
In European conference on principles of data mining and knowledge discovery, pages
259{271. Springer, 2006.
[76] J. Kleinberg. Cascading behavior in networks: Algorithmic and economic issues. Al-
gorithmic game theory, 24:613{632, 2007.
[77] M. Kruczkowski, E. Niewiadomska-Szynkiewicz, and A. Kozakiewicz. Fp-tree and svm
for malicious web campaign detection. In Asian Conference on Intelligent Information
and Database Systems, pages 193{201. Springer, 2015.
[78] S. Kudugunta and E. Ferrara. Deep neural networks for bot detection. Information
Sciences, 467(October):312{322, 2018.
[79] A. Lancichinetti, S. Fortunato, and F. Radicchi. Benchmark graphs for testing com-
munity detection algorithms. Physical review E, 78(4):046110, 2008.
[80] D. Lazer, R. Kennedy, G. King, and A. Vespignani. The parable of google
u: traps
in big data analysis. Science, 343(6176), 2014.
[81] D. Lazer, A. S. Pentland, L. Adamic, S. Aral, A. L. Barabasi, D. Brewer, N. Christakis,
N. Contractor, J. Fowler, M. Gutmann, et al. Life in the network: the coming age of
computational social science. Science, 323(5915):721, 2009.
[82] K. Lerman and R. Ghosh. Information contagion: An empirical study of the spread of
news on digg and twitter social networks. In Fourth International AAAI Conference
on Weblogs and Social Media, 2010.
[83] J. Leskovec, L. A. Adamic, and B. A. Huberman. The dynamics of viral marketing.
ACM Transactions on the Web (TWEB), 1(1):5, 2007.
122
[84] J. Leskovec, A. Krause, C. Guestrin, C. Faloutsos, C. Faloutsos, J. VanBriesen, and
N. Glance. Cost-eective outbreak detection in networks. In Proceedings of the 13th
ACM SIGKDD international conference on Knowledge discovery and data mining,
pages 420{429. ACM, 2007.
[85] K. Li, L. Zhang, and H. Huang. Social in
uence analysis: models, methods, and
evaluation. Engineering, 4(1):40{46, 2018.
[86] L. Luceri, A. Deb, A. Badawy, and E. Ferrara. Red bots do it better comparative
analysis of social bot partisan behavior. arXiv preprint arXiv:1902.02765, 2019.
[87] V. Mahajan, E. Muller, and F. M. Bass. New product diusion models in marketing:
A review and directions for research. Journal of marketing, 54(1):1{26, 1990.
[88] M. M. Malik, H. Lamba, C. Nakos, and J. Pfeer. Population bias in geotagged tweets.
People, 1(3,759.710):3{759, 2015.
[89] C. D. Manning, M. Surdeanu, J. Bauer, J. Finkel, S. J. Bethard, and D. McClosky.
The Stanford CoreNLP natural language processing toolkit. In Association for Com-
putational Linguistics (ACL) System Demonstrations, pages 55{60, 2014.
[90] J. Mattis. Summary of the 2018 national defense strategy of the united states of
america. Technical report, Department of Defense Washington United States, 2018.
[91] A. M. McCright and R. E. Dunlap. Combatting misinformation requires recognizing
its types and the factors that facilitate its spread and resonance. Journal of Applied
Research in Memory and Cognition, 6(4):389{396, 2017.
[92] P. Metaxas, E. Mustafaraj, K. Wong, L. Zeng, M. O'Keefe, and S. Finn. What do
retweets indicate? results from user survey and meta-review of research. In Ninth
International AAAI Conference on Web and Social Media, 2015.
[93] P. T. Metaxas and E. Mustafaraj. Social media and the elections. Science,
338(6106):472{473, 2012.
[94] P. T. Metaxas, E. Mustafaraj, and D. Gayo-Avello. How (not) to predict elections. In
3rd International Conference on Social Computing, pages 165{171, 2011.
[95] T. Mikolov, E. Grave, P. Bojanowski, C. Puhrsch, and A. Joulin. Advances in pre-
training distributed word representations. In Proceedings of the International Confer-
ence on Language Resources and Evaluation (LREC 2018), 2018.
[96] L. C. Minnite. The myth of voter fraud. Cornell University Press, 2017.
[97] A. Mislove, S. Lehmann, Y.-Y. Ahn, J.-P. Onnela, and J. N. Rosenquist. Understanding
the demographics of twitter users. ICWSM, 2011.
[98] S. M. Mohammad and P. D. Turney. Crowdsourcing a word-emotion association lexi-
con. 29(3):436{465, 2013.
123
[99] B. Mnsted, P. Sapie_ zy nski, E. Ferrara, and S. Lehmann. Evidence of complex con-
tagion of information in social media: An experiment using twitter bots. Plos One,
12(9):e0184148, 2017.
[100] F. Morstatter, J. Pfeer, H. Liu, and K. M. Carley. Is the sample good enough?
comparing data from twitter's streaming api with twitter's rehose. In ICWSM, 2013.
[101] F. Mosteller and L. W. Doob. The pre-election polls of 1948. Social Science Research
Council, 1949.
[102] G. L. Nemhauser, L. A. Wolsey, and M. L. Fisher. An analysis of approximations for
maximizing submodular set functions|i. Mathematical programming, 14(1):265{294,
1978.
[103] J. W. Pennebaker, M. E. Francis, and R. J. Booth. Linguistic inquiry and word count:
Liwc 2001. Mahway: Lawrence Erlbaum Associates, 71(2001):2001, 2001.
[104] G. Pennycook and D. G. Rand. Lazy, not biased: Susceptibility to partisan fake news
is better explained by lack of reasoning than by motivated reasoning. Cognition, 2018.
[105] G. Pennycook and D. G. Rand. Cognitive re
ection and the 2016 us presidential
election. Personality and Social Psychology Bulletin, 45(2), 2019.
[106] N. Persily. The 2016 us election: Can democracy survive the internet? Journal of
democracy, 28(2):63{76, 2017.
[107] E. Pitoura, P. Tsaparas, G. Flouris, I. Fundulaki, P. Papadakos, S. Abiteboul, and
G. Weikum. On measuring bias in online information. ACM SIGMOD Record,
46(4):16{21, 2018.
[108] I. Pozzana and E. Ferrara. Measuring bot and human behavioral dynamics. arXiv
preprint arXiv:1802.04286, 2018.
[109] J. Ratkiewicz, M. Conover, M. R. Meiss, B. Gon calves, A. Flammini, and F. Menczer.
Detecting and tracking political abuse in social media. ICWSM, 11:297{304, 2011.
[110] R. Rehurek and P. Sojka. Software framework for topic modelling with large corpora. In
In Proceedings of the LREC 2010 Workshop on New Challenges for NLP Frameworks.
Citeseer, 2010.
[111] F. N. Ribeiro, M. Ara ujo, P. Gon calves, M. A. Gon calves, and F. Benevenuto.
Sentibench-a benchmark comparison of state-of-the-practice sentiment analysis meth-
ods. EPJ Data Science, 5(1):1{29, 2016.
[112] M. Richardson and P. Domingos. Mining knowledge-sharing sites for viral marketing.
In Proceedings of the eighth ACM SIGKDD international conference on Knowledge
discovery and data mining, pages 61{70. ACM, 2002.
124
[113] H. Robinson, A. Wysocka, and C. Hand. Internet advertising eectiveness,the eect
of design on click-through rates for banner ads. International Journal of Advertising,
26(4):527{541, 2007.
[114] M. G. Rodriguez and B. Sch olkopf. In
uence maximization in continuous time diusion
networks. arXiv preprint arXiv:1205.1682, 2012.
[115] S. Rose, D. Engel, N. Cramer, and W. Cowley. Automatic keyword extraction from
individual documents. Text Mining: Applications and Theory, pages 1{20, 2010.
[116] D. Ruths and J. Pfeer. Social media for large studies of behavior. Science,
346(6213):1063{1064, 2014.
[117] M. Saher and J. Pathak. Malware and exploit campaign detection system and method,
Mar. 12 2015. US Patent App. 14/482,696.
[118] D. A. Scheufele and N. M. Krause. Science audiences, misinformation, and fake news.
PNAS, page 201805871, 2019.
[119] C. Shao, G. L. Ciampaglia, O. Varol, K.-C. Yang, A. Flammini, and F. Menczer. The
spread of low-credibility content by social bots. Nature communications, 9(1):4787,
2018.
[120] K. Shu, A. Sliva, S. Wang, J. Tang, and H. Liu. Fake news detection on social media: A
data mining perspective. ACM SIGKDD Explorations Newsletter, 19(1):22{36, 2017.
[121] G. Song, Y. Li, X. Chen, X. He, and J. Tang. In
uential node tracking on dynamic
social network: An interchange greedy approach. IEEE Transactions on Knowledge
and Data Engineering, 29(2):359{372, 2016.
[122] M. Stella, M. Cristoforetti, and M. De Domenico. In
uence of augmented humans in
online interactions during voting events. PloS one, 14(5):e0214210, 2019.
[123] M. Stella, E. Ferrara, and M. De Domenico. Bots sustain and in
ate striking opposition
in online social systems. arXiv preprint arXiv:1802.07292, 2018.
[124] V. Subrahmanian, A. Azaria, S. Durst, V. Kagan, A. Galstyan, K. Lerman, L. Zhu,
E. Ferrara, A. Flammini, F. Menczer, et al. The darpa twitter bot challenge. Computer,
49(6):38{46, 2016.
[125] C. R. Sunstein. # Republic: Divided democracy in the age of social media. Princeton
University Press, 2018.
[126] M. Tambuscio, D. F. Oliveira, G. L. Ciampaglia, and G. Ruo. Network segregation in
a model of misinformation and fact-checking. Journal of Computational Social Science,
1(2):261{275, 2018.
[127] M. Tambuscio, G. Ruo, A. Flammini, and F. Menczer. Fact-checking eect on viral
hoaxes: A model of misinformation spread in social networks. In Proceedings of the
24th international conference on World Wide Web, pages 977{982. ACM, 2015.
125
[128] Y. Tang, Y. Shi, and X. Xiao. In
uence maximization in near-linear time: A martingale
approach. In Proceedings of the 2015 ACM SIGMOD International Conference on
Management of Data, pages 1539{1554. ACM, 2015.
[129] Y. Tang, X. Xiao, and Y. Shi. In
uence maximization: Near-optimal time complexity
meets practical eciency. In Proceedings of the 2014 ACM SIGMOD international
conference on Management of data, pages 75{86. ACM, 2014.
[130] K. Toutanova, D. Klein, C. D. Manning, and Y. Singer. Feature-rich part-of-speech
tagging with a cyclic dependency network. In Proceedings of the 2003 Conference of the
North American Chapter of the Association for Computational Linguistics on Human
Language Technology-Volume 1, pages 173{180, 2003.
[131] U. A. Training and D. Command. Multi-domain battle: Evolution of combined arms
for the 21st century, 2025-2040, 2017.
[132] U. A. Training and D. Command. Tradoc pamphlet 525-3-1 \the us army in multi-
domain operations 2028,". Training and Doctrine Command, Ft. Eustis, VA,(6 De-
cember 2018), viii{x, 2018.
[133] O. Varol, E. Ferrara, C. Davis, F. Menczer, and A. Flammini. Online human-bot
interactions: Detection, estimation, and characterization, 2017.
[134] O. Varol, E. Ferrara, C. A. Davis, F. Menczer, and A. Flammini. Online human-bot
interactions: Detection, estimation, and characterization. In Int. AAAI Conference on
Web and Social Media, pages 280{289, 2017.
[135] O. Varol, E. Ferrara, F. Menczer, and A. Flammini. Early detection of promoted
campaigns on social media. EPJ Data Science, 6(1):13, 2017.
[136] S. Vosoughi, D. Roy, and S. Aral. The spread of true and false news online. Science,
359(6380):1146{1151, 2018.
[137] G. Wang, W. Jiang, J. Wu, and Z. Xiong. Fine-grained feature-based social in
uence
evaluation in online social networks. IEEE Transactions on parallel and distributed
systems, 25(9):2286{2296, 2013.
[138] T. Wang. The politics of voter suppression: Defending and expanding Americans' right
to vote. Cornell University Press, 2012.
[139] Y. Wang, G. Cong, G. Song, and K. Xie. Community-based greedy algorithm for
mining top-k in
uential nodes in mobile social networks. In Proceedings of the 16th
ACM SIGKDD international conference on Knowledge discovery and data mining,
pages 1039{1048. ACM, 2010.
[140] M. L. Williams, P. Burnap, and L. Sloan. Crime sensing with big data: The aordances
and limitations of using open-source communications to estimate crime patterns. The
British Journal of Criminology, 57(2), 2017.
126
[141] S. C. Woolley and D. R. Guilbeault. Computational propaganda in the united states
of america: Manufacturing consensus online. Computational Propaganda Research
Project, page 22, 2017.
[142] K.-C. Yang, O. Varol, C. A. Davis, E. Ferrara, A. Flammini, and F. Menczer. Arming
the public with ai to counter social bots. Human Behavior and Emerging Technologies,
1(1), 2019.
[143] E. Yildiz, D. Acemoglu, A. E. Ozdaglar, A. Saberi, and A. Scaglione. Discrete opinion
dynamics with stubborn agents. Available at SSRN 1744113, 2011.
[144] E. Yildiz, A. Ozdaglar, D. Acemoglu, A. Saberi, and A. Scaglione. Binary opinion
dynamics with stubborn agents. ACM Transactions on Economics and Computation
(TEAC), 1(4):19, 2013.
[145] S. Zannettou, T. Cauleld, E. De Cristofaro, M. Sirivianos, G. Stringhini, and J. Black-
burn. Disinformation warfare: Understanding state-sponsored trolls on twitter and
their in
uence on the web. In Companion Proceedings of The 2019 World Wide Web
Conference, pages 218{226. ACM, 2019.
[146] C. Zhou, P. Zhang, J. Guo, and L. Guo. An upper bound based greedy algorithm for
mining top-k in
uential nodes in social networks. In Proceedings of the 23rd Interna-
tional Conference on World Wide Web, pages 421{422. ACM, 2014.
[147] C. Zhou, P. Zhang, J. Guo, X. Zhu, and L. Guo. Ublf: An upper bound based approach
to discover in
uential nodes in social networks. In 2013 IEEE 13th International
Conference on Data Mining, pages 907{916. IEEE, 2013.
[148] C. Zhou, P. Zhang, W. Zang, and L. Guo. On the upper bounds of spread for greedy
algorithms in social network in
uence maximization. IEEE Transactions on Knowledge
and Data Engineering, 27(10):2770{2783, 2015.
[149] D. Zucchino. Thunder Run: The Armored Strike to Capture Baghdad. Open Road+
Grove/Atlantic, 2007.
127

Abstract (if available)

Abstract Modern society is a very complex system of people, institutions and ideas where individuals and organizations can become polarized into subgroups with conflicting ideology. With the widespread use of social media today, people’s perception of reality can be influenced by the messaging content to which they are exposed and the source from whom it comes. To that end, I developed a framework for modeling information operations and diffusion dynamics on social media networks. We present the context of Information Operations in the Multi-Domain Battle (MDB) paradigm which includes land, sea, air, space and cyberspace. As part of cyberspace, social media platforms have been used to spread false or misleading information effectively, particularly because of how easy and how fast it is to reach a consensus of users. After analyzing the 3,500 Russian Facebook ads purchased during the 2016 US Presidential campaign, we found that the liberal and conservative ads were just as effective and knowing that these ads came from the same group, we determined that the purpose was to sow chaos. However, with more sophisticated approaches, platforms and researchers have targeted the automated, inorganic user accounts. Therefore, the algorithmically controlled account users will have to adapt, and this research is one of the first attempts at targeting their adaptation. We studied 245K accounts that posted during the 2016 and 2018 elections, and bot behavior in 2016 was mechanical with a lot of retweets, but in 2018 bots aligned with human activity trends which might indicate bots are evolving. Likewise on the human side, there was a noticeable shift from retweets to replies which might indicate a desire to discuss their ideas instead of just retweeting other user’s content. We contend that the organized campaigns could benefit from an approach in Influence Maximization (IM) using algorithms that have theoretical guarantees over social media networks using various network diffusion models. This task is combinatorial complex, both approximation algorithms for the IM task and Monte Carlo simulations for the diffusion models must be utilized. Under this framework, we addressing gullible and stubborn agents by incorporating user information from a content-based and network-based approach could provide increased speed and reduced computations, while only marginally sacrificing the spread effectiveness.

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

Diffusion network inference and analysis for disinformation mitigation

PDF

Measuing and mitigating exposure bias in online social networks

PDF

Modeling social and cognitive aspects of user behavior in social media

PDF

Predicting and modeling human behavioral changes using digital traces

PDF

Time-sensitive social media mining and its applications

PDF

Towards trustworthy and data-driven social interventions

PDF

Leveraging programmability and machine learning for distributed network management to improve security and performance

PDF

Combining textual Web search with spatial, temporal and social aspects of the Web

PDF

Disentangling the network: understanding the interplay of topology and dynamics in network analysis

PDF

Understanding diffusion process: inference and theory

PDF

Domical: a new cooperative caching framework for streaming media in wireless home networks

PDF

Improving information diversity on social media

PDF

Computing cascades: how to spread rumors, win campaigns, stop violence and predict epidemics

PDF

Socially-informed content analysis of online human behavior

PDF

Modeling intermittently connected vehicular networks

PDF

Analysis and prediction of malicious users on online social networks: applications of machine learning and network analysis in social science

PDF

Graph embedding algorithms for attributed and temporal graphs

PDF

Spatiotemporal traffic forecasting in road networks

PDF

Aging analysis in large-scale wireless sensor networks

PDF

Heterogeneous graphs versus multimodal content: modeling, mining, and analysis of social network data

Asset Metadata

Creator Deb, Ashok Kumar (author)

Core Title Modeling information operations and diffusion on social media networks

School Viterbi School of Engineering

Degree Doctor of Philosophy

Degree Program Computer Science

Degree Conferral Date 2021-05

Publication Date 05/05/2021

Defense Date 08/14/2020

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag information diffusion,network analysis,OAI-PMH Harvest,social media networks

Format application/pdf (imt)

Language English

Contributor Electronically uploaded by the author (provenance)

Advisor Ferrara, Emilio (committee chair), Rice, Eric (committee member), Shahabi, Cyrus (committee member), Sukhatme, Gaurav (committee member)

Creator Email adeb@usc.edu,ashok.deb@gmail.com

Permanent Link (DOI) https://doi.org/10.25549/usctheses-oUC112720075

Unique identifier UC112720075

Identifier etd-DebAshokKu-9604.pdf (filename)

Legacy Identifier etd-DebAshokKu-9604

Document Type Dissertation

Format application/pdf (imt)

Rights Deb, Ashok Kumar

Type texts

Source 20210507-wayne-usctheses-batch-835-shoaf (batch), University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Access Conditions The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the author, as the original true and official version of the work, but does not grant the reader permission to use the work if the desired use is covered by copyright. It is the author, as rights holder, who must provide use permission if such use is covered by copyright. The original signature page accompanying the original submission of the work to the USC Libraries is retained by the USC Libraries and a copy of it may be obtained by authorized requesters contacting the repository e-mail address given.

Repository Name University of Southern California Digital Library

Repository Location USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA

Repository Email cisadmin@lib.usc.edu