University of Southern California Dissertations and Theses

Socially-informed content analysis of online human behavior

(USC Thesis Other)

Socially-informed content analysis of online human behavior

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content SOCIALLY-INFORMED CONTENT ANALYSIS OF ONLINE HUMAN BEHAVIOR
by
Julie Jiang
A Dissertation Presented to the
FACULTY OF THE USC GRADUATE SCHOOL
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulfillment of the
Requirements for the Degree
DOCTOR OF PHILOSOPHY
(COMPUTER SCIENCE)
May 2024
Copyright 2024 Julie Jiang

To my little sister.

Acknowledgments
This dissertation represents not just the culmination of my PhD research but also the symbolic distillation
of five fantastic years of my 20s. For this, I have many people to thank. This journey of professional and
personal growth has been made possible only by the constellation of people who have generously shared
their mentorship, wisdom, camaraderie, friendship, and love with me. At the core of this dissertation on
human behavior is my belief that the very essence of our humanity lies within our interpersonal interactions. It is with great privilege and honor that I extend my heartfelt thanks within these pages to those
who have enriched my life.
First and foremost, I thank my amazing advisor, Emilio Ferrara. His remarkable research acumen,
sharp wit, and occasional dry humor have been a constant source of inspiration. Without his guidance,
I would not have carved the path to becoming a computational social scientist, forging my own research
journey. Emilio’s insightful, constructive feedback, coupled with his generous encouragement, has been
invaluable. I thank Kristina Lerman, whose pivotal role in my emerging research career cannot be overstated. Her endless stream of new and exciting research ideas continually impresses me. My appreciation
also extends to my dissertation committee members, Pablo Barberá and Marlon Twyman, whose thoughtful and engaging comments have made this dissertation extra awesome, as well as my previous committee
members and collaborators, Xiang Ren and Jesse Thomason, who helped me co-author some of the proudest works I have published. Special thanks go to Joe Walter for his ingenious research ideas, exceedingly
iii

helpful and kind feedback, and witty emails that never fail to make me laugh. I have learned immensely
from each and every one of you, and for that, I am extremely grateful.
I extend my deepest thanks to my mentors from beyond USC. From Tufts, my gratitude goes to LiPing Liu, Soha Hassoun, and Nate Bragg for setting the foundations of my research career. From Snap
Research, I thank Francesco Barbieri, Leo Neves, Maarten Bos, Ron Dotsch, Neil Shah, Nils Murrugarrallerena, Vitor Sousa, and Shubham Vij. From Spotify Research, I thank Sam Way, Ang Li, Aditya Ponnada,
and Ben Lacker. It has been my privilege to have such incredible mentors guiding me.
Throughout my PhD journey, I have had the privilege of meeting a handful of researchers who have
also become dear friends. My heartfelt thanks go to Luca Luceri, one of my closest colleagues and friends,
from whom I have learned the virtues of patience and dedication in being a supportive mentor, as well as
the importance of maintaining a healthy work-life balance. I am grateful to have Bijean Ghafouri, who
serves as a wise sounding board when I find myself too absorbed in my thoughts and who is, just as
importantly, an extremely fun friend outside of work. My thanks also go to Ashok Deb, whose steadfast
optimism and kindness never cease to humble me. I am incredibly fortunate to have each of you to share in
every small triumph and to commiserate over every paper rejection. I also thank my lab mates and other
colleagues I met through work—Herbert Chang, Emily Chen, Alex Spangher, Alex Bisberg, Goran Murić,
Jinyi Ye, Shen Yan, Yilei Zeng, David Chu, Patrick Gerard, Francesco Pierri, Gabriela Pinto, Priyanka Dey,
Charles Beckham, Eun Cheol Choi, Adam Badawy, Basel Shbita, Yulin Yu, Wen Xie, Xiao Fu, Indira Sen,
Giuseppe Russo, Manoel Horta Ribeiro, and Dominik Bär. Our productive research collaborations and our
random rants about the PhD program have made my time here infinitely more enjoyable and fulfilling.
I am also profoundly grateful for the lifelong friends I have made. This journey would have been
impossible without the support of Min, Phil, and Rox, my three closest friends from college. I cherish our
in-depth, late-night discussions, where we chatted about everything in life and more. Your unwavering
support has seen me through the best and worst of times. Though we have all changed since our college
iv

days–a bit older, probably wiser, and perhaps more jaded–I treasure our closeness despite the physical
distances. Min, your genuineness, trustworthiness, and openness make you an irreplaceable friend in my
life. Phil, your high-achieving yet easy-going personality continually inspires me, and I know we will
always have each other’s back. Rox, your confidence is infectious, and your ability to listen has always
brought comfort to me. I am also fortunate to have made incredible friends in LA. Special thanks to
Mercedes, whose kindness and vibrancy light up every room, and Maja, who is often the most considerate
and funniest person in the room. I want to give a shout-out to my friends from college, LA, and beyond,
who have shared countless memorable holidays and excursions with me, including Jenn, Lydia, Steph, Ian,
Rachel, Florence, Noah, Olivia W., David Y., Brittany, Iszzy, Eeuny, Gab, the other Gab, Audrey, Daniel,
Kyara, Ixchel, Gianluca, Jordan, Tijana, Charles, Kim, Melody, Jaz, Felix, and Sunny. Every day feels extra
special when I am with you.
Above all, I thank my family. My academic pursuits were made possible only by their constant support. I am especially grateful for my mom, Grace, who has enthusiastically cheered me on since day one,
and my dad, Michael, who is always the pillar we depend on. You always believed in me, even when I
doubted myself. My sister Jamie, no longer the “baby” of the family but forever so in my heart, deserves
special thanks. Knowing that a younger sibling might be looking up to me provides a perpetual source of
motivation, especially considering she is not only intelligent and bright but also, more importantly, one of
the most empathetic, kind, and driven people I know. Jamie, while I may be a role model to you, it’s your
qualities that inspire me to be the best I can be. And finally, I cannot mention my family without talking
about my grandparents. My wai gong wai po raised me with nothing but love, patience, dedication, and a
lot of really great homemade food. I love you all.
Los Angeles, California
April 6, 2024
Julie
v

Table of Contents
Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iii
List of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . x
List of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xii
Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv
Chapter 1: Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 Computational Social Science . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.2 Social Network Homophily . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.3 Technical Challenges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.4 Research Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1.5 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
I Social Network Representation Learning 13
Chapter 2: Retweet-BERT: Learning from Language Features and Retweet Diffusions . . . . . . . . 16
2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.2 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.2.1 Ideology Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.2.2 Socially-Infused Text Mining . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
2.3 Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
2.3.1 Content Cues: Profiles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
2.3.2 Interaction Cues: Retweet Network . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
2.3.3 Data Preprocessing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
2.4 Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
2.4.1 Pseudo-Label Generation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
2.4.2 Methods for Polarity Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
2.4.3 Retweet-BERT Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
2.5 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
2.5.1 Automatic Evaluation on Seed Users . . . . . . . . . . . . . . . . . . . . . . . . . . 28
2.5.2 Human Evaluation on Held-out Users . . . . . . . . . . . . . . . . . . . . . . . . . 30
2.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
Chapter 3: Social-LLM: Learning from Social Network Data . . . . . . . . . . . . . . . . . . . . . . 34
3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
vi

3.2 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
3.3 Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
3.3.1 Content Cues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
3.3.2 Network Cues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
3.3.3 Social-LLM Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
3.3.4 Advantages and Disadvantages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
3.4 Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
3.4.1 Politics in COVID Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
3.4.2 2020 Presidential Election . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
3.4.3 Morality in COVID Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
3.4.4 Ukraine-Russia Suspended Accounts . . . . . . . . . . . . . . . . . . . . . . . . . . 43
3.4.5 Ukraine-Russia Hate Speech . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
3.4.6 Immigration Hate Speech . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
3.5 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
3.5.1 Baseline Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
3.5.2 Experimental Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
3.6 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
3.6.1 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
3.6.2 Visualization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
3.7 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
II Data-Driven Analysis of Online Human Behavior 56
Chapter 4: Social Media Polarization and Echo Chambers Surrounding COVID-19 . . . . . . . . . . 57
4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
4.2 Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
4.2.1 Interaction Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
4.2.2 Data Preprocessing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
4.3 Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
4.4 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
4.4.1 The Roles of Partisan Users . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
4.4.2 The Polarity of Influencers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
4.4.3 Echo Chambers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
4.4.4 Random Walk Controversy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
4.4.5 Popular Users Among the Left and Right . . . . . . . . . . . . . . . . . . . . . . . . 68
4.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
4.5.1 Implications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
4.5.2 Future Direction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
4.5.3 Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
Chapter 5: Social Approval and Network Homophily as Motivators of Online Toxicity . . . . . . . 73
5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
5.2 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
5.2.1 Homophily in Toxic Behavior . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
5.2.2 Social Motivators of Toxic Behavior . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
5.3 Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
5.3.1 Data Preprocessing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
vii

5.4 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
5.4.1 Measuring Toxicity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
5.4.2 Measuring Network Homophily . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
5.4.3 Measuring the Effects of Social Engagement on Toxicity . . . . . . . . . . . . . . . 79
5.5 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
5.5.1 Homophily in Toxic Behavior . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
5.5.2 The Effect of Social Engagement on Toxicity . . . . . . . . . . . . . . . . . . . . . . 83
5.6 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
5.6.1 Implications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
5.6.2 Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
5.6.3 Ethical Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
Chapter 6: Moral Values Underpinning COVID-19 Online Communication Patterns . . . . . . . . . 91
6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
6.2 Background and Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
6.2.1 The Moral Foundation Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
6.2.2 Morality and Politics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
6.2.3 Moral Homophily . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
6.2.4 Morality and COVID-19 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
6.3 Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
6.4 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
6.4.1 Detecting the Morality of Tweets and Users . . . . . . . . . . . . . . . . . . . . . . 98
6.4.2 Detecting User Groups Using MFT and Twitter Activity . . . . . . . . . . . . . . . 99
6.4.3 Detecting User Partisanship . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
6.5 RQ1: User Groups by Morality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
6.5.1 User Visualization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
6.6 RQ2: Moral Homophily . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
6.6.1 Homophily of Users . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
6.6.2 Homophily of User Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
6.7 RQ3: Bridging Moral Divides . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
6.8 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
6.8.1 Implications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
6.8.2 Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
6.8.3 Ethical Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
Chapter 7: Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
7.1 Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
7.2 Broader Impact . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135
Appendix A . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136
A.1 Heuristics-based Pseudo-Labeling Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136
A.2 Data Preprocessing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136
A.3 Hyperparameter Tuning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137
Appendix B . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
viii

B.1 Data Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
B.1.1 User Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
B.1.2 Bot Score and Homophily in Toxic Behavior . . . . . . . . . . . . . . . . . . . . . . 139
B.2 Social Engagement Experiment Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140
B.2.1 Social Engagement Estimation Model . . . . . . . . . . . . . . . . . . . . . . . . . . 140
B.3 Additional Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142
B.3.1 Results with Higher Bot Threshold . . . . . . . . . . . . . . . . . . . . . . . . . . . 142
B.3.2 Robustness Checks with Other Toxicity Measures. . . . . . . . . . . . . . . . . . . 143
ix

List of Tables
2.1 Retweet-BERT results for political leaning classification on seed users on the main CovidPolitical dataset (N = 79, 000) and the secondary Election-2020 dataset (N = 75, 000).
The best F1 (macro) scores for each model type are shown in bold, and the best overall
scores are underlined. Retweet-BERT outperforms all other models on both datasets. . . . 29
2.2 Retweet-BERT results on 85 users with human-annotated political-leaning labels from a
random sample of 100 users without seed labels. Retweet-BERT outperforms all models. . 30
3.1 Summary statistics of our Twitter datasets. . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
3.2 Social-LLM results on the classification task datasets evaluated using Macro-F1 scores.
The best model for each experiment is in bold, and the best baseline model is underlined. . 48
3.3 Social-LLM results on the regression task datasets evaluated using Pearson correlation
scores. The best model for each experiment is in bold, and the best baseline model is
underlined. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
5.1 Hate scores among users exhibit homophily in the hate speech dataset social network, as
indicated by both the network assortativity and the Pearson correlation between a user’s
hate score and the weighted average of their neighbors’ (***p < 0.001) . . . . . . . . . . . . 82
5.2 The number of anchor tweets and their corresponding number of unique users (bot score
<= 0.5) when the engagement metric that the anchor tweet received is substantially
lower or higher than predicted (k = 50) in the hate speech dataset. . . . . . . . . . . . . . 84
5.3 The average change in toxicity at k = 50 when an anchor tweet received substantially
higher or lower likes-per-quotes or retweets-per-quotes than expected in the hate speech
dataset. Statistical significance from a Mann-Whitney U test is indicated (** p < 0.01, ***
p < 0.001). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
6.1 Hypothetical tweets, adapted from real ones in the COVID-19 dataset, that contain
detected moral foundations values. Some tweets only contain the virtue or the vice of a
foundation, and some tweets contain both. . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
x

6.2 For every user group, we show the top five moral combinations used in messages that are
retweeted more often by out-group members than in-group members. The list is sorted
by the ratio C(X, X′
, m)/C(X, X, m). Highlighted moral foundations are the ones that
the user group does not favor (cf. Figure 6.2). . . . . . . . . . . . . . . . . . . . . . . . . . 109
A.1 Hashtags that are categorized as either left-leaning or right-leaning from the top 50 most
popular hashtags used in user profile descriptions in the COVID-19 dataset. . . . . . . . . 137
A.2 The Twitter handles, media URLs, and bias ratings from AllSides.com for the popular
media on Twitter. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138
B.1 The statistics of the number of followers and average numbers of likes, retweets, quotes,
and replies per user in the hate speech dataset. . . . . . . . . . . . . . . . . . . . . . . . . . 140
B.2 The change in hate score when a user received lower vs. higher than expected amount
of likes, retweets, replies, or quotes over a window of k = 50 tweets in the hate speech
dataset. We test the significance of the difference between the distributions using a
Mann-Whitney U test (** p < 0.01). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141
xi

List of Figures
2.1 The two key motivating components of Retweet-BERT. . . . . . . . . . . . . . . . . . . . . 17
2.2 Illustration of the proposed Retweet-BERT. We first train it in an unsupervised manner
on the retweet network (left) using a Siamese network structure, where the two BERT
networks share weights. We then train a new dense layer on top to predict polarity on a
labeled dataset (right). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
3.1 Overview of the Social-LLM method. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
3.2 Distribution of users’ bot scores prior to user preprocessing for the Immigration-Hate
datasets. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
3.3 Social-LLM Experient 5: Ablation study of user tweet embeddings on the Ukr-RusSuspended dataset. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
3.4 Social-LLM Experiment 6: Sensitivity to embedding dimension d. . . . . . . . . . . . . . . 52
3.5 Social-LLM Experiment 7: Sensitivity to training size. . . . . . . . . . . . . . . . . . . . . . 53
3.6 Visualization of Social-LLM embeddings. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
4.1 COVID-19 dataset statistics of left-leaning (bottom 20%), neutral (middle 20%), and rightleaning (top 20%) users partitioned by their verification status. The degree distributions
are taken from the retweet network. All triplets of distributions (left-leaning, neutral, and
right-leaning) are significant using a one-way ANOVA test (P < 0.001). . . . . . . . . . . 62
4.2 The proportion of users in the COVID-19 dataset in each decile of predicted political bias
scores that are (a) verified, (b) top 5% in the number of followers, (c) top 5% of in-degrees
in the retweet network (most retweeted by others), (c) top 5% of in-degrees in the mention
network (most mentioned by others), and (e) top 5% in PageRank in the retweet network. . 63
4.3 The distribution of left-leaning (bottom 20% of the polarity scores), center (middle 20%),
and right-leaning (top 20%) retweeters (y-axis) for users in the COVID-19 dataset across
the polarity score deciles (x-axis). The retweeted users are either verified or not verified. . 65
xii

4.4 The RWC(X, Y ) for every pair of polarity deciles X and Y on the retweet (left) and
mention (right) networks using Eq. 4.1. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
4.5 Users in the COVID-19 dataset with the highest number of retweeters from left- and
right-leaning users. The bar plots show the distribution of their unique retweeters by
political leaning. Users are also ranked by their total number of retweeters (i.e., #1
@realDonaldTrump means that @realDonaldTrump has the most retweeters). Numbers
appended to the end of the bars show their total number of retweeters. . . . . . . . . . . . 68
5.1 Distribution of the hate scores per user (average hate score of their original tweets) and
per original tweet in the hate speech dataset. . . . . . . . . . . . . . . . . . . . . . . . . . . 78
5.2 The four types of social engagement on dimensions of rebroadcast and endorsement.
Retweets represent rebroadcast and endorsement, likes represent endorsements, quotes are
rebroadcasts that can be either positive or negative, and replies do not rebroadcast and can
be either positive or negative. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
5.3 Changes in toxicity (y-axis) when an anchor tweet received lower (red bars) or higher
(blue bars) than the predicted amount of social engagement at different windows k (x-axis)
in the hate speech dataset. Changes that are significantly different between the lowerand the higher-than-predicted groups are indicated (Mann-Whitney U test, ** p < 0.01,
*** p < 0.001). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
5.4 Having higher amounts of likes and retweets than predicted would result in the biggest
increase in future toxicity in the hate speech dataset, and vice versa (k = 50). . . . . . . . 87
5.5 When an anchor tweet in the hate speech dataset receives substantially lower (red) or
higher (blue) amount of retweets than expected, the difference in maximum toxicity
(k = 50) is statistically significant (Mann-Whitney U test, ∗ ∗ p < 0.01). More retweets
lead to an increase in maximum toxicity, and vice versa . . . . . . . . . . . . . . . . . . . . 88
6.1 Distribution of raw user moral scores in the COVID-19 dataset. . . . . . . . . . . . . . . . 96
6.2 The average moral z-scores of each foundation for the four user groups in the COVID-19
dataset. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
6.3 Partisanship breakdown of the four user groups in the COVID-19 dataset. Blue bars
represent left-leaning users, and red bars represent right-leaning users. . . . . . . . . . . . 102
6.4 Distribution of the user metadata features for the four user groups in the COVID-19 dataset. 103
6.5 TSNE visualization of 100,000 sampled Social-LLM user embeddings from the COVID-19
dataset of the four user groups. These Social-LLM embeddings are learned from the users’
network cues, content cues, and moral leanings. . . . . . . . . . . . . . . . . . . . . . . . . 104
6.6 Retweet network assortativity of users’ moral scores over time in the COVID-19 dataset.
High assortativity indicates homophily. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
xiii

6.7 The ratio of how often COVID-19 dataset users in group X retweet users in group Y
divided by the null baseline amount. > 1 (red) cells indicate X is more likely to retweet
from Y than the null baseline, and < 1 (blue) cells indicate the opposite. . . . . . . . . . . 107
6.8 The ratio of how often COVID-19 dataset users in group X communicate with in-group
users (left) or out-group users (right) divided by the null baseline amount. > 1 indicates
X is more likely to retweet from Y than the null baseline. . . . . . . . . . . . . . . . . . . 108
B.1 Hate score network assortativity of users in the same bot score bin in the hate speech
dataset. All assortativity measures are significant (p < 0.001), indicating that users are all
assortative or homophilous with each other in terms of their hate scores. . . . . . . . . . . 141
B.2 Pearson correlation between two Perspective (toxicity) scores of each tweet in the hate
speech dataset. All values are statistically significant (p < 0.05). . . . . . . . . . . . . . . . 142
B.3 We display the effect of social engagement on all five toxicity attributes at k = 50 in the
hate speech dataset, similar to Figure 5.3. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143
xiv

Abstract
The explosive growth of social media has not only revolutionized communication but also brought challenges such as political polarization, misinformation, hate speech, and echo chambers. This dissertation
employs computational social science techniques to investigate these issues, understand the social dynamics driving negative online behaviors, and propose data-driven solutions for healthier digital interactions. I begin by introducing a scalable social network representation learning method that integrates
user-generated content with social connections to create unified user embeddings, enabling accurate prediction and visualization of user attributes, communities, and behavioral propensities. Using this tool, I
explore three interrelated problems: 1) COVID-19 discourse on Twitter, revealing polarization and asymmetric political echo chambers; 2) online hate speech, suggesting the pursuit of social approval motivates
toxic behavior; and 3) moral underpinnings of COVID-19 discussions, uncovering patterns of moral homophily and echo chambers, while also indicating moral diversity and plurality can improve message
reach and acceptance across ideological divides. These findings contribute to the advancement of computational social science and provide a foundation for understanding human behavior through the lens of
social interactions and network homophily.
xv

Chapter 1
Introduction
The explosive growth of social media platforms in recent years has facilitated an unparalleled level of online
communication and interaction between people. Staggering statistics highlight this growth: nearly half the
US adults use Instagram, and even more use Facebook and YouTube (Gottfried, 2024). Globally, as of 2024,
4.8 billion people are using social media, accounting for 60% of the global population and 93% of all internet
users (Nyst, 2023). This growth shows no plateauing, with TikTok being among the fastest-growing social
media apps from 2021 to 2023, accruing a 12% growth in user base (Gottfried, 2024). Investors remain
optimistic about the future of social media tech companies, as evidenced by Reddit’s recent valuation of
$6.4 billion for its initial public offering in March 2024 (Isaac and Hirsch, 2024). Moreover, new social
media apps continue to emerge, catering to ever-evolving consumer demands, such as Bluesky (2021),
BeReal (2020), and Threads (2023).
Nowadays, social media use is ubiquitous and intertwined with many aspects of our lives. It addresses
a fundamental part of human beings, which is the desire for interpersonal communication (Baumeister and
Leary, 1995). With the advances in technology, social media apps offer an easy way to stay connected with
friends and family, as well as to follow the lives of influencers and public figures. It also serves as a source
of entertainment, news, and education. For many individuals, particularly those who grew up in the digital
era, social media has become an indispensable tool for maintaining relationships and engaging with the
1

world around them. Platforms such as Facebook, Instagram, WeChat, Snapchat, TikTok, YouTube, Reddit,
Twitter (now X), and LinkedIn play a crucial role in connecting people with friends and family, providing
access to captivating content, and enabling participation in academic and professional communities. As
social media continues to evolve and adapt to user needs, its importance in our daily lives is likely to grow
further.
However, social media has also faced significant criticism, so much so that many people have voluntarily undertaken “digital detoxes” to resist access to social media apps and focus on their non-digital lives
(Syvertsen and Enli, 2020). Social media can negatively impact users’ mental health. For example, internal
research at Meta revealed Instagram’s negative impact on the mental health of young girls, propagating
disillusioned beliefs about body image (Wells et al., 2021). Equally troubling is the issue of addiction, as
heavy social media use can negatively impact one’s mental health, relationships, and academic or professional achievement (Kuss and Griffiths, 2011).
Additionally, the lack of adequate content moderation to filter harmful, abusive, or illegal material
poses serious risks for users. Online hate speech is estimated to be very pervasiveness (Siegel, 2020); one
cross-national survey estimates that a staggering half of young adults experience some form of hateful
messages (Keipi et al., 2016). Another example is the “Elsagate” controversy on YouTube, where malicious actors exploited the recommendation algorithm to spread inappropriate videos targeting children
(Maheshwari, 2017).
Moreover, social media can serve as a conduit for the spread of misinformation (Del Vicario et al.,
2016)–the unintentional spread of false information–and disinformation (Tucker et al., 2018)–the intentional spread of false information. This has severe consequences for matters of national security and
election integrity (Marwick and Lewis, 2017). For instance, the high-profile Cambridge Analytica and
Facebook scandal that broke the news in 2018 in which millions of user profiles were leaked for target
political advertising (Rosenberg and Frenkel, 2018), or the state-backed propaganda by Russia’s Internet
2

Research Agency (IRA) that used internet “trolls”, or fake social media accounts, to interfere in the 2016
US elections. (Bastos and Farkas, 2019). Social media can also act as a filter bubble and echo chamber,
supplying users with the type of information they already agree with, worsening ideological divides, and
amplifying beliefs (Barberá, 2020). The greater consequences of this have already been seen in matters
of political polarization (Kubin and Von Sikorski, 2021) and extremism radicalization (Thompson, 2011).
Relatedly, user privacy is also a concern as social media companies accumulate extensive user data (Zhang
et al., 2014), as was most recently seen in the recent 2024 US House initiative to ban TikTok due to privacy
concerns (Maheshwari et al., 2024).
Despite these significant drawbacks, we are unable to truly part ways with social media. It has evolved
into a constant source of connection—an arguably necessary component interwoven into both our personal
and professional lives. Even if an individual were to abstain from social media, the rest of the world remains
deeply connected and engaged with these platforms. This inescapable reality is not without its merits,
however. Social media can be an invaluable platform for fast, convenient communication. It can serve as
a useful tool for learning (Duffy, 2008) and education (Greenhow and Lewin, 2019). And anecdotally, the
entertaining content shared on these platforms, such as amusing dog videos, may even provide a mood
boost and improve mental well-being.
If we cannot escape the digital world, can we make it better? This question drives my research, aiming
to identify problems on social media and offer data-driven suggestions for improvement. Throughout this
dissertation, I hope to use my knowledge to explore specific issues and potential solutions related to social
media usage and its impacts. While a comprehensive solution is beyond the scope of a single work, this
research aims to contribute one piece towards the broader goal of enhancing the social media experience
and mitigating its negative consequences.
This dissertation is motivated by the desire to make online social network platforms better by mitigating
the undesirable, toxic, and ill-intentioned consequences that often arise. To this end, I consider a few cases of
3

negative online human behavior to examine the severity of the problem, provide potential explanations for
their occurrence, and predict human behavior. By leveraging the large-scale nature of social media data
and employing empirical, quantitative analyses, I aim to understand user motivations, social dynamics,
community effects, and influence pathways.
1.1 Computational Social Science
The availability of large-scale online behavioral traces has fueled research in computational social science
(CSS), a fusion of computer science and social science (Lazer et al., 2009; Cioffi-Revilla, 2014; Wallach,
2018; Zhang et al., 2020). Operating at the intersection of empirical, social, and computational sciences,
CSS provides both a theoretical framework for understanding social phenomena at a macro level and a
methodological tool for computationally validating and testing new theories (Cioffi-Revilla, 2021). Its scope
is broad, encompassing computational foundations on humans and social systems, information extraction,
social network analysis, social complexity, and social simulations, as outlined by Cioffi-Revilla (2021). Due
to its highly interdisciplinary nature, CSS has applications in various domains such as psychology, political
science, communications, economics, and public policy (Edelmann et al., 2020; Zhang et al., 2020).
In this work, my focus lies on the empirical, data-driven aspect of CSS, leveraging the novelty of
“big data” computing and advancements in computer science (Lazer et al., 2009, 2020). The methodology
employed here varies widely, including data analysis techniques, statistical methods, network science,
traditional machine learning, and deep learning (Zhang et al., 2020). As such, rapid technological advances
in deep learning have also been instrumental in this emerging field’s growth. CSS researchers are able to
use data-driven methods to analyze individual and collective human dynamics.
It is not lost on me the irony of utilizing CSS approaches, a field greatly bolstered by large-scale social
media data, to address issues that have arisen because of social media. Nonetheless, CSS methods provide a
valuable opportunity to glean empirical, data-driven insights that can inform strategies for mitigating the
4

negative consequences of modern online communication. The vast troves of social media data harbor the
potential to illuminate crucial mechanisms underlying human behavior on these platforms. By harnessing
computational methods to analyze this rich data, I aim to uncover insights that can guide solutions to
tackle the problems emerging in online systems.
1.2 Social Network Homophily
A core theme underlying my research is the concept of network homophily–the tendency of individuals
to associate and bond with others who are similar to themselves, resulting in social network connections
between users who share comparable characteristics, opinions, emotions, or other attributes (McPherson
et al., 2001). In network theory, this is also called assortative mixing, where nodes preferentially attach
themselves to other nodes with similar characteristics (Newman, 2003).
While homophily implies that “similarity breeds connection” (McPherson et al., 2001), a closely related
theory is social contagion, which posits the opposite: one user’s behavior influences their network connections to do the same (Christakis and Fowler, 2013), similar to how one might contract COVID from a close
contact. Social contagion can occur as a simple contagion, with adoption probability increasing monotonically with the number of exposures, or as a complex contagion involving more intricate dynamics (Hodas
and Lerman, 2014). Homophily and social contagion represent contrasting causal relationships. However,
the purpose of this research is not to argue for the presence of one mechanism over the other. As Shalizi
and Thomas (2011) aptly stated in their paper title, “homophily and contagion are generically confounded
in observational social network studies.” Regardless of whether network connections are formed through
homophily or contagion, the observable outcome remains the same: individuals with similar characteristics tend to be situated closer to each other in the network. Therefore, the primary objective of this work
is to leverage the theory of homophily to model and interpret the data at hand.
5

Throughout my dissertation, I will exercise caution to avoid making unsubstantiated causal claims.
The data utilized in this research is observational in nature, and drawing causal inferences from such data
is, at best, pseudo-causal and, at worst, entirely unjustified. Establishing robust causal relationships would
necessitate interventional studies, which are beyond the scope and capabilities of an independent academic
researcher. Even barring practical limitations, conducting such experiments is ethically questionable, as it
would involve subjecting users to potentially harmful content such as radicalism or hate speech.
Despite the inability to conclusively determine the underlying mechanism of homophily, the theory
itself can still be effectively utilized. Specifically, I leverage the concept of homophily to formulate research
hypotheses and interpret the findings of this study. The literature on user homophily in social networks
provides numerous examples of its application. For instance, on Twitter, users tend to gravitate towards
others who share similar political ideologies (Conover et al., 2011b; Barberá et al., 2015). Similarly, on
Facebook, users who spread conspiracy theories and those who disseminate scientific news are situated
in polarized and homogenous communities (Del Vicario et al., 2016). Additionally, one study suggests that
hateful messaging is also a homophilous trait (Nagar et al., 2022). Moreover, homophily manifests through
the lens of moral psychology, as demonstrated by Dehghani et al. (2016), who found that users sharing
certain moral foundations are more likely to be closely connected in their social network. These examples of homophily in social networks inspire thought-provoking interpretations of human behaviors and
opinions within the context of social network structures and can thus illuminate online social dynamics.
In addition to its utility in interpreting social phenomena, I will also draw from the concept of homophily in developing a novel graph representation learning method for social networks. Graph representation learning, also known as network representation learning, has emerged as a promising machine
learning approach for extracting insights from structured graph data. This technique aims to learn vector
representations, or embeddings, for each node in a graph, capturing relevant information for downstream
tasks like node classification or link prediction (Hamilton, 2020). Here, I treat users as nodes and social
6

network interactions as edges, applying a network representation learning algorithm to derive latent user
embeddings.
Many graph representation learning methods rely on and leverage the homophily property (Grover
and Leskovec, 2016; Hamilton et al., 2017; Kipf and Welling, 2017; Veličković et al., 2018). These methods
work to embed connected users, assuming that they are more similar, according to the homophily theory,
closer together in the latent space. Consequently, the resulting embedding vectors will exhibit greater
similarity for connected users. It is worth noting that while homophily is not strictly necessary for some
graph representation learning approaches like graph convolutional networks, which can derive meaningful
representations from heterophilous connections, graphs lacking homophily still present challenges for
graph representation learning (Ma et al., 2022a). However, in the context of social networks, homophily
predominates (McPherson et al., 2001).
1.3 Technical Challenges
A key aspect of this work is utilizing the social network to capture the rich relational signals from user-user
interaction data. I aim to go beyond simply considering the number of people a user interacts with and tap
into the identity of connected users to better understand individual users. This motivates using graph representation learning techniques such as graph neural networks (Goyal and Ferrara, 2018; Hamilton, 2020).
By representing users as nodes and user interactions or connections as edges, network representation
learning methods encode community properties, structural roles, and other properties in user embeddings
that are easily accessible for machine learning models.
Regrettably, these techniques often encounter difficulties in scaling computationally to handle large
datasets (Zhang et al., 2018), a challenge particularly pronounced by the scale of empirical social network
data (Wu et al., 2020; Ma et al., 2022b). The sheer volume of big data frequently surpasses the capabilities
of contemporary technology in terms of both hardware resources and processing power (Fan et al., 2014).
7

Additionally, their methodological constraints limit their compatibility with popular distributed training
methods such as parallelization (Serafini and Guan, 2021). Furthermore, many of these methods struggle
with inductive inferencing on new users without necessitating retraining (Goyal and Ferrara, 2018).
To address this challenge, I propose a novel social network representation learning method, SocialLLM, which integrates user content with social connections to create unified user representations. This
approach enables accurate prediction and insightful visualizations of user attributes, communities, and
behavioral propensities. Social-LLM leverages the simplest form of connections, where one edge connects
two users, to semi-supervise user embedding training via a Siamese network structure. The model aims
to embed two connected users closer in the latent dimension space. To facilitate the training process, I
also utilize large language models (Zhao et al., 2023). As social network data is mostly textual, the latest
language model technology can be used to obtain initial user embeddings based on texts, providing the
model with a substantial head-start in performance without relying on network information.
Social-LLM offers several practical advantages. First, by using pairs of nodes connected by an edge
as inputs, the data can be easily processed with stochastic gradient descent via batching, resulting in a
time complexity that is linear with respect to the number of edges. Second, since Social-LLM is semisupervised using edges, node labels are not required. This is particularly beneficial as node labels can
be costly to annotate for large datasets and may be inherently noisy and ambiguous due to the nature
of many social science problems. Moreover, annotations require the problem to be carefully thought out
ahead of time, tailoring to a single problem and limiting the generalizability of a dataset. Additionally, the
learned Social-LLM user embeddings can be highly flexible and used as features in downstream prediction
tasks, such as predicting user partisanship or clustering users for community analysis. The versatility and
reusability of these embeddings make them extremely useful. Finally, Social-LLM does not require any
social network data at the inference stage, substantially expanding its usefulness in producing inductive
user representations when applying the model to out-of-sample users.
8

1.4 Research Overview
In this dissertation, I explore the dynamics of online communication within the realm of pressing social
and political matters by focusing on three research topics. A central theme that binds them together is
the investigation into social dynamics and polarization across digital platforms, with a special emphasis
on the capacity of social media to enhance or mitigate specific behaviors. Although each study focuses
on a distinct facet of these dynamics, together, they illuminate the complex manner in which online social
networks shape user behavior, the spread of information, and ideological division.
Polarization in COVID-19 Discourse. Focusing on the politicization of COVID-19 discourse on Twitter, this research explores the dynamics of political polarization. I find significant polarization and the existence of two distinct and asymmetric echo chambers. The left-leaning users are in a larger echo chamber,
while the right-leaning users form more densely connected and isolated echo chambers. This emphasizes
the role of social media in reinforcing pre-existing beliefs and alludes to challenges in penetrating these
echo chambers with alternative information regarding a public health crisis.
Online Hate Speech. This research focuses on the role of social approval in driving online toxicity,
suggesting that the pursuit of social approval, rather than a direct intent to harm, motivates users to engage
in online hate speech. It emphasizes the impact of social networks and engagements (likes, retweets, etc.)
on escalating or reducing toxicity, providing insights into how social dynamics on platforms contribute
to the spread of hate speech. Coupled with the observation that hateful behavior is homophilous, this
research has important implications for understanding how hateful behavior is “networked” and socially
motivated.
9

Moralization and COVID-19. This research delves into the moral underpinnings of online discussions
about COVID-19, showing how moral psychology and political ideologies together shape users’ communication. It reveals patterns of moral homophily and the existence of a moral echo chamber. Promisingly, it
also suggests that moral diversity and plurality can improve the reach and acceptance of messages across
ideological divides.
Collectively, these studies shed light on how social media platforms serve as fertile grounds for the
reinforcement and amplification of existing beliefs, the formation of homophilous networks, and the escalation of behaviors like toxicity and polarization. They underscore the complex interplay between individual psychology, social approval mechanisms, political ideologies, and network structures in shaping
online discourse. These insights are crucial for developing strategies to mitigate polarization, combat online hate, and promote the dissemination of information. By providing a comprehensive understanding of
the social dynamics at play within online environments, this dissertation contributes to the growing body
of research aimed at fostering healthier online interactions and more informed public discourse.
1.5 Contributions
This dissertation analyzes the dynamics of online communication against the backdrop of significant social
and political issues, emphasizing the role of social media in magnifying behaviors and ideologies. Its contributions encompass a technical innovation as well as three explorations of computational social science
problems that shed light on harmful and fruitful communication practices online.
The technical cornerstone of this work, presented in Part I, is a scalable social network representation
learning algorithm. This algorithm excels at generating insightful user representations, thereby facilitating subsequent analyses such as user classification and community detection. I first introduce the preliminary model, Retweet-BERT, in Chapter 2 as a proof-of-concept. It leverages user profile descriptions
10

and retweet interactions for modeling political partisanship. I then implement enhancements in a SocialLLM, discussed in Chapter 3, to incorporate multimodal user content and a broader spectrum of network
interactions. I also comprehensively evaluate Social-LLM in user detection to showcase its utility in user
modeling tasks.
Part II conducts online behavior analysis to tackle distinct yet interrelated problems within online
communication. In Chapter 4, I investigate the political echo chambers that arise from COVID-19 pandemic
discussions, revealing how digital spaces can become echo chambers that could reinforce pre-existing
beliefs. In Chapter 5, I examine networked hate speech behavior, uncovering the driving force of social
approval behind online toxicity. In Chapter 6, I analyze COVID-19 discussions through the lens of moral
psychology to offer insights into communication dynamics based on moral inclinations, painting a nuanced
picture of online interaction extending beyond political ideology.
This dissertation underscores the need to dissect online dynamics to devise effective strategies for
accurate information dissemination and public health messaging, especially within polarized and echo
chamber-prone environments. It highlights the significance of leveraging social network data in computational social science, demonstrating how users’ social circles can reveal communication cues critical for
understanding online behavior.
These research studies primarily rely on social networks built on interactions, such as retweets. This
choice is driven by my focus on interactive and highly contextualized networks surrounding specific events
and topics. While my work may not delve into more stable networks, such as friendship or following
networks, it does yield clear behavioral insights into how people engage and allocate their attention within
social media.
My findings offer actionable insights for various stakeholders. Researchers can develop targeted interventions to help mitigate the negative consequences of polarization, hate speech, and moral misalignment,
ultimately promoting more constructive and inclusive online interactions. Social media platforms can
11

leverage these findings to update their policies, intervention strategies, and platform architectures in their
efforts to foster healthier online environments. Lawmakers can use this research to inform regulations
that address the challenges posed by the modern digital landscape. Educators can incorporate the insights
into digital literacy curricula, empowering individuals to navigate online spaces safely and effectively.
More broadly, the significance of this work extends beyond the realm of social media alone. It strives
to harness the potential of computational social science to address the multifaceted challenges presented
by the digital age. While social media serves as a primary focus, the methodologies and insights developed
here have broader applicability across various domains where human interaction intersects with digital
platforms. By shedding light on the intricate dynamics of online behavior, this research contributes to the
development of strategies aimed at promoting healthier and more productive online communities through
computational social science.
Some chapters in this dissertation are adapted from papers that have been published (Jiang et al., 2021,
2023c) or are available online as preprints (Jiang and Ferrara, 2023; Jiang et al., 2023b, 2024).
12

Part I
Social Network Representation Learning
13

Social media data has provided researchers with an exciting avenue for empirical data-driven mining
of human behavior (Freeman, 2004; Lazer et al., 2009), enabling the tracking of mass sentiment, health
trends, political polarization, mis/disinformation spread, and information diffusion. Social network data
comprises two crucial elements: content—what people share —and network–who, when, and how frequently users interact with each other. The text-based content aspect of social network data is easier to
process computationally due to recent advancements in large language models (LLMs).
Graph representation learning has emerged as a tool to learn information from networks effectively
(Hamilton, 2020). Graph representation learning would convert each node in the network to a vector
representation that could capture useful latent properties, such as global community properties and local
node structural properties. Common approaches to accomplish this include random walk and graph neural
network algorithms (Hamilton et al., 2017; Goyal and Ferrara, 2018). Random walk methods preserve the
probability of one node visiting another on a random walk over the graph (Perozzi et al., 2014; Grover
and Leskovec, 2016). Graph neural networks use a message-passing framework to exchange information
between connected nodes, aggregating k-hop neighborhood embeddings around a node during embedding
optimization (Hamilton et al., 2017; Kipf and Welling, 2017).
However, the accessibility of big data often surpasses the capabilities of modern technology in terms
of hardware resources and processing capabilities (Fan et al., 2014). While advancements in graph neural
networks have yielded significant progress in network representation, these methods often struggle to
mitigate training overhead. They could suffer from the “neighbor explosion” problem as the time and
memory required to model neighborhoods grows exponentially with the depth (Hamilton et al., 2017;
Duan et al., 2022). They also cannot use conventional approaches to speed up training, such as distributed
training, due to the cost of coordination required to update node embeddings across devices (Serafini and
Guan, 2021). To circumvent this practical constraint, many algorithms rely on sampling or decouplingbased approaches (Serafini and Guan, 2021; Duan et al., 2022). Additionally, most graph learning algorithms
14

are, inherently by design, unable to produce embeddings for unseen and unconnected nodes, limiting
their capacity to generalize to out-of-sample users without retraining (Grover and Leskovec, 2016; Kipf
and Welling, 2017). This challenge is particularly pronounced when modeling the network ties of largescale social networks (Ma et al., 2022b), which are much larger and sparser compared to other commonly
modeled networks (such as protein-protein interaction or citation networks).
In this part, we will introduce a pragmatic social network representation learning method designed
specifically to address these problems using social network connections and user content features. In
our social network setting, we treat users as nodes and social network connections between two users
as edges. Under the assumption of social network homophily (McPherson et al., 2001), we optimize user
representations by making users who are connected by edge more similar to one another. We also leverage
the recent advancements in large language models (Zhao et al., 2023) by incorporating the user’s textual
content.
This model contains several advantages. First, it is scalable to train since the training time complexity
is linearly proportional to the number of edges, and it also enables us to use stochastic gradient descent
for batch-wise processing. Second, the model is self-supervised and requires no labels, which also, in turn,
makes the learned representations reusable for any task. This is particularly advantageous as annotating human social network data can be costly, noisy, and inherently ambiguous. Finally, the method is
inductive since the social network is no longer needed during the inference stage. We are able to produce
representations for any user, provided the suitable user content features, during inference.
We begin with a preliminary version of the model called Retweet-BERT. This model serves as proof
of concept, using only user profile description features and retweet connections. We evaluate this model
on the task of user partisanship prediction. The success of this model motivates the Social-LLM, a more
general-purpose solution that can accommodate multimodel user content features and heterogeneous edge
types. We conduct a full-scale evaluation of Social-LLM on seven different real-world datasets.
15

Chapter 2
Retweet-BERT: Learning from Language Features and Retweet
Diffusions
2.1 Introduction
The rise of social media has brought forth an era of unprecedented connectivity, enabling individuals to
engage in online discussions on a wide range of topics. However, this increased connectivity has also amplified the issue of political polarization, which has become a major driver of many online conversations.
This chapter introduces a preliminary model for social network representation learning, focusing specifically on the problem of user partisanship prediction. By leveraging the vast amounts of data generated
through online interactions, we aim to develop a tool that can accurately predict users’ political affiliations
based on their social network structure and behavior.
Political polarization has been shown to play a significant role in various online discussions, ranging
from election-related topics, such as the 2010 US congressional midterm elections (Conover et al., 2011b)
or the 2012 US presidential election (Barberá et al., 2015), to seemingly unrelated issues like the 2020 global
health crisis. (Calvillo et al., 2020; Jiang et al., 2020). Literature suggests that political affiliations may have
an impact on people’s favorability of public health preventive measures (e.g., social distancing, wearing
16

Figure 2.1: The two key motivating components of Retweet-BERT.
masks) (Jiang et al., 2020), vaccine hesitancy (Hornsey et al., 2020; Peretti-Watel et al., 2020), and conspiracy
theories (Uscinski et al., 2020).
Divisive politicized discourse can be fueled by the presence of echo chambers, where users are mostly
exposed to information that aligns with ideas they already agree with, further reinforcing one’s positions
due to confirmation bias (Garrett, 2009; Barberá et al., 2015). Political polarization can contribute to the
emergence of echo chambers (Conover et al., 2011b; Cinelli et al., 2021), which may accelerate the spread
of misinformation and conspiracies (Del Vicario et al., 2016; Shu et al., 2017; Motta et al., 2020; Muric
et al., 2021; Rao et al., 2021). To facilitate research in online polarization, such as the COVID-19 infodemic,
we present Retweet-BERT, a lightweight tool to accurately detect user ideology in large Twitter datasets
(illustrated in Figure 2.1). Our method simultaneously captures (i) semantic features about the user’s
textual content in their profile descriptions (e.g., affiliations, ideologies, sentiment, and linguistics) and
(ii) the patterns of diffusion of information – i.e., the spread of a given message on the social network –
and how they can contribute to the formation of particular network structures (e.g., echo chambers). Prior
works on polarization primarily focus on only one of these aspects (Conover et al., 2011b,a; Barberá et al.,
2015; Wong et al., 2016; Preoţiuc-Pietro et al., 2017).
17

Figure 2.2: Illustration of the proposed Retweet-BERT. We first train it in an unsupervised manner on the
retweet network (left) using a Siamese network structure, where the two BERT networks share weights.
We then train a new dense layer on top to predict polarity on a labeled dataset (right).
There are two important assumptions behind Retweet-BERT. One is that the act of retweets implies
endorsement (Boyd et al., 2010), which further implies support for another’s ideology (Wong et al., 2016).
The other is that people who share similar ideologies also share similar textual content in their profile
descriptions, including not only similar keywords (e.g., “Vote Blue!”) and sentiment but also linguistics.
The idea of linguistic homophily among similar groups of people has been documented and explored in
the past (Yang and Eisenstein, 2017). People who adopt similar language styles have a higher likelihood of
friendship formation (Kovacs and Kleinbaum, 2020).
Retweet-BERT leverages both network structure and language cues to predict user ideology. Our
method is simple, intuitive, and scalable. The two steps to Retweet-BERT are:
1. Training in an unsupervised manner on the full dataset by learning representations based on users’
profile descriptions and retweet interactions,
2. Fine-tuning the model for polarity estimation on a smaller labeled subset.
18

An illustration of Retweet-BERT is shown in Figure 2.2. Crucially, our method does not require human
annotations. Instead, we label a small set of users heuristically based on hashtags and mentions of biased
new media outlets, as was done in prior works (Conover et al., 2011a; Badawy et al., 2018; Addawood et al.,
2019). In addition, since we only use profile descriptions instead of all of the users’ tweets, Retweet-BERT
can be easily deployed.
The datasets we use are two large-scale Twitter datasets collected in recent years. The COVID-19
Twitter dataset was collected from January to July 2020 for 232,000 active users. We demonstrate that
Retweet-BERT attains 96% cross-validated macro-F1 on this dataset and outperforms other state-of-theart methods based on large language models, graph embedding, etc. We also perform extensive evaluations
of our model on a second Twitter dataset on the 2020 presidential elections to showcase the reliability of
Retweet-BERT (97% macro-F1).
In sum, the contributions of this chapter are:
• We present Retweet-BERT, a simple and elegant approach for estimating user ideology based on
linguistic homophily and social network interactions.
• We conduct experiments and manual validations to highlight the effectiveness of Retweet-BERT on
two recent public Twitter datasets compared to baselines: COVID-19 and the 2020 US presidential
elections.
2.2 Related Work
2.2.1 Ideology Detection
There is growing interest in estimating expressed ideologies. Many works focused on opinion mining and
stance detection (Somasundaran and Wiebe, 2009; Walker et al., 2012; Abu-Jbara et al., 2013; Hasan and
Ng, 2014; Sridhar et al., 2015; Darwish et al., 2020). Of particular interest are political ideology detection
19

of textual data (Sim et al., 2013; Iyyer et al., 2014; Bamman and Smith, 2015) as well as of Twitter users
(Conover et al., 2011a,b; Barberá et al., 2015; Wong et al., 2016; Yang et al., 2016; Preoţiuc-Pietro et al.,
2017; Badawy et al., 2018, 2019; Xiao et al., 2020). There are two general strategies for identifying Twitter
user ideologies: content-based and network-based. Content-based strategies are concerned with the user’s
tweets and other textual data. An earlier study used hashtags in tweets to classify users’ political ideologies
(Conover et al., 2011a). Preoţiuc-Pietro et al. (2017) applied word embedding on tweets to detect tweets
of similar topics. Network-based strategies leverage cues from information diffusion to inform ideological
differences. These models observe that users interact more with people they share similar ideologies with
(Yang et al., 2016). Interactions can be retweets (Wong et al., 2016) or followings (Barberá, 2015). Xiao et al.
(2020) formulated a multi-relational network using retweets, mentions, likes, and follows to detect binary
ideological labels. Other works used a blend of both content- and network-based approaches (Badawy
et al., 2019). Hashtag-based methods were combined with label propagation to infer the leanings of users
from the retweet network (Conover et al., 2011a,b; Badawy et al., 2018). Closely related to our methodology,
Darwish et al. (2020) clustered users by projecting them on a space jointly characterized by their tweets,
hashtags, and retweeted accounts; however, this algorithm comes at a high computational cost.
2.2.2 Socially-Infused Text Mining
More related to our method is a recent line of work that learns from socially infused text data. Li and
Goldwasser (2019) combined user interactions and user sharing of news media to predict the bias of new
articles. Pan et al. (2016) used node structure, node content, and node labels to learn node representations to
classify categories of scientific publications. Yang and Eisenstein (2017) used social interactions to improve
sentiment detection by leveraging the idea of linguistics homophily. Johnson et al. (2017) used lexical,
behavioral, and social information to categorize tweets from politicians into various topics of political
issues. These works provide promising results for combining social network data and textual data.
20

This model: Retweet-BERT is unique from the approaches described above in two substantial ways: (i)
it combines both language features, particularly pretrained large language models for natural language
processing, and social network features for a more comprehensive estimation of user ideology, and (ii) it
is scalable to large datasets without supervision.
2.3 Data
We use two recent large-scale Twitter datasets. The primary dataset is on COVID-19 (Covid-Political)
from January 21 to July 31, 2020 (v2.7) (Chen et al., 2020). All tweets collected contain COVID-related
keywords. We also use a secondary dataset on the 2020 presidential election (Election-2020) collected
from March 1 to May 31, 2020 (Chen et al., 2021b). Both datasets are publicly available. Each tweet contains
user metadata, including their profile description, the number of followers, the user-provided location, etc.
Users can be verified, which means they are authenticated by Twitter in the interest of the public.
Although a number of Twitter accounts have since been banned by Twitter—notably, @realDonaldTrump was suspended in January 2021 (Twitter Inc., 2021)—our data collection was done in real-time,
and therefore all tweets by banned accounts are still in our dataset.
2.3.1 Content Cues: Profiles
For the purposes of this chapter, we do not use tweet contents but rather user profile descriptions. In addition to different users posting various numbers of tweets, our main assumption is that profile descriptions
are more descriptive of a user’s ideology than tweets. The profile description is a short biography that
is displayed prominently when clicking on a user. It usually includes personal descriptors (e.g., “Father”,
“Governor”, “Best-selling author”) and, when appropriate, the political ideology or activism they support
(e.g., “Democratic”, “#BLM”). Capped at 160 characters, these descriptions have to be short, which motivates users to convey essential information about themselves clearly, succinctly, and attractively. Previous
21

work established a positive link between the number of followers and the character length of the user
(Mention.com, 2018), which would suggest that more influential users will have a more meaningful profile.
2.3.2 Interaction Cues: Retweet Network
In this chapter, we use retweets to build the interaction network. Retweets refer only to tweets that were
shared verbatim. Retweets are distinct from quoted tweets, which are essentially retweets with additional
comments. We do not use the following network as it is rarely used due to the time-consuming nature of its
data collection (Martha et al., 2013). The retweet network GR is a weighted, directed graph where vertices
V are users and edges E are retweet connections. An edge (u, v) ∈ E indicates that user u retweeted
from user v, and the weight w(u, v) represents the number of retweets.
2.3.3 Data Preprocessing
We removed inactive users and users who are likely not in the U.S. (see Appendix A for details). Users in
our dataset must have posted more than one tweet. To remove biases from potential bots infiltrating the
dataset (Ferrara, 2020), we calculate bot scores using Davis et al. (2016), which assigns a score from 0 (likely
human) to 1 (likely bots), and remove the top 10% of users by bot scores as suggested by Ferrara (2020).
The Covid-Political dataset contains 232,000 users with 1.4 million retweet interactions. The average
degree of the retweet network is 6.15. Around 18k users (≈ 8%) are verified. The Election-2020 dataset
contains 115,000 users and 3.6 million retweet interactions.
2.4 Method
This section describes our proposed method to estimate the polarity of users as a binary classification problem. We first use heuristics-based methods to generate “pseudo"-labels for two polarized groups of users,
22

which are used as seed users for training and evaluating polarity estimation models. We then introduce
several baseline models followed by Retweet-BERT.
2.4.1 Pseudo-Label Generation
We consider two reliable measures to estimate political leanings for some users, which can be used for
model training and automatic, large-scale evaluation. These measures will be used to generate “pseudo"
political leaning labels for a subset of users (i.e., seed users). These seed users will be used as the training
set of users.
2.4.1.1 Hashtag-Based Method
The first method involves annotating the 50 most popular hashtags used in user profiles as left- or rightleaning, depending on what political party or candidate they support (or oppose). 17 of these hashtags
are classified as left-leaning (e.g., #Resist) and 12 as right-leaning (e.g., #MAGA). The list of hashtags can
be found in Appendix A. Users are labeled left-leaning if their profiles contain more left-leaning than
right-leaning hashtags and vice versa. We do not consider hashtags appearing in tweets because hashtags
in tweets can be used to reply to opposing ideology content (Conover et al., 2011b). Instead, following
prior work (Badawy et al., 2018; Addawood et al., 2019), we assume that hashtags appearing in users’
self-reported profile descriptions are better indicators of their true ideological affiliations.
2.4.1.2 News Media-Based Method
The second method utilizes media outlets mentioned in users’ tweets through mentions or retweets
(Badawy et al., 2019; Bovet and Makse, 2019; Ferrara et al., 2020). Following Ferrara et al. (2020), we
determined 29 prominent media outlets on Twitter. Each media outlet’s political bias is evaluated by the
non-partisan media watchdog AllSides.com on a scale of 1 to 5 (left, center-left, neutral, center-right, right).
If a user mentions any of these media outlets, either by retweeting the media outlet’s Twitter account or
23

by link sharing, the user is considered to have endorsed that media outlet. Given a user who has given
at least two endorsements to any of these media (to avoid those who are not extremely active in news
sharing), we calculate their media bias score from the average of the scores of their media outlets. A user
is considered left-leaning if their media bias score is equal to or below 2 and right-leaning if their score is
equal to or above 4.
2.4.1.3 Pseudo-Labeling Seed Users
Using a combination of the profile hashtag method and the media outlet method, we categorized 79,370
(≈ 34% of all) users as either left- or right-leaning. The first, hashtag-based, method alone was only able
to label around 16,000 users, while the second, media-based, method labeled around 49,000 users. The
two methods overlapped in labeling around 10,000 users. In case of any disagreements between the two
methods, which were exceedingly rare at only 200 instances, we defer to the first hashtag-based method.
These users are considered seed users for political leaning estimation. 75% of these seed users are leftleaning, a finding consistent with previous research which revealed that there are more liberal users on
Twitter (Wojcik and Hughes, 2019). In our secondary Election-2020 dataset, we tagged 75,301 seed users.
2.4.1.4 Pseudo-Labeling Validation
This pseudo-labeling method is limited in its capacity for labeling all users (i.e., low coverage ratio, covering only 34% of all users), but it serves as a good starting point for its simplicity. We validated this labeling
strategy by annotating 100 randomly sampled users from the main Covid-Political dataset. Two authors
independently annotated the data by considering both the tweets and the profile descriptions to determine
the users’ political leaning, keeping political neutrality to the extent possible. We then discussed and resolved any annotation differences until reaching a consensus. We attained a substantial inter-annotator
agreement (Cohen’s Kappa) of 0.85. 96 users’ annotated labels agree with the pseudo-labels, and 4 users’
24

labels cannot be conclusively determined manually. The high agreement with the pseudo-labels makes us
highly confident in the precision of our pseudo-label approach.
2.4.2 Methods for Polarity Estimation
While pseudo-labels can assign confident political leaning labels to a third of all users, they cannot determine the political leaning of the rest. To predict political leanings for all users, we explore several
representation learning methods based on users’ profile description and/or their retweet interactions. In
all of our methods in this section and the one that follows (our proposed method), We do not consider
users’ tweets. This is because the datasets contain sampled tweets based on keywords and do not encompass any user’s full tweet histories. Considering tweets in isolation can bias an algorithm for political
leaning detection.
2.4.2.1 Word Embeddings
As baselines, we use pre-trained Word2Vec (Mikolov et al., 2013) and GloVe (Pennington et al., 2014) word
embeddings from Gensim (Řehůřek and Sojka, 2010). The profile embeddings are formed by averaging the
embeddings of the profile tokens.
2.4.2.2 Large Language Models
Large language models (LLMs), particularly the BERT family of models (Devlin et al., 2019; Liu et al., 2019;
Sanh et al., 2019), are pre-trained language models that have led to significant performance gains across
many NLP tasks. We experiment with two different ways to apply LLMs for our task: (1) averaging the output embeddings of all words in the profile to form profile embeddings, and (2) fine-tuning an LLM through
the initial token embedding of the sentence (e.g., [CLS] for BERT, for RoBERTa) with a sequence classification head. We use the sequence classification head by Wolf et al. (2020), which adds a dense layer on
top of the pooled output of the LLM’s initial token embedding.
25

Reimers and Gurevych (2019) proposed Sentence-BERT (SBERT), which is a Siamese network optimized for sentence-level embeddings. Sentence-BERT outperforms naive transformer-based methods for
sentence-based tasks while massively reducing the time complexity. We directly retrieve profile embeddings for each user using Sentence-BERT’s pre-trained model for semantic textual similarity.
2.4.2.3 Network-Based Models
We explore network-based models such as node2vec (Grover and Leskovec, 2016), which learns node embeddings based on structural similarity and homophily, and label propagation, which deterministically
propagates labels using the network. Neither of these models can classify isolated nodes in the network.
We also experiment with GraphSAGE (Hamilton et al., 2017), an inductive graph neural network method
that utilizes node attributes to enable predictions for isolated nodes. We use the aforementioned profile
embeddings as node attributes.
All profile or network embeddings are subsequently fit with a logistic regression model for the classification task. Hyperparameter-tuning details can be found in Appendix A. The profiles are pre-processed
and tokenized according to the instructions for each language model.
With the exception of GraphSAGE, all of these aforementioned methods use either the textual features
of the profile description or the network content, but not both. Purely network-based models will do poorly
for nodes with only a few connections and may only be suitable for non-isolated nodes. Purely text-based
models will do poorly when there are insufficient textual features to inform the models.
2.4.3 Retweet-BERT Framework
2.4.3.1 Combining Textual and Social Content
To overcome the aforementioned issues, we propose Retweet-BERT (Figure 2.2), a sentence embedding
model that incorporates the retweet network. We base our model on the assumption that users who retweet
26

each other are more likely to share similar ideologies. As such, the intuition of our model is to encourage
the profile embeddings to be more similar for users who retweet each other. Retweet-BERT is trained
in two steps. The first step involves training in an unsupervised manner on the retweet network, and
the second step involves supervised fine-tuning on the labeled dataset for classification. Similar to the
training of Sentence-BERT (Reimers and Gurevych, 2019), the unsupervised training step of Retweet-BERT
uses a Siamese network structure. Specifically, using any of the aforementioned models that can produce
sentence-level embeddings, we apply it to a profile description to obtain the profile embedding si for user
i. For every positive retweet interaction from user i to j (i.e., (i, j) ∈ E), we optimize the objective:
X
k∈V,(i,k)̸∈E
max(||si − sj || − ||si − sk|| + ϵ, 0), (2.1)
where || · || is a distance metric and ϵ is a margin hyperparameter. We follow the default configuration as
in Sentence-BERT (Reimers and Gurevych, 2019), which uses the Euclidean distance and ϵ = 1. We then
freeze the learned weights and add a new layer on top to fine-tune on a labeled dataset for classification.
2.4.3.2 Negative Sampling
To optimize the training procedure during the unsupervised training step, we employ negative sampling.
We explore two types of negative sampling strategies. The first is a simple negative sampling (one-neg), in
which we randomly sample one other node k for every anchor node in each iteration (Mikolov et al., 2013).
For simplicity, we assume all nodes are uniformly distributed. The second is multiple negative sampling
(mult-neg), in which the negative examples are drawn from all other examples in the same batch (Henderson et al., 2017). For instance, if the batch of positive examples are [(si1, sj1),(si2, sj2), ...,(sin, sjn)],
then the negative examples for (sik, sjk), the pair at index k, are {sjk′} for k
′ ∈ [1, n] and k
′ ̸= k.
It is worth noting that Retweet-BERT disregards the directionality of the network and only considers
the immediate neighbors of all nodes. In practice, we find that doing so balances the trade-off between
27

training complexity and testing performance. Building on the convenience of Sentence-BERT for sentence
embeddings, we use the aforementioned Sentence-BERT models pre-trained for semantic textual similarity
as the base model for fine-tuning.
2.5 Results
We conduct two sets of evaluations to compare the methods: 1) cross-validation over the pseudo-labeled
seed users as an automatic, large-scale evaluation and 2) in-house human evaluation on a set of held-out
users as a complementary evaluation to the first one. We use the macro-averaged F1 score as the primary
metric due to data imbalance. We note that due to our setup, many of the aforementioned related works are
not directly comparable. We do not use the following network (Barberá, 2015; Xiao et al., 2020). We also
do not use manual labeling (Wong et al., 2016) or additional external sources to determine user ideology
(Wong et al., 2016; Preoţiuc-Pietro et al., 2017). We do include a comparison with the label propagation
method used in Conover et al. (2011a,b); Badawy et al. (2018) on the held-out users.
Finally, the best model (ours) is selected to classify all the remaining users (non-seed users) to obtain
their polarity leaning labels in the Covid-Political dataset. These labels are used to conduct a case study
of polarization COVID-19 on Twitter.
2.5.1 Automatic Evaluation on Seed Users
2.5.1.1 Baselines
We conduct a 5-fold cross-validation on the seed users (i.e., full set of training users) comparing RetweetBERT with baselines. In addition, we also use a random label predictor (based on the distribution of the
labels) and a majority label predictor model as additional baselines. Table 2.1 shows the cross-validated
results for political leaning classification on the seed users, Overall, the models perform comparatively
similarly between the two datasets. Of all models that do not consider the retweet network, fine-tuned
28

Table 2.1: Retweet-BERT results for political leaning classification on seed users on the main CovidPolitical dataset (N = 79, 000) and the secondary Election-2020 dataset (N = 75, 000). The best F1
(macro) scores for each model type are shown in bold, and the best overall scores are underlined. RetweetBERT outperforms all other models on both datasets.
Covid-Political Election-2020
Model Profile Network Acc. AUC F1 Acc. AUC F1
Random and Majority
Random ✗ ✗ 0.585 0.501 0.706 0.499 0.499 0.506
Majority ✗ ✗ 0.706 0.500 0.828 0.508 0.500 0.674
Average word embeddings
Word2Vec-google-news-300 ✓ ✗ 0.852 0.877 0.907 0.831 0.906 0.839
GloVe-wiki-gigaword-300 ✓ ✗ 0.856 0.875 0.909 0.835 0.908 0.844
Average LLM embeddings
BERT-base-uncased ✓ ✗ 0.859 0.882 0.910 0.837 0.912 0.844
BERT-large-uncased ✓ ✗ 0.862 0.887 0.911 0.842 0.913 0.848
DistilBERT-uncased ✓ ✗ 0.863 0.888 0.912 0.845 0.919 0.851
RoBERTa-base ✓ ✗ 0.870 0.898 0.917 0.853 0.925 0.859
RoBERTa-large ✓ ✗ 0.882 0.914 0.924 - - -
Fine-tuned LLMs
BERT-base-uncased ✓ ✗ 0.900 0.932 0.934 0.902 0.963 0.906
DistilBERT-uncased ✓ ✗ 0.899 0.931 0.934 0.899 0.962 0.904
RoBERTa-base ✓ ✗ 0.893 0.916 0.930 0.888 0.953 0.895
Sentence-BERT
SBERT-large-uncased ✓ ✗ 0.869 0.890 0.916 0.849 0.924 0.855
S-DistilBERT-uncased ✓ ✗ 0.864 0.885 0.913 0.843 0.917 0.849
S-RoBERTa-large ✓ ✗ 0.879 0.903 0.922 0.874 0.944 0.878
Graph embedding
Node2vec* ✗ ✓ 0.928 0.955 0.949 0.882 0.944 0.883
GraphSAGE + RoBERTa-base ✓ ✓ 0.789 0.725 0.873 - - -
Retweet-BERT (our model)
Retweet-DistilBERT-one-neg ✓ ✓ 0.900 0.933 0.935 - - -
Retweet-DistilBERT-mult-neg ✓ ✓ 0.935 0.965 0.957 0.973 0.984 0.973
Retweet-BERT-base-mult-neg ✓ ✓ 0.934 0.966 0.957 0.971 0.984 0.971
* Node2vec, a transductive-only model, can only be applied to non-isolated users in the retweet network
29

Table 2.2: Retweet-BERT results on 85 users with human-annotated political-leaning labels from a random
sample of 100 users without seed labels. Retweet-BERT outperforms all models.
Model Profile Network F1
RoBERTa-large (average) ✓ ✗ 0.892
BERT-base-uncased (fine-tuned) ✓ ✗ 0.908
S-RoBERTA-large (SBERT) ✓ ✗ 0.909
Label Propagation* ✗ ✓ 0.910
node2vec* ✗ ✓ 0.922
Retweet-BERT-base-mult-neg ✓ ✓ 0.932
LLMs are demonstrably better. Averaging LLM outputs and fine-tuning Sentence-BERTs lead to similar
results. For LLMs that have a base and large variant, where the large version has roughly twice the number
of tunable parameters as the base, we see very little added improvement with the large version, which may
be attributed to having to vastly reduce the batch size due to memory issues, which could hurt performance.∗ DistilBERT, a smaller and faster version of BERT, produces comparable or even better results than
BERT or RoBERTa. Though the network-based model, node2vec, achieves good performance, it can only
be applied on nodes that are not disconnected in the retweet network. While GraphSAGE can be applied
to all nodes, it vastly underperforms compared to other models due to its training complexity and time
efficiency (Wu et al., 2020).
Our proposed model, Retweet-BERT, delivers the best results using the DistilBERT base model and
the multiple negatives training strategy on both datasets. Other Retweet-BERT variants also achieve good
results, which shows our methodology can work robustly with any base language model.
2.5.2 Human Evaluation on Held-out Users
For further validation, we manually annotated the political leanings of 100 randomly sampled users without
seed labels. We annotated these users as either left- or right-leaning based on their tweets and their profile
descriptions. We were unable to determine the political leanings of 15 people. We take the best model from
∗
https://github.com/google-research/bert#out-of-memory-issues
30

each category in Table 2.1 and evaluate them on this labeled set. In this experiment, we also include labelpropagation, a simple but efficient method to propagate pseudo-labels through the network commonly
used in past work (Conover et al., 2011a,b; Badawy et al., 2018). However, label propagation and also
node2vec only predict labels for nodes connected to the training network (i.e., they are transductive), but
10 nodes were not connected and thus were excluded from their evaluation. The results are reported in
Table 2.2 for the 85 labeled users. With a macro-F1 of 0.932, Retweet-BERT outperforms all baselines,
further strengthening our confidence in our model.
2.6 Conclusion
In this chapter, we proposed Retweet-BERT, a simple and elegant method to estimate user political leanings
based on social network interactions (the social) and linguistic homophily (the textual). We evaluated our
model on two recent Twitter datasets and compared it with other state-of-the-art baselines to show that
Retweet-BERT achieves highly competitive performance (96%-97% macro-F1 scores). Our experiments
demonstrated the importance of including both the textual and the social components. Additionally, we
proposed a modeling pipeline that does not require manual annotation but only a training set of users
labeled heuristically through hashtags and news media mentions. In Chapter 4, we demonstrate how we
applied Retweet-BERT to understand the polarization and the partisan divide of COVID-19 discourse on
Twitter.
The effectiveness of Retweet-BERT is mainly attributed to the use of both social and textual data.
Using both modalities led to significant improvement gains over using only one. This finding has also
been validated in other contexts (Pan et al., 2016; Johnson et al., 2017; Yang and Eisenstein, 2017; Li and
Goldwasser, 2019), but ours is the first to apply this line of thought to detecting user ideology on social
media. Our model can be utilized by researchers to understand the political and ideological landscapes of
social media users.
31

Though we apply Retweet-BERT specifically to the retweet network on Twitter, we note that it can be
extended to any data with a social network structure and textual content, which is essentially any social
media. Though we use hashtags as the method to initiate weak labels in place of manual supervision,
other methods can be used depending on the social network platform, such as user-declared interests in
community groups (e.g., Facebook groups, Reddit Subreddits, YouTube channels).
This method is not without limitations. Since our method relied on mining both user profile descriptions and the retweet network, it was necessary to remove users who did not have profile descriptions
or sufficient retweet interactions (see Appendix A). As such, our dataset only contains some of the most
active and vocal users. The practical use of our model, consequently, should only be limited to active and
vocal users of Twitter. Additionally, we acknowledge that Retweet-BERT is most accurate on datasets of
polarizing topics where users can be distinguished almost explicitly through verbal cues. This is driven
by two reasons. First, polarizing datasets makes it clearer to evaluate detection performance. Second, and
more importantly, the applications of Retweet-BERT are realistically more useful when applied to controversial or polarizing topics. Since our detection method relies on users revealing explicit cues for their
political preference in their profile descriptions or their retweet activities, we focus on the top 20% (most
likely right-leaning) and the bottom 20% (most likely left-leaning) when conducting the case study on the
polarization of COVID-19 discussions. The decision to leave most users out is intentional: we only want
to compare users for which Retweet-BERT is most confident in predicting political bias. Detecting user
ideology is a difficult and largely ambiguous problem, even for humans (Elfardy and Diab, 2016). Cohen
and Ruths (2013) raised concerns that it is harder to predict the political leanings of the general Twitter
public, who are much more “modest” in vocalizing their political opinions. Thus, we focus our efforts on
detecting the more extreme cases of political bias in an effort to reduce false positives (predicting users
as politically biased when, in fact, they are neutral) over false negatives (predicting users as politically
neutral when, in fact, they are biased).
32

Another major limitation is that Retweet-BERT is restricted to using only user profile descriptions
and retweet interactions, leaving out many more detailed and fine-grained aspects of social network data.
In the next chapter, we discuss how to extend Retweet-BERT, a model that relies on retweets and LLM
embeddings of the profile description, to a more comprehensive model that considers all social network
interactions and content cues.
33

Chapter 3
Social-LLM: Learning from Social Network Data
3.1 Introduction
In the previous chapter, we introduced Retweet-BERT, a simple and scalable solution for learning user representation from their profile descriptions and retweet connections. It builds on the assumption of social
network homophily, which suggests that users connected with retweets are more likely to be characteristically and linguistically similar (McPherson et al., 2001; Kovacs and Kleinbaum, 2020), to develop user
embeddings that encode socially connected users in closer latent spaces. We evaluated its feasibility in
detecting user political partisanship.
However, the concept of the method can be generalized to more complex data and is not limited to any
specific tasks. In this chapter, we introduce Social-LLM, a natural extension of Retweet-BERT. Social-LLM
can use multimodal user features, meaning different types of user content features, and can also draw
information from heterogeneous types of network interactions. Similar to Retweet-BERT, Social-LLM remains a self-supervised representation learning method that does not need labels. It is also inductive,
generalizing to any users with content features during the inference stage. We conduct a thorough evaluation of Social-LLMs on 7 real-world, large-scale social media datasets showcasing its applicability for a
diverse range of user detection tasks. We also showcase the utility of using Social-LLM embeddings for
visualization. We provide an overview of the model and its application in Figure 3.1
34

Figure 3.1: Overview of the Social-LLM method.
3.2 Related Work
User detection is a crucial element in computational social science research, encompassing a spectrum of
areas such as identifying ideologies (Barberá et al., 2015; Jiang et al., 2020), spotting inauthentic accounts
(Davis et al., 2016; Masood et al., 2019), flagging toxic behavior (Ribeiro et al., 2018a; Jiang et al., 2023b),
recognizing influencers (Rios et al., 2019), and assessing vulnerability to misinformation (Aral and Walker,
2012; Ye et al., 2024). While considerable information lies in the social network structure itself, most
user characterization methods that utilize network features only consider high-level statistics like node
centrality measures (Saravia et al., 2016; Davis et al., 2016; Masood et al., 2019), failing to fully capture
the complex relational patterns among users. Other studies tackle the task of political ideology prediction
using Bayesian ideal point estimation (Barberá, 2015) or label propagation (Conover et al., 2011a; Badawy
et al., 2018), but such methods require a subset of labeled users, which both come at a data acquisition cost
and results in solutions tailored at very specific problems.
In recent years, graph representation learning techniques have been proposed as the state-of-the-art
method to model complex network information in an unsupervised manner (Grover and Leskovec, 2016;
Liu et al., 2018; Hamilton, 2020). These methods capture higher-order proximity and community structure
from network topology but often demand substantial computational resources and heightened hardware
requirements during training that can limit scalability (Zhang et al., 2018). Some approaches alleviate the
35

complexity with sampling strategies (Serafini and Guan, 2021; Ma et al., 2022b), though this inherently
trades off representational power (Wu et al., 2020). In this chapter, we preserve the full social network
structure but utilize it in the simplest manner by considering only first-order proximity (i.e., the edges
between users). We demonstrate that such a simple method is sufficient for accurate user characterization
on large-scale social media datasets without sacrificing computational traceability.
Our proposed Social-LLM method utilizes multi-relational user interaction data along with contentbased user features. The most similar prior works are TIMME (Xiao et al., 2020), a scalable graph neural
network (GNN) for user classification that leverages multi-relational social networks, and GEM (Liu et al.,
2018), another heterogeneous GNN designed for malicious account detection. While these GNN methods
can incorporate user content as node features, because they inherently rely on modeling the network
structure, they cannot be applied inductively to out-of-sample users without retraining. In contrast, SocialLLM can be applied inductively on any unseen user during inference.
Social-LLM builds upon our previous Retweet-BERT model (Jiang et al., 2023c), which learns user embeddings encoding political orientation by optimizing them to be similar for users who retweet each other’s
content. Retweet-BERT only requires user profile text and retweet interactions as input yet demonstrates
effective political leaning prediction compared to other approaches. Social-LLM generalizes Retweet-BERT
by modeling more extensive heterogeneous social relations beyond just retweets and incorporating other
user content information.
3.3 Method
We propose Social-LLM, a model that leverages network homophily and user features to learn user representations scalably. This representation model can then be applied inductively for a range of downstream
user detection tasks. Social-LLM draws from two types of social network features: content cues from each
user and network cues from social interactions.
36

3.3.1 Content Cues
The content cues are derived mainly from the textual content on their social media but can also be from
other contextual metadata. We primarily utilize users’ profile descriptions, which are self-provided minibiographies. For most user detection task purposes, users’ biography encodes a substantial amount of
personal information with personal descriptors (e.g., “Senator”) and, in some cases, self-identities and beliefs (e.g., “Democratic”). Capped at 160 characters, these descriptions have to be short, incentivizing users
to convey essential information they wish to share about themselves succinctly and attractively. The use
of Twitter profile descriptions, not the tweet texts, has proved useful in a large number of computational
social science research (Rogers and Jones, 2021; Jiang et al., 2023a,c). From a practical standpoint, using
user profiles instead of all of the tweets by a user also vastly reduces the complexity of the computation
problem as well as alleviates data collection challenges. In addition to profile descriptions, we also leverage, when applicable, user metadata features (e.g., follower counts, account creation date, etc.) and user
tweets.
3.3.2 Network Cues
Online social media platforms offer a variety of ways to engage with one another, such as by following,
liking, or re-sharing. These acts of social interaction can be gathered to form social networks. The Twitter
API enables us to obtain three types of social interactions: retweeting, mentioning, and following. Though
the following network is perhaps the most useful indication of user interaction, it is rarely in empirical
research used due to the API rate limits set by Twitter (Martha et al., 2013). As such, following other
Twitter research (e.g., Conover et al., 2011b; Ferrara et al., 2016), we use the retweet and mention networks
in this research. Retweet refers to the act of re-sharing another user’s tweet directly, without any additional
comments. Mention includes all other acts of mentioning (using ‘@’) another user, including retweeting
with comments (i.e., quoting), replying, or otherwise referencing another user in a tweet. We draw a
37

distinction between retweets and mentions because they may represent two distinct motivations of social
interaction: retweeting is usually understood as an endorsement (Boyd et al., 2010; Metaxas et al., 2015)
while mentioning could be used to criticize publicly (Hemsley et al., 2018).
3.3.3 Social-LLM Framework
We train Social-LLM in an unsupervised manner to learn user representations in a d−dimensional embedding space (Figure 3.1, step 1). Once we train the user representation module, we can apply the user
representation module to obtain user embeddings. Additional layers can be trained on top of any downstream user detection task (Figure 3.1, step 2).
3.3.3.1 User Representation Module
The user representation module takes in any user content features and produces a user embedding. Most
importantly, we use pre-trained LLM models for any textual content, such as their profile description.
This LLM model would be trainable in order to allow for fine-tuning in our training process. Other inputs,
such as the user metadata features, will go through a deep neural network. We can also model tweet
text through text embeddings, but to ensure equal input length, we precompute the text embeddings and
average them. We concatenate these outputs into one single embedding and apply another dense layer to
produce a single d-dimensional embedding ui for user i.
3.3.3.2 Unsupervised Training Via Siamese Architecture
The user representation module is wrapped in a Siamese model architecture in a manner similar to
Sentence-BERT (Reimers and Gurevych, 2019), which employs identical representation modules on two
sentences and optimizes the similarity of their embeddings if the sentences are deemed semantically similar. In our case, we apply an identical representation module on the user content cue and optimize the
resulting embeddings based on the network cues. A training instance of Social-LLM is a tuple (i, j, k)
38

where i and j are two users who are connected by a social network interaction (i.e., an edge) of type k.
We want to train the user embeddings ui and uj so that they are as similar as possible. Sentence-BERT
(Reimers and Gurevych, 2019) and Retweet-BERT (Chapter 2) achieve this by optimizing the cosine similarity of embeddings. However, we also want to consider (1) multiple edge types–modeling retweets distinct
from mentions–and (2) directionality–user A retweeting from user B is not the same as user B retweeting
from user A. To account for multiple edge types, we initialize a learnable weight matrix Wk
for every
edge type k. To account for directionality, we can use separate weight matrices Wkin and Wkout for the inand out-edges. We then calculate the cosine similarity scores between Wkui and Wkuj , or Wkinui and
Wkinuj in the directional case, as the final output. We can also account for edge weights by weighting
each training instance proportionally to their weight.
3.3.3.3 Multiple Negatives Ranking Loss
We optimize with a ranking loss function, pitching positive examples against negative examples. All edges
in the graph serve as positive examples, and all other pairs of users who are not connected by an edge
can serve as negative examples. To speed up the training procedure, we use the multiple negatives loss
function (Henderson et al., 2017) used in Retweet-BERT (Chapter 2). Essentially, all other pairs of users
in the same batch serve as negative examples. For instance, if the input batch contains positive examples
[(i1, j1),(i2, j2), ...], then {(ix, jy)} for all x ̸= y are negative examples. This would encourage users who
are truly connected in the graph to have more similar representations than users who are not. To minimize
training complexity, we alternate the training of different types of edges in a round-robin fashion. For
example, if we want to accommodate for both k = retweet and k = mention edges, we will train one
batch of retweet edges, followed by one batch of mention edges in a round-robin manner.
39

3.3.3.4 Downstream Task Application
The Social-LLM model produces reusable user representation that can be used on any downstream user
prediction tasks (Figure 3.1, step 2). We can fine-tune the representation module further or freeze the layers
and add task-specific fine-tuning layers on top.
3.3.4 Advantages and Disadvantages
Social-LLM builds on traditional user detection methods by adding social network components. There are
two main advantages of Social-LLM over similar GNN approaches.
• Ease of training: The time complexity of step 1 is O(|E|), and that of step 2 is even quicker at
O(|V |). Crucially, since we forgo training the complete graph and only focus on edges as if they are
independent, we can fit very large datasets via batching.
• Inductive capabilities: Since step 2 of the framework no longer relies on the network, we can
extend our model to produce embeddings for any users, provided we have their content information,
without needing their network information and refitting the whole model. This is called inductive
learning, and most graph-embedding approaches either cannot natively support this (e.g., Grover
and Leskovec, 2016; Zhang et al., 2019), or they do so at a significantly higher training complexity
(Hamilton et al., 2017).
• Reusability: The Social-LLM embedding training process is separate from the downstream applications so that we can reuse the embeddings for various applications, including user detection tasks,
user clustering, and user visualization.
The advantages of Social-LLM come with costs. Notably, we sacrifice precision and thoroughness
for speed and efficiency. Our model focuses only on first-order proximity, or users who are connected
immediately by an edge. This undoubtedly loses valuable information from the global network structures
40

Table 3.1: Summary statistics of our Twitter datasets.
# Rt # Mn Profile Meta Tweet Time Pred.
Dataset # Users Edges Edges Desc. Feat. Texts Span Label(s) Type
Covid-Political 79,370 180,928 - ✓ ✓ ✗ 6 Mo Partisanship (1) Cls.
Election2020 75,301 2,818,603 - ✓ ✗ ✗ 3 Mo Partisanship (1) Cls.
COVID-Morality 119,770 609,845 639,994 ✓ ✓ ✗ 2 Yr Morality (5) Reg.
Ukr-Rus-Suspended 56,440 135,053 255,476 ✓ ✓ ✓ 1 Mo Suspension (1) Cls.
Ukr-Rus-Hate 82,041 166,741 414,258 ✓ ✓ ✗ 1 Mo Toxicity (6) Reg.
Immigration-Hate-08 5,759 63,097 83,870 ✓ ✓ ✗ All Toxicity (5) Reg.
Immigration-Hate-05 2,188 4,827 7,993 ✓ ✓ ✗ All Toxicity (5) Reg.
or higher-order proximities. However, as we will demonstrate in this chapter, in the cases of many user
detection problems on social networks, it is sufficient to model the localized connections for a cheap boost
to performance compared to a framework that does not use the social network at all. For these large but
sparse real-world social network datasets, the more powerful graph embedding methods may require a lot
more training time, memory footprint, or hardware resources for a marginal gain in performance.
3.4 Data
We describe the dataset we use to validate our approach. The first two datasets, Covid-Political and
Election2020, were described in Chapter 2. These datasets focusing only on using user profile descriptions and retweet interactions to predict political partisanship. We include them in this chapter again for
completeness. To demonstrate the additional capabilities of Social-LLM, we introduce several new datasets
below that encompass more heterogeneous user metadata and network features. Our new datasets add diversity to the types of labels, prediction methods, time spans, and data sizes to demonstrate the robustness
of our approach. Table 3.1 displays the summary statistics of our dataset.
3.4.1 Politics in COVID Discussion
The COVID-19 pandemic left an unprecedented impact on everyone worldwide. Research has shown that
COVID-19 was politicized, with partisanship steering online discourse about the pandemic (Calvillo et al.,
41

2020; Jiang et al., 2020), motivating our prediction task of detecting user partisanship. Our dataset, CovidPolitics, is based on a real-time collection of COVID-19 tweets (Chen et al., 2020) between January 21 and
July 31, 2020, and was further preprocessed in §2.3 to remove bots and inactive users (Davis et al., 2016;
Yang et al., 2022). We include user metadata features of the initial follower count, final follower count, number of tweets, number of original tweets, number of days actively posting, and whether the user is verified.
The ground truth partisanship labels from Jiang et al. (2023c) are derived from two heuristics-labeling approaches, using annotated political hashtags used in user profile descriptions and the partisanship leaning
of new media URLs mentioned in users’ tweets. This dataset contains around 79,000 users and 181,000
retweet interactions. The distribution of users is unbalanced, with 75% of users labeled as left-leaning.
3.4.2 2020 Presidential Election
The 2020 US presidential election took place amidst the backdrop of the COVID-19 pandemic. The
Election-2020 dataset is based on a real-time collection of tweets regarding the 2020 US presidential
election (Chen et al., 2021b) from March 1 to May 31, 2020. This dataset was also preprocessed to remove
bots and inactive users §2.3. This dataset contains around 75,000 users and 2.8 million retweet interactions.
Since this is specifically a dataset on US politics, we are also interested in predicting user partisanship. We
use the same partisanship labeling approach as above. The distribution split is even, with around 50% of
the users labeled as left-leaning.
3.4.3 Morality in COVID Discussion
The Covid-Morality dataset spans tweets on COVID-19 from February 2020 to October 2021 (Chen et al.,
2020). Our task is to predict users’ moral foundations based on the Moral Foundation Theory: care, fairness,
loyalty, authority, and purity (Haidt and Joseph, 2004). Prior work found social networks exhibit moral
homophily, with moral values such as purity predicting network distances (Dehghani et al., 2016). Studies
42

also link morality to COVID-19 health decisions such as masking and vaccination (Chan, 2021; Díaz and
Cova, 2022; Francis and McNabb, 2022), suggesting morality’s role in online communication patterns (Jiang
et al., 2024).
We use the best off-the-shelf moral value detector (Guo et al., 2023), a data-fusion morality detector
technique fine-tuned on this specific dataset, to label tweets with the 10 moral inclinations. Retaining
tweets with >= 1 moral value, we filter for active users with >= 10 moral tweets per month. From this
set, we sample 150,000 users and build their retweet/mention network, keeping edges with weight >= 2
(Jiang et al., 2023c). User meta features include account age, follower count, following count, list count,
tweet count, favorite count, and whether they are verified. Our multi-output regression task predicts each
user’s average moral foundation scores. We aggregate the 10 labels into 5 foundations, scoring 1 if both the
virtue and the vice are present, 0.5 if only one is present, and 0 otherwise. We combine the virtue and vice
scores for each moral foundation because the morality detector separately identifies explicit expressions of
both the positive (e.g., care) and negative (e.g., harm) aspects. However, for our purposes, the presence of
either aspect reflects a moral disposition within that particular foundation. We then calculate the user-level
moral foundation score as the average across their tweets.
3.4.4 Ukraine-Russia Suspended Accounts
After the Ukraine-Russia war erupted in 2022, Russian disinformation campaigns were rampant on social media (Pierri et al., 2023b). We use tweets about the conflict from March 2022 (Chen and Ferrara,
2023), where many users were suspended by Twitter,∗ often newer accounts exhibiting suspicious behaviors (Pierri et al., 2023a). We hypothesize that suspended and non-suspended users have different communication patterns due to divergent motivations.
∗
https://help.twitter.com/en/managing-your-account/suspended-x-accounts
43

Our task predicts whether a user was eventually suspended. The raw dataset contains 10M nonsuspended and 1M suspended users. We filter for users with >= 10 tweets to remove inactive users. To
focus on likely human-operated accounts, we remove users with > 130 tweets that month (90th percentile)
as a rough de-spamming step since we no longer had access to Botometer at this time of research. We then
sample an equal proportion of non-suspended users. We build the retweet/mention network with edges
>= 2 weight (Jiang et al., 2023c), removing isolated users. The final Ukr-Rus-Suspended dataset is 58%
suspended users. We use the same user metadata features as in §3.4.3. We also include tweet texts since
the labels are not derived from the tweets.
3.4.5 Ukraine-Russia Hate Speech
Beyond misinformation, online discourse on the Ukraine-Russia war was riddled with toxicity (Thapa et al.,
2022). As a spin-off from Ukr-Rus-Suspended, we experiment with detecting users’ toxicity levels from
their Twitter behavior in this Ukr-Rus-Hate dataset. After filtering for users with >= 10 and <= 130
tweets, we use the Perspective API†
toxicity detector (Kim et al., 2021; Frimer et al., 2023) on users’ original
tweets. We use six toxicity scores (overall, identity attack, insult, profanity, threat) from 0-1. We filter for
users with >= 10 tweets rated for toxicity. Unlike Ukr-Rus-Suspended, we retain all network edges
regardless of weight due to smaller network density. User metadata features are the same as §3.4.3. The
prediction task is to detect each user’s overall toxicity level from their tweets, Twitter activity, and network
interactions. Modeling toxicity can provide insights into how toxic language and user behaviors propagate
through online networks during crisis events.
3.4.6 Immigration Hate Speech
We compile an Immigration-Hate dataset by collecting historical tweets from users who previously
posted hateful immigration content (Bianchi et al., 2022). From 18,803 annotated uncivil/intolerant tweets
†
https://perspectiveapi.com/
44

Figure 3.2: Distribution of users’ bot scores prior to user preprocessing for the Immigration-Hate datasets.
in 2020-2021 (Bianchi et al., 2022), we successfully rehydrated 8,790 tweets by 7,566 users (the rest of the
tweets were no longer available). Considering these hateful users, we used the Twitter API to collect up to
their most recent 3,200 tweets, yielding 21 million tweets. Here, we focus only on the 2.9 million original
tweets.
We apply the Perspective API toxicity detector (§3.4.5) and aggregate each user’s average toxicity
scores across their tweets. A Botometer (Davis et al., 2016; Yang et al., 2022) analysis revealed many bot
accounts (Figure 3.2). To mitigate the influence of bots, we remove users using two thresholds of bot score:
0.8, which is a conservative choice given the peak in the distribution of bot scores, and 0.5, which would
leave us substantially fewer users but with a higher certainty that they are genuine. Since the user set is
relatively small, we retain all network edges. User metadata features are the same as §3.4.3. The prediction
task aims to model each user’s propensity for hate speech from their toxicity levels, activity patterns, and
network position.
3.5 Evaluation
We evaluate Social-LLM by conducting extensive comparisons with baseline methods for user detection
tasks. We also include sensitivity and ablation studies.
45

3.5.1 Baseline Methods
For a thorough evaluation of our approach, we use a series of state-of-the-art baseline methods divided
into three camps: content-based, network-based, and hybrid methods. The content-based and networkbased models provide an alternative user embedding that we can utilize in the evaluation procedure (Figure
3.1, step 2). All input embeddings undergo similar training processes for target task prediction. For the
hybrid method, we use TIMME, an end-to-end user detection method that also uses both user features and
network features. We conduct a thorough hyperparameter tuning process for all of the baseline models.
3.5.1.1 Content-Based Methods
For Content-Based Methods, we primarily investigate using embeddings from pre-trained LLMs. Finetuning the LLMs for our specific purpose is also one option; however, doing so on the Covid-Politics and
Election2020 dataset did not deliver a substantial enough improvement to justify the added training cost.
Here, we experiment with the following three LLMs applied to the profile descriptions:
1. RoBERTa-base (Liu et al., 2019),
2. BERTweet (Nguyen et al., 2020), a RoBERTa fine-tuned on Twitter data,
3. SBERT-MPNet, a Sentence-BERT (Reimers and Gurevych, 2019) model based on MPNet (Song et al.,
2020) and is, as of late 2023, the best-performing Sentence-BERT model.‡
For datasets with additional metadata features, we also experiment with using only the raw metadata
features as the “user embeddings” as well as with concatenating the LLM embeddings with the raw metadata features. For Ukr-Rus-Suspended, we additionally experiment with applying the aforementioned
three LLMs on users’ tweets, averaging one LLM embedding per user.
‡
https://www.sbert.net/docs/pretrained_models.html (Accessed November 2023).
46

3.5.1.2 Network-Based Methods
We use two network-based methods as baselines: node2vec (Grover and Leskovec, 2016) and ProNE (Zhang
et al., 2019). While GraphSAGE (Hamilton et al., 2017) is another suitable choice for inductive graph
representation learning with node attributes, it is, in practice, to train on a large graph within reasonable
time limits and can, therefore, underperform (Jiang et al., 2023c). These network embedding methods
support weights and directions but also heterogeneous edge types. Therefore, we run a separate network
model on the (1) retweet edges only, (2) mention edges only, and (3) indiscriminately combining retweet
and mention edges as one.
3.5.1.3 Hybrid Method
We use TIMME (Xiao et al., 2020) as our hybrid method baseline, providing it with the user content features
and network features as our Social-LLM model. The original model was only designed for user classification tasks, but we modified the open-sourced code to enable regression. Since TIMME is designed to be
a multi-relational model, we mainly apply it on both retweet and mention edges, but we also experiment
with combining these edges indiscriminately.
3.5.2 Experimental Setup
For every dataset and its corresponding set of user embeddings, we conduct 10 repetitions of the training
and evaluation procedure, splitting the dataset randomly using 10 pre-selected seeds into 60%-20%-20%
train-val-test splits. The validation sets are used for early stopping and model selection. The classification
tasks are evaluated using Macro-F1, and the regression tasks are evaluated using Pearson’s correlation
averaged across multiple labels.
47

Table 3.2: Social-LLM results on the classification task datasets evaluated using Macro-F1 scores. The best
model for each experiment is in bold, and the best baseline model is underlined.
Content Network Election Covid- Ukr-RusFeatures Features 2020 Political Suspended
Experiment 1: LLMs
RoBERTa ✓ ✗ 80.11 78.41 56.21
BERTweet ✓ ✗ 79.31 78.42 55.69
SBERT-MPNet ✓ ✗ 86.47 82.99 56.79
Experiment 2 (Main): Baselines vs Social-LLM
(a) Profile LLM ✓ ✗ 86.47 82.99 56.79
(a) + (b) Metadata ✓ ✗ - 83.26 70.75
(a) + (b) + (c) Tweet LLMs ✓ ✗ - - 81.74
(d) node2vec ✗ ✓ - 88.65 72.33
(e) ProNE ✗ ✓ 76.28 64.04 77.95
(f) TIMME ✓ ✓ 84.81 81.85 72.91
Social-LLM ✓ ✓ 97.87 90.82 82.71
%↑ 13% 2% 1%
Experiment 3: Ablation on edge types in Social-LLM models
RT ✓ ✓ - - 70.71
MN ✓ ✓ - - 71.32
RT & MN (distinct) ✓ ✓ - - 71.99
RT + MN (indistinct) ✓ ✓ - - 72.10
Experiment 4: Ablation on edge directions and weights in Social-LLM models
(best edge combo model) ✓ ✓ 97.78 90.68 72.10
+ w ✓ ✓ 97.78 90.55 71.85
+ d ✓ ✓ 97.85 90.82 71.77
+ d + w ✓ ✓ 97.82 90.42 72.17
3.6 Results
3.6.1 Experiments
Below, we discuss our experimental results and interpret our findings. For model selection within the same
family of methods, we use the validation sets to select the final model. The results for the classification
tasks are shown in Table 3.2, and the results for the regression tasks are shown in Table 3.3. We display
the average value of the metric over 10 repeated random splits.
48

Table 3.3: Social-LLM results on the regression task datasets evaluated using Pearson correlation scores.
The best model for each experiment is in bold, and the best baseline model is underlined.
Content Network Covid- Ukr-Rus- Immigration-HateFeatures Features Morality Hate 05 08
Experiment 1: LLMs
RoBERTa ✓ ✗ 32.84 36.54 12.06 9.30
BERTweet ✓ ✗ 30.72 40.38 14.33 12.03
SBERT-MPNet ✓ ✗ 36.77 43.35 17.16 16.76
Experiment 2 (Main): Baselines vs Social-LLM
(a) Profile LLM ✓ ✗ 36.77 43.35 17.16 16.76
(a) + (b) Metadata ✓ ✗ - 45.38 17.72 17.32
(a) + (b) + (c) Tweet LLMs ✓ ✗ - - - -
(d) node2vec ✗ ✓ 50.53 39.97 10.70 12.18
(e) ProNE ✗ ✓ 51.13 45.38 5.47 14.30
(f) TIMME ✓ ✓ 30.47 43.46 20.98 18.67
Social-LLM ✓ ✓ 50.15 57.27 21.17 20.11
%↑ -2% 26% 1% 7%
Experiment 3: Ablation on edge types in Social-LLM models
RT ✓ ✓ 46.57 48.18 18.85 18.18
MN ✓ ✓ 45.33 49.55 18.73 17.92
RT & MN (distinct) ✓ ✓ 20.40 51.20 14.75 18.89
RT + MN (indistinct) ✓ ✓ 47.51 50.73 19.05 18.53
Experiment 4: Ablation on edge directions and weights in Social-LLM models
(best edge combo model) ✓ ✓ 47.51 50.73 18.53 19.05
+ w ✓ ✓ 46.98 51.46 18.70 18.81
+ d ✓ ✓ 50.15 57.19 18.95 18.77
+ d + w ✓ ✓ 46.89 49.35 17.67 19.21
3.6.1.1 Experiment 1 - Choice of LLMs
We first experiment with the choice of LLMs by running our prediction tasks using only the profile LLM
embeddings as features. This determines both the best baseline method for LLMs and which LLM we should
use in our Social-LLM models. We selected the clear winner, SBERT-MPNet, which outperformed RoBERTa
and BERTweet on all datasets. We note that, given rapid innovations in NLP and LLMs, SBERT-MPNet may
not be the best model or could soon be replaced by a better successor. However, the contribution of SocialLLM is not tied to a single LLM but rather a model training paradigm that can be paired with any LLM.
Our choices of LLMs are driven by ease of use, costs, and reproducibility (Bosley et al., 2023).
49

3.6.1.2 Experiment 2 - Main Experiments
In Tables 3.2 and 3.3 Experiment 2, we underline the best baseline model and boldface the best model (either
a baseline or the Social-LLM model) for each dataset. We also indicate the percentage change in performance using Social-LLM compared to the best-performing baseline. Regarding baseline methods, there
is no clear winner among the content-based, network-based, or hybrid models. Network-based models
face much higher variability in performance across datasets, pointing at issues when using solely network
features. Notably, we observe that Social-LLM is superior in nearly all cases, with improvements ranging
from a substantial 26% to a modest 1%. Using a one-sided t-test, we find that all improvements are statistically significant. Social-LLM performs comparatively worse than the baselines only in Covid-Morality,
where the network embedding models are slightly superior, but the Social-LLM model still demonstrates
commendable performance. In summary, Social-LLM emerges as the most consistent model, exhibiting
robustness across various tasks and data sizes.
3.6.1.3 Experiment 3 - Edge Type Ablation
We perform an edge-type ablation experiment to evaluate the importance of each edge type on datasets
containing both retweets and mention edges. When using only one type of edge, we find they perform
comparably. The combination of retweets and mentions as two distinct edge types occasionally results in
improved performance but can also lead to deteriorated outcomes. However, combining them indiscriminately as one edge type yields the best performance consistently. This suggests that both retweets and
mentions carry important signals, yet the distinctions between the two actions might not be substantial
enough to warrant differentiation for our objective tasks.
50

Figure 3.3: Social-LLM Experient 5: Ablation study of user tweet embeddings on the Ukr-Rus-Suspended
dataset.
3.6.1.4 Experiment 4 - Edge Weights and Directions
Using the best edge-type model (RT for Election2020 and Covid-Political, and RT + MN for all others),
we then experiment with adding edge weights (+ w) and edge directions (+ d). The inclusion of directions always yields better performance, and occasionally, the performance is further enhanced if we stack
on weights as well. The importance of directionality emphasizes the value of understanding the flow of
information exchange on social networks.
3.6.1.5 Experiment 5 - User Tweet Embeddings
On Ukr-Rus-Suspended, we experiment with additionally including user tweet embeddings as features. In
Figure 3.3, we see that using user tweet embeddings leads to an average improvement of 4% Macro-F1 between otherwise identically configured models. This experiment underscores the importance of including
user tweets, when applicable and suitable, in user prediction tasks.
3.6.1.6 Experiment 6 - Sensitivity to Dimension Size
For every dataset, we select the Social-LLM model with the best edge type, edge weight, and edge direction
configuration to plot the sensitivity to embedding dimension d. The results are presented in Figure 3.4.
51

Figure 3.4: Social-LLM Experiment 6: Sensitivity to embedding dimension d.
Performance generally increases with rising dimensions, with d = 258 being a popular choice; however,
we note that Social-LLM usually performs quite well even with very low dimensions.
3.6.1.7 Experiment 7: Sensitivity to Training Size
We compare how Social-LLM fares with baseline methods under varying conditions of training size in Figure 3.5. On Covid-Political, Election2020, and Ukr-Rus-Hate, Social-LLM consistently yields higher
performance even given a very small training size. In other datasets, Social-LLM demonstrates similar or
slightly inferior performance compared to the best-performing baseline but is never substantially worse
than the best baseline. Given the variability in the baseline models across datasets, the consistency and
robustness in Social-LLM performance is noteworthy.
3.6.2 Visualization
We further demonstrate Social-LLM’s interpretability by visualizing the user embeddings with t-SNE
(Van der Maaten and Hinton, 2008). As shown in Fig 3.6, the embeddings effectively capture distinctions
52

Figure 3.5: Social-LLM Experiment 7: Sensitivity to training size.
between political ideologies on Covid-Politicsand the Election2020 with clear separations between liberal and conservative users. For the Ukr-Rus-Suspended dataset, while there is no global separation
between suspended and non-suspended accounts, localized clustering emerges. We also see concentrations of more hateful users against less hateful users in the Ukr-Rus-Hate dataset. These visualizations
highlight Social-LLM’s ability to encode user attributes and behaviors into an interpretable embedding
space, offering insights into the structure and composition of social networks.
3.7 Conclusion
This chapter presents Social-LLM, a scalable method for learning user representations by integrating user
content and social network interactions. Our approach combines large language model embeddings of
user profiles with a simple yet effective modeling of the social graph structure. Instead of complex highorder proximity, we exploit the inherent sparsity and homophily in social networks by considering only
the direct edges between users. Leveraging 7 different large Twitter datasets spanning a diverse range of
user detection tasks, we showcase the advantage and robustness of our method. Importantly, once fitted on
53

Figure 3.6: Visualization of Social-LLM embeddings.
a social network, Social-LLM can generalize to new users and downstream tasks using only user content
features without requiring the original network data. Overall, Social-LLM provides an accurate, scalable,
and generalizable framework for characterizing users on social media through their digital footprints.
In the next part of this dissertation, I delve into the qualitative dimensions of this dissertation, focusing on understanding human behavior mining through social network analysis. This part harnesses the
capabilities of the Social-LLM method for generating user embeddings, which are then applied in diverse
contexts. In Chapter 4, I use the embeddings to categorize users by political partisanship to understand
the process and impact of political polarization. Chapter 5 leverages these embeddings as features within
a machine learning model that estimates the level of social approval received by hateful users. This approach enables the identification of anomalies where users receive significantly more or less approval
54

than anticipated, offering insights into the mechanisms of social validation in hate speech behavior. In
Chapter 6, I extend the application of Social-LLM embeddings to datasets enriched with information on
users’ moral foundations. By employing clustering techniques on these embeddings, the research uncovers distinct communities within the social network characterized by their moral leanings. This exploration
contributes to our understanding of the moral underpinnings that influence community formation and interaction online. These varied applications underscore the versatility of Social-LLM embeddings in computational social science and the analysis of online social networks, highlighting their potential to reveal
intricate patterns of human behavior and social structuring.
55

Part II
Data-Driven Analysis of Online Human Behavior
56

Chapter 4
Social Media Polarization and Echo Chambers Surrounding COVID-19
4.1 Introduction
As the unprecedented COVID-19 pandemic continues to put millions of people at home in isolation, online
communication, especially on social media, is seeing a staggering uptick in engagement (Koeze and Popper,
2020). Prior research has shown that COVID-19 has become a highly politicized subject matter, with
political preferences linked to beliefs (or disbelief) about the virus (Calvillo et al., 2020; Uscinski et al.,
2020), support for safe practices (Jiang et al., 2020), and willingness to return to activities (Naeim et al.,
2021). As the United States was simultaneously undergoing one of the largest political events – the 2020
presidential election – public health policies may have been undermined by those who disagree politically
with health officials and prominent government leaders. As it happens with topics that become politicized,
people may fall into echo chambers – the idea that one is only presented with the information they already
agree with, thereby reinforcing one’s confirmation bias (Garrett, 2009; Barberá et al., 2015).
Social media platforms have been criticized for enhancing political echo chambers and driving political
polarization (Conover et al., 2011b; Schmidt et al., 2017; Cinelli et al., 2021). In part, this is due to a conscious
decision made by users when choosing who or what to follow, selectively exposing themselves to contents
they already agree with (Garrett, 2009); but this may also be a consequence of the algorithms social media
platforms use to attract users (Schmidt et al., 2017). Numerous studies have shown that echo chambers
57

are prevalent on Twitter (Conover et al., 2011b; An et al., 2014; Colleoni et al., 2014; Barberá et al., 2015;
Cossard et al., 2020); however, most past works are done on topics that are political in nature.
In the case of COVID-19, the risks of political polarization and echo chambers can have dire consequences in politicizing this topic that is originally about public health. The lack of diversity in multiperspective and evidence-based information can present serious consequences on society by fueling the
spread of misinformation (Del Vicario et al., 2016; Shu et al., 2017; Motta et al., 2020). For instance, prior research revealed that conservative users push narratives contradicting public health experts (e.g., anti-mask)
and misinformation (e.g., voter fraud) (Chen et al., 2021a). Another research shows that the consumption
of conservative media is linked to an increase in conspiracy beliefs (Romer and Jamieson, 2021). Understanding the degree of polarization and the extent of echo chambers can help policymakers and public
health officials effectively relay accurate information and debunk misinformation to the public.
In this chapter, we focus on the issue of COVID-19 and present a large-scale empirical analysis of the
prevalence of echo chambers and the effect of polarization on social media. Our research is guided by the
following research questions surrounding COVID-19 discussions on Twitter:
• RQ1: What are the roles of partisan users on social media in spreading COVID-19 information?
How polarized are the most influential users?
• RQ2: Do echo chambers exist? And if so, what are the echo chambers, and how do they compare?
The technical challenge for addressing these questions is posed by the need to build a scalable and
reliable method to estimate user political leanings. To this end, we use the Retweet-BERT model we developed (Chapter 2), an end-to-end model that estimates user polarity from their profiles and retweets on
a spectrum from left- to right-leaning. Using the estimated polarity scores for all 232,000 Twitter users
in our data, we observe and compare the Twitter usage trends of partisan users. Our analyses show that
right-leaning users are more vocal in creating original content, more active in broadcasting information
58

(by retweeting), and more impactful through distributing information (by getting retweeted) than their
left-leaning counterparts. Moreover, influential users are usually highly partisan, a finding that holds irrespective of the influence measure used.
Finally, we provide evidence that political echo chambers are apparent at both political extremes,
though the degrees of cross-ideological interactions are highly asymmetrical: While communication channels remain open between left-leaning and neutral users, right-leaning users are found in a denselyconnected political bubble of their own. Information rarely travels in/out of the right-leaning echo chamber.
As our work offers unique insights into the polarization of COVID-19 discussions on Twitter, it carries
broader implications for identifying and combating misinformation spread, as well as strengthening the
online promotion of public health campaigns. Further, since communication across the two echo chambers
functions very differently, we stress that communication effectiveness must be evaluated separately for
people in each echo chamber.
4.2 Data
We use a large COVID-19 Twitter dataset collected by Chen et al. (2020), containing data from January 21
to July 31, 2020 (v2.7). All tweets collected contain keywords relevant to COVID-19. The tweets can be
original tweets, retweets, quoted tweets (retweets with comments), or replies. Each tweet also contains
the user’s profile description, the number of followers they have, and the user-provided location. Some
users are verified, meaning they are authenticated by Twitter in the interest of the public, reducing the
chance that they are fake or bot accounts (Hentschel et al., 2014). All users can optionally fill in their profile
descriptions, which can include personal descriptors (e.g., “Dog-lover”, “Senator”, “Best-selling author”) and
the political party or activism they support (e.g., “Republican”, “#BLM”).
59

4.2.1 Interaction Networks
The retweet network GR = (V, E) is modeled as a weighted, directed graph. Each user u ∈ V is a node
in the graph, each edge (u, v) ∈ E indicates that user u has retweeted from user v, and the weight of
an edge w(u, v) represents the number of retweets. We use the terms retweet interaction and edges of
the retweet network interchangeably. Similarly, we construct the mention network GM, where the edges
are mentions instead of retweets. A user can be mentioned through retweets, quoted tweets, replies, or
otherwise directly mentioned in any tweet.
4.2.2 Data Preprocessing
We restrict our attention to users who are likely in the United States, as determined by their self-provided
location (Jiang et al., 2020). Following Garimella et al. (2018b), we only retain edges in the retweet network
with weights of at least 2. Since retweets often imply endorsement (Boyd et al., 2010), a user retweeting
another user more than once would imply stronger endorsement and produce more reliable results. As
our analyses depend on user profiles, we remove users with no profile data. We also remove users with
degrees less than 10 (in- or out-degrees) in the retweet network, as these are mostly inactive Twitter users.
To remove biases from potential bots infiltrating the dataset (Ferrara, 2020), we calculate bot scores using
Davis et al. (2016), which estimates a score from 0 (likely human) to 1 (likely bots), and remove the top
10% of users by bot scores as suggested by Ferrara (2020).
Our final dataset contains 232,000 users with 1.4 million retweet interactions among them. The average
degree of the retweet network is 6.15. For the same set of users in the mention network, there are 10 million
mention interactions, with an average degree of 46.19. Around 18,000, or approximately 8% of all, users
are verified.
A subset of this dataset is pseudo-labeled and is referred to as the Covid-Political dataset in §2.3.
This subset only contains users that we are able to label heuristically as left-leaning or right-leaning from
60

the hashtags and news media URLs they use. This is used to develop a model to estimate user polarity for
all users.
4.3 Method
We use Social-LLM to estimate user political ideology. We first use the Social-LLM model that worked best
on the Covid-Political dataset, which is also the Retweet-BERT model in this case, to estimate political
ideology. We then train another machine learning model on the user embeddings to estimate user political
leaning, using pseudo-labeled seed users as the supervision. Finally, we infer polarity scores for the rest
of the users not pseudo-labeled, ranging from 0 (far-left) to 1 (far-right). Since there are more left-leaning
seed users in our dataset, the predicted polarity scores are naturally skewed towards 0 (left). Therefore,
we bin users by evenly distributed deciles of the polarity scores, with each decile containing exactly 10%
of all users.
4.4 Results
4.4.1 The Roles of Partisan Users
We first examine the characteristics of extremely polarized users, defined as the users in the bottom (leftleaning/far-left) or top (right-leaning/far-right) 20% of the polarity scores. As a point of comparison, we
also include neutral users who are in the middle 20% of the polarity scores. Considering various aspects
of user tweeting behaviors, we characterize the Twitter user roles as follows:
1. Information creators: those who create original content and are usually the source of new information.
2. Information braodcasters: those who foster the distribution of existing content, such as through
retweeting other people and promoting the visibility of other’s content.
61

Figure 4.1: COVID-19 dataset statistics of left-leaning (bottom 20%), neutral (middle 20%), and right-leaning
(top 20%) users partitioned by their verification status. The degree distributions are taken from the retweet
network. All triplets of distributions (left-leaning, neutral, and right-leaning) are significant using a oneway ANOVA test (P < 0.001).
3. Information distributors: those whose contents are likely to be seen by many people, either through
passive consumption by their followers or through broadcasting (retweeting) by others.
According to these definitions, a user can be all of these or none of these at the same time. In Figure
4.1, we plot several Twitter statistics regarding the polarized and neutral users, disaggregated by their
verification status.
Compared to unverified users, verified users are more likely information creators. This is unsurprising,
given that verified users can only be verified if they demonstrate they are of public interest and noteworthy.
Comparatively, left-leaning verified have the smallest fraction of original posts. However, this is reversed
for unverified users, with unverified left-leaning users having the highest fraction of original content and
unverified right-leaning users having little to no original content. We note that this may be related to the
distribution of bot scores. Figure 4.1(b) reveals that right-leaning users score significantly higher on the
bot scale. Since bots retweet significantly more than normal users (Ferrara et al., 2016), we cannot rule out
the possibility that right-leaning bots are confounding the analysis. However, users scoring the highest on
the bot scale have already been removed from the data.
Unverified right-leaning users, in comparison with their left-leaning counterparts, are more likely
information broadcasters as they have the highest out-degree distribution (Figure 4.1(c)). As out-degree
62

Figure 4.2: The proportion of users in the COVID-19 dataset in each decile of predicted political bias scores
that are (a) verified, (b) top 5% in the number of followers, (c) top 5% of in-degrees in the retweet network
(most retweeted by others), (c) top 5% of in-degrees in the mention network (most mentioned by others),
and (e) top 5% in PageRank in the retweet network.
.
measures the number of people a user retweets from, a user with a high out-degree functions critically in
information broadcasting. The fact that they also have very little original content (Figure 4.1(a)) further
suggests that unverified right-leaning users primarily retweet from others.
Finally, all right-leaning users function as information distributors regardless of their verification status. Their tweets are much more likely to be shared and consumed by others. Their high in-degree distribution indicates they get retweeted more often (Figure 4.1(d)), and the higher number of followers they
have indicates that their posts are likely seen by more people (Figure 4.1(e)).
As right-leaning users play larger roles in both the broadcasting and distributing of information, we
question if these users form a political echo chamber, wherein right-leaning users retweet frequently from,
but only from, users who are also right-leaning. As we will see later in the chapter, we indeed find evidence
that right-leaning users form a strong echo chamber.
4.4.2 The Polarity of Influencers
The above characterizes the Twitter activities of users who are extremely left or right-biased. However,
the majority of the social influence is controlled by a few key individuals (Wu et al., 2011; Lou and Tang,
2013; Zhang et al., 2015). In this section, we consider five measures of social influence: verification status,
number of followers, number of retweets, number of mentions, and PageRank in the retweet network (Page
63

et al., 1999). A user is considered influential if they are in the top 5% of all people according to the measure
of influence. Figure 4.2 reveals the proportion of users in each decile of polarity score that is influential.
We show that, consistent with all of the influence measures above, partisan users are more likely to be
found influential.
The verification status is correlated with partisan bias, with the proportion of verified users decreasing
linearly as we move from the most left- to the most right-leaning deciles of users (Figure 4.2(a)). 15% of
users in the 1st and 2nd deciles, which are most liberal, are verified, compared to less than 1% of users in the
extremely conservative 10th decile. As verified accounts generally mark the legitimacy and authenticity of
the user, the lack of far-right verified accounts opens up the question of whether there is a greater degree
of unverified information spreading in the right-leaning community. We stress, however, that our result
is cautionary. A closer investigation is needed to establish if there are other politically driven biases, such
as a liberal bias from Twitter as a moderating platform, that may contribute to the under-representation
of conservative verified users.
While being verified certainly aids visibility and authenticity, users do not need to be verified to be
influential. We observe bimodal distributions (U-shaped) in the proportion of users who are influential
with respect to their polarity according to three measures of influence: top most followed, retweeted,
and mentioned (Figure 4.2(b)-(d)), indicating that partisan users have more influence in these regards. In
particular, far-right users have some of the highest proportion of most-followed users. Far-left users are
more likely to be highly retweeted and mentioned, but the far-right also holds considerable influence in
those regards.
Lastly, we look at PageRank, a well-known algorithm for measuring node centrality in directed networks (Page et al., 1999). A node with a high PageRank is indicative of high influence and importance.
Much like the distribution of verified users, the proportion of users with high PageRank in each polarity
decile is correlated with how left-leaning the polarity decile (Figure 4.2(b)), which suggests that left-leaning
64

Figure 4.3: The distribution of left-leaning (bottom 20% of the polarity scores), center (middle 20%), and
right-leaning (top 20%) retweeters (y-axis) for users in the COVID-19 dataset across the polarity score
deciles (x-axis). The retweeted users are either verified or not verified.
users hold higher importance and influence. However, this phenomenon may also be an artifact of the
much larger left-leaning user base on Twitter.
4.4.3 Echo Chambers
As most influential users are partisan, we question if echo chambers exist and how prevalent they are. We
begin by exploring the partisan relationship between the retweeted and the retweeter, where the latter
is considered as the (immediate) audience of the former. Figure 4.3 plots the proportion of left-leaning,
neutral, or right-leaning retweeters for users in each of the 10 deciles of polarity scores, revealing that
users on both ends of the political spectrum reach an audience that primarily agrees with their political
stance. In fact, the far-left and far-right users have virtually no retweeters from supporters of the opposite
party. However, the echo chamber effect is much more prominent on the far-right. About 80% of the
audience reached by far-right users are also right. In comparison, only 40% of the audience reached by
far-left users are also left. There is little difference in the distribution of retweeters between verified and
unverified users.
65

Since the polarized users are mostly preoccupied in their echo chambers, the politically neutral users
(Figure 4.3, green) would serve the important function of bridging the echo chambers and allowing for
cross-ideological interactions. Most of them (30-40%) retweet from sources that are also neutral, and
around 20% of them retweet from very liberal sources. When it comes to broadcasting tweets from the
far-right, they behave similarly to the far-left retweeters: Almost no neutral users retweet from the farright. Such observations would imply a much stronger flow of communication between the far-left users
and neutral users, whereas the far-right users remain in a political bubble.
4.4.4 Random Walk Controversy
Previously, we explored the partisan relationship between users and their immediate audience. To quantify how information is disseminated throughout the Twitter-sphere and its relationship with user polarity,
we conduct random walks on the graphs to measure the degree of controversy between any two polarity deciles of users. Our method extends the Random Walk Controversy (RWC) score for two partitions
(Garimella et al., 2018b), which uses random walks to measure the empirical probability of any node from
one polarity decile being exposed to information from another.
A walk begins with a given node and recursively visits a random out-neighbor of the node. It terminates
when the maximum walk length is reached or if a node previously seen on the walk is revisited. Following
Garimella et al. (2018b), we also halt the walk if we reach an authoritative node, which we define as the
top 1000 nodes (≈ 4%) with the highest in-degree in any polarity decile. By stopping at nodes with high
in-degrees, we can capture how likely a node from one polarity decile would receive highly endorsed and
well-established information from another polarity decile. To quantify the controversy, we measure the
RWC from polarity decile A to B by estimating the empirical probability
RWC(A, B) = P r(start in A|end in B). (4.1)
66

Figure 4.4: The RWC(X, Y ) for every pair of polarity deciles X and Y on the retweet (left) and mention
(right) networks using Eq. 4.1.
The probability of walks starting in a partiion is conditional on the walks ending in a given partition to
control for varying distribution of high-degree vertices in each polarity decile. RWC yields a probability,
with a high RWC(A, B) implying that random walks landing in B started from A. Compared to the
original work (Garimella et al., 2018b), we simplify the definition of RWC as we do not need to consider
the varying number of users in each echo chamber.
We initiate the random walks 10,000 times randomly in each polarity decile for a maximum walk length
of 10. The RWC between any two polarity deciles for the retweet and mention networks are visualized
in Figure 4.4. For both networks, the RWC scores are higher along the diagonal, indicating that random
walks most likely terminate close to where they originated. Moreover, the intensities of the heatmap
visualizations confirm that there are two separate echo chambers. The right-leaning echo chamber (top
right corner) is much denser and smaller than the left-leaning echo chamber (bottom left corner). Any
walk in the retweet network that originates in polarity deciles 9 and 10 will terminate in polarity deciles 8
to 10 about 80% of the time. In contrast, walks that start in deciles 1–7 have a near equal, but overall much
67

Figure 4.5: Users in the COVID-19 dataset with the highest number of retweeters from left- and rightleaning users. The bar plots show the distribution of their unique retweeters by political leaning. Users
are also ranked by their total number of retweeters (i.e., #1 @realDonaldTrump means that @realDonaldTrump has the most retweeters). Numbers appended to the end of the bars show their total number of
retweeters.
smaller, probability of landing in deciles 1–7. In essence, users who are right-leaning form a smaller but
stronger echo chamber, while other users form a larger and more distributed echo chamber.
The RWC scores on the mention network confirm the presence of the two echo chambers, but the intensities are reduced. Compared to random walks on the retweet network, those on the mention network
are much more likely to end far away. As a result, while there are rarely any cross-ideological retweet interactions, there exists a greater degree of direct communication through mentions, likely done to speak to
or criticize the opposing side (Conover et al., 2011b). We note that because the RWC scores appear highly
symmetrical about the diagonals, there is little difference in the cross-ideological interaction between opposite directions of communication flow.
4.4.5 Popular Users Among the Left and Right
Retweeting is the best indication of active endorsement (Boyd et al., 2010) and is commonly used as the
best proxy for gauging popularity and virality on Twitter (Cha et al., 2010). Figure 4.5 shows the users who
are the most popular users among the left and the right according to the number of left- or right-leaning
retweeters they have.
68

Analyzing the identities of the top-most retweeted users by partisans gives us the first hint at the presence of political echo chambers. There is no overlap between the most retweeted users by the left- and
the right-leaning audience, and they tend to be politically aligned with the polarization of their audience.
Almost all users who are most retweeted by left-leaning users are Democratic politicians, liberal-leaning
pundits, or journalists working for left-leaning media. Notably, @ProjectLincoln is a political action committee formed by Republicans to prevent the re-election of the Republican incumbent President Trump.
Similarly, almost all users who are most retweeted by right-leaning users are Republican politicians, rightleaning pundits, or journalists working for right-leaning media. Despite its username, @Education4Libs is
a far-right account promoting QAnon, a far-right conspiracy group. As of January 2021, @Education4Libs
has already been banned by Twitter.
These popular users are not only popular among the partisan users but are considerably popular overall, as indicated by the high overall rankings by the number of total retweeters. With a few exceptions,
users who are popular among the left are more popular among the general public than users who are
popular among the right.
The distribution of the polarity of retweeters of these most popular users reveals another striking observation: The most popular users among the far-right rarely reach an audience that is not also right,
whereas those of the far-left reach a much wider audience in terms of polarity. Users who are popular
among the far-left hail the majority of their audience from non-partisan users (around 75%) and, importantly, draw a sizable proportion of far-right audience (around 5%). In contrast, users who are popular
among the far-right have an audience made up almost exclusively of the far-right (around 80%) and amass
only a negligible amount of far-left audience.
69

4.5 Discussion
In this chapter, we studied the extent of echo chambers and political polarization in COVID-19 conversations on Twitter in the US. Using Social-LLM–more specifically, the earlier version named Retweet-BERT–a
model that leverages user profile descriptions and retweet interactions to effectively and accurately measure the degree and direction of polarization, we provided insightful characterizations of partisan users
and the echo chambers in the Twitter-sphere to address our research questions.
RQ1. What are the roles of partisan users on social media in spreading COVID-19 information? How polarized are the most influential users? From characterizing partisan users, we find that right-leaning users
stand out as being more vocal, more active, and more impactful than their left-leaning counterparts.
Our finding that many influential users are partisan suggests that online prominence is linked with
partisanship. This result is in line with previous literature on the “price of bipartisanship,” which is that
bipartisan users must forgo their online influence if they expose information from both sides (Garimella
et al., 2018a). In another simulated study, Garibay et al. (2019) shows that polarization can allow influential
users to maintain their influence. Consequently, an important implication is that users may be incentivized
to capitalize on their partisanship to maintain or increase their online popularity, thereby further driving
polarization. Information distributed by highly polarized yet influential users can reinforce political predispositions that already exist, and any polarized misinformation spread by influencers risks being amplified.
RQ2. Do echo chambers exist? And if so, what are the echo chambers, and how do they compare? Though
COVID-19 is a matter of public health, we discover strong evidence of political echo chambers on this topic
on both ends of the political spectrum, particularly within the right-leaning community. Right-leaning
users are almost exclusively retweeted by users who are also right-leaning, whereas the left-leaning and
neutral users have a more proportionate distribution of retweeter polarity. From random walk simulations,
we find that information rarely travels in or out of the right-leaning echo chamber, forming a small yet
70

intense political bubble. In contrast, far-left and non-partisan users are much more receptive to information
from each other. Comparing users who are popular among the far-left and the far-right, we reveal that
users who are popular among the right are only popular among the right, whereas users who are popular
among the left are also popular among all users.
4.5.1 Implications
Despite Twitter’s laudable recent efforts in fighting misinformation and promoting fact-checking (Fowler,
2020), we shed light on the fact that communication is not just falsely manipulated but also hindered by
communication bubbles segregated by partisanship. It is imperative that we not only dispute misinformation but also relay true information to all users. As we have shown, outside information is extremely
difficult to get through to the right-leaning echo chamber, which could present unique challenges for public figures and health officials outside this echo chamber to effectively communicate information. Existing
research suggests that right-leaning users are more susceptible to anti-science narratives, misinformation,
and conspiracy theories (Calvillo et al., 2020; Uscinski et al., 2020; Romer and Jamieson, 2021; Chen et al.,
2021a), which, given the echo chambers they are situated in, can worsen with time. Our work has implications for helping officials develop public health campaigns, encourage safe practices, and combat vaccine
hesitancy effectively for different partisan audiences.
4.5.2 Future Direction
Though the question of whether social media platforms should moderate polarization is debated, we note
that how they can do so remains an open problem. It is unclear how much of the current polarization
is attributed to users’ selective exposure versus the platform’s recommendation algorithm. Moreover,
whether users are even aware that they are in an echo chamber and how many conscious decisions are
being made by the users to combat that remains to be studied in future work.
71

Another future avenue of research could focus on studying how misinformation travels in different
echo chambers. Since our study highlights that there is an alarmingly small number of far-right verified
users, and given that verified users are typically believed to share legitimate and authentic information,
further research is required to establish if the right-leaning echo chamber is at greater risk of being exposed
to false information from unverified users. Detailed content analysis on the tweets can reveal if there
are significant disparities in the narratives shared by left- and right-leaning users. Crucially, our work
provides a basis for more in-depth analyses of how and what kind of misinformation is spread in both
echo chambers.
4.5.3 Limitations
There are several limitations regarding this work. First, we cannot exclude any data bias. The list of
keywords was manually constructed, and the tweets collected are only a sample of all possible Tweets
containing these keywords. Since the data was collected based on keywords strictly related to COVID-19,
we only gathered data that were relevant to the virus and not tainted by political commentary. Therefore,
the data provides us with a natural setting to study the polarization of COVID-19 discourse on Twitter.
Second, our study hinges on the fact that retweets imply endorsement, which may be an oversimplification. To reduce noisy, isolated retweet interactions, we consider only retweets that have occurred at
least twice between any two users.
Finally, our political detection model is built on weakly-supervised labelings of users using politicallyrelevant hashtags and the polarization of news media as the sources of ground truth. We took a conservative approach and only seeded users who explicitly use politicized hashtags in their profile or have
repeatedly interacted with polarized new sources.
72

Chapter 5
Social Approval and Network Homophily as Motivators of Online
Toxicity
5.1 Introduction
The proliferation of hate messages in social media–commonly understood as expressions of hatred, discrimination, or attacks towards individuals or groups based on identity attributes such as race, gender, sex,
religion, ethnicity, citizenship, or nationality (Tsesis, 2002)–has garnered considerable research attention
over recent years (Paz et al., 2020; Ezeibe, 2021; Thomas et al., 2021; Frimer et al., 2023). Online hate has
important connections to cyberbullying (Chen et al., 2012) and online harassment (Thomas et al., 2021)
and, in extreme cases, to the incitement of violence and offline hate crimes (Castaño-Pulgarín et al., 2021;
Ezeibe, 2021; Müller and Schwarz, 2021; Wang et al., 2023). Research finds that young adults, LGBTQ+
minorities, and active social media users are especially vulnerable to online hate and harassment (Keipi
et al., 2016; Thomas et al., 2021). Moreover, evidence suggests the pervasiveness of online hate is growing
increasingly (Mathew et al., 2020; Thomas et al., 2021; Frimer et al., 2023), which emphasizes the urgency
to address problems of online hate and toxicity.
This research provides an initial empirical test of a new theory focusing on the motivations and gratifications associated with posting hate messages online. It posits that online hate is fueled by the social
73

approval that hate message producers receive from others (Walther, 2022). The theory suggests that online
hate behavior is not primarily motivated by the desire to harm prospective victims but rather to accrue
validation and encouragement from like-minded others. Within this framework, one’s propensity to express hateful messages should be related to a similar propensity among one’s social network; people who
share similar resentments and actions should be linked, which can facilitate mutual reinforcement of one
other’s hate behavior. Further, their hate messaging would be expected to become more extreme as they
obtain more reinforcement through social approval signals from others, potentially in the form of likes,
upvotes, or other forms of positive feedback. To test this theory, our research asks the following questions:
• RQ1: Is a user’s hatefulness related to how hateful their social network is?
• RQ2: Does receiving more social approval increase a user’s subsequent hateful behavior?
The contribution of this work is two-fold. First, we investigate whether online hateful behavior conforms to patterns of social network homophily–the idea that people who share similar characteristics,
interests, or behaviors are also frequently associated with one another–is a phenomenon repeatedly observed in many settings (McPherson et al., 2001; Kossinets and Watts, 2009). Earlier research suggests that
the expression of hate is also a homophilous trait (Nagar et al., 2022), which we corroborate in this work.
Second, we show that social approvals are linked to increased toxicity, whereas insufficient approvals are
linked to decreased toxicity. Our analysis suggests that hate speech is “networked”: the expressions of hate,
hostility, or extremism are not just isolated incidents by individual users but are influenced, amplified, and
sustained through the dynamics and structures of social networks. The insights from this research carry
important theoretical implications to advance our understanding of how social gratifications affect the
propagation of online hate and suggest alternative strategies to deter it.
74

5.2 Related Work
5.2.1 Homophily in Toxic Behavior
Social network homophily–the idea that people who share similar characteristics, interests, or behaviors
are also frequently associated with one another–is a phenomenon repeatedly frequently observed in many
settings (McPherson et al., 2001; Kossinets and Watts, 2009). Some suggest that the expression of hate is also
a homophilous trait (Nagar et al., 2022). In a comparison study of hateful and non-hateful Twitter users,
prior work has shown that hateful users have higher network centrality (Ribeiro et al., 2018b). Additionally,
semantic, syntactic, stylometric, and topical similarities exist among hateful users connected in a Twitter
network. Another work on COVID-19 discourse on YouTube highlights that YouTube commentators are
segregated by toxicity levels (Obadimu et al., 2021). Relatedly, a study on moral homophily found that
moral convergence in a cluster of users in an extremist social network predicts how often they spread
hateful messages (Atari et al., 2022). However, these works do not elucidate the extent to which social
network cues facilitate detecting toxic users. Drawing upon notions of social network homophily, we
explore whether we can utilize them to predict user toxicity labels from a smaller, more limited training
set, which would suggest that homophily plays an important role in easily locating toxic users.
5.2.2 Social Motivators of Toxic Behavior
There is a significant gap in research about the influence of social media messages on toxicity. While
several empirical studies report that tweets that are more uncivil, toxic, or otherwise outrageous generate
more signals of social approval in the forms of likes and retweets (Brady et al., 2017; Kim et al., 2021;
Frimer et al., 2023), scant research has examined the opposite: whether the reception of social approval to
one’s toxic messages encourages yet greater incivility and toxicity. This is the core proposition of the social
approval theory of online hate: The audience for an author’s hate messages (that appear nominally focused
75

on some target minority) is like-minded online peers and friends, whose signals of approval reinforce and
encourage more extreme hatred in an author’s subsequent messages. One possible illumination of this
dynamic is Frimer et al. (2023)’s examination of 11 years of tweets by US Congress members. Results
demonstrated increasing incivility among US politicians on Twitter. Moreover, they found that politicians
responded to likes and retweets for uncivil tweets by escalating their toxicity going forward (Frimer et al.,
2023; see also Brady et al., 2017; Shmargad et al., 2022). However, whether these findings generalize to
normal users is unclear since politicians may be more pressured to respond to constituents’ approval than
normal users. The present work helps to bridge this gap by examining the effects of social feedback on a
more representative sample of “average” yet hate-generating Twitter users.
5.3 Data
For this research, we use a Twitter dataset of hateful users. Twitter is a platform where users can share
tweets and potentially receive engagement and feedback from others, for instance, in the form of likes and
replies. We use the Immigration-Hate dataset that is also described in §3.4.6. Our dataset is based on a
dataset collected by (Bianchi et al., 2022), which collected tweets from 2020-2021 referencing US and UK
anti-immigration terms. These tweets are annotated for four sub-types of incivility (profanities, insults,
outrage, and character assassination) and two sub-types of intolerance (discrimination and hostility). In
their dataset, 18,803 tweets were annotated as uncivil, intolerant, or both. Since only the tweet IDs were
provided, we recreated the dataset using our own API credentials in January 2023. We fetched 8,790 tweets
(47% of the original dataset) produced by 7,566 unique users. The rest of the tweets were unavailable for
many possible reasons (e.g., a tweet was deleted, the user set their visibility to private, or the account
was suspended). Using these 7,566 users who are known to have posted hateful tweets, we collected our
dataset of these hateful users’ most recent tweets, up to 3,200 tweets per user (the maximum allowed by
Twitter’s historical API). Of the 21 million tweets we collected, 2.9 million of them are original tweets
76

(i.e., not retweets, quotes, or replies), which is the main focus of this research. Note that due to our data
collection setup, our dataset contains only users who expressed hate at some point during their Twitter
tenure, which appropriately limits the scope of this research yet limits it from generalizing to other users.
For each tweet, our data includes the tweet text and its engagement metrics: like counts, retweet
counts, quote counts, and reply counts. Due to data API limitations, we cannot systematically collect the
identity of users who engaged with a tweet (e.g., which user liked a post), nor can we collect the original
quote or reply messages. We additionally collect each user’s profile description and metadata, including
their account age, verification status, follower count, following count, statuses count (the number of total
tweets they shared), and listed count (the number of public lists of which the user is a member).
5.3.1 Data Preprocessing
As described in §3.4.6, this dataset may be heavily infiltrated by inauthentic bot accounts. Therefore,
for the rest of this study, we replicate all analyses at two thresholds of bot elimination: users with bot
scores <= 0.8, which is a conservative choice given the peak in the bot score distribution (see Figure
3.2), and users with bot scores <= 0.5. We also remove outlier users, those whose follower counts or
engagement metrics exceed three standard deviations from the mean after transformation. We further
remove outlier users with transformed follower counts or engagement metric counts that exceed three
standard deviations from the mean for each level of bot elimination. The follower count is log-transformed
to remove skewness, with 1 added to all counts to avoid 0s when taking the log. The engagement metrics
are also log-transformed and further normalized by the log of the follower count to make them comparable
across users with varying numbers of followers. Following Frimer et al. (2023), we also replace all 0s in
the engagement metrics with 0.1. See Appendix B for the statistics of these metrics.
After eliminating users with bot scores greater than 0.8 and other statistical outliers, 6,665 users, 2.5
million original tweets, and 19 million total tweets remain. After eliminating users with bot scores greater
77

Figure 5.1: Distribution of the hate scores per user (average hate score of their original tweets) and per
original tweet in the hate speech dataset.
than 0.5 and other statistical outliers, 2,985 users, 1 million original tweets, and 8.7 million total tweets
remain. In both cases, between 1-2% of users are verified.
Using all 21 million tweets in our dataset, we compile two social networks: a retweet network and a
mention network. These networks are compiled using all 21 million tweets. In these networks, users are
nodes, and edges represent either a retweet or mention interaction. Mentioning refers to all acts of mentioning (using ‘@’) that are not a retweet, which could include quoting, replying, or otherwise referencing
the user in a tweet. We disambiguate retweets from mentions because retweeting is usually considered a
form of endorsement (Boyd et al., 2010; Metaxas et al., 2015) while mentioning could be used to criticize
publicly (Hemsley et al., 2018). Each edge also comes with an edge weight, which is equal to the frequency
of the retweet or mention between the two users. Details of the networks can be found in Table 5.1.
5.4 Methods
5.4.1 Measuring Toxicity
To detect toxicity, we apply the Perspective API∗ on every original tweet. The Perspective API is a popular
hate detector used in many similar studies (Kim et al., 2021; Frimer et al., 2023). Using its flagship Toxicity
∗
https://perspectiveapi.com/
78

score, we compute a hate score per tweet on a scale of 0 (not toxic) to 1 (very toxic). Additionally, we
compute a single hate score for each user which is the average of the score scores of all their tweets.
Figure 5.1 shows the distributions of the tweet-level and user-level score scores.
5.4.2 Measuring Network Homophily
For the first research objective of determining whether toxic behavior is homophilous, we use the network
assortativity method, commonly used to measure how related the edges of a network are in terms of some
node attribute (Newman, 2003), which, in this case, is the average hate score of a user. It is computed as
the Pearson correlation of the average hate scores of every pair of users connected by a retweet or mention
edges, ranging from −1, which indicates that users are preferentially connected to users with the opposite
toxicity (disassortative) to 1, which indicates that users are preferentially connected to other users with
similar hate scores (assortative). However, network assortativity does not take into account edge weights,
and it neglects to consider that a user’s social network usually consists of multiple connections. Therefore,
we also compute the correlation between each user’s hate score and the weighted average toxicity of all
of their neighbors using the edge weights in the calculation of that weighted average.
5.4.3 Measuring the Effects of Social Engagement on Toxicity
The second research objective aims to understand how others’ social engagement with one’s hate message,
in the form of likes, retweets, replies, and quotes, affects the hatefulness of one’s subsequent posts. An
overly simple solution to this problem could be to binarize the likes, perhaps exploring whether any likes
or receiving more likes than some x amount would impact toxicity. However, this approach is unsuitable
considering that some users in our dataset receive very few likes on their tweets while others are accustomed to receiving hundreds of likes. One potential solution is to scale the social engagement metrics
based on users’ follower counts, which can be a reasonable proxy for their popularity. However, we also
79

need to adjust carefully for the varying impact of that social engagement. For instance, an additional “like”
may carry significantly greater weight for a less popular user than a more popular one.
To address the problems, we categorize tweets into ones receiving “high” or “low” (or neither) amounts
of social engagement by capturing the deviation of actual social engagement from the level of social engagement a user may have expected. We approximate the expected social engagement using the predictive
model described below. We then analyze the change in a user’s toxicity if the user expected much less or
much more than expected social engagement.
To facilitate the computational modeling of users, we use several forms of user representations that
are based mainly on language features. The first is the Social-LLM user representation model introduced
in Chapter 3, which is a social network representation learning method. The model we use is trained
with both retweet and mention edges, as well as users’ profile descriptions and metadata features. SocialLLM begins with an LLM representation of the users’ profile descriptions and continues to minimize the
distance between two users’ representations if the users are connected by a retweet or mention interaction.
This model, as demonstrated in Chapter 3, can be very useful in detecting the hatefulness of users by
combining social network cues with social media content cues. The low-dimensional Social-LLM user
embeddings are useful encapsulations of the user’s social network features. In addition, we also use the
LLM embedding of the user’s profile descriptions (Profile LLM) as well as the LLM of the tweet text (Tweet
LLM). Following Chapter 3, we use SBERT-MPNet (sentence-transformers/allmpnet-base-v2) as
the LLM for all embeddings.
To calculate the expected social engagement, we train a deep neural network to predict likes, retweets,
etc., for every tweet in our dataset based on the following features:
• Social-LLM user embeddings (Chapter 3) containing social network and social media content cues.
• Profile LLM embedding of the user’s profile description
80

• Tweet LLM embedding
• User metadata (follower count, verified, etc.)
• Hate score of the tweet
• Average hate scores of the past 50 tweets
• Average social engagement metrics of the past 50 tweets
• Other social engagement metrics this tweet had that are not the one being predicted (e.g., using
retweets, quotes, and replies to predict likes)
With a model to estimate expected social engagement, we can then calculate the difference between
the actual versus expected amount of social engagement. Since we are interested in when a user receives
substantially less or more engagement than they may expect, we standardize this difference and look at
instances where the z-score of the difference is smaller than −2 (less than expected) or more than 2 (more
than expected). We choose |2| because z-scores less than 2 may represent a fluctuation in social engagement
that would not be perceived by a user as other than a normal deviation within expectations. In contrast,
a higher z-score threshold, such as |3|, would result in statistical outliers and also yield too few instances
for a meaningful comparison.
These tweets that experienced a dramatically unexpected amount of social engagement would be referred to as anchor tweets. We then compare how the toxicity levels of a user change after the anchor
tweet, using varying temporal windows of 30, 50, or 80 tweets before and after the anchor tweet. Here,
we operate under the assumption that an unexpected amount of social engagement may alter a user’s behavior. As our social engagement data is not timestamped, it is possible that a tweet may have received
likes, retweets, etc., after the user has posted the next 30-80 tweets, but we believe this is unlikely. Our
assumption is that the majority of social engagement occurred well before users posted an additional 30-80
tweets.
81

Table 5.1: Hate scores among users exhibit homophily in the hate speech dataset social network, as indicated by both the network assortativity and the Pearson correlation between a user’s hate score and the
weighted average of their neighbors’ (***p < 0.001) .
Network Bot Score # Nodes # Edges Assort. Wgt. Avg. Corr.
Retweet <= 0.8 6,665 74,943 0.071*** 0.310***
Retweet <= 0.5 2,985 9,016 0.195*** 0.445***
Mention <= 0.8 6,665 104,802 0.057*** 0.244***
Mention <= 0.5 2,985 14,182 0.185*** 0.421***
5.5 Results
5.5.1 Homophily in Toxic Behavior
To investigate whether toxicity is a homophilous behavior, we compute network assortativity and weighted
average correlation metrics based on the social network and the user hate scores. Both metrics use the Pearson correlation method. The results shown in Table 5.1 demonstrate that the hate scores exhibit homophily
in both the retweet and mention networks. We isolate users by two bot score thresholds–where the higher
the bot score, the more likely the user is a bot–to eliminate the impact of bots. Both the network assortativity and the weighted average correlation of a user’s hate score and their neighbors are significantly
positive. To ensure the validity of these findings, we perform a robustness check by comparing our results
with those obtained from a null model. In the null model, we randomly shuffle the nodes on one end of
the edges so that each actual edge A ← B is now A ← C, where C is a random node. This procedure
yields nonsignificant Pearson correlations, all with absolute values less than 0.02 in every case, suggesting
that the social network reflects homophily in toxicity. Interestingly, homophily is more evident when we
consider only users with lower bot scores. One possible reason is that users who are more likely to be
genuine users rather than social bots tend to create more homophilous connections (see Appendix B for
further analysis).
82

Figure 5.2: The four types of social engagement on dimensions of rebroadcast and endorsement. Retweets
represent rebroadcast and endorsement, likes represent endorsements, quotes are rebroadcasts that can be
either positive or negative, and replies do not rebroadcast and can be either positive or negative.
5.5.2 The Effect of Social Engagement on Toxicity
Next, we analyze how social engagement may affect a user’s propensities toward toxicity. Before presenting the results, let us review the four types of social engagement signals and what they potentially implicate. On Twitter, one user may engage with another user’s post by liking, retweeting, quoting (retweeting
with additional comments), or replying (commenting). As illustrated in Figure 5.2, we can position each
form of social engagement jointly on the dimensions of rebroadcast and endorsement. By retweeting, one
rebroadcasts a tweet to one’s own followers on Twitter. Retweeting may connote exceptional social approval, as a retweet not only signifies endorsement but the flattering reflection of another user’s desire
for the original message to be seen by their own friends and followers (Boyd et al., 2010; Metaxas et al.,
2015). Liking a tweet also signifies endorsement but not the flattery implied by re-transmission. Quotes
are similar to retweets, except that a quoter adds additional comments of their own that can either support
or disparage the original tweet (Hemsley et al., 2018). Similarly, replies can support or undermine the original tweet, but replies do not rebroadcast the original tweet. Considering these possibilities, we contend
that retweeting conveys the strongest degree of social approval, followed by likes. Quotes and replies can
express social approval or disapproval, but without the textual content, it is impossible to determine which
sentiment it conveys. Of course, an individual can interact with a tweet in multiple ways, for instance, by
83

Table 5.2: The number of anchor tweets and their corresponding number of unique users (bot score <=
0.5) when the engagement metric that the anchor tweet received is substantially lower or higher than
predicted (k = 50) in the hate speech dataset.
Lower Than Predicted Higher Than Predicted
Metric # Ex # Users # Ex # Users
Likes 8,962 741 15,835 1,447
Retweets 16,969 1,062 63,779 2,022
Replies 16,496 642 30,743 1,800
Quotes 2,462 347 52,279 1,883
retweeting and liking, but we cannot determine this from our data. In the absence of engagement metrics
that explicitly signal disapproval (such as downvoting), we attempt to distinguish social approval from
disapproval by how these metrics relate to one another. For example, a tweet with relatively more replies
than likes could indicate that the tweet is perceived negatively by others. Conversely, if a tweet has both
a high number of quotes and retweets, the tweet may be viewed favorably.
To examine the potential impact of social engagement on toxicity, we focus on instances where users
experienced social approvals or disapprovals that significantly deviated from their expectations. To achieve
this, we construct four machine-learning models to predict four distinct social engagement signals per
tweet (see Materials and Methods). For example, in estimating the number of likes a tweet garners, we
incorporate metrics such as retweets, quotes, and replies, along with an extensive set of other features
derived from both user information and text content, aiming to predict the tweet’s social engagement. This
model can help identify tweets where the predicted engagement value substantially differs from the actual
value, suggesting situations where users received unexpected levels of social approval or disapproval.
We refer to these tweets as "anchor" tweets and analyze the average change in toxicity between the k
tweets preceding and following the anchor tweet. The following results presented focus on users with
bot scores less than 0.5. For robustness, we replicate our findings for users with bot scores less than 0.8
in Appendix B to demonstrate the consistency of our results. We display the number of anchor tweets
84

Figure 5.3: Changes in toxicity (y-axis) when an anchor tweet received lower (red bars) or higher (blue
bars) than the predicted amount of social engagement at different windows k (x-axis) in the hate speech
dataset. Changes that are significantly different between the lower- and the higher-than-predicted groups
are indicated (Mann-Whitney U test, ** p < 0.01, *** p < 0.001).
and their corresponding unique users that received significantly higher or lower engagement counts for
k = 50 in Table 5.2.
5.5.2.1 Likes and Retweets Increase Toxicity, but Replies Reduce It
Figure 5.3 illustrates the changes in a user’s toxicity when an anchor tweet experienced a substantially
higher or lower amount of social engagement. Let k = 50 indicate that we compare the past 50 tweets
before the anchor tweet with the 50 tweets following the anchor tweet, we see that anchor tweets that received substantially more likes, more retweets, or fewer replies than expected lead to a significantly greater
increase in toxicity than when the anchor tweet received substantially fewer likes, fewer retweets, or more
replies than expected. Though the net effect is relatively small, we emphasize the statistical significance
of our results. In particular, not receiving enough retweets seems to have a much more dramatic effect on
85

Table 5.3: The average change in toxicity at k = 50 when an anchor tweet received substantially higher
or lower likes-per-quotes or retweets-per-quotes than expected in the hate speech dataset. Statistical significance from a Mann-Whitney U test is indicated (** p < 0.01, *** p < 0.001).
Actual vs. Expected
Relative Metric Lower Higher Sig.
Likes-per-Quotes −0.0007 0.0005 ∗∗
Retweets-per-Quotes −0.0019 0.0005 ∗∗∗
users: users who received substantially fewer retweets than expected generated a larger net decrease in
subsequent toxicity (red bars) than if they received more retweets than expected (blue bars).
The effects of social engagement on toxicity may also reflect different temporal durations. While the
increased toxicity due to retweets replicates for all other values of k, it differs for likes and replies. For
likes, we observe a sustained increase in toxicity when the window is larger k = 50, 80 but not smaller;
for replies, the reduction of toxicity persists for smaller k = 30, 50 but not larger windows. One potential explanation is that giving likes requires little effort, and therefore “likes” connote less potent social
approval compared to more effortful verbalized approval messages in the form of quotes or replies, so the
effect of likes is more gradual and long-term. If a reply is negative, the explicit criticism can immediately
affect a user’s behavior, but the effect may not be long-lasting.
5.5.2.2 Relatively Fewer Quotes Increase Toxicity
It appears that quotes lead to no discernable changes in toxicity (Figure 5.3). However, it could be the case
that they do impact toxic behavior, yet because their impact could be both extremely negative or extremely
positive, the net effect may be canceled out. To test this possibility, we look at the relative number of
quotes compared to the number of likes or retweets, two engagement methods that we are quite certain
to be approvals and not disapprovals. We train a new social engagement model to predict x-per-quote,
where x could be likes, retweets, or replies. We observe that when there are more likes-per-quotes or
retweets-per-quotes, a user displays a significantly greater increase in toxicity (Table 5.3). The effect is
86

Figure 5.4: Having higher amounts of likes and retweets than predicted would result in the biggest increase
in future toxicity in the hate speech dataset, and vice versa (k = 50).
not significant when we compare replies-per-quotes. One possibility is that since likes and retweets are
relatively unambiguous positive social approval, a high number of likes/retweets in conjunction with a
low number of quotes indicates more social approval than disapproval, and vice versa.
5.5.2.3 The Compounded Impact of Both Likes and Retweets
Thus far, we have only considered engagement metrics in isolation. However, a user would presumably
be impacted by all forms of social engagement on their post at once. Therefore, we analyze the combined
effects of likes and retweets (Figure 5.4). When both likes and retweets are greater than expected, the
increase in subsequent toxicity doubles compared to when only one form is greater. The opposite is also
true: fewer likes in conjunction with fewer retweets reduce toxicity by a greater amount.
5.5.2.4 Retweets Escalate Maximum Toxicity
While we have shown that the toxicity levels are raised by social approval, we wonder whether social
approval influences how hateful one is in one’s most hateful tweet. Similar to our previous approach, we
compare the maximum toxicity in the tweets before and after an anchor tweet. Figure 5.5 shows that the
87

Figure 5.5: When an anchor tweet in the hate speech dataset receives substantially lower (red) or higher
(blue) amount of retweets than expected, the difference in maximum toxicity (k = 50) is statistically
significant (Mann-Whitney U test, ∗ ∗ p < 0.01). More retweets lead to an increase in maximum toxicity,
and vice versa
change in maximum toxicity is statistically significant when the anchor tweet experiences substantially
fewer or greater retweets. After an anchor tweet gets more retweets than expected, the user escalates their
maximum toxicity. In contrast, after anchor tweets receive fewer retweets than expected, users decrease
their maximum toxicity. This trend is only observed with retweets and not with likes, replies, or quotes
(which are not significantly different, not shown), suggesting that retweets hold unparalleled power in
altering online hate.
5.6 Discussion
In this chapter, we take a comprehensive look at millions of historical tweets by a set of known, hateful
users on Twitter. We make two meaningful contributions. First, we show that toxicity is homophilous
on social networks. Second, we find that a hateful user’s toxicity level can rise or fall when the user
experiences substantially less or more social engagement in the form of retweets (flattering rebroadcast and
endorsement), likes (endorsement), replies (either endorsement or criticism), and quotes (rebroadcast and
either endorsement or criticism). We find that users’ tweets grow significantly more toxic after they receive
more social approval from other users: more retweets, more likes, fewer replies, and comparatively fewer
88

quotes. Conversely, users’ tweets become less toxic after they receive insufficient social approval or social
engagement that may indicate disapproval rather than approval. In particular, retweets, which signal both
rebroadcast and endorsement, consistently is linked to a profound increase in users’ toxic behavior. These
findings can be extended by analyzing the relation between social engagement signals and higher-level
behavioral cues to explain the incentives and motivations of hateful actors through modeling techniques
such as inverse reinforcement learning (e.g., Luceri et al., 2020).
5.6.1 Implications
Our results support the social approval theory of online hate (Walther, 2022): hateful users could be motivated to appeal to their supporters and respond suitably to positive social reinforcement. It may be hateful
users are not primarily incentivized to harm the nominal target of their disparaging messages but rather
to gain favor from their hateful peers. In addition to advancing our understanding of toxic online behavior, we believe this work has tremendous potential to inform strategies to combat hate speech on not
only Twitter but also other online platforms. For one, this empirical evidence supports efforts to moderate
hateful content (e.g., Meta’s policy; Allan, 2017). Other directions include “shadowbanning”–hiding users’
posts from all but the user; see Jaidka et al. (2023)–or disabling likes, etc., on hateful posts to reduce the
effects of social reinforcement on toxicity.
5.6.2 Limitations
Our research has limitations. Most importantly, while our findings are wholly consistent with the social
approval theory, we recognize that we did not conduct any randomized, controlled experiment to demonstrate causality. Conducting such an experiment on this topic would not be ethical, as it would require us
to reward hateful users for discriminatory messages and thereby encourage online hatred. Associational
research with temporal order provides the best estimation of potentially causal theoretical relationships
89

under the circumstances (Davis, 1985). We also note other limitations due to data availability. Our data
is based on one hate-infused dataset of Twitter posts concerning specifically US/UK immigration (Bianchi
et al., 2022), and our sample may be skewed since many users from the seed dataset are likely to be suspended by Twitter or have deleted their own accounts. Additionally, in the absence of the quote and reply
texts that may either endorse or criticize a tweet, we cannot examine the full impact of social approval or
disapproval signals on hateful behavior. Lastly, our findings regarding the impact of social engagement on
future toxicity reveal small effect sizes. However, despite these shortcomings, the statistical significance
and the consistency of our findings across various robustness tests enhance the validity of our conclusions. Many other influences impinge on the tenor of hate messages, from real-world geopolitical conflicts
to mainstream media stories, not to mention changes in online content moderation policies. Even if the
exchange of social approval is a primary influence, there are undoubtedly many others as well.
5.6.3 Ethical Statement
We recognize that our research could be used by malicious actors to incentivize online hate through positive social reinforcement. We believe the benefits of understanding the social motivators of online hate
production outweigh the risks the research poses. This research was approved by the Institutional Review
Board (IRB).
90

Chapter 6
Moral Values Underpinning COVID-19 Online Communication Patterns
6.1 Introduction
The COVID-19 pandemic not only brought illness and death but also catalyzed a series of important
changes disrupting the lives of nearly everyone across the world. Actions that once constituted ordinary conduct, such as breathing freely without face coverings, became morally laden with accusations
of selfishness and harm (Los Angeles Times, 2020). Work swiftly transitioned from in-person to remote,
changing normative expectations of work (Gramlich, 2022). Research consistently underscores the intertwining of moral judgments with decisions pertaining to health-related behaviors, such as the intricate
linkages between morality and vaccine hesitancy (Reimer et al., 2022; Schmidtke et al., 2022).
In this chapter, we use moral psychology to explain online conversations surrounding COVID-19. The
Moral Foundation Theory (MFT) was created to explain the elements of human moral reasoning. It consists of five main foundations: care/harm, fairness/cheating, loyalty/betrayal, authority/subversion, and purity/degradation (Haidt and Joseph, 2004; Graham et al., 2013). These foundations serve as a moral compass,
shaping how we perceive and engage with the world. As individuals may prioritize these foundations differently, the misalignment of moral orientations between people often leads to disagreements and moral
conflicts (Haidt, 2012). Computational social scientists have also uncovered patterns of moral foundation
homophily in social networks (Dehghani et al., 2016), which could perpetuate a user’s existing beliefs.
91

Moral values have also been found to be linked with political ideology, with liberals and conservatives
preferring different sets of moral values (Haidt and Graham, 2007; Graham et al., 2009). Thus, the polarization of opinions relating to COVID-19 could be related to moral differences (Jiang et al., 2020).
There remains a notable research gap concerning the role of moral foundations in shaping communication dynamics, particularly regarding COVID-19. We know from Chapter 4 that COVID-19 was a
politicized issue on social media leading to communication echo chambers that could undermine public health strategy efforts (see Chapter 4); however, we don’t know to what extent this can be explained
by moral differences. Through the lens of moral psychology, this research aims to explore how moral
orientations influence user groups on Twitter discussing COVID-19, moral homophily in communication
patterns, and the resonance of moral themes across groups. This facilitates a deeper understanding of how
moral considerations influence online discussions during times of crisis. We pose the following research
questions:
• RQ1: What distinct user groups emerge based on MFT and network interactions when discussing
COVID-19 on Twitter?
• RQ2: Do communication patterns reflect moral homophily, and how do they compare across groups?
• RQ3: Which moral themes resonate more effectively with out-group members?
Our summary of contributions is as follows. First, we discover four main groups of users with vastly
different moral priorities and political partisanship. We paint a nuanced picture of the relationship between
morality and political ideology, demonstrating that moral orientations do not rigidly separate users across
the political spectrum. We also find that most user groups exhibit group-based homophily–the tendency
to communicate with in-group members. One group of users (group IV) who are primarily right-leaning
users with fairness and authority moral foundations also tend to only communicate with in-group members, suggesting a moral echo chamber. Finally, we find that messages with moral foundations that are
92

not typically favored by their authors and messages with moral pluralism tend to resonate better with
out-group members. We conclude with insights into user group dynamics and communication patterns,
emphasizing the importance of moral diversity for effective discourse.
6.2 Background and Related Work
6.2.1 The Moral Foundation Theory
The Moral Foundation Theory was proposed to explain variations in human moral reasoning. The original
MFT consists of the following five main foundations (Haidt and Joseph, 2004; Graham et al., 2013):
1. Care/harm: This foundation relates to our ability to feel empathy and compassion for others and our
willingness to alleviate their suffering. It emphasizes the importance of caring for and protecting
others, especially those who are vulnerable.
2. Fairness/cheating: This foundation emphasizes the importance of justice, equality, and fairness in our
interactions with others. It focuses on the belief that everyone should be treated fairly and equally.
3. Authority/subversion: This foundation relates to our respect for authority, hierarchy, and tradition.
It emphasizes the importance of following rules and obeying authority figures.
4. Loyalty/betrayal: This foundation underlies the sense of belonging to a group and the importance
of showing loyalty and allegiance to that group. It emphasizes the importance of being patriotic and
self-sacrificing for the betterment of the group.
5. Purity/degradation:
∗ This foundation relates to our sense of purity and cleanliness and the importance of self-control to avoid impure or degrading actions. It emphasizes the importance of protecting sanctity and avoiding contamination.
∗
‘Purity’ is sometimes referred to as ‘sanctity’ in some literature, but for consistency, we use ‘purity’ throughout this paper.
93

There is also a sixth foundation, liberty/oppression, proposed in Haidt (2012). This foundation relates
to our belief in individual freedom and autonomy, as well as our opposition to oppression and coercion.
However, due to its frequent omission in prior research on MFT text detection (Johnson and Goldwasser,
2018; Rojecki et al., 2021; Guo et al., 2023), there is a lack of a suitable detector for this foundation. Hence,
we do not address this foundation in our current study.
6.2.2 Morality and Politics
Research shows that morality binds and divides users by political orientation (Haidt and Graham, 2007;
Graham et al., 2009). Users with liberal ideology typically reflect higher individualizing moral foundations
of care and fairness, foundations that support the rights and welfare of individuals, whereas conservative
ideology typically endorses all five moral foundations equally, which includes the binding moral foundations of authority, loyalty, and purity. Koleva et al. (2012) further exemplified this by examining 20
politically salient issues, such as abortion and immigration. They found that the five moral foundations, in
particular purity, are better predictors of issue-specific opinion above ideology and demographic features.
This difference in moral attitude has been attributed to growing political polarization, where users on both
ends of the political spectrum are unable to resonate with each other due to moral incongruence (Haidt
and Graham, 2007) and overexaggerate their differences (Graham et al., 2012). Techniques of moral-based
reframing have thus been proposed as a way to bridge the political divide (Feinberg and Willer, 2019).
6.2.3 Moral Homophily
Moral values may also explain the connection and formation of communities through social network homophily, the phenomenon that we tend to be drawn to people we are similar to (McPherson et al., 2001;
Kossinets and Watts, 2009). In a large-scale analysis of social network data on the US government shutdown, moral purity can predict network ties, indicating purity homophily (Dehghani et al., 2016). Another
94

study comparing multilingual tweets explicitly mentioning morality in English and Japanese found that the
care, authority, and purity foundations are homophilous in English tweets, while the loyalty, authority and
purity foundations are homophilous in Japanese tweets (Singh et al., 2021). We theorize that for COVID-19,
some moral foundations may exhibit network homophily that informs our moral-based understanding of
online conversations.
6.2.4 Morality and COVID-19
Many divisive behaviors and controversial opinions regarding COVID-19 may be rooted in moral differences. Those with a higher care disposition are found to be more likely to follow health recommendations
(Díaz and Cova, 2022). A similar study by Chan (2021) also found that higher care and fairness predicts
compliance with the health strategies of staying at home, wearing masks, and social distancing, while
purity predicts non-compliance with wearing masks and social distancing. In the face of disease threats,
Ekici et al. (2021) found that fairness, care, and purity were the most important moral foundations that predicted people’s acceptability of moral transgressions. In a study of tweets on the COVID-19 mask mandate,
Mejova et al. (2023) found that authority and purity were associated with anti-masking sentiment while
fairness and loyalty were associated with pro-masking sentiment; there is also a de-emphasis on the care
foundation following the mask mandate.
By 2021, arguments loaded with moral judgments both for and against the COVID-19 vaccination
take center stage. In a study based in Great Britain, Schmidtke et al. (2022) found that vaccine hesitancy is
associated with higher moral needs of authority, liberty, and purity and less need ofcare. Moral foundations
were also shown to be good predictors of country-level vaccination rates in the US: purity predicted lower
vaccination rates, whereas fairness and loyalty predicted higher vaccination rates (Reimer et al., 2022).
Analyzing the sentiment in vaccine-related tweets revealed that pro-vaccination tweets carried more care
morals, while anti-vaccination tweets carried more liberty morals (Pacheco et al., 2022). On Facebook,
95

Figure 6.1: Distribution of raw user moral scores in the COVID-19 dataset.
pro-vaccination users identify more with authority, and anti-vaccination users identify more with liberty
(Beiró et al., 2023). The debate on vaccination is also linked to partisanship. Liberals discuss COVID
vaccination on Twitter with more emphasis on the moral virtues of care, fairness, liberty, and authority,
whereas conservatives leaned into the vices of oppression and harm (Borghouts et al., 2023).
Besides being moralized, COVID-19 was also politicized, which, as we previously analyzed in Chapter
4, may have led to the formation of online political echo chambers. Echo chambers are harmful to online ecosystems as segregated communication can lead to radicalism and extremism (O’Hara and Stevens,
2015). One research suggests that moral differences can explain partisan differences in vaccine hesitancy
(Bruchmann and LaPierre, 2022). Conservatives and liberals also differ in how they use morality in their
language. Conservatives are more likely to use moral vices rather than virtues in their tweets (Borghouts
et al., 2023; Rao et al., 2023). That said, there is promising research on how moral-based reframing of messages can be used to advocate mask-wearing among liberals and conservatives (Kaplan et al., 2023; Luttrell
and Trentadue, 2023).
96

6.3 Data
For this research, we also use the large dataset of real-time COVID-19 tweets collected by Chen et al.
(2020) that we used previously in Chapter 4. However, unlike the previous version of the dataset, we
now use a much longer subset of the data spanning the beginning of February 2020 to the end of October
2021 for 21 full months. We then take a longitudinal set of active users who tweeted at least 10 tweets
containing moral values in any given month, which means they each contain at least 10 data points of moral
foundation values. We describe how we detect moral values in the Methods section below. This procedure
resulted in a large dataset of 2 million users and 253 million tweets. To analyze communication patterns,
we also create the retweet and mention networks. Using users as nodes, we build the retweet network by
connecting users who retweet one another. Similarly, we build the mention network by connecting users
who do not retweet but rather quote (retweet with additional comment) or mention (‘@’) another user.
We draw the distinction between the retweet and mention network as there may be different underlying
motivations for each action. Retweeting usually implies endorsement (Boyd et al., 2010; Metaxas et al.,
2015) while mentioning could be used to endorse or criticize (Hemsley et al., 2018). When identifying
retweets or mentions between two target users, we use all available tweets, including those that do not
have any identifiable moral foundations. The edges are weighted by the number of times one user retweets
or mentions another. To allow for efficient computation, we create separate networks for each month of
our dataset and aggregate the results if necessary. On average, we have 90,000 nodes, 1 million retweet
edges, and 1.3 mention edges in our monthly networks.
A sample of 150,000 users and their respective edges from this dataset is called Covid-Morality, used
in the evaluation of Social-LLM in §3.4.3. We also use this subset in the methodology section below to for
user cluster detection.
97

Table 6.1: Hypothetical tweets, adapted from real ones in the COVID-19 dataset, that contain detected
moral foundations values. Some tweets only contain the virtue or the vice of a foundation, and some
tweets contain both.
Tweet Morals
The vaccine won’t harm you if you are in one of the approved groups, but it will protect you. Care, Harm
The choice to not wear a mask is my right. The choice to stay at home is yours. Fairness, Authority
New Zealand just announced it will provide the new Covid vaccine to any New Zealander who wants Fairness, Authority it–free of charge. They’re also making the vaccine available to all their Pacific Island neighbors.
It’s up to us to slow the spread, save lives, & keep businesses open. We have to work together. Care, Loyalty
I refuse to take this vaccination! It goes against my religious beliefs! Purity
Revolting. The fact that XXX got the vaccine before healthcare workers and first responders. Degradation
6.4 Methods
To address our research goals, we employ several computational methods. First, we use a moral foundation
model to detect the moral foundations present in each tweet and aggregate them to obtain overall moral
orientation scores for each user. Next, using user content and moral features as well as the social network
as input data, we apply Social-LLM (Chapter 3) to learn latent vector representations of the users based
on their moral foundations and communication patterns, which are then used to cluster users into distinct
groups. Finally, to analyze moral theme resonance across ideological lines, we apply a classifier to predict
users’ political leanings.
6.4.1 Detecting the Morality of Tweets and Users
We detect tweet morality using an MFT model fine-tuned specifically on this dataset (Guo et al., 2023). This
model adopts a data fusion technique to account for the fundamental shifts in morality based on the topics
of the dataset. It is based on a pre-trained BERT model (Devlin et al., 2019) that was fine-tuned on three
different Twitter datasets annotated for morality, including one that was specifically on COVID (Rojecki
et al., 2021). See Guo et al. (2023) for a detailed evaluation of the model. This MFT detector predicts the
presence of the 10 moral values—each foundation contains two opposite moral values for the virtue and
98

the vice—in a multi-label manner for every tweet, with 1 indicating the presence of a moral value and 0
indicating the absence of it. We present some example tweets with moral foundation labels in Table 6.1.
We then aggregate the virtues and vices of each foundation into one label. If a tweet has a score of 1 for the
care dimension but a score of 0 for the harm dimension, then it has a combined score of 0.5 in the care/harm
foundation. We choose to do this because the morality detector looks for explicit expressions of the virtue
(e.g., care) and the vice (e.g., harm) separately, but for our purposes, both reflect a moral disposition in that
foundation. The vast majority (87%) of tweets contain only a single moral foundation. 13% of the tweets
contain two moral foundations, of which the most popular combination is care and authority. Less than
1% of tweets have 3 or 4 moral foundations, and no tweets have all five foundations.
We also compute morality scores at the user level, which is the mean moral score of each five foundations across all the tweets by one user. The distribution of the aggregated user morality scores is shown
in Figure 6.1. The care foundation is the most frequently utilized foundation by a wide margin, and purity
is the least utilized foundation. To maintain cross-comparison of each morality, we standardize (z-scores)
the scores in each foundation to have a mean score of 0 and a standard deviation of 1.
6.4.2 Detecting User Groups Using MFT and Twitter Activity
In pursuit of our research goal–understanding the communication dynamics among users with various
moralities–we want to find salient groups of users in our dataset based on their moral foundations and
their communication preferences. For this task, we leverage Social-LLM, an unsupervised social network
user representation method that combines user content features and social network features. It works
by initializing user embeddings from their text and metadata features and optimizing them such that two
users who share a network connection would have similar embeddings. Social-LLM is substantially easier
to train than its graph neural network counterparts since it does not need (sub)graphs as inputs but rather
99

only the pairs of edges. Despite its simplicity, we have shown that it works comparably to many stateof-the-art baselines, including various fine-tuned LLMs or complex graph neural network methods, in its
ability to recover user moral foundation scores. See Chapter 3 for a detailed description of the model and
evaluations on this dataset. Given the experiment results from this dataset, we use the best hyperparameter configuration to train a Social-LLM model, which learns 128-dimensional embeddings using directed
retweet and mention edges, as well as a base language model of SBERT-MPNet (Reimers and Gurevych,
2019; Song et al., 2020).
In addition to profile descriptions and user metadata features, we also include the user moral foundation
scores in this version of the model so that the user representations include moral cues. Then, we apply
this trained model to our full dataset to obtain user embeddings for our 2 million users.
These learned user embeddings contain important information about users’ moral foundation values
and Twitter activities (including social network interactions, profile descriptions, and other user metadata
features). We then apply the k-means algorithm to the embeddings to uncover distinct user groups. After
experimenting with cluster numbers ranging from 2 to 10 by using the elbow methods on the inertia and
the silhouette scores, we select k = 4 as the most appropriate number of clusters. These four distinct user
groups, encapsulating the distinct moral orientations and Twitter behaviors within our dataset, warrant
further exploration and analysis.
6.4.3 Detecting User Partisanship
As morality is often linked to differences in political partisanship (Graham et al., 2009; Koleva et al., 2012),
and because numerous studies on COVID have shown that user opinions are politically divided (Jiang
et al., 2020; Rao et al., 2023), we also want to capture users’ political partisanship in this study. To this end,
we utilize another Social-LLM model trained for user political leaning detection. Compared to other political leaning detection methods on Twitter, Social-LLM has the advantage of learning crucial cues from not
100

Figure 6.2: The average moral z-scores of each foundation for the four user groups in the COVID-19 dataset.
only the textual features of the tweets but also social network interaction features. The latter is particularly
preferable since Twitter users are often politically segregated, especially on the topic of COVID-19 (Chapter 4). Applying the Social-LLM model for political leaning detection on COVID-19 datasets, we obtain
political-leaning labels for every user in our dataset. This particular model is also the one documented extensively in the Retweet-BERT chapter (Chapter 2). Similar to prior work on political polarization analysis
in Chapter 4, we bin the scores into quintiles to adjust for the left bias. Users falling within the 0-20% range
are labeled as very left-leaning, those in the 20-40% range as left-leaning, the middle 40-60% as moderate,
the 60-80% as right-leaning, and those in the 80-100% range as very right-leaning.
6.5 RQ1: User Groups by Morality
We first answer the research question: What are the characteristics of the groups of users tweeting about
COVID-19 based on their moral values, Twitter profiles, and network interactions? We show the average
moral scores of each group in Figure 6.2, a political partisanship breakdown in Figure 6.3, and some key
user metadata statistics in Figure 6.4. Below, we discuss the characteristics of each group.
101

Figure 6.3: Partisanship breakdown of the four user groups in the COVID-19 dataset. Blue bars represent
left-leaning users, and red bars represent right-leaning users.
Group I: Authority & Fairness. This group contains 493, 000 users (23%) who reflect a relatively
stronger morality in the authority (z = 0.15) and fairness foundations (z = 0.07). Its scores in the care
foundation are below average (z = −0.14). Considering user partisanship, we find that this group is predominantly occupied by left-leaning users (87% left-leaning), but very few are extremely left-leaning. The
metadata features of group I users stand out in several important ways. They have the fewest proportion
of verified users. On average, they have the least number of followers, followings, posts, and proportion
of tweets that are original. In sum, group I users appear to be the least active user group and most likely
do not have as many influential or popular users as the other groups. As Twitter users reflect an overall
left bias (Chapter 4), we theorize that group I users are average users with low influence.
Group II: Authority & Care. With 800,000, or 37% of users, this group is the largest user group in our
dataset. Its users are characterized by higher moral scores in the authority (z = 0.09) and care foundations
(z = 0.05). It scores comparatively lower in the purity foundation (z = −0.09). In terms of partisanship,
102

Figure 6.4: Distribution of the user metadata features for the four user groups in the COVID-19 dataset.
this group has a good balance of left- and right-leaning users, although it has a substantial amount of
far-left users (38%),
Group III: Care & Purity. The third group contains 603,000 (28%) users and exhibits stronger than
average morality in the care (z = 0.30) and purity (z = 0.19) foundations, along with a weaker than
average morality in the authority (z = −0.42) and fairness (z = −0.30) foundations. It has a balanced
representation of users across all political leanings, comprising 48% users on the right and 44% users on
the left.
Group IV: Fairness & Authority. The fourth and final group is the smallest user group, made up of only
274 (13%) users. This group is characterized by stronger than average foundations of fairness (z = 0.64)
and authority (z = 0.37), coupled with weaker than average foundations of care (z = −0.56) and purity
(z = −0.13). This group is also made up of predominantly right-leaning users (95%). Users who are
far-right occupy 84% of this group alone.
We note that the characteristics of these groups are discussed in relation to each other, not in absolute
terms. That is, for example, Group I users display lower care moral values than Group III users, but that
does not mean Group I users don’t utilize care morality. Of the four user groups, we see that Groups III
and IV users have very strong preferences in some moral foundations, whereas Group I and Group II have
more modest preferences. Further, We note that Group III and Group IV users have almost polar opposite
103

Figure 6.5: TSNE visualization of 100,000 sampled Social-LLM user embeddings from the COVID-19 dataset
of the four user groups. These Social-LLM embeddings are learned from the users’ network cues, content
cues, and moral leanings.
moral foundation inclinations. While Group III prefers care and purity, these are exactly the two moral
foundations that are least utilized by Group IV, and vice versa for Group IV’s preferred morality of fairness
and authority. Additionally, though both Group I and Group IV prefer the foundations authority and
fairness, they differ considerably in what foundations they don’t prefer, the strength of their morality, and
their political partisanship breakdown. Finally, while most of the five moral foundations are prominently
favored or unfavored by at least one of the four user groups, the loyalty foundation usage is used almost
indistinguishable among the user groups.
6.5.1 User Visualization
In Figure 6.5, we present a TSNE (Van der Maaten and Hinton, 2008) visualization of 100,000 sampled user
embeddings in each group. TSNE is a popular dimension-reduction technique of high-dimensional data
points to reveal structural proximity among users in each group. This plot shows a good separation of
every user group, with user groups II, III, and IV forming visible clusters. This may indicate that these
104

groups form homophilous communication bubbles. However, Group I users form a circular ring enclosing
other user embeddings. As we will see in the next section, this appears to be because users in this group
do not preferentially communicate with in-group members but rather interact equally with all users.
6.6 RQ2: Moral Homophily
In this section, we continue our analysis by examining whether there is a communication homophily
within in-group members, which may lead to communication echo chambers among people sharing similar
moral profiles.
6.6.1 Homophily of Users
As a preliminary analysis, we examine whether there is moral homophily in individual moral foundation
values at the user level. That is, do users share moral foundation values similar to those of their network?
We use network assortativity scores (Newman, 2003) to measure how (dis)similar two sets of nodes’ attributes are, given that each pair of nodes is connected by an edge. Using the standardized moral scores
as the users’ node attributes and the retweet or mention interaction as edges, we compute the network
assortativity values of every moral foundation on monthly subgraphs.
Figure 6.6 shows the results from the retweet networks, which are largely similar to the mention network. All of the Pearson correlation coefficients are positive and significant (all p < 0.001), indicating
moral assortative mixing or moral homophily. In particular, the care (µ = 0.43), fairness (µ = 0.40),
and authority (µ = 0.38) foundations indicate strong, consistent moral homophily. However, loyalty
(µ = 0.22) reflects much lower assortativity. Intriguingly, while most foundations reflect consistent levels
of homophily over time, the purity foundation (µ = 0.29) showcases fluctuations, at times recording both
the lowest and highest homophily values. This variance can be attributed to increased discussions related
to notable events surrounding masking (April 2020) and vaccines (early 2021), topics related to the purity
105

Figure 6.6: Retweet network assortativity of users’ moral scores over time in the COVID-19 dataset. High
assortativity indicates homophily.
foundation (Chan, 2021; Reimer et al., 2022). Notably, peaks in purity assortativity align with significant
events, such as the CDC’s official recommendation of face coverings in April 2020 and the widespread
discussion of vaccines during the first half of 2021. While these findings align partially with prior research
on purity homophily (Dehghani et al., 2016), we emphasize the consistent and strong homophily observed
in other moral foundations, particularly care, fairness, and authority.
6.6.2 Homophily of User Groups
Next, we evaluate whether there is homophily at the moral group level: do users preferentially communicate with in-group members, and if so, how do they cross-compare? Given that user group membership
is determined by their Social-LLM embeddings, which are learned from social network connections, it is
reasonable to expect group homophily. This might initially render the research question ill-posed. However, the importance of this section lies in comparing the degree of moral homophily among user groups
to illuminate differences in group-based communication strategies.
106

Figure 6.7: The ratio of how often COVID-19 dataset users in group X retweet users in group Y divided
by the null baseline amount. > 1 (red) cells indicate X is more likely to retweet from Y than the null
baseline, and < 1 (blue) cells indicate the opposite.
We cannot use assortativity to measure group homophily because group membership is categorical,
not numerical. However, we can empirically compute how often users from group X will retweet or
mention users from group Y compared to a null model. Let P(X
rt
←− Y ) be the actual proportion of
tweets published by user group Y that are retweeted by user group X out of all the retweets by user group
X. We then randomly re-assign the group identity of the users to compute the null model Prand(X
rt
←− Y ).
This procedure controls for the fact that the moral groups are not even in size, so some user groups don’t
appear to receive more communication only because they have more users. We then examine P/Prand,
which would be > 1 if X communicates with Y more frequently than the null baseline and < 1 if X
communicates less frequently with Y .
The results for the retweet network are shown in Figure 6.7. We omit the results from the mention
network, which are very similar. Users in group IV (right-leaning users with fairness and authority morals)
display the strongest homophily, preferentially retweeting from users in the same moral group more than
3x as much as the null baseline. This trend is followed by users in group III (politically balanced users with
107

Figure 6.8: The ratio of how often COVID-19 dataset users in group X communicate with in-group users
(left) or out-group users (right) divided by the null baseline amount. > 1 indicates X is more likely to
retweet from Y than the null baseline.
care and purity morals) and II (politically balanced users with authority and care morals) but to a lesser
extent. Users in groups II, III, and IV also retweet from the other three groups much less frequently than
the null baseline. Interestingly, group I (left-leaning users with authority and fairness morals) users do
not display preferential communication with in-group members. In fact, they retweeted themselves very
infrequently and retweeted from other groups (II and IV) more than expected. As we have seen in Figure
6.5, this could indicate that group I users are not characterized by strong homophilous social network ties
but rather serve as peripheral members of the Twittersphere interacting with all users.
Figure 6.8 compares in-group communication P/Prand(X ← X) with out-group communication
P/Prand(X ← X′
), where X′
include all the users not in X. We observe that group IV users have much
higher in-group communication compared to the null baseline, followed by groups III and II, whereas
group I does not favor in-group communication. However, we also see that groups II and III, the politically balanced user groups, have higher out-group communication compared to the null baseline. This is
not true for group IV, which clearly prefers in-group communication and not out-group communication.
This finding may signal that group IV, who are predominantly right-leaning extremists, is falling into a
communication echo chamber, a potentially harmful manifestation of online communication. Our findings
108

Table 6.2: For every user group, we show the top five moral combinations used in messages that are
retweeted more often by out-group members than in-group members. The list is sorted by the ratio
C(X, X′
, m)/C(X, X, m). Highlighted moral foundations are the ones that the user group does not favor
(cf. Figure 6.2).
Moral Foundations Ratio
I: Authority & Fairness
Care, Purity 2.316
Fairness 2.010
Care, Authority, Purity 1.895
Care, Loyalty 1.558
Care, Fairness, Loyalty 1.293
II: Authority & Care
Care, Loyalty, Purity 1.299
Fairness, Loyalty 1.151
Care, Fairness, Loyalty 1.137
Authority, Loyalty, Purity 1.114
Authority, Loyalty 1.099
Moral Foundations Ratio
III: Care & Purity
Fairness, Authority, Purity5.049
Care, Authority, Purity 3.098
Care, Purity 2.059
Care 1.566
Fairness, Authority 1.177
IV: Fairness & Authority
Care, Loyalty 4.053
Care, Authority, Loyalty 2.303
Care, Authority, Purity 1.822
Care, Loyalty 1.614
Authority 1.158
align with our prior research considering only user political orientation in Chapter 4), which showed that
right-leaning users discussing COVID on Twitter are situated in a tight-knit political echo chamber.
6.7 RQ3: Bridging Moral Divides
Previously, we found that users of certain moral typologies communicate frequently with in-group members (groups II, III, and IV; group IV also displays a lack of communication with out-group members. In
this section, we consider whether there is any pattern in the moralities or the combination of moralities
in messages that tend to travel across groups. Let m be the 5-dimensional multilabel indicator vector of
the moral foundations present in a tweet. We then count the frequency of the moral foundation combinations of every tweet published by user group X that was retweeted or mentioned by a user that is
not from group X, denoted by C(X, X′
, m). To draw a suitable comparison, we also count the moral
foundation combinations’ occurrences of retweets or mentions within a group, denoted by C(X, X, m).
The ratio C(X, X′
, m)/C(X, X, m) thus reflects the moral foundation combinations that were retweeted
more frequently by in-group members than out-group members. Sorting by this ratio, we identify the top
109

five moral foundation combinations that were retweeted by out-group members for every group in Table
6.2, highlighting the moral foundations that are not the preferred moral foundation by that specific user
group.
Key insights emerged from this analysis. First, messages with moral foundations that were less favored
by the user group were retweeted much more frequently by out-group members. Second, combinations
featuring multiple moral foundations garnered heightened traction among out-group members, especially
since tweets with multiple moral foundations are extremely rare: tweets with two moral foundations only
make up 13% of all tweets, and tweets with three or more moral foundations make up less than 1% of all
tweets. The implications are profound, suggesting avenues of facilitators of social network connections by
enhancing one’s moral diversity and moral plurality. We also offer support for research in moral reframing
as a tool to bridge ideological gulfs (Feinberg and Willer, 2019).
However, it’s important to acknowledge the limitations of this analysis. While we shed light on existing
communication patterns, we cannot definitively causal relationships. There might be untapped moral
combinations that could potentially resonate well with out-group members but remain unexplored within
our dataset. Nonetheless, these findings offer valuable insights into understanding the dynamics of moral
communication using observable data.
6.8 Discussion
In this paper, we present a large-scale empirical investigation of COVID-19 online conversation through
the lens of moral psychology. Using a large dataset of COVID-19 tweets spanning nearly two years, we offer
insights into the characteristics of the main types of users from a moral standpoint (RQ1), communication
patterns among users of different moral typologies (RQ2), and how the morality of messages leads to more
effective diverse communication (RQ3).
110

Based on users’ moral foundation profiles, user metadata features, and social network data, we use
social network representational learning to help uncover four distinct groups of users that differ in moral
preferences, strength, and political affiliations. Group I users represent low-influence, left-leaning users
who care mainly about authority and fairness, positioning themselves mostly at the peripherals of social
network communication. Group II users is a politically balanced group who care about authority and
care. The biggest contrast occurs between group III, who highly value care and purity, and group IV, who
highly value fairness and authority; they are diametrically opposed in that each group values exactly the
moral foundations that the other group does not. The two groups also differ considerably in their political
partisanship composition: group III users are politically balanced, while group IV users are predominantly
far-right.
Analyzing communication patterns based on morality, our results illustrate patterns of moral homophily. We consistently find high homophily in the care, fairness, and authority foundations. We also
partially align our results with findings on purity homophily (Dehghani et al., 2016), although we observe
purity homophily only at specific time points that may relate to real-world events. We also find group
homophily in groups II, III, and IV since they display preferential communication with their respective
groups. Notably, group IV also demonstrates a substantial lack of communication with out-group members, hinting at the potential existence of harmful moral echo chambers. These findings provide valuable
insights into the dynamics of moral homophily, emphasizing its presence across different moral foundations and its potential impact on group communication dynamics.
Finally, upon analyzing the moral foundation of messages that could bridge moral divides, we find that
moral diversity and moral pluralism may be useful approaches. Messages that contain moral foundations
their users don’t usually prefer, and messages that contain multiple moral foundations tend to be more
likely retweeted by out-group members.
111

6.8.1 Implications
The findings of our study offer insights and recommendations for practitioners, health agencies, and researchers. We emphasize significant communication variations among users of various moral preferences
regarding COVID-19. Importantly, we identify a group of users (IV), characterized by a preference for the
fairness and authority foundations and a right-leaning political orientation, as potentially more susceptible to morally homogeneous messages. Adding to the wealth of literature on moral ideals separating the
political left and right (Haidt and Graham, 2007; Graham et al., 2009), we observe that the moral lines are
not always so clear cut on this topic; users on both sides of the political spectrum can prioritize the same
moralities that are both individualizing (care or fairness) and binding (authority or purity). A deeper dive
into the complexity of our moral differences, in addition to political ideology factors, can lead to a more
comprehensive understanding of online communication. Finally, our research sheds light on the potential
usefulness of rhetorical tools focused on moral reframing to enhance the diversity of communication on
online platforms.
6.8.2 Limitations
The research conducted in this study is strictly observational, and no causal relationships can be implied
from the findings. The validity of the results is contingent upon the accuracy of various machine learning
models utilized in the study, including those for morality detection and political partisanship detection.
Additionally, it’s essential to note that the scope of the results is limited to the topic of COVID, and generalizability to other topics may not be assured. These methodological considerations and limitations should
be taken into account when interpreting and applying the study’s results.
112

6.8.3 Ethical Statement
The data used in our study is publicly accessible (Chen et al., 2020). Our study was exempt from review
by the Institutional Review Board (IRB) as it solely relied on publicly available data. During our analysis,
we protect user privacy by utilizing user IDs instead of screen names. Further, in the interest of user
confidentiality, we present only aggregated statistics in this chapter. In conclusion, we put forth ethical
recommendations, proposing the integration of moral psychology in comparable research endeavors and
the development of rhetorical tools specifically designed for moral reframing. This initiative aims to enrich
the diversity of communication on online platforms. Acknowledging a potential risk, we recognize the
possibility of malicious actors manipulating public opinion by manipulating moral foundations. However,
we contend that the positive outcomes and contributions of our research far outweigh this risk. The authors
declare no competing interests.
113

Chapter 7
Conclusion
Social media provides an online space where we can connect digitally with people everywhere in the world
and spread content almost instantaneously. Despite the benefits of faster communication, social media
also presents significant challenges, such as the spread of misinformation, online hate speech, privacy
concerns, and political polarization. Through this dissertation, I identify problems on social media and
offer data-driven insights by leveraging computational social science methodologies and the concept of
network homophily.
To address the technical challenge of modeling large and sparse real-world social network data, I propose a novel social network representation learning method called Social-LLM in Part I. Social-LLM integrates user content with social connections to create unified representations, enabling accurate prediction
and insightful visualizations of user attributes, communities, and behavioral propensities. The scalability,
inductiveness, and task-agnostic nature of Social-LLM render it an effective tool for modeling extensive
and complex social network data, thereby addressing a broad spectrum of open-ended research questions.
I conduct a comprehensive evaluation of this method across seven real-world social network datasets,
covering a wide variety of topics and detection tasks. This demonstrates Social-LLM’s versatility and its
potential to propel research in computational social science forward.
114

Part II of this dissertation explores the complex dynamics of online communication and behavior, particularly in the context of pressing social and political issues. I shed light on the ways in which social
media platforms can amplify and reinforce certain behaviors, ideologies, and polarization.
In Chapter 4, I investigate the politicization of COVID-19 discourse on Twitter, revealing the existence
of echo chambers, particularly within the right-leaning community. This research highlights the relationship between information dissemination and political preferences, emphasizing the need for effective
public health communication strategies.
Chapter 5 delves into the role of social approval in driving online hate messaging, suggesting that the
pursuit of social approval, rather than a direct desire to harm, may be the primary motivator for users
engaging in toxic behavior. The study establishes a connection between receiving social approval signals
and increases in subsequent toxicity, with retweeting playing a particularly prominent role in escalating
toxicity.
In Chapter 6, I examine the moral underpinnings of online discussions surrounding COVID-19, identifying distinct user groups characterized by differences in morality, political ideology, and communication
styles. This study uncovers patterns of moral homophily and the existence of a potential moral echo chamber while also highlighting the effectiveness of messages that incorporate diverse and multitude of moral
foundations in resonating with out-group members.
The unified theme connecting these studies is the exploration of how social media platforms shape
and amplify user behavior, ideologies, and polarization. Each study focuses on a specific aspect of online
communication–political discourse, moral foundations, or hate messaging–but they all contribute to a
broader understanding of the complex social dynamics at play within online environments. Collectively,
these studies serve the purpose of providing valuable insights into the mechanisms that contribute to
unhealthy and unproductive communication online.
115

7.1 Limitations
Certainly, this research is not without its constraints. A notable limitation is the data. The exclusive
use of a Twitter dataset might not encapsulate the entirety of social media users, presenting a significant
limitation in the generalizability of my research. For instance, US Twitter users are more likely to be
younger and more liberal-leaning than the general US population (Wojcik and Hughes, 2019). However,
given the inaccessibility of other datasets for academic researchers, and the popularity of Twitter as one of
the most utilized social media platforms, this research still holds significant merit. Additionally, my focus is
narrowly placed on retweets and mentions, sidelining potentially insightful social network dynamics like
follower relationships or other forms of social connections due to API rate limit challenges. Despite these
challenges, Twitter remains a valuable data source that offers a glimpse into large-scale human interactions
and user relationship dynamics.
Another considerable limitation lies in the scope of interpretation of the findings. The observational nature of the data, coupled with the limited information on each user, makes it difficult to establish any causal
relationships. Nevertheless, the insights garnered from this dissertation are of substantial importance and
lay the groundwork for future studies to further examine and validate these observations. Moreover, given
the ethical concerns surrounding certain behaviors, such as hate speech, observational studies often represent the most ethical approach. Deliberately exposing individuals to harmful content for research purposes
would be highly unethical, thus reinforcing the value and necessity of observational research despite its
limitations.
116

7.2 Broader Impact
More broadly, my research offers insights into the intricate relationship among social media, user engagement, and societal challenges. These findings have significant implications for a wide array of stakeholders,
including academic researchers, social media platforms, lawmakers, and the general public.
Academics have the opportunity to use this knowledge to address the adverse impacts of online divisiveness, echo chambers, and harmful behaviors. By uncovering the root causes of these issues, scholars
can pave the way for more effective approaches to foster healthier digital interactions and more enlightened societal conversations.
Social media platforms can use these insights to refine their operational policies, content moderation practices, and overall platform architecture. Gaining a deeper understanding of how their platforms
influence user actions and propagate certain viewpoints enables these firms to proactively create a more
welcoming and positive digital space. This might involve the development of machine learning algorithms,
guardrails, or moderation techniques that promote a diversity of viewpoints, diminish the dissemination
of false information, and prevent harmful conduct.
Lawmakers can leverage this research to craft regulations and laws that address the complexities of
today’s digital environment. As social media continues to mold public sentiment and dialogue, it’s vital
for legislative bodies to grasp the social mechanics within these platforms. The knowledge garnered from
this research offers a solid base for formulating policy decisions that uphold transparency, responsibility,
and the safeguarding of online user welfare.
The general public also stands to gain from the knowledge disseminated through this research. Highlighting the hazards associated with digital communication, such as informational echo chambers, societal
polarization, and the proliferation of hostile behavior, I highlight the need to educate the general public
on digital literacy. We should be considerate of our digital footprints and our digital consumption.
117

Through this dissertation, I attempt to decipher human behavior and social interactions in our rapidly
digitizing world. I emphasize the pivotal influence of social networks in shaping both personal and collective actions in online communication. These insights are instrumental in guiding research and initiatives
across diverse disciplines, such as public health, political science, and social psychology, illuminating future research into the complexity of human behavior in the digital age. While my focus here is on social
media applications, it’s essential to recognize that the significance of this work goes beyond social media
alone. It extends into the broader realm of computational social science, aiming to harness computational
methods to address the complex challenges of the digital era. Although social media is a central focus, the
methodologies and insights developed in this research hold applicability across any digital domain where
human interaction intersects with digital platforms.
118

References
Amjad Abu-Jbara, Ben King, Mona Diab, and Dragomir Radev. 2013. Identifying opinion subgroups in
Arabic online discussions. In Proceedings of the 51st Annual Meeting of the Association for Computational
Linguistics (Volume 2: Short Papers). Association for Computational Linguistics, 829–835. https://aclant
hology.org/P13-2144
Aseel Addawood, Adam Badawy, Kristina Lerman, and Emilio Ferrara. 2019. Linguistic cues to deception:
Identifying political trolls on social media. In Proceedings of the Thirteenth International AAAI Conference
on Web and Social Media (ICWSM ’19, Vol. 13). 15–25. https://doi.org/10.1609/icwsm.v13i01.3205
Richard Allan. 2017. Hard questions: Who should decide what is hate speech in an online global community?
Meta. Retrieved September 19, 2023 from https://about.fb.com/news/2017/06/hard-questions-hate-spe
ech/.
Jisun An, Daniele Quercia, Meeyoung Cha, Krishna Gummadi, and Jon Crowcroft. 2014. Sharing political
news: The balancing act of intimacy and socialization in selective exposure. EPJ Data Science 3, 1 (2014),
12. https://doi.org/10.1140/epjds/s13688-014-0012-2
Sinan Aral and Dylan Walker. 2012. Identifying influential and susceptible members of social networks.
Science 337, 6092 (2012), 337–341. https://doi.org/10.1126/science.121584
Mohammad Atari, Aida Mostafazadeh Davani, Drew Kogon, Brendan Kennedy, Nripsuta Ani Saxena, Ian
Anderson, and Morteza Dehghani. 2022. Morally homogeneous networks and radicalism. Social Psychological and Personality Science 13, 6 (2022), 999–1009. https://doi.org/10.1177/19485506211059329
Adam Badawy, Emilio Ferrara, and Kristina Lerman. 2018. Analyzing the digital traces of political manipulation: The 2016 Russian interference Twitter campaign. In 2018 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM ’18). IEEE, 258–265. https:
//doi.org/10.1109/ASONAM.2018.8508646
Adam Badawy, Kristina Lerman, and Emilio Ferrara. 2019. Who falls for online political manipulation?. In
Companion Proceedings of the 2019 World Wide Web Conference (WWW ’19). Association for Computing
Machinery, 162–168. https://doi.org/10.1145/3308560.3316494
David Bamman and Noah A. Smith. 2015. Open extraction of fine-grained political statements. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (EMNLP ’15). Association
for Computational Linguistics, 76–85. https://doi.org/10.18653/v1/D15-1008
Pablo Barberá. 2015. Birds of the same feather tweet together: Bayesian ideal point estimation using
Twitter data. Political Analysis 23, 1 (2015), 76–91. https://doi.org/10.1093/pan/mpu011
Pablo Barberá. 2020. Social media, echo chambers, and political polarization. In Social Media and
Democracy, Nathaniel Persily and Joshua A.Editors Tucker (Eds.). Cambridge University Press, 34–55.
https://doi.org/10.1017/9781108890960
119

Pablo Barberá, John T. Jost, Jonathan Nagler, Joshua A. Tucker, and Richard Bonneau. 2015. Tweeting from
left to right: Is online political communication more than an echo chamber? Psychological Science 26,
10 (2015), 1531–1542. https://doi.org/10.1177/0956797615594620
Marco Bastos and Johan Farkas. 2019. “Donald Trump is my president!”: The internet research agency
propaganda machine. Social Media + Society 5, 3 (2019), 2056305119865466. https://doi.org/10.1177/20
5630511986546
Roy F Baumeister and Mark R Leary. 1995. The need to belong: Desire for interpersonal attachments as a
fundamental human motivation. Psychological Bulletin 117, 3 (1995), 497–529. https://doi.org/10.1037/
0033-2909.117.3.497
Mariano Gastón Beiró, Jacopo D’Ignazi, Victoria Perez Bustos, María Florencia Prado, and Kyriaki Kalimeri.
2023. Moral narratives around the vaccination debate on Facebook. In Proceedings of the ACM Web
Conference 2023 (WWW ’23). Association for Computing Machinery, 4134–4141. https://doi.org/10.114
5/3543507.3583865
Federico Bianchi, Stefanie HIlls, Patricia Rossini, Dirk Hovy, Rebekah Tromble, and Nava Tintarev. 2022.
“It’s not just hate”: A multi-dimensional perspective on detecting harmful speech online. In Proceedings
of the 2022 Conference on Empirical Methods in Natural Language Processing (EMNLP ’22). Association
for Computational Linguistics, 8093–8099. https://doi.org/10.18653/v1/2022.emnlp-main.553
Judith Borghouts, Yicong Huang, Sydney Gibbs, Suellen Hopfer, Chen Li, and Gloria Mark. 2023. Understanding underlying moral values and language use of COVID-19 vaccine attitudes on Twitter. PNAS
Nexus 2, 3 (2023), pgad013. https://doi.org/10.1093/pnasnexus/pgad013
Mitchell Bosley, Musashi Jacobs-Harukawa, Hauke Licht, and Alexander Hoyle. 2023. Do We Still Need
BERT in the Age of GPT? Comparing the Benefits of Domain-Adaptation and In-Context-Learning Approaches to Using LLMs for Political Science Research. In 2023 Annual Meeting of the Midwest Political
Science Association (MPSA).
Alexandre Bovet and Hernán A Makse. 2019. Influence of fake news in Twitter during the 2016 US presidential election. Nature Communications 10, 1 (2019), 1–14. https://doi.org/10.1038/s41467-018-07761-2
Danah Boyd, Scott Golder, and Gilad Lotan. 2010. Tweet, tweet, retweet: Conversational aspects of retweeting on Twitter. In 2010 43rd Hawaii International Conference on System Sciences (HICSS ’10). IEEE, 1–10.
https://doi.org/10.1109/HICSS.2010.412
William J Brady, Killian McLoughlin, Tuan N Doan, and Molly J Crockett. 2021. How social learning
amplifies moral outrage expression in online social networks. Science Advances 7, 33 (2021), eabe5641.
https://doi.org/10.1126/sciadv.abe5641
William J Brady, Julian A Wills, John T Jost, Joshua A Tucker, and Jay J Van Bavel. 2017. Emotion shapes
the diffusion of moralized content in social networks. Proceedings of the National Academy of Sciences
114, 28 (2017), 7313–7318. https://doi.org/10.1073/pnas.161892311
Kathryn Bruchmann and Liya LaPierre. 2022. Moral foundations predict perceptions of moral permissibility of COVID-19 public health guideline violations in United States university students. Frontiers in
Psychology 12 (2022), 6442. https://doi.org/10.3389/fpsyg.2021.795278
Dustin P. Calvillo, Bryan J. Ross, Ryan J. B. Garcia, Thomas J. Smelter, and Abraham M. Rutchick. 2020.
Political ideology predicts perceptions of the threat of COVID-19 (and susceptibility to fake news about
120

it). Social Psychological and Personality Science 11, 8 (2020), 1119–1128. https://doi.org/10.1177/194855
062094053
Sergio Andrés Castaño-Pulgarín, Natalia Suárez-Betancur, Luz Magnolia Tilano Vega, and Harvey Mauricio Herrera López. 2021. Internet, social media and online hate speech. Systematic review. Aggression
and Violent Behavior 58 (2021), 101608. https://doi.org/10.1016/j.avb.2021.101608
Meeyoung Cha, Hamed Haddadi, Fabricio Benevenuto, and P Krishna Gummadi. 2010. Measuring user
influence in Twitter: The million follower fallacy. In Proceedings of the Fourth International AAAI Conference on Weblogs and Social Media (ICWSM ’10, Vol. 4). 10–17. https://doi.org/10.1609/icwsm.v4i1.14033
Eugene Y Chan. 2021. Moral foundations underlying behavioral compliance during the COVID-19 pandemic. Personality and Individual Differences 171 (2021), 110463. https://doi.org/10.1016/j.paid.2020.11
0463
Emily Chen, Herbert Chang, Ashwin Rao, Kristina Lerman, Geoffrey Cowan, and Emilio Ferrara. 2021a.
COVID-19 misinformation and the 2020 U.S. presidential election. Harvard Kennedy School (HKS) Misinformation Review (2021). https://doi.org/10.37016/mr-2020-57
Emily Chen, Ashok Deb, and Emilio Ferrara. 2021b. #Election2020: The first public Twitter dataset on the
2020 US presidential election. Journal of Computational Social Science (2021), 1–18. https://doi.org/10.1
007/s42001-021-00117-9
Emily Chen and Emilio Ferrara. 2023. Tweets in time of conflict: A public dataset tracking the Twitter
discourse on the war between Ukraine and Russia. In Proceedings of the Seventeenth International AAAI
Conference on Web and Social Media (ICWSM ’23, Vol. 17). 1006–1013. https://doi.org/10.1609/icwsm.v1
7i1.22208
Emily Chen, Kristina Lerman, and Emilio Ferrara. 2020. Tracking social media discourse about the COVID19 pandemic: Development of a public Coronavirus Twitter data set. JMIR Public Health and Surveillance
6, 2 (2020), e19273. https://doi.org/10.2196/19273
Ying Chen, Yilu Zhou, Sencun Zhu, and Heng Xu. 2012. Detecting offensive language in social media
for protection of adolescent online safety. In 2012 International Conference on Privacy, Security, Risk
and Trust and 2012 International Conference on Social Computing (SocialCom/PASSAT ’12). IEEE, 71–80.
https://doi.org/10.1109/SocialCom-PASSAT.2012.55
Nicholas A Christakis and James H Fowler. 2013. Social contagion theory: Examining dynamic social
networks and human behavior. Statistics in Medicine 32, 4 (2013), 556–577. https://doi.org/10.1002/sim.
5408
Matteo Cinelli, Gianmarco De Francisci Morales, Alessandro Galeazzi, Walter Quattrociocchi, and Michele
Starnini. 2021. The echo chamber effect on social media. Proceedings of the National Academy of Sciences
118, 9 (2021), e2023301118. https://doi.org/10.1073/pnas.2023301118
Claudio Cioffi-Revilla. 2014. Introduction to Computational Social Science. Springer Cham. https://doi.org/
10.1007/978-3-319-50131-4
Claudio Cioffi-Revilla. 2021. The scope of computational social science. In Handbook of Computational
Social Science, Volume 1, Uwe Engel, Anabel Quan-Haase, Sunny Liu, and Lars E Lyberg (Eds.). Routledge,
17–32. https://doi.org/10.4324/9781003024583
121

Raviv Cohen and Derek Ruths. 2013. Classifying political orientation on Twitter: It’s not easy!. In Proceedings of the Seventh International AAAI Conference on Weblogs and Social Media (ICWSM ’13, Vol. 7).
91–99. https://doi.org/10.1609/icwsm.v7i1.14434
Elanor Colleoni, Alessandro Rozza, and Adam Arvidsson. 2014. Echo chamber or public sphere? Predicting political orientation and measuring political homophily in Twitter using big data. Journal of
Communications 64, 2 (2014), 317–332. https://doi.org/10.1111/jcom.12084
Michael D. Conover, Bruno Goncalves, Jacob Ratkiewicz, Alessandro Flammini, and Filippo Menczer.
2011a. Predicting the political alignment of Twitter users. In 2011 IEEE Third International Conference
on Privacy, Security, Risk and Trust and 2011 IEEE Third International Conference on Social Computing
(PASSAT/SocialCom ’11). IEEE, 192–199. https://doi.org/10.1109/PASSAT/SocialCom.2011.34
Michael D. Conover, Jacob Ratkiewicz, Matthew R Francisco, Bruno Gonçalves, Filippo Menczer, and
Alessandro Flammini. 2011b. Political polarization on Twitter. In Proceedings of the Fifth International
AAAI Conference on Weblogs and Social Media (ICWSM ’11). 89–96. https://doi.org/10.1609/icwsm.v5i1
.14126
Alessandro Cossard, Gianmarco De Francisci Morales, Kyriaki Kalimeri, Yelena Mejova, Daniela Paolotti,
and Michele Starnini. 2020. Falling into the echo chamber: the Italian vaccination debate on Twitter.
In Proceedings of the Fourteenth International AAAI Conference on Web and Social Media (ICWSM ’20,
Vol. 14). 130–140. https://doi.org/10.1609/icwsm.v14i1.7285
Kareem Darwish, Peter Stefanov, Michaël Aupetit, and Preslav Nakov. 2020. Unsupervised user stance
detection on Twitter. In Proceedings of the Fourteenth International AAAI Conference on Web and Social
Media (ICWSM ’20, Vol. 14). 141–152. https://doi.org/10.1609/icwsm.v14i1.7286
Clayton Allen Davis, Onur Varol, Emilio Ferrara, Alessandro Flammini, and Filippo Menczer. 2016.
BotOrNot: A system to evaluate social bots. In Proceedings of the 25th International Conference Companion on World Wide Web (WWW ’16 Companion). 273–274. https://doi.org/10.1145/2872518.2889302
James A Davis. 1985. The Logic of Causal Order. SAGE Publications. https://doi.org/10.4135/9781412986212
Morteza Dehghani, Kate Johnson, Joe Hoover, Eyal Sagi, Justin Garten, Niki Jitendra Parmar, Stephen
Vaisey, Rumen Iliev, and Jesse Graham. 2016. Purity homophily in social networks. Journal of Experimental Psychology: General 145, 3 (2016), 366–375. https://doi.org/10.1037/xge0000139
Michela Del Vicario, Alessandro Bessi, Fabiana Zollo, Fabio Petroni, Antonio Scala, Guido Caldarelli, H Eugene Stanley, and Walter Quattrociocchi. 2016. The spreading of misinformation online. Proceedings of
the National Academy of Sciences 113, 3 (2016), 554–559. https://doi.org/10.1073/pnas.1517441113
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep
bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North
American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) (NAACL ’19). Association for Computational Linguistics, 4171–4186.
https://doi.org/10.18653/v1/N19-1423
Rodrigo Díaz and Florian Cova. 2022. Reactance, morality, and disgust: The relationship between affective dispositions and compliance with official health recommendations during the COVID-19 pandemic.
Cognition and Emotion 36, 1 (2022), 120–136. https://doi.org/10.1080/02699931.2021.1941783
122

Keyu Duan, Zirui Liu, Peihao Wang, Wenqing Zheng, Kaixiong Zhou, Tianlong Chen, Xia Hu, and
Zhangyang Wang. 2022. A comprehensive study on large-scale graph training: Benchmarking and rethinking. In Thirty-sixth Conference on Neural Information Processing Systems Datasets and Benchmarks
Track (NeuRIPS ’22). https://openreview.net/forum?id=2QrFr_U782Z
Peter Duffy. 2008. Using YouTube: Strategies for using new media in teaching and learning. In Enhancing
learning through technology: research on emerging technologies and pedagogies. World Scientific, 31–43.
https://doi.org/10.1142/9789812799456_0003
Achim Edelmann, Tom Wolff, Danielle Montagne, and Christopher A Bail. 2020. Computational social
science and sociology. Annual Review of Sociology 46 (2020), 61–81. https://doi.org/10.1146/annurev-s
oc-121919-054621
Hatice Ekici, Emine Yücel, and Sevim Cesur. 2021. Deciding between moral priorities and COVID-19
avoiding behaviors: A moral foundations vignette study. Current Psychology (2021), 1–17. https:
//doi.org/10.1007/s12144-021-01941-y
Heba Elfardy and Mona Diab. 2016. Addressing annotation complexity: The case of annotating ideological
perspective in Egyptian social media. In Proceedings of the 10th Linguistic Annotation Workshop held in
conjunction with ACL 2016. Association for Computational Linguistics, 79–88. https://doi.org/10.18653
/v1/W16-1710
Christian Ezeibe. 2021. Hate speech and election violence in Nigeria. Journal of Asian and African Studies
56, 4 (2021), 919–935. https://doi.org/10.1177/0021909620951208
Jianqing Fan, Fang Han, and Han Liu. 2014. Challenges of big data analysis. National Science Review 1, 2
(2014), 293–314. https://doi.org/10.1093/nsr/nwt032
Matthew Feinberg and Robb Willer. 2019. Moral reframing: A technique for effective and persuasive
communication across political divides. Social and Personality Psychology Compass 13, 12 (2019), e12501.
https://doi.org/10.1111/spc3.12501
Emilio Ferrara. 2020. What types of COVID-19 conspiracies are populated by Twitter bots? First Monday
25, 6 (2020). https://doi.org/10.5210/fm.v25i6.10633
Emilio Ferrara, Herbert Chang, Emily Chen, Goran Muric, and Jaimin Patel. 2020. Characterizing social
media manipulation in the 2020 US presidential election. First Monday 25, 11 (2020). https://doi.org/10
.5210/fm.v25i11.11431
Emilio Ferrara, Onur Varol, Clayton Davis, Filippo Menczer, and Alessandro Flammini. 2016. The rise of
social bots. Communications of the ACM 59, 7 (2016), 96–104. https://doi.org/10.1145/2818717
Geoffrey A. Fowler. 2020. Twitter and Facebook warning labels aren’t enough to save democracy. The Washington Post. Retrieved December 14, 2020 from https://www.washingtonpost.com/technology/2020/1
1/09/facebook-twitter-election-misinformation-labels.
Kathryn B Francis and Carolyn B McNabb. 2022. Moral decision-making during COVID-19: Moral
judgements, moralisation, and everyday behaviour. Frontiers in Psychology 12 (2022), 6484. https:
//doi.org/10.3389/fpsyg.2021.769177
Linton C. Freeman. 2004. The Development of Social Network Analysis: A Study in the Sociology of Science.
Empirical Press.
123

Jeremy A Frimer, Harinder Aujla, Matthew Feinberg, Linda J Skitka, Karl Aquino, Johannes C Eichstaedt,
and Robb Willer. 2023. Incivility is rising among American politicians on Twitter. Social Psychological
and Personality Science 14, 2 (2023), 259–269. https://doi.org/10.1177/19485506221083811
Ivan Garibay, Alexander V Mantzaris, Amirarsalan Rajabi, and Cameron E Taylor. 2019. Polarization in
social media assists influencers to become more influential: Analysis and two inoculation strategies.
Scientific Reports 9, 1 (2019), 1–9. https://doi.org/10.1038/s41598-019-55178-8
Kiran Garimella, Gianmarco De Francisci Morales, Aristides Gionis, and Michael Mathioudakis. 2018a.
Political discourse on social media: Echo chambers, gatekeepers, and the price of bipartisanship. In Proceedings of the 2018 World Wide Web Conference (WWW ’18). International World Wide Web Conferences
Steering Committee, 913–922. https://doi.org/10.1145/3178876.3186139
Kiran Garimella, Gianmarco De Francisci Morales, Aristides Gionis, and Michael Mathioudakis. 2018b.
Quantifying controversy on social media. ACM Transactions on Social Computing 1, 1, Article 3 (2018),
27 pages. https://doi.org/10.1145/3140565
R Kelly Garrett. 2009. Echo chambers online? Politically motivated selective exposure among Internet
news users. Journal of Computer-Mediated Communication 14, 2 (1 2009), 265–285. https://doi.org/10.1
111/j.1083-6101.2009.01440.x
Jeffrey Gottfried. 2024. Americans’ social media use. Pew Research Center. Retrieved April 1, 2024 from
https://www.pewresearch.org/internet/2024/01/31/americans-social-media-use/.
Palash Goyal and Emilio Ferrara. 2018. Graph embedding techniques, applications, and performance: A
survey. Knowledge-Based Systems 151 (2018), 78–94. https://doi.org/10.1016/j.knosys.2018.03.022
Jesse Graham, Jonathan Haidt, Sena Koleva, Matt Motyl, Ravi Iyer, Sean P Wojcik, and Peter H Ditto. 2013.
Moral foundations theory: The pragmatic validity of moral pluralism. In Advances in Experimental Social
Psychology. Vol. 47. Elsevier, 55–130. https://doi.org/10.1016/B978-0-12-407236-7.00002-4
Jesse Graham, Jonathan Haidt, and Brian A Nosek. 2009. Liberals and conservatives rely on different sets
of moral foundations. Journal of Personality and Social Psychology 96, 5 (2009), 1029. https://doi.org/10
.1037/a0015141
Jesse Graham, Brian A Nosek, and Jonathan Haidt. 2012. The moral stereotypes of liberals and conservatives: Exaggeration of differences across the political spectrum. PloS ONE 7, 12 (2012), e50092.
https://doi.org/10.1371/journal.pone.0050092
John Gramlich. 2022. Two years into the pandemic, Americans inch closer to a new normal. Pew Research
Center. Retrieved April 4, 2023 from https://www.pewresearch.org/2022/03/03/two-years-into-the-pan
demic-americans-inch-closer-to-a-new-normal/.
Christine Greenhow and Cathy Lewin. 2019. Social media and education: Reconceptualizing the boundaries of formal and informal learning. In Social Media and Education. Routledge, 6–30. h t tps:
//doi.org/10.1080/17439884.2015.1064954
Aditya Grover and Jure Leskovec. 2016. node2vec: Scalable feature learning for networks. In Proceedings
of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD ’16).
Association for Computing Machinery, 855–864. https://doi.org/10.1145/2939672.2939754
124

Siyi Guo, Negar Mokhberian, and Kristina Lerman. 2023. A data fusion framework for multi-domain
morality learning. In Proceedings of the Seventeenth International AAAI Conference on Web and Social
Media (ICWSM ’23, Vol. 17). 281–291. https://doi.org/10.1609/icwsm.v17i1.22145
Jonathan Haidt. 2012. The Righteous Mind: Why Good people are Divided by Politics and Religion. Vintage.
Jonathan Haidt and Jesse Graham. 2007. When morality opposes justice: conservatives have moral
intuitions that liberals may not recognize. Social Justice Research 20, 1 (2007), 98–116. h t tps:
//doi.org/10.1007/s11211-007-0034-z
Jonathan Haidt and Craig Joseph. 2004. Intuitive ethics: How innately prepared intuitions generate culturally variable virtues. Daedalus 133, 4 (2004), 55–66. https://www.jstor.org/stable/20027945
William L Hamilton. 2020. Graph Representation Learning. Morgan & Claypool Publishers.
William L. Hamilton, Zhitao Ying, and Jure Leskovec. 2017. Inductive representation learning on large
graphs. In Advances in Neural Information Processing Systems (NIPS ’17, Vol. 30). Curran Associates Inc.
https://proceedings.neurips.cc/paper_files/paper/2017/file/5dd9db5e033da9c6fb5ba83c7a7ebea9-Paper
.pdf
Kazi Saidul Hasan and Vincent Ng. 2014. Why are you taking this stance? Identifying and classifying
reasons in ideological debates. In Proceedings of the 2014 Conference on Empirical Methods in Natural
Language Processing (EMNLP ’14). Association for Computational Linguistics, 751–762. https://doi.org/
10.3115/v1/D14-1083
Jeff Hemsley, Jennifer Stromer-Galley, Bryan Semaan, and Sikana Tanupabrungsun. 2018. Tweeting to
the target: Candidates’ use of strategic messages and @mentions on Twitter. Journal of Information
Technology & Politics 15, 1 (2018), 3–18. https://doi.org/10.1080/19331681.2017.1338634
Matthew Henderson, Rami Al-Rfou, Brian Strope, Yun-Hsuan Sung, László Lukács, Ruiqi Guo, Sanjiv Kumar, Balint Miklos, and Ray Kurzweil. 2017. Efficient natural language response suggestion for smart
reply. arXiv preprint arXiv:1705.00652 (2017). https://arxiv.org/abs/1705.00652
Martin Hentschel, Omar Alonso, Scott Counts, and Vasileios Kandylas. 2014. Finding users we trust:
Scaling up verified Twitter users using their communication patterns. In Proceedings of the Eighth
International AAAI Conference on Weblogs and Social Media (ICWSM ’14, Vol. 8). 591–594. https:
//doi.org/10.1609/icwsm.v8i1.14569
Nathan O Hodas and Kristina Lerman. 2014. The simple rules of social contagion. Scientific Reports 4, 1
(2014), 4343. https://doi.org/10.1038/srep04343
Matthew J Hornsey, Matthew Finlayson, Gabrielle Chatwood, and Christopher T Begeny. 2020. Donald
Trump and vaccination: The effect of political identity, conspiracist ideation and presidential tweets on
vaccine hesitancy. Journal of Experimental Social Psychology 88 (2020), 103947. https://doi.org/10.1016/
j.jesp.2019.103947
Mike Isaac and Lauren Hirsch. 2024. Reddit prices I.P.O. at $34 a share, in a positive sign for tech. The New
York Times. Retrieved April 3, 2024 from https://www.nytimes.com/2024/03/20/technology/reddit-ipo
-stock-price.html.
Mohit Iyyer, Peter Enns, Jordan Boyd-Graber, and Philip Resnik. 2014. Political ideology detection using
recursive neural networks. In Proceedings of the 52nd Annual Meeting of the Association for Computational
Linguistics (Volume 1: Long Papers) (ACL ’14). Association for Computational Linguistics, 1113–1122.
https://doi.org/10.3115/v1/P14-1105
125

Kokil Jaidka, Subhayan Mukerjee, and Yphtach Lelkes. 2023. Silenced on social media: The gatekeeping
functions of shadowbans in the American Twitterverse. Journal of Communication 73, 2 (2023), 163–178.
https://doi.org/10.1093/joc/jqac050
Julie Jiang, Emily Chen, Kristina Lerman, and Emilio Ferrara. 2020. Political polarization drives online
conversations about COVID-19 in the United States. Human Behavior and Emerging Technologies 2, 3
(2020), 200–211. https://doi.org/10.1002/hbe2.202
Julie Jiang, Emily Chen, Luca Luceri, Goran Murić, Francesco Pierri, Ho-Chun Herbert Chang, and Emilio
Ferrara. 2023a. What are your pronouns? Examining gender pronoun usage on Twitter. In Workshop
Proceedings of the 17th International AAAI Conference on Web and Social Media (ICWSM ’23). https:
//doi.org/10.36190/2023.02
Julie Jiang and Emilio Ferrara. 2023. Social-LLM: Modeling user behavior at scale using language models
and social network data. arXiv preprint arXiv:2401.00893 (2023). https://arxiv.org/abs/2401.00893
Julie Jiang, Luca Luceri, and Emilio Ferrara. 2024. Moral values underpinning COVID-19 online communication patterns. arXiv preprint arXiv:2401.08789 (2024). https://arxiv.org/abs/2401.08789
Julie Jiang, Luca Luceri, Joseph B Walther, and Emilio Ferrara. 2023b. Social approval and network homophily as motivators of online toxicity. arXiv preprint arXiv:2310.07779 (2023). https://arxiv.org/abs/
2310.07779
Julie Jiang, Xiang Ren, and Emilio Ferrara. 2021. Social media polarization and echo chambers in the
context of COVID-19: Case study. JMIRx med 2, 3 (2021), e29570. https://doi.org/10.2196/29570
Julie Jiang, Xiang Ren, and Emilio Ferrara. 2023c. Retweet-BERT: Political leaning detection using language
features and information diffusion on social networks. In Proceedings of the Seventeenth International
AAAI Conference on Web and Social Media (ICWSM ’23, Vol. 17). 459–469. https://doi.org/10.1609/icws
m.v17i1.22160
Kristen Johnson and Dan Goldwasser. 2018. Classification of moral foundations in microblog political
discourse. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics
(Volume 1: Long Papers) (ACL ’18). Association for Computational Linguistics, 720–730. https://doi.or
g/10.18653/v1/P18-1067
Kristen Johnson, Di Jin, and Dan Goldwasser. 2017. Leveraging behavioral and social information for
weakly supervised collective classification of political discourse on Twitter. In Proceedings of the 55th
Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (ACL ’17). Association for Computational Linguistics, 741–752. https://doi.org/10.18653/v1/P17-1069
Jonas T Kaplan, Anthony Vaccaro, Max Henning, and Leonardo Christov-Moore. 2023. Moral reframing
of messages about mask-wearing during the COVID-19 pandemic. Scientific Reports 13, 1 (2023), 10140.
https://doi.org/10.1038/s41598-023-37075-3
Teo Keipi, Matti Näsi, Atte Oksanen, and Pekka Räsänen. 2016. Online Hate and Harmful Content: CrossNational Perspectives. Taylor & Francis.
Jin Woo Kim, Andrew Guess, Brendan Nyhan, and Jason Reifler. 2021. The distorting prism of social media:
How self-selection and exposure to incivility fuel online comment toxicity. Journal of Communication
71, 6 (2021), 922–946. https://doi.org/10.1093/joc/jqab034
126

Thomas N Kipf and Max Welling. 2017. Semi-supervised classification with graph convolutional networks.
In International Conference on Learning Representations (ICLR ’17). https://openreview.net/forum?id=
SJU4ayYgl
Ella Koeze and Nathaniel Popper. 2020. The virus changed the way we internet. The New York Times.
Retreived December 14, 2020 from https://www.nytimes.com/interactive/2020/04/07/technology/coro
navirus-internet-use.html.
Spassena P Koleva, Jesse Graham, Ravi Iyer, Peter H Ditto, and Jonathan Haidt. 2012. Tracing the threads:
How five moral concerns (especially purity) help explain culture war attitudes. Journal of Research in
Personality 46, 2 (2012), 184–194. https://doi.org/10.1016/j.jrp.2012.01.006
Gueorgi Kossinets and Duncan J Watts. 2009. Origins of homophily in an evolving social network. American
Journal of Sociology 115, 2 (2009), 405–450. https://doi.org/10.1086/599247
Balazs Kovacs and Adam M Kleinbaum. 2020. Language-style similarity and social networks. Psychological
Science 31, 2 (2020), 202–213. https://doi.org/10.1177/0956797619894557
Emily Kubin and Christian Von Sikorski. 2021. The role of (social) media in political polarization: A
systematic review. Annals of the International Communication Association 45, 3 (2021), 188–206. https:
//doi.org/10.1080/23808985.2021.1976070
Daria J Kuss and Mark D Griffiths. 2011. Online social networking and addiction—a review of the psychological literature. International Journal of Environmental Research and Public Health 8, 9 (2011), 3528–
3552. https://doi.org/10.3390/ijerph8093528
David Lazer, Alex Pentland, Lada Adamic, Sinan Aral, Albert-László Barabási, Devon Brewer, Nicholas
Christakis, Noshir Contractor, James Fowler, Myron Gutmann, Tony Jebara, Gary King, Michael Macy,
Deb Roy, and Marshall Van Alstyne. 2009. Computational social science. Science 323, 5915 (2009), 721–
723. https://doi.org/10.1126/science.1167742
David M. J. Lazer, Alex Pentland, Duncan J. Watts, Sinan Aral, Susan Athey, Noshir Contractor, Deen
Freelon, Sandra Gonzalez-Bailon, Gary King, Helen Margetts, Alondra Nelson, Matthew J. Salganik,
Markus Strohmaier, Alessandro Vespignani, and Claudia Wagner. 2020. Computational Social Science:
Obstacles and Opportunities. Science 369, 6507 (2020), 1060–1062.
Chang Li and Dan Goldwasser. 2019. Encoding social information with graph convolutional networks for
political perspective detection in news media. In Proceedings of the 57th Annual Meeting of the Association
for Computational Linguistics (ACL ’19). Association for Computational Linguistics, 2594–2604. https:
//doi.org/10.18653/v1/P19-1247
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke
Zettlemoyer, and Veselin Stoyanov. 2019. RoBERTa: A robustly optimized BERT pretraining approach.
arXiv preprint arXiv:1907.11692 (2019). https://arxiv.org/abs/1907.11692
Ziqi Liu, Chaochao Chen, Xinxing Yang, Jun Zhou, Xiaolong Li, and Le Song. 2018. Heterogeneous graph
neural networks for malicious account detection. In Proceedings of the 27th ACM International Conference
on Information and Knowledge Management (CIKM ’18). Association for Computing Machinery, 2077–
2085. https://doi.org/10.1145/3269206.3272010
Los Angeles Times. 2020. Letters to the editor: Refusing to wear a mask is the most brainless, selfish way to
assert your liberty. Los Angeles Times. Retrieved April 4, 2023 from https://www.latimes.com/opinion/
story/2020-06-20/refusing-to-wear-a-mask-brainless-selfish.
127

Tiancheng Lou and Jie Tang. 2013. Mining structural hole spanners through information diffusion in social
networks. In Proceedings of the 22nd international conference on World Wide Web (WWW ’13). 825–836.
https://doi.org/10.1145/2488388.2488461
Luca Luceri, Silvia Giordano, and Emilio Ferrara. 2020. Detecting troll behavior via inverse reinforcement learning: A case study of Russian trolls in the 2016 US election. In Proceedings of the Fourteenth International AAAI Conference on Web and Social Media (ICWSM ’20, Vol. 14). 417–427. https:
//doi.org/10.1609/icwsm.v14i1.7311
Andrew Luttrell and Joseph T Trentadue. 2023. Advocating for mask-wearing across the aisle: Applying
moral reframing in health communication. Health Communication (2023), 1–13. https://doi.org/10.108
0/10410236.2022.2163535
Hehuan Ma, Yu Rong, and Junzhou Huang. 2022b. Graph neural networks: Scalability. In Graph Neural
Networks: Foundations, Frontiers, and Applications, Lingfei Wu, Peng Cui, Jian Pei, and Liang Zhao (Eds.).
Springer Nature Singapore, 99–119. https://doi.org/10.1007/978-981-16-6054-2_6
Yao Ma, Xiaorui Liu, Neil Shah, and Jiliang Tang. 2022a. Is homophily a necessity for graph neural networks?. In International Conference on Learning Representations (ICLR ’22). https://openreview.net/for
um?id=ucASPPD9GKN
Sapna Maheshwari. 2017. On YouTube Kids, startling videos slip past filters. The New York Times. Retrieved
April 4, 2024 from https://www.nytimes.com/2017/11/04/business/media/youtube-kids-paw-patrol.ht
ml.
Sapna Maheshwari, David McCabe, and Annie Karni. 2024. House Passes Bill to Force TikTok Sale From
Chinese Owner or Ban the App. The New York Times. Retrieved April 3, 2024 from https://www.nytime
s.com/2024/03/13/technology/tiktok-ban-house-vote.html.
VenkataSwamy Martha, Weizhong Zhao, and Xiaowei Xu. 2013. A study on Twitter user-follower network:
a network based analysis. In Proceedings of the 2013 IEEE/ACM International Conference on Advances in
Social Networks Analysis and Mining (ASONAM ’13). IEEE, 1405–1409. https://doi.org/10.1145/2492517.
2500298
Alice Marwick and Rebecca Lewis. 2017. Media Manipulation and Disinformation Online. Technical Report.
Data & Society Research Institute. https://datasociety.net/output/media-manipulation-and-disinfo-onl
ine/
Faiza Masood, Ahmad Almogren, Assad Abbas, Hasan Ali Khattak, Ikram Ud Din, Mohsen Guizani, and
Mansour Zuair. 2019. Spammer detection and fake user identification on social networks. IEEE Access 7
(2019), 68140–68152. https://doi.org/10.1109/ACCESS.2019.2918196
Binny Mathew, Anurag Illendula, Punyajoy Saha, Soumya Sarkar, Pawan Goyal, and Animesh Mukherjee.
2020. Hate begets hate: A temporal study of hate speech. Proceedings of the ACM on Human-Computer
Interaction 4, CSCW2, Article 92 (2020), 24 pages. https://doi.org/10.1145/3415163
Miller McPherson, Lynn Smith-Lovin, and James M Cook. 2001. Birds of a feather: Homophily in social
networks. Annual Review of Sociology 27, 1 (2001), 415–444. https://doi.org/10.1146/annurev.soc.27.1.415
Yelena Mejova, Kyriaki Kalimeri, and Gianmarco De Francisci Morales. 2023. Authority without care:
Moral values behind the mask mandate response. In Proceedings of the Seventeenth International AAAI
Conference on Web and Social Media (ICWSM ’23, Vol. 17). 614–625. https://doi.org/10.1609/icwsm.v17i
1.22173
128

Mention.com. 2018. Mention’s Twitter engagement report 2018. Mention.com. Retrieved August 7, 2021
from https://mention.com/en/reports/twitter/bios/.
Panagiotis Metaxas, Eni Mustafaraj, Kily Wong, Laura Zeng, Megan O’Keefe, and Samantha Finn. 2015.
What do retweets indicate? Results from user survey and meta-review of research. In Proceedings of
the Ninth International AAAI Conference on Web and Social Media (ICWSM ’15, Vol. 9). 658–661. https:
//doi.org/10.1609/icwsm.v9i1.14661
Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Distributed representations of words and phrases and their compositionality. In Advances in Neural Information Processing
Systems (NIPS ’13). 3111–3119. https://proceedings.neurips.cc/paper_files/paper/2013/file/9aa42b3188
2ec039965f3c4923ce901b-Paper.pdf
Matt Motta, Dominik Stecula, and Christina Farhart. 2020. How right-leaning media coverage of COVID19 facilitated the spread of misinformation in the early stages of the pandemic in the US. Canadian
Journal of Political Science (2020), 1–8. https://doi.org/10.1017/S0008423920000396
Karsten Müller and Carlo Schwarz. 2021. Fanning the flames of hate: Social media and hate crime. Journal
of the European Economic Association 19, 4 (2021), 2131–2167. https://doi.org/10.1093/jeea/jvaa045
Goran Muric, Yusong Wu, and Emilio Ferrara. 2021. COVID-19 vaccine hesitancy on social media: building
a public Twitter dataset of anti-vaccine content, vaccine misinformation and conspiracies. JMIR Public
Health and Surveillance 7, 11 (2021), e30642. https://doi.org/10.2196/30642
Arash Naeim, Ryan Baxter-King, Neil Wenger, Annette L Stanton, Karen Sepucha, and Lynn Vavreck. 2021.
Effects of age, gender, health status, and political party on COVID-19–related concerns and prevention
behaviors: Results of a large, longitudinal cross-sectional survey. JMIR Public Health and Surveillance 7,
4 (2021), e24277. https://doi.org/10.2196/24277
Seema Nagar, Sameer Gupta, CS Bahushruth, Ferdous Ahmed Barbhuiya, and Kuntal Dey. 2022. Homophily
- a driving factor for hate speech on Twitter. In Complex Networks & Their Applications X. Springer
International Publishing, 78–88. https://doi.org/10.1007/978-3-030-93413-2_7
Mark EJ Newman. 2003. Mixing patterns in networks. Physical Review E 67, 2 (2003), 026126. https:
//doi.org/10.1103/PhysRevE.67.026126
Dat Quoc Nguyen, Thanh Vu, and Anh Tuan Nguyen. 2020. BERTweet: A pre-trained language model for
English tweets. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations (EMNLP ’20). 9–14. https://doi.org/10.18653/v1/2020.emnlp-demos.2
Annabelle Nyst. 2023. 134 social media statistics you need to know for 2023. Search Engine Journal. Retrieved
April 1, 2024 from https://www.searchenginejournal.com/social-media-statistics/480507/.
Adewale Obadimu, Tuja Khaund, Esther Mead, Thomas Marcoux, and Nitin Agarwal. 2021. Developing a
socio-computational approach to examine toxicity propagation and regulation in COVID-19 discourse
on YouTube. Information Processing & Management 58, 5 (2021), 102660. https://doi.org/10.1016/j.ipm.
2021.102660
Kieron O’Hara and David Stevens. 2015. Echo chambers and online radicalism: Assessing the internet’s
complicity in violent extremism. Policy & Internet 7, 4 (2015), 401–422. https://doi.org/10.1002/poi3.88
129

Maria Leonor Pacheco, Tunazzina Islam, Monal Mahajan, Andrey Shor, Ming Yin, Lyle Ungar, and Dan
Goldwasser. 2022. A holistic framework for analyzing the COVID-19 vaccine debate. In Proceedings of the
2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human
Language Technologies (NAACL ’22). Association for Computational Linguistics, 5821–5839. https:
//doi.org/10.18653/v1/2022.naacl-main.427
Lawrence Page, Sergey Brin, Rajeev Motwani, and Terry Winograd. 1999. The PageRank Citation Ranking:
Bringing Order to the Web. Technical Report. Stanford InfoLab.
Shirui Pan, Jia Wu, Xingquan Zhu, Chengqi Zhang, and Yang Wang. 2016. Tri-party deep network representation. In Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence (IJCAI
’16). AAAI Press, 1895–1901. https://www.ijcai.org/Proceedings/16/Papers/271.pdf
María Antonia Paz, Julio Montero-Díaz, and Alicia Moreno-Delgado. 2020. Hate speech: A systematized
review. Sage Open 10, 4 (2020). https://doi.org/10.1177/2158244020973022
Jeffrey Pennington, Richard Socher, and Christopher D Manning. 2014. GloVe: Global vectors for word
representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing
(EMNLP ’14). Association for Computational Linguistics, 1532–1543. https://doi.org/10.3115/v1/D14-1
162
Patrick Peretti-Watel, Valérie Seror, Sébastien Cortaredona, Odile Launay, Jocelyn Raude, Pierrea Verger,
Lisa Fressard, François Beck, Stéphane Legleye, Olivier L’Haridon, Damien Léger, and Jeremy Keith
Ward. 2020. A future vaccination campaign against COVID-19 at risk of vaccine hesitancy and politicisation. The Lancet Infectious Diseases 20, 7 (2020), 769–770. https://doi.org/10.1016/S1473-3099(20)30426-6
Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. 2014. DeepWalk: Online learning of social representations. In Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data
Mining (KDD ’14). Association for Computing Machinery, 701–710. https://doi.org/10.1145/2623330.26
23732
Francesco Pierri, Luca Luceri, Emily Chen, and Emilio Ferrara. 2023a. How does Twitter account moderation work? Dynamics of account creation and suspension on Twitter during major geopolitical events.
EPJ Data Science 12, 1 (2023), 43. https://doi.org/10.1140/epjds/s13688-023-00420-7
Francesco Pierri, Luca Luceri, Nikhil Jindal, and Emilio Ferrara. 2023b. Propaganda and misinformation
on Facebook and Twitter during the Russian invasion of Ukraine. In Proceedings of the 15th ACM Web
Science Conference 2023 (WebSci ’23). Association for Computing Machinery, 65–74. https://doi.org/10
.1145/3578503.3583597
Daniel Preoţiuc-Pietro, Ye Liu, Daniel Hopkins, and Lyle Ungar. 2017. Beyond binary labels: Political
ideology prediction of Twitter users. In Proceedings of the 55th Annual Meeting of the Association for
Computational Linguistics (Volume 1: Long Papers) (ACL ’17). Association for Computational Linguistics,
729–740. https://doi.org/10.18653/v1/P17-1068
Ashwin Rao, Siyi Guo, Sze-Yuh Nina Wang, Fred Morstatter, and Kristina Lerman. 2023. Pandemic
culture wars: partisan asymmetries in the moral language of COVID-19 discussions. arXiv preprint
arXiv:2305.18533 (2023). https://arxiv.org/abs/2305.18533
Ashwin Rao, Fred Morstatter, Minda Hu, Emily Chen, Keith Burghardt, Emilio Ferrara, and Kristina
Lerman. 2021. Political partisanship and antiscience attitudes in online discussions about COVID19: Twitter content analysis. Journal of Medical Internet Research 23, 6 (2021), e26692. h t tps:
//doi.org/10.2196/26692
130

Radim Řehůřek and Petr Sojka. 2010. Software framework for topic modelling with large corpora. In
Proceedings of the LREC 2010 Workshop on New Challenges for NLP Frameworks. 45–50.
Nils Karl Reimer, Mohammad Atari, Farzan Karimi-Malekabadi, Jackson Trager, Brendan Kennedy, Jesse
Graham, and Morteza Dehghani. 2022. Moral values predict county-level COVID-19 vaccination rates
in the United States. American Psychologist 77, 6 (2022), 743. https://doi.org/10.1037/amp0001020
Nils Reimers and Iryna Gurevych. 2019. Sentence-BERT: Sentence embeddings using Siamese BERTnetworks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing
and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP ’19). Association for Computational Linguistics, 3982–3992. https://doi.org/10.18653/v1/D19-1410
Manoel Horta Ribeiro, Pedro H Calais, Yuri A Santos, Virgílio AF Almeida, and Wagner Meira Jr. 2018a.
Characterizing and detecting hateful users on Twitter. In Proceedings of the Twelfth International AAAI
Conference on Web and Social Media (ICWSM ’18). 676–679. https://doi.org/10.1609/icwsm.v12i1.15057
Manoel Horta Ribeiro, Pedro H Calais, Yuri A Santos, Virgílio AF Almeida, and Wagner Meira Jr. 2018b.
“Like sheep among wolves": Characterizing hateful users on Twitter. Proceedings of WSDM workshop on
Misinformation and Misbehavior Mining on the Web (MIS2).
Sebastián A Rios, Felipe Aguilera, J David Nuñez-Gonzalez, and Manuel Graña. 2019. Semantically enhanced network analysis for influencer identification in online social networks. Neurocomputing 326
(2019), 71–81. https://doi.org/10.1016/j.neucom.2017.01.123
Nick Rogers and Jason J Jones. 2021. Using Twitter bios to measure changes in self-identity: Are Americans
defining themselves more politically over time? Journal of Social Computing 2, 1 (2021), 1–13. https:
//doi.org/10.23919/JSC.2021.0002
Andrew Rojecki, Elena Zheleva, and Lauren Levine. 2021. The Moral imperatives of self-quarantining. In
Annual Meeting of the American Political Science Association.
Daniel Romer and Kathleen Hall Jamieson. 2021. Patterns of media use, strength of belief in COVID-19
conspiracy theories, and the prevention of COVID-19 from March to July 2020 in the United States:
Survey study. Journal of Medical Internet Research 23, 4 (2021), e25215. https://doi.org/10.2196/25215
Matthew Rosenberg and Sheera Frenkel. 2018. Facebook’s role in data misuse sets off storms on two continents.
The New York Times. Retrieved April 4, 2024 from https://www.nytimes.com/2018/03/18/us/cambridg
e-analytica-facebook-privacy-data.html.
Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2019. DistilBERT, a distilled version of
BERT: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108 (2019).
Elvis Saravia, Chun-Hao Chang, Renaud Jollet De Lorenzo, and Yi-Shin Chen. 2016. MIDAS: Mental illness
detection and analysis via social media. In 2016 IEEE/ACM International Conference on Advances in Social
Networks Analysis and Mining (ASONAM ’16). IEEE, 1418–1421. https://doi.org/10.1109/ASONAM.201
6.7752434
Ana Lucía Schmidt, Fabiana Zollo, Michela Del Vicario, Alessandro Bessi, Antonio Scala, Guido Caldarelli,
H. Eugene Stanley, and Walter Quattrociocchi. 2017. Anatomy of news consumption on Facebook.
Proceedings of the National Academy of Sciences 114, 12 (2017), 3035–3039. https://doi.org/10.1073/pnas
.1617052114
131

Kelly Ann Schmidtke, Laura Kudrna, Angela Noufaily, Nigel Stallard, Magdalena Skrybant, Samantha Russell, and Aileen Clarke. 2022. Evaluating the relationship between moral values and vaccine hesitancy
in Great Britain during the COVID-19 pandemic: A cross-sectional survey. Social Science & Medicine
308 (2022), 115218. https://doi.org/10.1016/j.socscimed.2022.115218
Marco Serafini and Hui Guan. 2021. Scalable graph neural network training: The case for sampling. ACM
SIGOPS Operating Systems Review 55, 1 (2021), 68–76. https://doi.org/10.1145/3469379.3469387
Cosma Rohilla Shalizi and Andrew C Thomas. 2011. Homophily and contagion are generically confounded
in observational social network studies. Sociological Methods & Research 40, 2 (2011), 211–239. https:
//doi.org/10.1177/0049124111404820
Yotam Shmargad, Kevin Coe, Kate Kenski, and Stephen A Rains. 2022. Social norms and the dynamics of
online incivility. Social Science Computer Review 40, 3 (2022), 717–735. https://doi.org/10.1177/089443
9320985527
Kai Shu, Amy Sliva, Suhang Wang, Jiliang Tang, and Huan Liu. 2017. Fake news detection on social
media: A data mining perspective. ACM SIGKDD Explorations Newsletter 19, 1 (2017), 22–36. https:
//doi.org/10.1145/3137597.3137600
Alexandra A Siegel. 2020. Online hate speech. In Social Media and Democracy, Nathaniel Persily and
Joshua A.Editors Tucker (Eds.). Cambridge University Press, 56–88. https://doi.org/10.1017/97811088
90960
Yanchuan Sim, Brice D. L. Acree, Justin H. Gross, and Noah A. Smith. 2013. Measuring ideological
proportions in political speeches. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing (EMNLP ’13). Association for Computational Linguistics, 91–101. https:
//aclanthology.org/D13-1010
Maneet Singh, Rishemjit Kaur, Akiko Matsuo, SRS Iyengar, and Kazutoshi Sasahara. 2021. Morality-based
assertion and homophily on social media: A cultural comparison between English and Japanese languages. Frontiers in Psychology 12 (2021), 768856. https://doi.org/10.3389/fpsyg.2021.768856
Swapna Somasundaran and Janyce Wiebe. 2009. Recognizing stances in online debates. In Proceedings of
the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on
Natural Language Processing of the AFNLP (ACL-IJCNLP ’09). Association for Computational Linguistics,
226–234. https://aclanthology.org/P09-1026
Kaitao Song, Xu Tan, Tao Qin, Jianfeng Lu, and Tie-Yan Liu. 2020. MPNet: Masked and permuted pretraining for language understanding. Proceedings of the 34th International Conference on Neural Information Processing Systems 33 (2020), 16857–16867.
Dhanya Sridhar, James Foulds, Bert Huang, Lise Getoor, and Marilyn Walker. 2015. Joint models of disagreement and stance in online debate. In Proceedings of the 53rd Annual Meeting of the Association
for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) (ACL-IJCNLP ’15). Association for Computational Linguistics, 116–125.
https://doi.org/10.3115/v1/P15-1012
Trine Syvertsen and Gunn Enli. 2020. Digital detox: Media resistance and the promise of authenticity.
Convergence 26, 5-6 (2020), 1269–1283. https://doi.org/10.1177/1354856519847325
132

Surendrabikram Thapa, Aditya Shah, Farhan Ahmad Jafri, Usman Naseem, and Imran Razzak. 2022. A
multi-modal dataset for hate speech detection on social media: Case-study of Russia-Ukraine conflict. In
Proceedings of the 5th Workshop on Challenges and Applications of Automated Extraction of Socio-political
Events from Text (CASE ’22). Association for Computational Linguistics. https://doi.org/10.18653/v1/20
22.case-1.1
Kurt Thomas, Devdatta Akhawe, Michael Bailey, Dan Boneh, Elie Bursztein, Sunny Consolvo, Nicola Dell,
Zakir Durumeric, Patrick Gage Kelley, Deepak Kumar, Damon McCoy, Sarah Meiklejohn, Thomas Ristenpart, and Gianluca Stringhini. 2021. Sok: Hate, harassment, and the changing landscape of online
abuse. In 2021 IEEE Symposium on Security and Privacy (SP ’21). IEEE, 247–267. https://doi.org/10.1109/
SP40001.2021.00028
Robin Thompson. 2011. Radicalization and the use of social media. Journal of Strategic Security 4, 4 (2011),
167–190. https://www.jstor.org/stable/26463917
Alexander Tsesis. 2002. Destructive Messages: How Hate Speech Paves the Way for Harmful Social Movements.
NYU Press.
Joshua A Tucker, Andrew Guess, Pablo Barberá, Cristian Vaccari, Alexandra Siegel, Sergey Sanovich, Denis
Stukal, and Brendan Nyhan. 2018. Social Media, Political Polarization, and Political Disinformation: A
Review of the Scientific Literature. Technical Report. Hewlett Foundation. https://www.hewlett.org/libr
ary/social-media-political-polarization-political-disinformation-review-scientific-literature/
Twitter Inc. 2021. Permanent suspension of @realDonaldTrump. Twitter Inc. Retrieved October 17, 2021
from https://blog.twitter.com/en_us/topics/company/2020/suspension.
Joseph E Uscinski, Adam M Enders, Casey Klofstad, Michelle Seelig, John Funchion, Caleb Everett, Stefan
Wuchty, Kamal Premaratne, and Manohar Murthi. 2020. Why do people believe COVID-19 conspiracy
theories? Harvard Kenny School (HKS) Misinformation Review 1, 3 (2020). https://doi.org/10.37016/m
r-2020-015
Laurens Van der Maaten and Geoffrey Hinton. 2008. Visualizing data dsing t-SNE. Journal of Machine
Learning Research 9, 11 (2008). http://jmlr.org/papers/v9/vandermaaten08a.html
Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio.
2018. Graph attention networks. In International Conference on Learning Representations (ICLR ’18).
https://openreview.net/forum?id=rJXMpikCZ
Marilyn Walker, Pranav Anand, Rob Abbott, and Ricky Grant. 2012. Stance classification using dialogic
properties of persuasion. In Proceedings of the 2012 Conference of the North American Chapter of the
Association for Computational Linguistics: Human Language Technologies (NAACl ’12). Association for
Computational Linguistics, 592–596. https://aclanthology.org/N12-1072
Hanna Wallach. 2018. Computational social science ̸= computer science + social data. Communications of
the ACM 61, 3 (feb 2018), 42–44. https://doi.org/10.1145/3132698
Joseph B Walther. 2022. Social media and online hate. Current Opinion in Psychology (2022). https:
//doi.org/10.1016/j.copsyc.2021.12.010
Emily L Wang, Luca Luceri, Francesco Pierri, and Emilio Ferrara. 2023. Identifying and characterizing behavioral classes of radicalization within the QAnon conspiracy on Twitter. In Proceedings of
the Seventeenth International AAAI Conference on Web and Social Media (ICWSM ’23, Vol. 17). 890–901.
https://doi.org/10.1609/icwsm.v17i1.22197
133

Georgia Wells, Jeff Horwitz, and Deepa Seetharaman. 2021. Facebook knows Instagram is toxic for teen girls,
company documents show. The Wall Street Journal. Retrieved April 3, 2024 from https://www.wsj.com/
articles/facebook-knows-instagram-is-toxic-for-teen-girls-company-documents-show-11631620739.
Evan M Williams, Valerie Novak, Dylan Blackwell, Paul Platzman, Ian McCulloh, and Nolan Edward
Phillips. 2020. Homophily and transitivity in bot disinformation networks. In 2020 Seventh International Conference on Social Networks Analysis, Management and Security (SNAMS ’20). IEEE, 1–7.
https://doi.org/10.1109/SNAMS52053.2020.9336579
Stefan Wojcik and Adam Hughes. 2019. Sizing up Twitter users. Pew Research Center. Retrieved December
14, 2020 from https://www.pewresearch.org/internet/2019/04/24/sizing-up-twitter-users.
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen,
Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame,
Quentin Lhoest, and Alexander Rush. 2020. Transformers: State-of-the-Art Natural Language Processing. In : Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations (EMNLP ’20). Association for Computational Linguistics, 38–45. https:
//doi.org/10.18653/v1/2020.emnlp-demos.6
Felix Ming Fai Wong, Chee Wei Tan, Soumya Sen, and Mung Chiang. 2016. Quantifying political leaning
from tweets, retweets, and retweeters. IEEE Transactions on Knowledge and Data Engineering 28, 8 (2016),
2158–2172. https://doi.org/10.1109/TKDE.2016.2553667
Shaomei Wu, Jake M. Hofman, Winter A. Mason, and Duncan J. Watts. 2011. Who says what to whom on
Twitter. In Proceedings of the 20th International Conference on World Wide Web (WWW ’11). Association
for Computing Machinery, 705–714. https://doi.org/10.1145/1963405.1963504
Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and S Yu Philip. 2020. A comprehensive survey on graph neural networks. IEEE Transactions on Neural Networks and Learning Systems
32, 1 (2020), 4–24. https://doi.org/10.1109/TNNLS.2020.2978386
Zhiping Xiao, Weiping Song, Haoyan Xu, Zhicheng Ren, and Yizhou Sun. 2020. TIMME: Twitter ideologydetection via multi-task multi-relational embedding. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD ’20). Association for Computing Machinery, 2258–2268. https://doi.org/10.1145/3394486.3403275
Kai-Cheng Yang, Emilio Ferrara, and Filippo Menczer. 2022. Botometer 101: Social bot practicum for
computational social scientists. Journal of Computational Social Science 5 (2022), 1511–1528. https:
//doi.org/10.1007/s42001-022-00177-5
Xinxin Yang, Bo-Chiuan Chen, Mrinmoy Maity, and Emilio Ferrara. 2016. Social politics: Agenda setting
and political communication on social media. In Social Informatics, Vol. 10046. Springer International
Publishing, 330–344. https://doi.org/10.1007/978-3-319-47880-7_20
Yi Yang and Jacob Eisenstein. 2017. Overcoming language variation in sentiment analysis with social
attention. Transactions of the Association for Computational Linguistics 5 (2017), 295–307. https://doi.or
g/10.1162/tacl_a_00062
Jinyi Ye, Luca Luceri, Julie Jiang, and Emilio Ferrara. 2024. Susceptibility to unreliable information sources:
Swift adoption with minimal exposure. In Proceedings of the ACM Web Conference 2024 (WWW ’24). To
Appear.
134

Daokun Zhang, Jie Yin, Xingquan Zhu, and Chengqi Zhang. 2018. Network representation learning: A
survey. IEEE Transactions on Big Data 6, 1 (2018), 3–28. https://doi.org/10.1109/TBDATA.2018.2850013
Hui Zhang, Munmun De Choudhury, and Jonathan Grudin. 2014. Creepy but inevitable? The evolution of
social networking. In Proceedings of the 17th ACM Conference on Computer Supported Cooperative Work
& Social Computing (CSCW ’14). Association for Computing Machinery, 368–378. https://doi.org/10.1
145/2531602.2531685
Jie Zhang, Yuxiao Dong, Yan Wang, Jie Tang, and Ming Ding. 2019. ProNE: Fast and scalable network
representation learning. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial
Intelligence (IJCAI ’19). 4278–4284. https://doi.org/10.24963/ijcai.2019/594
Jing Zhang, Jie Tang, Juanzi Li, Yang Liu, and Chunxiao Xing. 2015. Who influenced you? Predicting
retweet via social influence locality. ACM Transactions on Knowledge Discovery From Data 9, 3, Article
25 (2015), 26 pages. https://doi.org/10.1145/2700398
Jun Zhang, Wei Wang, Feng Xia, Yu-Ru Lin, and Hanghang Tong. 2020. Data-driven computational social
science: A survey. Big Data Research 21 (2020), 100145. https://doi.org/10.1016/j.bdr.2020.100145
Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen
Zhang, Junjie Zhang, Zican Dong, Yifan Du, Chen Yang, Yushuo Chen, Zhipeng Chen, Jinhao Jiang,
Ruiyang Ren, Yifan Li, Xinyu Tang, Zikang Liu, Peiyu Liu, Jian-Yun Nie, and Ji-Rong Wen. 2023. A survey
of large language models. arXiv preprint arXiv:2303.18223 (2023). https://arxiv.org/abs/2303.18223
135

Appendix A
This appendix provides supplementary material for Chapter 2.
A.1 Heuristics-based Pseudo-Labeling Details
We show the exact hashtags in Table A.1 and media bias ratings in Table A.2 used in the heuristics-based
pseudo-labeling of user political leanings. In the labeling process, all hashtags are treated as case insensitive.
A.2 Data Preprocessing
We restrict our attention to users who are likely in the United States, as determined by their self-provided
location (Jiang et al., 2020). Following Garimella et al. (2018b), we only retain edges in the retweet network
with weights of at least 2. Since retweets often imply endorsement (Boyd et al., 2010), a user retweeting
another user more than once would imply a stronger endorsement and produce more reliable results. As
our analyses depend on user profiles, we remove users with no profile data. We also remove users with
degrees less than 10 (in- or out-degrees) in the retweet network, as these are mostly inactive Twitter users.
136

Table A.1: Hashtags that are categorized as either left-leaning or right-leaning from the top 50 most popular
hashtags used in user profile descriptions in the COVID-19 dataset.
Left Right
Resist MAGA
FBR KAG
TheResistance Trump2020
Resistance WWG1WGA
Biden2020 QAnon
VoteBlue Trump
VoteBlueNoMatterWho KAG2020
Bernie2020 Conservative
BlueWave BuildTheWall
BackTheBlue AmericaFirst
NotMyPresident TheGreatAwakening
NeverTrump TrumpTrain
Resister
VoteBlue2020
ImpeachTrump
BlueWave2020
YangGang
A.3 Hyperparameter Tuning
All models producing user (profile and/or network) embeddings are fit with a logistic regression model for
classification. We search over parameter {C: [1, 10, 100, 1000]} to find the best 5-fold CV value. We also use
randomized grid search to tune the base models. For node2vec, the search grid is {d: [128, 256, 512, 768], l:
[5, 10, 20, 80], r: [2, 5, 10], k: [10, 5], p: [0.25, 0.5, 1, 2, 4], q: [0.25, 0.5, 1, 2, 4]}. For GraphSAGE, the search
grid is {activation: [relu, sigmoid], S1: [10, 25, 50], S2: [5, 10, 20], negative samples: [5, 10, 20]}. Both
node2vec and GraphSAGE are trained for 10 epochs with hidden dimensions fixed to 128. Retweet-BERT
is trained for 1 epoch.
137

Table A.2: The Twitter handles, media URLs, and bias ratings from AllSides.com for the popular media on
Twitter.
Media (Twitter) URL Rating
@ABC abcnews.go.com 2
@BBCWorld bbc.com 3
@BreitbartNews breitbart.com 5
@BostonGlobe bostonglobe.com 2
@businessinsider businessinsider.com 3
@BuzzFeedNews buzzfeednews.com 1
@CBSNews cbsnews.com 2
@chicagotribune chicagotribune.com 3
@CNBC cnbc.com 3
@ CNN cnn.com 2
@DailyCaller dailycaller.com 5
@DailyMail dailymail.co.uk 5
@FoxNews foxnews.com 4
@HuffPost huffpost.com 1
InfoWars* infowars.com 5
@latimes latimes.com 2
@MSNBC msnbc.om 1
@NBCNews nbcnews.com 2
@nytimes nytimes.com 2
@NPR npr.org 3
@OANN oann.com 4
@PBS pbs.org 3
@Reuters reuters.com 3
@guardian theguardian.com 2
@USATODAY usatoday.com 3
@YahooNews yahoo.com 2
@VICE vice.com 1
@washingtonpost washingtonpost.com 2
@WSJ wsj.com 3
*The official Twitter account of InfoWars was
banned in 2018.
138

Appendix B
This appendix provides supplementary material for Chapter 5.
B.1 Data Details
B.1.1 User Statistics
Table B.1 displays the raw and transformed social engagement metrics of the users in our dataset.
B.1.2 Bot Score and Homophily in Toxic Behavior
For completeness, we analyze the relationship between a user’s bot tendencies and their homophily indepth. We divide the users by their bot scores in 0.1 increments and measure the homophily in toxic
behavior among users in the same bot score bin using network assortativity in Figure B.1. Note that
there are no users with bot scores between 0.1 and 0.2. We calculate the network assortativity of both the
retweet and mention networks, which are all significant (p < 0.0001). We find that hate score network
assortativity is high for both the very human-like users (bot score bin [0.0, 0.1)) and the very bot-like (bot
score bin [0.9, 1.0]) and low otherwise. One explanation is that bots and humans do not preferentially
attach themselves to each other, but rather, bots interact mostly with bots and humans interact mostly
with humans (see Williams et al., 2020). This finding motivates our choice to separate analyses in this
work by bot scores.
139

Table B.1: The statistics of the number of followers and average numbers of likes, retweets, quotes, and
replies per user in the hate speech dataset.
Raw Metrics Transformed
Min Max Mean SD Median Min Max Mean SD Median
Bot Score <= 0.5 (N = 2, 985)
Followers 1 1,232,710.00 3,359.18 32,838.93 557.00 0.30 6.09 2.73 0.71 2.75
Likes 0 13,356.06 19.86 277.85 1.01 -3.00 1.42 -0.03 0.35 0.00
Retweets 0 1,327.37 2.91 30.00 0.17 -3.32 1.13 -0.30 0.38 -0.28
Quotes 0 116.08 0.30 3.23 0.03 -4.02 0.48 -0.53 0.36 -0.48
Replies 0 1,492.49 1.53 28.37 0.23 -3.32 0.54 -0.28 0.32 -0.24
Bot Score <= 0.8 (N = 6, 664)
Followers 1 3,627,321.00 7,093.31 86,518.45 559.00 0.30 6.56 2.70 0.87 2.75
Likes 0 37,450.54 30.43 566.38 0.80 -5.39 1.42 -0.11 0.48 -0.04
Retweets 0 5,320.85 5.71 88.32 0.17 -7.88 1.13 -0.35 0.53 -0.29
Quotes 0 489.23 0.50 7.41 0.03 -8.60 0.48 -0.57 0.52 -0.47
Replies 0 1,492.49 1.96 28.47 0.20 -7.61 0.54 -0.34 0.46 -0.28
B.2 Social Engagement Experiment Details
B.2.1 Social Engagement Estimation Model
We select the best Social-LLM models from RQ1 for use in the RQ2 modeling of social engagement estimation. The best models based on test performance all incorporated retweet edges, mention edges, profile
description, and user metadata but differed regarding how the network feature was used. For users with
bot threshold = 0.5, the best model used the network edges as undirected and unweighted. For users with
bot threshold = 0.8, the best model used network edges as undirected but weighted. We use the following
hyperparameters/settings when training the social engagement prediction model: learning rate = 0.001,
number of epochs = 20, number of dense layers = 4, hidden units = 768, batch size = 32, ReLU activation,
and batch normalization.
For users with bot threshold = 0.5, our trained models produce R2 values of 0.577, 0.515, 0.371, and
0.273 for predicting the number of likes, retweets, replies, and quotes, respectively, per tweet. For users
with bot threshold = 0.8, our trained models produce R2 values of 0.531, 0.380, 0.167, and 0.467 for predicting the number of likes, retweets, replies, and quotes, respectively, per tweet.
140

Figure B.1: Hate score network assortativity of users in the same bot score bin in the hate speech dataset.
All assortativity measures are significant (p < 0.001), indicating that users are all assortative or homophilous with each other in terms of their hate scores.
Table B.2: The change in hate score when a user received lower vs. higher than expected amount of likes,
retweets, replies, or quotes over a window of k = 50 tweets in the hate speech dataset. We test the
significance of the difference between the distributions using a Mann-Whitney U test (** p < 0.01).
Lower Than Predicted Higher Than Predicted
Metric # Ex # Users ∆ Hate # Ex # Users ∆ Hate Diff. in ∆ Hate Sig.
Likes 2,397 748 -0.001421 82,488 1,900 0.000204 0.001624 **
Retweets 67 52 0.006862 143,592 2,537 0.000073 -0.006789 N.S
Replies 4 1 -0.004567 105,176 2,746 -0.000169 0.004398 N.S.
Quote 14,693 1,034 0.000789 157,379 3,656 0.000148 -0.000641 **
141

Figure B.2: Pearson correlation between two Perspective (toxicity) scores of each tweet in the hate speech
dataset. All values are statistically significant (p < 0.05).
B.3 Additional Results
B.3.1 Results with Higher Bot Threshold
We report the results for users with bot scores <= 0.8 in Table B.2 using a time window of k = 50. We
note that due to the change in data distribution, there is no longer a sufficient number of samples where
users experienced significantly fewer retweets and replies. Therefore, we cannot adequately analyze how
retweets and replies affect the toxic behaviors of these users. The results for likes and quotes corroborate
our main findings. When users experience more likes than expected instead of fewer likes than expected,
they will become more toxic in the future. However, if they receive more quotes than expected instead
of fewer quotes than expected, they will become less toxic in the future. Likes act as a positive social
reinforcement, whereas quotes act as a negative social reinforcement.
142

Figure B.3: We display the effect of social engagement on all five toxicity attributes at k = 50 in the hate
speech dataset, similar to Figure 5.3.
B.3.2 Robustness Checks with Other Toxicity Measures.
Besides the flagship TOXICITY score, the Perspective API also computes IDENTITY_ATTACK, INSULT,
PROFANITY, and THREAT scores, which are additional hateful messaging measures we use in robustness
checks. We display the correlations among all five toxicity attributes of the tweets in our dataset in Fig
B.2. Most similar to TOXICITY is INSULT at r = 0.95 (p < 0.001), and the least similar is THREAT at a
r = 0.45 (p < 0.001).
143

To ensure the validity of our results, we repeat our analysis with the four other toxicity attributes
in Figure B.3 as a robustness check. The positive impact of likes on increased toxicity also occurs in
INSULT and PROFANITY. The reduction of toxicity due to more replies is also found in PROFANITY and
THREAT. Interestingly, retweets positively affect subsequent toxicity in all five toxicity attributes, further
corroborating our contention about the potency of retweets.
144

Abstract (if available)

Abstract The explosive growth of social media has not only revolutionized communication but also brought challenges such as political polarization, misinformation, hate speech, and echo chambers. This dissertation employs computational social science techniques to investigate these issues, understand the social dynamics driving negative online behaviors, and propose data-driven solutions for healthier digital interactions. I begin by introducing a scalable social network representation learning method that integrates user-generated content with social connections to create unified user embeddings, enabling accurate prediction and visualization of user attributes, communities, and behavioral propensities. Using this tool, I explore three interrelated problems: 1) COVID-19 discourse on Twitter, revealing polarization and asymmetric political echo chambers; 2) online hate speech, suggesting the pursuit of social approval motivates toxic behavior; and 3) moral underpinnings of COVID-19 discussions, uncovering patterns of moral homophily and echo chambers, while also indicating moral diversity and plurality can improve message reach and acceptance across ideological divides. These findings contribute to the advancement of computational social science and provide a foundation for understanding human behavior through the lens of social interactions and network homophily.

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

Measuing and mitigating exposure bias in online social networks

PDF

Improving information diversity on social media

PDF

Mining and modeling temporal structures of human behavior in digital platforms

PDF

Learning social sequential decision making in online games

PDF

Modeling social and cognitive aspects of user behavior in social media

PDF

The spread of moral content in online social networks

PDF

Predicting and modeling human behavioral changes using digital traces

PDF

Modeling information operations and diffusion on social media networks

PDF

Three text-based approaches to the evolution of political values and attitudes

PDF

Artificial intelligence for low resource communities: Influence maximization in an uncertain world

PDF

Bound in hatred: a multi-methodological investigation of morally motivated acts of hate

PDF

Analysis and prediction of malicious users on online social networks: applications of machine learning and network analysis in social science

PDF

Towards social virtual listeners: computational models of human nonverbal behaviors

PDF

Learning distributed representations from network data and human navigation

PDF

Computational modeling of human behavior in negotiation and persuasion: the challenges of micro-level behavior annotations and multimodal modeling

PDF

Responsible artificial intelligence for a complex world

PDF

How misinformation exploits moral values and framing: insights from social media platforms and behavioral experiments

PDF

Understanding diffusion process: inference and theory

PDF

Statistical approaches for inferring category knowledge from social annotation

PDF

Towards socially assistive robot support methods for physical activity behavior change

Asset Metadata

Creator Jiang, Julie (author)

Core Title Socially-informed content analysis of online human behavior

School Viterbi School of Engineering

Degree Doctor of Philosophy

Degree Program Computer Science

Degree Conferral Date 2024-05

Publication Date 05/17/2024

Defense Date 05/03/2024

Publisher Los Angeles, California (original), University of Southern California (original), University of Southern California. Libraries (digital)

Tag artificial intelligence,big data,Communication,computational social science,computer science,echo chambers,hate speech,human-computer interaction,machine learning,misinformation,moral foundations,OAI-PMH Harvest,online human behavior,online human behavior mining,polarization,social network analysis,social science

Format theses (aat)

Language English

Contributor Electronically uploaded by the author (provenance)

Advisor Ferrara, Emilio (committee chair), Barberá, Pablo (committee member), Lerman, Kristina (committee member), Twyman, Marlon II (committee member)

Creator Email yioujian@usc.edu

Permanent Link (DOI) https://doi.org/10.25549/usctheses-oUC113939987

Unique identifier UC113939987

Identifier etd-JiangJulie-12947.pdf (filename)

Legacy Identifier etd-JiangJulie-12947

Document Type Dissertation

Format theses (aat)

Rights Jiang, Julie

Internet Media Type application/pdf

Type texts

Source 20240517-usctheses-batch-1154 (batch), University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Access Conditions The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the author, as the original true and official version of the work, but does not grant the reader permission to use the work if the desired use is covered by copyright. It is the author, as rights holder, who must provide use permission if such use is covered by copyright.

Repository Name University of Southern California Digital Library

Repository Location USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA

Repository Email cisadmin@lib.usc.edu