The emergence of the novel coronavirus (COVID-19) has caused 5,878,328 confirmed deaths worldwide as of February 2022, along with 423,437,674 confirmed infections [1]. Despite vaccine availability, vaccine hesitancy has inhibited public health officials’ efforts to mitigate the COVID-19 pandemic, especially in the United States [2]. Although some US elected officials have responded by issuing vaccine mandates, others have amplified vaccine hesitancy by broadcasting messages that minimize vaccine efficacy [3,4].

With 68% of American adults reporting social media as a source of their news diet [5], social media platforms such as Twitter have become important communication channels for US politicians to share their agendas [6]. As a result, social media have become a prominent source of political information and misinformation, including information surrounding COVID-19 vaccines [7-11]. The politically polarized nature of COVID-19 information on social media has given rise to an infodemic, wherein health attitudes often hinge more on political ideology than science [12-15]. As a result, political affiliation influences negative sentiment toward the vaccine [16]. Such negative sentiment may foster uncivil discourse toward the vaccines and mandates [17,18].

Incivility on social media platforms has been widely studied and discussed in both political and health contexts, among others [19-25]. However, to the best of our knowledge, incivility has not been studied in the context of discourse regarding COVID-19 vaccines and mandates. Specifically, there is little focus on the psychological processes that elicit uncivil vaccine discourse. We aimed to bridge this gap by conducting a computational analysis of tweets. In this study, we investigated the role of negative emotion in predicting uncivil posts about COVID-19 vaccine mandates on Twitter. Ultimately, we argue that a more nuanced understanding of the psychological processes underlying uncivil vaccine discourse has practical implications for public health interventions.

Incivility has become a salient point of discussion in social media research. However, scholars across fields have found it difficult to conceptualize incivility. Incivility has been defined in a variety of ways, including impoliteness, profanity, and specific discriminatory acts (eg, former US president Trump caught on a hot mic in 2016 praising nonconsensual sexual encounters with women) [26-29]. Papacharissi [29] supplements this definition by including threat—in this case to democracy—as uncivil. Other scholars have operationalized incivility as including the use of all capital letters, accusations of lying, pejorative speech, ideologically extreme language, exaggerated argument, and misinformation [26,30-33]. Despite these inconsistent operationalizations, incivility is a concept that is nuanced and varies across individuals, perhaps because it is bound by cultural perceptions and understandings of what uncivil discourse is [16,18]. Informed by the operational inconsistency of incivility outlined in the literature, we conceptualize incivility as a multifaceted construct encompassing a diversity of uncivil behaviors, including toxicity, severe toxicity, profanity, threats, insults, and identity attacks in discourse. Recent studies have argued that uncivil behaviors are related to toxicity on social media platforms [34]. Tromble [28] asserts that profanity and insulting language constitute key indicators of uncivil behaviors. Likewise, scholars have argued that identity attacks and threatening language that aims to morally attack individuals or groups are also aspects of incivility and uncivil discourse [35]. We now shift our attention to explaining what causes incivility.

Incivility does not have a single cause; instead, varying forms of uncivil behaviors are a result of diverse psychological processes. For example, a user may post profane content because they are anxious, whereas a user might make an insulting comment because they are angry. However, scholars often obscure these distinct underlying psychological mechanisms by conceptualizing incivility as a one-dimensional process with a unitary explanation [19,21]. In the context of COVID-19 vaccines and mandates, emotional responses such as anger and anxiety among other negative emotions are salient in the discourse about the pandemic [36,37]. In fact, studies have found negative emotions such as anger and anxiety to play a role in driving vaccine hesitancy [38]. We investigated 3 psychological processes that are likely to predict discourse incivility—namely, anxiety, anger, and sadness.

The sample comprised posts shared to Twitter, a popular platform for seeking and sharing health information on the web, including (mis)information about vaccination and vaccines [7-11]. We opted to curate a list of vaccine-related words and scraped tweets containing those words. We curated a list of words that we believed would collect tweets related to the vaccine, without introducing bias into the data set. For example, “shot” was not included, because we noticed that it scraped tweets about gunshots, which are unrelated to the COVID-19 vaccine. The text of the 8014 tweets contained terms related to COVID-19 vaccine mandates (eg, “Moderna,” “required,” and “mandating”) from September 14, 2020, to October 1, 2021. See Figure 1 for a flowchart of the data collection process.

Twitter’s code-free application programming interface (API) Tweet Downloader Tool (version 2) was used to extract posts about COVID-19 vaccine mandates. We were interested in words that would identify tweets about COVID-19 vaccine mandates rather than the COVID-19 vaccine generally. Thus, we filtered through a data set of 375,000 vaccine-related tweets posted from September 14, 2020, to October 1, 2021, to extrapolate tweets specifically related to vaccine mandates (eg, “forcing,” “required,” and “mandating”) from September 14, 2020, to October 1, 2021; the final sample contained 8014 tweets.

The data were analyzed using 2 different natural language processing approaches: (1) the Linguistic Inquiry and Word Count (LIWC) computational tool [59] and (2) the Google Perspective API [60].

No personally identifiable information was included in this study. The institutional review board recognizes that the analysis of publicly available data does not constitute human subjects research. This study only used information in the public domain; thus, ethical review and approval was not required.

Prior to hypothesis testing, we conducted a repeated measures ANOVA to assess whether to model dimensions of incivility together or separately. The main effect for the within-subjects factor was significant (F_4,32052=930.44; P<.001), indicating significant differences among identity attack, insult, profanity, threat, and severe toxicity (see Table 2).

Tukey comparisons were used to test marginal mean differences in each combination of incivility dimensions. There were significant differences between each combination, except identity attack and profanity (see Table 3). Thus, we concluded that the 5 dimensions of incivility should be assessed separately in the main analysis.

Incivility has been understood as a multifaceted construct, encompassing the breadth of conceptual and operational definitions offered in the literature. This study resolved discrepant operationalizations of incivility by introducing incivility as a multifaceted construct and explored the distinct emotional processes underlying 5 dimensions of discourse incivility. The findings reveal that 3 types of emotions—anxiety, anger, and sadness—were significantly associated with dimensions of incivility. With regard to the relationship between anxiety and incivility, we found that the anxiety was negatively associated with identity attacks and profanity expressed in Twitter posts. Individuals who expressed higher levels of anger were more likely to engage in all 5 dimensions of incivility, including profanity, insults, severe toxicity, identity attacks, and threats. Lastly, our findings revealed that sadness was positively associated with uncivil behavior, especially threats.

Individuals who expressed higher anxiety were less likely to engage in uncivil behaviors such as posting hateful comments targeting individuals with a specific identity or using profane language in their posts. We suspect that individuals’ anxiety may have decreased the level of uncivil expressions about vaccine mandate policy, because individuals who are anxious about COVID-19 and its health consequences are more likely to seek ways to contain the threat (ie, spread of COVID-19) and exhibit positive attitudes and behaviors toward policies related to restricting the spread of COVID-19. Namely, when novel threatening stimuli are encountered and feelings of anxiety are induced, people may be motivated to attend to the issue at hand [66]. In line with this idea, previous studies suggest that anxiety can be an indicator of a “functional fear” that predicts individuals’ positive attitudes and behaviors (eg, compliance) toward COVID-19–related measures and policies [67]. For instance, an extant work shows that COVID-19–related anxiety and health-related fears were associated with more protective health behaviors and higher vaccine acceptance [68,69].

It is noteworthy that anger, unlike anxiety or sadness, predicted all dimensions of incivility, demonstrating that this emotion is the strongest predictor of incivility.

Evidence from previous studies has shown that prolonged risk and uncertainty about the level of risk can elicit anger and conflict within the community [70]. People have experienced increased levels of anger during the pandemic [71], and those who express anger have also exhibited disbelief toward COVID-19 vaccines [72]. Moreover, it has been shown that political polarization regarding the issues of vaccination and vaccine mandates has further fueled public outrage among groups with conflicting political views [51,52]. Thus, the strong association between anger and uncivil behaviors can be due to both social disruptions caused by the wide spread of COVID-19 and political conflicts partly induced by media outlets.

Lastly, as the level of sadness increased, individuals were more likely to exhibit verbal intentions to inflict pain and hurt other individuals or groups. Such aggression toward other people, especially exhibiting intentions to hurt others, may be explained by depression and victimhood. Approximately over 2 years of the COVID-19 pandemic, individuals worldwide have experienced prolonged social isolation and lifestyle disruptions, which have led them to be depressed [56,57]. Furthermore, the direct health impacts of the spread of COVID-19 have caused many individuals to become the victims of multiple losses such as a loss of financial security, loss of family members, and loss of physical/mental health and general safety [73,74]. However, sadness may have been strongly associated with viewing themselves as victims of COVID-19, which could have led them to issue threats to others who were favorable toward vaccine mandates. Additionally, this victimhood mentality [58] among the unvaccinated may have also been high because they are prohibited from participating in relationships and activities available to those who are vaccinated. This prohibition may have led them to feel socially excluded and in turn prompt threats toward the outgroup members—proponents of vaccine mandates.

Although the findings shed light on the psychological processes underlying vaccine mandate incivility, this study is not without limitations. The LIWC computational tool does not measure the nuances afforded by human coders. Although we endeavored to minimize this limitation by using well-validated measures [59], future work might employ human coders to analyze the specific topics related to uncivil discourse. Additionally, we focused on posts shared to Twitter and therefore cannot generalize our findings about incivility to other social media platforms. Given the role of platform community norms in predicting incivility, future work should investigate how incivility manifests itself on different platforms. Likewise, Twitter users are wealthier, younger, and more liberal than the wider population of Americans [75], and the sample was limited to English-speaking Twitter users, which makes it difficult to generalize our findings to the entire US population. Additionally, we acknowledge that social media posting data could have been biased based on individuals’ geographical area (eg, city and state), whether they were local residents or visitors in the area at the time of the post, as well as the types of activities completed during the course of a day [76,77]. These factors may have contributed to our study findings. Lastly, we did not measure how many different users were included in each stage in the data collection process. Future work should elucidate the extent to which a small number of active users produce uncivil vaccine mandate content.

The results suggest that our multidimensional approach to incivility is a promising alternative to understanding and intervening in the psychological processes underlying uncivil vaccine discourse. Given the need for real-time monitoring and automated responses to the spread of health information and misinformation on the web, social media platforms can harness the Google Perspective API to offer users immediate, automated feedback when it detects that a comment is uncivil [78]. Furthermore, the Perspective API is available in 17 languages—from Arabic to Korean, enabling the study of uncivil health discourse in non-English posts. Future work should explore cross-cultural differences in uncivil health discourse.

Vaccine hesitancy still remains a threat to global health, and this work demonstrates that distinct emotional processes underlie distinct attitudes toward vaccines and vaccine-related policies. It is important for health practitioners and policy makers to first acknowledge negative emotions associated with vaccines and vaccine mandates while emphasizing the safety of COVID-19 vaccines in health campaigns, which would provide aid in reducing vaccine hesitancy. One avenue public health officials can take to combat vaccine hesitancy while simultaneously affirming discrete negative emotions toward the vaccine is by holding COVID-19 community listening sessions, where officials can hear directly from communities about COVID-19 concerns, including vaccination (see Figure 3 for an overview) [79]. After officials have a better understanding of the specific emotional processes underlying a communities’ vaccine hesitancy, public health campaigns can tailor messages to address these concerns (see Figure 3) [80,81].