This paper is in the following e-collection/theme issue:

Information Demand (32) JMIR Theme Issue: COVID-19 Special Issue (2452) Theme Issue: Novel Coronavirus (COVID-19) Outbreak Rapid Reports (1517) Information Seeking, Information Needs (465) COVID-19 Vaccination (146) Infoveillance and Social Listening (209)

Published on in Vol 1, No 1 (2021): Jan-Dec

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/32127, first published .
Online Search Behavior Related to COVID-19 Vaccines: Infodemiology Study

Online Search Behavior Related to COVID-19 Vaccines: Infodemiology Study

Online Search Behavior Related to COVID-19 Vaccines: Infodemiology Study

Authors of this article:

Lawrence An1, 2 Author Orcid Image ;   Daniel M Russell3 Author Orcid Image ;   Rada Mihalcea4 Author Orcid Image ;   Elizabeth Bacon1 Author Orcid Image ;   Scott Huffman3 Author Orcid Image ;   Ken Resnicow1, 5 Author Orcid Image
Article Authors Cited by (20) Tweetations (8) Metrics

    Original Paper

    1Center for Health Communications Research, Rogel Cancer Center, University of Michigan, Ann Arbor, MI, United States

    2Division of General Medicine, School of Medicine, University of Michigan, Ann Arbor, MI, United States

    3Google, Mountain View, CA, United States

    4Computer Science and Engineering Division, College of Engineering, University of Michigan, Ann Arbor, MI, United States

    5Department of Health Behavior & Health Education, University of Michigan School of Public Health, Ann Arbor, MI, United States

    Corresponding Author:

    Lawrence An, MD

    Center for Health Communications Research

    Rogel Cancer Center

    University of Michigan

    North Campus Research Complex, Building 16

    2800 Plymouth Rd.

    Ann Arbor, MI, 48109

    United States

    Phone: 1 734 763 6099

    Email: lcan@med.umich.edu


    Background: Vaccination against COVID-19 is an important public health strategy to address the ongoing pandemic. Examination of online search behavior related to COVID-19 vaccines can provide insights into the public's awareness, concerns, and interest regarding COVID-19 vaccination.

    Objective: The aim of this study is to describe online search behavior related to COVID-19 vaccines during the start of public vaccination efforts in the United States.

    Methods: We examined Google Trends data from January 1, 2021, through March 16, 2021, to determine the relative search volume for vaccine-related searches on the internet. We also examined search query log data for COVID-19 vaccine-related searches and identified 5 categories of searches: (1) general or other information, (2) vaccine availability, (3) vaccine manufacturer, (4) vaccine side-effects and safety, and (5) vaccine myths and conspiracy beliefs. In this paper, we report on the proportion and trends for these different categories of vaccine-related searches.

    Results: In the first quarter of 2021, the proportion of all web-based search queries related to COVID-19 vaccines increased from approximately 10% to nearly 50% of all COVID-19–related queries (P<.001). A majority of COVID-19 vaccine queries addressed vaccine availability, and there was a particularly notable increase in the proportion of queries that included the name of a specific pharmacy (from 6% to 27%; P=.01). Queries related to vaccine safety and side-effects (<5% of total queries) or specific vaccine-related myths (<1% of total queries) were uncommon, and the relative frequency of both types of searches decreased during the study period.

    Conclusions: This study demonstrates an increase in online search behavior related to COVID-19 vaccination in early 2021 along with an increase in the proportion of searches related to vaccine availability at pharmacies. These findings are consistent with an increase in public interest and intention to get vaccinated during the initial phase of public COVID-19 vaccination efforts.

    JMIR Infodemiology 2021;1(1):e32127

    doi:10.2196/32127

    Keywords

    online health information; behavior; search; COVID-19; vaccine; infodemiology; internet; trend; public health; awareness; concern; interest; public; misinformation; safety; side effect; availability 


    We are currently in the midst of a global pandemic caused by COVID-19. At all times, and particularly during a pandemic, it is critical for the public to have access to timely and accurate health information [1-3]. The internet is a major source of such health information [4-7]. Analysis of health-related web search behavior can provide critical insight into the public’s awareness and interest in specific health issues and their health concerns and information needs, health experiences, and health-related intentions and behaviors [8-11].

    Infodemiology is the scientific study of the “distribution and determinants of information in an electronic medium, specifically the internet, or in a population, with the ultimate aim to inform public health and health policy” [8]. To date, a number of infodemiology studies related to the COVID-19 pandemic have been published. Many of these studies identified an association between general COVID-19–related or symptom-specific search trends and COVID-19 case incidence and associated deaths [12-18]. In several cases, search behavior appeared to be an effective predictor of disease trends. Several studies have examined the occurrence and spread of COVID-19–related misinformation, which has been a public health challenge during the pandemic [19-25]. Another group of studies has examined search behavior to gain insights into public awareness, interest, attitudes, and behaviors related to COVID-19 [21,26-28]. Husain et al [27] found that countries that demonstrated a more rapid increase in public search interest regarding COVID-19 also tended to be more effective in their control of the pandemic.

    Vaccination against COVID-19 is a major public health strategy in the effort to end the pandemic [29]. As of Spring 2021, a number of effective COVID-19 vaccines have been available that substantially reduce the risk of COVID-19–related illness, hospitalizations, and death [30]. Understanding public awareness and interest in COVID-19 vaccines and willingness to vaccinate are critical to help guide vaccination efforts. Unfortunately, vaccine hesitancy is common and poses a major barrier to successful vaccination efforts [31]. Infodemiologic approaches can provide potentially important insights into the public’s awareness, interests, concerns, and intentions related to COVID-19 vaccination. Thus far, few infodemiological studies have focused on COVID-19 vaccines [32,33]. To address this critical gap, we examined the relative search volume using Google Trends data and also search query logs capturing users’ online search behaviors related to COVID-19 vaccines in the first quarter of 2021.


    Study Design

    This study describes users’ online search behavior via Google search engine for searches related to COVID-19 vaccines in the first quarter of 2021. Google is the dominant search engine in the United States, accounting for approximately 89% of the total search volume in the country as of January 2021 [34]. We focused on the time period from January 1, 2021, through March 16, 2021, which follows the initial emergency-use authorization by the US Food and Drug Administration (FDA) for the Pfizer and Moderna COVID-19 vaccines and the start of public vaccination efforts.

    Data Sources

    Google Trends

    Google Trends provides open access to time-series data related to Google search engine search volumes for specific terms [35]. Search query volume was normalized to a percentage scale (0% to 100%) to provide a measure of relative search volume (RSV), with 100% corresponding to the peak in search volume in any given time frame for that specific topic. By searching for multiple terms simultaneously, we were able to compare the RSV for different terms. For this report, we focus on Google Trends data for COVID-19 vaccine-related searches in the United States.

    Search Query Logs

    Search engine query logs record the specific language that users employ when conducting online searches and can provide insight into the users' information needs and interests and how these change over time [36-38]. To examine COVID-19 vaccine-related search behavior in greater detail, we compiled anonymized data from Google search query logs. That is, we examined a complete sample of English-language queries conducted in the United States during the search period (January 1, 2021, to March 16, 2021). We collected only queries that contain both the terms “COVID” and “vaccine.” This data set comprises over 45.4 million queries during the sampling period, which suggests that when people search for information about COVID-19 vaccines, they use a fairly limited number of common queries. For example, the top 150 most common queries related to COVID-19 account for approximately half of all queries in the data set. The distribution of search queries by volume is shown in Figure 1. This figure shows that a small number of queries accounts for a large proportion of the overall query volume. This query volume distribution curve essentially becomes asymptotic after the top 1000 most common queries. To create this search query log data set, we collected the top 5000 most common COVID-19 vaccine-related search queries for each day during the study period.

    ‎
    Figure 1. Frequency distribution of the 1000 most common search queries related to "COVID-19 vaccine.".
    View this figure

    Metrics

    Overview

    Metrics of interest for this study are based upon both Google Trends and the search query log data set. Specific Google Trend metrics include (1) comparison of the RSV for any searches related to COVID-19 and those related to COVID-19 vaccine, (2) general RSV for the term “vaccine” since 2005, and (3) comparison of RSV for specific vaccine myths and conspiracy beliefs identified in the search query logs.

    Our main interest with the search query log data set is to examine the distribution and trends of different types of COVID-19 vaccine-related searches. We developed a COVID-19 vaccine search query classifier using the following steps.

    Step 1: Identify Search Categories

    Two study authors (DMR and LA) performed independent manual review of a random sample of 1000 queries to identify common themes, as well as unique terms associated with each category. The two authors met to review and resolve any discrepancies and reached complete agreement on both categories and associated terms. Based on this review, we identified the following categories of search queries: (1) vaccine availability, (2) vaccine maker or manufacturer, (3) vaccine side effects or safety, (4) vaccine myth or conspiracy beliefs, and (5) general or other vaccine-related searches. A definition of each of these categories of searches, associated search terms, and examples is shown in Table 1. Because pharmacies were a major channel for distribution of COVID-19 vaccines, we also created a subcategory of vaccine availability queries that asked about COVID-19 vaccines in relation to pharmacies (eg, included the name of specific pharmacy chains).

    Table 1. Types of COVID-19 vaccine–related search queries.
    CategoryDefinitionAssociated termsExamples of specific queries
    AvailabilityQuery that included a term or phrase identifying locations where or time when COVID-19 vaccines might be availableNames of US states, counties, or cities, names of organizations or specific locations that provide COVID-19 vaccines (eg, pharmacies, hospitals or health systems, vaccination sites), when or where to get COVID-19 vaccines“ny covid vaccine”, “covid vaccine california”, “florida covid vaccine”, “covid vaccine near me”, “where to get covid vaccine”, “cvs covid vaccine”, “covid vaccine rite-aid”, “covid vaccine appointment”, “when can I get covid vaccine”
    Maker or manufacturerQuery that included the name of a COVID-19 vaccine maker or manufacturerNames of different COVID-19 vaccines, names of companies or organizations that developed or manufactured different vaccines“pfizer vaccine”, “moderna vaccine”, “johnson vaccine”, “j&j vaccine”
    Side effects or safetyQuery that included general or specific terms associated with side effects or safety of COVID-19 vaccinesSide effects, safety, specific vaccine-related worries and concerns“covid vaccine side effects”, “covid vaccine safety”, “reaction to covid vaccine”, “pregnant women covid vaccine”, “covid vaccine blood clot”, “problems with covid vaccine”, “covid vaccine fever”, “covid vaccine allergy”
    Myths or conspiraciesQuery that included general or specific terms associated with COVID-19 vaccine myths or conspiracy beliefsSpecific myths or conspiracy beliefs“covid vaccine infertility”, “does covid vaccine change dna”, “covid vaccine microchip”, “can I get covid from vaccine”, “covid vaccine 5G”
    General or otherQuery related to COVID-19 vaccine that included no additional terms or terms not associated with any of the above categoriesCOVID-19 vaccine or vaccination or other topics other than identified above“covid vaccine”, “covid-19 vaccine”, “coronavirus vaccine”, “covid vaccine update”, “covid vaccination rates”
    Step 2: Identify Terms Associated With Each Search Category

    One study author (LA) then manually reviewed an additional random sample of 5000 queries to identify any additional terms that might be associated with each search category. The results of this review were discussed with additional authors (DMR, KR, and RM) to create a final list of unique search query terms associated with each search category.

    Step 3: Create and Apply Search Query Classifier

    We created a rules-based classifier that assigned a query to one or more of the 5 categories based on the presence of a unique set of associated terms while accounting for common variations in spelling. Some search queries contained terms associated with multiple categories, and these queries were counted separately in each appropriate category. For example, a search for “Pfizer covid vaccine CVS” would be counted as a query related to vaccine availability (given the presence of the name of a specific pharmacy) and also as a query related to vaccine manufacturer H (given the presence of the name of a specific vaccine manufacturer).

    Step 4: Evaluate Performance of the Search Query Classifier

    This classifier was able to classify 90% of all queries in the entire COVID-19 vaccine search query log dataset. The remaining 10% of unclassified queries represented searches for additional vaccine-related information (eg, “covid vaccination rates,” or “how long does the covid vaccine last”) and are labeled as “other” and included as part of the category of “general or other” searches. After application of the classifier, an additional random sample of 1000 search queries with classifier results was reviewed separately by 2 authors (DMR and LA) to assess the accuracy of the classifier. These authors met to resolve any discrepancies in manual review and the results of this review were used to calculate the classifier's precision and recall for each of the search categories. The classifier performed well with precision of 99.8% to 100% and recall over 99.5% across all search query categories.

    Analysis

    Based on the search query log data set, we employed linear regression to examine the time trends for the proportion of different types of COVID-19 vaccine-related searches over time. For each week during the study period, we calculated a proportion corresponding to each of our categories of interest. The proportions examined in these analyses correspond to the above-described categories. The proportion of COVID-19 vaccine-related searches in each of these categories serves as the dependent variable in separate linear regression models. In each of these models, time (ie, week number since start of the study period) serves as the independent variable to examine the significance of trends over time.

    Ethics Review

    This was reviewed by the University of Michigan Institutional Review Board and judged to be exempt based upon its use of open access and anonymized aggregate data.


    Overview

    The relative search volumes for any searches related to COVID-19 and those related to COVID-19 vaccines are shown in Figure 2. This figure shows a clear increase in the relative volume of COVID-19 vaccine–related searches over the study period.

    In the beginning of January 2021, approximately 10% of all COVID-19–related queries were about vaccines. By March 2021, nearly 50% of all COVID-19–related searches were vaccine related. A linear regression was calculated to predict the fraction of queries about COVID-19 vaccines based on daily change during the sample period. A significant linear regression model was found (df=103; R2=0.76; beta coefficient for time=.31; P<.001), indicating that the RSV for COVID-19 vaccine queries increased over the study period.

    Figure 3 provides a broader historical context for the level of vaccine-related search interest. This figure shows the relative search volume for the term “vaccine” from January 2005 through the first quarter of 2021. The small peak in vaccine-related search volume occurring in October 2009 coincides with the H1N1 influenza epidemic [39]. The peak in vaccine-related searches in early 2021 is several fold higher than this prior peak in 2009.

    A breakdown of the proportion and trends for different types of COVID-19 vaccine-related queries based upon search query log data is shown in Figure 4. Trends for the proportions of different categories of COVID-19 vaccine-related searches are described below.

    ‎
    Figure 2. Relative search volume (RSV) for the terms “COVID” (blue) and “COVID vaccine” (red) from September 2020 through March 2021.
    View this figure
    ‎
    Figure 3. Relative search volume for the term “vaccine” from January 2005 through March 2021.
    View this figure
    ‎
    Figure 4. Trends for different categories of COVID-19 vaccine search queries: (A) overall availability and pharmacy, (B) vaccine manufacturer, (C) side effects and safety, and (D) myths and conspiracy beliefs.
    View this figure

    Vaccine Availability

    During the study period, a majority of searches were classified as related to vaccine availability, with the specific proportion ranging from 55% to 69% each week (Figure 4A). The high proportion of vaccine availability searches was consistent over the study period. Linear regression showed that the time trend for the proportion of queries related to vaccine availability during the study period was not significant (df=8; R2=0.19; beta coefficient for time=.81; P=.20).

    During the study period, there was a substantial increase in the subcategory of searches related to pharmacies. The proportion of COVID-19 vaccine-related queries that included a specific pharmacy name increased from 5.9% at the start of the study period to 27.2% at the end of the study period (Figure 4A). Linear regression showed the time trend for this change was positive and significant (df=8; R2=0.56; beta coefficient for time=2.51; P=.01).

    Vaccine Manufacturers

    Over the same period, the proportion of vaccine manufacturer-related searches (eg, Pfizer, Moderna, Janssen or Johnson & Johnson) averaged 10.4%. Linear regression shows that the time trend for the proportion of vaccine manufacturer-related searches during the study period was not significant (df=8; R2=0.38; beta coefficient for time=.61; P=.06).

    Side Effects and Safety

    The proportion of searches related to side effects or safety of COVID-19 vaccines was quite small, and the proportion actually decreased slightly (from 5.7% to 4.2%) over the study period. Linear regression showed a negative time trend for the proportion of queries related to vaccine side effects or safety during the study period (df=8; R2=0.51; beta coefficient for time=–0.23; P=.02).

    Myths

    During the study period, the overall proportion of COVID-19 vaccine-related queries that included mention of myths or conspiracies related to COVID-19 vaccines was quite low. This proportion actually decreased slightly (from 0.4% to 0.1%) over the study period. Linear regression showed a negative time trend for the proportion of queries related to vaccine myths or conspiracy beliefs during the study period (df=8; R2=0.58; beta coefficient for time=–0.01; P=.01). Searches related to specific myths or conspiracy beliefs included searches related to the COVID-19 vaccine and (1) infertility, (2) potential to cause change in DNA, (3) 5G and the vaccine, (4) microchips, and (5) contracting COVID-19 from the vaccine itself. The Google Trends RSV for searches related to these specific myths and conspiracy topics is presented in Figure 5. This figure reveals that searches related to COVID-19 vaccine and “infertility” and “DNA” were the most common vaccine myth-related searches.

    ‎
    Figure 5. Google Trends relative search volume for queries related to specific COVID-19 vaccine myths and conspiracy beliefs (red: vaccine + DNA, blue: vaccine + fertility, green: vaccine + microchip, gold: vaccine + 5G, and purple: “COVID from vaccine”).
    View this figure

    Principal Findings

    This study reports on online search behavior related to COVID-19 vaccines in the United States, from the first quarter of 2021. During this period, there was a clear increase in the volume of online searches for information about COVID-19 vaccines with a consistently high proportion of searches related to vaccine availability. Online search behavior is influenced strongly by external events and associated media coverage [40,41]. Critical events that are likely drivers of the observed patterns in online search behavior include emergency-use authorizations by the US FDA for 3 different vaccines (ie, Pfizer on December 11, 2020, Moderna on December 18, 2020, Johnson & Johnson on February 26, 2021) and the beginning of public vaccination efforts. During this time, web-based registration was also one of the major means to obtain a vaccine appointment. A particularly notable rise in the proportion of COVID-19 vaccine searches that include the names of specific pharmacies is consistent with the US national strategy that featured pharmacies as vaccination sites.

    We interpret these patterns of online search behavior related to COVID-19 vaccines, particularly the rise in pharmacy-related searches, as a sign of increased readiness and intentions to vaccinate among the US population during the study period. This interpretation is consistent with the findings of national tracking surveys in the United States that demonstrate a similar increase in intentions to vaccinate over the study period. Specifically, the Kaiser Family Foundation COVID-19 Tracking Survey shows that the proportion of US adults that either had been vaccinated or would want to get vaccinated as soon as possible increased from under 40% to over 60% during the study period [42]. The increase in online searches related to COVID-19 vaccines is also consistent with national survey findings that reported an increase in the proportion of US adults who reported they had “enough information about when or where to get the vaccine” also over the same period [42].

    It is interesting to consider how to interpret our findings regarding online search behavior related to information about COVID-19 vaccine side effects and safety. National surveys conducted during the study period show that among the majority of adults who had not yet been vaccinated, a substantial proportion were concerned about the long-term effects of COVID-19 vaccines (68%), the potential for serious side effects from COVID-19 (59%), or that the vaccines may not be safe (55%) [43]. Given the prevalence of these concerns about COVID-19 vaccine side effects and safety, the relatively low proportion and decreasing time trend for vaccine-related searches that addressed these topics is somewhat surprising. The low proportion of COVID-19 vaccine-related searches pertaining to side effects or safety could be due to a relatively low rate of active information seeking about these aspects among those who are hesitant to get the vaccine, active searching on the web for vaccine appointments among those individuals highly motivated to obtain the vaccine, or some combination of these factors. In such a situation (where some segments of the population are highly motivated to search actively for information while others are not), the relative frequency of searches related to different topics does not appear to provide a good representation of the level of public concern or interest in these topics.

    Our findings regarding the frequency of online searches related to COVID-19 vaccine myths or conspiracy theories can be similarly interpreted. Belief, or at least uncertainty, regarding COVID-19 vaccine myths is unfortunately common. During the same time period as this study, national surveys show that 34% of adults in the United States who had not been vaccinated either believed or were unsure about one or more common COVID-19 vaccine-related myths [43]. Nevertheless, at the same time, we found less than 1% of online searches related to COVID-19 vaccines addressed these topics and that this proportion actually decreased over the study period. These findings suggest that many individuals who either believe or are unsure about COVID-19 vaccine myths are not actively seeking additional information online to help clarify their understanding. Although making inferences regarding the population prevalence of a particular COVID-19 vaccine myth may be difficult based upon analysis of search behavior, it is possible that relative search volume might be useful in helping to assess which particular myths are more or less common. For example, the results of our RSV comparison for specific vaccine myths, direct evaluation of COVID-19 vaccine misinformation on the web, and national population surveys identify concerns about infertility related to COVID-19 vaccines as among the most common [33,43].

    Limitations

    Several limitations need to be considered while interpreting the results of this study. First, it is important to acknowledge these findings are based on an analysis of search behavior in the United States and are likely to be influenced by specific vaccine approvals and distribution plans in this country. Future work is needed to determine how COVID-19–related vaccine search behavior might differ across countries. Second, these results apply only to search queries conducted using the Google search engine. Although Google is the dominant search engine in the United States, future work is needed to understand how search behavior described here is similar or different for other search engines or on other platforms. For example, social media has been identified as a major source of exposure to COVID-19–related misinformation. Our finding of relatively low rates of active searching for COVID-19 vaccine myths and conspiracy beliefs might or might not apply to search behavior within social media platforms. Third, it is important to acknowledge the subjective nature of our approach for identifying and defining search themes and categories. Other teams could have certainly chosen to identify or define search categories (or subcategories) in other ways. Fourth, of relevance, we also acknowledge the uncertainty regarding our interpretation of the observed patterns of online search behavior representing an increase in interest or intention among the study population to take the COVID-19 vaccine. Although we believe the major study finding that a large increase specifically in pharmacy-related COVID-19 vaccine queries strongly suggests active searching on the web for the vaccine, future work that directly assesses users' information needs (eg, near-time or real-time surveys) would be needed to confirm this interpretation. Finally, it is important to note that the results presented here are based on the aggregate search volume measured at the population level. We, therefore, are not able to determine the degree to which changing patterns in online search behavior are due to changes in the number of individuals performing a specific search, or due to an increase in the number of searches performed by specific individuals, or some combination of these factors.

    Conclusions

    Despite these limitations, the findings presented here provide important information about the use of an infodemiologic approach to assess COVID-19 vaccine-related interest and intentions. During the study period, online search behavior related to COVID-19 vaccines suggested a possible historic high in public interest of vaccines. Furthermore, the specific type of vaccine related searches (eg, increased searches related to specific pharmacies and decreased searches related to vaccine side effects) is consistent with reduced vaccine hesitancy and greater intention to vaccinate. The relatively low occurrence of some types of searches (eg, COVID-19 vaccine myths and conspiracy beliefs) suggests that many individuals who lack or are uncertain about critical vaccine-related information are not engaged in active online search to address their information needs. Encouraging more active information-seeking, along with critical appraisal of health information on the web, could be an important strategy to combat misinformation about COVID-19 vaccines and increase vaccine confidence and intention to vaccinate among the general population.

    Conflicts of Interest

    None declared.

    1. Brørs G, Norman CD, Norekvål TM. Accelerated importance of eHealth literacy in the COVID-19 outbreak and beyond. Eur J Cardiovasc Nurs 2020 Aug 15;19(6):458-461 [FREE Full text] [CrossRef] [Medline]
    2. Chong YY, Cheng HY, Chan HYL, Chien WT, Wong SYS. COVID-19 pandemic, infodemic and the role of eHealth literacy. Int J Nurs Stud 2020 Aug;108:103644 [FREE Full text] [CrossRef] [Medline]
    3. Eysenbach G. How to fight an infodemic: the four pillars of infodemic management. J Med Internet Res 2020 Jun 29;22(6):e21820 [FREE Full text] [CrossRef] [Medline]
    4. Rice RE. Influences, usage, and outcomes of Internet health information searching: multivariate results from the Pew surveys. Int J Med Inform 2006 Jan;75(1):8-28. [CrossRef] [Medline]
    5. Bangerter LR, Griffin J, Harden K, Rutten LJ. Health information-seeking behaviors of family caregivers: analysis of the health information national trends survey. JMIR Aging 2019 Jan 14;2(1):e11237 [FREE Full text] [CrossRef] [Medline]
    6. Kontos E, Blake KD, Chou WS, Prestin A. Predictors of eHealth usage: insights on the digital divide from the Health Information National Trends Survey 2012. J Med Internet Res 2014 Jul 16;16(7):e172 [FREE Full text] [CrossRef] [Medline]
    7. Sherman LD, Patterson MS, Tomar A, Wigfall LT. Use of digital health information for health information seeking among men living with chronic disease: data from the Health Information National Trends Survey. Am J Mens Health 2020 Jan 23;14(1):1557988320901377 [FREE Full text] [CrossRef] [Medline]
    8. Eysenbach G. Infodemiology and infoveillance: framework for an emerging set of public health informatics methods to analyze search, communication and publication behavior on the Internet. J Med Internet Res 2009 Mar 27;11(1):e11 [FREE Full text] [CrossRef] [Medline]
    9. Stephens-Davidowitz S. Google searches can help us find emerging COVID-19 outbreaks. The New York Times. 2020 Apr 05.   URL: https://www.nytimes.com/2020/04/05/opinion/coronavirus-google-searches.html [accessed 2021-10-28]
    10. Vasconcellos-Silva PR, Carvalho DBF, Trajano V, de La Rocque LR, Sawada ACMB, Juvanhol LL. Using Google Trends data to study public interest in breast cancer screening in Brazil: why not a pink February? JMIR Public Health Surveill 2017 Apr 06;3(2):e17 [FREE Full text] [CrossRef] [Medline]
    11. Ginsberg J, Mohebbi MH, Patel RS, Brammer L, Smolinski MS, Brilliant L. Detecting influenza epidemics using search engine query data. Nature 2009 Feb 19;457(7232):1012-1014. [CrossRef] [Medline]
    12. Badell-Grau RA, Cuff JP, Kelly BP, Waller-Evans H, Lloyd-Evans E. Investigating the prevalence of reactive online searching in the covid-19 pandemic: infoveillance study. J Med Internet Res 2020 Oct 27;22(10):e19791 [FREE Full text] [CrossRef] [Medline]
    13. Higgins TS, Wu AW, Sharma D, Illing EA, Rubel K, Ting JY, Snot Force Alliance. Correlations of online search engine trends with coronavirus disease (COVID-19) incidence: infodemiology study. JMIR Public Health Surveill 2020 May 21;6(2):e19702 [FREE Full text] [CrossRef] [Medline]
    14. Jimenez AJ, Estevez-Reboredo RM, Santed MA, Ramos V. COVID-19 symptom-related google searches and local covid-19 incidence in spain: correlational study. J Med Internet Res 2020 Dec 18;22(12):e23518 [FREE Full text] [CrossRef] [Medline]
    15. Mattiuzzi C, Lippi G. Analysis of Google Searches for COVID-19 and its symptoms for predicting disease epidemiology in the United States. Acta Biomed 2020 Dec 04;92(1):e2021064 [FREE Full text] [CrossRef] [Medline]
    16. Mavragani A. Tracking COVID-19 in Europe: infodemiology approach. JMIR Public Health Surveill 2020 Apr 20;6(2):e18941 [FREE Full text] [CrossRef] [Medline]
    17. Rajan A, Sharaf R, Brown RS, Sharaiha RZ, Lebwohl B, Mahadev S. Association of search query interest in gastrointestinal symptoms with COVID-19 diagnosis in the United States: infodemiology study. JMIR Public Health Surveill 2020 Jul 17;6(3):e19354 [FREE Full text] [CrossRef] [Medline]
    18. Effenberger M, Kronbichler A, Shin JI, Mayer G, Tilg H, Perco P. Association of the COVID-19 pandemic with internet search volumes: a Google trends analysis. Int J Infect Dis 2020 Jun;95:192-197 [FREE Full text] [CrossRef] [Medline]
    19. Chen B, Chen X, Pan J, Liu K, Xie B, Wang W, et al. Dissemination and refutation of rumors during the COVID-19 outbreak in China: infodemiology study. J Med Internet Res 2021 Feb 15;23(2):e22427 [FREE Full text] [CrossRef] [Medline]
    20. Gerts D, Shelley CD, Parikh N, Pitts T, Watson Ross C, Fairchild G, et al. "Thought I'd share first" and other conspiracy theory tweets from the COVID-19 infodemic: exploratory study. JMIR Public Health Surveill 2021 Apr 14;7(4):e26527 [FREE Full text] [CrossRef] [Medline]
    21. Hou Z, Du F, Zhou X, Jiang H, Martin S, Larson H, et al. Cross-country comparison of public awareness, rumors, and behavioral responses to the COVID-19 epidemic: infodemiology study. J Med Internet Res 2020 Aug 03;22(8):e21143 [FREE Full text] [CrossRef] [Medline]
    22. Nsoesie EO, Cesare N, Müller M, Ozonoff A. COVID-19 misinformation spread in eight countries: exponential growth modeling study. J Med Internet Res 2020 Dec 15;22(12):e24425 [FREE Full text] [CrossRef] [Medline]
    23. Rovetta A, Bhagavathula AS. COVID-19-related web search behaviors and infodemic attitudes in Italy: infodemiological study. JMIR Public Health Surveill 2020 May 05;6(2):e19374 [FREE Full text] [CrossRef] [Medline]
    24. Rovetta A, Bhagavathula AS. Global infodemiology of COVID-19: analysis of Google web searches and Instagram hashtags. J Med Internet Res 2020 Aug 25;22(8):e20673 [FREE Full text] [CrossRef] [Medline]
    25. Sajjadi NB, Nowlin W, Nowlin R, Wenger D, Beal JM, Vassar M, et al. United States internet searches for "infertility" following COVID-19 vaccine misinformation. J Osteopath Med 2021 Apr 12;121(6):583-587 [FREE Full text] [CrossRef] [Medline]
    26. Hu Z, Yang Z, Li Q, Zhang A. The COVID-19 infodemic: infodemiology study analyzing stigmatizing search terms. J Med Internet Res 2020 Nov 16;22(11):e22639 [FREE Full text] [CrossRef] [Medline]
    27. Husain I, Briggs B, Lefebvre C, Cline DM, Stopyra JP, O'Brien MC, et al. Fluctuation of Public Interest in COVID-19 in the United States: retrospective analysis of Google Trends search data. JMIR Public Health Surveill 2020 Jul 17;6(3):e19969 [FREE Full text] [CrossRef] [Medline]
    28. Husnayain A, Shim E, Fuad A, Su EC. Understanding the community risk perceptions of the COVID-19 outbreak in South Korea: infodemiology study. J Med Internet Res 2020 Sep 29;22(9):e19788 [FREE Full text] [CrossRef] [Medline]
    29. COVID-19 Vaccination. Centers for Disease Control and Prevention.   URL: https://www.cdc.gov/vaccines/covid-19/index.html [accessed 2021-07-07]
    30. COVID-19 Vaccine Product Information. Centers for Disease Control and Prevention.   URL: https://www.cdc.gov/vaccines/covid-19/info-by-product/index.html [accessed 2021-07-07]
    31. Wood S, Schulman K. Beyond politics — promoting Covid-19 vaccination in the United States. N Engl J Med 2021 Feb 18;384(7):e23. [CrossRef]
    32. Pullan S, Dey M. Vaccine hesitancy and anti-vaccination in the time of COVID-19: a Google Trends analysis. Vaccine 2021 Apr 01;39(14):1877-1881 [FREE Full text] [CrossRef] [Medline]
    33. Islam MS, Kamal AM, Kabir A, Southern DL, Khan SH, Hasan SMM, et al. COVID-19 vaccine rumors and conspiracy theories: the need for cognitive inoculation against misinformation to improve vaccine adherence. PLoS One 2021 May 12;16(5):e0251605 [FREE Full text] [CrossRef] [Medline]
    34. Search Engine Market Share in United State of America-June 2021. Global Stats - Stat Counter.   URL: https://gs.statcounter.com/search-engine-market-share/all/united-states-of-america [accessed 2021-07-07]
    35. Arora VS, McKee M, Stuckler D. Google Trends: opportunities and limitations in health and health policy research. Health Policy 2019 Mar;123(3):338-341. [CrossRef] [Medline]
    36. Silvestri F. Mining query logs: turning search usage data into knowledge. Foundations and Trends in Information Retrieval 2009 Nov 29;4(1–2):1-174. [CrossRef]
    37. Kulkarni A, Teevan J, Svore K, Dumais ST. Understanding temporal query dynamics. In: WSDM '11.: Association for Computing Machinery; 2011 Presented at: Proceedings of the Fourth ACM International Conference on Web Search and Data Mining; February 2011; Hong Kong, China p. 167-176. [CrossRef]
    38. Dumais S, Jeffries R. Understanding user behavior through log data analysis. In: Olson J, Kellogg W, editors. Ways of Knowing in Human-Computer Interaction. New York: Springer-Verlag; 2014:349-372.
    39. Malik MT, Gumel A, Thompson LH, Strome T, Mahmud SM. “Google flu trends” and emergency department triage data predicted the 2009 pandemic H1N1 waves in Manitoba. Can J Public Health 2011 Jul 1;102(4):294-297. [CrossRef]
    40. Sousa-Pinto B, Anto A, Czarlewski W, Anto JM, Fonseca JA, Bousquet J. Assessment of the impact of media coverage on COVID-19-related Google Trends data: infodemiology study. J Med Internet Res 2020 Aug 10;22(8):e19611 [FREE Full text] [CrossRef] [Medline]
    41. Huynh Dagher S, Lamé G, Hubiche T, Ezzedine K, Duong TA. The influence of media coverage and governmental policies on Google queries related to COVID-19 cutaneous symptoms: infodemiology study. JMIR Public Health Surveill 2021 Feb 25;7(2):e25651 [FREE Full text] [CrossRef] [Medline]
    42. KFF COVID-19 Vaccine Monitor: March 2021. Kaiser Family Foundation. 2021.   URL: https://www.kff.org/coronavirus-covid-19/poll-finding/kff-covid-19-vaccine-monitor-march-2021/ [accessed 2021-10-07]
    43. KFF COVID-19 Vaccine Monitor: January 2021. Kaiser Family Foundation. 2021 Jan 21.   URL: https://www.kff.org/report-section/kff-covid-19-vaccine-monitor-january-2021-vaccine-hesitancy/ [accessed 2021-10-07]

    ‎
    FDA: Food and Drug Administration
    RSV: relative search volume

    Edited by G Eysenbach; submitted 15.07.21; peer-reviewed by N Sajjadi, X Zhou, J Jiang; comments to author 06.08.21; revised version received 11.10.21; accepted 13.10.21; published 12.11.21

    Copyright

    ©Lawrence An, Daniel M Russell, Rada Mihalcea, Elizabeth Bacon, Scott Huffman, Ken Resnicow. Originally published in JMIR Infodemiology (https://infodemiology.jmir.org), 12.11.2021.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Infodemiology, is properly cited. The complete bibliographic information, a link to the original publication on https://infodemiology.jmir.org/, as well as this copyright and license information must be included.