Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress
Clinical Collections
Advances in Alzheimer’s

At a glance: The ONWARDS insulin icodec trials

A round-up of the ONWARDS phase 3 clinical trials evaluating the novel weekly insulin icodec in people with diabetes.
ADVANCED SEARCH
Log in
Top
BMC Public Health
Published in: BMC Public Health 1/2016

Open Access 01-12-2015 | Research article

Comparing the quality of pro- and anti-vaccination online information: a content analysis of vaccination-related webpages

Authors: Gabriele Sak, Nicola Diviani, Ahmed Allam, Peter J. Schulz

Published in: BMC Public Health | Issue 1/2016

Login to get access

Abstract

Background

The exponential increase in health-related online platforms has made the Internet one of the main sources of health information globally. The quality of health contents disseminated on the Internet has been a central focus for many researchers. To date, however, few comparative content analyses of pro- and anti-vaccination websites have been conducted, and none of them compared the quality of information. The main objective of this study was therefore to bring new evidence on this aspect by comparing the quality of pro- and anti-vaccination online sources.

Methods

Based on past literature and health information quality evaluation initiatives, a 40-categories assessment tool (Online Vaccination Information Quality Codebook) was developed and used to code a sample of 1093 webpages retrieved via Google and two filtered versions of the same search engine. The categories investigated were grouped into four main quality dimensions: web-related design quality criteria (10 categories), health-specific design quality criteria (3 categories), health related content attributes (12 categories) and vaccination-specific content attributes (15 categories). Data analysis comprised frequency counts, cross tabulations, Pearson’s chi-square, and other inferential indicators.

Results

The final sample included 514 webpages in favor of vaccination, 471 against, and 108 neutral. Generally, webpages holding a favorable view toward vaccination presented more quality indicators compared to both neutral and anti-vaccination pages. However, some notable exceptions to this rule were observed. In particular, no differences were found between pro- and anti-vaccination webpages as regards vaccination-specific content attributes.

Conclusions

Our analyses showed that the overall quality of pro-vaccination webpages is superior to anti-vaccination online sources. The developed coding scheme was proven to be a helpful and reliable tool to judge the quality of vaccination-related webpages. Based on the results, we advance recommendations for online health information providers as well as directions for future research in this field.
Appendix
Available only for authorised users
Footnotes
1
“The customization of the search engines was done by limiting their search coverage to different pre-defined sets of websites […]. This manipulation was realized by configuring the context and annotation files of Google custom search engine” [22].
Google custom search engine is available at: https://​developers.​google.​com/​custom-search/​
 
2
American Accreditation HealthCare Commission: https://​www.​urac.​org/​
 
5
As explained by Hobson-West [43], the anti-vaccination group labeled as reformist is composed by those people “who are critical of vaccines but likely to provide at least partial support to vaccination”.
 
6
For the current study the category medical ownership was specifically designed to depict if on the page evaluated the name of a doctor or a medical corporation was present. However, it has to be noted that no disctinctions were made in case the webpage was affiliated with a single medical doctor or an internationally well-known health institution.
 
7
The developed evaluating tool (OVIQC) can be obtained by e-mail at the following address (gabriele.sak@usi.ch).
 
Literature
1.
2.
Hesse B, Nelson D, Kreps G, Croyle R, Arora N, Rimer B, et al. Trust and sources of health information. Arch Intern Med. 2005;165(22):2618–24.CrossRefPubMed
3.
Lustria MLA. Can interactivity make a difference? Effects of interactivity on the comprehension of and attitudes toward online health content. J Am Soc Inf Sci Technol. 2007;58(6):766–76.CrossRef
4.
5.
Eysenbach G, Powell J, Kuss O, Sa E. Empirical studies assessing the quality of health information for consumers on the World Wide Web. J Am Med Assoc. 2002;287(20):2691–700.CrossRef
6.
Risk A, Dzenowagis J. Review of internet health information quality initiatives. J Med Internet Resources. 2001;3:4:e28.CrossRef
7.
Berland GK, Elliott MN, Morales LM, Algazy JI, Kravitz RL, Broder MS, et al. Health information on the Internet: accessibility, quality, and readability in English and Spanish. J Am Med Assoc. 2001;285:2612–21.CrossRef
8.
Wilson P. How to find the good and avoid the bad or ugly: a short guide to tools for rating quality of health information on the Internet. Br Med J. 2002;324(7337):598–602.CrossRef
9.
Lopes CT, Ribeiro C. Data Certification Impact on Health Information Retrieval. In: Holziger A, Simonic KM, editors. Information Quality in E-Health: 7th Conference of the Workgroup Human-Computer Interaction and Usability Engineering of the Austrian Computer Society, USAB 2011, Proceedings. Graz, Austria: Springer; 2011.
10.
Barnes C, Harvey R, Wilde A, Hadzi-Pavlovic D, Wilhelm K, Mitchell PB. Review of the quality of information on bipolar disorder on the Internet. Aust N Z J Psychiatry. 2009;43:934–45.CrossRef
11.
Betsch C, Brewer NT, Brocard P, Davies P, Gaissmaier W, Haase N, et al. Opportunities and challenges of Web 2.0 for vaccination decisions. Vaccine. 2012;30(25):3727–33.CrossRefPubMed
12.
13.
Jones AM, Omer SB, Bednarczyk RA, Halsey NA, Moulton LH, Salmon DA. Parents’ source of vaccine information and impact on vaccine attitudes, beliefs, and nonmedical exemptions. Advances Prev Med. 2012;2012:1–8. doi: 10.​1155/​2012/​932741.CrossRef
14.
Stvilia B, Mon L, Yi YJ. A model for online consumer health information quality. J Am Med Assoc. 2009;60(9):1781–91.
15.
Habel MA, Liddon N, Stryker JE. The HPV Vaccine: A content Analysis of Online News Stories. J Women’s Health. 2009;18(3):401–7.CrossRef
16.
Goff SL, Mazor KM, Gagne SJ, Corey KC, Blake DR. Vaccine counseling: a content analysis of patient-physician discussions regarding the human papilloma virus vaccine. Vaccine. 2011;29(43):7343–9.CrossRefPubMed
17.
Madden K, Nan X, Briones R, Waks L. Sorting through search results: a content analysis of HPV vaccine information online. Vaccine. 2012;30(25):3741–6.CrossRefPubMed
18.
Bean SJ. Emerging and continuing trends in vaccine opposition website content. Vaccine. 2011;29(10):1874–80.CrossRefPubMed
19.
Wolfe RM, Sharp LK, Lipsky MS. Content and design attributes of antivaccination web sites. J Am Med Assoc. 2002;287(24):3245–8.CrossRef
20.
Wolfe RM, Sharp LK. Vaccination or immunization? The impact of search terms on the Internet. J Health Commun. 2005;10(6):537–51.CrossRefPubMed
21.
Kata A. Anti-vaccine activists, Web 2.0, and the postmodern paradigm – an overview of tactics and tropes used online by the anti-vaccination movement. Vaccine. 2012;30(25):3778–89.CrossRefPubMed
22.
Allam A, Schulz PJ, Nakamoto K. The impact of search engine selection and sorting criteria on vaccination beliefs and attitudes: two experiments manipulating Google output. J Med Internet Resources. 2014;16:4:e100. doi:10.​2196/​jmir.​2642.CrossRef
23.
Guardiola-Wanden-Berghe R, Gil-Peréz JD, Sanz-Velero J, Wanden-Berhge C. Evaluating the quality of websites relating to diet and eating disorder. Health Inf Libr J. 2011;28:294–301.CrossRef
24.
Martın Martınéz B. Valoracion de la calidad de las paginas Web en gastroenterologıa, hepatologıa y nutricion infantil [Doctoral Thesis]. Barcelona: University of Barcelona, Faculty of Medicine; 2007.
25.
Dellavalle RP, Hester EJ, Heilig LF, Drake AL, Kuntzman JW, Graber M, et al. Information science. Going, going, gone: lost Internet references. Science. 2003;302:787–8.CrossRefPubMed
26.
Wang RY, Strong DM. Beyond accuracy: what data quality means to data consumers. J Manag Inf Syst. 1996;12(4):5–34.CrossRef
27.
Jadad AR, Gagliardi A. Rating health information on the Internet: navigating to knowledge or to Babel? J Am Med Assoc. 1998;279(8):611–4.CrossRef
28.
Alexander JE, Tate MA. Web wisdom: How to evaluate and create information quality on the web. Mahwah, New Jersey: Erlbaum; 1999.
29.
Kim P, Eng TR, Deering DJ, Maxfield A. Published criteria for evaluating health related web sites: review. Br Med J. 1999;318:647.1.CrossRef
30.
Stevenson FA, Wallace G, Rivers P, Gerrett D. It’s the best of two evils’: a study of patients’ perceived information needs about oral steroids for asthma. Health Expect. 1999;2:185–94.CrossRefPubMedPubMedCentral
31.
Dedeke A. A conceptual framework for developing quality measures for information. In: Proceedings of 5th International Conference on Information Quality. 2000. p. 126–8.
32.
33.
Holmes-Rovner M. Patient choice modules for summaries of clinical effectiveness. Br Med J. 2001;322:664–7.CrossRef
34.
Department of Health (UK). Toolkit for producing patient information. London: HMSO; 2002.
35.
Gagliardi A, Jadad AR. Examination of instruments used to rate quality of health information on the Internet: chronicle of a voyage with an unclear destination. Br Med J. 2002;324(7337):569–73.CrossRef
36.
37.
Moult B, Franck LS, Brady H. Ensuring Quality Information for Patients: development and preliminary validation of a new instrument to improve the quality of written health care information. Health Expect. 2004;7:165–75.CrossRefPubMedPubMedCentral
38.
Bernstam EV, Shelton DM, Walji M, Meric-Bernstam F. Instruments to assess the quality of health information on the World Wide Web: what can our patients actually use? Int J Med Inform. 2005;74:13–9.CrossRefPubMed
39.
40.
Knight S, Burn J. Developing a framework for assessing Information Quality on the World Wide Web. Inform Sci J. 2005;8:159–72.
41.
Zimmermann R, Wolfe R, Fox J, Nowalk M, Troy J, Sharp L. Vaccine criticism on the World Wide Web. J Med Internet Res. 2005;7:2:e17.CrossRef
42.
43.
Hobson-West P. Trusting blindly can be the biggest risk of all: organized resistance to childhood vaccination in the UK. Sociol Health Illness. 2007;29(2):198–215.CrossRef
44.
Calero C, Caro A, Piattini M. An applicable data quality model for Web portal data consumers. World Wide Web. 2008;11(4):465–84.CrossRef
45.
Walsh TM, Volsko TA. Readability assessment of internet-based consumer health information. Respir Care. 2008;53(10):1310–5.PubMed
46.
Nagappa AN, Sam KG, Zarrin F, Saurabh H, Partha G, Pathak K. Evaluation of web sites for quality and contents of asthma patient education. J Young Pharm. 2009;1(3):278–83.CrossRef
47.
Kata A. A postmodern Pandora’s box: anti-vaccination misinformation on the Internet. Vaccine. 2010;28(7):1709–16.CrossRefPubMed
48.
49.
50.
51.
52.
Cohen J. A coefficient of agreement for nominal scales. Educ Psychol Meas. 1960;20:37–46.CrossRef
53.
Lombard M, Snyder-Duch J, Bracken CC. Content analysis in mass communication: assessment and reporting of intercoder reliability. Hum Commun Res. 2002;28(4):587–604.CrossRef
54.
Fagerlin A, Wang C, Ubel PA. Reducing the influence of anecdotal reasoning on people’s health care decisions: is a picture worth a thousand statistics. Med Decis Mak. 2005;25:398–405.CrossRef
55.
Meppelink CS, van Weert JCM, Haven C, Smit EG. The effectiveness of health animations in audiences with different health literacy levels: an experimental study. J Med Internet Res. 2015;17:1.CrossRef
56.
Eysenbach G. Medicine 2.0: social networking, collaboration, participation, apomediation, and openness. J Med Internet Res. 2008;10:3:e22.CrossRef
57.
Petty RE, Cacioppo JT. Communication and persuasion: central and peripheral routes to attitude change. New York: Springer; 1986.CrossRef
58.
Murray E, Burns J, See TS, Lai R, Nazareth I. Interactive Health Communication Applications for People with Chronic Disease, Cochrane Database of Systematic Reviews. 2005. In: Eysenbach G, editor. Credibility of health information and digital media: New perspectives and implications for youth. Digital media, youth, and credibility. Cambridge, MA: The MIT Press; 2008. p. 123–54.
59.
Wantland DJ, Portillo CJ, Holzemer WL, Slaughter R, McGhee EM. The Effectiveness of Web-Based vs. Non-Web-Based Interventions: A Meta-analysis of Behavioral Change Outcomes. Journal of Medical Internet Research. 2004; 6: e40. In: Eysenbach G, editor. Credibility of Health Information and Digital Media: New Perspectives and Implications for Youth. Digital Media, Youth, and Credibility. Cambridge, MA: The MIT Press; 2008. p. 123–54.
Metadata
Title
Comparing the quality of pro- and anti-vaccination online information: a content analysis of vaccination-related webpages
Authors
Gabriele Sak
Nicola Diviani
Ahmed Allam
Peter J. Schulz
Publication date
01-12-2015
Publisher
BioMed Central
Published in
BMC Public Health / Issue 1/2016
Electronic ISSN: 1471-2458
DOI
https://doi.org/10.1186/s12889-016-2722-9

Other articles of this Issue 1/2016

BMC Public Health 1/2016 Go to the issue

Debate

Is there a link between childhood adversity, attachment style and Scotland’s excess mortality? Evidence, challenges and potential research

Research article

Uptake and linkage into care over one year of providing HIV testing and counselling through community and health facility testing modalities in urban informal settlement of Kibera, Nairobi Kenya

Research article

A population-based analysis of increasing rates of suicide mortality in Japan and South Korea, 1985–2010

Study protocol

A transdisciplinary model to inform randomized clinical trial methods for electronic cigarette evaluation

Research article

Mother-to-child transmission of hepatitis B: extent of knowledge of physicians and midwives in Eastern region of Ghana

Research article

Unwanted sexual attention at work and long-term sickness absence: a follow-up register-based study