Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress 2023 EHA Congress 2023 ASCO Annual Meeting
Clinical Collections
Advances in Alzheimer’s

Keynote webinar | Spotlight on sleep in brain health

LIVE: Thursday 2nd May 2024, 18:00-19:30 (CEST)

Quality sleep is essential for health. But what happens to our brains when sleep patterns are disturbed? Join our experts to explore the interplay between sleep disruption and neurological diseases, and the questions that you need to be asking your patients to help you prevent the harmful effects of sleep deprivation.
ADVANCED SEARCH
Log in
Top
BMC Medical Informatics and Decision Making
Published in: BMC Medical Informatics and Decision Making 1/2018

Open Access 01-12-2018 | Research article

The QUEST for quality online health information: validation of a short quantitative tool

Authors: Julie M. Robillard, Jessica H. Jun, Jen-Ai Lai, Tanya L. Feng

Published in: BMC Medical Informatics and Decision Making | Issue 1/2018

Login to get access

Abstract

Background

Online health information is unregulated and can be of highly variable quality. There is currently no singular quantitative tool that has undergone a validation process, can be used for a broad range of health information, and strikes a balance between ease of use, concision and comprehensiveness. To address this gap, we developed the QUality Evaluation Scoring Tool (QUEST). Here we report on the analysis of the reliability and validity of the QUEST in assessing the quality of online health information.

Methods

The QUEST and three existing tools designed to measure the quality of online health information were applied to two randomized samples of articles containing information about the treatment (n = 16) and prevention (n = 29) of Alzheimer disease as a sample health condition. Inter-rater reliability was assessed using a weighted Cohen’s kappa (κ) for each item of the QUEST. To compare the quality scores generated by each pair of tools, convergent validity was measured using Kendall’s tau (τ) ranked correlation.

Results

The QUEST demonstrated high levels of inter-rater reliability for the seven quality items included in the tool (κ ranging from 0.7387 to 1.0, P < .05). The tool was also found to demonstrate high convergent validity. For both treatment- and prevention-related articles, all six pairs of tests exhibited a strong correlation between the tools (τ ranging from 0.41 to 0.65, P < .05).

Conclusions

Our findings support the QUEST as a reliable and valid tool to evaluate online articles about health. Results provide evidence that the QUEST integrates the strengths of existing tools and evaluates quality with equal efficacy using a concise, seven-item questionnaire. The QUEST can serve as a rapid, effective, and accessible method of appraising the quality of online health information for researchers and clinicians alike.
Appendix
Available only for authorised users
Footnotes
1
Please refer to Appendix 1 for characteristics of retrieved articles.
 
2
Please refer to Appendix 2 for a complete listing of currently available tools.
 
Literature
1.
2.
Stvilia B, Mon L, Yi YJ. A model for online consumer health information quality. J Am Soc Inf Sci Technol. 2009;60:1781–91.CrossRef
3.
Theodosiou L, Green J. Emerging challenges in using health information from the internet. Adv Psychiatr Treat. 2003;9:387–96.CrossRef
4.
McCully SN, Don BP, Updegraff JA. Using the internet to help with diet, weight, and physical activity: results from the health information National Trends Survey (HINTS). J Med Internet Res. 2013;15:e148.CrossRef
5.
Fahy E, Hardikar R, Fox A, Mackay S. Quality of patient health information on the internet: reviewing a complex and evolving landscape. Australas Med J. 2014;7:24–8.CrossRef
6.
7.
8.
Diviani N, van den Putte B, Giani S, van Weert JC. Low health literacy and evaluation of online health information: a systematic review of the literature. J Med Internet Res 2015;17:e112–e112.CrossRef
9.
Devine T, Broderick J, Harris LM, Wu H, Hilfiker SW. Making quality health websites a national public health priority: toward quality standards. J Med Internet Res 2016;18:e211–e211.CrossRef
10.
Arksey H, O’Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005;8:19–32.CrossRef
11.
Zeraatkar D, Obeda M, Ginsberg JS, Hirsh J. The development and validation of an instrument to measure the quality of health research reports in the lay media. BMC Public Health. 2017;17:343.CrossRef
12.
Närhi U, Pohjanoksa-Mäntylä M, Karjalainen A, Saari JK, Wahlroos H, Airaksinen MS, et al. The DARTS tool for assessing online medicines information. Pharm World Sci. 2008;30:898–906.CrossRef
13.
Oxman AD, Guyatt GH, Cook DJ, Jaeschke R, Heddle N, Keller J. An index of scientific quality for health reports in the lay press. J Clin Epidemiol. 1993;46:987–1001.CrossRef
14.
Hsu W-C, Bath PA. Development of a patient-oriented tool for evaluating the quality of breast cancer information on the internet. Stud Health Technol Inform. 2008;136:297–302.PubMed
15.
Seidman JJ, Steinwachs D, Rubin HR. Design and testing of a tool for evaluating the quality of diabetes consumer-information web sites. J Med Internet Res. 2003;5:e30.CrossRef
16.
Charnock D, Shepperd S, Needham G, Gann R. DISCERN: an instrument for judging the quality of written consumer health information on treatment choices. J Epidemiol Community Health. 1999;53:105–11.CrossRef
17.
Moult B, Franck LS, Brady H. Ensuring quality information for patients: development and preliminary validation of a new instrument to improve the quality of written health care information. Health Expect Int J Public Particip Health Care Health Policy. 2004;7:165–75.
18.
Jones J. Development of a self-assessment method for patients to evaluate health information on the Internet. Proc AMIA Symp. 1999:540–4.
19.
Aslani A, Pournik O, Abu-Hanna A, Eslami S. Web-site evaluation tools: a case study in reproductive health information. Stud Health Technol Inform. 2014;205:895–9.PubMed
20.
Eysenbach G, Powell J, Kuss O, Sa E-R. Empirical studies assessing the quality of health information for consumers on the world wide web: a systematic review. JAMA. 2002;287:2691–700.CrossRef
21.
Kim P, Eng TR, Deering MJ, Maxfield A. Published criteria for evaluating health related web sites: review. BMJ. 1999;318:647–9.CrossRef
22.
Roberts L. Health information and the internet: the 5 Cs website evaluation tool. Br J Nurs. 2010;19:322–5.CrossRef
23.
Breckons M, Jones R, Morris J, Richardson J. What do evaluation instruments tell us about the quality of complementary medicine information on the internet? J Med Internet Res. 2008;10:e3.CrossRef
24.
Bernstam EV, Shelton DM, Walji M, Meric-Bernstam F. Instruments to assess the quality of health information on the world wide web: what can our patients actually use? Int J Med Inf. 2005;74:13–9.CrossRef
25.
Gagliardi A, Jadad AR. Examination of instruments used to rate quality of health information on the internet: chronicle of a voyage with an unclear destination. BMJ. 2002;324:569–73.CrossRef
26.
Prusti M, Lehtineva S, Pohjanoksa-Mäntylä M, Bell JS. The quality of online antidepressant drug information: an evaluation of English and Finnish language web sites. Res Soc Adm Pharm. 2012;8:263–8.CrossRef
27.
Ademiluyi G, Rees CE, Sheard CE. Evaluating the reliability and validity of three tools to assess the quality of health information on the internet. Patient Educ Couns. 2003;50:151–5.CrossRef
28.
Sandvik H. Health information and interaction on the internet: a survey of female urinary incontinence. BMJ. 1999;319:29–32.CrossRef
29.
Chumber S, Huber J, Ghezzi P. A methodology to analyze the quality of health information on the internet: the example of diabetic neuropathy. Diabetes Educ. 2015;41:95–105.CrossRef
30.
Silberg W, Lundberg G, Musacchio R. Assessing, controlling, and assuring the quality of medical information on the internet: caveant lector et viewor—let the reader and viewer beware. JAMA. 1997;277:1244–5.CrossRef
31.
Robillard JM, Feng TL. Health advice in a digital world: quality and content of online information about the prevention of Alzheimer’s disease. J Alzheimers Dis. 2017;55:219–29.CrossRef
32.
33.
Robillard JM, Johnson TW, Hennessey C, Beattie BL, Illes J. Aging 2.0: health information about dementia on twitter. PLoS One. 2013;8:1–5.CrossRef
34.
Robillard JM, Illes J, Arcand M, Beattie BL, Hayden S, Lawrence P, et al. Scientific and ethical features of English-language online tests for Alzheimer’s disease. Alzheimers Dement Diagn Assess Dis Monit. 2015;1:281–8.
35.
Beitzel SM, Jensen EC, Chowdhury A, Frieder O, Grossman D. Temporal analysis of a very large topically categorized web query log. J Am Soc Inf Sci Technol. 2007;58:166–78.CrossRef
36.
Bouchier H, Bath PA. Evaluation of websites that provide information on Alzheimer’s disease. Health Informatics J. 2003;9:17–31.CrossRef
37.
Harland J, Bath P. Assessing the quality of websites providing information on multiple sclerosis: evaluating tools and comparing sites. Health Informatics J. 2007;13:207–21.CrossRef
38.
Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33:159–74.CrossRef
39.
Gibbons JD. Nonparametric statistical inference. 2nd ed. New York: Marcel Dekker; 1985.
40.
41.
Mullen E. Health literacy challenges in the aging population. Nurs Forum (Auckl). 2013;48:248–55.CrossRef
42.
Robertson-Lang L, Major S, Hemming H. An exploration of search patterns and credibility issues among older adults seeking online health information. Can J Aging Rev Can Vieil. 2011;30:631–45.CrossRef
43.
Kruse RL, Koopman RJ, Wakefield BJ, Wakefield DS, Keplinger LE, Canfield SM, et al. Internet use by primary care patients: where is the digital divide? Fam Med. 2012;44:342–7.PubMed
44.
Boyer C, Dolamic L. Automated detection of HONcode website conformity compared to manual detection: an evaluation. J Med Internet Res. 2015;17:e135.CrossRef
45.
46.
Eysenbach G, Köhler C. How do consumers search for and appraise health information on the world wide web? Qualitative study using focus groups, usability tests, and in-depth interviews. BMJ. 2002;324:573–7.CrossRef
47.
Provost M, Koompalum D, Dong D, Martin BC. The initial development of the WebMedQual scale: domain assessment of the construct of quality of health web sites. Int J Med Inf. 2006;75:42–57.CrossRef
48.
49.
Larson HJ, Jarrett C, Schulz WS, Chaudhuri M, Zhou Y, Dube E, et al. Measuring vaccine hesitancy: the development of a survey tool. Vaccine. 2015;33:4165–75.CrossRef
50.
Robertson G. Attitudes towards ageing and their impact on health and wellbeing in later life: an agenda for further analysis. Work Older People. 2016;20:214–8.CrossRef
51.
Prins MA, Verhaak PFM, Bensing JM, van der Meer K. Health beliefs and perceived need for mental health care of anxiety and depression—the patients’ perspective explored. Clin Psychol Rev 2008;28:1038–1058.CrossRef
52.
Metadata
Title
The QUEST for quality online health information: validation of a short quantitative tool
Authors
Julie M. Robillard
Jessica H. Jun
Jen-Ai Lai
Tanya L. Feng
Publication date
01-12-2018
Publisher
BioMed Central
Published in
BMC Medical Informatics and Decision Making / Issue 1/2018
Electronic ISSN: 1472-6947
DOI
https://doi.org/10.1186/s12911-018-0668-9

Other articles of this Issue 1/2018

BMC Medical Informatics and Decision Making 1/2018 Go to the issue

Technical advance

Towards stroke prediction using electronic health records

Debate

Healthcare information systems: the cognitive challenge

Research article

A rank weighted classification for plasma proteomic profiles based on case-based reasoning

Research article

SwissMTB: establishing comprehensive molecular cancer diagnostics in Swiss clinics

Research article

Presentation of laboratory test results in patient portals: influence of interface design on risk interpretation and visual search behaviour

Study protocol

Information sharing across generations and environments (InfoSAGE): study design and methodology protocol