Top

Published in:

01-12-2020 | Research

Effects of the Informed Health Choices podcast on the ability of parents of primary school children in Uganda to assess the trustworthiness of claims about treatment effects: one-year follow up of a randomised trial

Authors: Daniel Semakula, Allen Nsangi, Andrew D. Oxman, Matt Oxman, Astrid Austvoll-Dahlgren, Sarah Rosenbaum, Angela Morelli, Claire Glenton, Simon Lewin, Laetitia Nyirazinyoye, Margaret Kaseje, Iain Chalmers, Atle Fretheim, Christopher J. Rose, Nelson K. Sewankambo

Published in: Trials | Issue 1/2020

Abstract

Introduction

Earlier, we designed and evaluated an educational mass media intervention for improving people’s ability to think more critically and to assess the trustworthiness of claims (assertions) about the benefits and harms (effects) of treatments. The overall aims of this follow-up study were to evaluate the impact of our intervention 1 year after it was administered, and to assess retention of learning and behaviour regarding claims about treatments.

Methods

We randomly allocated consenting parents to listen to either the Informed Health Choices podcast (intervention) or typical public service announcements about health issues (control) over 7–10 weeks. Each intervention episode explained how the trustworthiness of treatment claims can be assessed by using relevant key concepts of evidence-informed decision-making. Participants listened to two episodes per week, delivered by research assistants. We evaluated outcomes immediately, and a year after the intervention. Primary outcomes were mean score and the proportion with a score indicating a basic ability to apply the key concepts (> 11 out of 18 correct answers) on a tool measuring people’s ability to critically appraise the trustworthiness of treatment claims. Skills decay/retention was estimated by calculating the relative difference between the follow-up and initial results in the intervention group, adjusting for chance. Statistical analyses were performed using R (R Core Team, Vienna, Austria; version 3.4.3).

Results

After 1 year, the mean score for parents in the intervention group was 58.9% correct answers, compared to 52.6% in the control (adjusted mean difference of 6.7% (95% CI 3.3% to 10.1%)). In the intervention group, 47.2% of 267 parents had a score indicating a basic ability to assess treatment claims compared to 39.5% of 256 parents in the control (adjusted difference of 9.8% more parents (95% CI 0.9% to 18.9%). These represent relative reductions of 29% in the mean scores and 33% in the proportion of parents with a score indicating a basic ability to assess the trustworthiness of claims about treatment effects.

Conclusions

Although listening to the Informed Health Choices podcast initially led to a large improvement in the ability of parents to assess claims about the effects of treatments, our findings show that these skills decreased substantially over 1 year. More active practice could address the substantial skills decay observed over 1 year.

Trial registration

Pan African Clinical Trial Registry (www.pactr.org), PACTR201606001676150. Registered on 12 June 2016.

Available only for authorised users

Mugyenyi P. Genocide by denial. How profiteering from HIV/AIDS killed millions. Kampala: Fountain Publishers; 2008.

Blumenkrantz, David In The new vision 11 November 1989. Thirty tons of soil: Nanyonga’s divine panacea. 1989. Available from: http://david-blumenkrantz.squarespace.com/new-page-3/. Cited 23 Nov 2018.

Kunihira NR, Nuwaha F, Mayanja R, Peterson S. Barriers to use of antiretroviral drugs in Rakai district of Uganda. Afr Health Sci. 2010;10:120–9 Makerere Medical School.PubMedPubMedCentral

Casiday R, Cresswell T, Wilson D, Panter-Brick C. A survey of UK parental attitudes to the MMR vaccine and trust in medical authority. Vaccine. 2006;24:177–84.PubMedCrossRef

Hadjikoumi I, Niekerk KV, Scott C. MMR catch up campaign: reasons for refusal to consent [3]. Arch Dis Child. 2006;91:621–2.PubMedPubMedCentralCrossRef

Mills E, Jadad AR, Ross C, Wilson K. Systematic review of qualitative studies exploring parental beliefs and attitudes toward childhood vaccination identifies common barriers to vaccination. J Clin Epidemiol. 2005;58:1081–8.PubMedCrossRef

Del Vicario M, Bessi A, Zollo F, Petroni F, Scala A, Caldarelli G, et al. The spreading of misinformation online. Proc Natl Acad Sci. 2016;113:554–9 National Academy of Sciences.PubMedCrossRefPubMedCentral

McMullan M. Patients using the Internet to obtain health information: how this affects the patient-health professional relationship. Patient Educ Couns. 2006;63(1-2):24–8. https://doi.org/10.1016/j.pec.2005.10.006.PubMedCrossRef

Ullrich PF Jr, Vaccaro AR. Patient education on the internet: opportunities and pitfalls. Spine (Phila Pa 1976). 2002;27(7):E185–E188. https://doi.org/10.1097/00007632-200204010-00019.PubMedCrossRef

10.

Wald HS, Dube CE, Anthony DC. Untangling the Web--the impact of Internet use on health care and the physician-patient relationship. Patient Educ Couns. 2007;68(3):218–24. https://doi.org/10.1016/j.pec.2007.05.016.PubMedCrossRef

11.

Schwitzer G. A guide to reading health care news stories. JAMA Intern Med. 2014;174(7):1183–6. https://doi.org/10.1001/jamainternmed.2014.1359.PubMedCrossRef

12.

Bonevski B, Wilson A, Henry DA. An analysis of news media coverage of complementary and alternative medicine. PLoS One. 2008;3:e2406 Public Library of Science.PubMedPubMedCentralCrossRef

13.

Motosko CC, Ault AK, Kimberly LL, Zakhem GA, Gothard MD, Ho RS, et al. Analysis of spin in the reporting of studies of topical treatments of photoaged skin. J Am Acad Dermatol. 2019;80(2):516–522.e12. https://doi.org/10.1016/j.jaad.2018.04.034.PubMedCrossRef

14.

Haneef R, Lazarus C, Ravaud P, Yavchitz A, Boutron I. Interpretation of results of studies evaluating an intervention highlighted in Google health news: a cross-sectional study of news. PLoS One. 2015;10:1–15 Public Library of Science.CrossRef

15.

Woloshin S, Schwartz LM, Moncur M, Gabriel S, Tosteson ANA. Assessing values for health: numeracy matters. Med Decis Mak. 2001;21:382–90.CrossRef

16.

Sillence E, Briggs P, Harris PR, Fishwick L. How do patients evaluate and make use of online health information? Soc Sci Med. 2007;64:1853–62.PubMedCrossRef

17.

Schwartz LM, Woloshin S, Black WC, Welch HG. The role of numeracy in understanding the benefit of screening mammography. Ann Intern Med. 1997;127:966–72.PubMedCrossRef

18.

Lokker N, Sanders L, Perrin EM, Kumar D, Finkle J, Franco V, et al. Parental misinterpretations of over-the-counter pediatric cough and cold medication labels. Pediatrics. 2009;123:1464–71.PubMedCrossRef

19.

Rothman RL, Housam R, Weiss H, Davis D, Gregory R, Gebretsadik T, et al. Patient understanding of food labels: the role of literacy and numeracy. Am J Prev Med. 2006;31:391–8.PubMedCrossRef

20.

Eysenbach G, Köhler C. How do consumers search for and appraise health information on the world wide web? Qualitative study using focus groups, usability tests, and in-depth interviews. BMJ. 2002;324(7337):573–7. https://doi.org/10.1136/bmj.324.7337.573.PubMedPubMedCentralCrossRef

21.

Glenton C, Nilsen ES, Carlsen B. Lay perceptions of evidence-based information - a qualitative evaluation of a website for back pain sufferers. BMC Health Serv Res. 2006;6:34.PubMedPubMedCentralCrossRef

22.

Oxman AD, Austvoll-Dahlgren A, Garratt A, Rosenbaum S. Understanding of key concepts relevant to assessing claims about treatment effects: a survey of Norwegian adults. IHC Working Paper 2017. ISBN 978-82-8082-819-4. Available online at https://www.informedhealthchoices.org/wp-content/uploads/2016/08/CURE-survey-report-2017-03-01.pdf. Accessed 29 Jan 2020.

23.

Hoffmann TC, Del Mar C. Patients’ expectations of the benefits and harms of treatments, screening, and tests: a systematic review. JAMA Intern Med. 2015;175:274–86.PubMedCrossRef

24.

Hoffmann TC, Del Mar C. Clinicians’ expectations of the benefits and harms of treatments, screening, and tests: a systematic review. JAMA Intern Med. 2017;177:407–19.PubMedCrossRef

25.

Brownlee S, Chalkidou K, Doust J, Elshaug AG, Glasziou P, Heath I, et al. Evidence for overuse of medical services around the world. Lancet. 2017;390:156–68.PubMedPubMedCentralCrossRef

26.

Frass M, Strassl RP, Friehs H, Müllner M, Kundi M, Kaye AD. Use and acceptance of complementary and alternative medicine among the general population and medical personnel: a systematic review, vol. 12. New Orleans: the Academic Division of Ochsner Clinic Foundation; 2012. p. 45–56.

27.

Glasziou P, Straus S, Brownlee S, Trevena L, Dans L, Guyatt G, et al. Evidence for underuse of effective medical services around the world. Lancet. 2017;390:169–77.PubMedCrossRef

28.

Jones G, Steketee RW, Black RE, Bhutta ZA, Morris SS. How many child deaths can we prevent this year? Lancet. 2003;362:65–71.PubMedCrossRef

29.

Pierce H, Gibby AL, Forste R. Caregiver decision-making: household response to child illness in sub-Saharan Africa. Popul Res Policy Rev. 2016;35:581.PubMedPubMedCentralCrossRef

30.

WHO (World Health Organisation). 10 facts on health inequities and their causes. 2017. Available from: https://www.who.int/features/factfiles/health_inequities/en/. Cited 15 Nov 2019.

31.

Lynch J, Smith GD, Harper S, Hillemeier M, Ross N, Kaplan GA, et al. Is income inequality a determinant of population health? Part 1. A systematic review. Milbank Q. 2004;82:5–99.PubMedPubMedCentralCrossRef

32.

Lynch J, Smith GD, Harper S, Hillemeier M. Is income inequality a determinant of population health? Part 2. U.S. national and regional trends in income inequality and age- and cause-specific mortality. Milbank Q. 2004;82:355–400.PubMedPubMedCentralCrossRef

33.

Cutler D, Deaton A, Lleras-Muney A. The determinants of mortality. J Econ Perspect. 2006;20:97–120.CrossRef

34.

Messias E. Income inequality, illiteracy rate, and life expectancy in Brazil. Am J Public Health. 2003;93:1294–6.PubMedPubMedCentralCrossRef

35.

Castle JC, Chalmers I, Atkinson P, Badenoch D, Oxman AD, Austvoll-Dahlgren A, et al. Establishing a library of resources to help people understand key concepts in assessing treatment claims - the “Critical thinking and Appraisal Resource Library” (CARL). PLoS One. 2017;12:e0178666.PubMedPubMedCentralCrossRef

36.

Cusack L, Del Mar CB, Chalmers I, Gibson E, Hoffmann TC. Educational interventions to improve people’s understanding of key concepts in assessing the effects of health interventions: a systematic review. Syst Rev. 2018;7:68.PubMedPubMedCentralCrossRef

37.

The Informed Health Choices Group. The informed health choices project: using evidence to change the world. 2017. Available from: www.informedhealthchoices.org. Cited 20 Jun 2017.

38.

Austvoll-Dahlgren A, Oxman AD, Chalmers I, Nsangi A, Glenton C, Lewin S, et al. Key concepts that people need to understand to assess claims about treatment effects. J Evid Based Med. 2015;8:112–25.PubMedCrossRef

39.

Chalmers I, Oxman AD, Austvoll-Dahlgren A, Ryan-Vig S, Pannell S, Sewankambo N, et al. Key concepts for Informed Health Choices: a framework for helping people learn how to assess treatment claims and make informed choices. BMJ Evid Based Med. 2018;23:29–33.PubMedCrossRef

40.

Semakula D, Nsangi A, Oxman AD, Sewankambo NK. Priority setting for resources to improve the understanding of information about claims of treatment effects in the mass media. J Evid Based Med. 2015;8:84–90.PubMedCrossRef

41.

Semakula D, Nsangi A, Oxman M, Rosenbaum S, Oxman A, Austvoll-Dahlgren A, et al. Development of mass media resources to improve the ability of parents of primary school children in Uganda to assess the trustworthiness of claims about the benefits and harms of treatments. In: IHC Working Paper; 2018. Report No: ISBN: 978-82-8082-903-0.

42.

The Informed Health Choices Group. Learning resources. 2017. Available from: http://www.informedhealthchoices.org/learning-resources/. Cited 28 Jun 2017.

43.

Semakula D, Nsangi A, Oxman AD, Oxman M, Austvoll-Dahlgren A, Rosenbaum S, et al. Effects of the Informed Health Choices podcast on the ability of parents of primary school children in Uganda to assess claims about treatment effects: a randomised controlled trial. Lancet. 2017;390:389–98 Elsevier.PubMedCrossRef

44.

Nsangi A, Semakula D, Oxman AD, Austvoll-Dahlgren A, Oxman M, Rosenbaum S, et al. Effects of the Informed Health Choices primary school intervention on the ability of children in Uganda to assess the reliability of claims about treatment effects: a cluster-randomised controlled trial. Lancet. 2017;390:374–88 Elsevier.PubMedCrossRef

45.

Nsangi A, Semakula D, Oxman AD, Austvoll-Dahlgren A, Oxman M, Rosenbaum S, et al. Effects of the Informed Health Choices primary school intervention on the ability of children in Uganda to assess the reliability of claims about treatment effects, 1-year follow-up: a cluster-randomised trial. Trials. 2020;21:27. https://doi.org/10.1186/s13063-019-3960-9.

46.

Semakula D, Nsangi A, Oxman AD, Glenton C, Lewin S, Rosenbaum S, et al. Informed Health Choices media intervention for improving people’s ability to critically appraise the trustworthiness of claims about treatment effects: a mixed-methods process evaluation of a randomised trial in Uganda. BMJ Open 2019;9:e031510. https://doi.org/10.1136/bmjopen-2019-031510.PubMedPubMedCentralCrossRef

47.

Nsangi A, Semakula D, Oxman AD, Glenton C, Lewin S, Rosenbaum S, et al. Informed health choices intervention to teach primary school children in low-income countries to assess claims about treatment effects: process evaluation. BMJ Open. 2019;9(9):e030787. Published 2019 Sep 11. https://doi.org/10.1136/bmjopen-2019-030787.PubMedPubMedCentralCrossRef

48.

Semakula D, Nsangi A, Oxman M, Austvoll-Dahlgren A, Rosenbaum S, Kaseje M, et al. Can an educational podcast improve the ability of parents of primary school children to assess the reliability of claims made about the benefits and harms of treatments: study protocol for a randomised controlled trial. Trials. 2017;18:31.PubMedPubMedCentralCrossRef

49.

Austvoll-Dahlgren A, Semakula D, Nsangi A; The IHC Group, et al. Measuring ability to assess claims about treatment effects: the development of the ‘Claim Evaluation Tools’BMJ Open 2017;7:e013184. https://doi.org/10.1136/bmjopen-2016-013184.PubMedPubMedCentralCrossRef

50.

Austvoll-Dahlgren A, Guttersrud Ø, Nsangi A, Semakula D, Oxman AD. Measuring ability to assess claims about treatment effects: a latent trait analysis of items from the 'Claim Evaluation Tools' database using Rasch modelling. BMJ Open. 2017;7(5):e013185. Published 2017 May 25. https://doi.org/10.1136/bmjopen-2016-013185.PubMedPubMedCentralCrossRef

51.

The Informed Health Choices Group. The claim evaluation tools. 2018.

52.

Semakula D, Nsangi A, Guttersrud Ø, Oxman AD, Sewankambo NK, Austvoll-Dhalgren A. Measuring ability to assess claims about treatment effects in English and Luganda: evaluation of multiple-choice questions from the “Claim Evaluation Tools” database using Rasch modelling. IHC Working Paper. 2017. Available from: http://www.informedhealthchoices.org/wp-content/uploads/2016/08/Claim-2nd-Rasch-analysis-in-Uganda-2017-03-17.pdf. Cited 15 Apr 2018.

53.

Davies A, Gerrity M, Nordheim L, Peter O, Opiyo N, Sharples J, et al. Measuring ability to assess claims about treatment effects: establishment of a standard for passing and mastery. IHC Working Paper. 2017. Available from: http://www.informedhealthchoices.org/wp-content/uploads/2016/08/Claim-cut-off-IHC-Working-Paper-2017-01-09.pdf. Cited 15 Apr 2018.

54.

Donner A. Sample size requirements for stratified cluster randomization designs. Stat Med. 1992;11:743–50.PubMedCrossRef

55.

Lee DS. Training, wages, and sample selection: estimating sharp bounds on treatment effects. Rev Econ Stud. 2009;76:1071–102.CrossRef

56.

White IR, Thomas J. Standardized mean differences in individually-randomized and cluster-randomized trials, with applications to meta-analysis. Clin Trials. 2005;2:141–51.PubMedCrossRef

57.

Abrami PC, Bernard RM, Borokhovski E, Waddington DI, Wade CA, Persson T. Strategies for teaching students to think critically: a meta-analysis. Rev Educ Res. 2015;85:275–314.CrossRef

58.

Arthur W Jr, Bennett W Jr, Stanush PL, McNelly TL. Factors that influence skill decay and retention: a quantitative review and analysis. Hum Perform. 1998;11:57–101.CrossRef

59.

Custers EJFM. Long-term retention of basic science knowledge: a review study. Adv Heal Sci Educ. 2010;15:109–28.CrossRef

60.

Prince M. Does active learning work? A review of the research. J Eng Educ. 2004;93:223–31.CrossRef

61.

Ritter F, Baxter G, Kim JW, Srinivasmurthy S. Learning and retention. In: Lee JD, Kirlik A, Dainoff MJ, editors. The Oxford handbook of cognitive engineering. Oxford: Oxford University Press; 2013.

62.

Cromley J. Learning to think, learning to learn: what the science of thinking and Learning has to offer adult education. Washington, DC: National Institute for Literacy; 2000.

63.

David L. Social learning theory (Bandura). In: Learning theories; 2015. Available from: https://www.learning-theories.com/social-learning-theory-bandura.html. Cited 15 Apr 2018.

64.

Smith J, Baltes P. A life-span perspective on thinking and problem-solving. In: Schwebel M, Maher CA, Fagley NS, editors. Promoting cognitive growth over the life span. Hillsdale: Lawrence Erlbaum Associates; 1990.

65.

Hartshorne JK, Germine LT. When does cognitive functioning peak? The asynchronous rise and fall of different cognitive abilities across the life span. Psychol Sci. 2015;26:433–43.PubMedCrossRef

66.

The Informed Health Choices Group. Guide for translating and adapting the Informed Health Choices (IHC) podcast. In: Informed health choices working paper. Oslo; 2017. Available from: http://www.informedhealthchoices.org/wp-content/uploads/2016/08/GUIDE-for-translating-and-adapting-the-IHC-Podcast.pdf. Cited 15 Apr 2018.

67.

Slavin R, Madden NA. Measures inherent to treatments in program effectiveness reviews. J Res Educ Eff. 2011;1:370–80.

Title: Effects of the Informed Health Choices podcast on the ability of parents of primary school children in Uganda to assess the trustworthiness of claims about treatment effects: one-year follow up of a randomised trial
Authors: Daniel Semakula
Allen Nsangi
Andrew D. Oxman
Matt Oxman
Astrid Austvoll-Dahlgren
Sarah Rosenbaum
Angela Morelli
Claire Glenton
Simon Lewin
Laetitia Nyirazinyoye
Margaret Kaseje
Iain Chalmers
Atle Fretheim
Christopher J. Rose
Nelson K. Sewankambo
Publication date: 01-12-2020
Publisher: BioMed Central
Published in: Trials / Issue 1/2020
Electronic ISSN: 1745-6215
DOI: https://doi.org/10.1186/s13063-020-4093-x

At a glance: The ONWARDS insulin icodec trials

Springer Medicine

Effects of the Informed Health Choices podcast on the ability of parents of primary school children in Uganda to assess the trustworthiness of claims about treatment effects: one-year follow up of a randomised trial

Abstract

Introduction

Methods

Results

Conclusions

Trial registration

At a glance: The ONWARDS insulin icodec trials

Springer Medicine

Abstract

Introduction

Methods

Results

Conclusions

Trial registration

Please log in to get access to this content

Other articles of this Issue 1/2020

Intra-arterial versus standard intravenous administration of lutetium-177-DOTA-octreotate in patients with NET liver metastases: study protocol for a multicenter, randomized controlled trial (LUTIA trial)

Evaluation of an intervention to support decisions on disclosure in the employment setting (DECIDES): study protocol of a longitudinal cluster-randomized controlled trial

A multi-centre open-label two-arm randomised superiority clinical trial of azithromycin versus usual care in ambulatory COVID-19: study protocol for the ATOMIC2 trial

Measurement of cerebrovascular reserve by multimodal imaging for cerebral arterial occlusion or stenosis patients: protocol of a prospective, randomized, controlled clinical study

Driving Pressure During General Anesthesia for Open Abdominal Surgery (DESIGNATION): study protocol of a randomized clinical trial

Acupuncture for diabetic peripheral neuropathy: study protocol for a randomized, placebo-controlled trial