Skip to main content
Top
Published in: BMC Medical Research Methodology 1/2019

Open Access 01-12-2019 | Air Pollutants | Research article

Languages for different health information readers: multitrait-multimethod content analysis of Cochrane systematic reviews textual summary formats

Authors: Jasna Karačić, Pierpaolo Dondio, Ivan Buljan, Darko Hren, Ana Marušić

Published in: BMC Medical Research Methodology | Issue 1/2019

Login to get access

Abstract

Background

Although subjective expressions and linguistic fluency have been shown as important factors in processing and interpreting textual facts, analyses of these traits in textual health information for different audiences are lacking. We analyzed the readability and linguistic psychological and emotional characteristics of different textual summary formats of Cochrane systematic reviews.

Methods

We performed a multitrait-multimethod cross-sectional study of Press releases available at Cochrane web site (n = 162) and corresponding Scientific abstracts (n = 158), Cochrane Clinical Answers (n = 35) and Plain language summaries in English (n = 156), French (n = 101), German (n = 41) and Croatian (n = 156). We used SMOG index to assess text readability of all text formats, and natural language processing tools (IBM Watson Tone Analyzer, Stanford NLP Sentiment Analysis and Linguistic Inquiry and Word Count) to examine the affective states and subjective information in texts of Scientific abstracts, Plain language summaries and Press releases.

Results

All text formats had low readability, with SMOG index ranging from a median of 15.6 (95% confidence interval (CI) 15.3–15.9) for Scientific abstracts to 14.7 (95% CI 14.4–15.0) for Plain language summaries. In all text formats, “Sadness” was the most dominantly perceived emotional tone and the style of writing was perceived as “Analytical” and “Tentative”. At the psychological level, all text formats exhibited the predominant “Openness” tone, and Press releases scored higher on the scales of “Conscientiousness”, “Agreeableness” and “Emotional range”. Press releases had significantly higher scores than Scientific abstracts and Plain language summaries on the dimensions of “Clout”, and “Emotional tone”.

Conclusions

Although the readability of Plain language summaries was higher than that of text formats targeting more expert audiences, the required literacy was much higher than the recommended US 6th grade level. The language of Press releases was generally more engaging than that of Scientific abstracts and Plain language summaries, which are written by the authors of systematic reviews. Preparation of textual summaries about health evidence for different audiences should take into account readers’ subjective experiences to encourage cognitive processing and reaction to the provided information.
Appendix
Available only for authorised users
Literature
1.
go back to reference No authors listed. Health literacy: report of the council on scientific affairs. Ad hoc committee on health literacy for the council on scientific affairs, American Medical Association. JAMA. 1999;281:552–7.CrossRef No authors listed. Health literacy: report of the council on scientific affairs. Ad hoc committee on health literacy for the council on scientific affairs, American Medical Association. JAMA. 1999;281:552–7.CrossRef
2.
go back to reference Berkman ND, Sheridan SL, Donahue KE, Halpern DJ, Crotty K. Low health literacy and health outcomes: an updated systematic review. Ann Intern Med. 2011;155:97–107.CrossRef Berkman ND, Sheridan SL, Donahue KE, Halpern DJ, Crotty K. Low health literacy and health outcomes: an updated systematic review. Ann Intern Med. 2011;155:97–107.CrossRef
3.
go back to reference Kim H, Xie B. Health literacy in the eHealth era: a systematic review of the literature. Patient Educ Couns. 2017;100:1073–82.CrossRef Kim H, Xie B. Health literacy in the eHealth era: a systematic review of the literature. Patient Educ Couns. 2017;100:1073–82.CrossRef
4.
go back to reference Kurtzman ET, Greene J. Effective presentation of health care performance information for consumer decision making: a systematic review. Patient Educ Couns. 2016;99:36–43.CrossRef Kurtzman ET, Greene J. Effective presentation of health care performance information for consumer decision making: a systematic review. Patient Educ Couns. 2016;99:36–43.CrossRef
5.
go back to reference Vargas CR, Koolen PG, Chuang DJ, Ganor O, Lee BT. Online patient resources for breast reconstruction: an analysis of readability. Plast Reconstr Surg. 2014;134:406–13.CrossRef Vargas CR, Koolen PG, Chuang DJ, Ganor O, Lee BT. Online patient resources for breast reconstruction: an analysis of readability. Plast Reconstr Surg. 2014;134:406–13.CrossRef
6.
go back to reference Williams AM, Muir KW, Rosdahl JA. Readability of patient education materials in ophthalmology: a single-institution study and systematic review. BMC Ophthalmol. 2016;16:133.CrossRef Williams AM, Muir KW, Rosdahl JA. Readability of patient education materials in ophthalmology: a single-institution study and systematic review. BMC Ophthalmol. 2016;16:133.CrossRef
7.
go back to reference Morony S, Flynn M, McCaffery KJ, Jansen J, Webster AC. Readability of written materials for ckd patients: a systematic review. Am J Kidney Dis. 2015;65:842–50.CrossRef Morony S, Flynn M, McCaffery KJ, Jansen J, Webster AC. Readability of written materials for ckd patients: a systematic review. Am J Kidney Dis. 2015;65:842–50.CrossRef
8.
go back to reference U.S. National Institutes of Health (NIH), National Cancer Institute. Readability testing. In: Pretesting in health communications: methods, examples, and resources for improving health messages and materials. Bethesda, MD: National Cancer Institute; 1982. U.S. National Institutes of Health (NIH), National Cancer Institute. Readability testing. In: Pretesting in health communications: methods, examples, and resources for improving health messages and materials. Bethesda, MD: National Cancer Institute; 1982.
9.
go back to reference Kutner M, Greenburg E, Jin Y, Paulsen C. The health literacy of America’s adults: results from the 2003 National Assessment of adult literacy. (NCES 2006–483). National Center for education Statistics. 2006. Kutner M, Greenburg E, Jin Y, Paulsen C. The health literacy of America’s adults: results from the 2003 National Assessment of adult literacy. (NCES 2006–483). National Center for education Statistics. 2006.
10.
go back to reference Beaunoyer E, Arsenault M, Lomanowska AM, Guitton MJ. Understanding online health information: evaluation, tools, and strategies. Patient Educ Couns. 2017;100:183–9.CrossRef Beaunoyer E, Arsenault M, Lomanowska AM, Guitton MJ. Understanding online health information: evaluation, tools, and strategies. Patient Educ Couns. 2017;100:183–9.CrossRef
11.
go back to reference Alter AL, Oppenhiemer DM. Uniting the tribes of fluency to form a metacognitive nation. Personal Social Psychol Rev. 2009;13:219–35.CrossRef Alter AL, Oppenhiemer DM. Uniting the tribes of fluency to form a metacognitive nation. Personal Social Psychol Rev. 2009;13:219–35.CrossRef
12.
go back to reference Pennebaker JW, Mehl MR, Niederhoffer KG. Psychological aspects of natural language use: our words, our selves. Annu Rev Psychol. 2003;54:547–77.CrossRef Pennebaker JW, Mehl MR, Niederhoffer KG. Psychological aspects of natural language use: our words, our selves. Annu Rev Psychol. 2003;54:547–77.CrossRef
15.
go back to reference Jelicic Kadic A, Fidahic M, Vujcic M, Saric F, Propadalo I, Marelja I, et al. Cochrane plain language summaries are highly heterogeneous with low adherence to the standards. BMC Med Res Methodol. 2016;16:61.CrossRef Jelicic Kadic A, Fidahic M, Vujcic M, Saric F, Propadalo I, Marelja I, et al. Cochrane plain language summaries are highly heterogeneous with low adherence to the standards. BMC Med Res Methodol. 2016;16:61.CrossRef
16.
go back to reference Luk A, Aslani P. Tools used to evaluate written medicine and health information: document and user perspectives. Health Educ Behav. 2011;38:389–403.CrossRef Luk A, Aslani P. Tools used to evaluate written medicine and health information: document and user perspectives. Health Educ Behav. 2011;38:389–403.CrossRef
17.
go back to reference Brangan S. Development of SMOG-Cro readability formula for healthcare communication and patient education. Coll Antropol. 2015;39:11–20.PubMed Brangan S. Development of SMOG-Cro readability formula for healthcare communication and patient education. Coll Antropol. 2015;39:11–20.PubMed
18.
go back to reference Vucemilo L, Borovecki A. Readability and content assessment of informed consent forms for medical procedures in Croatia. PLoS One. 2015;10:e0138017.CrossRef Vucemilo L, Borovecki A. Readability and content assessment of informed consent forms for medical procedures in Croatia. PLoS One. 2015;10:e0138017.CrossRef
23.
go back to reference Hirschberg J, Manning CD. Advances in natural language processing. Science. 2015;349:261–6.CrossRef Hirschberg J, Manning CD. Advances in natural language processing. Science. 2015;349:261–6.CrossRef
24.
go back to reference Socher R, Perelygin A, Wu J, Chuang J, Manning CD, Ng A, Potts, C. Recursive deep models for semantic compositionality over a sentiment treebank. In: Proceedings of the 2013 conference on empirical methods in natural language processing, Seattle, Washington, 18–21 October 2013. pp. 1631–1642. Socher R, Perelygin A, Wu J, Chuang J, Manning CD, Ng A, Potts, C. Recursive deep models for semantic compositionality over a sentiment treebank. In: Proceedings of the 2013 conference on empirical methods in natural language processing, Seattle, Washington, 18–21 October 2013. pp. 1631–1642.
25.
go back to reference Wagenmakers EJ, Love J, Marsman M, Jamil T, Ly A, Verhagen J, et al. Bayesian inference for psychology. Part II: example applications with JASP. Psychol Bull Rev. 2018;25:58–76.CrossRef Wagenmakers EJ, Love J, Marsman M, Jamil T, Ly A, Verhagen J, et al. Bayesian inference for psychology. Part II: example applications with JASP. Psychol Bull Rev. 2018;25:58–76.CrossRef
26.
go back to reference Wang LW, Miller MJ, Schmitt MR, Wen FK. Assessing readability formula differences with written health information materials: application, results, and recommendations. Res Social Adm Pharm. 2013;9:503–16.CrossRef Wang LW, Miller MJ, Schmitt MR, Wen FK. Assessing readability formula differences with written health information materials: application, results, and recommendations. Res Social Adm Pharm. 2013;9:503–16.CrossRef
27.
go back to reference Stokes A. The reliability of readability formulae. J Res Reading. 1978;1:21–34.CrossRef Stokes A. The reliability of readability formulae. J Res Reading. 1978;1:21–34.CrossRef
28.
go back to reference Contreras A, García-Alonso R, Echenique M, Daye-Contreras F. The SOL formulas for converting SMOG readability scores between health education materials written in Spanish, English, and French. J Health Commun. 1999;4(1):21–9.CrossRef Contreras A, García-Alonso R, Echenique M, Daye-Contreras F. The SOL formulas for converting SMOG readability scores between health education materials written in Spanish, English, and French. J Health Commun. 1999;4(1):21–9.CrossRef
29.
go back to reference Betschart P, Zumstein V, Hasan Ali O, Schmid HP, Abt D. Readability assessment of patient education material published by german-speaking associations of urology. Urol Int. 2018;100:79–84.CrossRef Betschart P, Zumstein V, Hasan Ali O, Schmid HP, Abt D. Readability assessment of patient education material published by german-speaking associations of urology. Urol Int. 2018;100:79–84.CrossRef
30.
go back to reference Hartley J. Three ways to improve the clarity of journal abstracts. Brit J Educ Psychol. 1994;64:331–43.CrossRef Hartley J. Three ways to improve the clarity of journal abstracts. Brit J Educ Psychol. 1994;64:331–43.CrossRef
31.
go back to reference Hartley J, Benjamin M. An evaluation of structured abstracts in journals published by the British Psychological Society. Brit J Educ Psychol. 1998;68:443–5.CrossRef Hartley J, Benjamin M. An evaluation of structured abstracts in journals published by the British Psychological Society. Brit J Educ Psychol. 1998;68:443–5.CrossRef
32.
go back to reference Plaven-Sigray P, Matheson GJ, Schiffler BC, Thompson WH. The readability of scientific texts is decreasing over time. eLife. 2017;6:e27725.CrossRef Plaven-Sigray P, Matheson GJ, Schiffler BC, Thompson WH. The readability of scientific texts is decreasing over time. eLife. 2017;6:e27725.CrossRef
33.
go back to reference Hinnant A, Len-Rios ME. Tacit understanding of health literacy. Interview and survey research with health journalists. Sci Commun. 2009;31:84–115.CrossRef Hinnant A, Len-Rios ME. Tacit understanding of health literacy. Interview and survey research with health journalists. Sci Commun. 2009;31:84–115.CrossRef
34.
go back to reference Buljan I, Malički M, Wager E, Puljak L, Hren D, Kellie F, West H, Alfirević Ž, Marušić A. No difference in knowledge obtained from infographic or plain language summary of a Cochrane systematic review: three randomized controlled trials. J Clin Epidemiol. 2018;97:86–94.CrossRef Buljan I, Malički M, Wager E, Puljak L, Hren D, Kellie F, West H, Alfirević Ž, Marušić A. No difference in knowledge obtained from infographic or plain language summary of a Cochrane systematic review: three randomized controlled trials. J Clin Epidemiol. 2018;97:86–94.CrossRef
35.
go back to reference Denecke K, Deng Y. Sentiment analysis in medical settings: new opportunities and challenges. Artif Intell Med. 2015;64:17–27.CrossRef Denecke K, Deng Y. Sentiment analysis in medical settings: new opportunities and challenges. Artif Intell Med. 2015;64:17–27.CrossRef
36.
go back to reference Falkenstein A, Tran B, Ludi D, Molkara A, Nguyen H, Tabuenca A, et al. Characteristics and correlates of word use in physician-patient communication. Ann Behav Med. 2016;50:664–77.CrossRef Falkenstein A, Tran B, Ludi D, Molkara A, Nguyen H, Tabuenca A, et al. Characteristics and correlates of word use in physician-patient communication. Ann Behav Med. 2016;50:664–77.CrossRef
37.
go back to reference Vater LB, Donohue JM, Arnold R, White DB, Chu E, Schenker Y. What are cancer centers advertising to the public?: a content analysis. Ann Intern Med. 2014;160:813–20.CrossRef Vater LB, Donohue JM, Arnold R, White DB, Chu E, Schenker Y. What are cancer centers advertising to the public?: a content analysis. Ann Intern Med. 2014;160:813–20.CrossRef
38.
go back to reference MacDonald SP. The language of journalism in treatments of hormone replacement news. Writ Commun. 2005;22:275–97.CrossRef MacDonald SP. The language of journalism in treatments of hormone replacement news. Writ Commun. 2005;22:275–97.CrossRef
39.
go back to reference Kimmerle J, Flemming D, Feinkohl I, Cress U. How laypeople understand the tentativeness of medical research news in the media. Sci Commun. 2015;37:173–89.CrossRef Kimmerle J, Flemming D, Feinkohl I, Cress U. How laypeople understand the tentativeness of medical research news in the media. Sci Commun. 2015;37:173–89.CrossRef
40.
go back to reference Gregory J, Miller S. Science in public: communication, culture, and credibility. New York: Plenum Trade; 1998. Gregory J, Miller S. Science in public: communication, culture, and credibility. New York: Plenum Trade; 1998.
41.
go back to reference Williams MV, Davis T, Parker RM, Weiss BD. The role of health literacy in patient-physician communication. Fam Med. 2002;34:383–9.PubMed Williams MV, Davis T, Parker RM, Weiss BD. The role of health literacy in patient-physician communication. Fam Med. 2002;34:383–9.PubMed
Metadata
Title
Languages for different health information readers: multitrait-multimethod content analysis of Cochrane systematic reviews textual summary formats
Authors
Jasna Karačić
Pierpaolo Dondio
Ivan Buljan
Darko Hren
Ana Marušić
Publication date
01-12-2019
Publisher
BioMed Central
Keyword
Air Pollutants
Published in
BMC Medical Research Methodology / Issue 1/2019
Electronic ISSN: 1471-2288
DOI
https://doi.org/10.1186/s12874-019-0716-x

Other articles of this Issue 1/2019

BMC Medical Research Methodology 1/2019 Go to the issue