Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress
Clinical Collections
Advances in Alzheimer’s

Keynote webinar | Spotlight on medication adherence

LIVE: Thursday 27th June 2024, 18:00-19:30 (CEST)
Join our expert panel to discover why you need to understand the drivers of non-adherence in your patients, and how you can optimize medication adherence in your clinics to drastically improve patient outcomes.
ADVANCED SEARCH
Log in
Top
BMC Medical Research Methodology
Published in: BMC Medical Research Methodology 1/2020

Open Access 01-12-2020 | Research article

Translation method is validity evidence for construct equivalence: analysis of secondary data routinely collected during translations of the Health Literacy Questionnaire (HLQ)

Authors: Melanie Hawkins, Christina Cheng, Gerald R. Elsworth, Richard H. Osborne

Published in: BMC Medical Research Methodology | Issue 1/2020

Login to get access

Abstract

Background

Cross-cultural research with patient-reported outcomes measures (PROMs) assumes that the PROM in the target language will measure the same construct in the same way as the PROM in the source language. Yet translation methods are rarely used to qualitatively maximise construct equivalence or to describe the intents of each item to support common understanding within translation teams. This study aimed to systematically investigate the utility of the Translation Integrity Procedure (TIP), in particular the use of item intent descriptions, to maximise construct equivalence during the translation process, and to demonstrate how documented data from the TIP contributes evidence to a validity argument for construct equivalence between translated and source language PROMs.

Methods

Analysis of secondary data was conducted on routinely collected data in TIP Management Grids of translations (n = 9) of the Health Literacy Questionnaire (HLQ) that took place between August 2014 and August 2015: Arabic, Czech, French (Canada), French (France), Hindi, Indonesian, Slovak, Somali and Spanish (Argentina). Two researchers initially independently deductively coded the data to nine common types of translation errors. Round two of coding included an identified 10th code. Coded data were compared for discrepancies, and checked when needed with a third researcher for final code allocation.

Results

Across the nine translations, 259 changes were made to provisional forward translations and were coded into 10 types of errors. Most frequently coded errors were Complex word or phrase (n = 99), Semantic (n = 54) and Grammar (n = 27). Errors coded least frequently were Cultural errors (n = 7) and Printed errors (n = 5).

Conclusions

To advance PROM validation practice, this study investigated a documented translation method that includes the careful specification of descriptions of item intents. Assumptions that translated PROMs have construct equivalence between linguistic contexts can be incorrect due to errors in translation. Of particular concern was the use of high level complex words by translators, which, if undetected, could cause flawed interpretation of data from people with low literacy. Item intent descriptions can support translations to maximise construct equivalence, and documented translation data can contribute evidence to justify score interpretation and use of translated PROMS in new linguistic contexts.
Appendix
Available only for authorised users
Footnotes
1
We use ‘patient-reported outcome measure’ (PROM) to encompass all health-related assessments that are based on individuals’ responses to self-report questionnaires, whether written or verbally administered.
 
Literature
1.
Beaton DE, et al. Guidelines for the process of cross-cultural adaptation of self-report measures. Spine. 2000;25(24):3186–91.PubMedCrossRef
2.
Byrne BM, Campbell TL. Cross-cultural comparisons and the presumption of equivalent measurement and theoretical structure a look beneath the surface. J Cross-Cult Psychol. 1999;30(5):555–74.CrossRef
3.
Epstein J, Santo RM, Guillemin F. A review of guidelines for cross-cultural adaptation of questionnaires could not bring out a consensus. J Clin Epidemiol. 2015;68(4):435–41.PubMedCrossRef
4.
Flaherty JA, et al. Developing instruments for cross-cultural psychiatric research. J Nerv Ment Dis. 1988;176(5):257–63.CrossRef
5.
Luo N, et al. Do English and Chinese EQ-5D versions demonstrate measurement equivalence? An exploratory study. Health Qual Life Outcomes. 2003;1(1):1.CrossRef
6.
Byrne BM, Watkins D. The issue of measurement invariance revisited. J Cross-Cult Psychol. 2003;34(2):155–75.CrossRef
7.
Wild D, et al. Multinational trials—recommendations on the translations required, approaches to using the same language in different countries, and the approaches to support pooling the data: the ISPOR patient-reported outcomes translation and linguistic validation good research practices task force report. Value Health. 2009;12(4):430–40.PubMedCrossRef
8.
Mullen MR. Diagnosing measurement equivalence in cross-national research. J Int Bus Stud. 1995;26(3):573–96.CrossRef
9.
Oliveri ME, Lawless R, Young JW. A validity framework for the use and development of exported assessments. Princeton, NJ: ETS; 2015.
10.
AERA, APA, and NCME. Standards for educational and psychological testing. Washington, DC: American Educational Research Association; 2014.
11.
Herdman M, Fox-Rushby J, Badia X. ‘Equivalence’and the translation and adaptation of health-related quality of life questionnaires. Qual Life Res. 1997;6(3):237–47.PubMedCrossRef
12.
Millsap RE. Statistical approaches to measurement invariance. London: Routledge; 2012.CrossRef
13.
Teresi JA, Jones RN. Methodological Issues in Examining Measurement Equivalence in Patient Reported Outcomes Measures: Methods Overview to the Two-Part Series,“Measurement Equivalence of the Patient Reported Outcomes Measurement Information System®(PROMIS®) Short Forms”. Psychol Test Assess Model. 2016;58(1):37.PubMedPubMedCentral
14.
Sireci SG, Patsula L, Hambleton RK. Statistical methods for identifying flaws in the test adaptation process. In: Hambleton RK, Merenda PF, Speielberger CD, editors. Adapting educational and psychological tests for cross-cultural assessment. Hillsdale, NJ: Lawrence Erlbaum; 2005. p. 93–115.
15.
Kankaraš M, Moors G. Researching measurement equivalence in cross-cultural studies. Psihologija. 2010;43(2):121–36.CrossRef
16.
Tennant A, Conaghan PG. The Rasch measurement model in rheumatology: what is it and why use it? When should it be applied, and what should one look for in a Rasch paper? Arthritis Care & Research. 2007;57(8):1358–62.CrossRef
17.
Byrne BM. Testing for multigroup equivalence of a measuring instrument: a walk through the process. Psicothema. 2008;20(4):872–82.PubMed
18.
Hawkins M, Elsworth GR, Osborne RH. Application of validity theory and methodology to patient-reported outcome measures (PROMs): building an argument for validity. Qual Life Res. 2018:1–16.
19.
Acquadro, C., et al., Emerging good practices for translatability assessment (TA) of patient-reported outcome (PRO) measures. 2018. 2(1): ssss.
20.
Sidani, S., et al., Cultural adaptation and translation of measures: an integrated method 2010. 33(2): p. 133–143.
21.
Wild D, et al. Principles of good practice for the translation and cultural adaptation process for patient-reported outcomes (PRO) measures: report of the ISPOR task force for translation and cultural adaptation. Value in Health. 2005;8(2):94–104.PubMedCrossRef
22.
23.
Kane MT. Validating the interpretations and uses of test scores. J Educ Meas. 2013;50(1):1–73.CrossRef
24.
Messick, S., Validity, in Educational Measurement, R. Linn. 1989, American Council on Education/Macmillan Publishing Company: New York.
25.
Hunt S. Cross-cultural comparability of quality of life measures. Drug Information J. 1993;27(2):395–400.CrossRef
26.
Sperber AD. Translation and validation of study instruments for cross-cultural research. Gastroenterology. 2004;126:S124–8.PubMedCrossRef
27.
Buchbinder R, et al. A validity-driven approach to the understanding of the personal and societal burden of low back pain: development of a conceptual and measurement model. Arthritis Research Ther. 2011;13(5):R152.CrossRef
28.
Beck CT, Bernal H, Froman RD. Methods to document semantic equivalence of a translated scale. Res Nurs Health. 2003;26(1):64–73.PubMedCrossRef
29.
Zumbo BD, Chan EK. Validity and validation in social, behavioral, and health sciences. Social indicators research series. Switzerland: Springer International Publishing; 2014.
30.
McKenna SP, Doward LC. The translation and cultural adaptation of patient-reported outcome measures. Value Health. 2005;8(2):89–91.PubMedCrossRef
31.
Herdman M, Fox-Rushby J, Badia X. A model of equivalence in the cultural adaptation of HRQoL instruments: the universalist approach. Qual Life Res. 1998;7(4):323–35.PubMedCrossRef
32.
Acquadro C, et al. Literature review of methods to translate health-related quality of life questionnaires for use in multinational clinical trials. Value Health. 2008;11(3):509–21.PubMedCrossRef
33.
Stewart AL, Napoles-Springer A. Health-Related-Quality-of-Life Assessments in Diverse Groups in the United States. Med Care. 2000;38(9):II-102–24.
34.
Downing SM, Haladyna TM. Handbook of test development: Lawrence Erlbaum Associates Publishers; 2006.
35.
American Educational Research Association, et al. Standards for educational and psychological testing. Washington, DC: American Educational Research Association; 1999.
36.
American Educational Research Association, American Psychological Association, and National Council on Measurement in Education. Standards for educational and psychological testing: American Educational Research Association; 1985.
37.
Messick S. Test validity and the ethics of assessment. Am Psychol. 1980;35(11):1012.CrossRef
38.
Messick S. Validity of psychological assessment: validation of inferences from persons' responses and performances as scientific inquiry into score meaning. Am Psychol. 1995;50(9):741.CrossRef
39.
Kane M. The argument-based approach to validation. Sch Psychol Rev. 2013;42(4):448–57.
40.
Loevinger J. Objective tests as instruments of psychological theory. Psychol Rep. 1957.
41.
Campbell DT, Fiske DW. Convergent and discriminant validation by the multitrait-multimethod matrix. Psychol Bull. 1959;56(2):81.PubMedCrossRef
42.
Dunnette MD, Borman WC. Personnel selection and classification systems. Annu Rev Psychol. 1979;30(1):477–525.CrossRef
43.
Cronbach, L.J., Test Validation, in Educational Measurement, R.L. Thorndike, W.H. Angoff, and E.F. Lindquist, Editors. 1971, American Council on Education Washington p 483-507.
44.
Messick S. The standard problem: meaning and values in measurement and evaluation. Am Psychol. 1975;30(10):955.CrossRef
45.
Guion RM. On Trinitarian doctrines of validity. Prof Psychol. 1980;11(3):385.
46.
Hubley AM, Zumbo BD. A dialectic on validity: where we have been and where we are going. J Gen Psychol. 1996;123(3):207–15.CrossRef
47.
48.
Sawatzky R, et al. Montreal accord on patient-reported outcomes (PROs) use series–paper 7: modern perspectives of measurement validation emphasize justification of inferences based on patient reported outcome scores. J Clin Epidemiol. 2017;89:154–9.PubMedCrossRef
49.
Kane MT. Explicating validity. Assessment in Education: Principles, Policy & Practice. 2016;23(2):198–211.
50.
Kuliś D, et al. EORTC quality of life group translation procedure. Brussels: EORTC Quality of Life Group; 2017.
51.
Reeve BB, et al. ISOQOL recommends minimum standards for patient-reported outcome measures used in patient-centered outcomes and comparative effectiveness research. Qual Life Res. 2013;22(8):1889–905.PubMedCrossRef
52.
Guillemin F, Bombardier C, Beaton D. Cross-cultural adaptation of health-related quality of life measures: literature review and proposed guidelines. J Clin Epidemiol. 1993;46(12):1417–32.PubMedCrossRef
53.
U.S. Food and Drug Administration, et al. Guidance for industry: patient-reported outcome measures: use in medical product development to support labeling claims. In: Federal Register. Maryland: U.S. Department of Health and Human Services Food and Drug Administration; 2009. p. 65132–3.
54.
Kolarčik P, et al. Validation of Slovak version of Health Literacy Questionnaire. In: The European Journal of Public Health; 2015. p. ckv176. 151.
55.
56.
57.
Osborne RH, et al. The grounded psychometric development and initial validation of the health literacy questionnaire (HLQ). BMC Public Health. 2013;13:658.PubMedPubMedCentralCrossRef
58.
Epstein J, et al. Cross-cultural adaptation of the health education impact questionnaire: experimental study showed expert committee, not back-translation, added value. J Clin Epidemiol. 2015;68(4):360–9.PubMedCrossRef
59.
Hagell P, et al. Effects of method of translation of patient-reported health outcome questionnaires: a randomized study of the translation of the rheumatoid arthritis quality of life (RAQoL) instrument for Sweden. Value Health. 2010;13(4):424–30.PubMedCrossRef
60.
Nutbeam D. Health promotion glossary. Health Promot Int. 1998;13(4):349–64.CrossRef
61.
Embretson SE, Reise SP. Item response theory for psychologists. Maheah. New Jersey: Lawrence Erlbaum associates, Publishers; 2000.
62.
63.
Beauchamp A, et al. Distribution of health literacy strengths and weaknesses across socio-demographic groups: a cross-sectional survey using the health literacy questionnaire (HLQ). BMC Public Health. 2015;15:678.PubMedPubMedCentralCrossRef
64.
Batterham RW, et al. The OPtimising HEalth LIterAcy (Ophelia) process: study protocol for using health literacy profiling and community engagement to create and implement health reform. BMC Public Health. 2014;14(1):694.PubMedPubMedCentralCrossRef
65.
Beauchamp A, et al. Systematic development and implementation of interventions to Optimise health literacy and access (Ophelia). BMC Public Health. 2017;17(1):230.PubMedPubMedCentralCrossRef
66.
Hawkins M, et al. The health literacy questionnaire (HLQ) at the patient-clinician interface: a qualitative study of what patients and clinicians mean by their HLQ scores. BMC Health Serv Res. 2017;17(1):309.PubMedPubMedCentralCrossRef
67.
Elsworth GR, Beauchamp A, Osborne RH. Measuring health literacy in community agencies: a Bayesian study of the factor structure and measurement invariance of the health literacy questionnaire (HLQ). BMC Health Serv Res. 2016;16(1):508.PubMedPubMedCentralCrossRef
68.
Kolarčik P, Belak A, Osborne RH. The Ophelia (OPtimise HEalth LIteracy and access) process. Using health literacy alongside grounded and participatory approaches to develop interventions in partnership with marginalised populations. Eur Health Psychologist. 2015;17(6):297–304.
69.
Kolarčik P, et al. Structural properties and psychometric improvements of the health literacy questionnaire in a Slovak population. Int J Public Health. 2017;62(5):591–604.PubMedCrossRef
70.
Nolte S, et al. The Health Literacy Questionnaire: Translation and construct replication in 1,058 persons in Germany. Eur J Public Health. 2015;25(suppl 3):ckv172. 043.
71.
Vamos S, et al. Exploring health literacy profiles of Texas university students. Health Behav Pol Rev. 2016;3(3):209–25.CrossRef
72.
Bo A, et al. National indicators of health literacy: ability to understand health information and to engage actively with healthcare providers - a population-based survey among Danish adults. BMC Public Health. 2014;14(1):1095.PubMedPubMedCentralCrossRef
73.
Mullan J, et al. Health literacy amongst health professional university students: a study using the health literacy questionnaire. Education Sciences. 2017;7(2):54.CrossRef
74.
Goodwin BC, et al. Health literacy and the health status of men with prostate cancer. Psycho-Oncology. 2018;27(10):2374–81.PubMedCrossRef
75.
Debussche X, et al. Characterisation of health literacy strengths and weaknesses among people at metabolic and cardiovascular risk: validity testing of the health literacy questionnaire. SAGE Open Medicine. 2018;6:2050312118801250.PubMedCrossRef
76.
Australian Institute of Health and Welfare. Australia's health 2018. In: Australia's health series no.16. AUS 221. Canberra: AIHW; 2018.
77.
Koller M, et al. Translation procedures for standardised quality of life questionnaires: the European Organisation for Research and Treatment of Cancer (EORTC) approach. Eur J Cancer. 2007;43(12):1810–20.PubMedCrossRef
78.
Spurgeon SL. Evaluating the unintended consequences of assessment practices: construct irrelevance and construct underrepresentation. Meas Eval Couns Dev. 2017;50(4):275–81.CrossRef
79.
Beaton D, et al. Recommendations for the cross-cultural adaptation of health status measures. New York: American Academy of Orthopaedic Surgeons; 1998. p. 1–9.
80.
Mokkink LB, et al. COSMIN checklist manual. Amsterdam: University Medical Center; 2012.
81.
82.
Chapman DW, Carter JF. Translation procedures for the cross cultural use of measurement instruments. Educ Eval Policy Anal. 1979;1(3):71–6.CrossRef
83.
Caines J, Bridglall BL, Chatterji M. Understanding validity and fairness issues in high-stakes individual testing situations. Qual Assur Educ. 2014;22(1):5–18.CrossRef
Metadata
Title
Translation method is validity evidence for construct equivalence: analysis of secondary data routinely collected during translations of the Health Literacy Questionnaire (HLQ)
Authors
Melanie Hawkins
Christina Cheng
Gerald R. Elsworth
Richard H. Osborne
Publication date
01-12-2020
Publisher
BioMed Central
Published in
BMC Medical Research Methodology / Issue 1/2020
Electronic ISSN: 1471-2288
DOI
https://doi.org/10.1186/s12874-020-00962-8

Other articles of this Issue 1/2020

BMC Medical Research Methodology 1/2020 Go to the issue

Research article

Modelling seizure rates rather than time to an event within clinical trials of antiepileptic drugs

Research article

Challenges of recruiting emergency department patients to a qualitative study: a thematic analysis of researchers’ experiences

Research Article

Baseline-adjusted proportional odds models for the quantification of treatment effects in trials with ordinal sum score outcomes

Research article

Integrated Care Search: development and validation of a PubMed search filter for retrieving the integrated care research evidence

Research article

Who responds? An examination of response rates to a national postal survey of Aboriginal and Torres Strait Islander adults, 2018-2019

Technical advance

A novel method for controlling unobserved confounding using double confounders