Skip to main content
Top
Published in: Health and Quality of Life Outcomes 1/2017

Open Access 01-01-2017 | Research

Reliability analysis of the Chinese version of the Functional Assessment of Cancer Therapy – Leukemia (FACT-Leu) scale based on multivariate generalizability theory

Authors: Qiong Meng, Zheng Yang, Yang Wu, Yuanyuan Xiao, Xuezhong Gu, Meixia Zhang, Chonghua Wan, Xiaosong Li

Published in: Health and Quality of Life Outcomes | Issue 1/2017

Login to get access

Abstract

Background

The Functional Assessment of Cancer Therapy–Leukemia (FACT-Leu) scale, a leukemia-specific instrument for determining the health-related quality of life (HRQOL) in patients with leukemia, had been developed and validated, but there have been no reports on the development of a simplified Chinese version of this scale. This is a new exploration to analyze the reliability of the HRQOL measurement using multivariate generalizability theory (MGT). This study aimed to develop a Chinese version of the FACT-Leu scale and evaluate its reliability using MGT to provide evidence to support the revision and improvement of this scale.

Methods

The Chinese version of the FACT-Leu scale was developed by four steps: forward translation, backward translation, cultural adaptation and pilot-testing. The HRQOL was measured for eligible inpatients with leukemia using this scale to provide data. A single-facet multivariate Generalizability Study (G-study) design was demonstrated to estimate the variance–covariance components and then several Decision Studies (D-studies) with varying numbers of items were analyzed to obtain reliability coefficients and to understand how much the measurement reliability could be vary as the number of items in MGT changes.

Results

One-hundred and one eligible inpatients diagnosed with leukemia were recruited and completed the HRQOL measurement at the time of admission to the hospital. In the G-study, the variation component of the patient-item interaction was largest while the variation component of the item was the smallest for the four of five domains, except for the leukemia-specific (LEUS) domain. In the D-study, at the level of domain, the generalizability coefficients (G) and the indexes of dependability (Ф) for four of the five domains were approximately equal to or greater than 0.80 except for the Emotional Well-being (EWB) domain (>0.70 but <0.80). For the overall scale, the composite G and composite Ф coefficients were greater than 0.90. Based on the G coefficient and Ф coefficient, two decision options for revising this scale considering the number of items were obtained: one is a 37-item version while the other is a 45-item version.

Conclusion

The Chinese version of the FACT-Leu scale has good reliability as a whole based on the results of MGT and the implementation of MGT could lead to more informed decisions in complex questionnaire design and improvement.
Appendix
Available only for authorised users
Literature
1.
go back to reference Chen W, Zheng R, Baade P, Zhang S, Zeng H, Bray F, Jemal A, Yu X, He J. Cancer Statistics in China, 2015. CA Cancer J Clin. 2016;66:115–32.CrossRefPubMed Chen W, Zheng R, Baade P, Zhang S, Zeng H, Bray F, Jemal A, Yu X, He J. Cancer Statistics in China, 2015. CA Cancer J Clin. 2016;66:115–32.CrossRefPubMed
2.
go back to reference Redaelli A, Stephens JM, Laskin BL, Pashos CL, Botteman MF. The burden and outcomes associated with four leukemias: AML, ALL, CLL and CML. Expert Rev Anticancer Ther. 2003;3:311–29.CrossRefPubMed Redaelli A, Stephens JM, Laskin BL, Pashos CL, Botteman MF. The burden and outcomes associated with four leukemias: AML, ALL, CLL and CML. Expert Rev Anticancer Ther. 2003;3:311–29.CrossRefPubMed
3.
go back to reference Stalfelt AM, Wadman B. Assessing quality of life in leukemia: presentation of an instrument for assessing quality of life in patients with blood malignancies. Qual Assur Health Care. 1993;5:201–11.CrossRefPubMed Stalfelt AM, Wadman B. Assessing quality of life in leukemia: presentation of an instrument for assessing quality of life in patients with blood malignancies. Qual Assur Health Care. 1993;5:201–11.CrossRefPubMed
4.
go back to reference Watson M, Zittoun R, Hall E, Solbu G, Wheatley K. A modular questionnaire for the assessment of longterm quality of life in leukaemia patients: the MRC/EORTC QLQ-LEU. Qual Life Res. 1996;5:15–9.CrossRefPubMed Watson M, Zittoun R, Hall E, Solbu G, Wheatley K. A modular questionnaire for the assessment of longterm quality of life in leukaemia patients: the MRC/EORTC QLQ-LEU. Qual Life Res. 1996;5:15–9.CrossRefPubMed
5.
go back to reference Efficace F, Baccarani M, Breccia M, Saussele S, Abel G, Caocci G, Guilhot F, Cocks K, Naeem A, Sprangers M, et al. International development of an EORTC questionnaire for assessing health-related quality of life in chronic myeloid leukemia patients: the EORTC QLQ-CML24. Qual Life Res. 2014;23:825–36.CrossRefPubMed Efficace F, Baccarani M, Breccia M, Saussele S, Abel G, Caocci G, Guilhot F, Cocks K, Naeem A, Sprangers M, et al. International development of an EORTC questionnaire for assessing health-related quality of life in chronic myeloid leukemia patients: the EORTC QLQ-CML24. Qual Life Res. 2014;23:825–36.CrossRefPubMed
6.
go back to reference Cella D, Jensen SE, Webster K, Hongyan D, Lai JS, Rosen S, Tallman MS, Yount S. Measuring health-related quality of life in leukemia: the Functional Assessment of Cancer Therapy--Leukemia (FACT-Leu) questionnaire. Value Health. 2012;15:1051–8.CrossRefPubMed Cella D, Jensen SE, Webster K, Hongyan D, Lai JS, Rosen S, Tallman MS, Yount S. Measuring health-related quality of life in leukemia: the Functional Assessment of Cancer Therapy--Leukemia (FACT-Leu) questionnaire. Value Health. 2012;15:1051–8.CrossRefPubMed
7.
go back to reference Cella DF, Tulsky DS, Gray G, Sarafian B, Linn E, Bonomi A, Silberman M, Yellen SB, Winicour P, Brannon J. The functional assessment of cancer therapy scale: development and validation of the general measure. J Clin Oncol. 1993;11:570–9.CrossRefPubMed Cella DF, Tulsky DS, Gray G, Sarafian B, Linn E, Bonomi A, Silberman M, Yellen SB, Winicour P, Brannon J. The functional assessment of cancer therapy scale: development and validation of the general measure. J Clin Oncol. 1993;11:570–9.CrossRefPubMed
8.
go back to reference Gage NA, Debra P, Hirn R. Increasing reliability of direct observation measurement approaches in emotional and/or behavioral disorders research using generalizability theory. Behav Disord. 2014;39:228–44. Gage NA, Debra P, Hirn R. Increasing reliability of direct observation measurement approaches in emotional and/or behavioral disorders research using generalizability theory. Behav Disord. 2014;39:228–44.
9.
go back to reference Berggraf L, Ulvenes PG, Wampold BE, Hoffart A, McCullough L. Properties of the Achievement of Therapeutic Objectives Scale (ATOS): a generalizability theory study. Psychother Res. 2012;22:327–47.CrossRefPubMed Berggraf L, Ulvenes PG, Wampold BE, Hoffart A, McCullough L. Properties of the Achievement of Therapeutic Objectives Scale (ATOS): a generalizability theory study. Psychother Res. 2012;22:327–47.CrossRefPubMed
10.
go back to reference Hall C. Comment: Generalizability theory and assessment in medical training. Neurology. 2015;85:1628.CrossRefPubMed Hall C. Comment: Generalizability theory and assessment in medical training. Neurology. 2015;85:1628.CrossRefPubMed
11.
go back to reference Iramaneerat C, Yudkowsky R, Myford CM, Downing SM. Quality control of an OSCE using generalizability theory and many-faceted Rasch measurement. Adv Health Sci Educ Theory Pract. 2008;13:479–93.CrossRefPubMed Iramaneerat C, Yudkowsky R, Myford CM, Downing SM. Quality control of an OSCE using generalizability theory and many-faceted Rasch measurement. Adv Health Sci Educ Theory Pract. 2008;13:479–93.CrossRefPubMed
12.
go back to reference Kang M, Kim Y, Rowe D. Reliability of peak stepping cadences using generalizability theory. Med Sci Sports Exerc. 2013;45:327.CrossRef Kang M, Kim Y, Rowe D. Reliability of peak stepping cadences using generalizability theory. Med Sci Sports Exerc. 2013;45:327.CrossRef
13.
go back to reference Preuss R. Using generalizability theory to develop clinical assessment protocols. Phys Ther. 2013;93:562–9.CrossRefPubMed Preuss R. Using generalizability theory to develop clinical assessment protocols. Phys Ther. 2013;93:562–9.CrossRefPubMed
14.
go back to reference Prion S, Gilbert G, Haerling K. Generalizability theory: an introduction with application to simulation evaluation. Clin Simul Nurs. 2016;12:546–54.CrossRef Prion S, Gilbert G, Haerling K. Generalizability theory: an introduction with application to simulation evaluation. Clin Simul Nurs. 2016;12:546–54.CrossRef
15.
go back to reference Ragan BG, Kang M, Marquez T, Bell GW, Zhu W. Graphic pain rating scale reliability using generalizability theory. Med Sci Sports Exerc. 2004;36:S295. Ragan BG, Kang M, Marquez T, Bell GW, Zhu W. Graphic pain rating scale reliability using generalizability theory. Med Sci Sports Exerc. 2004;36:S295.
16.
go back to reference Schünemann HJ, Norman G, Puhan MA, Ståhl E, Griffith L, Heels-Ansdell D, Montori VM, Wiklund I, Goldstein R, Mador MJ, Guyatt GH. Application of generalizability theory confirmed lower reliability of the standard gamble than the feeling thermometer. J Clin Epidemiol. 2007;60:1256–62.CrossRefPubMed Schünemann HJ, Norman G, Puhan MA, Ståhl E, Griffith L, Heels-Ansdell D, Montori VM, Wiklund I, Goldstein R, Mador MJ, Guyatt GH. Application of generalizability theory confirmed lower reliability of the standard gamble than the feeling thermometer. J Clin Epidemiol. 2007;60:1256–62.CrossRefPubMed
17.
go back to reference Stegner A, Cook D. Evaluating reliability and response bias in computer-administered pain rating scales: a generalizability theory analysis. J Pain. 2013;14:S22.CrossRef Stegner A, Cook D. Evaluating reliability and response bias in computer-administered pain rating scales: a generalizability theory analysis. J Pain. 2013;14:S22.CrossRef
18.
go back to reference Vangeneugden T, Laenen A, Geys H, Renard D, Molenberghs G. Applying concepts of generalizability theory on clinical trial data to investigate sources of variation and their impact on reliability. Biometrics. 2005;61:295–304.CrossRefPubMed Vangeneugden T, Laenen A, Geys H, Renard D, Molenberghs G. Applying concepts of generalizability theory on clinical trial data to investigate sources of variation and their impact on reliability. Biometrics. 2005;61:295–304.CrossRefPubMed
19.
go back to reference Volpe RJ, Briesch AM, Gadow KD. The efficiency of behavior rating scales to assess inattentive-overactive and oppositional defiant behaviors: Applying generalizability theory to streamline assessment. J Sch Psychol. 2011;49:131–55.CrossRefPubMed Volpe RJ, Briesch AM, Gadow KD. The efficiency of behavior rating scales to assess inattentive-overactive and oppositional defiant behaviors: Applying generalizability theory to streamline assessment. J Sch Psychol. 2011;49:131–55.CrossRefPubMed
20.
go back to reference Wasserman RH, Levy KN, Loken E. Generalizability theory in psychotherapy research: the impact of multiple sources of variance on the dependability of psychotherapy process ratings. Psychother Res. 2009;19:397–408.CrossRefPubMed Wasserman RH, Levy KN, Loken E. Generalizability theory in psychotherapy research: the impact of multiple sources of variance on the dependability of psychotherapy process ratings. Psychother Res. 2009;19:397–408.CrossRefPubMed
21.
go back to reference Paul CC, Jennifer J, Robert G, Mart Beth CG, Connolly G, Sarah RK, Jessica LH, Xin T. A generalizability theory analysis of group process ratings in the treatment of cocaine dependence. Psychother Res. 2011;21:252–66.CrossRef Paul CC, Jennifer J, Robert G, Mart Beth CG, Connolly G, Sarah RK, Jessica LH, Xin T. A generalizability theory analysis of group process ratings in the treatment of cocaine dependence. Psychother Res. 2011;21:252–66.CrossRef
22.
go back to reference Yang ZM, Zhang L. Generalizability theory and its applications. Beijing: Educational science Publishing House; 2003. p. 2–53. Yang ZM, Zhang L. Generalizability theory and its applications. Beijing: Educational science Publishing House; 2003. p. 2–53.
23.
go back to reference Chen D, Hu BY, Fan X, Li K. Measurement quality of the chinese early childhood program rating scale: an investigation using multivariate generalizability theory. J Psychoeduc Assess. 2014;32:236–48.CrossRef Chen D, Hu BY, Fan X, Li K. Measurement quality of the chinese early childhood program rating scale: an investigation using multivariate generalizability theory. J Psychoeduc Assess. 2014;32:236–48.CrossRef
24.
go back to reference Chavez LM, Garcia P, Ortiz N, Shrout PE. Applying generalizability theory methods to assess continuity and change on the Adolescent Quality of Life-Mental Health Scale (AQOL-MHS). Qual Life Res. 2016;25:3191–6.CrossRefPubMed Chavez LM, Garcia P, Ortiz N, Shrout PE. Applying generalizability theory methods to assess continuity and change on the Adolescent Quality of Life-Mental Health Scale (AQOL-MHS). Qual Life Res. 2016;25:3191–6.CrossRefPubMed
25.
go back to reference Wan C, Li H, Fan X, Yang R, Pan J, Chen W, Zhao R. Development and validation of the coronary heart disease scale under the system of quality of life instruments for chronic diseases QLICD-CHD: combinations of classical test theory and Generalizability theory. Health Qual Life Outcomes. 2014;12:82.CrossRefPubMedPubMedCentral Wan C, Li H, Fan X, Yang R, Pan J, Chen W, Zhao R. Development and validation of the coronary heart disease scale under the system of quality of life instruments for chronic diseases QLICD-CHD: combinations of classical test theory and Generalizability theory. Health Qual Life Outcomes. 2014;12:82.CrossRefPubMedPubMedCentral
26.
go back to reference Lei P, Lei G, Tian J, Zhou Z, Zhao M, Wan C. Development and validation of the irritable bowel syndrome scale under the system of quality of life instruments for chronic diseases QLICD-IBS: combinations of classical test theory and generalizability theory. Int J Color Dis. 2014;29:1245–55.CrossRef Lei P, Lei G, Tian J, Zhou Z, Zhao M, Wan C. Development and validation of the irritable bowel syndrome scale under the system of quality of life instruments for chronic diseases QLICD-IBS: combinations of classical test theory and generalizability theory. Int J Color Dis. 2014;29:1245–55.CrossRef
27.
go back to reference Wu J, Hu L, Zhang G, Liang Q, Meng Q, Wan C. Development and validation of the nasopharyngeal cancer scale among the system of quality of life instruments for cancer patients (QLICP-NA V2.0): combined classical test theory and generalizability theory. Qual Life Res. 2016;25:2087–100.CrossRefPubMed Wu J, Hu L, Zhang G, Liang Q, Meng Q, Wan C. Development and validation of the nasopharyngeal cancer scale among the system of quality of life instruments for cancer patients (QLICP-NA V2.0): combined classical test theory and generalizability theory. Qual Life Res. 2016;25:2087–100.CrossRefPubMed
28.
go back to reference Meng Q, Yang Z, Wan C, Luo J, Dai Y, Cun Y. Reliability analysis of quality of life instruments for cancer patients-gastric cancer (QLICP-GA) based on generalizability theory. Tumor. 2013;33:428–33. Meng Q, Yang Z, Wan C, Luo J, Dai Y, Cun Y. Reliability analysis of quality of life instruments for cancer patients-gastric cancer (QLICP-GA) based on generalizability theory. Tumor. 2013;33:428–33.
29.
go back to reference Li W, Luo J, Wan C, Li G, Lu Y, Yang H, Meng Q. Evaluation on reliability of the chinese version of functional assessment of cancer therapy - ovary cancer by classical test theory and generalizability theory. Chin Gen Pract. 2013;16:749–54. Li W, Luo J, Wan C, Li G, Lu Y, Yang H, Meng Q. Evaluation on reliability of the chinese version of functional assessment of cancer therapy - ovary cancer by classical test theory and generalizability theory. Chin Gen Pract. 2013;16:749–54.
30.
go back to reference Bravo G, Sene M, Arcand M. Reliability of health-related quality-of-life assessments made by older adults and significant others for health states of increasing cognitive impairment. Health Qual Life Outcomes. 2017;15:4.CrossRefPubMedPubMedCentral Bravo G, Sene M, Arcand M. Reliability of health-related quality-of-life assessments made by older adults and significant others for health states of increasing cognitive impairment. Health Qual Life Outcomes. 2017;15:4.CrossRefPubMedPubMedCentral
31.
go back to reference Nussbaum A. Multivariate generalizability theory in educational-measurement - an empirical-study. Appl Psychol Meas. 1984;8:219–30.CrossRef Nussbaum A. Multivariate generalizability theory in educational-measurement - an empirical-study. Appl Psychol Meas. 1984;8:219–30.CrossRef
32.
go back to reference Wu Y-F, Tzou H. A multivariate generalizability theory approach to standard setting. Appl Psychol Meas. 2015;39:507–24.CrossRef Wu Y-F, Tzou H. A multivariate generalizability theory approach to standard setting. Appl Psychol Meas. 2015;39:507–24.CrossRef
33.
go back to reference Brennan RL. Generalizability theory. New York: Springer-Verlag New York Inc; 2001.CrossRef Brennan RL. Generalizability theory. New York: Springer-Verlag New York Inc; 2001.CrossRef
34.
go back to reference Brennan RL. Manual for mGENOVA (Version 2.1). Iowa City: Iowa Testing Programs; 2001. p. 50. Brennan RL. Manual for mGENOVA (Version 2.1). Iowa City: Iowa Testing Programs; 2001. p. 50.
35.
go back to reference Winterstein BP, Willse JT, Kwapil TR, Silvia PJ. Assessment of Score Dependability of the Wisconsin Schizotypy Scales Using Generalizability Analysis. J Psychopathol Behav Assess. 2010;32:575–85.CrossRef Winterstein BP, Willse JT, Kwapil TR, Silvia PJ. Assessment of Score Dependability of the Wisconsin Schizotypy Scales Using Generalizability Analysis. J Psychopathol Behav Assess. 2010;32:575–85.CrossRef
36.
go back to reference Keller LA, Clauser BE, Swanson DB. Using multivariate generalizability theory to assess the effect of content stratification on the reliability of a performance assessment. Adv Health Sci Educ. 2010;15:717–33.CrossRef Keller LA, Clauser BE, Swanson DB. Using multivariate generalizability theory to assess the effect of content stratification on the reliability of a performance assessment. Adv Health Sci Educ. 2010;15:717–33.CrossRef
Metadata
Title
Reliability analysis of the Chinese version of the Functional Assessment of Cancer Therapy – Leukemia (FACT-Leu) scale based on multivariate generalizability theory
Authors
Qiong Meng
Zheng Yang
Yang Wu
Yuanyuan Xiao
Xuezhong Gu
Meixia Zhang
Chonghua Wan
Xiaosong Li
Publication date
01-01-2017
Publisher
BioMed Central
Published in
Health and Quality of Life Outcomes / Issue 1/2017
Electronic ISSN: 1477-7525
DOI
https://doi.org/10.1186/s12955-017-0664-2

Other articles of this Issue 1/2017

Health and Quality of Life Outcomes 1/2017 Go to the issue