Top

Published in:

Open Access 01-10-2011

The king’s foot of patient-reported outcomes: current practices and new developments for the measurement of change

Authors: Richard J. Swartz, Carolyn Schwartz, Ethan Basch, Li Cai, Diane L. Fairclough, Lori McLeod, Tito R. Mendoza, Bruce Rapkin, The SAMSI Psychometric Program Longitudinal Assessment of Patient-Reported Outcomes Working Group

Published in: Quality of Life Research | Issue 8/2011

Abstract

Purpose

Assessing change remains a challenge in patient-reported outcomes. In June 2009, a group of psychometricians, biostatisticians, and behavioral researchers from other disciplines convened as a Longitudinal Analysis of Patient-Reported Outcomes Working group as part of the Statistical and Applied Mathematical Sciences Institute Summer Psychometric program to discuss the complex issues that arise when conceptualizing and operationalizing “change” in patient-reported outcome (PRO) measures and related constructs. This white paper summarizes these issues and provides recommendations and possible paths for dealing with the complexities of measuring change.

Methods/Results

This article presents and discusses issues associated with: (1) conceptualizing and operationalizing change in PRO measures; (2) modeling change using state-of-the-art statistical methods; (3) impediments to detecting true change; (4) new developments to deal with these challenges; and (5) important gaps that are fertile ground for future research.

Conclusions

There was a consensus that important research still needs to be performed in order develop and refine high-quality PRO measures and statistical methods to analyze and model change in PRO constructs.

Brief History of Measurement Systems with a Chart of the Modernized Metric System. (1974). In N. B. O. Standards (Ed.), Washington, DC: Government Printing Office.

Swartz, R. J., Basch, E., Cai, L., Fairclough, D. L., McLeod, L. D., Mendoza, T. R., Rapkin, B., Schwartz, C. E., & The SAMSI Psychometric Program Longitudinal Assessment of Patient-Reported Outcomes Working Group. (2010). The king’s foot of patient-reported outcomes: Current practices and new developments for the measurement of change. from http://www.bepress.com/mdandersonbiostat/paper60.

Lord, F. M., & Novick, M. R. (1968). Statistical theories of mental test scores. Reading, Mass.: Addison-Wesley Pub. Co.

de Boeck, P., & Wilson, M. (2004). Explanatory item response models: A generalized linear and nonlinear approach. New York: Springer.

McDonald, R. P. (1999). Test theory: A unified treatment. Mahwah, N.J.: L. Erlbaum Associates.

Cronbach, L. I., & Furby, L. (1970). How we should measure “change”—or should we? Psychological Bulletin, 74(1), 68–80.CrossRef

Lord, F. M. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Lawrence Erlbaum.

van der Linden, W. J., & Hambleton, R. K. (Eds.). (1997). Handbook of modern item response theory. New York: Springer.

Embretson, S. E., & Reise, S. P. (2000). Item response theory for psychologists. Mahwah, N.J.: L. Erlbaum Associates.

10.

Norman, G. R., Wyrwich, K. W., & Patrick, D. L. (2007). The mathematical relationship among different forms of responsiveness coefficients. Quality of Life Research: An International Journal of Quality of Life Aspects of Treatment, Care and Rehabilitation, 16(5), 815–822.

11.

Fayers, P. M., & Hays, R. D. (2005). Assessing quality of life in clinical trials: Methods and practice (2nd ed.). Oxford: Oxford University Press.

12.

Beaton, D. E., Bombardier, C., Katz, J. N., & Wright, J. G. (2001). A taxonomy for responsiveness. Journal of Clinical Epidemilology, 54(12), 1204–1217.CrossRef

13.

Cohen, J. (1992). A power primer. Psychological Bulletin, 112(1), 155–159.PubMedCrossRef

14.

US Department of Health and Human Services. (2006). Food and drug administration. Guidance for industry patient-reported outcome measures: use in medical product development to support labeling claims draft guidance.

15.

de Vet, H. C., Terwee, C. B., Ostelo, R. W., Beckerman, H., Knol, D. L., & Bouter, L. M. (2006). Minimal changes in health status questionnaires: Distinction between minimally detectable change and minimally important change. Health and Quality of Life Outcomes, 4, 54.PubMedCrossRef

16.

Juniper, E. F., Guyatt, G. H., Willan, A., & Griffith, L. E. (1994). Determining a minimal important change in a disease-specific quality of life questionnaire. Journal of Clinical Epidemiology, 47(1), 81–87.PubMedCrossRef

17.

Norman, G. R., Sloan, J., & Wyrwich, K. W. (2003). Interpretation of changes in health-related quality-of-life. Medical Care, 4, 582–592.

18.

Wyrwich, K. W., Tierney, W. M., & Wolinsky, F. D. (1999). Further evidence supporting an SEM-based criterion for identifying meaningful intra-individual changes in health-related quality of life. Journal of Clinical Epidemiology, 52(9), 861–873.PubMedCrossRef

19.

Guyatt, G. H., Juniper, E. F., Walter, S. D., Griffith, L. E., & Goldstein, R. S. (1998). Interpreting treatment effects in randomised trials. British Medical Journal, 316, 690–693.PubMed

20.

Norman, G. R., Sridhar, F. G., Guyatt, G. H., & Walter, S. D. (2001). Relation of distribution- and anchor-based approaches in interpretation of changes in health related quality of life. Medical Care, 29, 1039–1047.CrossRef

21.

Revicki, D., Hays, R. D., Cella, D., & Sloan, J. (2008). Recommended methods for determining responsiveness and minimally important differences for patient-reported outcomes. Journal of Clinical Epidemiology, 61(2), 102–109.PubMedCrossRef

22.

Sloan, J., Cella, D., & Hays, R. D. (2005). Clinical significance of patient-reported questionnaire data: Another step towards consensus. Journal of Clinical Epidemiology, 58, 1217–1219.PubMedCrossRef

23.

Raudenbush, S. W., & Byrk, A. S. (2002). Heirarchical linear models: Applications and data analysis methods (2nd ed.). Thousand Oaks, CA: Sage.

24.

Reise, S. P., & Duan, N. (2003). Multilevel modeling: Methodological advances, issues, and applications. Mahwah, N.J.; London: Lawrence Erlbaum Associates.

25.

Demidenko, E. (2004). Mixed models: Theory and applications. Hoboken, NJ: Wiley.CrossRef

26.

Skrondal, A., & Rabe-Hesketh, S. (2004). Generalized latent variable modeling multilevel, longitudinal, and structural equation models. Boca Raton: Chapman & Hall/CRC.CrossRef

27.

Bollen, K. A., & Curran, P. J. (2006). Latent curve models: A structural equation approach. Hoboken, NJ: Wiley.

28.

Bollen, K. A. (1989). Structural equations with latent variables. New York: Wiley.

29.

Hedeker, D. R., & Gibbons, R. D. (2006). Longitudinal data analysis. Hoboken, NJ: Wiley.

30.

Muller, K. E., & Stewart, P. W. (2006). Linear model theory: Univariate, multivariate, and mixed models. Hoboken, NJ: Wiley.

31.

Little, R. J., & Rubin, D. B. (2002). Statistical analysis with missing data (2nd ed.). New York, NY: Wiley.

32.

MacCallum, R. C., Kim, C., Marlarkey, W. B., & Kiecolt-Glaser, J. K. (1997). Studying multivariate change using multilevel models and latent curve models. Multivariate Behavioral Research, 32, 215–253.CrossRef

33.

Bauer, D. J. (2003). Estimating multilevel linear models as structural equation models. Journal of Educational and Behavioral Statistics, 28, 135–167.CrossRef

34.

Ahmed, S., & Schwartz, C. E. (2009, in press). Quality of life assessments and response shift. In A. Steptoe (Ed.), Handbook of behavioral medicine: Methods and applications. New York, NY: Springer Science.

35.

Sprangers, M. A., & Schwartz, C. E. (1999). Integrating response shift into health-related quality of life research: a theoretical model. Social Science & Medicine (1982), 48(11), 1507–1515.

36.

Rapkin, B. D., & Schwartz, C. E. (2004). Toward a theoretical model of quality-of-life appraisal: Implications of findings from studies of response shift. Health and Quality of Life Outcomes, 2, 14.PubMedCrossRef

37.

Wyrwich, K. W., & Tardino, V. M. (2006). Understanding global transition assessments. Quality of Life Research: An International Journal of Quality of Life Aspects of Treatment, Care and Rehabilitation, 15(6), 995–1004.

38.

Reise, S. P., & Waller, N. G. (2009). Item response theory and clinical measurement. Annual Review of Clinical Psychology, 5, 25–46.CrossRef

39.

Garcia, S. F., Cella, D., Clauser, S. B., Flynn, K. E., Lad, T., Lai, J. S., et al. (2007). Standardizing patient-reported outcomes assessment in cancer clinical trials: a patient-reported outcomes measurement information system initiative.[erratum appears in J Clin Oncol. 2008 Feb 20;26(6):1018 Note: Lad, Thomas [added]]. Journal of Clinical Oncology, 25(32), 5106–5112.PubMedCrossRef

40.

Rose, M., Bjorner, J. B., Becker, J., Fries, J. F., & Ware, J. E. (2008). Evaluation of a preliminary physical function item bank supported the expected advantages of the Patient-Reported Outcomes Measurement Information System (PROMIS). Journal of Clinical Epidemiology, 61(1), 17–33.PubMedCrossRef

41.

Thissen, D., Reeve, B. B., Bjorner, J. B., & Chang, C. H. (2007). Methodological issues for building item banks and computerized adaptive scales. Quality of Life Research, 16(Suppl 1), 109–119.PubMedCrossRef

42.

Fairclough, D. L., Thijs, H., Huang, I. C., Finnern, H. W., & Wu, A. W. (2008). Handling missing quality of life data in HIV clinical trials: What is practical? Quality of Life Research: An International Journal of Quality of Life Aspects of Treatment, Care and Rehabilitation, 17(1), 61–73.

43.

Hogan, J. W., Roy, J., & Korkontzelou, C. (2004). Handling drop-out in longitudinal studies. Statistics in Medicine, 23(9), 1455–1497.PubMedCrossRef

44.

Michiels, B., Molenberghs, G., Bijnens, L., Vangeneugden, T., & Thijs, H. (2002). Selection models and pattern-mixture models to analyse longitudinal quality of life data subject to drop-out. Statistics in Medicine, 21(8), 1023–1041.PubMedCrossRef

45.

Oort, F. J. (2005). Using structural equation modeling to detect response shifts and true change. Quality of Life Research, 14(3), 587–598.PubMedCrossRef

46.

Edelen, M. O., Reeve, B. B., Edelen, M. O., & Reeve, B. B. (2007). Applying item response theory (IRT) modeling to questionnaire development, evaluation, and refinement. Quality of Life Research, 16(Suppl 1), 5–18.PubMedCrossRef

47.

Edwards, M. C., & Edelen, M. O. (2009). Special topics in item response theory. In R. E. Millsap & A. Maydeu-Olivares (Eds.), Handbook of quantitative methods in psychology (pp. 178–198). New York, NY: Sage.

48.

Tisak, J., & Meredith, W. (1989). Exploratory longitudinal factor analysis in multiple populations. Psychometrika, 54, 261–281.CrossRef

49.

Hill, C. D. (2006). Two models for longitudinal item response data. Unpublished doctoral dissertation, University of North Carolina—Chapel Hill.

50.

te Marvelde, J., Glas, C. A. W., Landeghem, G., & van Damme, J. (2006). Application of multidimensional item response theory models to longitudinal data. Educational and Psychological Measurement, 66(1), 5–34.CrossRef

51.

Cai, L. (2010). A Two-Tier Full-Information Item Factor Analysis Model with Applications. Psychometrika, 75(4), 581–612.CrossRef

52.

Schilling, S., & Bock, R. D. (2005). High-dimensional maximum marginal likelihood item factor analysis by adaptive quadrature. Psychometrika, 70, 533–555.

53.

Béguin, A. A., & Glas, C. A. W. (2001). MCMC estimation and some model-fit analysis of multidimensional IRT models. Psychometrika, 66, 541–561.CrossRef

54.

Cai, L. (2010). High-dimensional Exploratory Item Factor Analysis by a Metropolis-Hastings Robbins-Monro Algorithm. Psychometrika, 75(1), 33–57.CrossRef

55.

Cai, L. (2010). Metropolis-Hastings Robbins-Monro Algorithm for Confirmatory Item Factor Analysis. Journal of Educational and Behavioral Statistics, 35, 307–335.CrossRef

56.

Harding, G., Leidy, N. K., Meddis, D., Kleinman, L., Wagner, S., & O’Brien, C. (2009). Interpreting clinical trial results of patient-perceived onset of effect in asthma: Methods and results of a Delphi panel. Current Medical Research and Opinion, 25(6), 1563–1571.PubMedCrossRef

57.

Turner, D., Schünemann, H. J., Griffith, L. E., Beaton, D. E., Griffiths, A. M., Critch, J. N., et al. (2009). Using the entire cohort in the receiver operating characteristic analysis maximizes precision of the minimal important difference. Journal of Clinical Epidemiology, 62(4), 374–379.PubMedCrossRef

58.

McLeod, L. D., Nelson, L., & Lewis, C. (July 1, 2009). Personal Communication.

59.

Li, Y., & Rapkin, B. (2009). Classification and regression tree uncovered hierarchy of psychosocial determinants underlying quality-of-life response shift in HIV/AIDS. Journal of Clinical Epidemiology, 62(11), 1138–1147.PubMedCrossRef

60.

Schwartz, C. E., & Rapkin, B. D. (2004). Reconsidering the psychometrics of quality of life assessment in light of response shift and appraisal. Health and Quality of Life Outcomes, 2, 16.PubMedCrossRef

61.

Stone, A. A., Turkkan, J. S., Bachrach, C. A., Jobe, J. B., Kurtzman, H. S., & Cain, V. S. (Eds.). (2000). The science of self-report: Implications for research and practice. Mahwah, NJ: Lawrence Erlbaum.

62.

Bloem, E. F., van Zuuren, F. J., Koeneman, M. A., Rapkin, B. D., Visser, M. R. M., Koning, C. C. E., et al. (2008). Clarifying quality of life assessment: Do theoretical models capture the underlying cognitive processes? Quality of Life Research: An International Journal of Quality of Life Aspects of Treatment, Care and Rehabilitation, 17(8), 1093–1102.

63.

Rijmen, F., Tuerlinckx, F., De Boeck, P., & Kuppens, P. (2003). A nonlinear mixed model framework for item response theory. Psychological Methods, 8(2), 185–205.PubMedCrossRef

Title: The king’s foot of patient-reported outcomes: current practices and new developments for the measurement of change
Authors: Richard J. Swartz
Carolyn Schwartz
Ethan Basch
Li Cai
Diane L. Fairclough
Lori McLeod
Tito R. Mendoza
Bruce Rapkin
The SAMSI Psychometric Program Longitudinal Assessment of Patient-Reported Outcomes Working Group
Publication date: 01-10-2011
Publisher: Springer Netherlands
Published in: Quality of Life Research / Issue 8/2011
Print ISSN: 0962-9343
Electronic ISSN: 1573-2649
DOI: https://doi.org/10.1007/s11136-011-9863-1

Keynote webinar | Spotlight on medication adherence

Springer Medicine

The king’s foot of patient-reported outcomes: current practices and new developments for the measurement of change

Abstract

Purpose

Methods/Results

Conclusions

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Abstract

Purpose

Methods/Results

Conclusions

Please log in to get access to this content

Other articles of this Issue 8/2011

Erratum to: Health status and QOL instruments used in childhood cancer research: deciphering conceptual content using World Health Organization definitions

Concordance in the measurement of quality of life and health indicators between two methods of computer-assisted interviews: self-administered and by telephone

Children and adolescents’ health-related quality of life in relation to eczema, asthma and hay fever: results from a population-based cross-sectional study

Maternal, paternal, and family health-related quality of life in the context of pediatric inflammatory bowel disease

Clinical correlates of health-related quality of life among opioid-dependent patients

Quality of life among parents of children with cancer or brain tumors: the impact of child characteristics and parental psychosocial factors