Abstract
This chapter focuses on the examination of response shift in patient-reported outcomes (PROs) research, with particular attention to measurement validity and response processes. Response shift occurs when changes in PROs over time are the result of changes in how people interpret and respond to PRO measurement items at different points in time. Conceptual foundations of response shift are discussed, followed by a review of statistical methods for examining response shift. The chapter concludes with a discussion of opportunities and challenges for response shift research. We specifically consider that, although response shift is inherently about response processes and measurement validation, most of the methods and applications of response shift are descriptive in nature with an eye towards detecting and controlling for response shift. There is significant opportunity for theoretical and methodological development in focusing on understanding the mechanisms (mediators, moderators and other causes) by which response shift occurs when measuring PROs.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Aaronson, N. K., Ahmedzai, S., Bergman, B., Bullinger, M., Cull, A., Duez, N. J., et al. (1993). The European Organization for Research and Treatment of Cancer QLQ-C30: A quality-of-life instrument for use in international clinical trials in oncology. Journal of the National Cancer Institute, 85, 365–376.
Ahmed, S., Mayo, N. E., Wood-Dauphinee, S., Hanley, J. A., & Cohen, S. R. (2005). The structural equation modeling technique did not show a response shift, contrary to the results of the then test and the individualized approaches. Journal of Clinical Epidemiology, 58, 1125–1133. doi:10.1016/j.jclinepi.2005.03.003.
Ahmed, S., Sawatzky, R., Levesque, J. F., Ehrmann-Feldman, D., & Schwartz, C. E. (2014). Minimal evidence of response shift in the absence of a catalyst. Quality of Life Research, 23, 2421–2430. doi:10.1007/s11136-014-0699-3.
Anastasi, A. (1986). Evolving concepts of test validation. Annual Review of Psychology, 37, 1–15.
Anota, A., Bascoul-Mollevi, C., Conroy, T., Guillemin, F., Velten, M., Jolly, D., et al. (2014). Item response theory and factor analysis as a mean to characterize occurrence of response shift in a longitudinal quality of life study in breast cancer patients. Health and Quality of Life Outcomes, 12, 32. doi:10.1186/1477-7525-12-32.
Barclay-Goddard, R., Lix, L. M., Tate, R., Weinberg, L., & Mayo, N. E. (2009). Response shift was identified over multiple occasions with a structural equation modeling framework. Journal of Clinical Epidemiology, 62, 1181–1188. doi:10.1016/j.jclinepi.2009.03.014.
Beaumont, J. L., Lix, L. M., Yost, K. J., & Hahn, E. A. (2006). Application of robust statistical methods for sensitivity analysis of health-related quality of life outcomes. Quality of Life Research, 15, 349–356. doi:10.1007/s11136-005-2293-1.
Beeken, R., Eiser, C., & Sheeran, P. (2008). Response shift and Quality of Life (QOL): A priming study. Psychology & Health, 23, 63. doi:10.1007/s11136-010-9737-y.
Bernhard, J., Lowy, A., Maibach, R., & Hurny, C. (2001). Response shift in the perception of health for utility evaluation. An explorative investigation. European Journal of Cancer, 37, 1729–1735.
Boucekine, M., Loundou, A., Baumstarck, K., Minaya-Flores, P., Pelletier, J., Ghattas, B., & Auquier, P. (2013). Using the random forest method to detect a response shift in the quality of life of multiple sclerosis patients: A cohort study. BMC Medical Research Methodology, 13, 20. doi:10.1186/1471-2288-13-20.
Bray, J. H., Maxwell, S. E., & Howard, G. S. (1984). Methods of analysis with response-shift bias. Educational and Psychological Measurement, 44, 781–804.
Breiman, L. (2001). Random forests. Machine Learning, 45, 5–32.
Carroll, R. J., Ruppert, D., Stefanski, L. A., & Crainiceanu, C. M. (2006). Measurement error in nonlinear models: A modern perspective (2nd ed.). Boca Raton, FL: Chapman & Hall.
Cohen, S. R., Mount, B. M., Tomas, J. J., & Mount, L. F. (1996). Existential well-being is an important determinant of quality of life: Evidence from the McGill Quality of Life Questionnaire. Cancer, 77, 576–586.
Cronbach, L. J. (1988). Five perspectives on validity argument. In H. Wainer & H. I. Braun (Eds.), Validity (pp. 3–17). Hillsdale, NJ: Lawrence Erlbaum.
Cronbach, L. J., & Meehl, P. E. (1955). Construct validity in psychological tests. Psychological Bulletin, 52, 281–302. doi:10.1037/h0040957.
Department of Health. (2009). Guidance on the routine collection of Patient Reported Outcomes Measures (PROMs). Retrieved from http://www.dh.gov.uk/en/Publicationsandstatistics/Publications/PublicationsPolicyAndGuidance/DH_092647.
Fayers, P., & Machin, D. (2007). Quality of life: The assessment, analysis and interpretation of patient-reported outcomes. Chichester, UK: John Wiley & Sons.
Guilleux, A., Blanchin, M., Vanier, A., Guillemin, F., Falissard, B., Schwartz, C. E., et al. (2015). RespOnse Shift ALgorithm in Item response theory (ROSALI) for response shift detection with missing data in longitudinal patient-reported outcome studies. Quality of Life Research, 24, 553–564. doi:10.1007/s11136-014-0876-4.
Hickey, A. M., Bury, G., O’Boyle, C. A., Bradley, F., O’Kelly, F. D., & Shannon, W. (1996). A new short form individual quality of life measure (SEIQoL-DW): Application in a cohort of individuals with HIV/AIDS. British Medical Journal, 313, 29–33.
Huberty, C. J., & Wisenbaker, J. M. (1992). Variable importance in multivariate group comparisons. Journal of Educational and Behavioral Statistics, 17, 75–91.
Hubley, A. M., & Zumbo, B. D. (1996). A dialectic on validity: Where we have been and where we are going. Journal of General Psychology, 123, 207–215.
Hubley, A. M., & Zumbo, B. D. (2011). Validity and the consequences of test interpretation and use. Social Indicators Research, 103, 219–230. doi:10.1007/s11205-011-9843-4.
Hubley, A. M., & Zumbo, B. D. (2013). Psychometric characteristics of assessment procedures: An overview. In K. F. Geisinger, B. A. Bracken, J. F. Carlson, J. I. C. Hansen, N. R. Kuncel, S. P. Reise, & M. C. Rodriguez (Eds.), APA handbook of testing and assessment in psychology, Vol. 1: Test theory and testing and assessment in industrial and organizational psychology (pp. 3–19). Washington, DC: American Psychological Association.
Jobe, J. B. (2003). Cognitive psychology and self-reports: Models and methods. Quality of Life Research, 12, 219–227.
Kane, M. T. (2013). Validating the interpretations and uses of test scores. Journal of EducationalMeasurement, 50, 1–73. doi:10.1111/jedm.12000.
Kim, S., & Camilli, G. (2014). An item response theory approach to longitudinal analysis with application to summer setback in preschool language/literacy. Large-scale Assessments in Education, 2, 1–17. doi:10.1186/2196-0739-2-1.
King-Kallimanis, B. L., Oort, F. J., & Garst, G. J. A. (2010). Using structural equation modelling to detect measurement bias and response shift in longitudinal data. AStA Advances in Statistical Analysis, 94, 139–156. doi:10.1007/s10182-010-0129-y.
King-Kallimanis, B. L., Oort, F. J., Nolte, S., Schwartz, C. E., & Sprangers, M. A. (2011). Using structural equation modeling to detect response shift in performance and health-related quality of life scores of multiple sclerosis patients. Quality of Life Research, 20, 1527–1540. doi:10.1007/s11136-010-9844-9.
King-Kallimanis, B. L., Oort, F. J., Visser, M. R., & Sprangers, M. A. (2009). Structural equation modeling of health-related quality-of-life data illustrates the measurement and conceptual perspectives on response shift. Journal of Clinical Epidemiology, 62, 1157–1164. doi:10.1016/j.jclinepi.2009.04.004.
Li, Y., & Rapkin, B. (2009). Classification and regression tree uncovered hierarchy of psychosocial determinants underlying quality-of-life response shift in HIV/AIDS. Journal of Clinical Epidemiology, 62, 1138–1147. doi:10.1016/j.jclinepi.2009.03.021.
Lix, L. M., Chan, E. K. H., Sawatzky, R., Sajobi, T. T., Liu, J., Hopman, W., & Mayo, N. (2015). Response shift and disease activity in inflammatory bowel disease. Quality of Life Research, 25, 1751–1760. doi:10.1007/s11136-015-1188-z.
Lix, L. M., Sajobi, T. T., Sawatzky, R., Liu, J., Mayo, N. E., Huang, Y., et al. (2013). Relative importance measures for reprioritization response shift. Quality of Life Research, 22, 695–703. doi:10.1007/s11136-012-0198-3.
Lowy, A., & Bernhard, J. (2004). Quantitative assessment of changes in patients’ constructs of quality of life: An application of multilevel models. Quality of Life Research, 13, 1177–1185.
Martin, F., Camfield, L., Rodham, K., Kliempt, P., & Ruta, D. (2007). Twelve years’ experience with the Patient Generated Index (PGI) of quality of life: A graded structured review. Quality of Life Research, 16, 705–715. doi:10.1007/s11136-006-9152-6.
Mayo, N. E., Scott, S. C., Dendukuri, N., Ahmed, S., & Wood-Dauphinee, S. (2008). Identifying response shift statistically at the individual level. Quality of Life Research, 17, 627–639. doi:10.1007/s11136-008-9329-2.
McClimans, L., Bickenbach, J., Westerman, M., Carlson, L., Wasserman, D., & Schwartz, C. (2013). Philosophical perspectives on response shift. Quality of Life Research, 22, 1871–1878. doi:10.1007/s11136-012-0300-x.
Meade, A. W., Lautenschlager, G. J., & Hecht, J. E. (2005). Establishing measurement equivalence and invariance in longitudinal data with item response theory. International Journal of Testing, 5, 279–300. doi:10.1207/s15327574ijt0503_6.
Messick, S. (1989). Validity. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp. 13–103). New York, NY: Macmillan Publishing Co Inc..
Messick, S. (1995). Validity of psychological assessment: Validation of inferences from persons’ responses and performances as scientific inquiry into score meaning. American Psychologist, 50, 741–749.
Messick, S. (1998). Test validity: A matter of consequence. Social Indicators Research, 45, 35–44.
Millsap, R. E. (2010). Testing measurement invariance using item response theory in longitudinal data: An introduction. Child Development Perspectives, 4, 5–9. doi:10.1111/j.1750-8606.2009.00109.x.
Neuman, H. B., Park, J., Fuzesi, S., & Temple, L. K. (2012). Rectal cancer patients’ quality of life with a temporary stoma: Shifting perspectives. Diseases of the Colon and Rectum, 55, 1117–1124. doi:10.1097/DCR.0b013e3182686213.
Nolte, S., Elsworth, G. R., Sinclair, A. J., & Osborne, R. H. (2009). Tests of measurement invariance failed to support the application of the “then-test”. Journal of Clinical Epidemiology, 62, 1173–1180. doi:10.1016/j.jclinepi.2009.01.021.
Oort, F. J. (2005). Using structural equation modeling to detect response shifts and true change. Quality of Life Research, 14, 587–598.
Oort, F. J., Visser, M. R., & Sprangers, M. A. (2005). An application of structural equation modeling to detect response shifts and true change in quality of life data from cancer patients undergoing invasive surgery. Quality of Life Research, 14, 599–609.
Oort, F. J., Visser, M. R., & Sprangers, M. A. (2009). Formal definitions of measurement bias and explanation bias clarify measurement and conceptual perspectives on response shift. Journal of Clinical Epidemiology, 62, 1126–1137. doi:10.1016/j.jclinepi.2009.03.013.
Rapkin, B. D., & Schwartz, C. E. (2004). Toward a theoretical model of quality-of-life appraisal: Implications of findings from studies of response shift. Health and Quality of Life Outcomes, 2, 14. doi:10.1186/1477-7525-2-14.
Rapkin, B. D., & Schwartz, C. E. (2016). Distilling the essence of appraisal: A mixed methods study of people with multiple sclerosis. Quality of Life Research, 25, 793–805.
Ruta, D. A., Garratt, A. M., Leng, M., Russell, I. T., & MacDonald, L. M. (1994). A new approach to the measurement of quality of life. The Patient-Generated Index. Medical Care, 32, 1109–1126.
Sajobi, T. T., Fiest, K. M., & Wiebe, S. (2014). Changes in quality of life after epilepsy surgery: The role of reprioritization response shift. Epilepsia, 55, 1331–1338. doi:10.1111/epi.12697.
Sajobi, T. T., Lix, L. M., Clara, I., Walker, J., Graff, L. A., Rawsthorne, P., et al. (2012). Measures of relative importance for health-related quality of life. Quality of Life Research, 21, 1–11. doi:10.1007/s11136-011-9914-7.
Sajobi, T. T., Lix, L. M., Singh, G., Lowerison, M., Engbers, J., & Mayo, N. E. (2015). Identifying reprioritization response shift in a stroke caregiver population: A comparison of missing data methods. Quality of Life Research, 24, 529–540. doi:10.1007/s11136-014-0824-3.
Samejima, F. (1997). Graded response model. In W. J. van der Linden & R. K. Hambleton (Eds.), Handbook of modern item response theory (pp. 85–100). New York, NY: Springer.
Sawatzky, R., Chan, E. K. H., Zumbo, B. D., Bingham, C. O., Gardner, W., Jutai, J.,… Lix, L. M. (in press). Challenges and opportunities in patient-reported outcomes validation. Journal of Clinical Epidemiology. doi:10.1016/j.jclinepi.2016.12.002
Sawatzky, R., Gadermann, A., Ratner, P. A., Zumbo, B. D., & Lix, L. M. (2012). Identifying individuals with inflammatory bowel disease who experienced response shift: A latent class analysis. Quality of Life Research, 21, 33.
Schmitt, N. (1982). The use of analysis of covariance structures to assess beta and gamma change. Multivariate Behavioral Research, 17, 343–358.
Schmitt, N., Pulakos, E., & Lieblein, A. (1984). A comparison of three techniques to assess group-level beta and gamma change. Applied Psychological Measurement, 8, 249–260.
Schwartz, C. E., Ahmed, S., Sawatzky, R., Sajobi, T., Mayo, N., Finkelstein, J., et al. (2013). Guidelines for secondary analysis in search of response shift. Quality of Life Research, 22, 2663–2673. doi:10.1007/s11136-013-0402-0.
Schwartz, C. E., Bode, R., Repucci, N., Becker, J., Sprangers, M. A., & Fayers, P. M. (2006). The clinical significance of adaptation to changing health: A meta-analysis of response shift. Quality of Life Research, 15, 1533–1550. doi:10.1007/s11136-006-0025-9.
Schwartz, C. E., & Rapkin, B. A. (2012). Understanding appraisal processes underlying the thentest: A mixed methods investigation. Quality of Life Research, 21, 381–388. doi:10.1007/s11136-011-0023-4.
Schwartz, C. E., Sajobi, T. T., Verdam, M. G., Sebille, V., Lix, L. M., Guilleux, A., & Sprangers, M. A. (2015). Method variation in the impact of missing data on response shift detection. Quality of Life Research, 24, 521–528. doi:10.1007/s11136-014-0746-0.
Schwartz, C. E., & Sprangers, M. A. (2009). Reflections on genes and sustainable change: Toward a trait and state conceptualization of response shift. Journal of Clinical Epidemiology, 62, 1118–1123. doi:10.1016/j.jclinepi.2009.02.008.
Schwartz, C. E., & Sprangers, M. A. (2010). Guidelines for improving the stringency of response shift research using the thentest. Quality of Life Research, 19, 455–464. doi:10.1007/s11136-010-9585-9.
Sprangers, M. A., & Schwartz, C. E. (1999). Integrating response shift into health-related quality of life research: A theoretical model. Social Science and Medicine, 48, 1507–1515.
Thomas, D. R. (1992). Interpreting discriminant functions: A data analytic approach. Multivariate Behavioral Research, 27, 335–362. doi:10.1207/s15327906mbr2703_3.
Thomas, D. R., Hughes, E., & Zumbo, B. D. (1998). On variable importance in linear regression. Social Indicators Research, 45, 253–275.
Thomas, D. R., Zhu, P., Zumbo, B. D., & Dutta, S. (2008). On measuring the relative importance of explanatory variables in a logistic regression. Journal of Modern Applied Statistical Methods, 7, 21–38.
Thomas, D. R., & Zumbo, B. D. (1996). Using a measure of variable importance to investigate the standardization of discriminant coefficients. Journal of Educational and Behavioral Statistics, 21, 110–130. doi:10.3102/10769986021002110.
Tourangeau, R., Rips, L. J., & Rasinski, K. A. (2000). The psychology of survey response. Cambridge, U.K.: Cambridge University Press.
Verdam, M. G., Oort, F. J., & Sprangers, M. A. G. (2016). Using structural equation modeling to detect response shifts and true change in discrete variables: An application to the items of the SF-36. Quality of Life Research, 25, 1361–1383. doi:10.1007/s11136-015-1195-0.
Verdam, M. G., Oort, F. J., van der Linden, Y. M., & Sprangers, M. A. (2015). Taking into account the impact of attrition on the assessment of response shift and true change: A multigroup structural equation modeling approach. Quality of Life Research, 24, 541–551. doi:10.1007/s11136-014-0829-y.
Westerman, M. J., Hak, T., Sprangers, M. A., Groen, H. J., van der Wal, G., & The, A. M. (2008). Listen to their answers! Response behaviour in the measurement of physical and role functioning. Quality of Life Research, 17, 549–558. doi:10.1007/s11136-008-9333-6.
Zumbo, B. D. (2007a). Validity: Foundational issues and statistical methodology. In C. R. Rao & S. Sinharay (Eds.), Handbook of statistics (Vol. 26: Psychometrics, pp. 45–79). Amsterdam, the Netherlands: Elsevier Science.
Zumbo, B. D. (2007b). Three generations of DIF analyses: Considering where it has been, where it is now, and where it is going. Language Assessment Quarterly, 4, 223–233.
Zumbo, B. D. (2009). Validity as contextualized and pragmatic explanation, and its implications for validation practice. In R. W. Lissitz (Ed.), The concept of validity: Revisions, new directions and applications (pp. 65–82). Charlotte, NC: Information Age Publishing.
Zumbo, B. D., & Chan, E. K. H. (2014). Reflections on validation practices in the social, behavioral, and health sciences. In B. D. Zumbo & E. K. H. Chan (Eds.), Validity and validation in social, behavioral, and health sciences (Vol. 54, pp. 321–327). New York, NY: Springer International Publishing.
Acknowledgements
Writing of this chapter was supported by a CIHR Operating grant (#342467) held by RS, TTS, LML, and BZ; a Research Manitoba operating grant held by LM; an O’Brien Institute for Public Health Catalyst award held by TTS; Canada Research Chairs program funding for a Canada Research Chair in Patient-Reported Outcomes held by RS; a Manitoba Research Chair held by LL.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this chapter
Cite this chapter
Sawatzky, R., Sajobi, T.T., Brahmbhatt, R., Chan, E.K.H., Lix, L.M., Zumbo, B.D. (2017). Longitudinal Change in Response Processes: A Response Shift Perspective. In: Zumbo, B., Hubley, A. (eds) Understanding and Investigating Response Processes in Validation Research. Social Indicators Research Series, vol 69. Springer, Cham. https://doi.org/10.1007/978-3-319-56129-5_14
Download citation
DOI: https://doi.org/10.1007/978-3-319-56129-5_14
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-56128-8
Online ISBN: 978-3-319-56129-5
eBook Packages: Social SciencesSocial Sciences (R0)