Skip to main content
Top
Published in: Health and Quality of Life Outcomes 1/2018

Open Access 01-12-2018 | Research

Testing the construct validity of a health transition question using vignette-guided patient ratings of health

Authors: Michael M. Ward, Jinxiang Hu, Lori C. Guthrie, Maria Alba

Published in: Health and Quality of Life Outcomes | Issue 1/2018

Login to get access

Abstract

Background

A single-item transition question is often used to assess improvement or worsening in health, but its validity has not been tested extensively. The purpose of this study was to test the construct validity of a transition question by relating it to qualitative changes in patient’s self-rating of health guided by clinical vignettes.

Methods

We studied 169 patients with active rheumatoid arthritis (RA) before and after treatment escalation. At both assessments, patients scored their current health on a rating scale after first rating three vignettes describing mild, moderate, or severe RA. We classified patients into one of these three RA categories using a nearest-neighbor match. We then related the change in these self-rated categories between visits to responses to a transition question on visit 2.

Results

Sixty patients improved their RA vignette category after treatment, 86 remained in the same vignette category, and 23 worsened categories. On the transition question, 101 patients reported improvement, 48 reported no change, and 20 reported worsening, representing a modest association with changes in RA vignette categories (polychoric correlation r = 0.19). The association was stronger if patients who were in the mild RA category at both visits were also classified as improved if their self-rating changed from below to above their mild vignette rating (r = 0.23) and when incorporating the importance of changes on the transition question (r = 0.26).

Conclusion

Changes in health states, guided by clinical vignettes, support the construct validity of the transition question.
Literature
1.
go back to reference Guyatt GH, Norman GR, Juniper EF, Griffith LE. A critical look at transition ratings. J Clin Epidemiol. 2002;55:900–8.CrossRefPubMed Guyatt GH, Norman GR, Juniper EF, Griffith LE. A critical look at transition ratings. J Clin Epidemiol. 2002;55:900–8.CrossRefPubMed
2.
go back to reference Guyatt GH, Osoba D, Wu AW, Wyrwich KW, Norman GR, Clinical Significance Consensus Meeting Group. Methods to explain the clinical significance of health status measures. Mayo Clin Proc. 2002;77:371–83.CrossRefPubMed Guyatt GH, Osoba D, Wu AW, Wyrwich KW, Norman GR, Clinical Significance Consensus Meeting Group. Methods to explain the clinical significance of health status measures. Mayo Clin Proc. 2002;77:371–83.CrossRefPubMed
3.
go back to reference Wyrich KW, Bullinger M, Aaronson N, Hays RD, Patrick DL, Symonds T, et al. Estimating clinically significant differences in quality of life outcomes. Qual Life Res. 2005;14:285–95.CrossRef Wyrich KW, Bullinger M, Aaronson N, Hays RD, Patrick DL, Symonds T, et al. Estimating clinically significant differences in quality of life outcomes. Qual Life Res. 2005;14:285–95.CrossRef
4.
go back to reference Rose AJ, Sacks NC, Deshpande AP, Griffin SY, Cabral HJ, Kazis LE. Single-change items did not measure change in quality of life. J Clin Epidemiol. 2008;61:603–8.CrossRefPubMed Rose AJ, Sacks NC, Deshpande AP, Griffin SY, Cabral HJ, Kazis LE. Single-change items did not measure change in quality of life. J Clin Epidemiol. 2008;61:603–8.CrossRefPubMed
5.
go back to reference Wyrwich KW, Tardino VM. Understanding global transition assessments. Qual Life Res. 2006;15:995–1004.CrossRefPubMed Wyrwich KW, Tardino VM. Understanding global transition assessments. Qual Life Res. 2006;15:995–1004.CrossRefPubMed
6.
go back to reference Sprangers MAG, Schwartz CE. Integrating response shift into health-related quality of life research: a theoretical model. Soc Sci Med. 1999;48:1507–15.CrossRefPubMed Sprangers MAG, Schwartz CE. Integrating response shift into health-related quality of life research: a theoretical model. Soc Sci Med. 1999;48:1507–15.CrossRefPubMed
7.
go back to reference Schmitt J, DiFabio RP. The validity of prospective and retrospective global change criterion measures. Arch Phys Med Rehabil. 2005;86:2270–6.CrossRefPubMed Schmitt J, DiFabio RP. The validity of prospective and retrospective global change criterion measures. Arch Phys Med Rehabil. 2005;86:2270–6.CrossRefPubMed
8.
go back to reference Metz SM, Wyrwich KW, Babu AN, Kroenke K, Tierney WM, Wolinsky FD. Validity of patient-reported health-related quality of life global ratings of change using structural equation modeling. Qual Life Res. 2007;16:1193–202.CrossRefPubMed Metz SM, Wyrwich KW, Babu AN, Kroenke K, Tierney WM, Wolinsky FD. Validity of patient-reported health-related quality of life global ratings of change using structural equation modeling. Qual Life Res. 2007;16:1193–202.CrossRefPubMed
9.
go back to reference Kamper SJ, Ostelo RWJG, Knol KL, Maher CG, de Vet HCW, Hancock MJ. Global perceived effect scales provided reliable assessments of health transition in people with musculoskeletal disorders, but ratings are strongly influenced by current status. J Clin Epidemiol. 2010;63:760–6.CrossRefPubMed Kamper SJ, Ostelo RWJG, Knol KL, Maher CG, de Vet HCW, Hancock MJ. Global perceived effect scales provided reliable assessments of health transition in people with musculoskeletal disorders, but ratings are strongly influenced by current status. J Clin Epidemiol. 2010;63:760–6.CrossRefPubMed
10.
go back to reference Grøvle L, Haugen AJ, Hasvik E, Natvig B, Brox JI, Grotle M. Patients’ ratings of global perceived change during 2 years were strongly influenced by the current health status. J Clin Epidemiol. 2014;67:508–15.CrossRefPubMed Grøvle L, Haugen AJ, Hasvik E, Natvig B, Brox JI, Grotle M. Patients’ ratings of global perceived change during 2 years were strongly influenced by the current health status. J Clin Epidemiol. 2014;67:508–15.CrossRefPubMed
11.
go back to reference Norman GR, Stratford P, Regehr G. Methodological problems in the retrospective computation of responsiveness to change: the lesson of Cronbach. J Clin Epidemiol. 1997;50:869–79.CrossRefPubMed Norman GR, Stratford P, Regehr G. Methodological problems in the retrospective computation of responsiveness to change: the lesson of Cronbach. J Clin Epidemiol. 1997;50:869–79.CrossRefPubMed
13.
go back to reference Lattig F, Grob D, Kleinstueck FD, Porchet F, Jeszenszky D, Bartanusz V, et al. Ratings of global outcome at the first post-operative assessment after spinal surgery: how often do the surgeon and patient agree? Eur Spine J. 2009;18:386–94.CrossRefPubMedPubMedCentral Lattig F, Grob D, Kleinstueck FD, Porchet F, Jeszenszky D, Bartanusz V, et al. Ratings of global outcome at the first post-operative assessment after spinal surgery: how often do the surgeon and patient agree? Eur Spine J. 2009;18:386–94.CrossRefPubMedPubMedCentral
14.
go back to reference Nieuwkerk PT, Tollenaar MS, Oort FJ, Spangers MAG. Are retrospective measures of change in quality of life more valid than prospective measures? Med Care. 2007;45:199–205.CrossRefPubMed Nieuwkerk PT, Tollenaar MS, Oort FJ, Spangers MAG. Are retrospective measures of change in quality of life more valid than prospective measures? Med Care. 2007;45:199–205.CrossRefPubMed
15.
go back to reference Korfage IJ, de Koning HJ, Essink-Bot ML. Response shift due to diagnosis and primary treatment of localized prostate cancer: a then-test and a vignette study. Qual Life Res. 2007;16:1627–34.CrossRefPubMedPubMedCentral Korfage IJ, de Koning HJ, Essink-Bot ML. Response shift due to diagnosis and primary treatment of localized prostate cancer: a then-test and a vignette study. Qual Life Res. 2007;16:1627–34.CrossRefPubMedPubMedCentral
16.
go back to reference Dowd JB, Todd M. Does self-reported health bias the measurement of health inequalities in U.S. adults? Evidence using anchoring vignettes from the health and retirement study. J Gerontol B Psychol Sci Soc Sci. 2011;66:478–89.CrossRefPubMed Dowd JB, Todd M. Does self-reported health bias the measurement of health inequalities in U.S. adults? Evidence using anchoring vignettes from the health and retirement study. J Gerontol B Psychol Sci Soc Sci. 2011;66:478–89.CrossRefPubMed
17.
go back to reference Grol-Prokopczyk H, Freese J, Hauser RM. Using anchoring vignettes to assess group differences in general self-rated health. J Health Soc Behav. 2011;52:246–61.CrossRefPubMedPubMedCentral Grol-Prokopczyk H, Freese J, Hauser RM. Using anchoring vignettes to assess group differences in general self-rated health. J Health Soc Behav. 2011;52:246–61.CrossRefPubMedPubMedCentral
18.
go back to reference Kapteyn A, Smith JP, Van Soest A. Vignettes and self-reports of work disability in the United States and the Netherlands. Am Econ Rev. 2007;97:461–73.CrossRef Kapteyn A, Smith JP, Van Soest A. Vignettes and self-reports of work disability in the United States and the Netherlands. Am Econ Rev. 2007;97:461–73.CrossRef
19.
go back to reference Bagozzi RP, Yi Y, Phillips LW. Assessing construct validity in organizational research. Adm Sc Q. 1991;36:421–58.CrossRef Bagozzi RP, Yi Y, Phillips LW. Assessing construct validity in organizational research. Adm Sc Q. 1991;36:421–58.CrossRef
20.
go back to reference Ward MM, Guthrie LC, Alba MI. Clinically important changes in individual and composite measures of rheumatoid arthritis activity. Thresholds applicable in clinical trials. Ann Rheum Dis. 2015;74:1691–6.CrossRefPubMed Ward MM, Guthrie LC, Alba MI. Clinically important changes in individual and composite measures of rheumatoid arthritis activity. Thresholds applicable in clinical trials. Ann Rheum Dis. 2015;74:1691–6.CrossRefPubMed
21.
go back to reference Ward MM, Guthrie LC, Alba MI. Domain-specific transition questions demonstrated higher validity than global transition questions as anchors for clinically important improvement. J Clin Epidemiol. 2015;68:655–61.CrossRefPubMedPubMedCentral Ward MM, Guthrie LC, Alba MI. Domain-specific transition questions demonstrated higher validity than global transition questions as anchors for clinically important improvement. J Clin Epidemiol. 2015;68:655–61.CrossRefPubMedPubMedCentral
22.
go back to reference Hopkins DJ, King G. Improving anchoring vignettes: designing surveys to correct interpersonal incomparability. Public Opin Q. 2010;74:201–22.CrossRef Hopkins DJ, King G. Improving anchoring vignettes: designing surveys to correct interpersonal incomparability. Public Opin Q. 2010;74:201–22.CrossRef
23.
go back to reference Bakker C, Rutten M, van Doorslaer E, Bennett K, van der Linden S. Feasibility of utility assessment by rating scale and standard gamble in patients with ankylosing spondylitis or fibromyalgia. J Rheumatol. 1994;21:269–74.PubMed Bakker C, Rutten M, van Doorslaer E, Bennett K, van der Linden S. Feasibility of utility assessment by rating scale and standard gamble in patients with ankylosing spondylitis or fibromyalgia. J Rheumatol. 1994;21:269–74.PubMed
24.
go back to reference Jaeschke R, Singer J, Guyatt GH. Measurement of health status. Ascertaining the minimal clinically important difference. Control Clin Trials. 1989;10:407–15.CrossRefPubMed Jaeschke R, Singer J, Guyatt GH. Measurement of health status. Ascertaining the minimal clinically important difference. Control Clin Trials. 1989;10:407–15.CrossRefPubMed
25.
go back to reference Cover T, Hart P. Nearest neighbor pattern classification. IEEE Trans Inf Theory. 1967;13:21–7.CrossRef Cover T, Hart P. Nearest neighbor pattern classification. IEEE Trans Inf Theory. 1967;13:21–7.CrossRef
26.
go back to reference Cohen J. Statistical power analysis for the behavioral sciences. 2nd ed. Hillsdale: Erlbaum; 1988. Cohen J. Statistical power analysis for the behavioral sciences. 2nd ed. Hillsdale: Erlbaum; 1988.
27.
go back to reference Rice ME, Harris GT. Comparing effect sizes in follow-up studies: ROC area, Cohen's d, and r. Law Hum Behav. 2005;29:615–20.CrossRefPubMed Rice ME, Harris GT. Comparing effect sizes in follow-up studies: ROC area, Cohen's d, and r. Law Hum Behav. 2005;29:615–20.CrossRefPubMed
28.
go back to reference Chang E, Abrahamowicz M, Ferland D, Fortin PR. Comparison of the responsiveness of lupus disease activity measures to changes in systemic lupus erythematosus activity relevant to patients and physicians. J Clin Epidemiol. 2002;55:488–97.CrossRefPubMed Chang E, Abrahamowicz M, Ferland D, Fortin PR. Comparison of the responsiveness of lupus disease activity measures to changes in systemic lupus erythematosus activity relevant to patients and physicians. J Clin Epidemiol. 2002;55:488–97.CrossRefPubMed
29.
go back to reference Williams NH, Wilkinson C, Russell IT. Extending the Aberdeen back pain scale to include the whole spine: a set of outcome measures for the neck, upper and lower back. Pain. 2001;94:261–74.CrossRefPubMed Williams NH, Wilkinson C, Russell IT. Extending the Aberdeen back pain scale to include the whole spine: a set of outcome measures for the neck, upper and lower back. Pain. 2001;94:261–74.CrossRefPubMed
30.
go back to reference Locker D, Jokovic A, Clarke M. Assessing the responsiveness of measures of oral health-related quality of life. Community Dent Oral Epidemiol. 2004;32:10–8.CrossRefPubMed Locker D, Jokovic A, Clarke M. Assessing the responsiveness of measures of oral health-related quality of life. Community Dent Oral Epidemiol. 2004;32:10–8.CrossRefPubMed
31.
go back to reference Peracchi F, Rossetti C. Heterogeneity in health responses and anchoring vignettes. Empir Econ. 2012;42:513–38.CrossRef Peracchi F, Rossetti C. Heterogeneity in health responses and anchoring vignettes. Empir Econ. 2012;42:513–38.CrossRef
33.
go back to reference Murray CJL, Ozaltin E, Tandon A, Salomon J, Sadana R, Chatterji S. Empirical evaluation of the anchoring vignettes approach in health surveys. In: Murray CJL, Evans DB, editors. Health systems performance assessment: debates, methods and empiricism. Geneva: World Health Organization; 2003. p. 369–99. Murray CJL, Ozaltin E, Tandon A, Salomon J, Sadana R, Chatterji S. Empirical evaluation of the anchoring vignettes approach in health surveys. In: Murray CJL, Evans DB, editors. Health systems performance assessment: debates, methods and empiricism. Geneva: World Health Organization; 2003. p. 369–99.
34.
go back to reference Gupta ND, Kristensen N, Pozzoli D. External validation of the use of vignettes in cross-country health studies. Econ Model. 2010;27:854–65.CrossRef Gupta ND, Kristensen N, Pozzoli D. External validation of the use of vignettes in cross-country health studies. Econ Model. 2010;27:854–65.CrossRef
35.
go back to reference d’Uva TB, Lindeboom M, O’Donnell O, Van Doorslaer E. Slipping anchor? Testing the vignettes approach to identification and correction of reporting heterogeneity. J Hum Resour. 2011;46:875–906.CrossRefPubMedPubMedCentral d’Uva TB, Lindeboom M, O’Donnell O, Van Doorslaer E. Slipping anchor? Testing the vignettes approach to identification and correction of reporting heterogeneity. J Hum Resour. 2011;46:875–906.CrossRefPubMedPubMedCentral
36.
go back to reference Au N, Lorgelly PK. Anchoring vignettes for health comparisons: an analysis of response consistency. Qual Life Res. 2014;23:1721–31.CrossRefPubMed Au N, Lorgelly PK. Anchoring vignettes for health comparisons: an analysis of response consistency. Qual Life Res. 2014;23:1721–31.CrossRefPubMed
Metadata
Title
Testing the construct validity of a health transition question using vignette-guided patient ratings of health
Authors
Michael M. Ward
Jinxiang Hu
Lori C. Guthrie
Maria Alba
Publication date
01-12-2018
Publisher
BioMed Central
Published in
Health and Quality of Life Outcomes / Issue 1/2018
Electronic ISSN: 1477-7525
DOI
https://doi.org/10.1186/s12955-017-0832-4

Other articles of this Issue 1/2018

Health and Quality of Life Outcomes 1/2018 Go to the issue