Skip to main content
Top
Published in: Quality of Life Research 12/2020

01-12-2020 | Scleroderma

The importance of development standards for anchoring vignettes: an illustrative example from pediatric localized scleroderma quality of life

Authors: Christina K. Zigler, Heidi Jacobe, Kaveh Ardalan, Theresa M. Coles, Suzane Lane, Kaila L. Schollaert, Kathryn S. Torok

Published in: Quality of Life Research | Issue 12/2020

Login to get access

Abstract

Purpose

Anchoring vignettes (AVs) are a promising measurement technique to reduce bias in patient-reported outcome (PRO) measures by helping researchers understand differences in how individuals and groups interpret response options. However, little attention has been paid to ensure quality development of AVs, and their performance has not been well assessed in pediatric populations. In this study, we explore the application of a rigorous development process for AVs based upon current standards for PROs, as well as feasibility of AVs when administered to children and adolescents.

Methods

We developed AVs using a rigorous, patient-centered mixed methods process including three phases: (1) development, (2) a pilot study, and (3) a field test. Our proposed process included the generation of a conceptual framework based on the PRO, the Localized Scleroderma Quality of Life Instrument, and numerous vignette-specific considerations. We qualitatively explored readability and comprehension of the AVs (pilot study) and then analyzed ranking patterns within vignette sets (field test).

Results

Four sets of four vignettes were developed. Revisions were suggested at each phase of development. The pilot study demonstrated that children ≥ 10 years had no trouble indicating understanding of the AVs. In the field test, although appropriate rankings of vignettes were generally demonstrated by participants, the percentage of tied rankings was higher than expected in this pediatric group.

Conclusions

This work supports the need for rigorous developmental standards for AVs, as each stage of development suggested revisions. Additionally, AVs showed initial promise for use with pediatric populations; general feasibility and understanding were supported.
Literature
1.
go back to reference U. S. Department of Health and Human Services, F. a. D. A., Center for Drug Evaluation and Research, Center for Biologics Evaluation and Research, and Center for Devices and Radiological Health. (2006). Guidance for industry: Patient-reported outcome measures: use in medical product development to support labeling claims: Draft guidance. Health Qual Life Outcomes, 4, 79. https://doi.org/10.1186/1477-7525-4-79.CrossRef U. S. Department of Health and Human Services, F. a. D. A., Center for Drug Evaluation and Research, Center for Biologics Evaluation and Research, and Center for Devices and Radiological Health. (2006). Guidance for industry: Patient-reported outcome measures: use in medical product development to support labeling claims: Draft guidance. Health Qual Life Outcomes, 4, 79. https://​doi.​org/​10.​1186/​1477-7525-4-79.CrossRef
2.
go back to reference Reeve, B. B., Wyrwich, K. W., Wu, A. W., Velikova, G., Terwee, C. B., Snyder, C. F., et al. (2013). ISOQOL recommends minimum standards for patient-reported outcome measures used in patient-centered outcomes and comparative effectiveness research. Quality of Life Research, 22(8), 1889–1905. https://doi.org/10.1007/s11136-012-0344-y.CrossRefPubMed Reeve, B. B., Wyrwich, K. W., Wu, A. W., Velikova, G., Terwee, C. B., Snyder, C. F., et al. (2013). ISOQOL recommends minimum standards for patient-reported outcome measures used in patient-centered outcomes and comparative effectiveness research. Quality of Life Research, 22(8), 1889–1905. https://​doi.​org/​10.​1007/​s11136-012-0344-y.CrossRefPubMed
3.
go back to reference Eignor D.R. (2013). The standards for educational and psychological testing: American psychological association. Eignor D.R. (2013). The standards for educational and psychological testing: American psychological association.
4.
go back to reference King, G., Murray, C. J., Salomon, J. A., & Tandon, A. (2004). Enhancing the validity and cross-cultural comparability of measurement in survey research. American Political Science Review, 98(1), 191–207.CrossRef King, G., Murray, C. J., Salomon, J. A., & Tandon, A. (2004). Enhancing the validity and cross-cultural comparability of measurement in survey research. American Political Science Review, 98(1), 191–207.CrossRef
6.
go back to reference King, G., & Wand, J. (2007). Comparing incomparable survey responses: Evaluating and selecting anchoring vignettes. Political Analysis, 15(1), 46–66.CrossRef King, G., & Wand, J. (2007). Comparing incomparable survey responses: Evaluating and selecting anchoring vignettes. Political Analysis, 15(1), 46–66.CrossRef
7.
go back to reference Schwartz, C. E. (2010). Applications of response shift theory and methods to participation measurement: A brief history of a young field. Archives of Physical Medicine Rehabilitation, 91(9), S38–S43.CrossRef Schwartz, C. E. (2010). Applications of response shift theory and methods to participation measurement: A brief history of a young field. Archives of Physical Medicine Rehabilitation, 91(9), S38–S43.CrossRef
8.
go back to reference Bolt, D. M., Lu, Y., & Kim, J.-S. (2014). Measurement and control of response styles using anchoring vignettes: A model-based approach. Journal Psychological Methods, 19(4), 528.CrossRef Bolt, D. M., Lu, Y., & Kim, J.-S. (2014). Measurement and control of response styles using anchoring vignettes: A model-based approach. Journal Psychological Methods, 19(4), 528.CrossRef
9.
go back to reference Primi R., Zanon C., Santos D., De Fruyt F., John O.P. (2016). Anchoring vignettes. European Journal of Psychological Assessment. Primi R., Zanon C., Santos D., De Fruyt F., John O.P. (2016). Anchoring vignettes. European Journal of Psychological Assessment.
10.
go back to reference Vonkova, H., Bendl, S., & Papajoanu, O. (2017). How students report dishonest behavior in school: Self-assessment and anchoring vignettes. The Journal of Experimental Education, 85(1), 36–53.CrossRef Vonkova, H., Bendl, S., & Papajoanu, O. (2017). How students report dishonest behavior in school: Self-assessment and anchoring vignettes. The Journal of Experimental Education, 85(1), 36–53.CrossRef
11.
go back to reference Vonkova H., Zamarro G., Deberg V., Hitt C. (2015). Comparisons of student perceptions of teacher’s performance in the classroom: Using parametric anchoring vignette methods for improving comparability. EDRE Working Paper No. 2015–01 Vonkova H., Zamarro G., Deberg V., Hitt C. (2015). Comparisons of student perceptions of teacher’s performance in the classroom: Using parametric anchoring vignette methods for improving comparability. EDRE Working Paper No. 2015–01
12.
go back to reference Marksteiner T., Kuger S., Klieme E. (2018). The potential of anchoring vignettes to increase intercultural comparability of non-cognitive factors. Assessment in Education: Principles, Policy & Practice, 1–21. Marksteiner T., Kuger S., Klieme E. (2018). The potential of anchoring vignettes to increase intercultural comparability of non-cognitive factors. Assessment in Education: Principles, Policy & Practice, 1–21.
13.
go back to reference Vonkova, H., & Hrabak, J. (2015). The (in) comparability of ICT knowledge and skill self-assessments among upper secondary school students: The use of the anchoring vignette method. Computers Education, 85, 191–202.CrossRef Vonkova, H., & Hrabak, J. (2015). The (in) comparability of ICT knowledge and skill self-assessments among upper secondary school students: The use of the anchoring vignette method. Computers Education, 85, 191–202.CrossRef
15.
go back to reference Primi R., Santos D., John O.P., De Fruyt F., Hauck-Filho N. (2017). Dealing with person differential item functioning in social-emotional skill assessment using anchoring vignettes. In The Annual Meeting of the Psychometric Society, (pp. 275–286): Springer Primi R., Santos D., John O.P., De Fruyt F., Hauck-Filho N. (2017). Dealing with person differential item functioning in social-emotional skill assessment using anchoring vignettes. In The Annual Meeting of the Psychometric Society, (pp. 275–286): Springer
16.
go back to reference Grol-Prokopczyk, H., Verdes-Tennant, E., McEniry, M., & Ispány, M. (2015). Promises and pitfalls of anchoring vignettes in health survey research. Demography, 52(5), 1703–1728.CrossRef Grol-Prokopczyk, H., Verdes-Tennant, E., McEniry, M., & Ispány, M. (2015). Promises and pitfalls of anchoring vignettes in health survey research. Demography, 52(5), 1703–1728.CrossRef
21.
22.
go back to reference Hopkins, D. J., & King, G. (2010). Improving anchoring vignettes: Designing surveys to correct interpersonal incomparability. Public Opinion Quarterly, 74(2), 201–222.CrossRef Hopkins, D. J., & King, G. (2010). Improving anchoring vignettes: Designing surveys to correct interpersonal incomparability. Public Opinion Quarterly, 74(2), 201–222.CrossRef
23.
go back to reference Collins, D. (2003). Pretesting survey instruments: An overview of cognitive methods. Quality of Life Research, 12(3), 229–238.CrossRef Collins, D. (2003). Pretesting survey instruments: An overview of cognitive methods. Quality of Life Research, 12(3), 229–238.CrossRef
24.
go back to reference Wand, J. (2013). Credible comparisons using interpersonally incomparable data: Nonparametric scales with anchoring vignettes. American Journal of Political Science, 57(1), 249–262.CrossRef Wand, J. (2013). Credible comparisons using interpersonally incomparable data: Nonparametric scales with anchoring vignettes. American Journal of Political Science, 57(1), 249–262.CrossRef
25.
go back to reference Wand J., King G., Lau O. (2011). Anchors: Software for anchoring vignette data. Journal of Statistical Software. Wand J., King G., Lau O. (2011). Anchors: Software for anchoring vignette data. Journal of Statistical Software.
27.
go back to reference Murray, K. J., & Laxer, R. M. (2002). Scleroderma in children and adolescents. Rheumatic Diseases Clinics of North America, 28(3), 603–624.CrossRef Murray, K. J., & Laxer, R. M. (2002). Scleroderma in children and adolescents. Rheumatic Diseases Clinics of North America, 28(3), 603–624.CrossRef
Metadata
Title
The importance of development standards for anchoring vignettes: an illustrative example from pediatric localized scleroderma quality of life
Authors
Christina K. Zigler
Heidi Jacobe
Kaveh Ardalan
Theresa M. Coles
Suzane Lane
Kaila L. Schollaert
Kathryn S. Torok
Publication date
01-12-2020
Publisher
Springer International Publishing
Published in
Quality of Life Research / Issue 12/2020
Print ISSN: 0962-9343
Electronic ISSN: 1573-2649
DOI
https://doi.org/10.1007/s11136-020-02575-6

Other articles of this Issue 12/2020

Quality of Life Research 12/2020 Go to the issue