Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress 2023 EHA Congress 2023 ASCO Annual Meeting
Clinical Collections
Advances in Alzheimer’s

At a glance: The ONWARDS insulin icodec trials

A round-up of the ONWARDS phase 3 clinical trials evaluating the novel weekly insulin icodec in people with diabetes.
ADVANCED SEARCH
Log in
Top
Quality of Life Research
Published in: Quality of Life Research 6/2016

Open Access 01-06-2016 | Special Section: Response Shift Effects at Item Level (by invitation only)

Using a retrospective pretest instead of a conventional pretest is replacing biases: a qualitative study of cognitive processes underlying responses to thentest items

Authors: Elsbeth F. Taminiau-Bloem, Carolyn E. Schwartz, Florence J. van Zuuren, Margot A. Koeneman, Mechteld R. M. Visser, Carol Tishelman, Caro C. E. Koning, Mirjam A. G. Sprangers

Published in: Quality of Life Research | Issue 6/2016

Login to get access

Abstract

Background

The thentest design aims to detect and control for recalibration response shift. This design assumes (1) more consistency in the content of the cognitive processes underlying patients’ quality of life (QoL) between posttest and thentest assessments than between posttest and pretest assessments; and (2) consistency in the time frame and description of functioning referenced at pretest and thentest. Our objective is to utilize cognitive interviewing to qualitatively examine both assumptions.

Methods

We conducted think-aloud interviews with 24 patients with cancer prior to and after radiotherapy to elicit cognitive processes underlying their assessment of seven EORTC QLQ-C30 items at pretest, posttest and thentest. We used an analytic scheme based on the cognitive process models of Tourangeau et al. and Rapkin and Schwartz that yielded five cognitive processes. We subsequently used this input for quantitative analysis of count data.

Results

Contrary to expectation, the number of dissimilar cognitive processes between posttest and thentest was generally larger than between pretest and posttest across patients. Further, patients considered a range of time frames when answering the thentest questions. Moreover, patients’ description at the thentest of their pretest functioning was often not similar to that which was noted at pretest. Items referring to trouble taking a short walk, overall health and QoL were most often violating the assumptions.

Conclusions

Both assumptions underlying the thentest design appear not to be supported by the patients’ cognitive processes. Replacing the conventional pretest–posttest design with the thentest design may simply be replacing one set of biases with another.
Appendix
Available only for authorised users
Literature
1.
Taminiau-Bloem, E. F., van Zuuren, F. J., Koeneman, M. A., Rapkin, B. D., Visser, M. R. M., Koning, C. C. E., & Sprangers, M. A. G. (2010). A ‘short walk’ is longer before radiotherapy than afterwards: A qualitative study questioning the baseline and follow-up design. Health and Quality of Life Outcomes, 8, 69.CrossRefPubMedPubMedCentral
2.
Morak, M. J., Pek, C. J., Kompanje, E. J., Hop, W. C., Kazemier, G., & van Eijck, C. H. (2010). Quality of life after adjuvant intra-arterial chemotherapy and radiotherapy versus surgery alone in resectable pancreatic and periampullary cancer: A prospective randomized controlled study. Cancer, 116(4), 830–836.CrossRefPubMed
3.
Komblith, A. B., Huang, H. Q., Walker, J. L., Spirtos, N. M., Rotmensch, J., & Cella, D. (2009). Quality of life of patients with endometrial cancer undergoing laparoscopic international federation of gynecology and obstetrics staging compared with laparotomy: A Gynecologic Oncology Group study. Journal of Clinical Oncology, 27(32), 5337–5342.CrossRef
4.
Groenvold, M., Fayers, P. M., Petersen, M. A., & Mouridsen, H. T. (2006). Chemotherapy versus ovarian ablation as adjuvant therapy for breast cancer: Impact on health-related quality of life in a randomized trial. Breast Cancer Research, 98(3), 275–284.CrossRef
5.
Weeks, J. C., Nelson, H., Gelber, S., Sargent, D., Schroeder, G., & for the Clinical Outcomes of Surgical Therapy (COST) Study Group. (2003). Short-term quality-of-life outcomes following laparoscopic-assisted colectomy vs open colectomy for colon cancer. JAMA, 287(3), 321–328.CrossRef
6.
Sprangers, M. A. G., & Schwartz, C. E. (1999). Integrating response shift into health-related quality-of-life research: A theoretical model. Social Science and Medicine, 48(11), 1507–1515.CrossRefPubMed
7.
Schwartz, C. E., Feinberg, R. G., Jilinskaia, E., & Applegate, J. C. (1999). An evaluation of a psychosocial intervention for survivors of childhood cancer: Paradoxical effects of response shift over time. Psycho-Oncology, 8(4), 344–354.CrossRefPubMed
8.
Hagedoorn, M., Sneeuw, K. C. A., & Aaronson, N. K. (2002). Changes in physical functioning and quality of life in patients with cancer: Response shift and relative evaluation of one’s condition. Journal of Clinical Epidemiology, 55(2), 176–183.CrossRefPubMed
9.
Visser, M. R., Oort, F. J., & Sprangers, M. A. (2005). Methods to detect response shift in quality of life data: A convergent validity study. Quality of Life Research, 14(3), 629–639.CrossRefPubMed
10.
Rees, J., Clarke, M. G., Waldron, D., O’Boyle, C., Ewings, P., & MacDonagh, R. P. (2005). The measurement of response shift in patients with advanced prostate cancer and their partners. Health and Quality of Life Outcomes, 3, 21.CrossRefPubMedPubMedCentral
11.
Andrykowski, M. A., Donovan, K. A., & Jacobsen, P. B. (2009). Magnitude and correlates of response shift in fatigue ratings in women undergoing adjuvant therapy for breast cancer. Journal of Pain and Symptom Management, 37(3), 341–351.CrossRefPubMedPubMedCentral
12.
Razmjou, H., Yee, A., Ford, M., & Finkelstein, J. A. (2006). Response shift in outcome assessment in patients undergoing total knee arthroplasty. The Journal of Bone and Joint Surgery, 88(12), 2590–2595.CrossRefPubMed
13.
Rees, J., Waldron, D., O’Boyle, C., Ewings, P., & MacDonagh, R. (2003). Prospective vs retrospective assessment of lower urinary tract symptoms in patients with advanced prostate cancer: the effect of ‘response shift’. BJU International, 92(7), 703–706.CrossRefPubMed
14.
Jansen, S. J., Stiggelbout, A. M., Nooij, M. A., Noordijk, E. M., & Kievit, J. (2000). Response shift in quality of life measurement in early-stage breast cancer patients undergoing radiotherapy. Quality of Life Research, 9(6), 603–615.CrossRefPubMed
15.
Howard, G. S., Ralph, K. M., Gulanick, N. A., Maxwell, S. E., Nance, S. W., & Gerber, S. K. (1979). Internal invalidity in pretest-posttest self-report evaluations and a re-evaluation of retrospective pretests. Applied Psychological Measurement, 3(1), 1–23.CrossRef
16.
Schwartz, C. E., & Sprangers, M. A. G. (1999). Methodological approaches for assessing response shift in longitudinal health-related quality-of-life research. Social Science and Medicine, 48(11), 1531–1548.CrossRefPubMed
17.
Schwartz, C. E., & Sprangers, M. A. G. (2010). Guidelines for improving the stringency of response shift research using the thentest. Quality of Life Research, 19(4), 455–464.CrossRefPubMed
18.
Westerman, M. J., The, A. M., Sprangers, M. A. G., Groen, H. J. M., van der Wal, G., & Hak, T. (2007). Small-cell lung cancer patients are just ‘a little bit’ tired: Response shift and self-presentation in the measurement of fatigue. Quality of Life Research, 16(5), 853–861.CrossRefPubMedPubMedCentral
19.
Westerman, M. J., Hak, T., Sprangers, M. A. G., Groen, H. J. M., & van der Wal, G. (2008). The AM: Listen to their answers! Response behaviour in the measurement of physical and role functioning. Quality of Life Research, 17(4), 549–558.CrossRefPubMedPubMedCentral
20.
Bloem, E. F., van Zuuren, F. J., Koeneman, M. A., Rapkin, B. D., Visser, M. R. M., Koning, C. C. E., & Sprangers, M. A. G. (2008). Clarifying quality of life assessment: Do theoretical models capture the underlying cognitive processes? Quality of Life Research, 17(8), 1093–1102.CrossRefPubMed
21.
Tourangeau, R., Rips, L. J., & Rasinski, K. (2000). The psychology of survey response. New York: Cambridge University Press.CrossRef
22.
Rapkin, B. D., & Schwartz, C. E. (2004). Toward a theoretical model of quality-of-life appraisal: Implications of findings from studies of response shift. Health and Quality of Life Outcomes, 2, 14.CrossRefPubMedPubMedCentral
23.
Taminiau-Bloem, E. F., Van Zuuren, F. J., Visser, M. R., Tishelman, C., Schwartz, C. E., Koeneman, M. A., et al. (2011). Opening the black box of cancer patients’ quality-of-life change assessments: A think-aloud study examining the cognitive processes underlying responses to transition items. Psychology and Health, 26(11), 1414–1428.CrossRefPubMed
24.
Aaronson, N. K., Ahmedzai, S. A., & Bergman, B. (1993). A quality of life instrument for use in international clinical trials in oncology. Journal of the National Cancer Institute, 85(5), 365–376.CrossRefPubMed
25.
Garratt, A., Schmidt, L., Mackinstosh, A., & Fitzpatrick, R. (2002). Quality of life measurement: bibliographic study of patient assessed health outcome measures. British Medical Journal, 324, 1417–1422.CrossRefPubMedPubMedCentral
26.
Hak, T., van der Veer, K., Jansen, H. (2002). The three-step test-interview (TSTI): An observational instrument for pre-testing self-completion questionnaires. Paper for the International Conference on Questionnaire Development, Evaluation and Testing Methods (QDET). Charleston: South Carolina.
27.
Willis, G. B. (2005). Cognitive interviewing: a tool for improving questionnaire design. California: Thousand Oaks.
28.
Sprangers, M., & Hoogstraten, J. (1989). Pretesting effects in retrospective pretest-posttest designs. Journal of Applied Psychology, 74, 265–272.CrossRef
29.
30.
Bowden, J. A. (1996). Phenomenographic research. In J. A. Bowden & E. Walsh (Eds.), Undertaking phenomenographic research: The Warburton symposium. Melbourne: EQARD.
31.
Nieuwkerk, P. T., Tollenaar, M. S., Oort, F. J., & Sprangers, M. A. G. (2007). Are retrospective measures of change in quality of life more valid than prospective measures? Medical Care, 45(3), 199–205.CrossRefPubMed
32.
Ahmed, S., Mayo, N. E., Wood-Dauphinee, S., Hanley, J. A., & Cohen, S. R. (2004). Response shift influences estimates of change in health-related quality of life (HRQL) post-stroke. Journal of Clinical Epidemiology, 57(6), 561–570.CrossRefPubMed
33.
Sprangers, M. A. G., van Dam, F. S. A. M., Broersen, J., Lodder, L., Wever, L., Visser, M., et al. (1999). Revealing response shift in longitudinal research on fatigue: The use of the thentest approach. Acta Oncologica, 38, 709–718.CrossRefPubMed
34.
Hill, L. G., & Betz, D. L. (2005). Revisiting the retrospective pretest. American Journal of Evaluation, 26(4), 501–517.CrossRef
35.
Norman, G. (2003). Hi! How are you? Response shift, implicit theories and differing epistemologies. Quality of Life Research, 12(3), 239–249.CrossRefPubMed
36.
Eton, D. T. (2010). Why we need response shift: An appeal to functionalism. Quality of Life Research, 19(6), 929–930.CrossRefPubMed
37.
Blome, C., & Augustin, M. (2015). Measuring change in quality of life: Bias in prospective and retrospective evaluation. Value in Health, 18, 110–115.CrossRefPubMed
38.
Hermann, D. (1995). Reporting current, past and change health status. What we know about distortion. Medical Care, 33(4), 77–88.
39.
Cella, D., Hahn, E. A., & Dineen, K. (2002). Meaningful change in cancer-specific quality of life scores: Differences between improvement and worsening. Quality of Life Research, 11(3), 207–221.CrossRefPubMed
Metadata
Title
Using a retrospective pretest instead of a conventional pretest is replacing biases: a qualitative study of cognitive processes underlying responses to thentest items
Authors
Elsbeth F. Taminiau-Bloem
Carolyn E. Schwartz
Florence J. van Zuuren
Margot A. Koeneman
Mechteld R. M. Visser
Carol Tishelman
Caro C. E. Koning
Mirjam A. G. Sprangers
Publication date
01-06-2016
Publisher
Springer International Publishing
Published in
Quality of Life Research / Issue 6/2016
Print ISSN: 0962-9343
Electronic ISSN: 1573-2649
DOI
https://doi.org/10.1007/s11136-015-1175-4

Other articles of this Issue 6/2016

Quality of Life Research 6/2016 Go to the issue

OriginalPaper

Health-related quality of life after first-line anti-cancer treatments for advanced non-small cell lung cancer in clinical practice

OriginalPaper

The impact of taste and smell alterations on quality of life in head and neck cancer patients

OriginalPaper

Impact of clinical osteoarthritis of the hip, knee and hand on self-rated health in six European countries: the European Project on OSteoArthritis

Special Section: Response Shift Effects at Item Level (by invitation only)

Using structural equation modeling to detect response shifts and true change in discrete variables: an application to the items of the SF-36

OriginalPaper

Validation of the EORTC QLQ-INFO 25 questionnaire in Lebanese cancer patients: Is ignorance a Bliss?

Editorial

Introduction to special section on response shift at the item level