Skip to main content
Top

20-11-2023 | Original Research Article

Unravelling the Self-Report Versus Proxy-Report Conundrum for Older Aged Care Residents: Findings from a Mixed-Methods Study

Authors: Julie Ratcliffe, Kiri Lay, Matthew Crocker, Lidia Engel, Rachel Milte, Claire Hutchinson, Jyoti Khadka, David G. T. Whitehurst, Brendan Mulhern, Rosalie Viney, Richard Norman

Published in: The Patient - Patient-Centered Outcomes Research

Login to get access

Abstract

Objectives

No guidance currently exists as to the cognition threshold beyond which self-reported quality of life for older people with cognitive impairment and dementia is unreliable.

Methods

Older aged care residents (≥ 65 years) were randomly assigned to complete the EQ-5D-5L in computer-based (eye movements were tracked) or hard copy (participants were encouraged to ‘think aloud’) format. Cognition was assessed using the Mini-Mental State Examination (MMSE). Think aloud and eye tracking data were analysed by two raters, blinded to MMSE scores. At the participant level, predefined criteria were used to assign traffic light grades (green, amber, red). These grades indicate the extent to which extracted data elements provided evidence of self-report reliability. The MMSE-defined cognition threshold was determined following review of the distributions of assigned traffic light grades.

Results

Eighty-one residents participated and provided complete data (38 eye tracking, 43 think aloud). In the think aloud cohort, all participants with an MMSE score ≤ 23 (n = 10) received an amber or red grade, while 64% of participants with an MMSE score ≥ 24 (21 of 33) received green grades. In the eye tracking cohort, 68% of participants with an MMSE score ≥ 24 (15 of 22) received green grades. Of the 16 eye tracking participants with an MMSE score ≤ 23, 14 (88%) received an amber or red grade.

Conclusions

Most older residents with an MMSE score ≥ 24 have sufficient cognitive capacity to self-complete the EQ-5D-5L. More research is needed to better understand self-completion reliability for other quality-of-life instruments in cognitively impaired populations.
Appendix
Available only for authorised users
Literature
1.
go back to reference United Nations Department of Economic and Social Affairs, Population Division (2020). World population ageing 2020 highlights: living arrangements of older persons (ST/ESA/SER. A/451) United Nations Department of Economic and Social Affairs, Population Division (2020). World population ageing 2020 highlights: living arrangements of older persons (ST/ESA/SER. A/451)
2.
go back to reference Australian Institute of Health and Welfare. Admitted patient care 2017–18: Australian hospital statistics. Canberra: Australian Government; 2019. Australian Institute of Health and Welfare. Admitted patient care 2017–18: Australian hospital statistics. Canberra: Australian Government; 2019.
3.
go back to reference World Health Organisation. Integrated care for older people. Geneva: World Health Organisation; 2018. World Health Organisation. Integrated care for older people. Geneva: World Health Organisation; 2018.
4.
go back to reference Australian Institute of Health and Welfare. Gen aged care data: government spending on aged care. Canberra: Australian Government; 2020. Australian Institute of Health and Welfare. Gen aged care data: government spending on aged care. Canberra: Australian Government; 2020.
5.
go back to reference Department of Health and Aged Care. New residential aged care quality indicators. Australian Government, Canberra, 21 Sept. 2021. Department of Health and Aged Care. New residential aged care quality indicators. Australian Government, Canberra, 21 Sept. 2021.
6.
go back to reference Agarwal A, Pain T, Levesque J, et al. Patient-reported outcome measures (PROMs) to guide clinical care: recommendations and challenges. Med J Aust. 2022;216(1):9–11. CrossRefPubMed Agarwal A, Pain T, Levesque J, et al. Patient-reported outcome measures (PROMs) to guide clinical care: recommendations and challenges. Med J Aust. 2022;216(1):9–11. CrossRefPubMed
7.
go back to reference Sabah S, Knight R, Alvand A, Beard D, Price A. Early patient-reported outcomes from primary hip and knee arthroplasty have improved over the past seven years: an analysis of the NHS PROMs dataset. Bone Jt J. 2022;104-B(6):687–95. CrossRef Sabah S, Knight R, Alvand A, Beard D, Price A. Early patient-reported outcomes from primary hip and knee arthroplasty have improved over the past seven years: an analysis of the NHS PROMs dataset. Bone Jt J. 2022;104-B(6):687–95. CrossRef
8.
go back to reference Organisation for Economic Cooperation and Development (OECD). All hands on deck: Co-developing the first international survey of people living with chronic conditions - Stakeholder engagement in the design, development, and field trial implementation of the PaRIS survey, OECD Health Working Paper No. 149, Jan. 2023. Organisation for Economic Cooperation and Development (OECD). All hands on deck: Co-developing the first international survey of people living with chronic conditions - Stakeholder engagement in the design, development, and field trial implementation of the PaRIS survey, OECD Health Working Paper No. 149, Jan. 2023.
9.
go back to reference Ernstsson O, Janssen M, Heintz A. Collection and use of EQ-5D for follow-up, decision-making, and quality improvement in health care-the case of the Swedish National Quality Registries. J Patient Report Outcomes. 2020;4:78. CrossRef Ernstsson O, Janssen M, Heintz A. Collection and use of EQ-5D for follow-up, decision-making, and quality improvement in health care-the case of the Swedish National Quality Registries. J Patient Report Outcomes. 2020;4:78. CrossRef
10.
go back to reference Dean S, Al Sayah F, Johnson J. Measuring value in healthcare from a patients' perspective. J Patient Report Outcomes. 2021;5(Suppl 2):88. CrossRef Dean S, Al Sayah F, Johnson J. Measuring value in healthcare from a patients' perspective. J Patient Report Outcomes. 2021;5(Suppl 2):88. CrossRef
11.
go back to reference Timmons S, Manning E, Barrett A, et al. Dementia in older people admitted to hospital: a regional multi-hospital observational study of prevalence, associations and case recognition. Age Ageing. 2015;44(6):993–9. CrossRefPubMedPubMedCentral Timmons S, Manning E, Barrett A, et al. Dementia in older people admitted to hospital: a regional multi-hospital observational study of prevalence, associations and case recognition. Age Ageing. 2015;44(6):993–9. CrossRefPubMedPubMedCentral
12.
go back to reference Australian Institute of Health and Welfare. Dementia in Australia. Canberra: Australian Government; 2023. Australian Institute of Health and Welfare. Dementia in Australia. Canberra: Australian Government; 2023.
13.
go back to reference O’Shea E, Hopper L, Marques M, et al. A comparison of self and proxy quality of life ratings for people with dementia and their carers: a European prospective cohort study. Aging Ment Health. 2020;24(1):162–70. CrossRefPubMed O’Shea E, Hopper L, Marques M, et al. A comparison of self and proxy quality of life ratings for people with dementia and their carers: a European prospective cohort study. Aging Ment Health. 2020;24(1):162–70. CrossRefPubMed
14.
go back to reference Sousa M, Santos R, Simoes P, Conde-Sala J, Dourado M. Discrepancies between Alzheimer’s disease patients’ and caregivers’ ratings about patients’ quality of life. Alzheimer Dis Assoc Disord. 2018;32(3):240–6. CrossRefPubMed Sousa M, Santos R, Simoes P, Conde-Sala J, Dourado M. Discrepancies between Alzheimer’s disease patients’ and caregivers’ ratings about patients’ quality of life. Alzheimer Dis Assoc Disord. 2018;32(3):240–6. CrossRefPubMed
15.
go back to reference Hutchinson C, Worley A, Khadka J, Milte R, Cleland J, Ratcliffe J. Do we agree or disagree? A systematic review of the application of preference-based instruments in self and proxy reporting of quality of life in older people. Soc Sci Med. 2022;305: 115046. CrossRefPubMed Hutchinson C, Worley A, Khadka J, Milte R, Cleland J, Ratcliffe J. Do we agree or disagree? A systematic review of the application of preference-based instruments in self and proxy reporting of quality of life in older people. Soc Sci Med. 2022;305: 115046. CrossRefPubMed
16.
go back to reference Ayton D, Gardam M, Pritchard E, et al. Patient-reported outcome measures to inform care of people with dementia-a systematic scoping review. The Gerontologist. 2021;61(5):e185–94. CrossRefPubMed Ayton D, Gardam M, Pritchard E, et al. Patient-reported outcome measures to inform care of people with dementia-a systematic scoping review. The Gerontologist. 2021;61(5):e185–94. CrossRefPubMed
17.
go back to reference Brazier J, Ratcliffe J, Salomon J, Tsuchiya A. Measuring and valuing health benefits for economic evaluation. 2nd ed. Oxford: Oxford University Press; 2017. Brazier J, Ratcliffe J, Salomon J, Tsuchiya A. Measuring and valuing health benefits for economic evaluation. 2nd ed. Oxford: Oxford University Press; 2017.
18.
go back to reference Herdman M, Gudex C, Llloyd A, Janssen M, Kind P, Parkin D, Bonszel G, Badia X. Development and preliminary testing of the new five-level version of EQ-5D (EQ-5D-5L). Qual Life Res. 2011;20:1727–36. CrossRefPubMedPubMedCentral Herdman M, Gudex C, Llloyd A, Janssen M, Kind P, Parkin D, Bonszel G, Badia X. Development and preliminary testing of the new five-level version of EQ-5D (EQ-5D-5L). Qual Life Res. 2011;20:1727–36. CrossRefPubMedPubMedCentral
19.
go back to reference Lay K, Crocker M, Engel L, Ratcliffe J, Charlton S, Hutchinson C. How do older adults receiving aged care services understand and respond to the EQ-5D-5L? A think aloud study in residential care. Qual Health Res. 2023;32(11):3161–70. Lay K, Crocker M, Engel L, Ratcliffe J, Charlton S, Hutchinson C. How do older adults receiving aged care services understand and respond to the EQ-5D-5L? A think aloud study in residential care. Qual Health Res. 2023;32(11):3161–70.
20.
go back to reference Milte R, Crocker M, Lay K, Ratcliffe J, Mulhern B, Norman R, Viney R, Khadka J. Feasibility of self-reported health related quality of life assessment with older people in residential care: insights from the application of eye tracking technology. Qual Life Res. 2023;32(12):3557–69. CrossRefPubMedPubMedCentral Milte R, Crocker M, Lay K, Ratcliffe J, Mulhern B, Norman R, Viney R, Khadka J. Feasibility of self-reported health related quality of life assessment with older people in residential care: insights from the application of eye tracking technology. Qual Life Res. 2023;32(12):3557–69. CrossRefPubMedPubMedCentral
21.
go back to reference Hennink M, Kaiser B, Marconi V. Code saturation versus meaning saturation: how many interviews are enough? Qual Health Res. 2017;27(4):591–608. CrossRefPubMed Hennink M, Kaiser B, Marconi V. Code saturation versus meaning saturation: how many interviews are enough? Qual Health Res. 2017;27(4):591–608. CrossRefPubMed
22.
go back to reference Wang K, Barr C, Norman R, George S, Whitehead C, Ratcliffe J. Using eye tracking technology with older people in memory clinics to investigate the impact of mild cognitive impairment on choices for EQ-5D-5L health states preferences. Appl Health Econ Health Policy. 2021;19(1):111–21. CrossRefPubMed Wang K, Barr C, Norman R, George S, Whitehead C, Ratcliffe J. Using eye tracking technology with older people in memory clinics to investigate the impact of mild cognitive impairment on choices for EQ-5D-5L health states preferences. Appl Health Econ Health Policy. 2021;19(1):111–21. CrossRefPubMed
23.
go back to reference Cullen B, Evans J, Coen R, et al. A review of screening tests for cognitive impairment. J Neurol Neurosurg Psychiatry. 2007;78:790–9. CrossRefPubMed Cullen B, Evans J, Coen R, et al. A review of screening tests for cognitive impairment. J Neurol Neurosurg Psychiatry. 2007;78:790–9. CrossRefPubMed
24.
go back to reference Kahle-Wrobleski K, Corrada M, Li B, Kawas C. Sensitivity and specificity of the Mini-Mental State Examination for identifying dementia in the oldest-old: the 90+ Study. J Am Geriatr Soc. 2007;55(2):284–9. CrossRefPubMedPubMedCentral Kahle-Wrobleski K, Corrada M, Li B, Kawas C. Sensitivity and specificity of the Mini-Mental State Examination for identifying dementia in the oldest-old: the 90+ Study. J Am Geriatr Soc. 2007;55(2):284–9. CrossRefPubMedPubMedCentral
25.
go back to reference National Institute for Health and Care Excellence. Donepezil, galantamine, rivastigmine and memantine for the treatment of Alzheimer’s disease. Technology appraisal guidance June 2018. National Institute for Health and Care Excellence. Donepezil, galantamine, rivastigmine and memantine for the treatment of Alzheimer’s disease. Technology appraisal guidance June 2018.
26.
go back to reference Van den Haak M, De Jong M, Schellnes P. Retrospective vs concurrent think aloud protocols: testing the usability of an online library catalogue. Behav Inform Technol. 2003;22(5):339–51. CrossRef Van den Haak M, De Jong M, Schellnes P. Retrospective vs concurrent think aloud protocols: testing the usability of an online library catalogue. Behav Inform Technol. 2003;22(5):339–51. CrossRef
27.
go back to reference Whitty J, Walker R, Golenko X, Ratcliffe J. A think aloud study comparing the validity and acceptability of discrete choice and best worst scaling methods. PLoS ONE. 2014;9(4):e90635. CrossRefPubMedPubMedCentral Whitty J, Walker R, Golenko X, Ratcliffe J. A think aloud study comparing the validity and acceptability of discrete choice and best worst scaling methods. PLoS ONE. 2014;9(4):e90635. CrossRefPubMedPubMedCentral
29.
go back to reference Staub A, Rayner K. Eye movements and on-line comprehension processes. In: Gaskill G, editor. the Oxford Handbook of psycholinguistics. Oxford University Press; 2007. p. 327–42. Staub A, Rayner K. Eye movements and on-line comprehension processes. In: Gaskill G, editor. the Oxford Handbook of psycholinguistics. Oxford University Press; 2007. p. 327–42.
30.
go back to reference Guerriero C, Abrines Jaume N, Diaz-Ordaz K, Brown K, Wray J, Ashworth J, Abbiss M, Cairns J. Using animation to self-report health: a randomized experiment with children. The Patient. 2020;13:175–88. CrossRefPubMed Guerriero C, Abrines Jaume N, Diaz-Ordaz K, Brown K, Wray J, Ashworth J, Abbiss M, Cairns J. Using animation to self-report health: a randomized experiment with children. The Patient. 2020;13:175–88. CrossRefPubMed
31.
go back to reference Hutchinson C, Ratcliffe J, Cleland J, Walker R, Milte R, McBain C, Corlis M, Cornell V, Khadka J. The integration of mixed methods data to develop the quality of life-aged care consumers (QOL-ACC) instrument. BMC Geriatr. 2021;21(1):702. CrossRefPubMedPubMedCentral Hutchinson C, Ratcliffe J, Cleland J, Walker R, Milte R, McBain C, Corlis M, Cornell V, Khadka J. The integration of mixed methods data to develop the quality of life-aged care consumers (QOL-ACC) instrument. BMC Geriatr. 2021;21(1):702. CrossRefPubMedPubMedCentral
32.
go back to reference Keetharuth A, Taylor Buck E, Acquadro C, Conway K, Connell J, Barkham M, Carlton J, Ricketts T, Barber R, Brazier J. Integrating qualitative and quantitative data in the development of outcome measures: the case of the recovering quality of life (ReQoL) measures in mental health populations. Int J Environ Res Public Health. 2018;15(7):1342. CrossRefPubMedPubMedCentral Keetharuth A, Taylor Buck E, Acquadro C, Conway K, Connell J, Barkham M, Carlton J, Ricketts T, Barber R, Brazier J. Integrating qualitative and quantitative data in the development of outcome measures: the case of the recovering quality of life (ReQoL) measures in mental health populations. Int J Environ Res Public Health. 2018;15(7):1342. CrossRefPubMedPubMedCentral
33.
go back to reference Tourangeau R, Bradburn N. The psychology of survey response. In: Marsden P, Wright J, editors. Handbook of survey research. 2nd ed. Emerald publishing group; 2009. p. 315–46. Tourangeau R, Bradburn N. The psychology of survey response. In: Marsden P, Wright J, editors. Handbook of survey research. 2nd ed. Emerald publishing group; 2009. p. 315–46.
34.
go back to reference Beer C, Flicker L, Horner B, et al. Factors associated with self and informant ratings of the quality of life of people with dementia living in care facilities: a cross sectional study. PLoS ONE. 2010;5(12): e15621. CrossRefPubMedPubMedCentral Beer C, Flicker L, Horner B, et al. Factors associated with self and informant ratings of the quality of life of people with dementia living in care facilities: a cross sectional study. PLoS ONE. 2010;5(12): e15621. CrossRefPubMedPubMedCentral
35.
go back to reference Burks H, des Bordes J, Chadha R, Holmes H, Rianon N. Quality of life assessment in older adults with dementia: a systematic review. Dement Geriatr Cognit Disord. 2021;50:103–10. CrossRef Burks H, des Bordes J, Chadha R, Holmes H, Rianon N. Quality of life assessment in older adults with dementia: a systematic review. Dement Geriatr Cognit Disord. 2021;50:103–10. CrossRef
36.
go back to reference Neumann P, Sander G, Russell L, Seigel J, Ganiats T. Cost-effectiveness in health and medicine. Oxford, UK: Oxford University Press; 2016. CrossRef Neumann P, Sander G, Russell L, Seigel J, Ganiats T. Cost-effectiveness in health and medicine. Oxford, UK: Oxford University Press; 2016. CrossRef
37.
go back to reference Basch E. New frontiers in patient-reported outcomes: adverse event reporting, comparative effectiveness, and quality assessment. Annu Rev Med. 2014;65:307–17. CrossRefPubMed Basch E. New frontiers in patient-reported outcomes: adverse event reporting, comparative effectiveness, and quality assessment. Annu Rev Med. 2014;65:307–17. CrossRefPubMed
38.
go back to reference Lavallee D, Chenok K, Love R, Petersen C, et al. Incorporating patient-reported outcomes into health care to engage patients and enhance care. Health Aff. 2016;35:575–82. CrossRef Lavallee D, Chenok K, Love R, Petersen C, et al. Incorporating patient-reported outcomes into health care to engage patients and enhance care. Health Aff. 2016;35:575–82. CrossRef
39.
go back to reference US Preventive Services Task Force. Screening for cognitive impairment in older adults. US Preventive Services Task Force Recommendation Statement. JAMA. 2020;323(8):757–63. CrossRef US Preventive Services Task Force. Screening for cognitive impairment in older adults. US Preventive Services Task Force Recommendation Statement. JAMA. 2020;323(8):757–63. CrossRef
40.
go back to reference World Health Organisation and Alzheimers Diseases International. Dementia: a public health priority. Berlin: World Health Organisation; 2012. World Health Organisation and Alzheimers Diseases International. Dementia: a public health priority. Berlin: World Health Organisation; 2012.
41.
go back to reference Griffiths A, Smith S, Martin A, Meads D, Kelley R, Surr C. Exploring self-report and proxy-report quality-of-life measures for people living with dementia in care homes. Qual Life Res. 2020;29:463–72. CrossRefPubMed Griffiths A, Smith S, Martin A, Meads D, Kelley R, Surr C. Exploring self-report and proxy-report quality-of-life measures for people living with dementia in care homes. Qual Life Res. 2020;29:463–72. CrossRefPubMed
Metadata
Title
Unravelling the Self-Report Versus Proxy-Report Conundrum for Older Aged Care Residents: Findings from a Mixed-Methods Study
Authors
Julie Ratcliffe
Kiri Lay
Matthew Crocker
Lidia Engel
Rachel Milte
Claire Hutchinson
Jyoti Khadka
David G. T. Whitehurst
Brendan Mulhern
Rosalie Viney
Richard Norman
Publication date
20-11-2023
Publisher
Springer International Publishing
Published in
The Patient - Patient-Centered Outcomes Research
Print ISSN: 1178-1653
Electronic ISSN: 1178-1661
DOI
https://doi.org/10.1007/s40271-023-00655-6