Top

BMC Health Services Research

Published in:

Open Access 01-12-2018 | Research article

Qualitative assessment of the primary care outcomes questionnaire: a cognitive interview study

Authors: Mairead Murphy, Sandra Hollinghurst, Chris Salisbury

Published in: BMC Health Services Research | Issue 1/2018

Abstract

Background

The Primary Care Outcomes Questionnaire (PCOQ) is a new patient-reported outcome measure designed specifically for primary care. This paper describes the developmental process of improving the item quality and testing the face validity of the PCOQ through cognitive interviews with primary care patients.

Methods

Two formats of the PCOQ were developed and assessed: the PCOQ-Status (which has an adjectival scale) and the PCOQ-Change (which has the same items as the PCOQ-Status, but a transitional scale). Three rounds of cognitive interviews were held with twenty patients from four health centres in Bristol. Patients seeking healthcare were recruited directly by their GP or practice nurse, and others not currently seeking healthcare were recruited from patient participation groups. An adjusted form of Tourangeau’s model of cognitive processing was used to identify problems. This contained four categories: general comprehension, temporal comprehension, decision process, and response process. The resultant pattern of problems was used to assess whether the items and scales were working as intended, and to make improvements to the questionnaires.

Results

The problems identified in the PCOQ-Status reduced from 41 in round one to seven in round three. It was noted that the PCOQ-Status seemed to be capturing a subjective view of health which might not vary with age or long-term conditions. However, as it is designed to be evaluative (measuring change over time) as opposed to discriminative (measuring change between different groups of people), this does not present a problem for validity. The PCOQ-Status was both understood by patients and was face valid. The PCOQ-Change had less face validity, and was misunderstood by three out of six patients in round 1. It was not taken forward after this round.

Conclusions

The cognitive interviews successfully contributed to the development of the PCOQ. Through this study, the PCOQ-Status was found to be well understood by patients, and it was possible to improve comprehension through each round of interviews. The PCOQ-Change was poorly understood and, given that this corroborates existing research, this may call into question the use of transitional questionnaires generally.

Available only for authorised users

Fortin M, Bravo G, Hudon C, et al. Prevalence of multimorbidity among adults seen in family practice. Ann Fam Med. 2005;3(3):223–8. https://doi.org/10.1370/afm.272. [published Online First: 2005/06/02]

Olayiwola JN, Anderson D, Jepeal N, et al. Electronic consultations to improve the primary care-specialty care Interface for cardiology in the medically underserved: a cluster-randomized controlled trial. Ann Fam Med. 2016;14(2):133–40. https://doi.org/10.1370/afm.1869.

Sharma AE, Willard-Grace R, Hessler D, et al. What happens after health coaching? Observational study 1 year following a randomized controlled trial. Ann Fam Med. 2016;14(3):200–7. https://doi.org/10.1370/afm.1924.

Barnes R. ISRCTN registry: footprints in primary care. 24/06/2015 ed, 2015.

Donabedian A. The quality of care. How can it be assessed? J Am Med Assoc. 1988;260(12):1743–8.CrossRef

Fitzpatrick R. Patient-reported outcomes and performance measurement. In: Smith P, Mossialos M, Leatherman S, et al., editors. Performance measurement for health system improvement: experiences, challenges and Prospect. Cambridge: Cambridge University Press; 2009. p. 63–86.

Salisbury C, Procter S, Stewart K, et al. The content of general practice consultations: cross-sectional study based on video recordings. Br J Gen Pract. 2013;63(616):751–9. https://doi.org/10.3399/bjgp13X674431.

Ware JE Jr, Sherbourne CD. The MOS 36-item short-form health survey (SF-36). I. Conceptual framework and item selection. Med Care. 1992;30(6):473–83.CrossRefPubMed

EuroQol BR. The current state of play. Health Policy. 1996;37(1):53–72.CrossRef

10.

Venning P, Durie A, Roland M, et al. Randomised controlled trial comparing cost effectiveness of general practitioners and nurse practitioners in primary care. BMJ. 2000;320(7241):1048–53.CrossRefPubMedPubMedCentral

11.

RK MK, Cragg DK, Hastings AM, et al. Comparison of out of hours care provided by patients' own general practitioners and commercial deputising services: a randomised controlled trial. II: the outcome of care. BMJ. 1997;314(7075):190–3.CrossRef

12.

Paterson C. Measuring outcomes in primary care: a patient generated measure, MYMOP, compared with the SF-36 health survey. BMJ 1996;312(7037):1016–1020. doi: https://doi.org/10.1136/bmj.312.7037.1016

13.

Paterson C. University of Bristol website, PHC section, MYMOP: University of Bristol; 2012 [Available from: http://www.bristol.ac.uk/primaryhealthcare/resources/mymop/strengthsandweaknesses/ Accessed 25 Apr 2014.

14.

Howie JG, Heaney DJ, Maxwell M, et al. A comparison of a patient enablement instrument (PEI) against two established satisfaction scales as an outcome measure of primary care consultations. Fam Pract. 1998;15(2):165–71.CrossRefPubMed

15.

Howie JG, Heaney DJ, Maxwell M, et al. Quality at general practice consultations: cross sectional survey. Br Med J. 1999;319(7212):738–43. [published Online First: 1999/09/17]CrossRef

16.

Streiner DL, Norman GR. Health measurement scales: a practical guide to their development and use. New York, USA: Oxford University Press; 2008.CrossRef

17.

Valderas JM, Fitzpatrick R, Roland M. Using health status to measure NHS performance: another step into the dark for the health reform in England. BMJ Qual Saf. 2012;21(4):352–3. https://doi.org/10.1136/bmjqs-2011-000184.

18.

Murphy M, Salisbury C, Hollinghurst S. Can the outcome of primary care be measured by a patient reported outcome measure? Br J Gen Pract. 2014;64(629):647–8. https://doi.org/10.3399/bjgp14X683017.

19.

Reeve BB, Wyrwich KW, Wu AW, et al. ISOQOL recommends minimum standards for patient-reported outcome measures used in patient-centered outcomes and comparative effectiveness research. Qual Life Res. 2013;22(8):1889–905. https://doi.org/10.1007/s11136-012-0344-y.

20.

Mokkink LB, Terwee CB, Patrick DL, et al. The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments: an international Delphi study. Qual Life Res. 2010;19(4):539–49. https://doi.org/10.1007/s11136-010-9606-8.

21.

Fitzpatrick R, Davey C, Buxton MJ, et al. Evaluating patient-based outcome measures for use in clinical trials. Health Technol Assess. 1998;2(14):i–iv. 1-74

22.

Murphy M, Hollinghurst S, Turner K, et al. Patient and practitioners' views on the most important outcomes arising from primary care consultations: a qualitative study. BMC Fam Pract. 2015;16:108. https://doi.org/10.1186/s12875-015-0323-9.

23.

Murphy M, Hollinghurst S, Salisbury C. Agreeing the content of a patient-reported outcome measure for primary care: a Delphi consensus study. Health Expect. 2016; https://doi.org/10.1111/hex.12462.

24.

Murphy M, Hollinghurst S, Cowlishaw S, et al. Psychometric testing of the primary care outcomes questionnaire. Br J Gen Prac Pending publ. 2018; https://doi.org/10.3399/bjgp14X683017.

25.

Ruger JP. Health capability: conceptualization and operationalization. Am J Public Health. 2010;100(1):41–9. https://doi.org/10.2105/AJPH.2008.143651.

26.

Black N. Patient reported outcome measures could help transform healthcare. Br Med J. 2013;346:f167. https://doi.org/10.1136/bmj.f167. [published Online First: 2013/01/30]

27.

Kamper SJ, Maher CG, Mackay G. Global rating of change scales: a review of strengths and weaknesses and considerations for design. J Man Manip Ther. 2009;17(3):163–70.CrossRefPubMedPubMedCentral

28.

Reilly D, Mercer SW, Bikker AP, et al. Outcome related to impact on daily living: preliminary validation of the ORIDL instrument. BMC Health Serv Res. 2007;7:139.CrossRefPubMedPubMedCentral

29.

Haddad S, Potvin L, Roberge D, et al. Patient perception of quality following a visit to a doctor in a primary care unit. Fam Pract. 2000;17(1):21–9.CrossRefPubMed

30.

Knäuper B, Carrière K, Chamandy M, et al. How aging affects self-reports. Euro J Ageing. 2016;13(2):185–93. https://doi.org/10.1007/s10433-016-0369-0.

31.

Choi BC, Pak AW. A catalog of biases in questionnaires. Prev Chronic Dis. 2005;2(1):A13.PubMed

32.

Watt T, Rasmussen AK, Groenvold M, et al. Improving a newly developed patient-reported outcome for thyroid patients, using cognitive interviewing. Qual Life Res. 2008;17(7):1009–17. https://doi.org/10.1007/s11136-008-9364-z.

33.

Willis G. Cognitive interviewing - a how to guide: Research Triangle Institute, 1999.

34.

Beatty P. The dynamics of cognitive interviewing. In: Presser S, editor. Methods for testing and evaluating survey questionnaires. Hoboken, N.J: John Wiley & Sons. p. xvi–2004. 606 p.

35.

Al-Janabi H, Flynn TN, Coast J. Development of a self-report measure of capability wellbeing for adults: the ICECAP-A. Qual Life Res. 2012;21(1):167–76. https://doi.org/10.1007/s11136-011-9927-2.

36.

Al-Janabi H, Keeley T, Mitchell P, et al. Can capabilities be self-reported? A think aloud study. Soc Sci Med. 2013;87:116–22. https://doi.org/10.1016/j.socscimed.2013.03.035.

37.

Matza LS, Boye KS, Stewart KD, et al. A qualitative examination of the content validity of the EQ-5D-5L in patients with type 2 diabetes. Health Qual Life Outcomes. 2015;13:192. https://doi.org/10.1186/s12955-015-0373-7.

38.

Tourangeau R. Cognitive sciences and survey methods. In: Jabine T, Straf M, Tanur J, et al., editors. Cognitive aspects of survey methodology: building a bridge between disciplines. Washington, DC: National Academy Press; 1984. p. 73–100.

39.

Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care. 2007;19(6):349–57. https://doi.org/10.1093/intqhc/mzm042.

40.

Horwood J, Pollard B, Ayis S, et al. Listening to patients: using verbal data in the validation of the Aberdeen measures of impairment, activity limitation and participation restriction (Ab-IAP). BMC Musculoskelet Disord 2010;11:182. doi: https://doi.org/10.1186/1471-2474-11-182.

41.

Mallinson S. Listening to respondents: a qualitative assessment of the short-form 36 health status questionnaire. Soc Sci Med. 2002;54(1):11–21.CrossRefPubMed

42.

Guyatt G, Walter S, Norman G. Measuring change over time: assessing the usefulness of evaluative instruments. J Chronic Dis. 1987;40(2):171–8.CrossRefPubMed

43.

Mc Dowell I. Measuring health. 2nd ed. New York: Oxford University Press; 2006.CrossRef

44.

Ware J. Scales for measuring general health perceptions. Health Serv Res. 1976;11:396–415.PubMedPubMedCentral

45.

Herrmann D. Reporting current, past, and changed health status. What we know about distortion. Med Care. 1995;33(4 Suppl):AS89–94.PubMed

46.

Guyatt GH, Norman GR, Juniper EF, et al. A critical look at transition ratings. J Clin Epidemiol. 2002;55(9):900–8.CrossRefPubMed

47.

Campbell JL, Fletcher E, Britten N, et al. Telephone triage for management of same-day consultation requests in general practice (the ESTEEM trial): a cluster-randomised controlled trial and cost-consequence analysis. Lancet. 2014;384(9957):1859–68. https://doi.org/10.1016/S0140-6736(14)61058-8.

48.

Salisbury C, Montgomery AA, Hollinghurst S, et al. Effectiveness of PhysioDirect telephone assessment and advice services for patients with musculoskeletal problems: pragmatic randomised controlled trial. Br Med J. 2013;346:f43. https://doi.org/10.1136/bmj.f43. (jan29 3)

49.

Saldana J. Chapter 1: an introduction to codes and coding. In: SAGE Publications, editor. The coding manual for qualitative researchers. London: SAGE publications; 2009.

50.

Conrad FG, Blair J. Data quality in cognitive interviews: the case of verbal reports. In: Presser S, editor. Methods for testing and evaluating survey questionnaires. Hoboken, N.J: John Wiley & Sons; 2004. p. xvi. 606 p.

51.

Lozano LM, García-Cueto E, Muñiz J. Effect of the number of response categories on the reliability and validity of rating scales. Methodology. 2008;4(2):73–9. https://doi.org/10.1027/1614-2241.4.2.73.

52.

Horwood J, Sutton E, Coast J. Evaluating the face validity of the ICECAP-O capabilities measure: a “think aloud” study with hip and knee Arthroplasty patients. Appl Res Qual Life. 2013;9(3):667–682. https://doi.org/10.1007/s11482-013-9264-4

Title: Qualitative assessment of the primary care outcomes questionnaire: a cognitive interview study
Authors: Mairead Murphy
Sandra Hollinghurst
Chris Salisbury
Publication date: 01-12-2018
Publisher: BioMed Central
Published in: BMC Health Services Research / Issue 1/2018
Electronic ISSN: 1472-6963
DOI: https://doi.org/10.1186/s12913-018-2867-6

At a glance: The ONWARDS insulin icodec trials

Springer Medicine

Qualitative assessment of the primary care outcomes questionnaire: a cognitive interview study

Abstract

Background

Methods

Results

Conclusions

At a glance: The ONWARDS insulin icodec trials

Springer Medicine

Abstract

Background

Methods

Results

Conclusions

Please log in to get access to this content

Other articles of this Issue 1/2018

Understanding Indigenous Australians’ experiences of cancer care: stakeholders’ views on what to measure and how to measure it

Elusive implementation: an ethnographic study of intersectoral policymaking for health

Clinicians’ attitude towards a placebo-controlled randomised clinical trial investigating the effect of neuraminidase inhibitors in adults hospitalised with influenza

Facilitating the implementation of clinical technology in healthcare: what role does a national agency play?

A qualitative exploration of contextual factors that influence dissemination and implementation of evidence-based chronic disease prevention across four countries

SCOPEOUT: sustainability and spread of quality improvement activities in long-term care- a mixed methods approach