Top

Trials

Published in:

Open Access 01-12-2016 | Methodology

The use of qualitative methods to inform Delphi surveys in core outcome set development

Authors: T. Keeley, P. Williamson, P. Callery, L. L. Jones, J. Mathers, J. Jones, B. Young, M. Calvert

Published in: Trials | Issue 1/2016

Abstract

Background

Core outcome sets (COS) help to minimise bias in trials and facilitate evidence synthesis. Delphi surveys are increasingly being used as part of a wider process to reach consensus about what outcomes should be included in a COS. Qualitative research can be used to inform the development of Delphi surveys. This is an advance in the field of COS development and one which is potentially valuable; however, little guidance exists for COS developers on how best to use qualitative methods and what the challenges are. This paper aims to provide early guidance on the potential role and contribution of qualitative research in this area. We hope the ideas we present will be challenged, critiqued and built upon by others exploring the role of qualitative research in COS development.

This paper draws upon the experiences of using qualitative methods in the pre-Delphi stage of the development of three different COS. Using these studies as examples, we identify some of the ways that qualitative research might contribute to COS development, the challenges in using such methods and areas where future research is required.

Results

Qualitative research can help to identify what outcomes are important to stakeholders; facilitate understanding of why some outcomes may be more important than others, determine the scope of outcomes; identify appropriate language for use in the Delphi survey and inform comparisons between stakeholder data and other sources, such as systematic reviews. Developers need to consider a number of methodological points when using qualitative research: specifically, which stakeholders to involve, how to sample participants, which data collection methods are most appropriate, how to consider outcomes with stakeholders and how to analyse these data. A number of areas for future research are identified.

Conclusions

Qualitative research has the potential to increase the research community’s confidence in COS, although this will be dependent upon using rigorous and appropriate methodology. We have begun to identify some issues for COS developers to consider in using qualitative methods to inform the development of Delphi surveys in this article.

Friedman LM, Furberg CD, DeMets DL. Fundamentals of clinical trials. 4th ed. New York: Springer; 2010.CrossRef

ICH Harmonised Tripartite Guideline. Statistical principles for clinical trials. International Conference on Harmonisation E9 Expert Working Group. Stat Med. 1999;18(15):1905–42.

Williamson PR, Altman DG, Blazeby JM, Clarke M, Devane D, Gargon E, et al. Developing core outcome sets for clinical trials: issues to consider. Trials. 2012;13. doi: 10.1186/1745-6215-13-132

Macefield RC, Boulind CE, Blazeby JM. Selecting and measuring optimal outcomes for randomised controlled trials in surgery. Langenbeck’s archives of surgery/Deutsche Gesellschaft fur Chirurgie. 2014;399(3):263–72. doi:10.1007/s00423-013-1136-8.CrossRefPubMedPubMedCentral

Tovey D. The impact of Cochrane Reviews. Cochrane Database Syst Rev. 2011;2011:ED000007. doi:10.1002/14651858.ED000007.

Hirsch BR, Califf RM, Cheng SK, Tasneem A, Horton J, Chiswell K, et al. Characteristics of oncology clinical trials: insights from a systematic analysis of ClinicalTrials.gov. JAMA Intern Med. 2013;173(11):972–9. doi:10.1001/jamainternmed.2013.627.CrossRefPubMed

Dwan K, Altman DG, Arnaiz JA, Bloom J, Chan AW, Cronin E, et al. Systematic review of the empirical evidence of study publication bias and outcome reporting bias. PLoS One. 2008;3(8):e3081. doi:10.1371/journal.pone.0003081.CrossRefPubMedPubMedCentral

Kirkham JJ, Dwan KM, Altman DG, Gamble C, Dodd S, Smyth R, et al. The impact of outcome reporting bias in randomised controlled trials on a cohort of systematic reviews. BMJ. 2010;340:c365. doi:10.1136/bmj.c365.CrossRefPubMed

McGauran N, Wieseler B, Kreis J, Schuler YB, Kolsch H, Kaiser T. Reporting bias in medical research – a narrative review. Trials. 2010;11:37. doi:10.1186/1745-6215-11-37.CrossRefPubMedPubMedCentral

10.

Williamson P, Clarke M. The COMET (Core Outcome Measures in Effectiveness Trials) Initiative: its role in improving Cochrane Reviews. Cochrane Database Syst Rev. 2012;5:ED000041. doi:10.1002/14651858.ED000041.PubMed

11.

Sinha IP, Altman DG, Beresford MW, Boers M, Clarke M, Craig J, et al. Standard 5: selection, measurement, and reporting of outcomes in clinical trials in children. Pediatrics. 2012;129:S146–S52. doi:10.1542/peds.2012-0055H.CrossRefPubMed

12.

Gargon E, Williamson PR, Altman DG, Blazeby JM, Clarke M. The COMET Initiative database: progress and activities from 2011 to 2013. Trials. 2014;15:279. doi:10.1186/1745-6215-15-279.CrossRefPubMedPubMedCentral

13.

Gargon E, Gurung B, Medley N, Altman DG, Blazeby JM, Clarke M, et al. Choosing important health outcomes for comparative effectiveness research: a systematic review. PLoS One. 2014;9(6):e99111. doi:10.1371/journal.pone.0099111.CrossRefPubMedPubMedCentral

14.

Koroshetz W. A core set of trial outcomes for every medical discipline? BMJ. 2015;350:h85. doi:10.1136/bmj.h85.CrossRefPubMed

15.

Jones J, Hunter D. Consensus methods for medical and health-services research. BMJ. 1995;311(7001):376–80.CrossRefPubMedPubMedCentral

16.

Hasson F, Keeney S, McKenna H. Research guidelines for the Delphi survey technique. J Adv Nurs. 2000;32(4):1008–15.PubMed

17.

Allard A, Fellowes A, Shilling V, Janssens A, Beresford B, Morris C. Key health outcomes for children and young people with neurodisability: qualitative research with young people and parents. BMJ Open. 2014;4(4):e004611. doi:10.1136/bmjopen-2013-004611.CrossRefPubMedPubMedCentral

18.

Sanderson T, Morris M, Calnan M, Richards P, Hewlett S. Patient perspective of measuring treatment efficacy: the rheumatoid arthritis patient priorities for pharmacologic interventions outcomes. Arthritis Care Res. 2010;62(5):647–56. doi:10.1002/acr.20151.CrossRef

19.

Sinha IP, Gallagher R, Williamson PR, Smyth RL. Development of a core outcome set for clinical trials in childhood asthma: a survey of clinicians, parents, and young people. Trials. 2012;13:103. doi:10.1186/1745-6215-13-103.CrossRefPubMedPubMedCentral

20.

Morris C, Janssens A, Shilling V, Allard A, Fellowes A, Tomlinson R, et al. Meaningful health outcomes for paediatric neurodisability: stakeholder prioritisation and appropriateness of patient reported outcome measures. Health Qual Life Outcomes. 2015;13:87. doi:10.1186/s12955-015-0284-7.CrossRefPubMedPubMedCentral

21.

Bruce I, Harman N, Williamson P, Tierney S, Callery P, Mohiuddin S, et al. The management of Otitis Media with Effusion in children with cleft palate (mOMEnt): a feasibility study and economic evaluation. Health Technol Assess. 2015;19(68):1–374. doi:10.3310/hta19680.CrossRefPubMedPubMedCentral

22.

Sanderson T, Morris M, Calnan M, Richards P, Hewlett S. What outcomes from pharmacologic treatments are important to people with rheumatoid arthritis? Creating the basis of a patient core set. Arthritis Care Res. 2010;62(5):640–6. doi:10.1002/acr.20034.CrossRef

23.

Hewlett S, Cockshott Z, Byron M, Kitchen K, Tipler S, Pope D, et al. Patients’ perceptions of fatigue in rheumatoid arthritis: overwhelming, uncontrollable, ignored. Arthritis Rheum. 2005;53(5):697–702. doi:10.1002/art.21450.CrossRefPubMed

24.

Saketkoo LA, Mittoo S, Frankel S, LeSage D, Sarver C, Phillips K, et al. Reconciling healthcare professional and patient perspectives in the development of disease activity and response criteria in connective tissue disease-related interstitial lung diseases. J Rheumatol. 2014;41(4):792–8. doi:10.3899/jrheum.131251.CrossRefPubMedPubMedCentral

25.

Patton MQ. Qualitative research and evaluation methods. 3rd ed. SAGE: Thousand Oaks, CA; 2002.

26.

Glaser BG, Strauss AL. The discovery of grounded theory: strategies for qualitative research. Chicago: Aldine de Gruyter; 1967.

27.

Spencer L, Ritchie J, O’Connor W. Analysis: practices, principles and processes. In: Ritchie J, Lewis J, editors. Qualitative research practice. London: SAGE; 2009. p. 199–218.

28.

Ritchie J, Lewis J, Elam G. Designing and selecting samples. In: Ritchie J, Lewis J, editors. Qualitative research practice. London: SAGE; 2009. p. 77–108.

29.

Mathers J, Keeley T, Jones L, Calvert M, Williamson P, Jones J, et al. Using qualitative research to understand what outcomes matter to patients: direct and indirect approaches to outcome elicitation. Trials. 2015;16(Suppl 2);O39.

30.

Keeley T, Khan H, Pinfold V, Williamson P, Mathers J, Davies L, et al. Core outcome sets for use in effectiveness trials involving people with bipolar and schizophrenia in a community-based setting (PARTNERS2): study protocol for the development of two core outcome sets. Trials. 2015;16:47. doi:10.1186/s13063-015-0553-0.CrossRefPubMedPubMedCentral

31.

Waters AM, Tudur Smith C, Young B, Jones TM. The CONSENSUS study: protocol for a mixed methods study to establish which outcomes should be included in a core outcome set for oropharyngeal cancer. Trials. 2014;15:168. doi:10.1186/1745-6215-15-168.CrossRefPubMedPubMedCentral

32.

Harman NL, Bruce IA, Kirkham JJ, Tierney S, Callery P, O’Brien K, et al. The Importance of integration of stakeholder views in core outcome set development: otitis media with effusion in children with cleft palate. PLoS One. 2015;10(6):e0129514. doi:10.1371/journal.pone.0129514.CrossRefPubMedPubMedCentral

33.

NIHR. PARTNERS2: development and pilot trial of primary care based collaborative care for people with serious mental illness. 2014. http://www.nihr.ac.uk/funding/funded-research/funded-research.htm?postid=2207. Accessed 12 Feb 2016.

Title: The use of qualitative methods to inform Delphi surveys in core outcome set development
Authors: T. Keeley
P. Williamson
P. Callery
L. L. Jones
J. Mathers
J. Jones
B. Young
M. Calvert
Publication date: 01-12-2016
Publisher: BioMed Central
Published in: Trials / Issue 1/2016
Electronic ISSN: 1745-6215
DOI: https://doi.org/10.1186/s13063-016-1356-7

Keynote webinar | Spotlight on sleep in brain health

Springer Medicine

The use of qualitative methods to inform Delphi surveys in core outcome set development

Abstract

Background

Results

Conclusions

Keynote webinar | Spotlight on sleep in brain health

Springer Medicine

Abstract

Background

Results

Conclusions

Please log in to get access to this content

Other articles of this Issue 1/2016

Vitamin D intervention in preschoolers with viral-induced asthma (DIVA): a pilot randomised controlled trial

BReastfeeding Attitude and Volume Optimization (BRAVO) trial: study protocol for a randomized controlled trial

European COMPARative Effectiveness research on blended Depression treatment versus treatment-as-usual (E-COMPARED): study protocol for a randomized controlled, non-inferiority trial in eight European countries

Community-based Rehabilitation Intervention for people with Schizophrenia in Ethiopia (RISE): study protocol for a cluster randomised controlled trial

Perioperative hemodynamic optimization using the photoplethysmography in colorectal surgery (the PANEX3 trial): study protocol for a randomized controlled trial

Efficacy of transcranial direct current stimulation (tDCS) in reducing consumption in patients with alcohol use disorders: study protocol for a randomized controlled trial