Skip to main content
Top
Published in: Journal of General Internal Medicine 1/2016

Open Access 01-01-2016 | Original Research

Patient and Stakeholder Engagement in the PCORI Pilot Projects: Description and Lessons Learned

Authors: Laura P. Forsythe, PhD, MPH, Lauren E. Ellis, MA, Lauren Edmundson, BA, Raj Sabharwal, MPH, Alison Rein, MS, Kristen Konopka, MPH, Lori Frank, PHD

Published in: Journal of General Internal Medicine | Issue 1/2016

Login to get access

ABSTRACT

BACKGROUND

Patients and healthcare stakeholders are increasingly becoming engaged in the planning and conduct of biomedical research. However, limited research characterizes this process or its impact.

OBJECTIVE

We aimed to characterize patient and stakeholder engagement in the 50 Pilot Projects funded by the Patient-Centered Outcomes Research Institute (PCORI), and identify early contributions and lessons learned.

DESIGN

A self-report instrument was completed by researchers between 6 and 12 months following project initiation.

PARTICIPANTS

Forty-seven principal investigators or their designees (94 % response rate) participated in the study.
MAIN MEASURES
Self-report of types of stakeholders engaged, stages and levels of engagement, facilitators and barriers to engagement, lessons learned, and contributions from engagement were measured.

KEY RESULTS

Most (83 %) reported engaging more than one stakeholder in their project. Among those, the most commonly reported groups were patients (90 %), clinicians (87 %), health system representatives (44 %), caregivers (41 %), and advocacy organizations (41 %). Stakeholders were commonly involved in topic solicitation, question development, study design, and data collection. Many projects engaged stakeholders in data analysis, results interpretation, and dissemination. Commonly reported contributions included changes to project methods, outcomes or goals; improvement of measurement tools; and interpretation of qualitative data. Investigators often identified communication and shared leadership strategies as “critically important” facilitators (53 and 44 % respectively); lack of stakeholder time was the most commonly reported challenge (46 %). Most challenges were only partially resolved. Early lessons learned included the importance of continuous and genuine partnerships, strategic selection of stakeholders, and accommodation of stakeholders’ practical needs.

CONCLUSIONS

PCORI Pilot Projects investigators report engaging a variety of stakeholders across many stages of research, with specific changes to their research attributed to engagement. This study identifies early lessons and barriers that should be addressed to facilitate engagement. While this research suggests potential impact of stakeholder engagement, systematic characterization and evaluation of engagement at multiple stages of research is needed to build the evidence base.
Appendix
Available only for authorised users
Literature
1.
go back to reference Concannon TW, Fuster M, Saunders T, Patel K, Wong JB, Leslie LK, et al. A systematic review of stakeholder engagement in comparative effectiveness and patient-centered outcomes research. J Gen Intern Med. 2014;29(12):1692–701.PubMedPubMedCentralCrossRef Concannon TW, Fuster M, Saunders T, Patel K, Wong JB, Leslie LK, et al. A systematic review of stakeholder engagement in comparative effectiveness and patient-centered outcomes research. J Gen Intern Med. 2014;29(12):1692–701.PubMedPubMedCentralCrossRef
2.
go back to reference Deverka PA, Lavallee DC, Desai PJ, Esmail LC, Ramsey SD, Veenstra DL, et al. Stakeholder participation in comparative effectiveness research: defining a framework for effective engagement. J Comp Eff Res. 2012;1(2):181–94.PubMedPubMedCentralCrossRef Deverka PA, Lavallee DC, Desai PJ, Esmail LC, Ramsey SD, Veenstra DL, et al. Stakeholder participation in comparative effectiveness research: defining a framework for effective engagement. J Comp Eff Res. 2012;1(2):181–94.PubMedPubMedCentralCrossRef
3.
go back to reference Shippee ND, Domecq Garces JP, Prutsky Lopez GJ, Wang Z, Elraiyah TA, Nabhan M, et al. Patient and service user engagement in research: a systematic review and synthesized framework. Health Expect. 2013. Shippee ND, Domecq Garces JP, Prutsky Lopez GJ, Wang Z, Elraiyah TA, Nabhan M, et al. Patient and service user engagement in research: a systematic review and synthesized framework. Health Expect. 2013.
5.
go back to reference Mullins CD, Abdulhalim AM, Lavallee DC. Continuous patient engagement in comparative effectiveness research. JAMA. 2012;307(15):1587–8.PubMedCrossRef Mullins CD, Abdulhalim AM, Lavallee DC. Continuous patient engagement in comparative effectiveness research. JAMA. 2012;307(15):1587–8.PubMedCrossRef
6.
go back to reference Hanley B, Bradburn J, Barnes M, Evans C, Goodare H, Kelson M, et al. Involving the public in NHS public health, and social care research: briefing notes for researchers. Eastleigh: INVOLVE; 2004. Hanley B, Bradburn J, Barnes M, Evans C, Goodare H, Kelson M, et al. Involving the public in NHS public health, and social care research: briefing notes for researchers. Eastleigh: INVOLVE; 2004.
7.
go back to reference Macaulay AC, Commanda LE, Freeman WL, Gibson N, McCabe ML, Robbins CM, et al. Participatory research maximises community and lay involvement. North American primary care research group. BMJ. 1999;319(7212):774–8.PubMedPubMedCentralCrossRef Macaulay AC, Commanda LE, Freeman WL, Gibson N, McCabe ML, Robbins CM, et al. Participatory research maximises community and lay involvement. North American primary care research group. BMJ. 1999;319(7212):774–8.PubMedPubMedCentralCrossRef
8.
go back to reference Green LW, Mercer SL. Can public health researchers and agencies reconcile the push from funding bodies and the pull from communities? Am J Public Health. 2001;91(12):1926–9.PubMedPubMedCentralCrossRef Green LW, Mercer SL. Can public health researchers and agencies reconcile the push from funding bodies and the pull from communities? Am J Public Health. 2001;91(12):1926–9.PubMedPubMedCentralCrossRef
11.
go back to reference Viswanathan M, Ammerman A, Eng E, Garlehner G, Lohr KN, Griffith D, et al. Community-based participatory research: assessing the evidence. Evid Rep Technol Assess (Summ). 2004(99):1–8. Viswanathan M, Ammerman A, Eng E, Garlehner G, Lohr KN, Griffith D, et al. Community-based participatory research: assessing the evidence. Evid Rep Technol Assess (Summ). 2004(99):1–8.
12.
go back to reference Whitlock EP, Lopez SA, Chang S, Helfand M, Eder M, Floyd N. AHRQ series paper 3: identifying, selecting, and refining topics for comparative effectiveness systematic reviews: AHRQ and the effective health-care program. J Clin Epidemiol. 2010;63(5):491–501.PubMedCrossRef Whitlock EP, Lopez SA, Chang S, Helfand M, Eder M, Floyd N. AHRQ series paper 3: identifying, selecting, and refining topics for comparative effectiveness systematic reviews: AHRQ and the effective health-care program. J Clin Epidemiol. 2010;63(5):491–501.PubMedCrossRef
16.
go back to reference Patient Protection and Affordable Care Act of 2010, Pub. L. no. 111–148, 124 Stat. 727, Sect. 6301. Patient Protection and Affordable Care Act of 2010, Pub. L. no. 111–148, 124 Stat. 727, Sect. 6301.
17.
go back to reference Frank L, Basch E, Selby JV. Patient-centered outcomes research I. The PCORI perspective on patient-centered outcomes research. JAMA. 2014;312(15):1513–4.PubMedCrossRef Frank L, Basch E, Selby JV. Patient-centered outcomes research I. The PCORI perspective on patient-centered outcomes research. JAMA. 2014;312(15):1513–4.PubMedCrossRef
18.
go back to reference Forsythe LP, Szydlowski V, Murad MH, Ip S, Wang Z, Elraiyah TA, et al. A systematic review of approaches for engaging patients for research on rare diseases. J Gen Intern Med. 2014;29(Suppl 3):S788–800.PubMedCrossRef Forsythe LP, Szydlowski V, Murad MH, Ip S, Wang Z, Elraiyah TA, et al. A systematic review of approaches for engaging patients for research on rare diseases. J Gen Intern Med. 2014;29(Suppl 3):S788–800.PubMedCrossRef
19.
go back to reference Boote J, Baird W, Sutton A. Public involvement in the design and conduct of clinical trials: a narrative review of case examples. Trials. 2011;12(Suppl 1):A82.PubMedCentralCrossRef Boote J, Baird W, Sutton A. Public involvement in the design and conduct of clinical trials: a narrative review of case examples. Trials. 2011;12(Suppl 1):A82.PubMedCentralCrossRef
20.
go back to reference Boote J, Baird W, Sutton A. Involving the public in systematic reviews: a narrative review of organizational approaches and eight case examples. J Comp Eff Res. 2012;1(5):409–20.PubMedCrossRef Boote J, Baird W, Sutton A. Involving the public in systematic reviews: a narrative review of organizational approaches and eight case examples. J Comp Eff Res. 2012;1(5):409–20.PubMedCrossRef
21.
go back to reference Brett J, Staniszewska S, Mockford C, Herron-Marx S, Hughes J, Tysall C, et al. Mapping the impact of patient and public involvement on health and social care research: a systematic review. Health Expect. 2014;17(5):637–50.PubMedCrossRef Brett J, Staniszewska S, Mockford C, Herron-Marx S, Hughes J, Tysall C, et al. Mapping the impact of patient and public involvement on health and social care research: a systematic review. Health Expect. 2014;17(5):637–50.PubMedCrossRef
22.
go back to reference Staley K. Exploring impact: public involvement in NHS, public health, and social care research. Eastleigh: INVOLVE; 2009. Staley K. Exploring impact: public involvement in NHS, public health, and social care research. Eastleigh: INVOLVE; 2009.
23.
go back to reference Staniszewska S, Brett J, Mockford C, Barber R. The GRIPP checklist: strengthening the quality of patient and public involvement reporting in research. Int J Technol Assess Health Care. 2011;27(4):391–9.PubMedCrossRef Staniszewska S, Brett J, Mockford C, Barber R. The GRIPP checklist: strengthening the quality of patient and public involvement reporting in research. Int J Technol Assess Health Care. 2011;27(4):391–9.PubMedCrossRef
24.
go back to reference Workman T, Maurer M, Carman K. Unresolved tensions in consumer engagement in CER: a US research perspective. J Comp Eff Res. 2013;2(2):127–34.PubMedCrossRef Workman T, Maurer M, Carman K. Unresolved tensions in consumer engagement in CER: a US research perspective. J Comp Eff Res. 2013;2(2):127–34.PubMedCrossRef
25.
go back to reference Esmail L, Moore E, Rein A. Evaluating patient and stakeholder engagement in research: moving from theory to practice. J Comp Eff Res. 2015;4(2):133–45.PubMedCrossRef Esmail L, Moore E, Rein A. Evaluating patient and stakeholder engagement in research: moving from theory to practice. J Comp Eff Res. 2015;4(2):133–45.PubMedCrossRef
26.
go back to reference Barber R, Boote JD, Cooper CL. Involving consumers successfully in NHS research: a national survey. Health Expect. 2007;10(4):380–91.PubMedCrossRef Barber R, Boote JD, Cooper CL. Involving consumers successfully in NHS research: a national survey. Health Expect. 2007;10(4):380–91.PubMedCrossRef
27.
go back to reference Hanley B, Truesdale A, King A, Elbourne D, Chalmers I. Involving consumers in designing, conducting, and interpreting randomised controlled trials: questionnaire survey. BMJ. 2001;322(7285):519–23.PubMedPubMedCentralCrossRef Hanley B, Truesdale A, King A, Elbourne D, Chalmers I. Involving consumers in designing, conducting, and interpreting randomised controlled trials: questionnaire survey. BMJ. 2001;322(7285):519–23.PubMedPubMedCentralCrossRef
28.
go back to reference Qualtrics Software, Version 57764. Provo, UT: Qualtrics Research Suite; 2014. Qualtrics Software, Version 57764. Provo, UT: Qualtrics Research Suite; 2014.
29.
go back to reference Frank L, Forsythe L, Ellis L, Schrandt S, Sheridan S, Gerson J, et al. Conceptual and practical foundations of patient engagement in research at the patient-centered outcomes research institute. Qual Life Res. 2015;24(5):1033–41.PubMedPubMedCentralCrossRef Frank L, Forsythe L, Ellis L, Schrandt S, Sheridan S, Gerson J, et al. Conceptual and practical foundations of patient engagement in research at the patient-centered outcomes research institute. Qual Life Res. 2015;24(5):1033–41.PubMedPubMedCentralCrossRef
30.
go back to reference Mitton C, Smith N, Peacock S, Evoy B, Abelson J. Public participation in health care priority setting: a scoping review. Health Policy. 2009;91(3):219–28.PubMedCrossRef Mitton C, Smith N, Peacock S, Evoy B, Abelson J. Public participation in health care priority setting: a scoping review. Health Policy. 2009;91(3):219–28.PubMedCrossRef
31.
go back to reference Mallery C, Ganachari D, Fernandez J, Smeeding L, Robinson S, Moon M, et al. Innovative Methods in Stakeholder Engagement: An Environmental Scan. Prepared by the American Institutes for Research under contract No. HHSA 290 2010 0005 C. AHRQ Publication NO. 12-EHC097-EF. Rockville, MD2012. Mallery C, Ganachari D, Fernandez J, Smeeding L, Robinson S, Moon M, et al. Innovative Methods in Stakeholder Engagement: An Environmental Scan. Prepared by the American Institutes for Research under contract No. HHSA 290 2010 0005 C. AHRQ Publication NO. 12-EHC097-EF. Rockville, MD2012.
32.
go back to reference McKenzie A, Hanley B. Consumer and Community Participation in Health and Medical Research: A Practical Guide for Health and Medical Research Organizations. Australia The University of Western Australia and The Telethon Institute for Child Research; 2009. McKenzie A, Hanley B. Consumer and Community Participation in Health and Medical Research: A Practical Guide for Health and Medical Research Organizations. Australia The University of Western Australia and The Telethon Institute for Child Research; 2009.
33.
go back to reference Minogue V, Girdlestone J. Building capacity for service user and carer involvement in research: the implications and impact of best research for best health. Int J Health Care Qual Assur. 2010;23(4):422–35.PubMedCrossRef Minogue V, Girdlestone J. Building capacity for service user and carer involvement in research: the implications and impact of best research for best health. Int J Health Care Qual Assur. 2010;23(4):422–35.PubMedCrossRef
34.
go back to reference Lopez MH, Holve E, Rein A, Winkler J. Involving Patients and Consumers in Research: New Opportunites for Meaningful Engagement in Research and Quality Improvement. EDM Forum, AcademyHealth. June 2012. Lopez MH, Holve E, Rein A, Winkler J. Involving Patients and Consumers in Research: New Opportunites for Meaningful Engagement in Research and Quality Improvement. EDM Forum, AcademyHealth. June 2012.
35.
go back to reference Saunders C, Crossing S, Girgis A, Butow P, Penman A. Operationalising a model framework for consumer and community participation in health and medical research. Aust New Zealand Health Policy. 2007;4:13.PubMedPubMedCentralCrossRef Saunders C, Crossing S, Girgis A, Butow P, Penman A. Operationalising a model framework for consumer and community participation in health and medical research. Aust New Zealand Health Policy. 2007;4:13.PubMedPubMedCentralCrossRef
36.
go back to reference Stacciarini JM, Shattell MM, Coady M, Wiens B. Review: community-based participatory research approach to address mental health in minority populations. Community Ment Health J. 2011;47(5):489–97.PubMedCrossRef Stacciarini JM, Shattell MM, Coady M, Wiens B. Review: community-based participatory research approach to address mental health in minority populations. Community Ment Health J. 2011;47(5):489–97.PubMedCrossRef
37.
go back to reference Stewart RJ, Caird J, Oliver K, Oliver S. Patients’ and clinicians’ research priorities. Health Expect. 2011;14(4):439–48.PubMedCrossRef Stewart RJ, Caird J, Oliver K, Oliver S. Patients’ and clinicians’ research priorities. Health Expect. 2011;14(4):439–48.PubMedCrossRef
38.
go back to reference Curtis P, Slaughter-Mason S, Thielke A, Gordon C, Pettinari C, Ryan K, et al. PCORI Expert Interviews Project: Final Report. Portland, OR 2012. Curtis P, Slaughter-Mason S, Thielke A, Gordon C, Pettinari C, Ryan K, et al. PCORI Expert Interviews Project: Final Report. Portland, OR 2012.
39.
go back to reference Forsythe LP, Frank L, Walker KO, Anise A, Wegener N, Weisman H, et al. Patient and Clinician Views on Comparative Effectiveness Research and Engagement in Research. J Comp Eff Res. in press. Forsythe LP, Frank L, Walker KO, Anise A, Wegener N, Weisman H, et al. Patient and Clinician Views on Comparative Effectiveness Research and Engagement in Research. J Comp Eff Res. in press.
40.
go back to reference Oliver S, Clarke-Jones L, Rees R, Milne R, Buchanan P, Gabbay J, et al. Involving consumers in research and development agenda setting for the NHS: developing an evidence-based approach. Health Technol Assess. 2004;8(15):1–148. 15.PubMedCrossRef Oliver S, Clarke-Jones L, Rees R, Milne R, Buchanan P, Gabbay J, et al. Involving consumers in research and development agenda setting for the NHS: developing an evidence-based approach. Health Technol Assess. 2004;8(15):1–148. 15.PubMedCrossRef
41.
go back to reference Westfall JM, Fagnan LJ, Handley M, Salsberg J, McGinnis P, Zittleman LK, et al. Practice-based research is community engagement. J Am Board Fam Med. 2009;22(4):423–7.PubMedCrossRef Westfall JM, Fagnan LJ, Handley M, Salsberg J, McGinnis P, Zittleman LK, et al. Practice-based research is community engagement. J Am Board Fam Med. 2009;22(4):423–7.PubMedCrossRef
42.
go back to reference Caron-Flinterman JF, Broerse JE, Bunders JF. The experiential knowledge of patients: a new resource for biomedical research? Soc Sci Med. 2005;60(11):2575–84.PubMedCrossRef Caron-Flinterman JF, Broerse JE, Bunders JF. The experiential knowledge of patients: a new resource for biomedical research? Soc Sci Med. 2005;60(11):2575–84.PubMedCrossRef
43.
go back to reference Domecq JP, Prutsky G, Elraiyah T, Wang Z, Nabhan M, Shippee N, et al. Patient engagement in research: a systematic review. BMC Health Serv Res. 2014;14:89.PubMedPubMedCentralCrossRef Domecq JP, Prutsky G, Elraiyah T, Wang Z, Nabhan M, Shippee N, et al. Patient engagement in research: a systematic review. BMC Health Serv Res. 2014;14:89.PubMedPubMedCentralCrossRef
45.
go back to reference Nilsen ES, Myrhaug HT, Johansen M, Oliver S, Oxman AD. Methods of consumer involvement in developing healthcare policy and research, clinical practice guidelines and patient information material. Cochrane Database Syst Rev. 2006;3:CD004563.PubMed Nilsen ES, Myrhaug HT, Johansen M, Oliver S, Oxman AD. Methods of consumer involvement in developing healthcare policy and research, clinical practice guidelines and patient information material. Cochrane Database Syst Rev. 2006;3:CD004563.PubMed
46.
go back to reference Staniszewska S, Jones N, Newburn M, Marshall S. User involvement in the development of a research bid: barriers, enablers and impacts. Health Expect. 2007;10(2):173–83.PubMedCrossRef Staniszewska S, Jones N, Newburn M, Marshall S. User involvement in the development of a research bid: barriers, enablers and impacts. Health Expect. 2007;10(2):173–83.PubMedCrossRef
47.
go back to reference Lophatananon A, Tyndale-Biscoe S, Malcolm E, Rippon HJ, Holmes K, Firkins LA, et al. The James Lind alliance approach to priority setting for prostate cancer research: an integrative methodology based on patient and clinician participation. BJU Int. 2011;108(7):1040–3.PubMedCrossRef Lophatananon A, Tyndale-Biscoe S, Malcolm E, Rippon HJ, Holmes K, Firkins LA, et al. The James Lind alliance approach to priority setting for prostate cancer research: an integrative methodology based on patient and clinician participation. BJU Int. 2011;108(7):1040–3.PubMedCrossRef
48.
go back to reference Lindenmeyer A, Hearnshaw H, Sturt J, Ormerod R, Aitchison G. Assessment of the benefits of user involvement in health research from the Warwick diabetes care research user group: a qualitative case study. Health Expect. 2007;10(3):268–77.PubMedCrossRef Lindenmeyer A, Hearnshaw H, Sturt J, Ormerod R, Aitchison G. Assessment of the benefits of user involvement in health research from the Warwick diabetes care research user group: a qualitative case study. Health Expect. 2007;10(3):268–77.PubMedCrossRef
49.
go back to reference Edwards V, Wyatt K, Logan S, Britten N. Consulting parents about the design of a randomized controlled trial of osteopathy for children with cerebral palsy. Health Expect. 2011;14(4):429–38.PubMedCrossRef Edwards V, Wyatt K, Logan S, Britten N. Consulting parents about the design of a randomized controlled trial of osteopathy for children with cerebral palsy. Health Expect. 2011;14(4):429–38.PubMedCrossRef
50.
go back to reference Terry SF, Terry PF, Rauen KA, Uitto J, Bercovitch LG. Advocacy groups as research organizations: the PXE international example. Nat Rev Genet. 2007;8(2):157–64.PubMedCrossRef Terry SF, Terry PF, Rauen KA, Uitto J, Bercovitch LG. Advocacy groups as research organizations: the PXE international example. Nat Rev Genet. 2007;8(2):157–64.PubMedCrossRef
51.
go back to reference Barber R, Beresford P, Boote J, Cooper C, Faulkner A. Evaluating the impact of service user involvement on research: a prospective case study. Int J Consum Stud. 2011;35(6):609–15.CrossRef Barber R, Beresford P, Boote J, Cooper C, Faulkner A. Evaluating the impact of service user involvement on research: a prospective case study. Int J Consum Stud. 2011;35(6):609–15.CrossRef
52.
go back to reference Fern LA, Taylor RM, Whelan J, Pearce S, Grew T, Brooman K, et al. The art of age-appropriate care: reflecting on a conceptual model of the cancer experience for teenagers and young adults. Cancer Nurs. 2013;36(5):E27–e38.PubMedCrossRef Fern LA, Taylor RM, Whelan J, Pearce S, Grew T, Brooman K, et al. The art of age-appropriate care: reflecting on a conceptual model of the cancer experience for teenagers and young adults. Cancer Nurs. 2013;36(5):E27–e38.PubMedCrossRef
Metadata
Title
Patient and Stakeholder Engagement in the PCORI Pilot Projects: Description and Lessons Learned
Authors
Laura P. Forsythe, PhD, MPH
Lauren E. Ellis, MA
Lauren Edmundson, BA
Raj Sabharwal, MPH
Alison Rein, MS
Kristen Konopka, MPH
Lori Frank, PHD
Publication date
01-01-2016
Publisher
Springer US
Published in
Journal of General Internal Medicine / Issue 1/2016
Print ISSN: 0884-8734
Electronic ISSN: 1525-1497
DOI
https://doi.org/10.1007/s11606-015-3450-z

Other articles of this Issue 1/2016

Journal of General Internal Medicine 1/2016 Go to the issue

Clinical Practice: Clinical Images

Horner’s, Heterochromia, and Harlequins

Obesity Clinical Trial Summary

At a glance: The STEP trials

A round-up of the STEP phase 3 clinical trials evaluating semaglutide for weight loss in people with overweight or obesity.

Developed by: Springer Medicine

Highlights from the ACC 2024 Congress

Year in Review: Pediatric cardiology

Watch Dr. Anne Marie Valente present the last year's highlights in pediatric and congenital heart disease in the official ACC.24 Year in Review session.

Year in Review: Pulmonary vascular disease

The last year's highlights in pulmonary vascular disease are presented by Dr. Jane Leopold in this official video from ACC.24.

Year in Review: Valvular heart disease

Watch Prof. William Zoghbi present the last year's highlights in valvular heart disease from the official ACC.24 Year in Review session.

Year in Review: Heart failure and cardiomyopathies

Watch this official video from ACC.24. Dr. Biykem Bozkurt discusses last year's major advances in heart failure and cardiomyopathies.