Top

Published in:

Open Access 01-12-2024 | Research

Participatory methods used in the evaluation of medical devices: a comparison of focus groups, interviews, and a survey

Authors: Kas Woudstra, Marcia Tummers, Catharina J. M. Klijn, Lotte Sondag, Floris Schreuder, Rob Reuzel, Maroeska Rovers

Published in: BMC Health Services Research | Issue 1/2024

Abstract

Background

Stakeholder engagement in evaluation of medical devices is crucial for aligning devices with stakeholders’ views, needs, and values. Methods for these engagements have however not been compared to analyse their relative merits for medical device evaluation. Therefore, we systematically compared these three methods in terms of themes, interaction, and time-investment.

Methods

We compared focus groups, interviews, and an online survey in a case-study on minimally invasive endoscopy-guided surgery for patients with intracerebral haemorrhage. The focus groups and interviews featured two rounds, one explorative focussing on individual perspectives, and one interactive focussing on the exchange of perspectives between participants. The comparison between methods was made in terms of number and content of themes, how participants interact, and hours invested by all researchers.

Results

The focus groups generated 34 themes, the interviews 58, and the survey 42. Various improvements for the assessment of the surgical procedure were only discussed in the interviews. In focus groups, participants were inclined to emphasise agreement and support, whereas the interviews consisted of questions and answers. The total time investment for researchers of focus groups was 95 h, of interviews 315 h, and survey 81 h.

Conclusions

Within the context of medical device evaluation, interviews appeared to be the most appropriate method for understanding stakeholder views since they provide a scope and depth of information that is not generated by other methods. Focus groups were useful to rapidly bring views together. Surveys enabled a quick exploration. Researchers should account for these methodological differences and select the method that is suitable for their research aim.

Available only for authorised users

Royal College of Surgeons. Future Surg. 2022 [cited 2022 Jul 14]. https://futureofsurgery.rcseng.ac.uk/#start. Accessed 4 April 2024.

Marcus HJ, Bennett A, Chari A, Day T, Hirst A, Hughes-Hallett A, et al. IDEAL-D framework for device innovation. Ann Surg. 2022;275(1):73–9.CrossRefPubMed

Food and Drug Administration (FDA) Patient Engagement in the Design and Conduct of Medical Device Clinical Studies Guidance for Industry, Food and Drug Administration Staff, and Other Stakeholders. 2022. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/patient-engagement-design-and-conduct-medical-device-clinical-studies. Accessed 4 April 2024.

The European Parliament and the Council of the European Union Regulation (EU) 2017/745 of the European Parliament and of the Council of 5 April 2017 on medical devices, amending Directive 2001/83/EC, Regulation (EC) No 178/2002 and Regulation (EC) No 1223/2009 and repealing Council Directives 90/385/EEC and 93/42/EEC. http://data.europa.eu/eli/reg/2017/745/oj. Accessed 4 April 2024.

Bitkina OV, Kim HK, Park J. Usability and user experience of medical devices: an overview of the current state, analysis methodologies, and future challenges. Int J Ind Ergon. 2020;76(November 2019):102932. Available from: https://doi.org/10.1016/j.ergon.2020.102932.CrossRef

Bitkina OV, Kim HK, Park J. Usability and user experience of medical devices: an overview of the current state, analysis methodologies, and future challenges. Int J Ind Ergon. 2020;76(November 2018):102932. Available from: https://doi.org/10.1016/j.ergon.2020.102932.CrossRef

Hansen NB, Dindler C, Halskov K, Iversen OS, Bossen C, Basballe DA, et al. How participatory designworks: mechanisms and effects. ACM International Conference Proceeding Series; 2019. p. 30–41.

Harder MK, Burford G, Hoover E. What is participation? Design leads the way to a cross-disciplinary framework. Des Issues. 2013;29(4):41–58. Available from: www.magno-design.com.CrossRef

Zakery A, Saremi MS. Knowledge and intellectual capital in internationalizing SMEs, case study in technology-based health companies. J Intellect Cap. 2021;22(2):219–42.CrossRef

10.

Guest G, Namey E, McKenna K. How many focus groups are enough? Building an evidence base for nonprobability sample sizes. Field Methods. 2017;29(1):3–22.CrossRef

11.

van Velthoven MH, Wang W, Wu Q, Li Y, Scherpbier RW, Du X, et al. Comparison of text messaging data collection vs face-to-face interviews for public health surveys: a cluster randomized crossover study of care-seeking for childhood pneumonia and diarrhoea in rural China. J Glob Health. 2018;8(1):010802.

12.

Vadi MG, Malkin MR, Lenart J, Stier GR, Gatling JW, Applegate RL. Comparison of web-based and face-to-face interviews for application to an anesthesiology training program: a pilot study. Int J Med Educ. 2016;7:102–8.CrossRefPubMedPubMedCentral

13.

Woudstra K, Reuzel R, Rovers M, Tummers M. An overview of stakeholders, methods, topics, and challenges in participatory approaches used in the development of medical devices: a scoping review. Int J Health Policy Manag. 2022;(Article in). Available from: https://www.ijhpm.com/article_4317.html.

14.

van Asch CJ, Luitse MJ, Rinkel GJ, van der Tweel I, Algra A, Klijn CJ. Incidence, case fatality, and functional outcome of intracerebral haemorrhage over time, according to age, sex, and ethnic origin: a systematic review and meta-analysis. Lancet Neurol. 2010;9(2):167–76. Available from: https://doi.org/10.1016/S1474-4422(09)70340-0.CrossRefPubMed

15.

Sondag L, Schreuder FHBM, Boogaarts HD, Rovers MM, Vandertop WP, Dammers R, et al. Neurosurgical intervention for supratentorial intracerebral hemorrhage. Ann Neurol. 2020;88(2):239–50. Available from: https://doi.org/10.1002/ana.25732.CrossRefPubMedPubMedCentral

16.

Malterud K, Siersma VD, Guassora AD. Sample size in qualitative interview studies: guided by information power. Qual Health Res. 2016;26(13):1753–60.CrossRefPubMed

17.

Carlsen B, Glenton C. What about N? A methodological study of sample-size reporting in focus group studies. BMC Med Res Methodol. 2011;11(26):1–10.

18.

Boddy CR. Sample size for qualitative interviews. Qual Mark Res Int J. 2003;2015(19):426–32.

19.

The European Parliament and The Council of The European Union. Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation) (Text with EEA relevance). http://data.europa.eu/eli/reg/2016/679/oj. Accessed 4 April 2024.

20.

Greenspan SB, Gordon KL, Whitcomb SA, Lauterbach AA. Use of video conferencing to facilitate focus groups for qualitative data collection. Am J Qual Res. 2021;5(1):85–93.CrossRef

21.

Keemink JR, Sharp RJ, Dargan AK, Forder JE. Reflections on the use of synchronous online focus groups in social care research. Int J Qual Methods. 2022;1:21.

22.

Thunberg S, Arnell L. Pioneering the use of technologies in qualitative research–a research review of the use of digital interviews. Int J Soc Res Methodol. 2022;25(6):757–68.CrossRef

23.

Reuzel R. Interactive technology assessment of paediatric cochlear implantation. Poiesis Praxis. 2004;2(2–3):119–37.CrossRef

24.

Guba EG, Lincoln YS. Fourth generation evaluation. Newbury Park: SAGE Publications Inc.; 1992.

25.

Moret-Hartman M, Reuzel R, Grin J, Kramers C, van der Wilt GJ. Strengthening evaluation through interactive problem structuring: a case study of hospital care after attempted suicide. Evaluation. 2011;17(1):37–52.CrossRef

26.

Tran VT, Porcher R, Tran VC, Ravaud P. Predicting data saturation in qualitative surveys with mathematical models from ecological research. J Clin Epidemiol. 2017;82:71–78.e2. Available from: https://www.sciencedirect.com/science/article/pii/S0895435616305431.CrossRefPubMed

27.

Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. 2006;3(2):77–101.CrossRef

28.

Morgan DL, Hoffman K. A system for coding the interaction in focus groups and dyadic interviews. Qual Rep. 2018;23(3):519–31.

29.

Keyton J. Interaction Process Analysis (IPA). In: Brauner E, Boos M, Kolbe M, editors. The Cambridge Handbook of Group Interaction Analysis. Cambridge: Cambridge University Press; 2018. p. 441–50.CrossRef

30.

Dearnley C. A reflection on the use of semi-structured interviews. Nurse Res. 2005;13(1):19–28.CrossRefPubMed

31.

Stuckey H. The first step in data analysis: transcribing and managing qualitative research data. J Soc Health Diabetes. 2014;02(01):6–8.CrossRef

32.

Woolley K, Edwards KL, Glazebrook C. Focus group or individual interviews for exploring children’s health behaviour: the example of physical activity. Adv Pediatr Res. 2018;05(02):1–10.CrossRef

33.

Rat AC, Pouchot J, Guillemin F, Baumann M, Retel-Rude N, Spitz E, et al. Content of quality-of-life instruments is affected by item-generation methods. Int J Qual Health Care. 2007;19(6):390–8.CrossRefPubMed

34.

Fern EF. The use of focus groups for idea generation: the effects of group size, acquaintanceship, and moderator on response quantity and quality. J Mark Res. 1982;19(1):1.CrossRef

35.

Namey E, Guest G, McKenna K, Chen M. Evaluating bang for the buck: a cost-effectiveness comparison between individual interviews and focus groups based on thematic saturation levels. Am J Eval. 2016;37(3):425–40.CrossRef

36.

Griffin A, Hauser JR. The voice of the customer. Mark Sci. 1993;12(1):1–27.CrossRef

37.

Coenen M, Stamm TA, Stucki G, Cieza A. Individual interviews and focus groups in patients with rheumatoid arthritis: a comparison of two qualitative methods. Qual Life Res. 2012;21(2):359–70.CrossRefPubMed

38.

Anggraeni M, Gupta J, Verrest HJLM. Cost and value of stakeholders participation: a systematic literature review. Environ Sci Policy. 2019;101:364–73.

Title: Participatory methods used in the evaluation of medical devices: a comparison of focus groups, interviews, and a survey
Authors: Kas Woudstra
Marcia Tummers
Catharina J. M. Klijn
Lotte Sondag
Floris Schreuder
Rob Reuzel
Maroeska Rovers
Publication date: 01-12-2024
Publisher: BioMed Central
Published in: BMC Health Services Research / Issue 1/2024
Electronic ISSN: 1472-6963
DOI: https://doi.org/10.1186/s12913-024-10887-3

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Participatory methods used in the evaluation of medical devices: a comparison of focus groups, interviews, and a survey

Abstract

Background

Methods

Results

Conclusions

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Abstract

Background

Methods

Results

Conclusions

Please log in to get access to this content

Other articles of this Issue 1/2024

Development of the Conversational Health Literacy Assessment Tool for maternity care (CHAT-maternity-care): participatory action research

The development of a novel sexual health promotion intervention for young people with mental ill-health: the PROSPEct project

Diagnostic evaluation of institutions as a basis for designing the Brazilian maturity model of telehealth services

Implementation of the Dementia Isolation Toolkit in long-term care improves awareness but does not reduce moral distress amongst healthcare providers

Long-term care staffs’ experience in facilitating the use of videoconferencing by cognitively impaired long-term care residents during the COVID-19 pandemic: a mixed-methods study

Rehabilitation in primary care for an ageing population: a secondary analysis from a scoping review of rehabilitation delivery models