Skip to main content
Top
Published in: BMC Medical Research Methodology 1/2012

Open Access 01-12-2012 | Research article

Do health care institutions value research? A mixed methods study of barriers and facilitators to methodological rigor in pediatric randomized trials

Authors: Michele P Hamm, Shannon D Scott, Terry P Klassen, David Moher, Lisa Hartling

Published in: BMC Medical Research Methodology | Issue 1/2012

Login to get access

Abstract

Background

Pediatric randomized controlled trials (RCTs) are susceptible to a high risk of bias. We examined the barriers and facilitators that pediatric trialists face in the design and conduct of unbiased trials.

Methods

We used a mixed methods design, with semi-structured interviews building upon the results of a quantitative survey. We surveyed Canadian (n=253) and international (n=600) pediatric trialists regarding their knowledge and awareness of bias and their perceived barriers and facilitators in conducting clinical trials. We then interviewed 13 participants from different subspecialties and geographic locations to gain a more detailed description of how their experiences and attitudes towards research interacted with trial design and conduct.

Results

The survey response rate was 23.0% (186/807). 68.1% of respondents agreed that bias is a problem in pediatric RCTs and 72.0% felt that there is sufficient evidence to support changing some aspects of how trials are conducted. Knowledge related to bias was variable, with inconsistent awareness of study design features that may introduce bias into a study. Interview participants highlighted a lack of formal training in research methods, a negative research culture, and the pragmatics of trial conduct as barriers. Facilitators included contact with knowledgeable and supportive colleagues and infrastructure for research.

Conclusions

A lack of awareness of bias and negative attitudes towards research present significant barriers in terms of conducting methodologically rigorous pediatric RCTs. Knowledge translation efforts must focus on these issues to ensure the relevance and validity of trial results.
Appendix
Available only for authorised users
Literature
1.
go back to reference Moss RL, Henry MCW, Dimmitt RA, Rangel S, Geraghty N, Skarsgard ED: The role of prospective randomized clinical trials in pediatric surgery: state of the art?. J Pediatr Surg. 2001, 36: 1182-1186. 10.1053/jpsu.2001.25749.CrossRefPubMed Moss RL, Henry MCW, Dimmitt RA, Rangel S, Geraghty N, Skarsgard ED: The role of prospective randomized clinical trials in pediatric surgery: state of the art?. J Pediatr Surg. 2001, 36: 1182-1186. 10.1053/jpsu.2001.25749.CrossRefPubMed
2.
go back to reference Welk B, Afshar K, MacNeily AE: Randomized controlled trials in pediatric urology: room for improvement. J Urol. 2006, 176: 306-310. 10.1016/S0022-5347(06)00560-X.CrossRefPubMed Welk B, Afshar K, MacNeily AE: Randomized controlled trials in pediatric urology: room for improvement. J Urol. 2006, 176: 306-310. 10.1016/S0022-5347(06)00560-X.CrossRefPubMed
3.
go back to reference Dulai SK, Slobogean BLT, Beauchamp RD, Mulpuri K: A quality assessment of randomized clinical trials in pediatric orthopaedics. J Pediatr Orthop. 2007, 27: 573-581. 10.1097/bpo.0b013e3180621f3e.CrossRefPubMed Dulai SK, Slobogean BLT, Beauchamp RD, Mulpuri K: A quality assessment of randomized clinical trials in pediatric orthopaedics. J Pediatr Orthop. 2007, 27: 573-581. 10.1097/bpo.0b013e3180621f3e.CrossRefPubMed
4.
go back to reference Uman LS, Chambers CT, McGrath PJ, Kisely S, Matthews D, Hayton K: Assessing the quality of randomized controlled trials examining psychological interventions for pediatric procedural pain: recommendations for quality improvement. J Pediatr Psychol. 2010, 35: 693-703. 10.1093/jpepsy/jsp104.CrossRefPubMed Uman LS, Chambers CT, McGrath PJ, Kisely S, Matthews D, Hayton K: Assessing the quality of randomized controlled trials examining psychological interventions for pediatric procedural pain: recommendations for quality improvement. J Pediatr Psychol. 2010, 35: 693-703. 10.1093/jpepsy/jsp104.CrossRefPubMed
5.
go back to reference Nor Aripin KNB, Choonara I, Sammons HM: A systematic review of paediatric randomised controlled drug trials published in 2007. Arch Dis Child. 2010, 95: 469-473. 10.1136/adc.2009.173591.CrossRef Nor Aripin KNB, Choonara I, Sammons HM: A systematic review of paediatric randomised controlled drug trials published in 2007. Arch Dis Child. 2010, 95: 469-473. 10.1136/adc.2009.173591.CrossRef
6.
go back to reference Thomson D, Hartling L, Cohen E, Vandermeer B, Tjosvold L, Klassen TP: Controlled trials in children: quantity, methodological quality and descriptive characteristics of pediatric controlled trials published 1948–2006. PLoS One. 2010, 5: e13106-10.1371/journal.pone.0013106.CrossRefPubMedPubMedCentral Thomson D, Hartling L, Cohen E, Vandermeer B, Tjosvold L, Klassen TP: Controlled trials in children: quantity, methodological quality and descriptive characteristics of pediatric controlled trials published 1948–2006. PLoS One. 2010, 5: e13106-10.1371/journal.pone.0013106.CrossRefPubMedPubMedCentral
7.
go back to reference DeMauro SB, Giaccone A, Kirpalani H, Schmidt B: Quality of reporting of neonatal and infant trials in high-impact journals. Pediatrics. 2011, 128: e639-PubMed DeMauro SB, Giaccone A, Kirpalani H, Schmidt B: Quality of reporting of neonatal and infant trials in high-impact journals. Pediatrics. 2011, 128: e639-PubMed
8.
go back to reference Hartling L, Ospina M, Liang Y, Dryden DM, Hooton N, Seida JK, Klassen TP: Risk of bias versus quality assessment of randomised controlled trials: cross sectional study. BMJ. 2009, 339: b4012-10.1136/bmj.b4012.CrossRefPubMedPubMedCentral Hartling L, Ospina M, Liang Y, Dryden DM, Hooton N, Seida JK, Klassen TP: Risk of bias versus quality assessment of randomised controlled trials: cross sectional study. BMJ. 2009, 339: b4012-10.1136/bmj.b4012.CrossRefPubMedPubMedCentral
9.
go back to reference Crocetti MT, Amin DD, Scherer R: Assessment of risk of bias among pediatric randomized controlled trials. Pediatrics. 2010, 126: 298-305. 10.1542/peds.2009-3121.CrossRefPubMed Crocetti MT, Amin DD, Scherer R: Assessment of risk of bias among pediatric randomized controlled trials. Pediatrics. 2010, 126: 298-305. 10.1542/peds.2009-3121.CrossRefPubMed
10.
go back to reference Hamm MP, Hartling L, Milne A, Tjosvold L, Vandermeer B, Thomson D, Curtis S, Klassen TP: A descriptive analysis of a representative sample of pediatric randomized controlled trials published in 2007. BMC Pediatr. 2010, 10: 96-10.1186/1471-2431-10-96.CrossRefPubMedPubMedCentral Hamm MP, Hartling L, Milne A, Tjosvold L, Vandermeer B, Thomson D, Curtis S, Klassen TP: A descriptive analysis of a representative sample of pediatric randomized controlled trials published in 2007. BMC Pediatr. 2010, 10: 96-10.1186/1471-2431-10-96.CrossRefPubMedPubMedCentral
11.
go back to reference Chalmers I, Glasziou P: Avoidable waste in the production and reporting of research evidence. Lancet. 2009, 374: 86-89. 10.1016/S0140-6736(09)60329-9.CrossRefPubMed Chalmers I, Glasziou P: Avoidable waste in the production and reporting of research evidence. Lancet. 2009, 374: 86-89. 10.1016/S0140-6736(09)60329-9.CrossRefPubMed
13.
go back to reference Als-Nielsen B, Gluud LL, Gluud C: Methodological quality and treatment effects in randomised trials: a review of six empirical studies. 12th Cochrane colloquium. 2004, Ottawa, Ontario, Canada: The Cochrane Collaboration, Oct 2–6 Als-Nielsen B, Gluud LL, Gluud C: Methodological quality and treatment effects in randomised trials: a review of six empirical studies. 12th Cochrane colloquium. 2004, Ottawa, Ontario, Canada: The Cochrane Collaboration, Oct 2–6
14.
go back to reference Pildal J, Hrobjartsson A, Jorgensen KJ, Hilden J, Altman DG, Gotzsche PC: Impact of allocation concealment on conclusions drawn from meta-analyses of randomized trials. Int J Epidemiol. 2007, 36: 847-857. 10.1093/ije/dym087.CrossRefPubMed Pildal J, Hrobjartsson A, Jorgensen KJ, Hilden J, Altman DG, Gotzsche PC: Impact of allocation concealment on conclusions drawn from meta-analyses of randomized trials. Int J Epidemiol. 2007, 36: 847-857. 10.1093/ije/dym087.CrossRefPubMed
15.
go back to reference Abraha I, Duca PG, Montedori A: Empirical evidence of bias: modified intention to treat analysis of randomised trials affects estimates of intervention efficacy. Z Evid Fortbild Qual Gesundhwes. 2008, 102 (Suppl VI): 9- Abraha I, Duca PG, Montedori A: Empirical evidence of bias: modified intention to treat analysis of randomised trials affects estimates of intervention efficacy. Z Evid Fortbild Qual Gesundhwes. 2008, 102 (Suppl VI): 9-
16.
go back to reference Von Elm E, Rollin A, Blumle A, Senessie C, Low N, Egger M: Selective reporting of outcomes of drug trials? comparison of study protocols and published articles. 14th Cochrane colloquium. 2006, Dublin, Ireland: The Cochrane Collaboration, Oct 23–26 Von Elm E, Rollin A, Blumle A, Senessie C, Low N, Egger M: Selective reporting of outcomes of drug trials? comparison of study protocols and published articles. 14th Cochrane colloquium. 2006, Dublin, Ireland: The Cochrane Collaboration, Oct 23–26
17.
go back to reference Dwan K, Altman DG, Amaiz JA, Bloom J, Chan AW, Cronin E, Decullier E, Easterbrook PJ, Von Elm E, Gamble C, Ghersi D, Ioannidis JP, Simes J, Williamson PR: Systematic review of the empirical evidence of study publication bias and outcome reporting bias. PLoS One. 2008, 3: e3081-10.1371/journal.pone.0003081.CrossRefPubMedPubMedCentral Dwan K, Altman DG, Amaiz JA, Bloom J, Chan AW, Cronin E, Decullier E, Easterbrook PJ, Von Elm E, Gamble C, Ghersi D, Ioannidis JP, Simes J, Williamson PR: Systematic review of the empirical evidence of study publication bias and outcome reporting bias. PLoS One. 2008, 3: e3081-10.1371/journal.pone.0003081.CrossRefPubMedPubMedCentral
18.
go back to reference Bassler D, Ferreira-Gonzalez I, Briel M, Cook DJ, Devereaux PJ, Heels-Ansdell D, Kirpalani H, Meade MO, Montori VM, Rozenberg A, Schunemann HJ, Guyatt GH: Systematic reviewers neglect bias that results from trials stopped early for benefit. J Clin Epidemiol. 2007, 60: 869-873. 10.1016/j.jclinepi.2006.12.006.CrossRefPubMed Bassler D, Ferreira-Gonzalez I, Briel M, Cook DJ, Devereaux PJ, Heels-Ansdell D, Kirpalani H, Meade MO, Montori VM, Rozenberg A, Schunemann HJ, Guyatt GH: Systematic reviewers neglect bias that results from trials stopped early for benefit. J Clin Epidemiol. 2007, 60: 869-873. 10.1016/j.jclinepi.2006.12.006.CrossRefPubMed
19.
go back to reference Montori VM, Devereaux PJ, Adhikari NK, Burns KE, Eggert CH, Briel M, Lacchetti C, Leung TW, Darling E, Bryant DM, Bucher HC, Schunemann HJ, Meade MO, Cook DJ, Erwin PJ, Sood A, Sood R, Lo B, Thompson CA, Zhou Q, Mills E, Guyatt GH: Randomized trials stopped early for benefit: a systematic review. JAMA. 2005, 294: 2203-2209. 10.1001/jama.294.17.2203.CrossRefPubMed Montori VM, Devereaux PJ, Adhikari NK, Burns KE, Eggert CH, Briel M, Lacchetti C, Leung TW, Darling E, Bryant DM, Bucher HC, Schunemann HJ, Meade MO, Cook DJ, Erwin PJ, Sood A, Sood R, Lo B, Thompson CA, Zhou Q, Mills E, Guyatt GH: Randomized trials stopped early for benefit: a systematic review. JAMA. 2005, 294: 2203-2209. 10.1001/jama.294.17.2203.CrossRefPubMed
20.
go back to reference Bekelman JE, Li Y, Gross CP: Scope and impact of financial conflicts of interest in biomedical research: a systematic review. JAMA. 2003, 298: 454-465.CrossRef Bekelman JE, Li Y, Gross CP: Scope and impact of financial conflicts of interest in biomedical research: a systematic review. JAMA. 2003, 298: 454-465.CrossRef
21.
go back to reference Lexchin J, Bero LA, Djulbegovic B, Clark O: Pharmaceutical industry sponsorship and research outcome and quality: systematic review. BMJ. 2003, 326: 1167-1170. 10.1136/bmj.326.7400.1167.CrossRefPubMedPubMedCentral Lexchin J, Bero LA, Djulbegovic B, Clark O: Pharmaceutical industry sponsorship and research outcome and quality: systematic review. BMJ. 2003, 326: 1167-1170. 10.1136/bmj.326.7400.1167.CrossRefPubMedPubMedCentral
22.
go back to reference Sismondo S: Pharmaceutical company funding and its consequences: a qualitative systematic review. Contemp Clin Trials. 2008, 29: 109-113. 10.1016/j.cct.2007.08.001.CrossRefPubMed Sismondo S: Pharmaceutical company funding and its consequences: a qualitative systematic review. Contemp Clin Trials. 2008, 29: 109-113. 10.1016/j.cct.2007.08.001.CrossRefPubMed
23.
go back to reference Caldwell PHY, Butow PN, Craig JC: Pediatricians’ attitudes toward randomized controlled trials involving children. J Pediatr. 2002, 141: 798-803. 10.1067/mpd.2002.129173.CrossRefPubMed Caldwell PHY, Butow PN, Craig JC: Pediatricians’ attitudes toward randomized controlled trials involving children. J Pediatr. 2002, 141: 798-803. 10.1067/mpd.2002.129173.CrossRefPubMed
24.
go back to reference Cohen E, Shaul RZ: Beyond the therapeutic orphan: children and clinical trials. Pediatr Health. 2008, 2: 151-159. 10.2217/17455111.2.2.151.CrossRef Cohen E, Shaul RZ: Beyond the therapeutic orphan: children and clinical trials. Pediatr Health. 2008, 2: 151-159. 10.2217/17455111.2.2.151.CrossRef
25.
go back to reference Rheims S, Cucherat M, Arzimanoglou A, Ryvlin P: Greater response to placebo in children than in adults: a systematic review and meta-analysis in drug-resistant partial epilepsy. PLoS Med. 2008, 5: e166-10.1371/journal.pmed.0050166.CrossRefPubMedPubMedCentral Rheims S, Cucherat M, Arzimanoglou A, Ryvlin P: Greater response to placebo in children than in adults: a systematic review and meta-analysis in drug-resistant partial epilepsy. PLoS Med. 2008, 5: e166-10.1371/journal.pmed.0050166.CrossRefPubMedPubMedCentral
26.
go back to reference Tishler CL, Reiss NS: Pediatric drug-trial recruitment: enticement without coercion. Pediatrics. 2011, 127: 949-954. 10.1542/peds.2010-2585.CrossRefPubMed Tishler CL, Reiss NS: Pediatric drug-trial recruitment: enticement without coercion. Pediatrics. 2011, 127: 949-954. 10.1542/peds.2010-2585.CrossRefPubMed
27.
go back to reference Ballard HO, Shook LA, Desai NS, Anand KJ: Neonatal research and the validity of informed consent obtained in the perinatal period. J Perinatol. 2004, 24: 409-415. 10.1038/sj.jp.7211142.CrossRefPubMed Ballard HO, Shook LA, Desai NS, Anand KJ: Neonatal research and the validity of informed consent obtained in the perinatal period. J Perinatol. 2004, 24: 409-415. 10.1038/sj.jp.7211142.CrossRefPubMed
28.
go back to reference Creswell JW, Plano Clark VL: Designing and conducting mixed methods research. 2011, Los Angeles, CA: Sage Publications, 2 Creswell JW, Plano Clark VL: Designing and conducting mixed methods research. 2011, Los Angeles, CA: Sage Publications, 2
29.
30.
go back to reference Funk SG, Champagne MT, Wiese RA, Tornquist EM: BARRIERS: the barriers to research utilization scale. Appl Nurs Res. 1991, 4: 39-45. 10.1016/S0897-1897(05)80052-7.CrossRefPubMed Funk SG, Champagne MT, Wiese RA, Tornquist EM: BARRIERS: the barriers to research utilization scale. Appl Nurs Res. 1991, 4: 39-45. 10.1016/S0897-1897(05)80052-7.CrossRefPubMed
31.
go back to reference Cabana MD, Rand CS, Powe NR: Why don’t physicians follow clinical practice guidelines? a framework for improvement. JAMA. 1999, 282: 1458-1465. 10.1001/jama.282.15.1458.CrossRefPubMed Cabana MD, Rand CS, Powe NR: Why don’t physicians follow clinical practice guidelines? a framework for improvement. JAMA. 1999, 282: 1458-1465. 10.1001/jama.282.15.1458.CrossRefPubMed
32.
go back to reference Kajermo KN, Boström AM, Thompson DS, Hutchinson AM, Estabrooks CA, Wallin L: The BARRIERS scale – the barriers to research utilization scale: a systematic review. Implement Sci. 2010, 5: 32-10.1186/1748-5908-5-32.CrossRefPubMedPubMedCentral Kajermo KN, Boström AM, Thompson DS, Hutchinson AM, Estabrooks CA, Wallin L: The BARRIERS scale – the barriers to research utilization scale: a systematic review. Implement Sci. 2010, 5: 32-10.1186/1748-5908-5-32.CrossRefPubMedPubMedCentral
33.
go back to reference Guest G, Bunce A, Johnson L: How many interviews are enough? an experiment with data saturation and variability. Field Methods. 2006, 18: 59-82. 10.1177/1525822X05279903.CrossRef Guest G, Bunce A, Johnson L: How many interviews are enough? an experiment with data saturation and variability. Field Methods. 2006, 18: 59-82. 10.1177/1525822X05279903.CrossRef
34.
go back to reference Hsieh HF, Shannon SE: Three approaches to qualitative content analysis. Qual Health Res. 2005, 15: 1277-1288. 10.1177/1049732305276687.CrossRefPubMed Hsieh HF, Shannon SE: Three approaches to qualitative content analysis. Qual Health Res. 2005, 15: 1277-1288. 10.1177/1049732305276687.CrossRefPubMed
35.
go back to reference Morse J, Niehaus L: Mixed methods design: principles and procedures. 2009, Walnut Creek, CA: Left Coast Press Morse J, Niehaus L: Mixed methods design: principles and procedures. 2009, Walnut Creek, CA: Left Coast Press
36.
go back to reference Caldwell PHY, Murphy SB, Butow PN, Craig JC: Clinical trials in children. Lancet. 2004, 364: 803-811. 10.1016/S0140-6736(04)16942-0.CrossRefPubMed Caldwell PHY, Murphy SB, Butow PN, Craig JC: Clinical trials in children. Lancet. 2004, 364: 803-811. 10.1016/S0140-6736(04)16942-0.CrossRefPubMed
37.
go back to reference Unguru Y: The successful integration of research and care: how pediatric oncology became the subspecialty in which research defines the standard of care. Pediatr Blood Cancer. 2011, 56: 1019-1025. 10.1002/pbc.22976.CrossRefPubMed Unguru Y: The successful integration of research and care: how pediatric oncology became the subspecialty in which research defines the standard of care. Pediatr Blood Cancer. 2011, 56: 1019-1025. 10.1002/pbc.22976.CrossRefPubMed
38.
go back to reference Marshall JC, Cook DJ, Canadian Critical Care Trials Group: Investigator-led clinical research consortia: the Canadian critical care trials group. Crit Care Med. 2009, 37 (1 Suppl): S165-S172.CrossRefPubMed Marshall JC, Cook DJ, Canadian Critical Care Trials Group: Investigator-led clinical research consortia: the Canadian critical care trials group. Crit Care Med. 2009, 37 (1 Suppl): S165-S172.CrossRefPubMed
39.
go back to reference Klassen TP, Acworth J, Bialy L, Black K, Chamberlain JM, Cheng N, Dalziel S, Fernandes RM, Fitzpatrick E, Johnson DW, Kuppermann N, Macias CG, Newton M, Osmond MH, Plint A, Valerio P, Waisman Y, PERN: Pediatric emergency research networks: a global initiative in pediatric emergency medicine. Pediatr Emerg Care. 2010, 26: 541-543. 10.1097/PEC.0b013e3181e5bec1.CrossRefPubMed Klassen TP, Acworth J, Bialy L, Black K, Chamberlain JM, Cheng N, Dalziel S, Fernandes RM, Fitzpatrick E, Johnson DW, Kuppermann N, Macias CG, Newton M, Osmond MH, Plint A, Valerio P, Waisman Y, PERN: Pediatric emergency research networks: a global initiative in pediatric emergency medicine. Pediatr Emerg Care. 2010, 26: 541-543. 10.1097/PEC.0b013e3181e5bec1.CrossRefPubMed
40.
go back to reference Hartling L, Wittmeier KDM, van der Lee JH, Klassen TP, Craig JC, Offringa M, StaR Child Health group: StaR child health: developing evidence-based guidance for the design, conduct, and reporting of pediatric trials. Clin Pharmacol Ther. 2011, 90: 727-731. 10.1038/clpt.2011.212.CrossRefPubMed Hartling L, Wittmeier KDM, van der Lee JH, Klassen TP, Craig JC, Offringa M, StaR Child Health group: StaR child health: developing evidence-based guidance for the design, conduct, and reporting of pediatric trials. Clin Pharmacol Ther. 2011, 90: 727-731. 10.1038/clpt.2011.212.CrossRefPubMed
41.
go back to reference Wilson PM, Petticrew M, Calnan M, Nazarteh I: Effects of a financial incentive on health researchers’ response to an online survey: a randomized controlled trial. J Med Internet Res. 2010, 12: e13-10.2196/jmir.1251.CrossRefPubMedPubMedCentral Wilson PM, Petticrew M, Calnan M, Nazarteh I: Effects of a financial incentive on health researchers’ response to an online survey: a randomized controlled trial. J Med Internet Res. 2010, 12: e13-10.2196/jmir.1251.CrossRefPubMedPubMedCentral
42.
go back to reference VanGeest JB, Johnson TP, Welch VL: Methodologies for improving response rates in surveys of physicians. Eval Health Prof. 2007, 30: 303-321. 10.1177/0163278707307899.CrossRefPubMed VanGeest JB, Johnson TP, Welch VL: Methodologies for improving response rates in surveys of physicians. Eval Health Prof. 2007, 30: 303-321. 10.1177/0163278707307899.CrossRefPubMed
Metadata
Title
Do health care institutions value research? A mixed methods study of barriers and facilitators to methodological rigor in pediatric randomized trials
Authors
Michele P Hamm
Shannon D Scott
Terry P Klassen
David Moher
Lisa Hartling
Publication date
01-12-2012
Publisher
BioMed Central
Published in
BMC Medical Research Methodology / Issue 1/2012
Electronic ISSN: 1471-2288
DOI
https://doi.org/10.1186/1471-2288-12-158

Other articles of this Issue 1/2012

BMC Medical Research Methodology 1/2012 Go to the issue