Skip to main content
Top
Published in: Trials 1/2018

Open Access 01-12-2018 | Methodology

Are randomised controlled trials positivist? Reviewing the social science and philosophy literature to assess positivist tendencies of trials of social interventions in public health and health services

Authors: Chris Bonell, Graham Moore, Emily Warren, Laurence Moore

Published in: Trials | Issue 1/2018

Login to get access

Abstract

Background

We have previously proposed that trials of social interventions can be done within a “realist” research paradigm. Critics have countered that such trials are irredeemably positivist and asked us to explain our philosophical position.

Methods

We set out to explore what is meant by positivism and whether trials adhere to its tenets (of necessity or in practice) via a narrative literature review of social science and philosophical discussions of positivism, and of the trials literature and three case studies of trials.

Results

The philosophical literature described positivism as asserting: (1) the epistemic primacy of sensory information; (2) the requirement that theoretical terms equate with empirical terms; (3) the aim of developing universal laws; and (4) the unity of method between natural and social sciences. Regarding (1), it seems that rather than embodying the epistemic primacy of sensory data, randomised controlled trials (RCTs) of social interventions in health embrace an anti-positivist approach aiming to test hypotheses derived deductively from prior theory. Considering (2), while some RCTs of social interventions appear to limit theorisation to concepts with empirical analogues, others examine interventions underpinned by theories engaging with mechanisms and contextual contingencies not all of which can be measured. Regarding (3), while some trialists and reviewers in the health field do limit their role to estimating statistical trends as a mechanistic form of generalisation, this is not an inevitable feature of RCT-based research. Trials of social interventions can instead aim to generalise at the level of theory which specifies how mechanisms are contingent on context. In terms of (4), while RCTs are used to examine biomedical as well as social interventions in health, RCTs of social interventions are often distinctive in using qualitative analyses of data on participant accounts to examine questions of meaning and agency not pursued in the natural sciences.

Conclusion

We conclude that the most appropriate paradigm for RCTs of social interventions is realism not positivism.
Literature
1.
go back to reference Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M. Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ. 2008;337:a1655.CrossRefPubMedPubMedCentral Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M. Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ. 2008;337:a1655.CrossRefPubMedPubMedCentral
2.
go back to reference Weinberger M, Oddone EZ, Henderson WG, Smith DM, Huey J, Giobbie-Hurder A, Feussner JR. Multisite randomized controlled trials in health services research: scientific challenges and operational issues. Med Care. 2001;39(6):627–34.CrossRefPubMed Weinberger M, Oddone EZ, Henderson WG, Smith DM, Huey J, Giobbie-Hurder A, Feussner JR. Multisite randomized controlled trials in health services research: scientific challenges and operational issues. Med Care. 2001;39(6):627–34.CrossRefPubMed
3.
go back to reference Banerjee AV, Duflo E. The experimental approach to development economics. Annual Review of Economics. 2009;1:151–78.CrossRef Banerjee AV, Duflo E. The experimental approach to development economics. Annual Review of Economics. 2009;1:151–78.CrossRef
4.
go back to reference Torgerson CJ, Torgerson DJ, Birks YF, Porthouse J. A comparison of randomised controlled trials in health and education. Br Educ Res J. 2005;31:761–85.CrossRef Torgerson CJ, Torgerson DJ, Birks YF, Porthouse J. A comparison of randomised controlled trials in health and education. Br Educ Res J. 2005;31:761–85.CrossRef
5.
go back to reference Campbell DT. Methods for the experimenting society. Am J Eval. 1991;12(3):223–60.CrossRef Campbell DT. Methods for the experimenting society. Am J Eval. 1991;12(3):223–60.CrossRef
6.
go back to reference Popper K. The poverty of historicism. London: Routledge & Kegan Paul; 1957. Popper K. The poverty of historicism. London: Routledge & Kegan Paul; 1957.
7.
go back to reference Green J, Thorogood N. Qualitative methods for health research. London: Sage; 2004. Green J, Thorogood N. Qualitative methods for health research. London: Sage; 2004.
8.
go back to reference Marchal B, Westhorp G, Wong G, Van Belle S, Greenhalgh T, Kegels G, Pawson R. Realist RCTs of complex interventions - an oxymoron. Soc Sci Med. 2013;94:124–8.CrossRefPubMed Marchal B, Westhorp G, Wong G, Van Belle S, Greenhalgh T, Kegels G, Pawson R. Realist RCTs of complex interventions - an oxymoron. Soc Sci Med. 2013;94:124–8.CrossRefPubMed
9.
go back to reference Pearce W, Raman S. The new randomised controlled trials (RCT) movement in public policy: challenges of epistemic governance. Policy Sci. 2014;47:387–402.CrossRef Pearce W, Raman S. The new randomised controlled trials (RCT) movement in public policy: challenges of epistemic governance. Policy Sci. 2014;47:387–402.CrossRef
10.
go back to reference Rowe M, Oltmann C. Randomised controlled trials in educational research: ontological and epistemological limitations. Afr J Health Prof Educ. 2016;8(1):6–8. Rowe M, Oltmann C. Randomised controlled trials in educational research: ontological and epistemological limitations. Afr J Health Prof Educ. 2016;8(1):6–8.
11.
go back to reference Tones K, Green J. Health Promotion: Planning and Strategies. London: Sage; 2004. Tones K, Green J. Health Promotion: Planning and Strategies. London: Sage; 2004.
12.
go back to reference Bonell C, Fletcher A, Morton M, Lorenc T. ‘Realist Randomised Controlled Trials’: a new approach to evaluating complex public health interventions. Soc Sci Med. 2012;75(12):2299–306.CrossRefPubMed Bonell C, Fletcher A, Morton M, Lorenc T. ‘Realist Randomised Controlled Trials’: a new approach to evaluating complex public health interventions. Soc Sci Med. 2012;75(12):2299–306.CrossRefPubMed
13.
go back to reference Pawson R, Tilley N. Realistic evaluation. London: Sage; 1997. Pawson R, Tilley N. Realistic evaluation. London: Sage; 1997.
14.
go back to reference Bhaskar R. The possibility of naturalism: a philosophical critique of the contemporary human sciences. Brighton: Harvester; 1979. Bhaskar R. The possibility of naturalism: a philosophical critique of the contemporary human sciences. Brighton: Harvester; 1979.
15.
go back to reference Bhaskar R. A realist theory of science. Leeds: Leeds Books; 1975. Bhaskar R. A realist theory of science. Leeds: Leeds Books; 1975.
16.
go back to reference Blaikie N. Approaches to social enquiry. Cambridge: Polity Press; 1993. Blaikie N. Approaches to social enquiry. Cambridge: Polity Press; 1993.
17.
go back to reference Russell B. A history of Western philosophy. London: Simon & Schuster, Inc; 1945. Russell B. A history of Western philosophy. London: Simon & Schuster, Inc; 1945.
18.
go back to reference Hacking I. Representing and intervening: introductory topics in the philosopy of natural science. Cambridge: Cambridge University Press; 1983.CrossRef Hacking I. Representing and intervening: introductory topics in the philosopy of natural science. Cambridge: Cambridge University Press; 1983.CrossRef
19.
go back to reference Kolakowski L. Positivist philosphy: from Hume to the Vienna circle. Harmondsworth: Penguin; 1972. Kolakowski L. Positivist philosphy: from Hume to the Vienna circle. Harmondsworth: Penguin; 1972.
20.
go back to reference Weber M. The methodology of the social sciences. Glencoe: Free Press; 1949. Weber M. The methodology of the social sciences. Glencoe: Free Press; 1949.
21.
go back to reference Solomon D, Watson MS, Delucchi KL, Schaps E, Battistich V. Enhancing children's prosocial behavior in the classroom. Am Educ Res J. 1988;25(4):527–54.CrossRef Solomon D, Watson MS, Delucchi KL, Schaps E, Battistich V. Enhancing children's prosocial behavior in the classroom. Am Educ Res J. 1988;25(4):527–54.CrossRef
22.
go back to reference Battistich V, Solomon D, Watson M, Solomon J, Schaps E. Effects of an elementary school program to enhance prosocial behavior on children's cognitive-social problem solving skills and strategies. J Appl Dev Psychol. 1989;10:147–69.CrossRef Battistich V, Solomon D, Watson M, Solomon J, Schaps E. Effects of an elementary school program to enhance prosocial behavior on children's cognitive-social problem solving skills and strategies. J Appl Dev Psychol. 1989;10:147–69.CrossRef
23.
go back to reference Bernstein B. Class, codes and control, vol. 3: towards a theory of educational transmission. London: Routledge; 1975.CrossRef Bernstein B. Class, codes and control, vol. 3: towards a theory of educational transmission. London: Routledge; 1975.CrossRef
24.
go back to reference Jamal F, Fletcher A, Shackleton A, Elbourne D, Viner R, Bonell C. The three stages of building and testing mid-level theories in a realist RCT: a case-example. Trials. 2015;16:466.CrossRefPubMedPubMedCentral Jamal F, Fletcher A, Shackleton A, Elbourne D, Viner R, Bonell C. The three stages of building and testing mid-level theories in a realist RCT: a case-example. Trials. 2015;16:466.CrossRefPubMedPubMedCentral
25.
go back to reference Markham WA, Aveyard P. A new theory of health promoting schools based on human functioning, school organisation and pedagogic practice. Soc Sci Med. 2003;56(6):1209–20.CrossRefPubMed Markham WA, Aveyard P. A new theory of health promoting schools based on human functioning, school organisation and pedagogic practice. Soc Sci Med. 2003;56(6):1209–20.CrossRefPubMed
26.
go back to reference Rose G. The strategy of preventive medicine. Oxford: Oxford university Press; 1992. Rose G. The strategy of preventive medicine. Oxford: Oxford university Press; 1992.
27.
go back to reference Campbell R, Starkey F, Holliday J, Audrey S, Bloor M, Parry-Langdon N, Hughes R, Moore L. An informal school-based peer-led intervention for smoking prevention in adolescence (ASSIST): a cluster randomised trial. Lancet. 2008;371:1595–602.CrossRefPubMedPubMedCentral Campbell R, Starkey F, Holliday J, Audrey S, Bloor M, Parry-Langdon N, Hughes R, Moore L. An informal school-based peer-led intervention for smoking prevention in adolescence (ASSIST): a cluster randomised trial. Lancet. 2008;371:1595–602.CrossRefPubMedPubMedCentral
28.
go back to reference Moore GF, Audrey S, Barker M, Bond L, Bonell C, Hardeman W, Moore L, O’Cathain A, Tinati T, Wight D. Process evaluation of complex interventions: Medical Research Council guidance. Br Med J. 2015;350:h1258.CrossRef Moore GF, Audrey S, Barker M, Bond L, Bonell C, Hardeman W, Moore L, O’Cathain A, Tinati T, Wight D. Process evaluation of complex interventions: Medical Research Council guidance. Br Med J. 2015;350:h1258.CrossRef
29.
go back to reference Van Belle S, Wong G, Westhorp G, Pearson M, Emmel N, Manzano A, Marchal B. Can “realist” randomised controlled trials be genuinely realist? Trials. 2016;17:313.CrossRefPubMedPubMedCentral Van Belle S, Wong G, Westhorp G, Pearson M, Emmel N, Manzano A, Marchal B. Can “realist” randomised controlled trials be genuinely realist? Trials. 2016;17:313.CrossRefPubMedPubMedCentral
30.
go back to reference Steckler L, Linnan A, editors. Process evaluation for public health interventions and research. San Francisco: Jossey-Bass; 2004. Steckler L, Linnan A, editors. Process evaluation for public health interventions and research. San Francisco: Jossey-Bass; 2004.
31.
go back to reference Audrey S, Holliday J, Campbell R. It’s good to talk: adolescent perspectives of an informal, peer-led intervention to reduce smoking. Soc Sci Med. 2006;63(2):320–34.CrossRefPubMed Audrey S, Holliday J, Campbell R. It’s good to talk: adolescent perspectives of an informal, peer-led intervention to reduce smoking. Soc Sci Med. 2006;63(2):320–34.CrossRefPubMed
32.
go back to reference Audrey S, Holliday J, Campbell R. Commitment and compatibility: teachers’ perspectives on the implementation of an effective school-based, peer-led smoking intervention. Health Educ J. 2008;67(2):74–90.CrossRef Audrey S, Holliday J, Campbell R. Commitment and compatibility: teachers’ perspectives on the implementation of an effective school-based, peer-led smoking intervention. Health Educ J. 2008;67(2):74–90.CrossRef
33.
go back to reference Moher D, Hopewell S, Schulz KF, Montori V, Gøtzsche PC, Devereaux PJ, Elbourne D, Egger M, Altman DG. CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. Br Med J. 2010;340:c869.CrossRef Moher D, Hopewell S, Schulz KF, Montori V, Gøtzsche PC, Devereaux PJ, Elbourne D, Egger M, Altman DG. CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. Br Med J. 2010;340:c869.CrossRef
34.
go back to reference Olds DL, Robinson J, O’Brien R, Luckey DW, Pettitt LM, Henderson CR, Ng RK, Sheff KL, Korfmacher J, Hiatt S, et al. Home visiting by paraprofessionals and by nurses: a randomized, controlled trial. Pediatrics. 2002;110(3):486–96.CrossRefPubMed Olds DL, Robinson J, O’Brien R, Luckey DW, Pettitt LM, Henderson CR, Ng RK, Sheff KL, Korfmacher J, Hiatt S, et al. Home visiting by paraprofessionals and by nurses: a randomized, controlled trial. Pediatrics. 2002;110(3):486–96.CrossRefPubMed
35.
go back to reference Robling M, Bekkers MJ, Bell K, Butler CC, Cannings-John R, Channon S, Corbacho Martin B, Gregory JW, Hood K, Kemp A, et al. Effectiveness of a nurse-led intensive home-visitation programme for first-time teenage mothers (Building Blocks): a pragmatic randomised controlled trial. Lancet. 2016;387(10014):146–55.CrossRefPubMedPubMedCentral Robling M, Bekkers MJ, Bell K, Butler CC, Cannings-John R, Channon S, Corbacho Martin B, Gregory JW, Hood K, Kemp A, et al. Effectiveness of a nurse-led intensive home-visitation programme for first-time teenage mothers (Building Blocks): a pragmatic randomised controlled trial. Lancet. 2016;387(10014):146–55.CrossRefPubMedPubMedCentral
36.
go back to reference Littell JH, Corcoran J, Pillai V. Systematic reviews and meta-analysis. Oxford: Oxford University Press; 2008.CrossRef Littell JH, Corcoran J, Pillai V. Systematic reviews and meta-analysis. Oxford: Oxford University Press; 2008.CrossRef
37.
go back to reference Schünemann H, Brożek J, Guyatt G, Oxman AD. Introduction to GRADE handbook. In: Handbook for grading the quality of evidence and the strength of recommendations using the GRADE approach. Oxford: Cochrane Collaboration; 2013. Schünemann H, Brożek J, Guyatt G, Oxman AD. Introduction to GRADE handbook. In: Handbook for grading the quality of evidence and the strength of recommendations using the GRADE approach. Oxford: Cochrane Collaboration; 2013.
38.
go back to reference Battistich V, Solomon D, Delucchi KL. Effects of a program to enhance prosocial development on adjustment. Boston: Annual Meeting of the American Psychological Association; 1990. Battistich V, Solomon D, Delucchi KL. Effects of a program to enhance prosocial development on adjustment. Boston: Annual Meeting of the American Psychological Association; 1990.
39.
go back to reference Battistich V. Effects of a school-based program to enhance prosocial development on children's peer relations and social adjustment. J Res Character Educ. 2003;1:1–7. Battistich V. Effects of a school-based program to enhance prosocial development on children's peer relations and social adjustment. J Res Character Educ. 2003;1:1–7.
40.
go back to reference Langford R, Campbell R, Magnus D, Bonell CP, Murphy SM, Waters E, Komro KA, Gibbs LF. The WHO health promoting school framework for improving the health and well-being of students and staff. Cochrane Database Syst Rev. 2011;1:CD008958. 2014 Langford R, Campbell R, Magnus D, Bonell CP, Murphy SM, Waters E, Komro KA, Gibbs LF. The WHO health promoting school framework for improving the health and well-being of students and staff. Cochrane Database Syst Rev. 2011;1:CD008958. 2014
41.
go back to reference Barry MM, Clarke AM, Jenkins R, Patel V. A systematic review of the effectiveness of mental health promotion interventions for young people in low and middle income countries. BMC Public Health. 2013;13:835. Barry MM, Clarke AM, Jenkins R, Patel V. A systematic review of the effectiveness of mental health promotion interventions for young people in low and middle income countries. BMC Public Health. 2013;13:835.
42.
go back to reference Thomas RE, McLellan J, Perera R. Effectiveness of school-based smoking prevention curricula: systematic review and metaanalysis. BMJ Open. 2015;5:e006976.CrossRefPubMedPubMedCentral Thomas RE, McLellan J, Perera R. Effectiveness of school-based smoking prevention curricula: systematic review and metaanalysis. BMJ Open. 2015;5:e006976.CrossRefPubMedPubMedCentral
43.
go back to reference Movsisyan A, Melendez-Torres GJ, Montgomery P. Outcomes in systematic reviews of complex interventions never reached “high” GRADE ratings when compared with those of simple interventions. J Clin Epidemiol. 2016;78:22–33.CrossRefPubMed Movsisyan A, Melendez-Torres GJ, Montgomery P. Outcomes in systematic reviews of complex interventions never reached “high” GRADE ratings when compared with those of simple interventions. J Clin Epidemiol. 2016;78:22–33.CrossRefPubMed
44.
go back to reference Popper K. The open society and its enemies, Vol. 2: Hegel and Marx. London: Routledge & Kegan Paul; 1945. Popper K. The open society and its enemies, Vol. 2: Hegel and Marx. London: Routledge & Kegan Paul; 1945.
45.
go back to reference Gardner F, Montgomery P, Knerr W. Transporting evidence-based parenting programs for child problem behavior (age 3–10) between countries: systematic review and meta-analysis. J Clin Child Adolesc Psychol. 2016;45(6):749–62.CrossRefPubMed Gardner F, Montgomery P, Knerr W. Transporting evidence-based parenting programs for child problem behavior (age 3–10) between countries: systematic review and meta-analysis. J Clin Child Adolesc Psychol. 2016;45(6):749–62.CrossRefPubMed
46.
go back to reference Bonell C, Allen E, Christie D, Elbourne D, Fletcher A, Grieve R, LeGood R, Mathiot A, Scott S, Wiggins M, et al. Initiating change locally in bullying and aggression through the school environment (INCLUSIVE): study protocol for a cluster randomised controlled trial. Trials. 2014;15:381.CrossRefPubMedPubMedCentral Bonell C, Allen E, Christie D, Elbourne D, Fletcher A, Grieve R, LeGood R, Mathiot A, Scott S, Wiggins M, et al. Initiating change locally in bullying and aggression through the school environment (INCLUSIVE): study protocol for a cluster randomised controlled trial. Trials. 2014;15:381.CrossRefPubMedPubMedCentral
47.
go back to reference Starkey F, Moore L, Campbell R, Sidaway M, Bloor M. Rationale, design and conduct of a comprehensive evaluation of a school-based peer-led anti-smoking intervention in the UK: the ASSIST cluster randomised trial. BMC Public Health. 2005;5(1):43.CrossRefPubMedPubMedCentral Starkey F, Moore L, Campbell R, Sidaway M, Bloor M. Rationale, design and conduct of a comprehensive evaluation of a school-based peer-led anti-smoking intervention in the UK: the ASSIST cluster randomised trial. BMC Public Health. 2005;5(1):43.CrossRefPubMedPubMedCentral
48.
go back to reference Ragin CC, Drass KA, Davey S. Fuzzy-set / qualitative comparative analysis 2.0. Tucson: Department of Sociology, University of Arizona; 2006. Ragin CC, Drass KA, Davey S. Fuzzy-set / qualitative comparative analysis 2.0. Tucson: Department of Sociology, University of Arizona; 2006.
49.
go back to reference Borenstein M, Hedges L, Higgins J, Rothstein H. Introduction to meta-analysis. Chichester: John Wiley & Sons Ltd; 2009.CrossRef Borenstein M, Hedges L, Higgins J, Rothstein H. Introduction to meta-analysis. Chichester: John Wiley & Sons Ltd; 2009.CrossRef
50.
go back to reference Fisher R. The Design of experiments (5th edition). Edinburgh: Oliver & Boyd; 1949. Fisher R. The Design of experiments (5th edition). Edinburgh: Oliver & Boyd; 1949.
51.
go back to reference Oakley A, Strange V, Bonell C, Allen E, Stephenson J. Process evaluation in randomised controlled trials of complex interventions. BMJ. 2006;332(7538):413–6.CrossRefPubMedPubMedCentral Oakley A, Strange V, Bonell C, Allen E, Stephenson J. Process evaluation in randomised controlled trials of complex interventions. BMJ. 2006;332(7538):413–6.CrossRefPubMedPubMedCentral
55.
go back to reference Oakley A, Strange V, Bonell C, Allen E, Stephenson J. Integrating process evaluation in the design of randomised controlled trials of complex interventions: the example of the RIPPLE Study. Br Med J. 2006;332:413–6.CrossRef Oakley A, Strange V, Bonell C, Allen E, Stephenson J. Integrating process evaluation in the design of randomised controlled trials of complex interventions: the example of the RIPPLE Study. Br Med J. 2006;332:413–6.CrossRef
56.
go back to reference Popper K. The logic of scientific discovery. London: Routledge & Kegan Paul; 1959. Popper K. The logic of scientific discovery. London: Routledge & Kegan Paul; 1959.
57.
go back to reference Giddens A. The constitution of society. Cambridge: Polity Press; 1984. Giddens A. The constitution of society. Cambridge: Polity Press; 1984.
58.
go back to reference Groff R. Critical realism, post-positivism, and the possibility of knowledge. New York: Routledge; 2004. Groff R. Critical realism, post-positivism, and the possibility of knowledge. New York: Routledge; 2004.
59.
go back to reference Pawson R. Evidence-based policy: a realist perspective. Thousand Oaks: Sage; 2006.CrossRef Pawson R. Evidence-based policy: a realist perspective. Thousand Oaks: Sage; 2006.CrossRef
60.
go back to reference Bonell C, Fletcher A, Fitzgerald-Yau N, Hale D, Allen E, Elbourne D, Jones R, Bond L, Wiggins M, Miner A, et al. A pilot randomised controlled trial of the INCLUSIVE intervention for initiating change locally in bullying and aggression through the school environment: final report. Health Technol Assess. 2013;19(53):1–109 Bonell C, Fletcher A, Fitzgerald-Yau N, Hale D, Allen E, Elbourne D, Jones R, Bond L, Wiggins M, Miner A, et al. A pilot randomised controlled trial of the INCLUSIVE intervention for initiating change locally in bullying and aggression through the school environment: final report. Health Technol Assess. 2013;19(53):1–109
Metadata
Title
Are randomised controlled trials positivist? Reviewing the social science and philosophy literature to assess positivist tendencies of trials of social interventions in public health and health services
Authors
Chris Bonell
Graham Moore
Emily Warren
Laurence Moore
Publication date
01-12-2018
Publisher
BioMed Central
Published in
Trials / Issue 1/2018
Electronic ISSN: 1745-6215
DOI
https://doi.org/10.1186/s13063-018-2589-4

Other articles of this Issue 1/2018

Trials 1/2018 Go to the issue