Skip to main content
Top
Published in: BMC Medicine 1/2013

Open Access 01-12-2013 | Guideline

RAMESES publication standards: realist syntheses

Authors: Geoff Wong, Trish Greenhalgh, Gill Westhorp, Jeanette Buckingham, Ray Pawson

Published in: BMC Medicine | Issue 1/2013

Login to get access

Abstract

Background

There is growing interest in realist synthesis as an alternative systematic review method. This approach offers the potential to expand the knowledge base in policy-relevant areas -for example, by explaining the success, failure or mixed fortunes of complex interventions. No previous publication standards exist for reporting realist syntheses. This standard was developed as part of the RAMESES (Realist And MEta-narrative Evidence Syntheses: Evolving Standards) project. The project's aim is to produce preliminary publication standards for realist systematic reviews.

Methods

We (a) collated and summarized existing literature on the principles of good practice in realist syntheses; (b) considered the extent to which these principles had been followed by published syntheses, thereby identifying how rigor may be lost and how existing methods could be improved; (c) used a three-round online Delphi method with an interdisciplinary panel of national and international experts in evidence synthesis, realist research, policy and/or publishing to produce and iteratively refine a draft set of methodological steps and publication standards; (d) provided real-time support to ongoing realist syntheses and the open-access RAMESES online discussion list so as to capture problems and questions as they arose; and (e) synthesized expert input, evidence syntheses and real-time problem analysis into a definitive set of standards.

Results

We identified 35 published realist syntheses, provided real-time support to 9 on-going syntheses and captured questions raised in the RAMESES discussion list. Through analysis and discussion within the project team, we summarized the published literature and common questions and challenges into briefing materials for the Delphi panel, comprising 37 members. Within three rounds this panel had reached consensus on 19 key publication standards, with an overall response rate of 91%.

Conclusion

This project used multiple sources to develop and draw together evidence and expertise in realist synthesis. For each item we have included an explanation for why it is important and guidance on how it might be reported. Realist synthesis is a relatively new method for evidence synthesis and as experience and methodological developments occur, we anticipate that these standards will evolve to reflect further methodological developments. We hope that these standards will act as a resource that will contribute to improving the reporting of realist syntheses.
To encourage dissemination of the RAMESES publication standards, this article is co-published in the Journal of Advanced Nursing and is freely accessible on Wiley Online Library (http://​www.​wileyonlinelibra​ry.​com/​journal/​jan). Please see related article http://​www.​biomedcentral.​com/​1741-7015/​11/​20 and http://​www.​biomedcentral.​com/​1741-7015/​11/​22
Appendix
Available only for authorised users
Literature
1.
go back to reference Pawson R, Evidence-based Policy: A Realist Perspective. 2006, London: Sage Pawson R, Evidence-based Policy: A Realist Perspective. 2006, London: Sage
2.
go back to reference Pearson M, Coomber R: The challenge of external validity in policy-relevant systematic reviews: a case study from the field of substance misuse. Addiction. 2010, 105: 136-145. 10.1111/j.1360-0443.2009.02713.x.CrossRefPubMed Pearson M, Coomber R: The challenge of external validity in policy-relevant systematic reviews: a case study from the field of substance misuse. Addiction. 2010, 105: 136-145. 10.1111/j.1360-0443.2009.02713.x.CrossRefPubMed
3.
go back to reference Dixon-Woods M, Agarwal S, Jones D, Young B, Sutton A: Synthesising qualitative and quantitative evidence: a review of possible methods. J Health Serv Res Policy. 2005, 10: 45-53. 10.1258/1355819052801804.CrossRefPubMed Dixon-Woods M, Agarwal S, Jones D, Young B, Sutton A: Synthesising qualitative and quantitative evidence: a review of possible methods. J Health Serv Res Policy. 2005, 10: 45-53. 10.1258/1355819052801804.CrossRefPubMed
4.
go back to reference Barnett-Page E, Thomas J: Methods for the synthesis of qualitative research: a critical review. BMC Med Res Methodol. 2009, 9: 59-10.1186/1471-2288-9-59.CrossRefPubMedPubMedCentral Barnett-Page E, Thomas J: Methods for the synthesis of qualitative research: a critical review. BMC Med Res Methodol. 2009, 9: 59-10.1186/1471-2288-9-59.CrossRefPubMedPubMedCentral
5.
go back to reference Pope P, Mays N, Popay J: Synthesizing Qualitative and Quantitative Health Research: A Guide to Methods. 2007, Maidenhead, UK: Open University Press Pope P, Mays N, Popay J: Synthesizing Qualitative and Quantitative Health Research: A Guide to Methods. 2007, Maidenhead, UK: Open University Press
6.
go back to reference Gough D, Oliver S, Thomas J: An Introduction to Systematic Reviews. 2012, London: Sage Gough D, Oliver S, Thomas J: An Introduction to Systematic Reviews. 2012, London: Sage
7.
go back to reference Kastner M, Tricco A, Soobiah C, Lillie E, Perrier L, Horsley T, Welch V, Cogo E, Antony J, Straus S: What is the most appropriate knowledge synthesis method to conduct a review? Protocol for a scoping review. BMC Med Res Methodol. 2012, 12: 114-10.1186/1471-2288-12-114.CrossRefPubMedPubMedCentral Kastner M, Tricco A, Soobiah C, Lillie E, Perrier L, Horsley T, Welch V, Cogo E, Antony J, Straus S: What is the most appropriate knowledge synthesis method to conduct a review? Protocol for a scoping review. BMC Med Res Methodol. 2012, 12: 114-10.1186/1471-2288-12-114.CrossRefPubMedPubMedCentral
8.
go back to reference Merton R: On Theoretical Sociology. Five Essays, Old and New. 1967, New York: The Free Press Merton R: On Theoretical Sociology. Five Essays, Old and New. 1967, New York: The Free Press
9.
go back to reference Astbury B, Leeuw FL: Unpacking black boxes: mechanisms and theory building in evaluation. Am J Eval. 2010, 31: 363-381. 10.1177/1098214010371972.CrossRef Astbury B, Leeuw FL: Unpacking black boxes: mechanisms and theory building in evaluation. Am J Eval. 2010, 31: 363-381. 10.1177/1098214010371972.CrossRef
10.
go back to reference Shepperd S, Lewin L, Straus S, Clarke M, Eccles M, Fitzpatrick R, Wong G, Sheikh A: Can we systematically review studies that evaluate complex interventions. PLoS Med. 2009, 6: e1000086-10.1371/journal.pmed.1000086.CrossRefPubMedPubMedCentral Shepperd S, Lewin L, Straus S, Clarke M, Eccles M, Fitzpatrick R, Wong G, Sheikh A: Can we systematically review studies that evaluate complex interventions. PLoS Med. 2009, 6: e1000086-10.1371/journal.pmed.1000086.CrossRefPubMedPubMedCentral
11.
go back to reference Rycroft-Malone J, McCormack B, Hutchinson AM, DeCorby K, Bucknall TK, Kent B, Schultz A, Snelgrove-Clarke E, Stetler CB, Titler M, Wallin L, Wilson V: Realist synthesis: illustrating the method for implementation research. Implement Sci. 2012, 7: 33-10.1186/1748-5908-7-33.CrossRefPubMedPubMedCentral Rycroft-Malone J, McCormack B, Hutchinson AM, DeCorby K, Bucknall TK, Kent B, Schultz A, Snelgrove-Clarke E, Stetler CB, Titler M, Wallin L, Wilson V: Realist synthesis: illustrating the method for implementation research. Implement Sci. 2012, 7: 33-10.1186/1748-5908-7-33.CrossRefPubMedPubMedCentral
12.
go back to reference Robert E, Ridde V, Marchal B, Fournier P: Protocol: a realist review of user fee exemption policies for health services in Africa. BMJ Open. 2012, 2: e000706-10.1136/bmjopen-2011-000706.CrossRefPubMedPubMedCentral Robert E, Ridde V, Marchal B, Fournier P: Protocol: a realist review of user fee exemption policies for health services in Africa. BMJ Open. 2012, 2: e000706-10.1136/bmjopen-2011-000706.CrossRefPubMedPubMedCentral
13.
go back to reference Pawson R: Evidence-based policy: the promise of 'realist synthesis'. Evaluation. 2002, 8: 340-358. 10.1177/135638902401462448.CrossRef Pawson R: Evidence-based policy: the promise of 'realist synthesis'. Evaluation. 2002, 8: 340-358. 10.1177/135638902401462448.CrossRef
14.
go back to reference Pawson R, Greenhalgh T, Harvey G, Walshe K: Realist Synthesis - an Introduction. ESRC Working Paper Series. 2004, London: Economic and Social Research Council Pawson R, Greenhalgh T, Harvey G, Walshe K: Realist Synthesis - an Introduction. ESRC Working Paper Series. 2004, London: Economic and Social Research Council
15.
go back to reference Pawson R, Greenhalgh T, Harvey G, Walshe K: Realist review-a new method of systematic review designed for complex policy interventions. J Health Serv Res Policy. 2005, 10 (Suppl 1): 21-34.CrossRefPubMed Pawson R, Greenhalgh T, Harvey G, Walshe K: Realist review-a new method of systematic review designed for complex policy interventions. J Health Serv Res Policy. 2005, 10 (Suppl 1): 21-34.CrossRefPubMed
16.
go back to reference Moher D, Hopewell S, Schulz K, Montori V, Gotzsche P, Devereaux P, Elbourne D, Egger M, Altman D: CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trial. BMJ. 2010, 340: c869-10.1136/bmj.c869. for the CONSORT GroupCrossRefPubMedPubMedCentral Moher D, Hopewell S, Schulz K, Montori V, Gotzsche P, Devereaux P, Elbourne D, Egger M, Altman D: CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trial. BMJ. 2010, 340: c869-10.1136/bmj.c869. for the CONSORT GroupCrossRefPubMedPubMedCentral
17.
go back to reference Brouwers M, Kho M, Browman G, Burgers J, Cluzeau F, Feder G, Fervers B, Graham I, Grimshaw J, Hanna S, Littlejohns P, Makarski J, Zitzelsberger L: AGREE II: advancing guideline development, reporting and evaluation in healthcare. Can Med Assoc J. 2010, 182: E839-842. 10.1503/cmaj.090449. AGREE Next Steps ConsortiumCrossRef Brouwers M, Kho M, Browman G, Burgers J, Cluzeau F, Feder G, Fervers B, Graham I, Grimshaw J, Hanna S, Littlejohns P, Makarski J, Zitzelsberger L: AGREE II: advancing guideline development, reporting and evaluation in healthcare. Can Med Assoc J. 2010, 182: E839-842. 10.1503/cmaj.090449. AGREE Next Steps ConsortiumCrossRef
18.
go back to reference Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JP, Clarke M, Devereaux PJ, Kleijnen J, Moher D: The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration. BMJ. 2009, 339: b2700-10.1136/bmj.b2700.CrossRefPubMedPubMedCentral Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JP, Clarke M, Devereaux PJ, Kleijnen J, Moher D: The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration. BMJ. 2009, 339: b2700-10.1136/bmj.b2700.CrossRefPubMedPubMedCentral
19.
go back to reference Davidoff F, Batalden P, Stevens D, Ogrinc G, Mooney S, SQUIRE Development Group: Publication guidelines for improvement studies in health care: evolution of the SQUIRE project. Ann Intern Med. 2008, 149: 670-676.CrossRefPubMed Davidoff F, Batalden P, Stevens D, Ogrinc G, Mooney S, SQUIRE Development Group: Publication guidelines for improvement studies in health care: evolution of the SQUIRE project. Ann Intern Med. 2008, 149: 670-676.CrossRefPubMed
20.
go back to reference Greenhalgh T, Wong G, Westhorp G, Pawson R: Protocol - realist and meta-narrative evidence synthesis: evolving standards (RAMESES). BMC Med Res Methodol. 2011, 11: 115-10.1186/1471-2288-11-115.CrossRefPubMedPubMedCentral Greenhalgh T, Wong G, Westhorp G, Pawson R: Protocol - realist and meta-narrative evidence synthesis: evolving standards (RAMESES). BMC Med Res Methodol. 2011, 11: 115-10.1186/1471-2288-11-115.CrossRefPubMedPubMedCentral
22.
go back to reference Wong G, Greenhalgh T, Westhorp G, Pawson R: Realist methods in medical education research: what are they and what can they contribute. Med Educ. 2012, 46: 89-96. 10.1111/j.1365-2923.2011.04045.x.CrossRefPubMed Wong G, Greenhalgh T, Westhorp G, Pawson R: Realist methods in medical education research: what are they and what can they contribute. Med Educ. 2012, 46: 89-96. 10.1111/j.1365-2923.2011.04045.x.CrossRefPubMed
23.
go back to reference Wong G: The internet in medical education: a worked example of a realist review. Synthesising Qualitative Research: Choosing the Right Approach. Edited by: Hannes K, Lockwood C. Chicester. 2011, UK: Wiley-Blackwell Wong G: The internet in medical education: a worked example of a realist review. Synthesising Qualitative Research: Choosing the Right Approach. Edited by: Hannes K, Lockwood C. Chicester. 2011, UK: Wiley-Blackwell
25.
go back to reference Dieleman M, Gerretsen B, van der Wilt GJ: Human resource management interventions to improve health workers' performance in low and middle income countries: a realist review. Health Res Policy Syst. 2009, 7: 7-10.1186/1478-4505-7-7.CrossRefPubMedPubMedCentral Dieleman M, Gerretsen B, van der Wilt GJ: Human resource management interventions to improve health workers' performance in low and middle income countries: a realist review. Health Res Policy Syst. 2009, 7: 7-10.1186/1478-4505-7-7.CrossRefPubMedPubMedCentral
26.
go back to reference Wong G, Pawson R, Owen L: Policy guidance on threats to legislative interventions in public health: a realist synthesis. BMC Public Health. 2011, 11: 222-10.1186/1471-2458-11-222.CrossRefPubMedPubMedCentral Wong G, Pawson R, Owen L: Policy guidance on threats to legislative interventions in public health: a realist synthesis. BMC Public Health. 2011, 11: 222-10.1186/1471-2458-11-222.CrossRefPubMedPubMedCentral
27.
go back to reference Kane SS, Gerretsen B, Scherpbier R, Dal Poz M, Dieleman M: A realist synthesis of randomised control trials involving use of community health workers for delivering child health interventions in low and middle income countries. BMC Health Serv Res. 2010, 10: 286-10.1186/1472-6963-10-286.CrossRefPubMedPubMedCentral Kane SS, Gerretsen B, Scherpbier R, Dal Poz M, Dieleman M: A realist synthesis of randomised control trials involving use of community health workers for delivering child health interventions in low and middle income countries. BMC Health Serv Res. 2010, 10: 286-10.1186/1472-6963-10-286.CrossRefPubMedPubMedCentral
28.
go back to reference Jackson L, Langille L, Lyons R, Hughes J, Martin D, Winstanley V: Does moving from a high-poverty to lower-poverty neighborhood improve mental health? A realist review of 'Moving to Opportunity'. Health Place. 2009, 15: 961-970. 10.1016/j.healthplace.2009.03.003.CrossRefPubMed Jackson L, Langille L, Lyons R, Hughes J, Martin D, Winstanley V: Does moving from a high-poverty to lower-poverty neighborhood improve mental health? A realist review of 'Moving to Opportunity'. Health Place. 2009, 15: 961-970. 10.1016/j.healthplace.2009.03.003.CrossRefPubMed
29.
go back to reference Wong G, Greenhalgh T, Pawson R: Internet-based medical education: a realist review of what works, for whom and in what circumstances. BMC Med Educ. 2010, 10: 12-10.1186/1472-6920-10-12.CrossRefPubMedPubMedCentral Wong G, Greenhalgh T, Pawson R: Internet-based medical education: a realist review of what works, for whom and in what circumstances. BMC Med Educ. 2010, 10: 12-10.1186/1472-6920-10-12.CrossRefPubMedPubMedCentral
30.
go back to reference Jagosh J, Macaulay AC, Pluye P, Salsberg J, Bush PL, Henderson J, Sirett E, Wong G, Cargo M, Herbert CP, Seifer SD, Green LW, Greenhalgh T: Uncovering the benefits of participatory research: implications of a realist review for health research and practice. Milbank Q. 2012, 90: 311-346. 10.1111/j.1468-0009.2012.00665.x.CrossRefPubMedPubMedCentral Jagosh J, Macaulay AC, Pluye P, Salsberg J, Bush PL, Henderson J, Sirett E, Wong G, Cargo M, Herbert CP, Seifer SD, Green LW, Greenhalgh T: Uncovering the benefits of participatory research: implications of a realist review for health research and practice. Milbank Q. 2012, 90: 311-346. 10.1111/j.1468-0009.2012.00665.x.CrossRefPubMedPubMedCentral
31.
go back to reference Dieleman M, Kane S, Zwanikken P, Gerretsen B: Realist Review and Synthesis of Retention Studies for Health Workers in Rural and Remote Areas. 2011, Geneva, Switzerland: World Health Organization Dieleman M, Kane S, Zwanikken P, Gerretsen B: Realist Review and Synthesis of Retention Studies for Health Workers in Rural and Remote Areas. 2011, Geneva, Switzerland: World Health Organization
32.
go back to reference Pawson R, Wong G, Owen L: Known knowns, known unknowns, unknown unknowns: the predicament of evidence-based policy. Am J Eval. 2011 Pawson R, Wong G, Owen L: Known knowns, known unknowns, unknown unknowns: the predicament of evidence-based policy. Am J Eval. 2011
Metadata
Title
RAMESES publication standards: realist syntheses
Authors
Geoff Wong
Trish Greenhalgh
Gill Westhorp
Jeanette Buckingham
Ray Pawson
Publication date
01-12-2013
Publisher
BioMed Central
Published in
BMC Medicine / Issue 1/2013
Electronic ISSN: 1741-7015
DOI
https://doi.org/10.1186/1741-7015-11-21

Other articles of this Issue 1/2013

BMC Medicine 1/2013 Go to the issue