Skip to main content
Top
Published in: Health Research Policy and Systems 1/2015

Open Access 01-12-2016 | Research

Evidence-based medicine – an appropriate tool for evidence-based health policy? A case study from Norway

Authors: Kirsti Malterud, Anne Karen Bjelland, Kari Tove Elvbakken

Published in: Health Research Policy and Systems | Issue 1/2015

Login to get access

Abstract

Background

Evidence-based policy (EBP), a concept modelled on the principles of evidence-based medicine (EBM), is widely used in different areas of policymaking. Systematic reviews (SRs) with meta-analyses gradually became the methods of choice for synthesizing research evidence about interventions and judgements about quality of evidence and strength of recommendations. Critics have argued that the relation between research evidence and service policies is weak, and that the notion of EBP rests on a misunderstanding of policy processes. Having explored EBM standards and knowledge requirements for health policy decision-making, we present an empirical point of departure for discussing the relationship between EBM and EBP.

Methods

In a case study exploring the Norwegian Knowledge Centre for the Health Services (NOKC), an independent government unit, we first searched for information about the background and development of the NOKC to establish a research context. We then identified, selected and organized official NOKC publications as an empirical sample of typical top-of-the-line knowledge delivery adhering to EBM standards. Finally, we explored conclusions in this type of publication, specifically addressing their potential as policy decision tools.

Results

From a total sample of 151 SRs published by the NOKC in the period 2004–2013, a purposive subsample from 2012 (14 publications) advised major caution about their conclusions because of the quality or relevance of the underlying documentation. Although the case study did not include a systematic investigation of uptake and policy consequences, SRs were found to be inappropriate as universal tools for health policy decision-making.

Conclusions

The case study demonstrates that EBM is not necessarily suited to knowledge provision for every kind of policy decision-making. Our analysis raises the question of whether the evidence-based movement, represented here by an independent government organization, undertakes too broad a range of commissions using strategies that seem too confined. Policymaking in healthcare should be based on relevant and transparent knowledge, taking due account of the context of the intervention. However, we do not share the belief that the complex and messy nature of policy processes in general is compatible with the standards of EBM.
Literature
1.
go back to reference Orton L, Lloyd-Williams F, Taylor-Robinson D, O’Flaherty M, Capewell S. The use of research evidence in public health decision making processes: systematic review. PLoS One. 2011;6:e21704.CrossRef Orton L, Lloyd-Williams F, Taylor-Robinson D, O’Flaherty M, Capewell S. The use of research evidence in public health decision making processes: systematic review. PLoS One. 2011;6:e21704.CrossRef
2.
go back to reference Klein R. From evidence-based medicine to evidence-based policy? J Health Serv Res Policy. 2000;5:65–6.CrossRef Klein R. From evidence-based medicine to evidence-based policy? J Health Serv Res Policy. 2000;5:65–6.CrossRef
3.
4.
go back to reference Cochrane AL. Archie Cochrane in his own words. Selections arranged from his 1972 introduction to “Effectiveness and Efficiency: Random Reflections on the Health Services” 1972. Control Clin Trials. 1989;10:428–33.CrossRef Cochrane AL. Archie Cochrane in his own words. Selections arranged from his 1972 introduction to “Effectiveness and Efficiency: Random Reflections on the Health Services” 1972. Control Clin Trials. 1989;10:428–33.CrossRef
5.
go back to reference Hill GB. Archie Cochrane and his legacy: An internal challenge to physicians’ autonomy? J Clin Epidemiol. 2000;53:1189–92.CrossRef Hill GB. Archie Cochrane and his legacy: An internal challenge to physicians’ autonomy? J Clin Epidemiol. 2000;53:1189–92.CrossRef
6.
go back to reference Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isn’t. BMJ. 1996;312:71–2.CrossRef Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isn’t. BMJ. 1996;312:71–2.CrossRef
7.
go back to reference Levin A. The Cochrane Collaboration Ann Intern Med. 2001;135:309–12.PubMed Levin A. The Cochrane Collaboration Ann Intern Med. 2001;135:309–12.PubMed
8.
go back to reference Smith R, Rennie D. Evidence-based medicine--an oral history. JAMA. 2014;311:365–7.CrossRef Smith R, Rennie D. Evidence-based medicine--an oral history. JAMA. 2014;311:365–7.CrossRef
9.
go back to reference Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-Coello P, et al. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ. 2008;336:924–6.CrossRef Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-Coello P, et al. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ. 2008;336:924–6.CrossRef
11.
go back to reference Oxman AD, Guyatt GH. The science of reviewing research. Ann N Y Acad Sci. 1993;703:125–33. Discussion 133–4.CrossRef Oxman AD, Guyatt GH. The science of reviewing research. Ann N Y Acad Sci. 1993;703:125–33. Discussion 133–4.CrossRef
12.
go back to reference Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gotzsche PC, Ioannidis JP, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. J Clin Epidemiol. 2009;62:e1–34.CrossRef Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gotzsche PC, Ioannidis JP, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. J Clin Epidemiol. 2009;62:e1–34.CrossRef
14.
go back to reference Greenhalgh T. Papers that summarise other papers (systematic reviews and meta-analyses). BMJ. 1997;315:672–5.CrossRef Greenhalgh T. Papers that summarise other papers (systematic reviews and meta-analyses). BMJ. 1997;315:672–5.CrossRef
15.
go back to reference Frey K, Ledermann S. Introduction: evidence-based policy: a concept in geographical and substantive expansion. Ger Policy Stud. 2010;6:1–15. Frey K, Ledermann S. Introduction: evidence-based policy: a concept in geographical and substantive expansion. Ger Policy Stud. 2010;6:1–15.
16.
go back to reference Canadian Task Force on the Periodic Health Examination. The periodic health examination. Canadian Task Force on the Periodic Health Examination. Can Med Assoc J. 1979;121:1193–254. Canadian Task Force on the Periodic Health Examination. The periodic health examination. Canadian Task Force on the Periodic Health Examination. Can Med Assoc J. 1979;121:1193–254.
18.
go back to reference Yin RK. Case study research: design and methods. 4th ed. Los Angeles: Sage; 2009. Yin RK. Case study research: design and methods. 4th ed. Los Angeles: Sage; 2009.
19.
go back to reference St.prp. nr 1 (2003-2004) for budsjetterminen 2004. [Royal proposition - Government budget 2004, in Norwegian]. Oslo: Government of Norway; 2003. St.prp. nr 1 (2003-2004) for budsjetterminen 2004. [Royal proposition - Government budget 2004, in Norwegian]. Oslo: Government of Norway; 2003.
20.
go back to reference St.prp. nr 1S (2013-2014) for budsjetterminen 2014. [Royal proposition - Government budget 2014, in Norwegian]. Oslo: Government of Norway; 2013. St.prp. nr 1S (2013-2014) for budsjetterminen 2014. [Royal proposition - Government budget 2014, in Norwegian]. Oslo: Government of Norway; 2013.
22.
go back to reference Straus SE, Glasziou P, Richardson WS, Haynes RB. Evidence-based medicine: How to practice and teach it. Edinburgh: Churchill Livingstone/Elsevier; 2011. Straus SE, Glasziou P, Richardson WS, Haynes RB. Evidence-based medicine: How to practice and teach it. Edinburgh: Churchill Livingstone/Elsevier; 2011.
23.
go back to reference Evidence-Based Medicine Working Group. Evidence-based medicine. A new approach to teaching the practice of medicine. JAMA. 1992;268:2420–5.CrossRef Evidence-Based Medicine Working Group. Evidence-based medicine. A new approach to teaching the practice of medicine. JAMA. 1992;268:2420–5.CrossRef
24.
go back to reference Greenhalgh T. How to read a paper: the basics of evidence-based medicine. Chichester: John Wiley & Sons; 2014. Greenhalgh T. How to read a paper: the basics of evidence-based medicine. Chichester: John Wiley & Sons; 2014.
25.
go back to reference Lewin S, Lavis JN, Oxman AD, Bastias G, Chopra M, Ciapponi A, et al. Supporting the delivery of cost-effective interventions in primary health-care systems in low-income and middle-income countries: an overview of systematic reviews. Lancet. 2008;372:928–39.CrossRef Lewin S, Lavis JN, Oxman AD, Bastias G, Chopra M, Ciapponi A, et al. Supporting the delivery of cost-effective interventions in primary health-care systems in low-income and middle-income countries: an overview of systematic reviews. Lancet. 2008;372:928–39.CrossRef
29.
go back to reference La Rochelle P. Basic learning concepts in EBM: the bidimentional hierarchy of evidence. Evid Based Med. 2014;19:83–4.CrossRef La Rochelle P. Basic learning concepts in EBM: the bidimentional hierarchy of evidence. Evid Based Med. 2014;19:83–4.CrossRef
30.
go back to reference Patton MQ. Qualitative research & evaluation methods: integrating theory and practice. 4th ed. Thousand Oaks: SAGE Publications; 2015. Patton MQ. Qualitative research & evaluation methods: integrating theory and practice. 4th ed. Thousand Oaks: SAGE Publications; 2015.
31.
go back to reference Segal JZ. Health and the Rhetoric of Medicine. Carbondale: Southern Illinois University Press; 2008. Segal JZ. Health and the Rhetoric of Medicine. Carbondale: Southern Illinois University Press; 2008.
32.
go back to reference Chalmers I. The Cochrane collaboration: preparing, maintaining, and disseminating systematic reviews of the effects of health care. Ann N Y Acad Sci. 1993;703:156–63. Discussion 163–5.CrossRef Chalmers I. The Cochrane collaboration: preparing, maintaining, and disseminating systematic reviews of the effects of health care. Ann N Y Acad Sci. 1993;703:156–63. Discussion 163–5.CrossRef
33.
go back to reference Malterud K. Reflexivity and metapositions: strategies for appraisal of clinical evidence. J Eval Clin Pract. 2002;8:121–6.CrossRef Malterud K. Reflexivity and metapositions: strategies for appraisal of clinical evidence. J Eval Clin Pract. 2002;8:121–6.CrossRef
34.
go back to reference Greenhalgh T, Howick J, Maskrey N. Evidence based medicine: a movement in crisis? BMJ. 2014;348:g3725.CrossRef Greenhalgh T, Howick J, Maskrey N. Evidence based medicine: a movement in crisis? BMJ. 2014;348:g3725.CrossRef
35.
go back to reference Pope C. Resisting evidence: the study of evidence-based medicine as a contemporary social movement. Health. 2003;7:267–82.CrossRef Pope C. Resisting evidence: the study of evidence-based medicine as a contemporary social movement. Health. 2003;7:267–82.CrossRef
36.
go back to reference McWhinney IR. Changing models: the impact of Kuhn’s theory on medicine. Fam Pract. 1984;1:3–8.CrossRef McWhinney IR. Changing models: the impact of Kuhn’s theory on medicine. Fam Pract. 1984;1:3–8.CrossRef
37.
go back to reference McWhinney IR. An acquaintance with particulars…”. Fam Med. 1989;21:296–8.PubMed McWhinney IR. An acquaintance with particulars…”. Fam Med. 1989;21:296–8.PubMed
38.
go back to reference Malterud K. The legitimacy of clinical knowledge: towards a medical epistemology embracing the art of medicine. Theor Med. 1995;16:183–98.CrossRef Malterud K. The legitimacy of clinical knowledge: towards a medical epistemology embracing the art of medicine. Theor Med. 1995;16:183–98.CrossRef
39.
go back to reference Montgomery K. How doctors think: clinical judgement and the practice of medicine. Oxford: Oxford University Press; 2006. Montgomery K. How doctors think: clinical judgement and the practice of medicine. Oxford: Oxford University Press; 2006.
40.
go back to reference Malterud K. The art and science of clinical knowledge: evidence beyond measures and numbers. Lancet. 2001;358:397–400.CrossRef Malterud K. The art and science of clinical knowledge: evidence beyond measures and numbers. Lancet. 2001;358:397–400.CrossRef
41.
go back to reference Greenhalgh T. Why do we always end up here? Evidence-based medicine’s conceptual cul-de-sacs and some off-road alternative routes. J Prim Health Care. 2012;4:92–7.PubMed Greenhalgh T. Why do we always end up here? Evidence-based medicine’s conceptual cul-de-sacs and some off-road alternative routes. J Prim Health Care. 2012;4:92–7.PubMed
42.
go back to reference Glenton C, Colvin CJ, Carlsen B, Swartz A, Lewin S, Noyes J, et al. Barriers and facilitators to the implementation of lay health worker programmes to improve access to maternal and child health: qualitative evidence synthesis. Cochrane Database Syst Rev. 2013;10:Cd010414. Glenton C, Colvin CJ, Carlsen B, Swartz A, Lewin S, Noyes J, et al. Barriers and facilitators to the implementation of lay health worker programmes to improve access to maternal and child health: qualitative evidence synthesis. Cochrane Database Syst Rev. 2013;10:Cd010414.
43.
go back to reference Kessel L, Flesner P, Andresen J, Erngaard D, Tendal B, Hjortdal J. Antibiotic prevention of postcataract endophthalmitis: a systematic review and meta-analysis. Acta Ophthalmol. 2015;93:303–17.CrossRef Kessel L, Flesner P, Andresen J, Erngaard D, Tendal B, Hjortdal J. Antibiotic prevention of postcataract endophthalmitis: a systematic review and meta-analysis. Acta Ophthalmol. 2015;93:303–17.CrossRef
44.
go back to reference Lee YJ, Park JE, Jeon BR, Lee SM, Kim SY, Lee YK. Is prostate-specific antigen effective for population screening of prostate cancer? A systematic review. Ann Lab Med. 2013;33:233–41.CrossRef Lee YJ, Park JE, Jeon BR, Lee SM, Kim SY, Lee YK. Is prostate-specific antigen effective for population screening of prostate cancer? A systematic review. Ann Lab Med. 2013;33:233–41.CrossRef
45.
go back to reference Barnett K, Mercer SW, Norbury M, Watt G, Wyke S, Guthrie B. Epidemiology of multimorbidity and implications for health care, research, and medical education: a cross-sectional study. Lancet. 2012;380:37–43.CrossRef Barnett K, Mercer SW, Norbury M, Watt G, Wyke S, Guthrie B. Epidemiology of multimorbidity and implications for health care, research, and medical education: a cross-sectional study. Lancet. 2012;380:37–43.CrossRef
46.
go back to reference Treweek S, Flottorp S, Fretheim A, Havelsrud K, Kristoffersen DT, Oxman A, et al. Guidelines in general practice--are they read and are they used? Tidsskr Nor Laegeforen. 2005;125:300–3. [in Norwegian] Treweek S, Flottorp S, Fretheim A, Havelsrud K, Kristoffersen DT, Oxman A, et al. Guidelines in general practice--are they read and are they used? Tidsskr Nor Laegeforen. 2005;125:300–3. [in Norwegian]
48.
go back to reference Dobrow MJ, Goel V, Upshur RE. Evidence-based health policy: context and utilisation. Soc Sci Med. 2004;58:207–17.CrossRef Dobrow MJ, Goel V, Upshur RE. Evidence-based health policy: context and utilisation. Soc Sci Med. 2004;58:207–17.CrossRef
49.
go back to reference Sturmberg JP, O’Halloran DM, Martin CM. Understanding health system reform – a complex adaptive systems perspective. J Eval Clin Pract. 2012;18:202–8.CrossRef Sturmberg JP, O’Halloran DM, Martin CM. Understanding health system reform – a complex adaptive systems perspective. J Eval Clin Pract. 2012;18:202–8.CrossRef
50.
go back to reference Campbell-Scherer D. The 11th hour--time for EBM to return to first principles? Evid Based Med. 2012;17:103–4.CrossRef Campbell-Scherer D. The 11th hour--time for EBM to return to first principles? Evid Based Med. 2012;17:103–4.CrossRef
51.
go back to reference Shaw R. Broadening the evidence base and the mind when thinking about mixed methods research. Evid Based Med. 2015;20:80.CrossRef Shaw R. Broadening the evidence base and the mind when thinking about mixed methods research. Evid Based Med. 2015;20:80.CrossRef
53.
go back to reference Greenhalgh T, Wieringa S. Is it time to drop the ‘knowledge translation’ metaphor? A critical literature review. J R Soc Med. 2011;104:501–9.CrossRef Greenhalgh T, Wieringa S. Is it time to drop the ‘knowledge translation’ metaphor? A critical literature review. J R Soc Med. 2011;104:501–9.CrossRef
54.
go back to reference Oliver K, Lorenc T, Innvaer S. New directions in evidence-based policy research: a critical analysis of the literature. Health Res Policy Syst. 2014;12:34.CrossRef Oliver K, Lorenc T, Innvaer S. New directions in evidence-based policy research: a critical analysis of the literature. Health Res Policy Syst. 2014;12:34.CrossRef
55.
go back to reference Fernandez A, Sturmberg J, Lukersmith S, Madden R, Torkfar G, Colagiuri R, et al. Evidence-based medicine: is it a bridge too far? Health Res Policy Syst. 2015;13:66.CrossRef Fernandez A, Sturmberg J, Lukersmith S, Madden R, Torkfar G, Colagiuri R, et al. Evidence-based medicine: is it a bridge too far? Health Res Policy Syst. 2015;13:66.CrossRef
56.
go back to reference Haraway D. Situated knowledges; the science question in feminism and the privilege of partial perspective. In: Haraway D, editor. Simians, Cyborgs, and Women: The Reinvention of Nature. New York: Routledge; 1991. p. 183–201. Haraway D. Situated knowledges; the science question in feminism and the privilege of partial perspective. In: Haraway D, editor. Simians, Cyborgs, and Women: The Reinvention of Nature. New York: Routledge; 1991. p. 183–201.
Metadata
Title
Evidence-based medicine – an appropriate tool for evidence-based health policy? A case study from Norway
Authors
Kirsti Malterud
Anne Karen Bjelland
Kari Tove Elvbakken
Publication date
01-12-2016
Publisher
BioMed Central
Published in
Health Research Policy and Systems / Issue 1/2015
Electronic ISSN: 1478-4505
DOI
https://doi.org/10.1186/s12961-016-0088-1

Other articles of this Issue 1/2015

Health Research Policy and Systems 1/2015 Go to the issue