Skip to main content
Top
Published in: BMC Health Services Research 1/2019

Open Access 01-12-2019 | Research article

Cross-cultural adaptation, internal consistency, test-retest reliability and feasibility of the German version of the evidence-based practice inventory

Authors: Tobias Braun, Katja Ehrenbrusthoff, Carolin Bahns, Lisa Happe, Christian Kopkow

Published in: BMC Health Services Research | Issue 1/2019

Login to get access

Abstract

Background

A psychometrically robust measurement instrument is prerequisite to tailor and monitor interventions aiming to improve evidence-based practice (EBP). The recently developed “Evidence-based Practice Inventory” (EBPI) questionnaire (five dimensions) provides a sound inventory for a comprehensive assessment of adherence to EBP, and identification of barriers and facilitators for EBP. The aims of this study were to establish a German language version of the EBPI and to examine the instrument’s reliability in a diverse sample of healthcare professionals.

Methods

The English version of the EBPI was translated, adopted and subsequently test-retest reliability of the German language EBPI was examined in a nationwide online survey. Participants working in Germany were invited to complete the questionnaire twice.
For each EBPI dimension, internal consistency reliability (Cronbach’s alpha) and the relative test-retest reliability (intraclass correlation coefficient, ICC) were calculated. The standard error of measurement, limits of agreement and minimal detectable change values were estimated to quantify measurement error.

Results

A German language version of the EBPI was established. In the online survey, the EBPI was initially completed by 889 healthcare professionals. At follow-up, 344 individuals (39%) completed the questionnaire (74% female; mean work experience: 13.6 years).
The ICCs for the five dimensions varied between 0.78 and 0.86. The standard error of measurement varied between 6.5 and 8.8% of the respective dimension scale range, and the limits of agreement between 24 and 37%. For internal consistency reliability, alpha varied between 0.64 and 0.90. There were neither floor nor ceiling effects, nor any other relevant feasibility issues.

Conclusions

The German language EBPI can be used to assess EBP adherence of healthcare professionals in clinical practice, and to identify barriers and facilitators for an EBP conform behaviour. Results on test-retest reliability indicate that the EBPI produces reliable scores when used for group comparisons, but the questionnaire seems insufficiently reliable for individual measurements over time. Methods of item response theory or Rasch measurement theory should be used for further evaluation and revision of the EBPI, informed by the results of this study.

Trial registration

German Clinical Trials Register (DRKS00013792). Registered 19 January 2018.
Appendix
Available only for authorised users
Literature
1.
go back to reference Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isn't. BMJ. 1996;312:71–2.CrossRef Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isn't. BMJ. 1996;312:71–2.CrossRef
2.
go back to reference Albarqouni L, Hoffmann T, Straus S, Olsen NR, Young T, Ilic D, et al. Core competencies in evidence-based practice for health professionals. JAMA Netw Open. 2018;1:e180281.CrossRef Albarqouni L, Hoffmann T, Straus S, Olsen NR, Young T, Ilic D, et al. Core competencies in evidence-based practice for health professionals. JAMA Netw Open. 2018;1:e180281.CrossRef
3.
go back to reference Haynes RB, Devereaux PJ, Guyatt GH. Clinical expertise in the era of evidence-based medicine and patient choice. ACP J Club. 2002;136:A11–4.PubMed Haynes RB, Devereaux PJ, Guyatt GH. Clinical expertise in the era of evidence-based medicine and patient choice. ACP J Club. 2002;136:A11–4.PubMed
4.
go back to reference Dawes M, Summerskill W, Glasziou P, Cartabellotta A, Martin J, Hopayian K, et al. Sicily statement on evidence-based practice. BMC Med Educ. 2005;5:1.CrossRef Dawes M, Summerskill W, Glasziou P, Cartabellotta A, Martin J, Hopayian K, et al. Sicily statement on evidence-based practice. BMC Med Educ. 2005;5:1.CrossRef
5.
go back to reference Heiwe S, Kajermo KN, Tyni-Lenné R, Guidetti S, Samuelsson M, Andersson I-L, Wengström Y. Evidence-based practice: attitudes, knowledge and behaviour among allied health care professionals. Int J Qual Health Care. 2011;23:198–209.CrossRef Heiwe S, Kajermo KN, Tyni-Lenné R, Guidetti S, Samuelsson M, Andersson I-L, Wengström Y. Evidence-based practice: attitudes, knowledge and behaviour among allied health care professionals. Int J Qual Health Care. 2011;23:198–209.CrossRef
6.
go back to reference Upton D, Stephens D, Williams B, Scurlock-Evans L. Occupational Therapists' attitudes, knowledge, and implementation of evidence-based practice: a systematic review of published research. Br J Occup Ther. 2014;77:24–38.CrossRef Upton D, Stephens D, Williams B, Scurlock-Evans L. Occupational Therapists' attitudes, knowledge, and implementation of evidence-based practice: a systematic review of published research. Br J Occup Ther. 2014;77:24–38.CrossRef
7.
go back to reference Smith CA, Coyle ME, de LS, Johnson NP. Evidence-based research and practice: attitudes of reproduction nurses, counsellors and doctors. Reprod BioMed Online. 2014;29:3–9.CrossRef Smith CA, Coyle ME, de LS, Johnson NP. Evidence-based research and practice: attitudes of reproduction nurses, counsellors and doctors. Reprod BioMed Online. 2014;29:3–9.CrossRef
8.
go back to reference Young JM, Ward JE. Evidence-based medicine in general practice: beliefs and barriers among Australian GPs. J Eval Clin Pract. 2001;7:201–10.CrossRef Young JM, Ward JE. Evidence-based medicine in general practice: beliefs and barriers among Australian GPs. J Eval Clin Pract. 2001;7:201–10.CrossRef
9.
go back to reference Jette D, Bacon K, Batty C, Carlson M, Ferland A, Hemingway R, et al. Evidence-based practice: beliefs, attitudes, knowledge, and behaviors of physical therapists. Phys Ther. 2003;83:786–805.PubMed Jette D, Bacon K, Batty C, Carlson M, Ferland A, Hemingway R, et al. Evidence-based practice: beliefs, attitudes, knowledge, and behaviors of physical therapists. Phys Ther. 2003;83:786–805.PubMed
10.
go back to reference Young T, Rohwer A, Volmink J, Clarke M. What are the effects of teaching evidence-based health care (EBHC)? Overview of systematic reviews. PLoS One. 2014;9:e86706.CrossRef Young T, Rohwer A, Volmink J, Clarke M. What are the effects of teaching evidence-based health care (EBHC)? Overview of systematic reviews. PLoS One. 2014;9:e86706.CrossRef
11.
go back to reference Mokkink LB, Terwee CB, Patrick DL, Alonso J, Stratford PW, Knol DL, et al. The COSMIN study reached international consensus on taxonomy, terminology, and definitions of measurement properties for health-related patient-reported outcomes. J Clin Epidemiol. 2010;63:737–45.CrossRef Mokkink LB, Terwee CB, Patrick DL, Alonso J, Stratford PW, Knol DL, et al. The COSMIN study reached international consensus on taxonomy, terminology, and definitions of measurement properties for health-related patient-reported outcomes. J Clin Epidemiol. 2010;63:737–45.CrossRef
12.
go back to reference de Vet HCW, Terwee CB, Mokkink LB, Knol DL. Measurement in medicine: a practical guide. Cambridge, New York: Cambridge University Press; 2011.CrossRef de Vet HCW, Terwee CB, Mokkink LB, Knol DL. Measurement in medicine: a practical guide. Cambridge, New York: Cambridge University Press; 2011.CrossRef
13.
go back to reference Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet. 1986;1:307–10.CrossRef Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet. 1986;1:307–10.CrossRef
14.
go back to reference Leung K, Trevena L, Waters D. Systematic review of instruments for measuring nurses' knowledge, skills and attitudes for evidence-based practice. J Adv Nurs. 2014;70:2181–95.CrossRef Leung K, Trevena L, Waters D. Systematic review of instruments for measuring nurses' knowledge, skills and attitudes for evidence-based practice. J Adv Nurs. 2014;70:2181–95.CrossRef
15.
go back to reference Shaneyfelt T, Baum KD, Bell D, Feldstein D, Houston TK, Kaatz S, et al. Instruments for evaluating education in evidence-based practice: a systematic review. JAMA. 2006;296:1116–27.CrossRef Shaneyfelt T, Baum KD, Bell D, Feldstein D, Houston TK, Kaatz S, et al. Instruments for evaluating education in evidence-based practice: a systematic review. JAMA. 2006;296:1116–27.CrossRef
16.
go back to reference Upton D, Upton P. Development of an evidence-based practice questionnaire for nurses. J Adv Nurs. 2006;53:454–8.CrossRef Upton D, Upton P. Development of an evidence-based practice questionnaire for nurses. J Adv Nurs. 2006;53:454–8.CrossRef
17.
go back to reference Koehn ML, Lehman K. Nurses' perceptions of evidence-based nursing practice. J Adv Nurs. 2008;62:209–15.CrossRef Koehn ML, Lehman K. Nurses' perceptions of evidence-based nursing practice. J Adv Nurs. 2008;62:209–15.CrossRef
18.
go back to reference Fernández-Domínguez JC, Sesé-Abad A, Morales-Asencio JM, Sastre-Fullana P, Pol-Castañeda S, de Pedro-Gómez JE. Content validity of a health science evidence-based practice questionnaire (HS-EBP) with a web-based modified Delphi approach. Int J Qual Health Care. 2016;28:764–73.PubMed Fernández-Domínguez JC, Sesé-Abad A, Morales-Asencio JM, Sastre-Fullana P, Pol-Castañeda S, de Pedro-Gómez JE. Content validity of a health science evidence-based practice questionnaire (HS-EBP) with a web-based modified Delphi approach. Int J Qual Health Care. 2016;28:764–73.PubMed
19.
go back to reference Fernández-Domínguez JC, de Pedro-Gómez JE, Morales-Asencio JM, Bennasar-Veny M, Sastre-Fullana P, Sesé-Abad A. Health sciences-evidence based practice questionnaire (HS-EBP) for measuring transprofessional evidence-based practice: creation, development and psychometric validation. PLoS One. 2017;12:e0177172.CrossRef Fernández-Domínguez JC, de Pedro-Gómez JE, Morales-Asencio JM, Bennasar-Veny M, Sastre-Fullana P, Sesé-Abad A. Health sciences-evidence based practice questionnaire (HS-EBP) for measuring transprofessional evidence-based practice: creation, development and psychometric validation. PLoS One. 2017;12:e0177172.CrossRef
20.
go back to reference Kaper NM, Swennen MHJ, van Wijk AJ, Kalkman CJ, van Rheenen N, van der Graaf Y, van der Heijden GJMG. The "evidence-based practice inventory": reliability and validity was demonstrated for a novel instrument to identify barriers and facilitators for evidence based practice in health care. J Clin Epidemiol. 2015;68:1261–9.CrossRef Kaper NM, Swennen MHJ, van Wijk AJ, Kalkman CJ, van Rheenen N, van der Graaf Y, van der Heijden GJMG. The "evidence-based practice inventory": reliability and validity was demonstrated for a novel instrument to identify barriers and facilitators for evidence based practice in health care. J Clin Epidemiol. 2015;68:1261–9.CrossRef
21.
go back to reference Epstein J, Santo RM, Guillemin F. A review of guidelines for cross-cultural adaptation of questionnaires could not bring out a consensus. J Clin Epidemiol. 2015;68:435–41.CrossRef Epstein J, Santo RM, Guillemin F. A review of guidelines for cross-cultural adaptation of questionnaires could not bring out a consensus. J Clin Epidemiol. 2015;68:435–41.CrossRef
22.
go back to reference von Elm E, Altman DG, Egger M, Pocock SJ, Gotzsche PC, Vandenbroucke JP. The strengthening the reporting of observational studies in epidemiology (STROBE) statement: guidelines for reporting observational studies. J Clin Epidemiol. 2008;61:344–9.CrossRef von Elm E, Altman DG, Egger M, Pocock SJ, Gotzsche PC, Vandenbroucke JP. The strengthening the reporting of observational studies in epidemiology (STROBE) statement: guidelines for reporting observational studies. J Clin Epidemiol. 2008;61:344–9.CrossRef
23.
go back to reference Eysenbach G. Improving the quality of web surveys: the checklist for reporting results of internet E-surveys (CHERRIES). J Med Internet Res. 2004;6:e34.CrossRef Eysenbach G. Improving the quality of web surveys: the checklist for reporting results of internet E-surveys (CHERRIES). J Med Internet Res. 2004;6:e34.CrossRef
24.
go back to reference Mokkink LB, de Vet HCW, Prinsen CAC, Patrick DL, Alonso J, Bouter LM, Terwee CB. COSMIN risk of Bias checklist for systematic reviews of patient-reported outcome measures. Qual Life Res. 2018;27:1171–9.CrossRef Mokkink LB, de Vet HCW, Prinsen CAC, Patrick DL, Alonso J, Bouter LM, Terwee CB. COSMIN risk of Bias checklist for systematic reviews of patient-reported outcome measures. Qual Life Res. 2018;27:1171–9.CrossRef
25.
go back to reference Kottner J, Audige L, Brorson S, Donner A, Gajewski BJ, Hrobjartsson A, et al. Guidelines for reporting reliability and agreement studies (GRRAS) were proposed. J Clin Epidemiol. 2011;64:96–106.CrossRef Kottner J, Audige L, Brorson S, Donner A, Gajewski BJ, Hrobjartsson A, et al. Guidelines for reporting reliability and agreement studies (GRRAS) were proposed. J Clin Epidemiol. 2011;64:96–106.CrossRef
26.
go back to reference Beaton DE, Bombardier C, Guillemin F, Ferraz MB. Guidelines for the process of cross-cultural adaptation of self-report measures. Spine. 2000;25:3186–91.CrossRef Beaton DE, Bombardier C, Guillemin F, Ferraz MB. Guidelines for the process of cross-cultural adaptation of self-report measures. Spine. 2000;25:3186–91.CrossRef
27.
go back to reference Pohontsch N, Meyer T. Cognitive interviewing – a tool to develop and validate questionnaires. Rehabilitation (Stuttg). 2015;54:53–9.CrossRef Pohontsch N, Meyer T. Cognitive interviewing – a tool to develop and validate questionnaires. Rehabilitation (Stuttg). 2015;54:53–9.CrossRef
28.
go back to reference Willis GB, Artino AR. What do our respondents think We're asking? Using cognitive interviewing to improve medical education surveys. J Grad Med Educ. 2013;5:353–6.CrossRef Willis GB, Artino AR. What do our respondents think We're asking? Using cognitive interviewing to improve medical education surveys. J Grad Med Educ. 2013;5:353–6.CrossRef
29.
go back to reference Terwee CB, Mokkink LB, Knol DL, Ostelo RWJG, Bouter LM. Vet HCW de. Rating the methodological quality in systematic reviews of studies on measurement properties: a scoring system for the COSMIN checklist. Qual Life Res. 2012;21:651–7.CrossRef Terwee CB, Mokkink LB, Knol DL, Ostelo RWJG, Bouter LM. Vet HCW de. Rating the methodological quality in systematic reviews of studies on measurement properties: a scoring system for the COSMIN checklist. Qual Life Res. 2012;21:651–7.CrossRef
30.
go back to reference Hobart JC, Cano SJ, Warner TT, Thompson AJ. What sample sizes for reliability and validity studies in neurology? J Neurol. 2012;259:2681–94.CrossRef Hobart JC, Cano SJ, Warner TT, Thompson AJ. What sample sizes for reliability and validity studies in neurology? J Neurol. 2012;259:2681–94.CrossRef
31.
go back to reference Feldt LS, Woodruff DJ, Salih FA. Statistical inference for coefficient alpha. Appl Psychol Meas. 1987;11:93–103.CrossRef Feldt LS, Woodruff DJ, Salih FA. Statistical inference for coefficient alpha. Appl Psychol Meas. 1987;11:93–103.CrossRef
32.
go back to reference Terwee CB, Bot SDM, de Boer MR, van der Windt DAWM, Knol DL, Dekker J, et al. Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol. 2007;60:34–42.CrossRef Terwee CB, Bot SDM, de Boer MR, van der Windt DAWM, Knol DL, Dekker J, et al. Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol. 2007;60:34–42.CrossRef
33.
go back to reference Scientific Advisory Committee of the Medical Outcomes Trust. Assessing health status and quality-of-life instruments: attributes and review criteria. Qual Life Res. 2002;11:193–205.CrossRef Scientific Advisory Committee of the Medical Outcomes Trust. Assessing health status and quality-of-life instruments: attributes and review criteria. Qual Life Res. 2002;11:193–205.CrossRef
34.
go back to reference Altman DG, Bland JM. Measurement in medicine: the analysis of method comparison studies. The Statistician. 1983;32:307–17.CrossRef Altman DG, Bland JM. Measurement in medicine: the analysis of method comparison studies. The Statistician. 1983;32:307–17.CrossRef
35.
go back to reference Brehm MA, Scholtes VA, Dallmeijer AJ, Twisk JW, Harlaar J. The importance of addressing heteroscedasticity in the reliability analysis of ratio-scaled variables: an example based on walking energy-cost measurements. Dev Med Child Neurol. 2012;54:267–73.CrossRef Brehm MA, Scholtes VA, Dallmeijer AJ, Twisk JW, Harlaar J. The importance of addressing heteroscedasticity in the reliability analysis of ratio-scaled variables: an example based on walking energy-cost measurements. Dev Med Child Neurol. 2012;54:267–73.CrossRef
36.
go back to reference Euser AM, Dekker FW, Le Cessie S. A practical approach to Bland-Altman plots and variation coefficients for log transformed variables. J Clin Epidemiol. 2008;61:978–82.CrossRef Euser AM, Dekker FW, Le Cessie S. A practical approach to Bland-Altman plots and variation coefficients for log transformed variables. J Clin Epidemiol. 2008;61:978–82.CrossRef
37.
go back to reference Stratford PW, Binkley JM, Riddle DL. Health status measures: strategies and analytic methods for assessing change scores. Phys Ther. 1996;76:1109–23.CrossRef Stratford PW, Binkley JM, Riddle DL. Health status measures: strategies and analytic methods for assessing change scores. Phys Ther. 1996;76:1109–23.CrossRef
38.
go back to reference Pallant J. SPSS survival manual: a step by step guide to data analysis using SPSS. Crows Nest: Allen & Unwin; 2011. Pallant J. SPSS survival manual: a step by step guide to data analysis using SPSS. Crows Nest: Allen & Unwin; 2011.
39.
go back to reference van Baalen B, Odding E, van Woensel MPC, Roebroeck ME. Reliability and sensitivity to change of measurement instruments used in a traumatic brain injury population. Clin Rehabil. 2006;20:686–700.CrossRef van Baalen B, Odding E, van Woensel MPC, Roebroeck ME. Reliability and sensitivity to change of measurement instruments used in a traumatic brain injury population. Clin Rehabil. 2006;20:686–700.CrossRef
40.
go back to reference Epstein J, Osborne RH, Elsworth GR, Beaton DE, Guillemin F. Cross-cultural adaptation of the health education impact questionnaire: experimental study showed expert committee, not back-translation, added value. J Clin Epidemiol. 2015;68:360–9.CrossRef Epstein J, Osborne RH, Elsworth GR, Beaton DE, Guillemin F. Cross-cultural adaptation of the health education impact questionnaire: experimental study showed expert committee, not back-translation, added value. J Clin Epidemiol. 2015;68:360–9.CrossRef
44.
go back to reference Dillman DA. Mail and internet surveys: the tailored design method--2007 update with new internet, visual, and mixed-mode guide. Hoboken: Wiley; 2011. Dillman DA. Mail and internet surveys: the tailored design method--2007 update with new internet, visual, and mixed-mode guide. Hoboken: Wiley; 2011.
46.
go back to reference Dysart AM, Tomlin GS. Factors related to evidence-based practice among US occupational therapy clinicians. Am J Occup Ther. 2002;56:275–84.CrossRef Dysart AM, Tomlin GS. Factors related to evidence-based practice among US occupational therapy clinicians. Am J Occup Ther. 2002;56:275–84.CrossRef
47.
go back to reference Gerst T, Hibbeler B. Gesundheitsfachberufe: Auf dem Weg in die Akademisierung. Deutsches Ärzteblatt. 2012;109:A2458–61. Gerst T, Hibbeler B. Gesundheitsfachberufe: Auf dem Weg in die Akademisierung. Deutsches Ärzteblatt. 2012;109:A2458–61.
48.
go back to reference Terwee CB, Prinsen CAC, Chiarotto A, Westerman MJ, Patrick DL, Alonso J, et al. COSMIN methodology for evaluating the content validity of patient-reported outcome measures: a Delphi study. Qual Life Res. 2018;27:1159–70.CrossRef Terwee CB, Prinsen CAC, Chiarotto A, Westerman MJ, Patrick DL, Alonso J, et al. COSMIN methodology for evaluating the content validity of patient-reported outcome measures: a Delphi study. Qual Life Res. 2018;27:1159–70.CrossRef
49.
go back to reference Watine J. What sort of bioethical values are the evidence-based medicine and the GRADE approaches willing to deal with? J Med Ethics. 2011;37:184–6.CrossRef Watine J. What sort of bioethical values are the evidence-based medicine and the GRADE approaches willing to deal with? J Med Ethics. 2011;37:184–6.CrossRef
50.
go back to reference Watine J, Wils J, Augereau C. Clinical practice guidelines: potential misconceptions of the GRADE approach. J Clin Epidemiol. 2014;67:7–9.CrossRef Watine J, Wils J, Augereau C. Clinical practice guidelines: potential misconceptions of the GRADE approach. J Clin Epidemiol. 2014;67:7–9.CrossRef
51.
go back to reference Prinsen CAC, Mokkink LB, Bouter LM, Alonso J, Patrick DL, de Vet HCW, Terwee CB. COSMIN guideline for systematic reviews of patient-reported outcome measures. Qual Life Res. 2018;27:1147–57.CrossRef Prinsen CAC, Mokkink LB, Bouter LM, Alonso J, Patrick DL, de Vet HCW, Terwee CB. COSMIN guideline for systematic reviews of patient-reported outcome measures. Qual Life Res. 2018;27:1147–57.CrossRef
52.
go back to reference Petrillo J, Cano SJ, McLeod LD, Coon CD. Using classical test theory, item response theory, and Rasch measurement theory to evaluate patient-reported outcome measures: a comparison of worked examples. Value Health. 2015;18:25–34.CrossRef Petrillo J, Cano SJ, McLeod LD, Coon CD. Using classical test theory, item response theory, and Rasch measurement theory to evaluate patient-reported outcome measures: a comparison of worked examples. Value Health. 2015;18:25–34.CrossRef
53.
go back to reference Hobart JC, Cano SJ, Zajicek JP, Thompson AJ. Rating scales as outcome measures for clinical trials in neurology: problems, solutions, and recommendations. Lancet Neurol. 2007;6:1094–105.CrossRef Hobart JC, Cano SJ, Zajicek JP, Thompson AJ. Rating scales as outcome measures for clinical trials in neurology: problems, solutions, and recommendations. Lancet Neurol. 2007;6:1094–105.CrossRef
Metadata
Title
Cross-cultural adaptation, internal consistency, test-retest reliability and feasibility of the German version of the evidence-based practice inventory
Authors
Tobias Braun
Katja Ehrenbrusthoff
Carolin Bahns
Lisa Happe
Christian Kopkow
Publication date
01-12-2019
Publisher
BioMed Central
Published in
BMC Health Services Research / Issue 1/2019
Electronic ISSN: 1472-6963
DOI
https://doi.org/10.1186/s12913-019-4273-0

Other articles of this Issue 1/2019

BMC Health Services Research 1/2019 Go to the issue