Top

Published in:

Open Access 01-12-2016 | Systematic review

Psychometric properties of implementation measures for public health and community settings and mapping of constructs against the Consolidated Framework for Implementation Research: a systematic review

Authors: Tara Clinton-McHarg, Sze Lin Yoong, Flora Tzelepis, Tim Regan, Alison Fielding, Eliza Skelton, Melanie Kingsland, Jia Ying Ooi, Luke Wolfenden

Published in: Implementation Science | Issue 1/2015

Abstract

Background

Recent reviews have synthesised the psychometric properties of measures developed to examine implementation science constructs in healthcare and mental health settings. However, no reviews have focussed primarily on the properties of measures developed to assess innovations in public health and community settings. This review identified quantitative measures developed in public health and community settings, examined their psychometric properties, and described how the domains of each measure align with the five domains and 37 constructs of the Consolidated Framework for Implementation Research (CFIR).

Methods

MEDLINE, PsycINFO, EMBASE, and CINAHL were searched to identify publications describing the development of measures to assess implementation science constructs in public health and community settings. The psychometric properties of each measure were assessed against recommended criteria for validity (face/content, construct, criterion), reliability (internal consistency, test-retest), responsiveness, acceptability, feasibility, and revalidation and cross-cultural adaptation. Relevant domains were mapped against implementation constructs defined by the CFIR.

Results

Fifty-one measures met the inclusion criteria. The majority of these were developed in schools, universities, or colleges and other workplaces or organisations. Overall, most measures did not adequately assess or report psychometric properties. Forty-six percent of measures using exploratory factor analysis reported >50 % of variance was explained by the final model; none of the measures assessed using confirmatory factor analysis reported root mean square error of approximation (<0.06) or comparative fit index (>0.95). Fifty percent of measures reported Cronbach’s alpha of <0.70 for at least one domain; 6 % adequately assessed test-retest reliability; 16 % of measures adequately assessed criterion validity (i.e. known-groups); 2 % adequately assessed convergent validity (r > 0.40). Twenty-five percent of measures reported revalidation or cross-cultural validation. The CFIR constructs most frequently assessed by the included measures were relative advantage, available resources, knowledge and beliefs, complexity, implementation climate, and other personal resources (assessed by more than ten measures). Five CFIR constructs were not addressed by any measure.

Conclusions

This review highlights gaps in the range of implementation constructs that are assessed by existing measures developed for use in public health and community settings. Moreover, measures with robust psychometric properties are lacking. Without rigorous tools, the factors associated with the successful implementation of innovations in these settings will remain unknown

Available only for authorised users

Davies P, Walker A, Grimshaw J. A systematic review of the use of theory in the design of guideline dissemination and implementation strategies and interpretation of the results of rigorous evaluations. Implement Sci. 2010;5:14.PubMedPubMedCentral

Nilsen P. Making sense of implementation theories, models and frameworks. Implement Sci. 2015;10:53.PubMedPubMedCentral

Tabak RG, Khoong EC, Chambers D, Brownson RC. Bridging research and practice: models for dissemination and implementation research. Am J Prev Med. 2012;43:337–50.PubMedPubMedCentral

Martinez RG, Lewis CC, Weiner BJ. Instrumentation issues in implementation science. Implement Sci. 2014;9:118.PubMedPubMedCentral

Rabin BA, Purcell P, Naveed S, Moser RP, Henton MD, Proctor EK, Brownson RC, Glasgow RE. Advancing the application, quality and harmonization of implementation science measures. Implement Sci. 2012;7:119.PubMedPubMedCentral

Brennan SE, Bosch M, Buchan H, Green SE. Measuring organizational and individual factors thought to influence the success of quality improvement in primary care: a systematic review of instruments. Implement Sci. 2012;7:121.PubMedPubMedCentral

Chaudoir SR, Dugan AG, Barr CH. Measuring factors affecting implementation of health innovations: a systematic review of structural, organizational, provider, patient, and innovation level measures. Implement Sci. 2013;8:22.PubMedPubMedCentral

Chor KHB, Wisdom JP, Olin SCS, Hoagwood KE, Horwitz SM. Measures for predictors of innovation adoption. Adm Policy Ment Health. 2014;42:545–73.

Scott T, Mannion R, Davies H, Marshall M. The quantitative measurement of organizational culture in health care: a review of the available instruments. Health Serv Res. 2003;38:923–45.PubMedPubMedCentral

10.

Weiner BJ, Amick H, Lee SYD. Conceptualization and measurement of organizational readiness for change: a review of the literature in health services research and other fields. Med Care Res Rev. 2008;65:379–436.PubMed

11.

King T, Byers JF. A review of organizational culture instruments for nurse executives. J Nurs Adm. 2007;37:21.PubMed

12.

Emmons KM, Weiner B, Fernandez ME, Tu SP. Systems antecedents for dissemination and implementation: a review and analysis of measures. Health Educ Behav. 2012;39:87–105.PubMed

13.

Squires JE, Estabrooks CA, Gustavsson P, Wallin L. Individual determinants of research utilization by nurses: a systematic review update. Implement Sci. 2011;6:1.PubMedPubMedCentral

14.

McDowell I. Measuring health: a guide to rating scales and questionnaires. New York: Oxford University Press; 2006.

15.

Hersen M. Clinician’s handbook of adult behavioral assessment. Boston: Elsevier Academic Press; 2006.

16.

Lewis CC, Fischer S, Weiner BJ, Stanick C, Kim M, Martinez RG. Outcomes for implementation science: an enhanced systematic review of instruments using evidence-based rating criteria. Implement Sci. 2015;10:155.PubMedPubMedCentral

17.

Lewis CC, Stanick CF, Martinez RG, Weiner BJ, Kim M, Barwick M, Comtois KA. The Society for Implementation Research Collaboration Instrument Review Project: a methodology to promote rigorous evaluation. Implement Sci. 2015;10:2.PubMedPubMedCentral

18.

Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, Griffey R, Hensley M. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health. 2011;38:65–76.PubMed

19.

Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4:50.PubMedPubMedCentral

20.

The SIRC Instrument Review Project (IRP): A systematic review and synthesis of implementation science instruments [http://www.societyforimplementationresearchcollaboration.org/sirc-projects/sirc-instrument-project]

21.

Sampson M, McGowan J, Cogo E, Grimshaw J, Moher D, Lefebvre C. An evidence-based practice guideline for the peer review of electronic search strategies. J Clin Epidemiol. 2009;62:944–52.PubMed

22.

Jenuwine E, Floyd J. Comparison of Medical Subject Headings and text-word searches in MEDLINE to retrieve studies on sleep in healthy individuals. J Med Libr Assoc. 2004;92:349–53.PubMedPubMedCentral

23.

Clinton-McHarg T, Carey M, Sanson-Fisher R, Shakeshaft A, Rainbird K. Measuring the psychosocial health of adolescent and young adult (AYA) cancer survivors: a critical review. Health Qual Life Outcomes. 2010;8:25.PubMedPubMedCentral

24.

Tzelepis F, Rose SK, Sanson-Fisher RW, Clinton-McHarg T, Carey ML, Paul CL. Are we missing the Institute of Medicine’s mark? A systematic review of patient-reported outcome measures assessing quality of patient-centred cancer care. BMC Cancer. 2014;14:41.PubMedPubMedCentral

25.

American Educational Research Association, American Psychological Association, National Council on Measurement in Education. Standards for educational and psychological testing. Washington, DC: American Educational Research Association; 2014.

26.

Mokkink L, Terwee C, Patrick D, Alonso J, Stratford P, Knol D, Bouter L, de Vet H. The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments: an international Delphi study. Qual Life Res. 2010;19:539–49.PubMedPubMedCentral

27.

Anastasi A, Urbina S. Psychological testing. Upper Saddle River: Prentice Hall; 1997.

28.

Lohr KN, Aaronson NK, Alonso J, Audrey-Burnam M, Patrick DL, Perrin EB, Roberts JS. Evaluating quality-of-life and health status instruments: development of scientific review criteria. Clin Ther. 1996;18:979–92.PubMed

29.

Kaiser HF. Directional statistical decisions. Psychol Rev. 1960;67:160–7.PubMed

30.

Tabachnick BG, Fidell LS. Using multivariate statistics. Boston: Pearson; 2013.

31.

Hu L, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Struct Equation Model. 1999;6:1–55.

32.

Schmitt TA. Current methodological considerations in exploratory and confirmatory factor analysis. J Psychoeduc Assess. 2011;29:304–21.

33.

Cohen J. Statistical power analysis for the behavioral sciences. Hillsdale: Lawrence Erlbaum Associates; 1988.

34.

Rubin A, Bellamy J. Practitioner’s guide to using research for evidence-based practice. Hoboken: John Wiley & Sons; 2012.

35.

Marx RG, Menezes A, Horovitz L, Jones EC, Warren RF. A comparison of two time intervals for test-retest reliability of health status instruments. J Clin Epidemiol. 2003;56:730–5.PubMed

36.

Streiner DL, Norman GR. Health measurement scales: a practical guide to their development and use. New York: Oxford University Press; 2008.

37.

Pedhazur EJ, Schmelkin LP. Measurement, design, and analysis: an integrated approach. Hillsdale: Lawrence Erlbaum Associates; 1991.

38.

Aldridge JM, Laugksch RC, Fraser BJ. School-level environment and outcomes-based education in South Africa. Learn Environ Res. 2006;9:123–47.

39.

Bowen GL, Rose RA, Ware WB. The reliability and validity of the school success profile learning organization measure. Eval Program Plann. 2006;29:97–104.

40.

Canfield JP, Teasley ML, Abell N, Randolph KA. Validation of a Mckinney-Vento Act implementation scale. Res Soc Work Pract. 2012;22:410–9.

41.

Chatterji M. Measuring leader perceptions of school readiness for reforms: use of an iterative model combining classical and Rasch methods. J Appl Meas. 2001;3:455–85.

42.

Deschesnes M, Trudeau F, Kebe M. Psychometric properties of a scale focusing on perceived attributes of a health promoting school approach. Can J Public Health. 2009;100:389–92.PubMed

43.

Gingiss PL, Gottlieb NH, Brink SG. Measuring cognitive characteristics associated with adoption and implementation of health innovations in schools. Am J Health Promot. 1994;8:294–301.

44.

Gingiss PL, Gottlieb NH, Brink SG. Increasing teacher receptivity toward use of tobacco prevention education programs. J Drug Educ. 1994;24:163–76.PubMed

45.

Hayes DM. Toward the development and validation of a curriculum coordinator Role-efficacy Belief Instrument for sexuality education. J Sex Educ Ther. 1992;18:127–35.

46.

Hume A, McIntosh K. Construct validation of a measure to assess sustainability of school-wide behavior interventions. Psychol Sch. 2013;50:1003–14.

47.

Kingery PM, Holcomb JD, Jibaja-Rusth M, Pruitt BE, Buckner WP. The health teaching self-efficacy scale. J Health Educ. 1994;25:68–76.

48.

Lambert LG, Monroe A, Wolff L. Mississippi elementary school teachers’ perspectives on providing nutrition competencies under the framework of their school wellness policy. J Nutr Educ Behav. 2010;42:271–6.PubMed

49.

McIntosh K, MacKay LD, Hume AE, Doolittle J, Vincent CG, Horner RH, Ervin RA. Development and initial validation of a measure to assess factors related to sustainability of school-wide positive behavior support. J Posit Behav Interv. 2011;13:208–18.

50.

Mellin EA, Bronstein L, Anderson-Butcher D, Amorose AJ, Ball A, Green J. Measuring interprofessional team collaboration in expanded school mental health: model refinement and scale development. J Interprof Care. 2010;24:514–23.PubMed

51.

Steckler A, Goodman RM, McLeroy KR, Davis S, Koch G. Measuring the diffusion of innovative health promotion programs. Am J Health Promot. 1992;6:214–24.PubMed

52.

Tuytens M, Devos G. Teachers’ perception of the new teacher evaluation policy: a validity study of the policy characteristics scale. Teach Teach Educ. 2009;25:924–30.

53.

Atkinson NL. Developing a questionnaire to measure perceived attributes of eHealth innovations. Am J Health Behav. 2007;31:612–21.PubMed

54.

Chung KC. Gender, culture and determinants of behavioural intents to adopt mobile commerce among the Y generation in transition economies: evidence from Kazakhstan. Behav Inf Technol. 2014;33:743–56.

55.

Chung K-C, Holdsworth DK. Culture and behavioural intent to adopt mobile commerce among the Y Generation: comparative analyses between Kazakhstan, Morocco and Singapore. Young Consumers. 2012;13:224–41.

56.

Davis FD. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Q. 1989;13:319–40.

57.

Pillay H, Irving K, McCrindle A. Developing a diagnostic tool for assessing tertiary students’ readiness for online learning. Int J Learn Tech. 2006;2:92–104.

58.

Pillay H, Irving K, Tones M. Validation of the diagnostic tool for assessing tertiary students’ readiness for online learning. High Educ Res Dev. 2007;26:217–34.

59.

Saeed KA, Abdinnour S. Understanding post-adoption IS usage stages: an empirical assessment of self-service information systems. Inf Syst J. 2013;23:219–44.

60.

Talukder M, Quazi A. The impact of social influence on individuals’ adoption of innovation. J Org Comp Elect Com. 2011;21:111–35.

61.

Fang Y, Yang S, Feng B, Ni Y, Zhang K. Pharmacists’ perception of pharmaceutical care in community pharmacy: a questionnaire survey in Northwest China. Health Soc Care Community. 2011;19:189–97.PubMed

62.

Kansanaho HM, Puumalainen II, Varunki MM, Airaksinen MSA, Aslani P. Attitudes of Finnish community pharmacists toward concordance. Ann Pharmacother. 2004;38:1946–53.PubMed

63.

Roberts AS, Benrimoj SI, Chen TF, Williams KA, Aslani P. Practice change in community pharmacy: quantification of facilitators. Ann Pharmacother. 2008;42:861–8.PubMed

64.

Cochran JK, Bromley ML, Swando MJ. Sheriff’s deputies’ receptivity to organizational change. Policing. 2002;25:507–29.

65.

Taxman FS, Henderson C, Young D, Farrell J. The impact of training interventions on organizational readiness to support innovations in juvenile justice offices. Adm Policy Ment Health. 2014;41:177–88.PubMedPubMedCentral

66.

Christensson L, Unosson M, Bachrach-Lindstrom M, Ek AC. Attitudes of nursing staff towards nutritional nursing care. Scand J Caring Sci. 2003;17:223–31.PubMed

67.

Randall R, Nielsen K, Tvedt SD. The development of five scales to measure employees’ appraisals of organizational-level stress management interventions. Work Stress. 2009;23:1–23.

68.

Boothroyd RA, Greenbaum PE, Wang W, Kutash K, Friedman RM. Development of a measure to assess the implementation of children’s systems of care: the systems of care implementation survey (SOCIS). J Behav Health Serv Res. 2011;38:288–302.PubMed

69.

Chang SE, Pan YHV. Exploring factors influencing mobile users’ intention to adopt multimedia messaging service. Behav Inf Technol. 2011;30:659–72.

70.

Collis B, Pals N. A model for predicting an individual’s use of a telematics application for a learning-related purpose. Int J Educ Telecommunications. 2000;6:63-103.

71.

Collis B, Peters O, Pals N. A model for predicting the educational use of information and communication technologies. Instruct Sci. 2001;29:95–125.

72.

Greenbaum PE, Wang W, Boothroyd R, Kutash K, Friedman RM. Multilevel confirmatory factor analysis of the systems of care implementation survey (SOCIS). J Behav Health Serv Res. 2011;38:303–26.PubMed

73.

Hall GE, George AA, Rutherford WL. Measuring stages of concern about innovation: a manual for use of the SOC questionnaire. Austin: Research and Development Center for Teacher Education, The University of Texas; 1977.

74.

Hall GE, George A, Rutherford W. Measuring stages of concern about the innovation: a manual for use of the SoC questionnaire. Austin: Research and Development Center for Teacher Education, The University of Texas; 1979.

75.

Monthuy-Blanc J, Bouchard S, Maiano C, Seguin M. Factors influencing mental health providers’ intention to use telepsychotherapy in first nations communities. Transcult Psychiatry. 2013;50:323–43.PubMed

76.

Bess KD, Perkins DD, McCown DL. Testing a measure of organizational learning capacity and readiness for transformational change in human services. J Prev Interv Community. 2010;39:35–49.

77.

Bouckenooghe D, Devos G, Van den Broeck H. Organizational change questionnaire-climate of change, processes, and readiness: development of a new instrument. J Psychol. 2009;143:559–99.PubMed

78.

Caldwell DF, O’Reilly CA. The determinants of team-based innovation in organizations the role of social influence. Small Group Res. 2003;34:497–517.

79.

Chwelos P, Benbasat I, Dexter AS. Research report: empirical test of an EDI adoption model. Inf Syst Res. 2001;12:304–21.

80.

Dahlan N, Ramayah T, Mei LL. Readiness to adopt data mining technologies: an exploratory study of telecommunication employees in Malaysia. In: Karagiannis D, Reimer U, editors. Practical aspects of knowledge management, vol. 2569. Berlin: Springer-Verlag; 2002. p. 75–86.

81.

Hanusaik N, O’Loughlin JL, Kishchuk N, Eyles J, Robinson K, Cameron R. Building the backbone for organisational research in public health systems: development of measures of organisational capacity for chronic disease prevention. J Epidemiol Community Health. 2007;61:742–9.PubMedPubMedCentral

82.

Holt DT, Armenakis AA, Feild HS, Harris SG. Readiness for organizational change: the systematic development of a scale. J Appl Behav Sci. 2007;43:232–55.

83.

Judge TA, Thoresen CJ, Pucik V, Welbourne TM. Managerial coping with organizational change: a dispositional perspective. J Appl Psychol. 1999;84:107.

84.

Jung J, Nitzsche A, Neumann M, Wirtz M, Kowalski C, Wasem J, Stieler-Lorenz B, Pfaff H. The Worksite Health Promotion Capacity Instrument (WHPCI): development, validation and approaches for determining companies’ levels of health promotion capacity. BMC Public Health. 2010;10:1–10.

85.

Molla A, Licker PS. eCommerce adoption in developing countries: a model and instrument. Inf Manage. 2005;42:877–99.

86.

Molla A, Licker PS. Perceived e-readiness factors in e-commerce adoption: an empirical investigation in a developing country. Int J Electron Commerce. 2005;10:83–110.

87.

Moore GC, Benbasat I. Development of an instrument to measure the perceptions of adopting an information technology innovation. Inf Syst Res. 1991;2:192–222.

88.

Peltier JW, Schibrowsky JA, Zhao Y. Understanding the antecedents to the adoption of CRM technology by small entrepreneurs vs owner-managers. Int Small Bus J. 2009;27:307–36.

89.

Ravichandran T. Swiftness and intensity of administrative innovation adoption: an empirical study of TQM in information systems. Decision Sci. 2000;31:691.

90.

Saffu K, Walker JH, Hinson R. An empirical study of perceived strategic value and adoption constructs: the Ghanaian case. Manag Decis. 2007;45:1083–101.

91.

Strating MMH, Nieboer AP. Norms for creativity and implementation in healthcare teams: testing the group innovation inventory. Int J Qual Health Care. 2010;22:275–82.PubMedPubMedCentral

92.

Zeitz G, Johannesson R, Ritchie JE. An employee survey measuring total quality management practices and culture development and validation. Group Org Manag. 1997;22:414–44.

93.

Taxman FS, Young DW, Wiersema B, Rhodes A, Mitchell S. The national criminal justice treatment practices survey: multilevel survey methods and procedures. J Subst Abuse Treat. 2007;32:225–38.PubMedPubMedCentral

94.

Lin SP, Yang HY. Exploring key factors in the choice of e-health using an asthma care mobile service model. Telemed e-Health. 2009;15:884–90.

95.

Davis FD, Bagozzi RP, Warshaw PR. User acceptance of computer-technology—a comparison of 2 theoretical-models. Manage Sci. 1989;35:982–1003.

96.

Venkatesh V, Davis FD. A theoretical extension of the technology acceptance model: four longitudinal field studies. Manage Sci. 2000;46:186–204.

97.

Molla A, Licker PL. PERM: a model of eCommerce adoption in developing countries. In: Khosrow-Pour M, editor. Issues and trends of information technology management in contemporary organizations, vol. 1. Hershey: Idea Group Publishing; 2002. p. 527–30.

98.

Al-Hudhaif SA, Alkubeyyer A. E-commerce adoption factors in Saudi Arabia. Int J Bus Manage. 2011;6:122.

99.

Aldridge JM, Fraser BJ. Teachers’ views of their school climate and its relationship with teacher self-efficacy and job satisfaction. Learn Environ Res. 2016;19:291.

100.

Alharbi S, Drew S. Using the technology acceptance model in understanding academics’ behavioural intention to use learning management systems. Int J Adv Comput Sci Appl (IJACSA). 2014;5.

101.

Bailey Jr DB, Palsha SA. Qualities of the Stages of Concern Questionnaire and implications for educational innovations. J Educ Res. 1992;85:226–32.

102.

Berkowitz R, Bowen G, Benbenishty R, Powers JD. A cross-cultural validity study of the school success profile learning organization measure in Israel. Child Schools. 2013;35:137–46.

103.

Canfield JP. The McKinney-Vento Act implementation scale: a second validation study. J Child Poverty. 2014;20:47–64.

104.

Cheung D, Hattie J, Ng D. Reexamining the Stages of Concern Questionnaire: a test of alternative models. J Educ Res. 2001;94:226–36.

105.

Christensson L, Bachrach-Lindstrom M. Adapting “the staff attitudes to nutritional nursing care scale” to geriatric nursing care. J Nutr Health Aging. 2009;13:102–7.PubMed

106.

Cetinkaya B. Understanding teachers in the midst of reform: teachers’ concerns about reformed sixth grade mathematics curriculum in Turkey. Eurasia J Math Sci Technol Educ. 2012;8:155–66.

107.

Godoe P, Johansen T. Understanding adoption of new technologies: technology readiness and technology acceptance as an integrated concept. J Eur Psychol Stud. 2012;3:38–52.

108.

Knapp P, Raynor DK, Thistlethwaite JE, Jones MB. A questionnaire to measure health practitioners’ attitudes to partnership in medicine taking: LATCon II. Health Expect. 2009;12:175–86.PubMedPubMedCentral

109.

Richardson JW. Technology adoption in Cambodia: measuring factors impacting adoption rates. J Int Dev. 2011;23:697–710.

110.

Shotsberger PG, Crawford AR. On the elusive nature of measuring teacher change: an examination of the stages of concern questionnaire. Eval Res Educ. 1999;13:3–17.

111.

Tan J, Tyler K, Manica A. Business-to-business adoption of eCommerce in China. Inf Manage. 2007;44:332–51.

112.

Van Den Berg R, Ros A. The permanent importance of the subjective reality of teachers during educational innovation: a concerns-based approach. Am Educ Res J. 1999;36:879–906.

113.

Edwards P, Roberts I, Clarke M, DiGuiseppi C, Pratap S, Wentz R, Kwan I, Cooper R. Methods to increase response rates to postal questionnaires. Cochrane Database Syst Rev. 2007;2:MR000008.

114.

Schafer J. Multiple imputation: a primer. Stat Methods Med Res. 1999;8:3–15.PubMed

115.

Macfarlane A, O’Reilly-de Brun M, de Brun T, Dowrick C, O’Donnell C, Mair F, Spiegel W, van den Muijsenbergh M, van Weel Baumgarten E, Lionis C, et al. Healthcare for migrants, participatory health research and implementation science—better health policy and practice through inclusion. The RESTORE project. Eur J Gen Pract. 2014;20:148–52.PubMed

116.

Husted JA, Cook RJ, Farewell VT, Gladman DD. Methods for assessing responsiveness: a critical review and recommendations. J Clin Epidemiol. 2000;53:459–68.PubMed

117.

Guyatt G, Walter S, Norman G. Measuring change over time—assessing the usefulness of evaluative instruments. J Chronic Dis. 1987;40:171–8.PubMed

118.

Groves RM. Nonresponse rates and nonresponse bias in household surveys. Public Opin Q. 2006;70:646–75.

119.

Armstrong JS, Overton TS. Estimating nonresponse bias in mail surveys. J Mark Res. 1977;14:396–402.

120.

Rogers EM. Diffusion of innovations. 3rd ed. New York: Free Press; 1983.

121.

Hall GE, Hord SM. Change in schools: facilitating the process. Albany: State University of New York Press; 1987.

122.

Bandura A. Social foundations of thought and action: a social cognitive theory. Englewood Cliffs: Prentice-Hall; 1986.

123.

Bronstein LR. A model for interdisciplinary collaboration. Soc Work. 2003;48:297–306.PubMed

124.

Bronstein LR. Index of interdisciplinary collaboration. Soc Work Res. 2002;26:113–26.

125.

Litwin G, Stringer R. Motivation and organizational climate. Manage Int Rev. 1969;9:163.

126.

Taylor J, Bowers D. Survey of organizations: a machine scored standardized questionnaire. Ann Arbor: Institute for Social Research, University of Michigan; 1972.

127.

Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q. 2004;82:581–629.PubMedPubMedCentral

128.

Oldenburg B, Parcel GS. Diffusion of innovations. In: Glanz K, Rimer BK, Viswanath K, editors. Health behavior and health education: theory, research, and practice. San Francisco: John Wiley & Sons; 2002. p. 312–34.

129.

Goldman KD. Perceptions of innovations as predictors of implementation levels: the diffusion of a nation wide health education campaign. Health Educ Behav. 1994;21:433–45.

130.

Parcel GS, O’Hara-Tompkins NM, Harrist RB, Basen-Engquist KM, McCormick LK, Gottlieb NH, Eriksen MP. Diffusion of an effective tobacco prevention program. Part II: evaluation of the adoption phase. Health Educ Res. 1995;10:297–307.PubMed

131.

Lafferty CK. Diffusion of an asset building innovation in three Portage County school districts: A model of individual change. Kent: Kent State University; 2001.

132.

Fullan M. The new meaning of educational change. New York: Teachers College Press; 2001.

133.

Bandura A, McClelland DC. Social learning theory. Prentice-Hall: Englewood Cliffs; 1977.

134.

Gibson S, Dembo MH. Teacher efficacy—a construct-validation. J Educ Psychol. 1984;76:569–82.

135.

Lewin K. Field theory in social science: selected theoretical papers. New York: Harper; 1951.

136.

Moos RH. The social climate scales: an overview. Palo Alto: Consulting Psychologists Press; 1974.

137.

Fisher DL, Fraser BJ. School climate and teacher professional development. South Pac J Teach Educ. 1991;19:17–32.

138.

Fisher DL, Fraser BJ. Validity and use of school environment instruments. J Classroom Interact. 1991;26(2):13–8.

139.

Bowen GL. Organizational culture profile. Chapel Hill: Bowen & Associates; 1997.

140.

Orthner DK, Cook PC, Sabah Y, Rosenfeld J. Measuring organizational learning in human services. Development and validation of the organizational learning capacity assessment. Miami: Workforce Issues in Social Work; 2005.

141.

Cameron KS, Bright D, Caza A. Exploring the relationships between organizational virtuousness and performance. Am Behav Sci. 2004;47:766–90.

142.

McIntosh K, Horner RH, Sugai G. Sustainability of systems-level evidence-based practices in schools: current knowledge and future directions. In: Sailor W, Dunlap G, Sugai G, Horner RH, editors. Handbook of positive behavior support. Berlin: Springer Science & Business Media; 2009. p. 327–52.

143.

Nysveen H, Pedersen PE, Thorbjornsen H. Intentions to use mobile services: antecedents and cross-service comparisons. J Acad Mark Sci. 2005;33:330–46.

144.

Hupcey JE, Penrod J, Morse JM, Mitcham C. An exploration and advancement of the concept of trust. J Adv Nurs. 2001;36:282–93.PubMed

145.

Bauer HH, Barnes SJ, Reichardt T, Neumann MM. Driving consumer acceptance of mobile marketing: a theoretical framework and empirical study. J Electron Commerce Res. 2005;6:181–92.

146.

Hofstede G. Culture’s consequences: international differences in work-related values. Beverly Hills: Sage Publications; 1980.

147.

Yoo B, Donthu N. The effects of marketing education and individual cultural values on marketing ethics of students. J Mark Educ. 2002;24:92–103.

148.

Bolton T. Perceptual factors that influence the adoption of videotex technology: results of the channel 2000 field test. J Broadcasting. 1983;27:141–53.

149.

Bandura A. Self-efficacy mechanism in human agency. Am Psychol. 1982;37:122–47.

150.

Beach LR, Mitchell TR. A contingency model for the selection of decision strategies. Acad Manage Rev. 1978;3:439–49.

151.

Johnson EJ, Payne JW. Effort and accuracy in choice. Manage Sci. 1985;31:395–414.

152.

Payne JW. Contingent decision behavior. Psychol Bull. 1982;92:382.

153.

Swanson EB. Measuring user attitudes in MIS research: a review. Omega. 1982;10:157–65.

154.

Swanson EB. Information channel disposition and use. Decision Sci. 1987;18:131–45.

155.

Saga VL, Zmud RW. The nature and determinants of IT acceptance, routinization, and infusion. In: Levine L, editor. Diffusion, transfer, and implementation of information technology. Pittsburgh: North-Holland; 1993. p. 67–86.

156.

Fishbein M. A theory of reasoned action: some applications and implications. Nebr Symp Motiv. 1979;27:65–116.

157.

Frambach RT, Schillewaert N. Organizational innovation adoption—a multi-level framework of determinants and opportunities for future research. J Bus Res. 2002;55:163–76.

158.

Venkatesh V, Morris MG, Davis GB, Davis FD. User acceptance of information technology: toward a unified view. MIS Q. 2003;27:425–78.

159.

Igbaria M, Guimaraes T, Davis G. Testing the antecedents of microcomputer usage via a structural equation model. J Manage Inf Syst. 1995;11:87–114.

160.

Igbaria M, Zinatelli N, Cragg P, Cavaye A. Personal computing acceptance factors in small firms: a structural equation model. MIS Q. 1997;21:279–305.

161.

Al-Gahtani SS, King M. Attitudes, satisfaction and usage: factors contributing to each in the acceptance of information technology. Behav Inf Technol. 1999;18:277–97.

162.

Taylor S, Todd PA. Understanding information technology usage—a test of competing models. Inf Syst Res. 1995;6:144–76.

163.

Lam T, Cho V, Qu H. A study of hotel employee behavioral intentions towards adoption of information technology. Int J Hosp Manag. 2007;26:49–65.

164.

Lewis W, Agarwal R, Sambamurthy V. Sources of influence on beliefs about information technology use: an empirical study of knowledge workers. MIS Q. 2003;27:657–78.

165.

Selwyn N. Students’ attitudes toward computers: validation of a computer attitude scale for 16-19 education. Comput Educ. 1997;28:35–41.

166.

Huang HM. Student perceptions in an online mediated environment. Int J Instr Media. 2002;29:405.

167.

Watkins R, Leigh D, Triner D. Assessing readiness for e-learning. Perform Improv Q. 2004;17:66.

168.

Smith PJ, Murphy KL, Mahoney SE. Towards identifying factors underlying readiness for online learning: an exploratory study. Distance Educ. 2003;24:57–67.

169.

Smith PJ. Learning preferences and readiness for online learning. Educ Psychol. 2005;25:3–12.

170.

Muse HE. The web-based community college student: an examination of factors that lead to success and risk. Internet Higher Educ. 2003;6:241–61.

171.

Osborn V. Identifying at-risk students in videoconferencing and web-based distance education. Am J Distance Educ. 2001;15:41–54.

172.

Roblyer MD, Marshall JC. Predicting success of virtual high school students: preliminary results from an educational success prediction instrument. J Res Comput Educ. 2002;35:241–55.

173.

Scott WR. Organizations: rational, natural, and open systems. London: Prentice Hall International; 1998.

174.

Benrimoj SI, Roberts AS. Providing patient care in community pharmacies in Australia. Ann Pharmacother. 2005;39:1911–7.PubMed

175.

Roberts AS, Benrimoj SIC, Chen TF, Williams KA, Hopp TR, Aslani P. Understanding practice change in community pharmacy: a qualitative study in Australia. Res Soc Adm Pharm. 2005;1:546–64.

176.

Raynor D, Thistlethwaite J, Hart K, Knapp P. Are health professionals ready for the new philosophy of concordance in medicine taking? Int J Pharm Pract. 2001;9:81–4.

177.

Hepler CD, Strand LM. Opportunities and responsibilities in pharmaceutical care. Am J Hosp Pharm. 1990;47:533–43.PubMed

178.

Tesluk PE, Farr JL, Mathieu JE, Vance RJ. Generalization of employee involvement training to the job setting: individual and situational effects. Pers Psychol. 1995;48:607.

179.

Orthner DK, Cook P, Sabah Y, Rosenfeld J. Organizational learning: a cross-national pilot-test of effectiveness in children’s services. Eval Program Plann. 2006;29:70–8.

180.

Scott SG, Bruce RA. Determinants of innovative behavior: a path model of individual innovation in the workplace. Acad Manage J. 1994;37:580–607.

181.

Bass BM, Avolio BJ. Improving organizational effectiveness through transformational leadership. New York: SAGE Publications, Inc; 1994.

182.

Arnold JA, Arad S, Rhoades JA, Drasgow F. The empowering leadership questionnaire: the construction and validation of a new scale for measuring leader behaviors. J Organ Behav. 2000;21:249–69.

183.

Podsakoff PM, MacKenzie SB, Moorman RH, Fetter R. Transformational leader behaviors and their effects on followers’ trust in leader, satisfaction, and organizational citizenship behaviors. Leadersh Q. 1990;1:107–42.

184.

Martino S, Ball SA, Gallon SL, Hall D, Garcia M, Ceperich S, Farentinos C, Hamilton J, Hausotter W. Motivational interviewing assessment: supervisory tools for enhancing proficiency. Salem: Northwest Frontier Addiction Technology Transfer Center, Oregon Health and Science University; 2006.

185.

Farrell J, Young DW, Taxman FS. Effects of organizational factors on use of juvenile supervision practices. Crim Justice Behav. 2011;38:565–83.

186.

Caldwell DF, Chatman JA, O’Reilly CA. Building organizational commitment: a multifirm study. J Occup Psychol. 1990;63:245–61.

187.

Saksvik P, Nytro K, Tvedt SD. Healthy organizational change. In: Houdmont J, Leka S, editors. Occupational health psychology: European perspectives on research, education and practice, vol. 2. Nottingham: ISMAI Publishers; 2007. p. 81–90.

188.

Saksvik P, Tvedt SD, Nytro K, Andersen GR, Andersen TK, Buvik MP, Torvatn H. Developing criteria for healthy organizational change. Work Stress. 2007;21:243–63.

189.

Fishbein M, Ajzen I. Belief, attitude, intention, and behavior: an introduction to theory and research. Boston: Addison-Wesley Pub. Co.; 1975.

190.

Hartwick J, Barki H. Explaining the role of user participation in information system use. Manage Sci. 1994;40:440–65.

191.

Hurt HT, Joseph K, Cook CD. Scales for the measurement of innovativeness. Hum Commun Res. 1977;4:58–65.

192.

Zaltman G. Metaphorically speaking: new technique uses multidisciplinary ideas to improve qualitative research. Mark Res. 1996;8:13.

193.

Hall GE, Wallace RD, Dossett WA. A developmental conceptualization of the adoption process within educational institutions. Austin: Research and Development Center for Teacher Education, The University of Texas; 1973.

194.

Newlove BW, Hall GE. A manual for assessing open-ended statements of concern. Austin: Research and Development Center for Teacher Education, The University of Texas; 1976.

195.

Davis FD. User acceptance of information technology: system characteristics, user perceptions and behavioral impacts. Int J Man Mach Stud. 1993;38:475–87.

196.

Venkatesh V. Determinants of perceived ease of use: integrating control, intrinsic motivation, and emotion into the technology acceptance model. Inf Syst Res. 2000;11:342–65.

197.

Venkatesh V, Davis FD. Extrinsic and intrinsic motivation to use computers in the work place. J Appl Psychol. 2000;22:1111–32.

198.

Kwon TH, Zmud RW. Unifying the fragmented models of information systems implementation. In: Boland RJ, Hirschheim R, editors. Critical issues in information systems research. New York: John Wiley & Sons, Inc; 1987. p. 227–51.

199.

Davenport TH, Harris JG, De Long DW, Jacobson AL. Data to knowledge to results: building an analytic capability. Calif Manage Rev. 2001;43:117.

200.

Plsek PE. Collaborating across organizational boundaries to improve the quality of care. Am J Infect Control. 1997;25:85–95.PubMed

201.

Kilo CM. A framework for collaborative improvement: lessons from the Institute for Healthcare Improvement’s Breakthrough Series. Qual Manage Healthc. 1998;6:1–14.

202.

Kivimaki M, Elovainio M. A short version of the team climate inventory: development and psychometric properties. J Occup Organ Psychol. 1999;72:241–6.

203.

Strating MMH, Nieboer AP. Psychometric test of the team climate inventory-short version investigated in Dutch quality improvement teams. BMC Health Serv Res. 2009;9:1.

204.

Anderson N, West MA. Team climate inventory: manual and user’s guide. Windsor: ASE; 1994.

205.

Iacovou CL, Benbasat I, Dexter AS. Electronic data interchange and small organizations: adoption and impact of technology. MIS Q. 1995;19:465–85.

206.

Raymond L, Pare G. Measurement of information technology sophistication in small manufacturing businesses. Inf Resourc Manage J. 1992;5:4–16.

207.

Ferguson S. The benefits and barriers to adoption of EDI. Vancouver: University of British Columbia; 1992.

208.

Marsick VJ, Watkins KE. Demonstrating the value of an organization’s learning culture: the dimensions of the learning organization questionnaire. Adv Dev Hum Resourc. 2003;5:196.

209.

Goodman RM, Speers MA, McLeroy K, Fawcett S, Kegler M, Parker E, Smith SR, Sterling TD, Wallerstein N. Identifying and defining the dimensions of community capacity to provide a basis for measurement. Health Educ Behav. 1998;25:258–78.PubMed

210.

Crisp BR, Swerissen H, Duckett SJ. Four approaches to capacity building in health: consequences for measurement and accountability. Health Promot Int. 2000;15:99–107.

211.

Hawe P, King L, Noort M, Jordens C, Lloyd B. Indicators to help with capacity building in health promotion. Sydney: NSW Health Department; 2000.

212.

Dean JW, Bowen DE. Management theory and total quality—improving research and practice through theory development. Acad Manage Rev. 1994;19:392–418.

213.

Miller D, Friesen PH. Innovation in conservative and entrepreneurial firms—2 models of strategic momentum. Strateg Manage J. 1982;3:1–25.

214.

Saraph JV, Benson PG, Schroeder RG. An instrument for measuring the critical factors of quality management. Decision Sci. 1989;20:810–29.

215.

Van de Ven AH, Poole MS. Explaining development and change in organizations. Acad Manage Rev. 1995;20:510–40.

216.

Hunt VD. Quality in America: how to implement a competitive quality program. Burr Ridge: Irwin Professional Publishing; 1992.

217.

Grandzol JR. Implementing total quality: Critical relationships. Philadelphia: Temple University, Department of Management Science and Operations Management; 1996.

218.

Hackman JR, Oldham GR. Work redesign. Reading: Addison-Wesley; 1980.

219.

Schwartz R, Smith C, Speers MA, Dusenbury LJ, Bright F, Hedlund S, Wheeler F, Schmid TL. Capacity building and resource needs of state health agencies to implement community-based cardiovascular disease programs. J Public Health Policy. 1993;14:480–94.PubMed

220.

Hawe P, Noort M, King L, Jordens C. Multiplying health gains: the critical role of capacity-building within health promotion programs. Health Policy. 1997;39:29–42.PubMed

221.

Riley BL, Taylor SM, Elliott SJ. Determinants of implementing heart health promotion activities in Ontario public health units: a social ecological perspective. Health Educ Res. 2001;16:425–41.PubMed

222.

Chatterji M, Sentovich C, Ferron J, Rendina-Gobioff G. Using an iterative model to conceptualize, pilot test, and validate scores from an instrument measuring teacher readiness for educational reforms. Educ Psychol Meas. 2002;62:444–65.

223.

Pearlin LI, Menaghan EG, Lieberman MA, Mullan JT. The stress process. J Health Soc Behav. 1981;22:337–56.PubMed

224.

Trumbo DA. Individual and group correlates of attitudes toward work-related changes. J Appl Psychol. 1961;45:338.

225.

Miller VD, Johnson JR, Grau J. Antecedents to willingness to participate in a planned organizational change. J Appl Commun Res. 1994;22:59–80.

226.

Mayer RC, Davis JH. The effect of the performance appraisal system on trust for management: a field quasi-experiment. J Appl Psychol. 1999;84:123.

227.

Watson D, Clark LA, Tellegen A. Development and validation of brief measures of positive and negative affect: the PANAS scales. J Pers Soc Psychol. 1988;54:1063–70.PubMed

228.

Hong SM, Faedda S. Refinement of the Hong psychological reactance scale. Educ Psychol Meas. 1996;56:173–82.

Title: Psychometric properties of implementation measures for public health and community settings and mapping of constructs against the Consolidated Framework for Implementation Research: a systematic review
Authors: Tara Clinton-McHarg
Sze Lin Yoong
Flora Tzelepis
Tim Regan
Alison Fielding
Eliza Skelton
Melanie Kingsland
Jia Ying Ooi
Luke Wolfenden
Publication date: 01-12-2016
Publisher: BioMed Central
Published in: Implementation Science / Issue 1/2015
Electronic ISSN: 1748-5908
DOI: https://doi.org/10.1186/s13012-016-0512-5

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Psychometric properties of implementation measures for public health and community settings and mapping of constructs against the Consolidated Framework for Implementation Research: a systematic review

Abstract

Background

Methods

Results

Conclusions

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Abstract

Background

Methods

Results

Conclusions

Please log in to get access to this content

Other articles of this Issue 1/2015

Reporting behaviour change interventions: do the behaviour change technique taxonomy v1, and training in its use, improve the quality of intervention descriptions?

Effectiveness of an intervention in increasing the provision of preventive care by community mental health services: a non-randomized, multiple baseline implementation trial

A cluster-randomised controlled trial of values-based training to promote autonomously held recovery values in mental health workers

Sustaining complex interventions in long-term care: a qualitative study of direct care staff and managers

A review of methods and tools to assess the implementation of government policies to create healthy food environments for preventing obesity and diet-related non-communicable diseases

Erratum to: ‘A modified theoretical framework to assess implementation fidelity of adaptive public health interventions’