Top

Published in:

Open Access 01-12-2018 | Research article

Operationalizing the ‘pragmatic’ measures construct using a stakeholder feedback and a multi-method approach

Authors: Cameo F. Stanick, Heather M. Halko, Caitlin N. Dorsey, Bryan J. Weiner, Byron J. Powell, Lawrence A. Palinkas, Cara C. Lewis

Published in: BMC Health Services Research | Issue 1/2018

Abstract

Context

Implementation science measures are rarely used by stakeholders to inform and enhance clinical program change. Little is known about what makes implementation measures pragmatic (i.e., practical) for use in community settings; thus, the present study’s objective was to generate a clinical stakeholder-driven operationalization of a pragmatic measures construct.

Evidence acquisition

The pragmatic measures construct was defined using: 1) a systematic literature review to identify dimensions of the construct using PsycINFO and PubMed databases, and 2) interviews with an international stakeholder panel (N = 7) who were asked about their perspectives of pragmatic measures.

Evidence synthesis

Combined results from the systematic literature review and stakeholder interviews revealed a final list of 47 short statements (e.g., feasible, low cost, brief) describing pragmatic measures, which will allow for the development of a rigorous, stakeholder-driven conceptualization of the pragmatic measures construct.

Conclusions

Results revealed significant overlap between terms related to the pragmatic construct in the existing literature and stakeholder interviews. However, a number of terms were unique to each methodology. This underscores the importance of understanding stakeholder perspectives of criteria measuring the pragmatic construct. These results will be used to inform future phases of the project where stakeholders will determine the relative importance and clarity of each dimension of the pragmatic construct, as well as their priorities for the pragmatic dimensions. Taken together, these results will be incorporated into a pragmatic rating system for existing implementation science measures to support implementation science and practice.

Glasgow RE, Riley WT. Pragmatic measures: what they are and why we need them. Am J Prev Med. 2013;45(2):237–43. https://doi.org/10.1016/j.amepre.2013.03.010.CrossRefPubMed

Kroenke K, Spitzer RL. The PHQ-9: a new depression diagnostic and severity measure. Psychiatr Ann. 2002;32(9):509–15. https://doi.org/10.3928/0048-5713-20020901-06.CrossRef

Achenbach TM. Advancing assessment of children and adolescents: commentary on evidence-based assessment of child and adolescent disorders. J Clin Child Adolesc Psychol. 2005;34(3):541–7. https://doi.org/10.1207/s15374424jccp3403_9.CrossRefPubMed

Glisson C, Landsverk J, Schoenwald S, et al. Assessing the organizational social context (OSC) of mental health services: implications for research and practice. Admin Pol Ment Health. 2008;35(1–2):98–113. https://doi.org/10.1007/s10488-007-0148-5.CrossRef

Glisson C. Assessing and changing organizational culture and climate for effective services. Res Soc Work Pract. 2007;17(6):736–47. https://doi.org/10.1177/1049731507301659.CrossRef

Glisson C, Schoenwald SK, Hemmelgarn A, et al. Randomized trial of MST and ARC in a two-level evidence-based treatment implementation strategy. J Consult Clin Psychol. 2010;78(4):537–50. https://doi.org/10.1037/a0019160.CrossRefPubMedPubMedCentral

Glisson C, Williams NJ, Hemmelgarn A, Proctor E, Green P. Increasing clinicians' EBT exploration and preparation behavior in youth mental health services by changing organizational culture with ARC. Behav Res Ther. 2016;76:40–6. https://doi.org/10.1016/j.brat.2015.11.008.CrossRefPubMed

Williams NJ, Glisson C. Testing a theory of organizational culture, climate and youth outcomes in child welfare systems: a United States national study. Child Abuse Negl. 2014;38(4):757–67. https://doi.org/10.1016/j.chiabu.2013.09.003.CrossRefPubMed

Williams NJ, Glisson C. Reducing turnover is not enough: the need for proficient organizational cultures to support positive youth outcomes in child welfare. Child Youth Serv Rev. 2013;35(11). https://doi.org/10.1016/j.childyouth.2013.09.002.CrossRef

10.

Hunsley J, Mash EJ. Evidence-based assessment. Annu Rev Clin Psychol. 2007;3:29–51. https://doi.org/10.1146/annurev.clinpsy.3.022806.091419.CrossRefPubMed

11.

Mokkink LB, Terwee CB, Patrick DL, et al. The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments: an international Delphi study. Qual Life Res. 2010;19(4):539–49. https://doi.org/10.1007/s11136-010-9606-8.CrossRefPubMedPubMedCentral

12.

Chambers DA, Azrin ST. Research and services partnerships: partnership: a fundamental component of dissemination and implementation research. Psychiatr Serv. 2013;64(6):509–11. https://doi.org/10.1176/appi.ps.201300032.CrossRefPubMed

13.

Minkler M, Salvatore AL. Participatory approaches for study design and analysis in dissemination and implementation research. In: Brownson RC, Colditz GA, Proctor EK, editors. Dissemination and implementation research in health: translating science to practice. New York: Oxford University Press; 2012. p. 192–212.CrossRef

14.

Palinkas LA, Horwitz SM, Green CA, Wisdom JP, Duan N, Hoagwood K. Purposeful sampling for qualitative data collection and analysis in mixed method implementation research. Admin Pol Ment Health. 2015;42(5):533–44. https://doi.org/10.1007/s10488-013-0528-y.CrossRef

15.

Patton MQ. Qualitative research and evaluation methods. 3rd ed. Thousand Oaks, CA: Sage Publications; 2002.

16.

Benton D. Grounded theory. In: Cormack D, editor. The research process in nursing. 2nd ed. London: Blackwell; 1991.

17.

Morgan DL. Qualitative content analysis: a guide to paths not taken. Qual Health Res. 1993;3(1):112–21. https://doi.org/10.1177/104973239300300107.CrossRefPubMed

18.

Stratford PW, Riddle DL. Assessing sensitivity to change: choosing the appropriate change coefficient. Health Qual Life Outcomes. 2005;3:23. https://doi.org/10.1186/1477-7525-3-23.CrossRefPubMedPubMedCentral

19.

Hendrickson AR, Massey PD, Cronan TP. On the test-retest reliability of perceived usefulness and perceived ease of use scales. MIS Q. 1993;17(2):227. https://doi.org/10.2307/249803.CrossRef

20.

Glasgow RE, Nutting PA, King DK, et al. Randomized effectiveness trial of a computer-assisted intervention to improve diabetes care. Diabetes Care. 2005;28(1):33–9.CrossRef

21.

Jensen-Doss A, Hawley KM. Understanding barriers to evidence-based assessment: clinician attitudes toward standardized assessment tools. J Clin Child Adolesc Psychol. 2010;39(6):885–96. https://doi.org/10.1080/15374416.2010.517169.CrossRefPubMedPubMedCentral

22.

Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4(50). https://doi.org/10.1186/1748-5908-4-50.

23.

Proctor E, Landsverk J, Aarons G, Chambers D, Glisson C, Mittman B. Implementation research in mental health services: an emerging science with conceptual, methodological, and training challenges. Admin Pol Ment Health. 2009;36(1):24–34. https://doi.org/10.1007/s10488-008-0197-4.CrossRef

24.

Lewis CC, Stanick CF, Martinez RG, et al. The Society for Implementation Research Collaboration Instrument Review Project: a methodology to promote rigorous evaluation. Implement Sci. 2015;10(2). https://doi.org/10.1186/s13012-014-0193-x.

Title: Operationalizing the ‘pragmatic’ measures construct using a stakeholder feedback and a multi-method approach
Authors: Cameo F. Stanick
Heather M. Halko
Caitlin N. Dorsey
Bryan J. Weiner
Byron J. Powell
Lawrence A. Palinkas
Cara C. Lewis
Publication date: 01-12-2018
Publisher: BioMed Central
Published in: BMC Health Services Research / Issue 1/2018
Electronic ISSN: 1472-6963
DOI: https://doi.org/10.1186/s12913-018-3709-2

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Operationalizing the ‘pragmatic’ measures construct using a stakeholder feedback and a multi-method approach

Abstract

Context

Evidence acquisition

Evidence synthesis

Conclusions

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Abstract

Context

Evidence acquisition

Evidence synthesis

Conclusions

Please log in to get access to this content

Other articles of this Issue 1/2018

Re-excision and survival following breast conserving surgery in early stage breast cancer patients: a population-based study

Staff empowerment and engagement in a magnet® recognized and joint commission international accredited academic centre in Belgium: a cross-sectional survey

It’s hard to play ball: A qualitative study of knowledge exchange and silo effects in public health

Image and perception of physicians as barriers to inter-disciplinary cooperation? – the example of German occupational health physicians in the rehabilitation process: a qualitative study

Qualitative research investigating the mental health care service gap in Chinese burn injury patients

Positive predictive value of ICD-10 codes for acute myocardial infarction in Japan: a validation study at a single center