Top

Administration and Policy in Mental Health and Mental Health Services Research

Published in:

01-01-2018 | Original Article

Concordance Between Administrator and Clinician Ratings of Organizational Culture and Climate

Authors: Rinad S. Beidas, Nathaniel J. Williams, Philip D. Green, Gregory A. Aarons, Emily M. Becker-Haimes, Arthur C. Evans, Ronnie Rubin, Danielle R. Adams, Steven C. Marcus

Published in: Administration and Policy in Mental Health and Mental Health Services Research | Issue 1/2018

Abstract

Organizational culture and climate are important determinants of behavioral health service delivery for youth. The Organizational Social Context measure is a well validated assessment of organizational culture and climate that has been developed and extensively used in public sector behavioral health service settings. The degree of concordance between administrators and clinicians in their reports of organizational culture and climate may have implications for research design, inferences, and organizational intervention. However, the extent to which administrators’ and clinicians’ reports demonstrate concordance is just beginning to garner attention in public behavioral health settings in the United States. We investigated the concordance between 73 administrators (i.e., supervisors, clinical directors, and executive directors) and 247 clinicians in 28 child-serving programs in a public behavioral health system. Findings suggest that administrators, compared to clinicians, reported more positive cultures and climates. Organizational size moderated this relationship such that administrators in small programs (<466 youth clients served annually) provided more congruent reports of culture and climate in contrast to administrators in large programs (≥466 youth clients served annually) who reported more positive cultures and climates than clinicians. We propose a research agenda that examines the effect of concordance between administrators and clinicians on organizational outcomes in public behavioral health service settings.

In this manuscript, we focus on molar organizational climate.

We use the term concordance going forward to reflect this concept.

The final sample included 22 organizations because one organization was included from a previous wave of data collection but was no longer included in the top 29 agencies.

Participation rates reflect best estimates of program staffing at the time of data collection. Total staff for each program were based on reports by clinical and/or executive directors, as the city of Philadelphia does not routinely collect data on staffing within organizations and programs. In 4 programs, clinical directors reported fewer therapists in their agency than participated in the study; in these instances, we rounded participation rates down to 100%.

Aarons, G. A., Ehrhart, M. G., Farahnak, L. R., & Sklar, M. (2014). Aligning leadership across systems and organizations to develop a strategic climate for evidence-based practice implementation. Annual Review of Public Health, 35, 255–274. doi:10.1146/annurev-publhealth-032013-182447.CrossRefPubMedPubMedCentral

Aarons, G. A., Ehrhart, M. G., Farahnak, L. R., Sklar, M., & Horowitz, J. (2015). Discrepancies in leader and follower ratings of transformational leadership: Relationship with organizational culture in mental health. Administration and Policy in Mental Health. doi:10.1007/s10488-015-0672-7.

Aarons, G. A., Ehrhart, M. G., Torres, E. L., Finn, N. K., & Beidas, R. S. (2016). The humble leader: Association of discrepancies in leader and follower ratings of implementation leadership with organizational climate in mental health organizations. Psychiatric Services. doi:10.1176/appi.ps.201600062.

Aarons, G. A., & Sawitzky, A. C. (2006a). Organizational climate partially mediates the effect of culture on work attitudes and staff turnover in mental health services. Administration and Policy in Mental Health, 33(3), 289–301. doi:10.1007/s10488-006-0039-1.CrossRefPubMedPubMedCentral

Aarons, G. A., & Sawitzky, A. C. (2006b). Organizational culture and climate and mental health provider attitudes toward evidence-based practice. Psychological Services, 3(1), 61–72. doi:10.1037/1541-1559.3.1.61.CrossRefPubMedPubMedCentral

Aarons, G. A., Sommerfeld, D. H., Hecht, D. B., Silovsky, J. F., & Chaffin, M. J. (2009). The impact of evidence-based practice implementation and fidelity monitoring on staff turnover: Evidence for a protective effect. Journal of Consulting and Clinical Psychology, 77(2), 270. doi:10.1037/a0013223.CrossRefPubMedPubMedCentral

Aarons, G. A., Sommerfeld, D. H., & Willging, C. E. (2011). The soft underbelly of system change: The role of leadership and organizational climate in turnover during statewide behavioral health reform. Psychological Services, 8(4), 269–281.CrossRefPubMedPubMedCentral

Antonakis, J., & Atwater, L. (2002). Leader distance: A review and a proposed theory. Leadership Quarterly, 13(6), 673–704. doi:10.1016/S1048-9843(02)00155-8.CrossRef

Avolio, B. J., Zhu, W. C., Koh, W., & Bhatia, P. (2004). Transformational leadership and organizational commitment: Mediating role of psychological empowerment and moderating role of structural distance. Journal of Organizational Behavior, 25(8), 951–968. doi:10.1002/job.283.CrossRef

Bashshur, M. R., Hernández, A., & González-Romá, V. (2011). When managers and their teams disagree: A longitudinal look at the consequences of differences in perceptions of organizational support. Journal of Applied Psychology, 96(3), 558. doi:10.1037/a0022675.CrossRefPubMed

Beidas, R. S., Aarons, G. A., Barg, F. K., Evans, A., Hadley, T., Hoagwood, K. E., … Mandell, D. S. (2013). Policy to implementation: Evidence-based practice in community mental health–study protocol. Implementation Science, 8(1), 1. doi:10.1186/1748-5908-8-38.CrossRef

Beidas, R. S., Marcus, S., Aarons, G. A., Hoagwood, K. E., Schoenwald, S. K., Evans, A. C., … Mandell, D. S. (2015). Predictors of community therapists’ use of therapy techniques in a large public mental health system. JAMA Pediatrics, 169(4), 374–382. doi:10.1001/jamapediatrics.2014.3736.CrossRefPubMedPubMedCentral

Beidas, R. S., Marcus, S., Wolk, C. B., Powell, B., Aarons, G. A., Evans, A. C., … Mandell, D. S. (2016). A prospective examination of clinician and supervisor turnover within the context of implementation of evidence-based practices in a publicly-funded mental health system. Administration and Policy in Mental Health, 43(5), 640–649. doi:10.1007/s10488-015-0673-6.CrossRefPubMedPubMedCentral

Beidas, R. S., Wolk, C. L., Walsh, L. M., Evans, A. C. Jr., Hurford, M. O., & Barg, F. K. (2014). A complementary marriage of perspectives: Understanding organizational social context using mixed methods. Implementation Science, 9, 175. doi:10.1186/s13012-014-0175-z.CrossRefPubMedPubMedCentral

Cameron, K. S., & Quinn, R. E. (2011). Diagnosing and changing organizational culture: Based on the competing values framework (3 edn.). San Francisco, CA: Jossey-Bass.

Carlfjord, S., Andersson, A., Nilsen, P., Bendtsen, P., & Lindberg, M. (2010). The importance of organizational climate and implementation strategy at the introduction of a new working tool in primary health care. Journal of Evaluation in Clinical Practice, 16(6), 1326–1332. doi:10.1111/j.1365-2753.2009.01336.x.CrossRefPubMed

Chan, D. (1998). Functional relations among constructs in the same content domain at different levels of analysis: A typology of composition models. Journal of Applied Psychology, 83(2), 234–246. doi:10.1037/0021-9010.83.2.234.CrossRef

Cole, M. S., Carter, M. Z., & Zhang, Z. (2013). Leader-team congruence in power distance values and team effectiveness: The mediating role of procedural justice climate. Journal of Applied Psychology, 98(6), 962. doi:10.1037/a0034269.CrossRefPubMed

Cooke, R. A., & Rousseau, D. M. (1988). Behavioral norms and expectations: A quantitative approach to the assessment of organizational culture. Group & Organization Studies, 13(3), 245–273. doi:10.1177/105960118801300302.CrossRef

Cunningham, C. T., Quan, H., Hemmelgarn, B., Noseworthy, T., Beck, C. A., Dixon, E., … Jetté, N. (2015). Exploring physician specialist response rates to web-based surveys. BMC Medical Research Methodology, 15(1), 1. doi:10.1186/s12874-015-0016-z.CrossRef

Ehrhart, M., Schneider, B., & Macey, W. H. (2014). Organizational climate and culture: An introduction to theory, research, and practice. New York, NY: Routledge.

Fleenor, J. W., McCauley, C. D., & Brutus, S. (1996). Self-other rating agreement and leader effectiveness. Leadership Quarterly, 7(4), 487–506. doi:10.1016/S1048-9843(96)90003-X.CrossRef

Gibson, C. B., Cooper, C. D., & Conger, J. A. (2009). Do you see what we see? The complex effects of perceptual distance between leaders and teams. Journal of Applied Psychology, 94(1), 62. doi:10.1037/a0013073.CrossRefPubMed

Glisson, C. (2002). The organizational context of children’s mental health services. Clinical Child and Family Psychology Review, 5(4), 233–253. doi:10.1023/A:1020972906177.CrossRefPubMed

Glisson, C., & Green, P. (2011). Organizational climate, services, and outcomes in child welfare systems. Child Abuse & Neglect, 35(8), 582–591. doi:10.1016/j.chiabu.2011.04.009.CrossRef

Glisson, C., Green, P., & Williams, N. J. (2012). Assessing the Organizational Social Context (OSC) of child welfare systems: Implications for research and practice. Child Abuse & Neglect, 36(9), 621–632. doi:10.1016/j.chiabu.2012.06.002.CrossRef

Glisson, C., & Hemmelgarn, A. (1998). The effects of organizational climate and interorganizational coordination on the quality and outcomes of children’s service systems. Child Abuse & Neglect, 22(5), 401–421. doi:10.1016/S0145-2134(98)00005-2.CrossRef

Glisson, C., Landsverk, J., Schoenwald, S. K., Kelleher, K., Hoagwood, K. E., Mayberg, S., … Research Network on Youth Mental Health. (2008a). Assessing the Organizational Social Context (OSC) of mental health services: Implications for research and practice. Administration and Policy in Mental Health, 35(1–2), 98–113. doi:10.1007/s10488-007-0148-5.CrossRefPubMed

Glisson, C., Schoenwald, S. K., Hemmelgarn, A., Green, P., Dukes, D., Armstrong, K. S., & Chapman, J. E. (2010). Randomized trial of MST and ARC in a two-level evidence-based treatment implementation strategy. Journal of Consulting and Clinical Psychology, 78(4), 537–550. doi:10.1037/a0019160.CrossRefPubMedPubMedCentral

Glisson, C., Schoenwald, S. K., Kelleher, K., Landsverk, J., Hoagwood, K. E., Mayberg, S., … Research Network on Youth Mental Health. (2008b). Therapist turnover and new program sustainability in mental health clinics as a function of organizational culture, climate, and service structure. Administration and Policy in Mental Health, 35(1–2), 124–133. doi:10.1007/s10488-007-0152-9.CrossRefPubMed

Glisson, C., Williams, N. J., Hemmelgarn, A., Proctor, E., & Green, P. (2016). Aligning organizational priorities with ARC to improve youth mental health service outcomes. Journal of Consulting and Clinical Psychology, 84(8), 713–725. doi:10.1037/ccp0000107.CrossRefPubMedPubMedCentral

Hansen, L. O., Williams, M. V., & Singer, S. J. (2011). Perceptions of hospital safety climate and incidence of readmission. Health Services Research, 46(2), 596–616. doi:10.1111/j.1475-6773.2010.01204.x.CrossRefPubMedPubMedCentral

Hartmann, C. W., Rosen, A. K., Meterko, M., Shokeen, P., Zhao, S., Singer, S., … Gaba, D. M. (2008). An overview of patient safety climate in the VA. Health Services Research, 43(4), 1263–1284. doi:10.1111/j.1475-6773.2008.00839.x.CrossRefPubMedPubMedCentral

Hartnell, C. A., Ou, A. Y., & Kinicki, A. (2011). Organizational culture and organizational effectiveness: A meta-analytic investigation of the competing values framework’s theoretical suppositions. Journal of Applied Psychology, 96(4), 677–694. doi:10.1037/a0021987.CrossRefPubMed

Hasson, H., Gilbert-Ouimet, M., Baril-Gingras, G., Brisson, C., Vezina, M., Bourbonnais, R., & Montreuil, S. (2012). Implementation of an organizational-level intervention on the psychosocial environment of work comparison of managers’ and employees’ views. Journal of Occupational and Environmental Medicine, 54(1), 85–91. doi:10.1097/JOM.0b013e31823ccb2f.CrossRefPubMed

Hasson, H., von Thiele Schwarz, U., Nielsen, K., & Tafvelin, S. (2016). Are we all in the same boat? The role of perceptual distance in organizational health interventions. Stress and Health. doi:10.1002/smi.2703.

Hedeker, D., & Gibbons, R. D. (2006). Longitudinal data analysis (Vol. 451). Hoboken, NJ: Wiley.

Hoagwood, K. E., Burns, B. J., Kiser, L., Ringeisen, H., & Schoenwald, S. K. (2001). Evidence-based practice in child and adolescent mental health services. Psychiatric Services, 52(9), 1179–1189. doi:10.1176/appi.ps.52.9.1179.CrossRefPubMed

James, L. R., Hater, J. J., Gent, M. J., & Bruni, J. R. (1978). Psychological climate: Implications from cognitive social-learning theory and interactional psychology. Personnel Psychology, 31(4), 783–813. doi:10.1111/j.1744-6570.1978.tb02124.x.CrossRef

Kimberly, J. R. (1976). Organizational size and the structuralist perspective: A review, critique, and proposal. Administrative Science Quarterly, 21(4), 571–597. doi:10.2307/2391717.CrossRef

Klein, K. J., & Kozlowski, S. W. (2000). Multilevel theory, research, and methods in organizations: Foundations, extensions, and new directions. San Francisco, CA: Jossey-Bass.

Martin, A. J., Jones, E. S., & Callan, V. J. (2006). Status differences in employee adjustment during organizational change. Journal of Managerial Psychology, 21(2), 145–162. doi:10.1108/02683940610650758.CrossRef

Olin, S. S., Williams, N., Pollock, M., Armusewicz, K., Kutash, K., Glisson, C., & Hoagwood, K. E. (2014). Quality indicators for family support services and their relationship to organizational social context. Administration and Policy in Mental Health, 41(1), 43–54. doi:10.1007/s10488-013-0499-z.CrossRefPubMed

Ostroff, C., Kinicki, A., & Tamkins, M. (2003). Organizational culture and climate (Vol. 12). Hoboken, NJ: Wiley.

Raudenbush, S. W. (2004). HLM 6: Hierarchical linear and nonlinear modeling. Lincolnwood, IL: Scientific Software International.

Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: Applications and data analysis methods (Vol. 1). Thousand Oaks, CA: Sage.

Rindfuss, R. R., Choe, M. K., Tsuya, N. O., Bumpass, L. L., & Tamaki, E. (2015). Do low survey response rates bias results? Evidence from Japan. Demographic Research, 32, 797. doi:10.4054/DemRes.2015.32.26.CrossRef

Rogers, E. M. (2003). Elements of diffusion. In Diffusion of innovations (5th edn., pp. 1–38). New York, NY: Free Press.

Rosen, A. K., Singer, S., Shibei, Z., Shokeen, P., Meterko, M., & Gaba, D. (2010). Hospital safety climate and safety outcomes: Is there a relationship in the VA? Medical Care Research and Review, 67(5), 590–608. doi:10.1177/1077558709356703.CrossRefPubMed

Rothman, K. J. (1990). No adjustments are needed for multiple comparisons. Epidemiology (Cambridge, Mass), 1(1), 43–46.CrossRef

Schneider, B., Ehrhart, M. G., & Macey, W. H. (2013). Organizational climate and culture. Annual Review of Psychology, 64, 361–388. doi:10.1146/annurev-psych-113011-143809.CrossRefPubMed

Schoenwald, S. K., Chapman, J. E., Kelleher, K., Hoagwood, K. E., Landsverk, J., Stevens, J., … Research Network on Youth Mental Health. (2008). A survey of the infrastructure for children’s mental health services: Implications for the implementation of empirically supported treatments (ESTs). Administration and Policy in Mental Health, 35(1–2), 84–97. doi:10.1007/s10488-007-0147-6.CrossRefPubMed

Singer, S., Lin, S., Falwell, A., Gaba, D., & Baker, L. (2009). Relationship of safety climate and safety performance in hospitals. Health Services Research, 44(2 Pt 1), 399–421. doi:10.1111/j.1475-6773.2008.00918.x.CrossRefPubMedPubMedCentral

Van Velsor, E., Taylor, S., & Leslie, J. (1993). An examination of the relationships among self-perception accuracy, self-awareness, gender, and leader effectiveness. Human Resource Management, 32(2–3), 249–264. doi:10.1002/hrm.3930320205.CrossRef

Verbeke, W., Volgering, M., & Hessels, M. (1998). Exploring the conceptual expansion within the field of organizational behaviour: Organizational climate and organizational culture. Journal of Management Studies, 35(3), 303–329. doi:10.1111/1467-6486.00095.CrossRef

Weisz, J. R. (1997). Therapist background questionnaire. Los Angeles: University of California.

Williams, N. J., & Glisson, C. (2014). Testing a theory of organizational culture, climate and youth outcomes in child welfare systems: A United States national study. Child Abuse & Neglect, 38(4), 757–767. doi:10.1016/j.chiabu.2013.09.003.CrossRef

Wolf, D. A. P. S., Dulmus, C., Maguin, E., Keesler, J., & Powell, B. (2014). Organizational leaders’ and staff members’ appraisals of their work environment within a children’s social service system. Human Service Organizations Management Leadership & Governance, 38(3), 215–227. doi:10.1080/23303131.2014.884032.CrossRef

Yammarino, F. J., & Atwater, L. E. (1997). Do managers see themselves as others see them? Implications of self-other rating agreement for human resources management. Organizational Dynamics, 25(4), 35–44. doi:10.1016/S0090-2616(97)90035-8.CrossRef

Zyphur, M. J., Zammuto, R. F., & Zhang, Z. (2016). Multilevel latent polynomial regression for modeling (in)congruence across organizational groups: The case of organizational culture research. Organizational Research Methods, 19(1), 53–79. doi:10.1177/1094428115588570.CrossRef

Title: Concordance Between Administrator and Clinician Ratings of Organizational Culture and Climate
Authors: Rinad S. Beidas
Nathaniel J. Williams
Philip D. Green
Gregory A. Aarons
Emily M. Becker-Haimes
Arthur C. Evans
Ronnie Rubin
Danielle R. Adams
Steven C. Marcus
Publication date: 01-01-2018
Publisher: Springer US
Published in: Administration and Policy in Mental Health and Mental Health Services Research / Issue 1/2018
Print ISSN: 0894-587X
Electronic ISSN: 1573-3289
DOI: https://doi.org/10.1007/s10488-016-0776-8

At a glance: The STEP trials

Springer Medicine

Concordance Between Administrator and Clinician Ratings of Organizational Culture and Climate

Abstract

At a glance: The STEP trials

Springer Medicine

Abstract

Please log in to get access to this content

Other articles of this Issue 1/2018

Mental Health Service Use Among Young Adults: A Communication Framework for Program Development

Identifying and Reducing Disparities in Mental Health Outcomes Among American Indians and Alaskan Natives Using Public Health, Mental Healthcare and Legal Perspectives

Association Between Mental Health Staffing Level and Primary Care-Mental Health Integration Level on Provision of Depression Care in Veteran’s Affairs Medical Facilities

Making the Case for a Feasible Evaluation Method of Available E-Mental Health Products

The Relationship Between Race, Patient Activation, and Working Alliance: Implications for Patient Engagement in Mental Health Care

Monitoring Treatment Progress and Providing Feedback is Viewed Favorably but Rarely Used in Practice