Implementing the Illness Management and Recovery Program in Community Mental Health Settings: Facilitators and Barriers

The illness management and recovery program is a standardized intervention that was developed on the basis of a thorough review of research on teaching illness self-management strategies to people with severe mental illness ( 1 ). Five empirically based strategies were identified from this body of research and incorporated into the illness management and recovery program: providing psychoeducation to improve understanding about mental illness and treatment, using cognitive-behavioral approaches to improve medication adherence, providing training in the prevention of psychiatric relapses, using social skills training to strengthen social support and buffer stress, and teaching coping skills to reduce the severity and distress of persistent symptoms. Important aspects of the program are the emphases on helping clients set personally meaningful goals for recovery and a strong therapeutic alliance aimed at achieving these goals. Illness management and recovery can be provided either in an individual or group format and generally takes between five and ten months to complete. Consumers progressively work through ten modules. These modules are listed in the box on the next page.

Various pilot studies have suggested that illness management and recovery can significantly improve client functioning in various psychosocial domains ( 2 , 3 ). These include ability to cope with symptoms, basic functioning, and overall well-being. A recent randomized controlled trial indicated that clients assigned to illness management and recovery showed greater improvements in knowledge, progress toward goals, and overall outcomes, compared with those who received care as usual ( 4 ).

The Surgeon General's report concludes that interventions such as illness management and recovery are not available to a great majority of adults with severe mental illnesses ( 5 ). The government and consumer advocacy organizations recognize that facilitating the widespread adoption of these interventions in community mental health care settings is an urgent priority ( 6 , 7 , 8 ). However, there is little research on barriers and facilitators to the effective implementation of illness management and recovery. Addressing this issue will fill a significant gap in the literature ( 5 , 6 , 7 , 8 , 9 ).

Given the evidence supporting illness management and recovery as an effective practice, the intervention was selected for a study of implementation processes as part of the National Implementing Evidence-Based Practices Project ( 10 ). This project examined the process and outcome of implementing these practices in various community mental health centers across the United States over a two-year period. Fifty-three centers participated in this research, 12 of which implemented illness management and recovery.

These centers received an implementation resource kit for the illness management and recovery program and received training aimed at assisting implementation. The resource kit included a manual, handouts for each module, practitioner guidelines for each module, information brochures, an introductory video, a practitioners' training video, a fidelity scale, and outcome measures. Sites also received training and consultations over a two-year period. The first year involved the delivery of training by a consultant trainer, who conducted an initial one- or two-day workshop, followed by further training and consultation as requested. There was variation across sites in the use of the consultant trainer's services. At some sites, the consultant trainer spent at least one day a month at the center over the two-year period. At other sites, training was provided only once per quarter over the first year of implementation. Each site appointed an illness management recovery program leader. These components became known as the "evidence-based practice implementation model." The study presented here aimed to discern prominent facilitators and barriers to successful implementation of illness management and recovery by qualitatively analyzing the implementation processes at these 12 centers.

Methods

Procedures

Four states volunteered to implement illness management and recovery. More detailed information is given on participating states, relevant state-level data, and the role of state mental health authorities in other articles ( 11 , 12 , 13 ). To summarize, various mechanisms were used among these states to recruit public-sector community mental health agencies as study sites. Some chose among solicited proposals; others used less formal procedures. At each site, illness management and recovery was implemented in programs for people with severe mental illness. Implementation efforts occurred over a two-year period, from 2002–2003 to 2004–2005 (sites had different start dates).

Standardized instructions (described below) regarding systematic observation of implementation efforts were designed and distributed by a central coordinating center (Dartmouth Psychiatric Research Center) to ensure rigor and comparability across sites. Each site had a supervised researcher (an implementation monitor) who functioned as an independent observer of implementation, documenting the process both qualitatively and quantitatively. Following the instructions given by Dartmouth Psychiatric Research Center, the implementation monitor made monthly visits (occasionally by phone) that were written up as field notes. The implementation monitor also conducted interviews every six months with the illness management and recovery program leader and the consultant trainer. These were taped and later transcribed. The implementation monitor also wrote down his or her own responses to the same questions posed to the program leader and the consultant trainer. These interviews followed a standardized topic guide designed to elicit opinions and experiences regarding barriers to and facilitators of implementation. Topics covered in the interview included "reasons for success," "reasons for shortcomings," and "assessment of the implementation package." Examples of questions include, "For those elements that are not high-fidelity, what has gotten in the way?" The implementation monitor also interviewed the agency director and the clinical director at baseline and 24 months; all these interviews were taped and transcribed. Transcriptions and field notes were imported into Atlas.ti qualitative software.

The quantitative measurement of implementation performance was based on assessments of fidelity to the practice model, which were conducted at baseline and every six months thereafter. Further information on these processes is given in another article ( 10 ). Consultant trainers and implementation monitors conducted these assessments by using a standard illness management and recovery fidelity scale with 13 items, such as the use of motivational interviewing, the use of cognitive-behavioral techniques, follow-up on personal recovery goals, and involvement of significant others. Interrater agreement was evaluated across pairs of independent raters and found to be high for this fidelity scale (intraclass correlation=.97, N=50). The Dartmouth Medical School Institutional Review Board approved the overall study, and similar approval for each state was obtained through the local principal investigator's affiliated university.

Analysis

Twenty-six dimensions hypothesized to be important to the implementation process, nested within five broad domains, were designated through expert consensus a priori after review of relevant literature ( Table 1 ). The five domains were prioritization, leadership, workforce, workflow, and reinforcement. This deductive a priori approach to data analysis is recommended as optimal for large multisite, multi-investigator studies. It creates standard categories that can be compared across sites with ease ( 14 ).

Enlarge table

Implementation monitors were instructed to code the transcribed qualitative interviews and the site-visit field notes in Atlas.ti according to this dimensional schema. Once final interviews and site visits were completed and coded, implementation monitors generated 26 Atlas.ti output files (a list of all text allotted under a code) for each dimension. Through careful content analysis of this list, implementation monitors then judged the bearing that each dimension had as a facilitator of or barrier to overall implementation, summarizing their judgment in a dimensional display table. Each dimension was also given a summary score from 2 (high facilitator) to -2 (high barrier). This summary score was created in order to facilitate cross-site comparison and also to allow analysts a means of quickly assessing and categorizing the impact of a dimension.

At the end of the two-year implementation period, implementation monitors wrote summary site reports in which the dimensional display table was included, along with a brief synopsis discussing and analyzing the implementation process over the 24 months in narrative format. These reports followed a standardized template.

For this article, the site reports were assembled in rank order from highest fidelity to lowest fidelity. The first and second authors (hereafter referred to as evaluators) then independently reviewed and evaluated each site report. This form of multiple evaluation is considered a robust check and balance on rigor in qualitative research ( 15 ). The overall aim was to distill prominent facilitators and barriers to implementation. Prominence was defined as dimensions that were labeled with a 2 or a -2 in the dimension display, that co-occurred in the synopsis, and that were described in strong, forceful language in both the dimension display and the synopsis.

Evaluators listed prominent dimensions for each site report, sometimes synthesizing dimensions into broader themes where the site reports implied dimensional overlap. This process resulted in two lists, one for each evaluator, documenting evaluator-identified barriers and facilitators for each site. Once this process was finished, the evaluators compared their lists of overall dimensional and thematic extractions, formulating the final schema presented in the results. At this stage, the two last authors read a subsample of site reports in order to examine the verisimilitude of the first two authors' final schema. All authors agreed on four broad emergent themes as strongly influencing the success (or lack thereof) of implementation.

Results

The four factors emerging as prominent themes affecting implementation were leadership, organizational culture, training, and staff and supervision. These are described in turn below. These themes are broad and subsume some of the narrower dimensions listed in Table 1 . The results thus blend a priori themes deduced before the study began and a posteriori themes induced from the empirical data. This is a common approach in qualitative research, providing structure to the analytical process while simultaneously giving the perspicacious analyst freedom to build and synthesize in light of incoming data ( 14 ). There is obvious overlap between themes; however, they are presented separately for ease of comprehension. Two case studies are presented at the end of the Results section to embody the impact of the emergent themes on implementation.

Leadership

As seen in Table 1 , leadership was preidentified at the start of the study as a broad domain potentially influencing implementation. Strong and active leadership appeared to characterize high-fidelity sites, with both evaluators identifying leadership as the strongest facilitator at the four highest-fidelity sites. Leadership was exercised at various organizational strata. We identified four strata of leadership that had an influence on implementation. These were leadership by the state mental health authority, the agency director, the community support program director, and the illness management and recovery program leader. Implementation appeared to be most effective when there was creative synergy between leadership at these different levels. This was often the case, because strong leadership at higher levels generally permeated down to leaders at lower levels. Similarly, we found that low-fidelity sites were marked by a leadership void. Both evaluators identified this as the strongest barrier at the four lowest-fidelity sites. Some sites appeared to lack state mental health authority leadership, whereas others lacked agency or program leadership. Again, this operated in a synergistic manner, with low-fidelity sites lacking leadership across the organizational spectrum. For example, in one state, the state mental health authority was underfunded and appeared to lack the time to champion new programs, such as illness management and recovery. This was mirrored by a lack of enthusiasm displayed by leaders at community mental health centers in the state.

Organizational culture

Organizational culture was not one of the preidentified domains or dimensions listed in Table 1 . However we created this superordinate theme after noticing that preidentified dimensions such as "attitude," "reward," "policies," and "change culture" all reflected different components of what could broadly be labeled "organizational culture." The highest-fidelity sites were marked by a strong culture of innovation accompanied by a positive attitude to new practice in general and to illness management and recovery in particular. This may be a corollary of previously described state mental health authority and agency leadership, policies, and attitudes that encourage innovation and cutting-edge practice. Both evaluators identified this theme as one of the top three facilitators in four of the five highest-fidelity sites. These sites had successfully implemented evidence-based practices in the recent past and considered themselves models of vanguard agencies. These agencies thus embraced implementing the illness management and recovery program as an opportunity to develop expertise in a new, recovery-oriented, and consumer-centered approach. In contrast, low-fidelity sites appeared much more conservative and marked by organizational inertia. These sites had not embraced new practices in the recent past and expressed a strong preference toward the status quo. Attitudes toward practice change were generally negative. Implementing illness management and recovery was thus seen as a burden or a threat rather than an opportunity. For some agencies, maintaining the status quo meant staying afloat in troubled waters, marked by diminishing budgets and state mental health authority instability, with the view that implementing a new practice would consume precious resources and energy.

Training

Training was one of the preidentified dimensions hypothesized as important in the implementation process. We found that the provision of high-quality training in illness management and recovery by competent and respected trainers appeared to be a key factor in some of the high-fidelity sites, with one evaluator noting training as an important facilitator at three of the top six sites and the other evaluator noting its import at two of the top six sites. In low-fidelity sites, the picture was more complex. In a few lower-fidelity sites, program leaders stated that training was insufficient and of low quality. However in the four lowest-fidelity sites, neither evaluator identified training as a significant barrier. When taking the wider data set into consideration, we cautiously posit that the few complaints about trainers by program leaders were due more to interpersonal conflict or to an effort to shift blame. In fact, the quality of training was generally extolled at the four lowest-fidelity sites by the implementation monitor, the program leader, and other agency staff. This suggests that training may act synergistically in the presence of other facilitating variables, such as strong leadership. However, strong training may be insufficient to overcome the tide of wider macro-level barriers, such as lack of leadership and an inert organizational culture. This may especially be the case where intensity of training is dependent upon leadership request. Leaders in inert organizational cultures were not proactive in engaging trainers or incorporating their suggestions. Training therefore appeared to have some influence on successful implementation, but it appeared dependent upon strong leadership and an organizational culture that was open to change.

Staff and supervision

Supervision was one of the 26 preidentified dimensions posited as important to implementation. There was no dimension labeled "staff," although many comments related to staff skills, attitudes, and abilities were coded under dimensions such as "staffing," "attitude," and "mastery." The data suggested a strong link between quality and commitment of supervisors and their staff. The highest-fidelity sites were characterized by staff that was strongly committed to consumer well-being and recovery, displaying a positive attitude to the new practice. Many of these staff members also had previous experience with interventions involving similar clinical skills (for example, cognitive-behavioral therapy) and were eager to learn and practice something considered innovative and cutting edge. Supervisors in high-fidelity sites seemed equally convinced of the desirability and efficacy of illness management and recovery. This intervention was seen as an opportunity for staff and supervisors to expand their skills and experience. One evaluator identified staff and supervision in four of the six highest-fidelity sites, the other in five of them.

In contrast, low-fidelity sites had serious staffing problems across many dimensions. Many had high turnover of frontline practitioners, seemingly as a result of low pay, demoralization, and unappreciative or unsupportive leadership. Staff and supervisors at these low-fidelity sites seemed uninterested in learning new practices, and they seemed to perceive illness management and recovery implementation as adding to an already heavy workload. Some were also unconvinced about illness management and recovery as an intervention. For example, staff at some low-fidelity sites complained that the written materials were dense and difficult for clients, especially those who had literacy problems, whereas staff at high-fidelity sites did not express this concern. Others did not see how illness management and recovery differed from extant "wellness" programs. This negative attitude among staff and supervisors could be an inevitable outcome of lack of leadership and negative organizational culture. Both evaluators identified this as a significant barrier in four of the bottom six sites.

Case studies

The following case studies are included in this article to embody the themes outlined in the Results section. The first is from the highest-fidelity community mental health center; the second, from the center scoring second lowest in fidelity. The latter was chosen because the lowest-fidelity site had missing data.

High-fidelity case study . This site had a baseline fidelity of 2.2, which rose to 4.2 at six months and reached 4.5 at two years. (Fidelity was measured with the 13-item fidelity scale mentioned in the Methods section. Possible scores range from 1 to 5, with higher scores indicating better fidelity.) All four factors identified above coalesced at this site. The implementation monitor noted that "the local mental health authority is very pro evidence-based practice—[it] had representatives who has [sic] historically provided forward-thinking and fiscal guidance and support for the agencies' new initiatives." Indeed the state mental health authority provided generous resources to support the project throughout the two-year implementation period. This seemed to encourage clinical leadership, which was also strong.

The implementation monitor noted that "The team leader and the program leader were able to provide excellent supervision and mentoring in house. … Clinicians take pride in their programs." The agency has a well-utilized open-door policy, where staff can seek advice from any available clinical professional. This suggests a strong agency commitment to mentoring, as well as proactive staff willing to initiate supervisory contact. There is a strong culture of innovation at this agency, which is strongly recovery oriented. The implementation monitor noted that the "agency is committed to recovery … [it] has [a] history of striving for excellence and innovation." It was regularly chosen as a site for pilot studies and had successfully implemented other innovative practices already. The community support program leader stated that this was done "because it fit with our mission of balancing practicality with high standards. It was very successful, beginning three years ago and still going strong." Good leadership manifested itself in hiring and allocating decisions. The consultant trainer noted that the staff were "enthusiastic and energized … filled with high-achievers" who were a model of success. From this pool, illness management and recovery clinicians were chosen on the basis of their being "eager" or "positive." The implementation monitor noted that "a major strength is the level of education and the training of staff selected for illness management and recovery. Most have master's degrees and experience with evidence-based practices." Leaders, staff, and consumers utilized regular feedback from the research team to improve performance. News of success further motivated involvement, and action steps were taken to ensure continued success. All this intertwined to ensure smooth implementation at this site.

Low-fidelity case study. This site had baseline fidelity of 1.6, which rose to 3.6 at six and 18 months but dropped to 2.8 at 24 months. There was a leadership void at all levels of implementation. The implementation monitor noted that "A great deal of financial and leadership upheaval at the state mental health authority in early implementation led to decreased involvement at the state level. Competing commitment led agency leaders to become less interested … the evidence-based practice program leader was viewed as lacking in initiative and not successful at identifying and removing barriers." These problems in leadership appeared to demoralize staff, to the extent that by month 12 all illness management and recovery therapists had resigned. The implementation monitor noted that "Several clinicians stated that their reason for leaving was because of disagreement with the evidence-based practice program leader's leadership. The clinicians felt the program leader was not responsive to their needs and was too controlling." For several months, the program leader was the only illness management and recovery practitioner and had reduced ability to provide illness management and recovery services because of time spent recruiting new clinicians. These newly hired clinicians were very inexperienced. They reported trouble understanding illness management and recovery and were skeptical about its benefits. The agency was not marked by a culture of innovation. It had never attempted to implement an evidence-based practice in the past. The implementation monitor noted that its recent past was "a period of great instability and lack of funding." The implementation monitor reported that the consultant trainer attempted various strategies to energize the leaders and clinicians at this agency, such as providing additional training in illness management and recovery for new clinicians. However, these met with very limited success, because agency inertia and indifference appeared to work against the consultant trainer's efforts to advance implementation.

Discussion and conclusions

The analysis identified four themes that contributed to the success (or lack thereof) of implementing the illness management and recovery program: leadership, organizational culture, training, and staff and supervision. Among these factors, the role of leadership appeared to be particularly important. Strong leadership was noted to be present at all of the high-fidelity sites. Leadership at the highest levels of state mental health authority and the agency director interacted synergistically with leadership at the community support program and illness management and recovery program levels. In contrast, a leadership void characterized the low-fidelity sites. Although the other three themes were generally present at successful implementation sites and absent at unsuccessful sites, for each theme there were exceptions to the rule. That is, a few high-fidelity sites lacked a strong organizational culture, excellent staff and supervision, or good training and consultation (though never all three), whereas some low-fidelity sites were strong on some of these other factors (again rarely all of them).

The data suggest that the four themes interact with one another in contributing to successful program implementation, with stronger factors compensating for weaker ones. Figure 1 depicts a model summarizing these hypothesized relationships. This figure is grounded in the empirical data analyzed in this article, but it may best be considered a model for further testing, rather than a precise specification of processes.

The importance of leadership to organizational change and innovation has been frequently noted ( 16 , 17 , 18 ). We speculate that the role of leadership is especially critical in situations in which there are few external pressures, resources, or contingencies that support the change, which is often the case for clinics implementing new practices such as illness management and recovery. Indeed, this was generally the case for the agencies in this study. Leadership may be particularly important for new practices, such as illness management and recovery (in contrast to older practices, such as assertive community treatment), in that it can forcefully counter the inevitable skepticism and doubt encountered when new practices are being implemented. This can be seen in our data, where many frontline workers in low-fidelity sites were skeptical of illness management and recovery. Resources are important to implementing and sustaining changes and improvements in organizations ( 19 , 20 ). Effective leadership may facilitate access to critical resources within an organization that enable changes in practice, such as supplementary training and consultation and high-quality supervision. Strengthening leadership on the ground may be a helpful intervention in ensuring successful implementation. This may come through support and incentives from state mental health authorities ( 11 , 12 , 13 ).

Leadership may also play an important role in creating an organizational culture of innovation and change. Organizational culture involves the shared expectations, perceptions, and attitudes among providers that contribute to the psychological climate of how people perceive their work environment and the customs of how work is done ( 21 ). Staff members who feel valued by their agency and an organizational culture that embraces change and is not overly averse to risk may facilitate the implementation of a new practice, despite the uncertainties of long-term funding and fiscal support for that practice. Staff at low-fidelity sites tended to view implementation of illness management and recovery as a burden and distraction from business as usual, rather than an opportunity to improve the quality of their services. Organizations characterized by a culture that is rule bound and invested in maintaining the status quo may also be susceptible to one important obstacle identified by Davidson and colleagues ( 22 )—the belief that the current emphasis on recovery-oriented services is a passing fad. Because illness management and recovery somewhat redistributes power and agency back to the client and is formulated within the wider recovery paradigm, it may be particularly difficult to implement at agencies not fully committed to recovery. Agency collective beliefs about client capabilities may reinforce the inherent inertia of all organizations, further reducing the likelihood of successful change ( 23 ). Strategizing to overcome these potential barriers should be a key priority for those leading the implementation of illness management and recovery in community mental health settings.

Acknowledgments and disclosures

This research was supported by grant 036805 from the Robert Wood Johnson Foundation, contract 280-02-8070 from the Center for Mental Health Services of the Substance Abuse and Mental Health Services Administration, a gift from the West Family Foundation, and funds from the Johnson & Johnson-Dartmouth Community Mental Health Program. The authors thank the members of the National Implementing Evidence-Based Practices Project team and the participants at the sites.

The authors report no competing interests.