Community Mental Health Provider Modifications to Cognitive Therapy: Implications for Sustainability

When evidence-based psychosocial interventions (EBPs) are implemented and disseminated in practice settings, modifications to the EBP models are common, and perhaps even necessary. However, the frequency and impact of EBP modifications in mental health service settings remain poorly understood. The few studies that have investigated modifications to EBPs have focused on group interventions and have found that modifications are intended to promote a better fit between the intervention and the setting or recipients (1,2). However, understanding the impact of such modifications on key outcomes, including consumer-level outcomes such as symptoms and functioning, and on long-term implementation, is also critical (3). Yet the variety of possible modifications to EBPs has created difficulty in classifying such changes rigorously and reliably and in identifying their impact.

Better characterization of commonly occurring modifications is an important first step to understanding their impact on key outcomes of interest. In addition, data regarding common modifications in a particular context can guide professionals tasked with training and implementation in the numerous initiatives to implement psychosocial EBPs throughout public mental health settings (4–6). Thus, in this study, we applied a framework for classifying modifications to a range of modifications described by clinicians who had been trained in cognitive therapy, an established EBP.

Methods

Cognitive therapy teaches individuals to develop and use skills for modifying problematic beliefs and behaviors through a variety of interventions (7). Sessions are structured to include brief summaries of previous and current sessions, agenda and goal setting, skill development, feedback from clients, and practice assignments (7). Cognitive therapy has an extensive evidence base for a variety of psychiatric disorders (8).

This study was conducted in the context of a cognitive therapy training program within a provider network in a large urban mental health system between 2007 and 2012. Clinicians were trained to use cognitive therapy to address depression, suicidal behaviors, and conditions that commonly co-occur with depression among adults in community mental health settings (9). Training consisted of 24 workshop hours, followed by six months of weekly consultation. Twenty-seven clinicians agreed to participate in interviews regarding their experience using the therapy after the training. Twenty of them had demonstrated the ability to conduct cognitive therapy sessions at an acceptable level of skill and fidelity at the end of the training, and all indicated that they used cognitive therapy interventions in their routine practice. Sixty-seven percent (N=18) of the participants were female, and participants had a mean±SD of 5.7±1.6 years of experience working in mental health treatment settings. Eighty-one percent (N=21) had a master’s degree, 7% (N=2) had completed some graduate work (such as toward a master’s degree), and 12% (N=4) had a bachelor’s degree. Seventy percent (N=19) of participants were Caucasian, 19% (N=5) were black, and 4% (N=1) were Asian; 7% (N=2) were also Latino, and 7% (N=2) were multiracial or endorsed a different race or ethnicity.

The study was approved by university and municipal institutional review boards, and clinicians provided informed consent. Hour-long, semistructured interviews included specific questions that assessed participants’ implementation of cognitive therapy, including any modifications they made to its delivery or content and the rationale for those modifications.

We used a coding scheme to identify modifications to cognitive therapy that was previously developed on the basis of articles describing modifications to EBPs across various interventions and settings (10). The coding scheme consists of four contextual modifications (for example, change in setting or format), 12 content modifications (such as adding or omitting components), seven levels at which modifications could occur (such as consistently throughout a system or organization or only for particular clients), and a code for modifications to training or evaluation processes. Content modifications were rated for the level at which they occurred and for the type and nature of the modification. Modification codes were applied to interview responses related to cognitive therapy modification. The two primary study raters (one bachelor’s level and one doctorate level) overlapped on 23% (N=41) of the identified interview segments, thus allowing calculation of rater agreement. The primary rater’s data are reported, but where disagreement existed in the ratings that overlapped, the two raters arrived at consensus through discussion.

Results

Cohen’s kappa statistic was .80 for the nature of the modification and .95 for the level at which the modifications occurred, suggesting substantial to almost perfect interrater agreement (11).

Across the 27 interviews with clinicians, 175 modifications were identified. All clinicians described at least one modification. Table 1 lists modifications, their relative frequencies, and the number of clinicians endorsing each. Contextual modifications were rarely endorsed by this sample of clinicians, although delivery of cognitive therapy in different settings (such as at a client’s home) or through a different format (in a group or by telephone, for example) were occasionally noted.

The most common content modification was tailoring the intervention to the needs of the client. Most frequently, clinicians endorsed changing terminology or language. Loosening the structure of cognitive therapy, such as not following an agenda for therapy sessions, was also common. Clinicians acknowledged that they integrated elements of cognitive therapy into a different therapeutic approach when they believed that a full cognitive therapy protocol was not an ideal fit either for their client or, more commonly, for themselves as clinicians.

More than half of the clinicians acknowledged that they sometimes drifted or departed from a cognitive therapy approach. In contrast to most other modifications in the coding scheme, which often appeared to be strategically determined when considering the needs of individual clients, drift was described as occurring less strategically and more in reaction to challenging situations. In approximately half of the interview segments for which drift was identified, it appeared to signal a discontinuation of cognitive therapy with individual clients. In the remaining half, clinicians indicated that they would return to a cognitive therapy approach in a subsequent session if the client seemed more able or willing to engage in the treatment.

Other content modifications that were endorsed somewhat less frequently included integrating another intervention (such as motivational interviewing or spiritual counseling) into cognitive therapy, adding specific interventions or modules (for example, a relapse prevention module), lengthening or shortening sessions or protocol length, and skipping particular interventions or strategies. On a few occasions, clinicians described what they believed to be modifications that raters judged to be consistent with cognitive therapy principles (problem solving, for example).

Content modifications most commonly occurred to address the needs or clinical presentations of individual clients. However, one-third of the modifications occurred at the clinician level, meaning that clinicians indicated that there were certain modifications that they routinely applied to cognitive therapy to increase their own comfort level or sense of fit between cognitive therapy and their personal style. Clinicians also stated that they adapted cognitive therapy for clients in particular diagnostic or demographic groups. Modifications across an entire clinic or organization were rarely endorsed.

Discussion

In this study, we classified the types of modifications that clinicians made to cognitive therapy in community mental health settings. Four of the modifications identified in the coding scheme—tailoring, integrating cognitive therapy into other approaches, loosening the structured elements of the session, and drift—accounted for 65% of the changes that were identified. These changes have potentially important implications for several key implementation outcomes. Perhaps with the exception of drift, these modifications appear to improve clinician satisfaction with cognitive therapy, and clinicians believed that some modifications increased the intervention’s acceptability to clients. However, whether these changes affect other client-level outcomes, such as symptoms and functioning, remains to be seen.

Although no studies have examined the impact of adaptations that increase acceptability of cognitive therapy or other psychotherapies, some research findings imply that lower fidelity may have a negative impact on client-level outcomes (12). Additional research is necessary to fully understand the implications of such changes. More than half of the clinicians acknowledged that they at least occasionally drifted away from a cognitive therapy approach, and others indicated that they borrow elements of cognitive therapy but work primarily from another approach, which suggests that cognitive therapy may not be fully sustained in its original form throughout the system over the long term. Thus, when training clinicians in EBPs, it may be beneficial to proactively address the potential for modifications. Discussion and consultation about adaptation that upholds core elements of the intervention, as well as when and how another therapeutic approach might be appropriate, may provide useful guidance to clinicians who are likely to be confronted with complex client presentations.

Some limitations are important to note. First, ratings were based on clinician descriptions of modifications, rather than on observations of sessions. Other modifications may have occurred, or clinician and observer perspectives may have differed concerning what constituted a modification to cognitive therapy. Interviews provided valuable insight into why clinicians made changes to cognitive therapy, but the addition of observation may have led to greater precision. Also, we did not investigate the relationship between the modifications that were identified and implementation- or client-level outcomes. These are important topics for future research. Finally, our sample comprised a small number of clinicians from agencies in one mental health system and focused on a single EBP. Additional research on other EBPs, within other settings, is under way and will be necessary to better understand how, when, and with what impact EBPs are modified during implementation efforts.

Conclusions

This study provides further evidence that raters can reliably use a coding system to classify modifications to EBPs. In contrast to a previous study that applied this classification system to descriptions of adaptations that were mostly planned by treatment developers and researchers in advance of implementation (10), this report describes the variety of modifications that were made by individual clinicians when implementing an EBP in their everyday practice. Thus these findings have important implications for training and consultation as well as for future research on implementation and sustainability of EBPs. Further research on the impact of EBP modifications on implementation and client-level outcomes is necessary.