Adding Cognitive Remediation to Employment Support Services: A Randomized Controlled Trial

Many people with serious mental illness want to work (1). Generally, work provides a sense of purpose and improves quality of life for people recovering from serious mental illness. More specifically, work is associated with improved self-esteem, less severe symptoms, better economic standing, greater satisfaction with finances, and an improved sense of recovery (2, 3). Despite the high unemployment rate among patients with serious mental health problems (4), many have a strong interest in working throughout adulthood (5).

Individual placement and support (IPS) is an evidence-based program designed to improve employment success for individuals with mental illness (6, 7). Several systematic reviews and meta-analyses have found that employment rates from IPS programs are more than twice as high as rates from standard vocational rehabilitation (8–10). When IPS is successful, clients typically find jobs in the first 3 months of receiving services. Although this result is quite robust, a substantial portion (45%) of clients also fail to benefit from the service despite many months of effort (6).

Among those who want to work, cognitive deficits often present a barrier to competitive employment. Cognitive difficulties are apparent before the onset of psychosis and span multiple domains, including attention, speed of information processing, working memory, verbal learning and memory, and executive functioning (11, 12). Furthermore, cognitive difficulties persist into remission and predict overall adaptive functioning. These deficits are associated with poorer employment outcomes and may partly explain the discrepancy between desire and ability to work (13).

Cognitive remediation therapy (CRT) is an evidence-based treatment for cognitive impairments in schizophrenia. Over the past several decades, >50 randomized controlled trials have been conducted on cognitive remediation for persons with serious mental illness. Most meta-analyses have concluded that CRT effectively improves cognitive functioning in patients with schizophrenia (14–16). Findings from these meta-analyses suggest that the efficacy of CRT is enhanced when it is delivered in conjunction with other evidence-based programs, such as supported employment (14).

The Cognitive Remediation Expert Working Group (17) defined CRT as a “behavioural training intervention targeting cognitive deficit using scientific principles of learning, with the ultimate goal of improving functional outcomes.” This group identified the core components of CRT as cognitive remediation therapists, cognitive exercise, procedures to develop problem-solving strategies, and approaches that facilitate transfer to functioning in the world.

This study focused on a subgroup of clients in an IPS program who had not found work after 3 months of employment support services and thus were thought to benefit from the addition of cognitive remediation. Our study design was based on previous research that found that adding cognitive remediation for clients who had not responded to supported employment after 3 months resulted in improved cognitive functioning and better employment outcomes during the follow-up period (18). This study used the same criterion to define a diminished likelihood of employment success, focusing on IPS clients who had not found work after 3 months of using employment support services. Our first hypothesis was that the addition of cognitive remediation to IPS for this subgroup would improve work outcomes, including finding competitive employment, working more hours per week, and earning more in wages. Our second hypothesis was that CRT would improve cognitive functioning and thereby mediate improved employment outcomes.

Methods

The study procedures were registered with ClinicalTrials.gov (NCT03483701) and approved by the Fraser Health Research Ethics Board (2016-096) and the University of British Columbia Clinical Research Ethics Board (H18-01826).

The study comprised a randomized, single-blind design focused on IPS clients who had not found work after 3 months of using employment support services. We used a power analysis to determine the sample size in this study and estimated an effect size of d=0.49 on competitive work on the basis of a previous study with the same treatment approach (18). With a standard α=0.05 (one-tailed) and power of 0.8 in a Mann-Whitney test (two groups), we estimated that a sample size of N=106 participants was needed to detect statistically significant differences.

Sites

The study initially took place at 10 community mental health centers associated with Fraser Health Authority in Greater Vancouver, British Columbia, Canada. After one-third of the recruitment period had passed, the study expanded to four additional centers in the adjacent health authority, which were part of the same IPS program and were managed by the same agency. The study was conducted from April 2017 to February 2020.

Participants

All participants had been enrolled in the IPS program for at least 3 months. Eligibility criteria included being ages 19–60 years, currently not working or being dissatisfied with current work and looking for different work, showing expressed interest in work, and showing no evidence of traumatic brain injury or loss of consciousness for >5 minutes.

Participants agreed to be randomly assigned to either continue IPS alone or receive CRT in addition to IPS. Random assignment was performed by the principal investigator (D.H.E.) after the initial assessment was complete, in accordance with a list prepared before the start of recruitment. That list, in permuted blocks of four drawn from a random number table, assigned about 50% of participants to each group (CRT+IPS, N=50; IPS alone, N=47) in chronological order, according to their date of consent. To protect against bias, the principal investigator had no contact with the participants before the random assignment and had no access to the baseline measures.

Assessments

All participants were assessed at three time points: baseline and 3- and 9-month follow-ups. The baseline assessment included a brief diagnostic interview that used the Mini-International Neuropsychiatric Interview, version 7.0.2 (19), to confirm a diagnosis and the Brief Psychiatric Rating Scale (BPRS-24) to assess symptom severity (20). Cognitive functioning at baseline and 3 months was measured with the Measurement and Treatment Research to Improve Cognition in Schizophrenia (MATRICS) Consensus Cognitive Battery (21), which provides measures within seven cognitive domains: speed of processing, attention and vigilance, working memory, verbal learning, visual learning, reasoning and problem solving, and social cognition. The social cognition subtest was replaced with Trails-B (22) to provide an estimate of executive functioning. Outcome measures at the 3- and 9-month follow-up interviews included success at gaining a competitive job, total hours of competitive employment, and wages earned. Assessments were conducted by trained master’s-level interviewers who were blind to group assignment. If the blind was broken between the 3-month and 9-month assessments, the assessment was completed by another interviewer. Participants were provided $30 gift cards for their participation in the assessments but were not remunerated for participation in the IPS or CRT interventions.

IPS

IPS services provided to clients in both groups were based in outpatient mental health clinics. Clients in both groups regularly met with IPS employment counselors to identify work goals, prepare résumés and cover letters, and discuss specific job application processes. Client meetings were held both in the mental health clinics and in community locations where privacy was assured.

The project included two fidelity reviews, held 1 year apart, to ensure that IPS services for all clients were delivered according to the standard. The trained independent auditor used the IPS Fidelity Scale (23) to assess staffing, organization, and services. The fidelity scale has 25 items, each using a 5-point Likert scale, such that scores range from 25 to 125, with higher scores indicating better fidelity. The first fidelity review found that the program delivered services with good fidelity (score of 100 out of 125). The second fidelity review found that the program delivered services with fair fidelity (93 of 125), reflecting changes in management structure and personnel that resulted in less supervision time and focus.

CRT: Thinking Skills for Work (TSW)

Once baseline assessments were completed, about half of the participants (N=50) received 3 months of CRT, consisting of two concurrent components: computerized cognitive exercise practice and learning coping strategies for cognitive challenges with a registered psychologist. The first component, computerized cognitive exercises, utilized CogPack, version 9.2. Participants engaged in 24 sessions of computerized cognitive exercises, approximately twice per week for 12 weeks, at a mental health center or on their personal computer at home. Following a standardized training curriculum (13), participants completed a broad range of exercises, including those focused on attention and concentration, information processing speed, learning and memory, and executive functions. Participants received performance scores reflecting accuracy and speed after completion of each exercise; scores were recorded by the participant and were referenced later to reinforce performance progress. Unlike previous studies of the TSW program (18, 24), participants did not receive strategy coaching on the computerized cognitive exercises.

The second CRT component, learning strategies to cope with cognitive challenges, comprised weekly individual meetings between the CRT participants and a psychologist. Concurrent with the computerized exercises, the sessions focused on managing cognitive challenges that could interfere with finding and keeping work, following the TSW manual (13). The topics discussed included coping with attention and concentration difficulties, coping with memory difficulties, developing routines for job searches and at home, improving thinking speed, planning, and problem solving. The CRT intervention was provided by a clinical psychologist (A.M.N.B.), with consultation provided by a senior psychologist (Susan McGurk, Ph.D., Boston University), who designed the TSW program.

Analyses

All data were analyzed in SPSS, version 21. Group differences in baseline characteristics and attrition over time were assessed with chi-square and t tests. Because data for employment hours worked and wages earned were skewed, Mann-Whitney U tests were conducted to compare employment outcomes between the two treatment groups. Differences between the treatment groups in change in cognition were assessed with repeated-measures univariate analysis of variance (ANOVA), with cognitive variables as within-participant factors and treatment group as a between-participant factor. Analyses were conducted by using intent-to-treat principles. Specifically, for a few participants who were lost to attrition, the employment outcomes were set to zero.

Results

Sample Characteristics

In total, 125 IPS participants were assessed for eligibility, and 97 met the inclusion criteria. Eligible participants completed the baseline assessments and were then randomly assigned to CRT+IPS (N=50) or the IPS-only (N=47) group. At baseline, the two groups showed no statistically significant differences in age, education, symptom severity, and psychiatric diagnosis (Table 1). Similarly, the two groups did not significantly differ in cognitive measures.

Sample retention over the follow-up period was high: 89% (N=86) of participants completed assessments at the 9-month follow-up. Of the 11% (N=11) for whom follow-up data were not collected, only one participant actively withdrew consent. We observed no differential attrition by age, education, or sex (Table 1). Three of four symptom measures on the BPRS-24 showed no differences between those who were lost to follow-up and those who completed the 9-month assessment. However, those who had dropped out by the 9-month assessment had higher baseline scores on BPRS-24 dysphoria (t=−2.37, df=94, p<0.05). (A CONSORT diagram is available in an online supplement to this article.)

Cognitive Remediation

In the CRT+IPS group, participants completed on average 10 of 12 individual sessions on coping strategies for managing cognitive challenges, averaging 11.6 hours. Completion of the self-directed computerized cognitive exercises was modest: CRT participants completed on average 12 of 24 computerized training sessions, with 70% (N=35 of 50) completing six or more sessions. The completion of six sessions of computerized cognitive training has been defined by McGurk and colleagues (18) as the cutoff for minimal exposure to the TSW program.

Employment Outcomes

A chi-square analysis comparing the number of participants employed after 3 months indicated that significantly more participants in the CRT+IPS group (N=25 of 48, 52%) were working than those in the IPS-only group (N=12 of 42, 29%) (χ²=4.37, df=1, p<0.05, odds ratio [OR]=2.83, 95% confidence interval [CI]=1.22–6.60). Differences between the two groups were maintained at the 9-month follow-up (CRT+IPS: N=27, 60%; IPS only: N=14, 34%; χ²=6.43, df=1, p<0.01; OR=2.91, 95% CI=1.27–6.65) (Figure 1). Participants in the CRT+IPS group worked more hours than those in the IPS-only group at 3 months (although these results were not statistically significant) and at 9 months (U=893, p<0.05) (Figure 2). The CRT+IPS group also earned more wages than the IPS-only group at both 3 months (U=153, p<0.05) and 9 months (U=204, p<0.01) (Figure 3).

Cognitive Outcomes

At baseline, the MATRICS scores for both groups were approximately one standard deviation below the norm for healthy adults, and no statistically significant differences were detected between the two groups. When we used the repeated-measures ANOVA, the composite cognitive score showed a time effect, whereby both groups showed significant improvement in cognitive scores over time (F=25.02, df=1 and 88, p<0.001), but no significant time × group interaction. Examination of individual cognitive measures showed significant time effects for three cognitive domains—speed of information processing (assessed with the Trail Making Test and Brief Assessment of Cognition in Schizophrenia), working memory (Wechsler Memory Scale–III), and visual learning (Brief Visuospatial Memory Test–Revised)—which could be attributable to practice effects (Table 2). The only significant time × group interaction was in the domain of attention and vigilance (Continuous Performance Test), indicating that participants in the IPS-only group had significantly greater improvement, a result that was the opposite of what we had hypothesized.

To examine whether the overall null finding in cognitive outcomes was a result of the low rate of completion of the computerized training component, we compared the cognitive scores of participants who completed at least 50% (N=25) of the computerized training sessions (“treatment exposed”) with those who completed <40% (N=24) of the computerized training sessions (“treatment incomplete”). We observed significant interactions for only two of the nine measures of cognitive functioning (Hopkins Verbal Learning Test–Revised: F=8.87, df=1 and 46, p<0.01; Letter-Number Span: F=5.42, df=1 and 46, p<0.05); neither the composite measure nor the remaining seven individual measures showed significant differences. Thus, those in the CRT subgroup did not achieve consistent gains because of greater exposure to the computerized exercises compared with those in the treatment-incomplete subgroup.

Discussion

This randomized controlled trial examined the effect of adding CRT to supported employment services for participants who had not found work after 3 months. The findings of our trial suggest that CRT significantly improves work outcomes for mental health clients who are slow to succeed with standard IPS. The increase in competitive work for the treatment group was substantial, given that these clients were selected because they had not benefited from supported employment programs alone. Indeed, the percentages of clients receiving both CRT and IPS who found work (3-month follow-up: N=25 of 48, 52%; 9-month follow-up: N=27 of 45, 60%) were similar to the overall success rate for IPS (55%) reported in meta-analyses (6). Focusing on clients who have not benefited from supported employment alone resulted in a more efficient use of this limited resource.

Unlike previous studies (18, 24) reporting that participants in the CRT+IPS group have improved cognitive functioning compared with those receiving only IPS, we did not observe changes in composite cognitive scores as measured by the MATRICS, despite significant changes in employment outcomes for those in the CRT+IPS group. One potential explanation for the lack of effects of CRT on cognitive functioning in our study may be the absence of strategy coaching in the cognitive exercises. Some participants completed the computerized training program on their home computer, precluding an opportunity to provide consistent strategy coaching.

The absence of changes in cognitive scores did not appear to be the result of insufficient computerized practice, but a dosage effect has been noted in similar studies, indicating that cognitive improvements are associated with massed practice (a learning procedure associated with practice trials occurring closely together in time [14, 25]). In this study, improvements in cognitive scores were not necessary for improvements in employment outcome. These results are similar to findings from studies using cognitive adaptation training, a compensatory approach to cognitive training, that showed no evidence for cognition-mediated improvements in functional outcomes (26). However, these findings contrast with a recent study that found that compensatory cognitive training (CCT) in the absence of restorative cognitive training improves working memory but has no effect on attainment of competitive work (27). One possible explanation for the improvement in employment outcomes in the present study is the specificity and emphasis of the TSW program on how to deal with cognitive challenges in order to achieve employment goals. Although both CCT and TSW feature compensatory strategies to allow people to work around their cognitive impairments and focus on linking the use of strategies to goals and roles in the community, TSW specifically focuses on efforts to improve thinking skills to seek and keep employment. Another possible explanation relates to the usefulness of specific therapy components. Although the therapeutic alliance is central to psychotherapy, its potential role in TSW was not addressed in this study. However, cognitive improvements have been found to be associated with massed practice, but improvements in functioning were associated only with therapeutic alliance (25).

This study had several important limitations. First, to remain consistent with IPS principles, we kept exclusion criteria to a minimum. As in most studies with this clientele, we did not use cognitive impairment as an entry criterion into the trial. Improvement in cognitive abilities may be more difficult to detect if baseline cognitive abilities are relatively intact. Second, participant eligibility was not based on a single diagnosis, resulting in a diagnostically heterogeneous group composition, including a significant portion of participants with a mood disorder. The cognitive problems of individuals with a mood disorder tend to be less prevalent, less pervasive, and less disabling than the problems of those with psychotic disorders (28). Cognitive remediation may work differently for individuals with different psychiatric disorders—for example, persons with a schizophrenia spectrum disorder may benefit more from improving their cognitive abilities with computerized cognitive exercises, whereas individuals with a major mood disorder may benefit more from learning coping strategies for managing cognitive challenges that could be created or exacerbated by changes in mood; thus, the effect of psychiatric diagnosis on changes in cognitive abilities is unclear. However, the diagnostically heterogeneous group composition could also be viewed as a strength of the study because it increased the generalizability of the findings to the broader population of individuals who do not show an early benefit from using IPS services.

Future research is needed to assess the relative contribution of various CRT program components to cognitive functioning and work outcomes. The findings suggest that the TSW program leads to better employment outcomes. However, in the absence of explicit strategy coaching to improve performance on computerized cognitive exercises, we did not see changes in cognitive scores. More research is needed to examine whether the computerized cognitive training is critical for improving work outcomes, whether teaching coping strategies to address cognitive challenges is sufficient, or whether the two components produce a larger effect in employment outcomes when combined.

Conclusions

The efficacy of CRT for reducing cognitive impairment among people with severe and persistent mental illness is well established. In this study, we added the established components of CRT (17) to IPS for clients who were having trouble finding work and compared their outcomes with those of clients receiving IPS services alone. For these clients, the addition of CRT to IPS improved employment outcomes to the level usually seen in IPS services generally and thus resulted in a more efficient use of limited resources. However, these results also point to the need to better understand the components crucial to CRT and to match client characteristics to effective intervention strategies.