Supported employment for adults with severe mental illness
Abstract
Background
People who suffer from severe mental disorder experience high rates of unemployment. Supported employment is an approach to vocational rehabilitation that involves trying to place clients in competitive jobs without any extended preparation. The Individual placement and support (IPS) model is a carefully specified form of supported employment.
Objectives
1. To review the effectiveness of supported employment compared with other approaches to vocational rehabilitation or treatment as usual.
2. Secondary objectives were to establish how far:
(a) fidelity to the IPS model affects the effectiveness of supported employment,
(b) the effectiveness of supported employment can be augmented by the addition of other interventions.
Search methods
We searched the Cochrane Schizophrenia Group Trials Register (February 2010), which is compiled by systematic searches of major databases, handsearches and conference proceedings.
Selection criteria
All relevant randomised clinical trials focusing on people with severe mental illness, of working age (normally 16 to 70 years), where supported employment was compared with other vocational approaches or treatment as usual. Outcomes such as days in employment, job stability, global state, social functioning, mental state, quality of life, satisfaction and costs were sought.
Data collection and analysis
Two review authors (YK and KK) independently extracted data. For binary outcomes, we calculated risk ratio (RR) and its 95% confidence interval (CI), on an intention‐to‐treat basis. For continuous data, we estimated mean difference (MD) between groups and its 95% (CI). We employed a fixed‐effect model for analyses. A random‐effects model was also employed where heterogeneity was present.
Main results
A total of 14 randomised controlled trials were included in this review (total 2265 people). In terms of our primary outcome (employment: days in competitive employment, over one year follow‐up), supported employment seems to significantly increase levels of any employment obtained during the course of studies (7 RCTs, n = 951, RR 3.24 CI 2.17 to 4.82, very low quality of evidence). Supported employment also seems to increase length of competitive employment when compared with other vocational approaches (1 RCT, n = 204, MD 70.63 CI 43.22 to 94.04, very low quality evidence). Supported employment also showed some advantages in other secondary outcomes. It appears to increase length (in days) of any form of paid employment (2 RCTs, n = 510, MD 84.94 CI 51.99 to 117.89, very low quality evidence) and job tenure (weeks) for competitive employment (1 RCT, n = 204, MD 9.86 CI 5.36 to 14.36, very low quality evidence) and any paid employment (3 RCTs, n = 735, MD 3.86 CI ‐2.94 to 22.17, very low quality evidence). Furthermore, one study indicated a decreased time to first competitive employment in the long term for people in supported employment (1 RCT, n = 204, MD ‐161.60 CI ‐225.73 to ‐97.47, very low quality evidence). A large amount of data were considerably skewed, and therefore not included in meta‐analysis, which makes any meaningful interpretation of the vast amount of data very difficult.
Authors' conclusions
The limited available evidence suggests that supported employment is effective in improving a number of vocational outcomes relevant to people with severe mental illness, though there appears to exist some overall risk of bias in terms of the quality of individual studies. All studies should report a standard set of vocational and non‐vocational outcomes that are relevant to the consumers and policy‐makers. Studies with longer follow‐up should be conducted to answer or address the critical question about durability of effects.
PICOs
Plain language summary
Supported employment for adults with severe mental illness
People with mental health problems experience high rates of unemployment. There are various schemes delivering support to people with mental health problems who are trying to find employment. Supported employment tries to place people into competitive jobs. People are placed quickly in normal work settings where they receive intensive support and training from ‘job coaches’.
Individual placement and support (IPS) is a more specified scheme that includes: finding local jobs; a rapid job search; customer choice in what they want from the employment service; close working between employment and mental health teams; attention to people’s preferred job, their strengths and work experience; ongoing and, if necessary, long‐term individual support; and the benefits of counselling. Employment specialists act to identify people’s job interests, assist with job finding, give job support and engage other support services. IPS uses assertive outreach to deliver training, advice and vocational support in the community. Augmented supported employment is where employment support is given with other supplementary techniques, such as social skills training, motivational classes and various types of rehabilitation. Other approaches are many and varied, including: job workshops; job counselling; peer support; partnerships with business; and the Clubhouse model, which involves training, work experience, peer support and transitional employment and IPS because they do not search for immediate and competitive employment. However, all approaches involve periods of preparation, education and on‐the‐job training.
This review compares supported employment and IPS with other approaches for finding employment. Drawing from a total of 2259 people with mental health problems in 14 studies, the review has two main findings: 1) Supported employment increases the length and time of people’s employment; 2) People on supported employment find jobs quicker. Supported employment and IPS are better than other approaches in these two respects, but there is limited information or measurable differences on other important issues for service users.
For example, there is little information on issues such as improving quality of life, impact on people’s mental health, days in hospital and costs. Furthermore, the review built its main findings on limited statistical evidence drawn mainly from studies carried out in North America and Europe. Future studies should address a fuller range of information and outcomes. Longer studies are needed to see how long the effects of supported employment last.
This plain language summary has been written by a consumer Ben Gray from RETHINK.
Authors' conclusions
Summary of findings
| Supported employment versus other vocational approaches for adults with severe mental illness | ||||||
| Patient or population: patients with adults with severe mental illness | ||||||
| Outcomes | Illustrative comparative risks* (95% CI) | Relative effect | No of Participants | Quality of the evidence | Comments | |
| Assumed risk | Corresponding risk | |||||
| Other vocational approaches | Supported employment | |||||
| Employment ‐ obtained any job during the study (high=better) | Study population | RR 2.62 | 951 | ⊕⊝⊝⊝ | ||
| 202 per 10001 | 530 per 1000 | |||||
| Moderate | ||||||
| 260 per 10001 | 681 per 1000 | |||||
| Employment ‐ days in competitive employment (primary outcome) ‐ long term | The mean employment ‐ days in competitive employment (primary outcome) ‐ long term in the control groups was | The mean employment ‐ days in competitive employment (primary outcome) ‐ long term in the intervention groups was | 204 | ⊕⊝⊝⊝ | ||
| Employment ‐ days in any form of paid employment ‐ long term | The mean employment ‐ days in any form of paid employment ‐ long term in the control groups was | The mean employment ‐ days in any form of paid employment ‐ long term in the intervention groups was | 510 | ⊕⊝⊝⊝ | ||
| Employment ‐ job tenure for competitive employment (weeks) ‐ long term | The mean employment ‐ job tenure for competitive employment (weeks) ‐ long term in the control groups was | The mean employment ‐ job tenure for competitive employment (weeks) ‐ long term in the intervention groups was | 204 | ⊕⊝⊝⊝ | ||
| Employment ‐ job tenure for any paid employment (weeks) ‐ long term | The mean employment ‐ job tenure for any paid employment (weeks) ‐ long term in the control groups was | The mean employment ‐ job tenure for any paid employment (weeks) ‐ long term in the intervention groups was | 423 | ⊕⊝⊝⊝ | ||
| Time (days) to first competitive employment ‐ long term | The mean time (days) to first competitive employment ‐ long term in the control groups was | The mean time (days) to first competitive employment ‐ long term in the intervention groups was | 204 | ⊕⊝⊝⊝ | ||
| *The basis for the assumed risk (e.g. the median control group risk across studies) is provided in footnotes. The corresponding risk (and its 95% confidence interval) is based on the assumed risk in the comparison group and the relative effect of the intervention (and its 95% CI). | ||||||
| GRADE Working Group grades of evidence | ||||||
| 1 Median control risk across studies. | ||||||
Background
Description of the condition
People who suffer from severe mental disorder experience high rates of unemployment. A review of eight controlled trials demonstrated that employment rates for people with schizophrenia, even with optimal support, ranged from only 30% to 80%, with a median of 60% across these studies (Bond 2004). These low employment rates reflect the disability caused by severe mental illness, but they may also reflect discrimination (unemployment rates are higher than in other disabled groups) (ONS 1998) and the low priority given to employment by psychiatric services (Lehman 1995). Despite high unemployment rates amongst people with severe mental illness, surveys have consistently shown that most want to work (Hatfield 1992; Lehman 1995; Shepherd 1994). Mental health issues said to be linked to unemployment include: cognitive impairment, psychotic symptoms, negative symptoms, fear of losing benefits, stigma and lack of access to employment services (Bond 1991; Bond 2008b; Cook 2006; Rosenheck 2006; Rutman 1994).
Description of the intervention
Supported employment is an approach to vocational rehabilitation that involves trying to place clients in competitive jobs without any extended preparation (Bond 1992). Originally developed for people with learning disabilities, supported employment has been defined as 'paid work that takes place in normal work settings with provision for ongoing support services' (Becker 1994; Bond 1999). Proponents of supported employment had two objections to pre‐vocational training, which adheres to the key principle that a period of preparation is necessary before entering competitive employment (Bilby 1992; Bond 1997a). First, they argued that it promoted dependency and deterred clients from finding competitive employment. Second, they argued that pre‐vocational training was not effective in developing work skills. Instead of pre‐vocational training, they proposed trying to place clients as quickly as possible in competitive employment positions, where they would receive intensive on‐the‐job support and training from personnel known as 'job coaches' (Anthony 1987).
The individual placement and support (IPS) model is a carefully specified form of supported employment that is based on close adherence to seven key principles (Mueser 2004). These principles are: (a) the goal is competitive employment in work settings integrated into a community's economy; (b) services are based on clients' choices; (c) clients are expected to obtain jobs directly, rather than following lengthy pre‐employment training (rapid job search); (d) attention to patient preference in the job search; (e) integration between employment services and mental health treatment teams; (f) ongoing individual support; and (g) systematic benefits counselling (Bond 2008a).
Adherence to individual placement and support guidelines may be measured using a fidelity scale (Bond 1997b). In IPS, employment specialists serve on clients’ treatment teams alongside other staff, such as case managers and psychiatrists. Each employment specialist provides the full range of vocational services to each client, including engagement in services, identifying job interests and vocational assessment, job finding and job support. IPS uses assertive outreach (Stein 1998) to deliver vocational services in the community rather than at mental health or rehabilitation agencies (Bond 1997b).
How the intervention might work
Supported employment is defined as 'paid work that takes place in normal work settings with provision for ongoing support services' (Becker 1994; Bond 1999). It helps people with mental illness to work by placing them as quickly as possible in competitive employment positions, where they would receive intensive on‐the‐job support and training from personnel known as job coaches (Anthony 1987). Listed below are critical components which are common in successful supported employment programs (Bond 2001).
-
The agency providing the services is committed to help clients with attaining competitive employment.
-
A rapid job search rather than lengthy pre‐employment assessment, training, and counselling is provided for clients.
-
Staff and clients find individualised job placements according to client preferences, strengths, and work experiences.
-
Follow‐along supports are provided indefinitely.
-
The program is closely integrated with the mental health treatment team.
Supported employment is also provided for clients with autism (Keel 1997), mental retardation (Walsh 1994), and traumatic brain injuries (Wehman 2003), though its effectiveness for these populations is yet to be confirmed.
Why it is important to do this review
A previous Cochrane review (Crowther 2001) and another systematic review (Twamley 2003) have examined the effectiveness of various types of vocational rehabilitation for individuals with severe mental illness, including supported employment, but as several new trials of supported employment have been published recently a review focusing purely on supported employment is required. These trials have been covered in two narrative reviews (Bond 2004; Bond 2008a), but there have been no formal meta‐analytic summaries as yet.
Objectives
-
To review the effectiveness of supported employment compared with other approaches to vocational rehabilitation and treatment as usual.
-
Secondary objectives are to establish how far:
-
fidelity to the IPS model affects the effectiveness of supported employment;
-
the effectiveness of supported employment can be augmented by the addition of other interventions.
-
Methods
Criteria for considering studies for this review
Types of studies
Randomised controlled trials (RCTs) that assess the effects of supported employment in people with severe mental illness. We excluded quasi‐randomised studies, such as those allocating by using alternate days of the week.
Types of participants
The supported employment was not designed for a specific diagnostic group nor was it applied in a diagnostic‐specific way in everyday practice. Therefore, for the purpose of this review, the main requirements of participants were that they were similar to those who typically present to the supported employment services. Specific inclusion criteria were that a majority of clients in the trial were (a) of working age (normally 16 to 70 years); (b) unemployed; and (c) suffering from severe mental illness, defined as: schizophrenia and schizophrenia‐like disorders; bipolar disorders; or depression with psychotic features. Trials were included where a majority of participants (more than 50%) were suffering from schizophrenia and schizophrenia‐like disorders; bipolar disorders; or depression with psychotic features. Substance abuse and post traumatic stress disorder were not considered severe mental illness, but trials were eligible if participants had a problem with substance abuse and/or comorbidity of post traumatic stress disorder in addition to severe mental illness. We excluded trials where a majority of participants (more than 50%) were suffering from a learning disability as the sole psychiatric diagnosis.
Types of interventions
Three interventions of interest were defined: supported employment (including Individual Placement and Support (IPS), and Augmented Supported Employment), other vocational approaches and treatment as usual.
1. Supported employment
Supported employment is a technique designed to help mentally ill people obtain and keep competitive employment. Supported employment aims to help clients obtain competitive work as quickly as possible and provides ongoing support to help them keep their employment (Bond 2001; Mueser 2004).
1.1 Individual placement and support (IPS)
IPS is a carefully specified approach to supported employment that requires close adherence to the seven principles described above. Fidelity to the IPS model can be assessed using an IPS Fidelity Scale (Becker 2001). IPS is classified into two categories: (i) Low fidelity IPS and (ii) High fidelity IPS. The seven key principles described in Description of the intervention are taken into consideration to assess the fidelity (Bond 1997b). Low fidelity IPS is defined as a) the programme itself does not satisfy one or more of the seven key principles, for example, if the same personnel are in charge of employment services and clinical services; b) although the programme does satisfy all the seven criteria, the quality assessment reveals that the actual delivery of the programme did not satisfy one or more of the seven key principles; or c) the quality of the actual delivery was not assessed. High fidelity IPS is when the programme satisfies all seven criteria.
Fidelity of IPS was assessed by the following two‐step procedure.
-
Two review authors (YK and KK) independently selected RCTs that assured fidelity of IPS using the IPS scale (Bond 1997b). Trials that did not fulfil this criterion were rated as low fidelity IPS.
-
The same two review authors checked the selected articles. If the detailed description, especially in terms of engagement and intensity, indicated low fidelity of IPS conducted in some of the RCTs, fidelity of such IPS was rated as low in this review. If not, fidelity of the IPS was classified as high. The reason for judgement for the low fidelity IPS is presented in Characteristics of included studies.
1.2 Augmented supported employment
Supported employment can be augmented with other interventions, such as motivational interventions, social skills training and cognitive rehabilitation (Bell 2008; Drake 2008; McGurk 2007; Mueser 2005; Tsang 2007; Wallace 2004).
2. Other vocational approaches
Other vocational approaches are described in detail in another Cochrane review (Crowther 2001) and include sheltered workshop; prevocational training classes; job counselling; and the Clubhouse model ‐ this model provides (a) work experiences through clubhouse work units; (b) transitional employment (the participant works for a limited period in a paid position in a real workplace, but the position is “owned” by the employment agency rather than the participant); and (c) peer support. In this model, the participant graduates from helping to maintain a patient‐led "clubhouse", to transitional employment, and finally to competitive employment; and diversified placement approach ‐ principles of this approach are: (a) goal of paid employment including but not limited to a competitive one; (b) gradualism (members move gradually through the vocational continuum); (c) flexibility in movement between placements; (d) peer support; and (f) partnerships with the business community (Bond 2004). All of these approaches differ from supported employment in that they do not place an emphasis on an immediate search for competitive employment, but prefer a period of preparation, before seeking competitive employment. We planned to treat them as a single control intervention.
3. Treatment as usual
Treatment as usual is defined as standard psychiatric care for participants in the trial, without any specific vocational component. It is assumed that both intervention and control participants will be receiving treatment as usual, which would normally include: medication, medication management, case management, and supportive psychotherapy (Bond 2008c).
Types of outcome measures
We grouped outcomes into short term (less than six months) medium term (six months to one year) and long term (over one year: a follow‐up duration of 12 months was also considered as long term)
Primary outcomes
1. Employment: days in competitive employment (long term)
Secondary outcomes
1. Employment
1.1 Days in competitive employment (medium term)
1.2 Days in any form of paid employment (such as competitive employment, transitional employment, or sheltered employment with wage)
1.3 Earnings in the first year
1.4 Job tenure (weeks/work/person: for competitive employment and any paid employment)
1.5 Time to first competitive employment
2. Education
2.1 Days in any form of employment or education (including training courses or full or part‐time education)
3. Leaving the study early (i.e. number of participants who dropped‐out from service)
3.1 For any reason
3.2 Specific reason (as defined by individual studies)
4. Global state
4.1 Relapse
4.2 Time to relapse
4.3 No clinically important change in global state
4.4 Not any change in global state
5. Mental state
5.1 No clinically important change in general mental state
5.2 Not any change in general mental state
5.3 Average endpoint general mental state score
5.4 Average change in general mental state scores
5.5 No clinically important change in specific symptoms
5.6 Not any change in specific symptoms
5.7 Average endpoint specific symptom score
5.8 Average change in specific symptom scores
6. Service Use
6.1 Mean days in hospital
6.2 Number of participants admitted to hospital/re‐hospitalised
7. Quality of life
7.1 No clinically important change in quality of life
7.2 Not any change in quality of life
7.3 Average endpoint quality of life score
7.4 Average change in quality of life scores
7.5 No clinically important change in specific aspects of quality of life
7.6 Not any change in specific aspects of quality of life
7.7 Average endpoint specific aspects of quality of life
7.8 Average change in specific aspects of quality of life
8. Social/General functioning
8.1 Average endpoint general functioning score (when Global Assessment of Functioning (GAF) was rated in symptoms and disability separately, a lower score was considered as general GAF score, and extracted and integrated in a meta‐analysis)
8.2 Average change in general functioning scores
8.3 No clinically important change in specific aspects of functioning, such as social or life skills
8.4 Not any change in specific aspects of functioning, such as social or life skills
8.5 Average endpoint specific aspects of functioning, such as social or life skills
8.6 Average change in specific aspects of functioning, such as social or life skills
9. Adverse effects
9.1 Not any general adverse effects
9.2 Average endpoint general adverse effect score
9.3 Average change in general adverse effect scores
9.4 No clinically important change in specific adverse effects
9.5 Not any change in specific adverse effects
9.6 Average endpoint specific adverse effects
9.7 Average change in specific adverse effects
9.8 Death ‐ natural and suicide
10. Economic Costs (excluding housing costs)
10.1 Direct costs
10.2 Indirect costs
11. Summary of findings
We used the GRADE approach to interpret findings (Schünemann 2008) and used GRADE profiler (GRADEPRO) to import data from RevMan 5.1 (Review Manager) to create 'Summary of findings' tables. These tables provide outcome‐specific information concerning the overall quality of evidence from each included study in the comparison, the magnitude of effect of the interventions examined, and the sum of available data on all outcomes we rated as important to patient‐care and decision making. We selected the following main outcomes for inclusion in the Summary of findings table:
1. Employment ‐ obtained any job during the study
2. Employment ‐ days in competitive employment (primary outcome) ‐ long term
3. Employment ‐ days in any form of paid employment ‐ long term
4. Employment ‐ job tenure for competitive employment (weeks) ‐ long term
5. Employment ‐ job tenure for any paid employment (weeks) ‐ long term
6. Time (days) to first competitive employment
Search methods for identification of studies
Electronic searches
The Cochrane Schizophrenia Group Trials Register (Feb 2010) was searched using the phrase:
[( *employ* or ((*supp* or *transitional*) and (*employ* or *work*) or ((*psychosocial* or *psycho‐social* or *psychiatric* or *occupational or *soc* or *work* or *job* or *counsel*) and *rehab*) or *sheltered work* or *vocatio* or *fountain house* or *fountain‐house* or *clubhouse* or *club‐house* or *occupat* or *job* or *work therap* or *delivery of health care* or *delivery of integrated delivery* in title, abstract and index fields in REFERENCE) or (*vocat* or work* or *employ* or * job* or *occupat* or * placem* or *rehab*) in STUDY interventions)]
This register is compiled by systematic searches of major databases, handsearches and conference proceedings (see group module).
Searching other resources
1. Reference searching
The sensitivity of the search strategy was examined by comparing the results of the search with the reference lists of the identified reviews and trials to determine how many cited trials had not been detected.
2. Personal contact
We contacted researchers working in the field to identify unpublished studies.
Data collection and analysis
Selection of studies
Two review authors (YK and KK) independently inspected all the citations identified by the search and requested all potentially relevant articles, contacting the trial authors where necessary. Once the full articles had been obtained, two review authors independently decided whether the studies met the inclusion criteria. In the event of a disagreement, a third reviewer adjudicated and made a final decision. If it was not possible to obtain sufficient information to judge whether a study met inclusion criteria, it was placed in the list of studies awaiting assessment until such information became available.
Data extraction and management
1. Extraction
Two review authors (YK and KK) independently extracted data from the selected trials using a double‐entry method. In the event of a difference between the review authors, they sought to resolve the difference by further scrutiny of the original trial reports, and involved a third review author and/or contacted the authors for further information.
2. Management
We extracted data onto standard, simple forms.
3. Scale‐derived data
We included continuous data from rating scales only if: (a) the psychometric properties of the measuring instrument had been described in a peer‐reviewed journal (Marshall 2000); (b) the measuring instrument was not written or modified by one of the trialists; (c) the measuring instrument was either (i) a self‐report or (ii) completed by an independent rater or relative (not the therapist).
Assessment of risk of bias in included studies
Review authors YK, KK and/or MH worked independently by using criteria described in the Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2011) to assess trial quality. This new set of criteria is based on evidence of associations between overestimate of effect and high risk of bias of the article such as sequence generation, allocation concealment, blinding, incomplete outcome data and selective reporting.
Where inadequate details of randomisation and other characteristics of trials were provided, we contacted authors of the studies in order to obtain additional information.
We noted the level of risk of bias in both the text of the review and in the summary of findings Table for the main comparison.
Measures of treatment effect
1. Binary outcomes
Where binary outcomes (proportions) were used, we calculated fixed‐effect risk ratios (RR) (Furukawa 2002), with 95% confidence intervals (CIs) for each outcome. In the event of significant heterogeneity, we used a random‐effects model. The RR was chosen over the odds ratio because the latter tends to overstate effect size when event rates are high (Higgins 2011).
2. Continuous data
2.1 Summary statistic
For continuous outcomes, we estimated a mean difference (MD) between groups. We preferred not to calculate effect size measures (standardised mean difference SMD). However, if scales of very considerable similarity were used, we would have presumed there was a small difference in measurement, and would have calculated effect size and transformed the effect back to the units of one or more of the specific instruments.
2.2 Endpoint versus change data
Since there is no principal statistical reason why endpoint and change data should measure different effects (Higgins 2011), we used scale endpoint data which was easier to interpret from a clinical point of view. If endpoint data were not available, we used change data.
2.3 Skewed data
Continuous data on clinical and social outcomes are often not normally distributed. To avoid the pitfall of applying parametric tests to non‐parametric data, we aimed to apply the following standards to all data before inclusion: (a) standard deviations (SDs) and means are reported in the paper or obtainable from the authors; (b) when a scale starts from the finite number zero, the SD, when multiplied by two, is less than the mean (as otherwise the mean is unlikely to be an appropriate measure of the centre of the distribution, (Altman 1996); (c) if a scale starts from a positive value (such as the Positive and Negative Syndrome Scale (PANSS) which can have values from 30 to 210), the calculation described above is modified to take the scale starting point into account. In these cases skew is present if 2SD > (S‐S min), where S is the mean score and S min is the minimum score. Endpoint scores on scales often have a finite start and end point and these rules can be applied. Skewed endpoint data from studies of less than 200 participants were entered as 'other data' within the data and analysis section rather than into a statistical analysis. Skewed data pose less of a problem when looking at means if the sample size was large (over 200 participants) and they were entered into syntheses. When continuous data are presented on a scale which includes a possibility of negative values (such as change data), it is difficult to tell whether data are skewed or not, skewed change data were entered into statistical analysis.
2.4 Data synthesis
When standard errors instead of SDs were presented, the former were converted to SDs. If SDs were not reported and could not be calculated from available data, authors were asked to supply the data. In the absence of data from authors, the mean SD from other studies was used.
Unit of analysis issues
1. Cluster trials
Studies increasingly employ ‘cluster randomisation’ (such as randomisation by clinician or practice) but analysis and pooling of clustered data poses problems. Firstly, authors often fail to account for intra class correlation in clustered studies, leading to a ‘unit of analysis’ error (Divine 1992) whereby P values are spuriously low, confidence intervals unduly narrow and statistical significance overestimated. This causes type I errors (Bland 1997; Gulliford 1999).
Where clustering had not been accounted for in primary studies, we presented the data in a table, with a (*) symbol to indicate the presence of a probable unit of analysis error. In subsequent versions of this review we will contact first authors of studies to obtain intra class correlation coefficients (ICCs) of their clustered data and adjust for this by using accepted methods (Gulliford 1999). Where clustering had been incorporated into the analysis of primary studies, we also presented these data as if from a non‐cluster randomised study, but adjusted for the clustering effect.
The binary data as presented in a report should be divided by a ‘design effect’ (Raj 2009). This is calculated using the mean number of participants per cluster (m) and the ICC [Design effect = 1+(m‐1)*ICC] (Donner 2002). If the ICC had not been reported it was assumed to be 0.1 (Ukoumunne 1999).
2. Studies with multiple treatment groups
Where a study involved more than two treatment groups, if relevant, the additional treatment groups were presented in additional relevant comparisons. Data were not double counted. Where the additional treatment groups were not relevant, these data were not reproduced.
Dealing with missing data
1. Overall loss of credibility
At some degree of loss of follow‐up, data must lose credibility (Xia 2007). Where more than 40% of data were unaccounted for, we did not reproduce these data or use them within analyses.
2. Binary
In the case where attrition for a binary outcome was between 0% and 40% and outcomes of these people were described, we included these data as reported. Where these data were not clearly described, data were presented on a 'once‐randomised‐always‐analyse' basis, assuming an intention‐to‐treat analysis. Those lost to follow‐up were all assumed to have a negative outcome. For example, for the outcome of employment, those who were lost to follow‐up were all considered to be unemployed. A final sensitivity analysis was undertaken to test how prone the primary outcomes were to change when 'completed' data only were compared to the intention‐to‐treat to treat analysis using the negative assumption.
3. Continuous
In the case where attrition for a continuous outcome was between 0% and 40% and completer‐only data were reported, we reproduced these.
4. Intention‐to‐treat (ITT)
Intention‐to‐treat (ITT) was used when available. We anticipated that in some studies, in order to undertake an ITT analysis, the method of last observation carried forward (LOCF) was employed within the study report. As with all methods of imputation to deal with missing data, LOCF introduces uncertainty about the reliability of the results. Therefore, where LOCF data had been used in the analysis, they were indicated in the review.
Assessment of heterogeneity
1. Clinical heterogeneity
We considered all included studies, hoping to use all studies together. Where clear unforeseen issues were apparent that may have added obvious clinical heterogeneity, we noted these issues, considered them in the analyses and undertook sensitivity analyses for the primary outcome.
2. Statistical
2.1 Visual inspection
We visually inspected graphs to investigate the possibility of statistical heterogeneity.
2.2 Employing the I2 statistic
Heterogeneity between studies was investigated by using the I2 method (Higgins 2003) and the Chi2 'P' value. The former provides an estimate of the percentage of variation in observed results thought unlikely to be due to chance. A value equal to or greater than 50% was taken to indicate heterogeneity and the reason for heterogeneity was explored. If the inconsistency was high and the clear reasons were found, the data were presented separately.
Assessment of reporting biases
Data from all identified and selected trials were entered into a funnel graph (trial effect versus trial size) in an attempt to investigate overt publication bias. The possible existence of small study effects was examined by Egger’s regression method (Egger 1997) as well as by visual inspection of the graph (see Results).
Data synthesis
In the absence of significant heterogeneity, a fixed‐effect model was used. However, if significant heterogeneity was demonstrated, a random‐effects model was used for analysis. Where available, the analyses were based on intention‐to‐treat data from the individual studies. The data from included trials were combined in a meta‐analysis if they were sufficiently homogeneous, both clinically and statistically.
Subgroup analysis and investigation of heterogeneity
1. Pre‐planned subgroup analyses
Subgroup analyses should be performed and interpreted with caution because multiple analyses will lead to false positive conclusions (Oxman 1992). However, we performed the following subgroup analyses, where possible, for the following a priori reasons.
(a) High fidelity IPS versus other vocational approaches.
(b) Augmented supported employment versus other vocational approaches.
2. Regression analyses
If we had included a sufficient number of trials (roughly nine to 11) per independent variable, meta‐regression would have been performed to determine whether various study‐level characteristics affect effect sizes. Possible effect modifiers to be examined in future updates of this review include: study location (USA versus other countries), study location (urban versus rural) and the local unemployment rate. STATA would have been used to perform the meta‐regression (STATA 2005).
Sensitivity analysis
We examined the robustness of our findings by excluding (i) studies with less than 80% follow‐up on the variable at the time point (ii) skewed data (iii) trials with a high risk of bias or where the overall risk of bias was unclear, and (iv) studies where IPS was augmented with other interventions.
Results
Description of studies
See:Characteristics of included studies; Characteristics of excluded studies; Characteristics of ongoing studies.
Results of the search
Electronic searches identified 500 references with 9 additional records identified through other sources. After duplicates were removed, we screened 509 records. 107 potentially relevant records were obtained and scrutinised and 59 of these reports did not meet the inclusion criteria (see Characteristics of excluded studies) and had to be excluded. Fourteen trials are included (Figure 1).
Study flow diagram.
Included studies
Two review authors (YK and KK) checked the strict eligibility of the identified studies. They agreed on 98 articles (91.6 %) but disagreed on eight articles. The latter were discussed with a third review author (TAF) and according to the consensus among the authors remaining discrepancies were resolved, One additional study (Bayer 2008) was written in German and judged not eligible by one review author (TAF) who can read and understand German. Forty‐seven articles describing 20 studies were finally judged eligible. Of these, five trials (eight articles) (Bejerholm 2009 (SE142);McFarlane 2000; McFarlane 2008 (SE147); Michon 2010; Nuechterlein 2008) are either still underway; are being written‐up; in process of publication; or required further details and therefore cannot be included in the present version of the review.
1. Study size
A total of 2265 participants were included from 14 trials. The median sample size per arm was 70 participants (range 20‐156).
2. Study design
Twelve studies were two‐arm studies, the remaining (Mueser 2004; Tsang 2009) were multi‐arm studies.
3. Participants
All 14 studies recruited outpatients. Thirteen RCTs were conducted in the setting of community psychiatric/mental health service. Participants were adults with severe mental illness, with schizophrenia or schizoaffective disorder well represented. Eleven RCTs used DSM‐IV (Diagnostic and Statistical Manual) or ICD 10 (International Classification of Diseases) criteria and three studies adopted DSM‐III‐R criteria. Two studies used other diagnostic criteria (Indiana Department of Mental Health Criteria, OPCRIT, and the Federal Center for Mental Health Services' criteria) (Burns 2007; Gold 2006). No description was given about diagnostic criteria in one RCT (Wong 2008). Men were well represented. There were insufficient data to assess representation of people from ethnic minorities.
4. Interventions and comparators
All included studies compared supported employment with other vocational approaches. Among these, 13 studies (Bond 2007; Burns 2007; Drake 1996; Drake 1999; Gold 2006; Howard 2010; Killackey 2008; Latimer 2006; Lehman 2002; Mueser 2004; Tsang 2009; Twamley 2008; Wong 2008) implemented individual placement and support (IPS) as supported employment and one RCT (Macias 2006) adopted supported employment, which was not defined as IPS.
Tsang 2009 consisted of integrated supported employment (ISE), IPS and traditional vocational rehabilitation, and Mueser 2004 involved IPS, standard services including supported employment, and psychosocial rehabilitation program. Only one study (Tsang 2009) implemented augmented supported employment. Detailed information of interventions adopted in each site from the Employment Intervention Demonstration Program (EIDP) (Gold 2006; Lehman 2002; Macias 2006; Mueser 2004; Blankertz 1997; Cook 2005; McFarlane 2002 (SE175); Toprac 2002) were also obtained via the Internet (EIDP website).
In terms of fidelity check of implementation of supported employment, 12 studies used the IPS fidelity scale (Bond 1997b) and two adopted other procedures defined in each study (Drake 1996; Macias 2006).
5. Outcomes
Of the 14 included studies, 13 reported vocational and/or non‐vocational data (either as dichotomous or as continuous outcomes) that could be entered into a meta‐analysis. All included studies reported attrition due to any reason; 13 studies provided data for this outcome.
5.1 Primary outcomes and secondary outcomes
Seven out of the 14 included studies reported data for our primary outcome of interest of days in competitive employment. At least one study provided data on each of the remaining secondary outcomes, except for time spent in education, and relapse.
5.2 Outcome scales
5.2.1 Mental state
5.2.1.1 Positive and Negative Syndrome Scale ‐ PANSS (Kay 1987)
This is a 30‐item scale, each of which can be defined on a seven‐point scoring system from absent to extreme. It has three subscales for measuring the severity of general psychopathology, positive symptoms (PANSS‐P), and negative symptoms (PANSS‐N). A low score indicates lesser severity. Two studies (Bond 2007; Burns 2007) reported data from this scale.
5.2.1.2 Brief Psychiatric Rating Score ‐ BPRS (Overall 1962)
This scale is used to assess the severity of abnormal mental state. A revised 18‐item scale is commonly used, though the original scale has 16 items. Each item is defined on a seven‐point scale varying from 'not present' to 'extremely severe', scoring from 0‐6 or 1‐7. Scores can range from 0‐126, with high scores indicating more severe symptoms. Only one study reported this outcome (Drake 1999).
5.2.1.3 Hospital Anxiety and Depression Scale ‐ HADS (Zigmond 1983).
This scale is a self‐rating instrument for anxiety and depression in patients with both somatic and mental problems. The scale consists of 14 items on a four‐point Likert scale (range 0–3). The total score is the sum of the 14 items, and for each subscale the score is the sum of the respective seven items. One study (Burns 2007) reported data from this scale.
5.2.2 Quality of Life
5.2.2.1 Quality of Life Interview ‐ QOLI (Lehman 1982)
The scale is a self‐rating instrument in which participants respond on a Likert scale for all items. Two different Likert scales were used; the first Likert scale ranged from one, “terrible” to 10 “delighted.”and the second ranged from one, “not at all” to five, “at least once a day.”The QOLI assesses objective and subjective quality of life indicators and includes such areas as leisure activities, social relationships, living situations, health, employment and vocational services, and finances. Scoring was performed by adding up scores on all items to obtain a total quality of life score. The original version consists of 143 items and the abbreviated version has thirteen sections consisting of 35 total items. Three studies (Bond 2007, Drake 1999, and Twamley 2008) reported data from this scale.
5.2.2.2 Lancashire Quality of Life Profile European version ‐ LQoLP‐EU (Gaite 2000)
This scale was originally developed from the QOLI. It is a structured interview comprising 105 items. It includes nine domains: work and education (seven items); leisure and participation (eight items); religion (four items); finances (seven items); living situation (12 items); legal status and safety (five items); family relations (seven items); social relations (six items) ; and health (10 items).The interview can also assess positive and negative affect; self‐esteem; global well‐being; quality of life of the patient independent of the patient's own opinion. Two studies (Burns 2007 and Howard 2010) reported data from this scale.
5.2.3 General functioning score
5.2.3.1 Global Assessment Scale ‐ GAS or Global Assessment of Functioning ‐ GAF (Endicott 1976)
This scale is a clinician‐rated scale of overall functioning on a scale of 1‐100. Lower scores indicate poorer functioning. Three studies reported data from this scale (Burns 2007; Drake 1999; Howard 2010).
Excluded studies
Of the 107 references retrieved for more detailed evaluation, 58 articles did not meet our inclusion criteria and were excluded. Reasons for this are presented in the 'Characteristics of excluded studies' table.
Awaiting classification
Two studies are considered as awaiting classification (McFarlane 2000; Michon 2010).
Ongoing Studies
Three studies are ongoing (Bejerholm 2009 (SE142); McFarlane 2008 (SE147); Nuechterlein 2008).
Risk of bias in included studies
See: Included studies, Figure 2, Figure 3.
'Risk of bias' graph: review authors' judgements about each risk of bias item presented as percentages across all included studies.
'Risk of bias' summary: review authors' judgements about each risk of bias item for each included study.
Our judgment about the overall risk of bias in the individual studies is illustrated in Figure 2 and Figure 3. Overall, the methodological quality of the included studies was moderate, with a high risk of bias for blinding across all included studies due to the open nature of the intervention. This type of reporting has been associated with an overestimate of the estimate of effect (Schulz 1995) and this should be considered when interpreting the results.
Allocation
Nine studies reported the methods of generating random sequence, in which "a computer originated schedule" was used, while enough description was not given and risk of bias was unclear with regard to sequence generation in six trials. In terms of allocation concealment, seven studies reported enough details, but risk of bias was unclear in the eight remaining studies. Therefore, we were not assured that bias was minimised during the allocation procedure in the other studies, yet the great majority of them reported that the participants allocated to each treatment group were "similar", "the same", "not significantly different", "comparable" or "matched".
Blinding
It is in the nature of studies for psychosocial interventions, that it is not possible for participants or those administering the intervention to be masked to the participants' allocation status. Therefore, none of the studies were double‐blinded. The reported results may exaggerate estimates of treatment effect (Boutron 2004). Nevertheless, three studies reported that the rater, assessor or interviewer was blind to the assignment (Howard 2010; Tsang 2009; Twamley 2008).
Incomplete outcome data
Total attrition rate was moderate and ranged from 2% (Drake 1999) to 32% (Lehman 2002). In six studies, the total dropout rates were more than 20% (Bond 2007; Gold 2006; Lehman 2002; Macias 2006; Tsang 2009; Twamley 2008).
Selective reporting
All included studies were rated as high quality in reporting outcome with a low risk of reporting bias. In terms of the Employment Intervention Demonstration Program (EIDP), data from one of the eight RCTs which seemed eligible (Arizona site) were not available due to lack of publication (EIDP website). This might be a source of publication bias.
Other potential sources of bias
Relatively small sample size in many of the studies might have reduced the opportunity to have comparable groups thus threatening internal validity, and might also have resulted in under‐powered studies, thus increasing chances of false positive outcomes. Moreover, the comparatively short duration of follow‐up might be insufficient to answer or address the critical question about durability of effects. In one study (Gold 2006), project redesign and deviation from a pre‐specified random assignment process may have compromised study validity.
Effects of interventions
COMPARISON 1: Supported employment versus other vocational approaches
1.1 Employment (continuous outcomes)
Eleven studies (Bond 2007; Drake 1996; Drake 1999; Gold 2006; Killackey 2008; Latimer 2006; Macias 2006; Mueser 2004; Tsang 2009; Twamley 2008; Wong 2008) reported competitive employment outcomes. All data for this outcome were skewed; however, as per our protocol, where studies had n = 200 participants or more, these were included in data synthesis. Data from the remaining studies were presented as 'Other data' in the Data and analyses section. Data demonstrate high levels of heterogeneity are presented using a random effects model.
1.1.1 days in competitive employment (primary outcome) ‐ long term
For days in competitive employment, we found only one relevant trial (n = 204) (Mueser 2004) which demonstrated statistically significant favour for supported employment over other vocational approaches (MD 70.63 CI 43.22 to 98.04, Analysis 1.1).
1.1.2 days in any form of paid employment ‐ long term
In this subgroup we found two relevant trials (n = 510). There was statistically significant favour for supported employment over other vocational approaches (MD 84.94 CI 51.99 to 117.89, Analysis 1.1); however, with moderate levels of heterogeneity (Chi2=1.85; df=1; P=0.173; I2=46%).
1.1.3 job tenure for competitive employment (weeks) ‐ long term
In this subgroup we only found one relevant trial (n = 204) (Mueser 2004). There was statistically significant favour for supported employment over other vocational approaches (MD 9.86 CI 5.36 to 14.36, Analysis 1.1).
1.1.4 job tenure for any paid employment (weeks) ‐ long term
Data from two studies showed no significant difference between groups (n = 423, Analysis 1.1), with substantial levels of heterogeneity (Chi2=3.73; df=1; P=0.054; I2=73%).
1.2 Employment (skewed data)
Skewed data from studies of less than n = 200 were entered as 'Other data' in the Data and analyses section; all data need interpreting with caution. Data from the majority of studies that reported employment outcomes of days in in employment and job tenure are heavily skewed, and are best inspected by viewing Analysis 1.2.
1.2.1 Days in competitive employment
Data at short and medium term suggest a trend favouring supported employment over other vocational approaches for days spent in competitive employment; however these data need interpreting in light of the considerable skew present (Analysis 1.2).
1.2.2 Days in any form of paid employment
There was indication that there were more days spent in any form of paid employment for supported employment at both medium and long term (Analysis 1.2); again, data are skewed and need interpreting with caution.
1.2.3 Job tenure for competitive employment (weeks)
Taking into account the considerable skewed data, at medium term, results indicated more weeks of job tenure for other vocational approaches, and the opposite at long term; again, data are skewed and need interpreting with caution (Analysis 1.2).
1.2.4 Job tenure for any paid employment (weeks)
Data are considerably skewed and are best inspected by viewing the 'Other data' table (Analysis 1.2).
1.2.5 Earnings in the first year
Greater earnings in the first year were associated with participants receiving supported employment; again, data are skewed and need interpreting with caution (Analysis 1.2).
1.3 Employment (dichotomous outcomes)
1.3.1 obtained competitive job during the study (high=better)
Seven studies reported data for this outcome (n = 951); there was a statistically significant difference between supported employment and other vocational approaches (RR 3.24 CI 2.17 to 4.82, Analysis 1.2). However, data demonstrated substantial levels of heterogeneity and are analysed using a random effects model (Chi2=22.87; df=6; P=0.0008; I2=74%).
1.4 Time (days) to first competitive employment
1.4.1 long term
Again, all data for this outcome were skewed; however, as per our protocol, where studies had n = 200 participants or more, these were included in the data synthesis. Data from the remaining studies were presented using 'Other data' tables in the Data and analyses section. There was evidence that supported employment was associated with less days to first competitive employment than other vocational approaches at long term (1 RCT, n = 204, MD ‐161.60, 95% CI ‐225.73 to ‐97.47, Analysis 1.4).
1.5 Time (days) to first competitive employment (skewed data)
Skewed data from studies of less than n = 200 were entered into 'Other data' tables in the Data and analyses section; all data need interpreting with caution (Analysis 1.5).
1.6 Leaving the study early for any reason
There was no significant difference between groups for leaving the study early in the short term (1 RCT, n = 92) or medium term (2 RCTs, n = 191). By long term, there was a statistically significant difference in favour of supported employment over other vocational approaches (RR 0.66 CI 0.52 to 0.84, Analysis 1.6) with moderate levels of heterogeneity (Chi2=13.62; df=9; P=0.137; I2=34%).
1.7 Mental state: Average endpoint specific symptom score (high = worse) ‐ long term
There was no evidence that supported employment was associated with a lower or higher endpoint specific symptom score than other vocational approaches when using the BPRS, PANSS or HADS scales (Analysis 1.7).
1.8 Service use: 1. Mean days in hospital (skew)
Data are considerably skewed and are best inspected by viewing the 'Other data' table (Analysis 1.8).
1.9 Service use: 2. Number of participants admitted to hospital
In the long term, there was no evidence that supported employment was associated with a lower or higher rate of participants admitted to hospital/re‐hospitalised than other vocational approaches at long term (2 RCTs, n = 455, RR 0.71, 95% CI 0.53 to 0.96, Analysis 1.9).
1.10 Quality of Life: Average endpoint QOL‐QOLI ‐ various subscales (high = better)
There was no evidence that supported employment was associated with a lower or higher average endpoint quality of life score than other vocational approaches across the measured quality of life domains (Analysis 1.10)
1.11 Global/Social functioning: Average endpoint general functioning score ‐ GAS (high = better)
There was no evidence that supported employment was associated with a lower or higher average endpoint general functioning score than other vocational approaches at long term (3 RCTs, n = 623, Analysis 1.11).
1.12 Adverse effects: Death ‐ natural and suicide
There was no evidence that supported employment was associated with a lower or higher risk of death than other vocational approaches at long term (1 RCT, n = 312, Analysis 1.12).
1.13 Economic Costs (excluding housing costs): Direct costs (GPB £, skewed)
There was no evidence that supported employment was associated with lower or higher economic costs than other vocational approaches; data are considerably skewed and need interpreting with cation (Analysis 1.13).
2. Comparison 2. subgroup analysis: High fidelity IPS versus other vocational approaches
We included RCTs in which high fidelity of IPS was assured using the IPS scale (Bond 1997b) in this subgroup analysis. Though Howard 2010 fulfilled these criteria, the study was excluded from the analysis because the detailed description in the article indicated the low fidelity of IPS conducted in the RCT (See: Characteristics of included studies). Where data were considerably skewed in studies of less than n = 200, we excluded these data.
2.1 Employment
2.1.1 Days in competitive employment (primary outcome)
There was evidence that high fidelity IPS was associated with more days in competitive employment than other vocational approaches in the long term (1 RCT, n = 306, MD 99.80, 95% CI 69.50 to 130.10, Analysis 2.1).
2.1.2 Job tenure for any paid employment
There was no evidence that high fidelity IPS was associated with more or less longer or shorter job tenure for any paid employment than other vocational approaches in the long term (1 RCT, n = 225, Analysis 2.1).
2.2 Leaving the study early for any reason
There was no evidence that high fidelity IPS was associated with lower or higher rate of participants leaving the study early for any reason than other vocational approaches at short or medium term; however, significantly more people left the study early when receiving other vocational approaches, with moderate heterogeneity present overall (P = 0.02; I2 = 49%, Analysis 2.2).
2.3 Mental state: average endpoint specific symptom score
There was no evidence that high fidelity IPS was associated with a lower or higher endpoint specific symptom score than other vocational approaches when using either the BPRS, PANSS or HAD scales (Analysis 2.3).
2.4 Service Use
2.4.1 Number of participants admitted to hospital/re‐hospitalised
There was no evidence that high fidelity IPS was associated with a lower or higher rate of participants admitted to hospital/re‐hospitalised than other vocational approaches (Analysis 2.4).
2.5 Quality of life: Average endpoint quality of life scores
2.5.1 Average endpoint QoL‐QoLI ‐ various subscales
There was no evidence that high fidelity IPS was associated with a lower or higher average endpoint quality of life score than other vocational approaches (Analysis 2.5)
2.6 Global/Social functioning: Average endpoint general functioning score
There was no evidence that high fidelity IPS was associated with a lower or higher average endpoint general functioning score than other vocational approaches in the long term (Analysis 2.6).
2.7 Adverse effects: Death ‐ natural and suicide
There was no evidence that high fidelity IPS was associated with a lower or higher risk of death than other vocational approaches (Analysis 2.7).
3. COMPARISON 3: subgroup analysis: Augmented supported employment versus other vocational approaches
This subgroup analysis was not conducted due to the reason described in Summary of main results.
4. Funnel Plot Analysis
As stated in the protocol, analyses were carried out as head‐to head comparisons. Where available, the funnel plot analyses did not suggest evidence of publication bias (Figure 4, Figure 5, Figure 6), however, for many comparisons the presence of publication bias was not examined because there were insufficient trials to allow meaningful formal assessment using funnel plots.
Funnel plot of comparison: 1 Supported employment versus other vocational approaches, outcome: 1.1 Employment.
Funnel plot of comparison: 1 Supported employment versus other vocational approaches, outcome: 1.3 Employment.
Funnel plot of comparison: 1 Supported employment versus other vocational approaches, outcome: 1.6 Leaving the study early for any reason.
5. Regression Analysis
We did not conduct regression analysis because we did not have a sufficient number of trials for most variables, including the primary outcome.
6. Sensitivity Analysis
6.1 Excluding trials with less than 80% follow‐up on the variable at the time point
Results from this sensitivity analysis did not materially change the main findings.
6.2 Excluding skewed data
No data were available for four items with regard to average endpoint specific symptom score (positive and negative symptoms of PANSS, and anxiety and depression of HADS) in this sensitivity analysis. This sensitivity analysis was not performed for the outcomes specified in the protocol 1.1 to 1.5 (days in competitive employment, days in any form of paid employment, earnings in the first year, job tenure, time to first competitive employment) and 10.1 (direct costs) (see Measures of treatment effect), because all included continuous data were skewed in terms of these outcomes. Only studies with more than n = 200 were included in data synthesis, with the remaining studies reported in separate data tables, making it difficult to draw any meaningful conclusions from the data.
6.3 Excluding trials with a high risk of bias or where the overall risk of bias was unclear.
This sensitivity analysis was not performed, because all included trials were with at least one risk of bias.
6.4 Excluding trials where IPS was augmented with other interventions.
Results from this sensitivity analysis did not materially change the main findings.
Discussion
Summary of main results
A total of 14 randomised controlled trials (2265 participants) were included in this review.
COMPARISON 1: Supported employment versus other vocational approaches
In terms of primary outcome, employment: days in competitive employment (long term, i.e. over one year of follow‐up), supported employment seems to increase the length of competitive employment when compared with other vocational approaches. However, the vast majority of the data were considerably skewed, making it impossible to draw any meaningful conclusions from these data. However, binary employment outcomes of obtaining competitive employment during the study clearly and significantly favoured supported employment over other vocational approaches.
Supported employment also showed advantage in other secondary outcomes. It seems to increase the length of any form of paid employment and job tenure for competitive employment, and decrease the time to first competitive employment. In terms of job tenure for any paid employment, no statistically significant difference was observed between supported employment and other vocational approaches.
With regard to earnings in the first year, endpoint global state, endpoint specific symptom scores, days in hospital, rate of hospitalisation, endpoint quality of life scores, endpoint global functioning score, death and direct costs, no significant difference was observed between supported employment and other vocational approaches.
COMPARISON 2: subgroup analysis: High fidelity Individual Placement and Support IPS) versus other vocational approaches
Results from this subgroup analysis did not materially change the main findings in COMPARISON 1 except that no data were available for direct costs.
Few data were obtained indicating that fidelity to the IPS model affects the effectiveness of supported employment.
One of the 14 included studies, Tsang 2009 adopted augmented supported employment as an experimental intervention and could contribute to a subgroup analysis including only augmented supported employment (COMPARISON 3: subgroup analysis: Augmented supported employment versus other vocational approaches). However, this subgroup analysis did not prove useful for exploring if the effectiveness of supported employment can be augmented by the addition of other interventions, because the study did not provide data about days in competitive employment (long term) and could not contribute to the primary outcome.
Sensitivity analysis
Results from a sensitivity analysis excluding studies with less than 80% follow‐up on the variable at the time point did not materially change the main findings in COMPARISON 1, except that no statistically significant difference was observed in job tenure for competitive employment between supported employment and other vocational approaches.
Overall completeness and applicability of evidence
1. Completeness
1.1 Duration of follow‐up
Though the majority of studies presented long‐term data, i.e. over one year of follow‐up, this might still not be enough to answer or address the critical question about durability of effects.
1.2 Coverage of outcomes
Of the 14 identified studies, 10 (71%) at maximum were able to contribute to any vocational or non‐vocational outcome that we had specified in the protocol. For example, eight studies contributed to "Days in any form of paid employment" and seven studies to "Job tenure for any paid employment by weeks," whereas only one study provided data in format we were able to pool for our primary outcome "Days in competitive employment" or five studies for "Average endpoint QOL‐life in general". No studies reported data on relapse except average endpoint global state score, and only one or two studies provided data for general functioning or costs. Adverse effects were not reported, except death.
2. Applicability
2.1 Origin
A percentage of 52.9% of the total number of participants included in the review were from USA, while 29.4% were from Europe.The sample also included people from Canada, Australia and China. Given that a large part of the total sample came from North America and Europe, the present review findings are still lacking applicability to developing countries and, more generally, to countries where mental health systems are not community‐based.
2.2 People
Variability of participants recruited for trials is likely to reflect the heterogeneity of patients a clinician faces in daily practice when treating people with severe mental illness. This variability was in regard to diagnosis (where participants were affected by a wide diagnostic group including schizophrenic, affective and personality disorder). On average, studies included people with a long history of illness; only Killackey 2008 included participants with a first episode of psychosis. This fits with the concept of severe mental illness, where this label includes certain criteria relating to length of illness.
2.3 Interventions
All the included studies compared supported employment with other vocational approaches and there was no study comparing supported employment with treatment as usual, as defined in the protocol. This would not violate the applicability of the results, because most of current psychiatric services provide at least one type of vocational approach (supported employment or pre‐vocational rehabilitations).
Quality of the evidence
The biggest caveat of the current data‐set is that only seven out of 14 (50%) of the identified studies contributed to our primary outcome. For secondary outcomes, eight studies (57%) contributed to days in any form of paid employment but for the others typically between two and five studies only were contributory. This represents a substantial risk of outcome reporting bias (Furukawa 2007). In terms of the quality of individual studies, there also appears to exist some overall risk of bias, as illustrated in Figure 2. This would mean, therefore, a moderate risk of overestimation of positive effects in the current systematic review. Future RCTs in this area should attempt to report all clinically important outcomes, preferably in a standardised format, and also to improve quality of study methodology and its reporting.
The following pre‐planned subgroup analyses did not fully answer the question explained in the secondary objectives.
(a) High fidelity IPS versus other vocational approaches.
(b) Augmented supported employment versus other vocational approaches.
Future reviews should include comparisons listed below.
(c) High fidelity IPS versus low fidelity IPS.
(d) Augmented supported employment versus supported employment without augmentation.
Potential biases in the review process
The present review is not exempt from some potential biases. We have worked mainly with published reports, and only in few cases with unpublished material. Only a minority of the identified trials contributed to any of the primary or secondary outcomes. This may have lead to a reporting and publishing bias. In several cases, our original protocol was not specific enough and a need for subsequent clarification arose and post hoc decisions had to be taken (see Differences between protocol and review). In some cases this could have affected the review results. Nevertheless, much information would have been lost if we chose to exclude trials where diagnoses of a part of the participants did not fulfil the definition of severe mental illness described in the original protocol.
Agreements and disagreements with other studies or reviews
A previous Cochrane review (Crowther 2001) and another systematic review (Twamley 2003) have examined effectiveness of various types of vocational rehabilitation for individuals with severe mental illness, including supported employment. In addition, there are two narrative reviews (Bond 2004; Bond 2008a). Their results are in line with this review. All the previous reviews indicated that supported employment could improve vocational and non‐vocational conditions in this population.
'Risk of bias' graph: review authors' judgements about each risk of bias item presented as percentages across all included studies.
'Risk of bias' summary: review authors' judgements about each risk of bias item for each included study.
Funnel plot of comparison: 1 Supported employment versus other vocational approaches, outcome: 1.1 Employment.
Funnel plot of comparison: 1 Supported employment versus other vocational approaches, outcome: 1.3 Employment.
Funnel plot of comparison: 1 Supported employment versus other vocational approaches, outcome: 1.6 Leaving the study early for any reason.
Comparison 1 Supported employment versus other vocational approaches, Outcome 1 Employment.
| Study | Intervention | Mean | SD | N |
| days in competitive employment (primary outcome) ‐ medium term | ||||
| Latimer 2006 | Supported employment | 34.5 | 63.5 | 75 |
| Latimer 2006 | Other vocational approach | 14.5 | 42.5 | 74 |
| Twamley 2008 | Supported employment | 55.89 | 68.62 | 28 |
| Twamley 2008 | Other vocational approach | 16.59 | 36.69 | 22 |
| days in competitive employment (primary outcome) ‐ long term | ||||
| Bond 2007 | Supported employment | 161.5 | 197.5 | 96 |
| Bond 2007 | Other vocational approach | 81.65 | 206.2 | 98 |
| Drake 1996 | Supported employment | 75.88 | 105.32 | 74 |
| Drake 1996 | Other vocational approach | 25.64 | 50.01 | 69 |
| Drake 1999 | Supported employment | 75.5 | 109.0 | 76 |
| Drake 1999 | Other vocational approach | 6.0 | 26.2 | 76 |
| Gold 2006 | Supported employment | 113.0 | 142.5 | 66 |
| Gold 2006 | Other vocational approach | 36.0 | 80.0 | 77 |
| days in any form of paid employment ‐ medium term | ||||
| Killackey 2008 | Supported employment | 43.15 | 46.1 | 20 |
| Killackey 2008 | Other vocational approach | 19.0 | 50.35 | 21 |
| Latimer 2006 | Supported employment | 85.0 | 99.5 | 75 |
| Latimer 2006 | Other vocational approach | 70.5 | 93.5 | 74 |
| days in any form of paid employment ‐ long term | ||||
| Bond 2007 | Supported employment | 173.1 | 192.8 | 96 |
| Bond 2007 | Other vocational approach | 219.4 | 237.3 | 98 |
| Gold 2006 | Supported employment | 145.0 | 157.5 | 66 |
| Gold 2006 | Other vocational approach | 116.0 | 134.5 | 77 |
| Wong 2008 | Supported employment | 136.0 | 161.0 | 46 |
| Wong 2008 | Other vocational approach | 74.0 | 141.0 | 46 |
| job tenure for competitive employment (weeks) ‐ medium term | ||||
| Latimer 2006 | Supported employment | 8.9 | 12.2 | 75 |
| Latimer 2006 | Other vocational approach | 11.1 | 13.5 | 74 |
| job tenure for competitive employment (weeks) ‐ long term | ||||
| Bond 2007 | Supported employment | 22.03 | 29.22 | 96 |
| Bond 2007 | Other vocational approach | 8.61 | 20.44 | 98 |
| Gold 2006 | Supported employment | 21.5 | 24.1 | 66 |
| Gold 2006 | Other vocational approach | 17.3 | 14.3 | 77 |
| Wong 2008 | Supported employment | 26.6 | 32.0 | 46 |
| Wong 2008 | Other vocational approach | 15.0 | 28.6 | 46 |
| job tenure for any paid employment (weeks) ‐ medium term | ||||
| Latimer 2006 | Supported employment | 15.8 | 18.0 | 75 |
| Latimer 2006 | Other vocational approach | 18.0 | 16.9 | 74 |
| job tenure for any paid employment (weeks) ‐ long term | ||||
| Gold 2006 | Supported employment | 22.0 | 23.5 | 66 |
| Gold 2006 | Other vocational approach | 20.6 | 17.8 | 77 |
| Macias 2006 | Supported employment | 21.8 | 26.3 | 88 |
| Macias 2006 | Other vocational approach | 13.1 | 15.9 | 89 |
| Tsang 2009 | Supported employment | 17.88 | 17.62 | 108 |
| Tsang 2009 | Other vocational approach | 1.05 | 4.5 | 55 |
| Earnings from paid employment ‐ CAN ($) ‐ long term | ||||
| Latimer 2006 | Supported employment | 1,296 | 2,575 | not stated |
| Latimer 2006 | Other vocational approach | 1,022 | 2,434 | not stated |
| Earnings from paid employment ‐ USD ($) per hour ‐ long term | ||||
| Lehman 2002 | Supported employment | 5.07 | 1.06 | not stated |
| Lehman 2002 | Other vocational approach | 5.31 | 3.70 | not stated |
| Earnings from paid employment ‐ USD ($) ‐ long term | ||||
| Bond 2007 | Supported employment | 5,199 | 8,473 | 1,494 |
| Bond 2007 | Other vocational approach | 5,244 | 7,579 | 2,797 |
| Drake 1996 | Supported employment | 3,394 | 5,446 | not stated |
| Drake 1996 | Other vocational approach | 1,077 | 2,237 | not stated |
| Drake 1999 | Supported employment | 1,875 | 392 | not stated |
| Drake 1999 | Other vocational approach | 153 | 83 | not stated |
| Macias 2006 | Supported employment (Clubhouse) | 2,887 | 5,058 | not stated |
| Macias 2006 | Other vocational approach (ACT) | 2,507 | 4,325 | not stated |
| Wong 2008 | Supported employment | 2,300 | 3,100 | not stated |
| Wong 2008 | Other vocational approach | 1,100 | 2,200 | not stated |
| Earnings from paid employment ‐ AUS ($) ‐ medium term | ||||
| Killackey 2008 | Supported employment | 4449 | 5067 | 2432 |
| Killackey 2008 | Treatment as usual | 3615 | 1,2473 | 0 |
Comparison 1 Supported employment versus other vocational approaches, Outcome 2 Employment (skewed).
Comparison 1 Supported employment versus other vocational approaches, Outcome 3 Employment.
Comparison 1 Supported employment versus other vocational approaches, Outcome 4 Time (days) to first competitive employment.
| Study | Intervention | Mean | SD | N |
| medium term | ||||
| Latimer 2006 | Supported employment | 126.3 | 95.6 | 75 |
| Latimer 2006 | Other vocational approach | 152.9 | 123.3 | 75 |
| Twamley 2008 | Supported employment | 92.75 | 90.71 | 28 |
| Twamley 2008 | Other vocational approach | 171.17 | 102.4 | 22 |
| long term | ||||
| Bond 2007 | Supported employment | 156.41 | 122.26 | 96 |
| Bond 2007 | Other vocational approach | 193.44 | 156.5 | 98 |
| Drake 1999 | Supported employment | 125.6 | 142.8 | 76 |
| Drake 1999 | Other vocational approach | 293.4 | 636.4 | 76 |
| Gold 2006 | Supported employment | 201.6 | 189.0 | 66 |
| Gold 2006 | Other vocational approach | 344.4 | 175.7 | 77 |
| Lehman 2002 | Supported employment | 164.36 | 156.51 | 47 |
| Lehman 2002 | Other vocational approach | 287 | 235.24 | 12 |
| Wong 2008 | Supported employment | 72.0 | 77.0 | 46 |
| Wong 2008 | Other vocational approach | 118.0 | 143.0 | 46 |
Comparison 1 Supported employment versus other vocational approaches, Outcome 5 Time (days) to first competitive employment (skewed).
Comparison 1 Supported employment versus other vocational approaches, Outcome 6 Leaving the study early for any reason.
Comparison 1 Supported employment versus other vocational approaches, Outcome 7 Mental state: Average endpoint specific symptom score (high = worse) ‐ long term.
| Study | Intervention | Mean | SD | N |
| long term | ||||
| Bond 2007 | Supported employment | 4.86 | 12.71 | 96 |
| Bond 2007 | Other vocational approach | 8.32 | 22.6 | 98 |
| Drake 1999 | Supported employment | 10.0 | 20.84 | 76 |
| Drake 1999 | Other vocational approach | 7.7 | 15.6 | 76 |
Comparison 1 Supported employment versus other vocational approaches, Outcome 8 Service use: 1. Mean days in hospital (skewed).
Comparison 1 Supported employment versus other vocational approaches, Outcome 9 Service use: 2. Number of participants admitted to hospital.
Comparison 1 Supported employment versus other vocational approaches, Outcome 10 Quality of Life: Average endpoint QOL‐QOLI ‐ various subscales (high = better).
Comparison 1 Supported employment versus other vocational approaches, Outcome 11 Global/Social functioning: Average endpoint general functioning score ‐ GAS (high = better).
Comparison 1 Supported employment versus other vocational approaches, Outcome 12 Adverse effects: Death ‐ natural and suicide.
| Study | Intervention | Mean | SD | N |
| long term | ||||
| Howard 2010 | Supported employment | 9571.0 | 11217.0 | 109 |
| Howard 2010 | Other vocational approach | 11932.0 | 13694.0 | 110 |
Comparison 1 Supported employment versus other vocational approaches, Outcome 13 Economic costs: 1. Direct costs (British £, skewed).
Comparison 2 Sub‐group analyses: High fidelity IPS vs other vocational approaches, Outcome 1 Employment.
Comparison 2 Sub‐group analyses: High fidelity IPS vs other vocational approaches, Outcome 2 Leaving the study early for any reason.
Comparison 2 Sub‐group analyses: High fidelity IPS vs other vocational approaches, Outcome 3 Mental state: Average endpoint specific symptom score (high = worse) ‐ long term.
Comparison 2 Sub‐group analyses: High fidelity IPS vs other vocational approaches, Outcome 4 Service use: Number of participants admitted to hospital.
Comparison 2 Sub‐group analyses: High fidelity IPS vs other vocational approaches, Outcome 5 Quality of Life: Average endpoint QOL‐QOLI ‐ various subscales (high = better).
Comparison 2 Sub‐group analyses: High fidelity IPS vs other vocational approaches, Outcome 6 Global/Social functioning: Average endpoint general functioning score ‐ GAS (high = better).
Comparison 2 Sub‐group analyses: High fidelity IPS vs other vocational approaches, Outcome 7 Adverse effects: Death ‐ natural and suicide.
Comparison 3 Sensitivity analysis: Excluding trials with less than 80% follow‐up on the variable at the time point, Outcome 1 Employment.
Comparison 3 Sensitivity analysis: Excluding trials with less than 80% follow‐up on the variable at the time point, Outcome 2 Time (days) to first competitive employment.
Comparison 3 Sensitivity analysis: Excluding trials with less than 80% follow‐up on the variable at the time point, Outcome 3 Leaving the study early for any reason.
Comparison 3 Sensitivity analysis: Excluding trials with less than 80% follow‐up on the variable at the time point, Outcome 4 Service use: 2. Number of participants admitted to hospital.
Comparison 3 Sensitivity analysis: Excluding trials with less than 80% follow‐up on the variable at the time point, Outcome 5 Quality of Life: Average endpoint QOL‐QOLI ‐ various subscales (high = better).
Comparison 4 Sensitivity analysis: Excluding trials where IPS was augmented with other interventions, Outcome 1 Employment.
Comparison 4 Sensitivity analysis: Excluding trials where IPS was augmented with other interventions, Outcome 2 Leaving the study early for any reason.
| Supported employment versus other vocational approaches for adults with severe mental illness | ||||||
| Patient or population: patients with adults with severe mental illness | ||||||
| Outcomes | Illustrative comparative risks* (95% CI) | Relative effect | No of Participants | Quality of the evidence | Comments | |
| Assumed risk | Corresponding risk | |||||
| Other vocational approaches | Supported employment | |||||
| Employment ‐ obtained any job during the study (high=better) | Study population | RR 2.62 | 951 | ⊕⊝⊝⊝ | ||
| 202 per 10001 | 530 per 1000 | |||||
| Moderate | ||||||
| 260 per 10001 | 681 per 1000 | |||||
| Employment ‐ days in competitive employment (primary outcome) ‐ long term | The mean employment ‐ days in competitive employment (primary outcome) ‐ long term in the control groups was | The mean employment ‐ days in competitive employment (primary outcome) ‐ long term in the intervention groups was | 204 | ⊕⊝⊝⊝ | ||
| Employment ‐ days in any form of paid employment ‐ long term | The mean employment ‐ days in any form of paid employment ‐ long term in the control groups was | The mean employment ‐ days in any form of paid employment ‐ long term in the intervention groups was | 510 | ⊕⊝⊝⊝ | ||
| Employment ‐ job tenure for competitive employment (weeks) ‐ long term | The mean employment ‐ job tenure for competitive employment (weeks) ‐ long term in the control groups was | The mean employment ‐ job tenure for competitive employment (weeks) ‐ long term in the intervention groups was | 204 | ⊕⊝⊝⊝ | ||
| Employment ‐ job tenure for any paid employment (weeks) ‐ long term | The mean employment ‐ job tenure for any paid employment (weeks) ‐ long term in the control groups was | The mean employment ‐ job tenure for any paid employment (weeks) ‐ long term in the intervention groups was | 423 | ⊕⊝⊝⊝ | ||
| Time (days) to first competitive employment ‐ long term | The mean time (days) to first competitive employment ‐ long term in the control groups was | The mean time (days) to first competitive employment ‐ long term in the intervention groups was | 204 | ⊕⊝⊝⊝ | ||
| *The basis for the assumed risk (e.g. the median control group risk across studies) is provided in footnotes. The corresponding risk (and its 95% confidence interval) is based on the assumed risk in the comparison group and the relative effect of the intervention (and its 95% CI). | ||||||
| GRADE Working Group grades of evidence | ||||||
| 1 Median control risk across studies. | ||||||
| Outcome or subgroup title | No. of studies | No. of participants | Statistical method | Effect size |
| 1 Employment Show forest plot | 3 | Mean Difference (IV, Random, 95% CI) | Subtotals only | |
| 1.1 days in competitive employment (primary outcome) ‐ long term | 1 | 204 | Mean Difference (IV, Random, 95% CI) | 70.63 [43.22, 98.04] |
| 1.2 days in any form of paid employment ‐ long term | 2 | 510 | Mean Difference (IV, Random, 95% CI) | 84.94 [51.99, 117.89] |
| 1.3 job tenure for competitive employment (weeks) ‐ long term | 1 | 204 | Mean Difference (IV, Random, 95% CI) | 9.86 [5.36, 14.36] |
| 1.4 job tenure for any paid employment (weeks) ‐ long term | 2 | 423 | Mean Difference (IV, Random, 95% CI) | 3.86 [‐5.66, 13.38] |
| 2 Employment (skewed) Show forest plot | Other data | No numeric data | ||
| 2.1 days in competitive employment (primary outcome) ‐ medium term | Other data | No numeric data | ||
| 2.2 days in competitive employment (primary outcome) ‐ long term | Other data | No numeric data | ||
| 2.3 days in any form of paid employment ‐ medium term | Other data | No numeric data | ||
| 2.4 days in any form of paid employment ‐ long term | Other data | No numeric data | ||
| 2.5 job tenure for competitive employment (weeks) ‐ medium term | Other data | No numeric data | ||
| 2.6 job tenure for competitive employment (weeks) ‐ long term | Other data | No numeric data | ||
| 2.7 job tenure for any paid employment (weeks) ‐ medium term | Other data | No numeric data | ||
| 2.8 job tenure for any paid employment (weeks) ‐ long term | Other data | No numeric data | ||
| 2.9 Earnings from paid employment ‐ CAN ($) ‐ long term | Other data | No numeric data | ||
| 2.10 Earnings from paid employment ‐ USD ($) per hour ‐ long term | Other data | No numeric data | ||
| 2.11 Earnings from paid employment ‐ USD ($) ‐ long term | Other data | No numeric data | ||
| 2.12 Earnings from paid employment ‐ AUS ($) ‐ medium term | Other data | No numeric data | ||
| 3 Employment Show forest plot | 7 | Risk Ratio (M‐H, Random, 95% CI) | Subtotals only | |
| 3.1 obtained any job during the study (high=better) | 7 | 951 | Risk Ratio (M‐H, Random, 95% CI) | 3.24 [2.17, 4.82] |
| 4 Time (days) to first competitive employment Show forest plot | 1 | Mean Difference (IV, Fixed, 95% CI) | Subtotals only | |
| 4.1 long term | 1 | 204 | Mean Difference (IV, Fixed, 95% CI) | ‐161.60 [‐225.73, ‐97.47] |
| 5 Time (days) to first competitive employment (skewed) Show forest plot | Other data | No numeric data | ||
| 5.1 medium term | Other data | No numeric data | ||
| 5.2 long term | Other data | No numeric data | ||
| 6 Leaving the study early for any reason Show forest plot | 13 | 2114 | Risk Ratio (M‐H, Random, 95% CI) | 0.76 [0.57, 1.01] |
| 6.1 short term | 1 | 92 | Risk Ratio (M‐H, Random, 95% CI) | 0.33 [0.01, 7.98] |
| 6.2 medium term | 2 | 191 | Risk Ratio (M‐H, Random, 95% CI) | 1.92 [0.98, 3.76] |
| 6.3 long term | 10 | 1831 | Risk Ratio (M‐H, Random, 95% CI) | 0.66 [0.52, 0.84] |
| 7 Mental state: Average endpoint specific symptom score (high = worse) ‐ long term Show forest plot | 3 | Mean Difference (IV, Fixed, 95% CI) | Subtotals only | |
| 7.1 BPRS specific symptom score | 1 | 152 | Mean Difference (IV, Fixed, 95% CI) | ‐1.90 [‐5.71, 1.91] |
| 7.2 PANSS positive symptoms | 2 | 446 | Mean Difference (IV, Fixed, 95% CI) | ‐0.01 [‐0.97, 0.96] |
| 7.3 PANSS negative symptoms | 2 | 446 | Mean Difference (IV, Fixed, 95% CI) | ‐2.12 [‐3.20, ‐1.05] |
| 7.4 PANSS cognitive | 1 | 194 | Mean Difference (IV, Fixed, 95% CI) | ‐1.20 [‐3.09, 0.69] |
| 7.5 PANSS specific symptom score | 1 | 194 | Mean Difference (IV, Fixed, 95% CI) | ‐3.05 [‐8.01, 1.91] |
| 7.6 PANSS emotional discomfort symptoms | 1 | 194 | Mean Difference (IV, Fixed, 95% CI) | 0.17 [‐1.17, 1.51] |
| 7.7 PANSS hostility/excitement | 1 | 194 | Mean Difference (IV, Fixed, 95% CI) | ‐0.17 [‐0.80, 0.46] |
| 7.8 HADS anxiety | 1 | 252 | Mean Difference (IV, Fixed, 95% CI) | ‐0.20 [‐1.30, 0.90] |
| 7.9 HADS depression | 1 | 252 | Mean Difference (IV, Fixed, 95% CI) | ‐0.10 [‐1.20, 1.00] |
| 8 Service use: 1. Mean days in hospital (skewed) Show forest plot | Other data | No numeric data | ||
| 8.1 long term | Other data | No numeric data | ||
| 9 Service use: 2. Number of participants admitted to hospital Show forest plot | 2 | Risk Ratio (M‐H, Fixed, 95% CI) | Subtotals only | |
| 9.1 long term | 2 | 455 | Risk Ratio (M‐H, Fixed, 95% CI) | 0.71 [0.53, 0.96] |
| 10 Quality of Life: Average endpoint QOL‐QOLI ‐ various subscales (high = better) Show forest plot | 5 | Mean Difference (IV, Fixed, 95% CI) | Subtotals only | |
| 10.1 LQoLP‐EU life in general | 5 | 867 | Mean Difference (IV, Fixed, 95% CI) | 0.04 [‐0.10, 0.18] |
| 10.2 financial | 1 | 194 | Mean Difference (IV, Fixed, 95% CI) | 0.10 [‐0.32, 0.52] |
| 10.3 job satisfaction | 1 | 152 | Mean Difference (IV, Fixed, 95% CI) | 0.0 [‐0.53, 0.53] |
| 10.4 housing | 1 | 152 | Mean Difference (IV, Fixed, 95% CI) | 0.0 [‐0.46, 0.46] |
| 10.5 town | 1 | 152 | Mean Difference (IV, Fixed, 95% CI) | 0.20 [‐0.29, 0.69] |
| 10.6 leisure | 2 | 346 | Mean Difference (IV, Fixed, 95% CI) | 0.01 [‐0.25, 0.28] |
| 10.7 services | 1 | 152 | Mean Difference (IV, Fixed, 95% CI) | 0.0 [‐0.38, 0.38] |
| 10.8 vocational services | 1 | 152 | Mean Difference (IV, Fixed, 95% CI) | 0.0 [‐0.51, 0.51] |
| 10.9 time spent with others | 1 | 194 | Mean Difference (IV, Fixed, 95% CI) | ‐0.15 [‐0.48, 0.18] |
| 10.10 socialisation | 1 | 194 | Mean Difference (IV, Fixed, 95% CI) | 0.16 [‐0.09, 0.41] |
| 11 Global/Social functioning: Average endpoint general functioning score ‐ GAS (high = better) Show forest plot | 3 | Mean Difference (IV, Fixed, 95% CI) | Subtotals only | |
| 11.1 long term | 3 | 623 | Mean Difference (IV, Fixed, 95% CI) | ‐0.70 [‐2.82, 1.41] |
| 12 Adverse effects: Death ‐ natural and suicide Show forest plot | 1 | Risk Ratio (M‐H, Fixed, 95% CI) | Subtotals only | |
| 12.1 long term | 1 | 312 | Risk Ratio (M‐H, Fixed, 95% CI) | 1.5 [0.25, 8.85] |
| 13 Economic costs: 1. Direct costs (British £, skewed) Show forest plot | Other data | No numeric data | ||
| 13.1 long term | Other data | No numeric data | ||
| Outcome or subgroup title | No. of studies | No. of participants | Statistical method | Effect size |
| 1 Employment Show forest plot | 2 | Mean Difference (IV, Random, 95% CI) | Subtotals only | |
| 1.1 days in any form of paid employment ‐ long term | 1 | 306 | Mean Difference (IV, Random, 95% CI) | 99.80 [69.50, 130.10] |
| 1.2 job tenure for any paid employment ‐ long term | 1 | 225 | Mean Difference (IV, Random, 95% CI) | ‐1.16 [‐8.50, 6.18] |
| 2 Leaving the study early for any reason Show forest plot | 13 | 2114 | Risk Ratio (M‐H, Random, 95% CI) | 0.76 [0.57, 1.01] |
| 2.1 short term | 1 | 92 | Risk Ratio (M‐H, Random, 95% CI) | 0.33 [0.01, 7.98] |
| 2.2 medium term | 2 | 191 | Risk Ratio (M‐H, Random, 95% CI) | 1.92 [0.98, 3.76] |
| 2.3 long term | 10 | 1831 | Risk Ratio (M‐H, Random, 95% CI) | 0.66 [0.52, 0.84] |
| 3 Mental state: Average endpoint specific symptom score (high = worse) ‐ long term Show forest plot | 3 | Mean Difference (IV, Fixed, 95% CI) | Subtotals only | |
| 3.1 BPRS specific symptom score | 1 | 152 | Mean Difference (IV, Fixed, 95% CI) | ‐1.90 [‐5.71, 1.91] |
| 3.2 PANSS positive symptoms | 2 | 446 | Mean Difference (IV, Fixed, 95% CI) | ‐0.01 [‐0.97, 0.96] |
| 3.3 PANSS negative symptoms | 2 | 446 | Mean Difference (IV, Fixed, 95% CI) | ‐2.12 [‐3.20, ‐1.05] |
| 3.4 PANSS cognitive | 1 | 194 | Mean Difference (IV, Fixed, 95% CI) | ‐1.20 [‐3.09, 0.69] |
| 3.5 PANSS hostility/excitement | 1 | 194 | Mean Difference (IV, Fixed, 95% CI) | ‐0.17 [‐0.80, 0.46] |
| 3.6 PANSS emotional discomfort symptoms | 1 | 194 | Mean Difference (IV, Fixed, 95% CI) | 0.17 [‐1.17, 1.51] |
| 3.7 PANSS specific symptom score | 1 | 194 | Mean Difference (IV, Fixed, 95% CI) | ‐3.05 [‐8.01, 1.91] |
| 3.8 HADS anxiety | 1 | 252 | Mean Difference (IV, Fixed, 95% CI) | ‐0.20 [‐1.30, 0.90] |
| 3.9 HADS depression | 1 | 252 | Mean Difference (IV, Fixed, 95% CI) | ‐0.10 [‐1.20, 1.00] |
| 4 Service use: Number of participants admitted to hospital Show forest plot | 2 | Risk Ratio (M‐H, Fixed, 95% CI) | Subtotals only | |
| 4.1 long term | 2 | 455 | Risk Ratio (M‐H, Fixed, 95% CI) | 0.71 [0.53, 0.96] |
| 5 Quality of Life: Average endpoint QOL‐QOLI ‐ various subscales (high = better) Show forest plot | 4 | Mean Difference (IV, Random, 95% CI) | Subtotals only | |
| 5.1 LQoLP‐EU life in general | 4 | 648 | Mean Difference (IV, Random, 95% CI) | ‐0.02 [‐0.19, 0.14] |
| 5.2 job satisfaction | 1 | 152 | Mean Difference (IV, Random, 95% CI) | 0.0 [‐0.53, 0.53] |
| 5.3 housing | 1 | 152 | Mean Difference (IV, Random, 95% CI) | 0.0 [‐0.46, 0.46] |
| 5.4 town | 1 | 152 | Mean Difference (IV, Random, 95% CI) | 0.20 [‐0.29, 0.69] |
| 5.5 leisure | 2 | 346 | Mean Difference (IV, Random, 95% CI) | 0.01 [‐0.25, 0.28] |
| 5.6 services | 1 | 152 | Mean Difference (IV, Random, 95% CI) | 0.0 [‐0.38, 0.38] |
| 5.7 vocational services | 1 | 152 | Mean Difference (IV, Random, 95% CI) | 0.0 [‐0.51, 0.51] |
| 5.8 time spent with others | 1 | 194 | Mean Difference (IV, Random, 95% CI) | ‐0.15 [‐0.48, 0.18] |
| 5.9 socialisation | 1 | 194 | Mean Difference (IV, Random, 95% CI) | 0.16 [‐0.09, 0.41] |
| 6 Global/Social functioning: Average endpoint general functioning score ‐ GAS (high = better) Show forest plot | 2 | Mean Difference (IV, Fixed, 95% CI) | Subtotals only | |
| 6.1 long term | 2 | 404 | Mean Difference (IV, Fixed, 95% CI) | ‐0.70 [‐3.08, 1.67] |
| 7 Adverse effects: Death ‐ natural and suicide Show forest plot | 1 | Risk Ratio (M‐H, Fixed, 95% CI) | Subtotals only | |
| 7.1 long term | 1 | 312 | Risk Ratio (M‐H, Fixed, 95% CI) | 1.5 [0.25, 8.85] |
| Outcome or subgroup title | No. of studies | No. of participants | Statistical method | Effect size |
| 1 Employment Show forest plot | 2 | Mean Difference (IV, Random, 95% CI) | Subtotals only | |
| 1.1 days in competitive employment (primary outcome) | 1 | 204 | Mean Difference (IV, Random, 95% CI) | 70.63 [43.22, 98.04] |
| 1.2 days in any form of paid employment | 2 | 510 | Mean Difference (IV, Random, 95% CI) | 84.94 [51.99, 117.89] |
| 1.3 job tenure for competitive employment (weeks) | 1 | 204 | Mean Difference (IV, Random, 95% CI) | 9.86 [5.36, 14.36] |
| 1.4 job tenure for any paid employment (weeks) | 1 | 204 | Mean Difference (IV, Random, 95% CI) | 8.56 [2.01, 15.11] |
| 2 Time (days) to first competitive employment Show forest plot | 1 | 204 | Mean Difference (IV, Random, 95% CI) | ‐161.60 [‐225.73, ‐97.47] |
| 3 Leaving the study early for any reason Show forest plot | 6 | 1054 | Risk Ratio (M‐H, Random, 95% CI) | 0.92 [0.54, 1.58] |
| 3.1 medium term | 2 | 191 | Risk Ratio (M‐H, Random, 95% CI) | 5.27 [1.36, 20.34] |
| 3.2 long term | 4 | 863 | Risk Ratio (M‐H, Random, 95% CI) | 0.65 [0.40, 1.05] |
| 4 Service use: 2. Number of participants admitted to hospital Show forest plot | 1 | Risk Ratio (M‐H, Fixed, 95% CI) | Subtotals only | |
| 4.1 long term | 1 | 312 | Risk Ratio (M‐H, Fixed, 95% CI) | 0.63 [0.44, 0.90] |
| 5 Quality of Life: Average endpoint QOL‐QOLI ‐ various subscales (high = better) Show forest plot | 4 | Mean Difference (IV, Random, 95% CI) | Subtotals only | |
| 5.1 LQoLP‐EU life in general ‐ long term | 4 | 817 | Mean Difference (IV, Random, 95% CI) | 0.04 [‐0.13, 0.20] |
| Outcome or subgroup title | No. of studies | No. of participants | Statistical method | Effect size |
| 1 Employment Show forest plot | 2 | Mean Difference (IV, Random, 95% CI) | Subtotals only | |
| 1.1 job tenure for any paid employment (weeks) ‐ long term | 2 | 423 | Mean Difference (IV, Random, 95% CI) | 3.86 [‐5.66, 13.38] |
| 2 Leaving the study early for any reason Show forest plot | 9 | 1364 | Risk Ratio (M‐H, Random, 95% CI) | 0.89 [0.60, 1.33] |
| 2.1 medium term | 3 | 241 | Risk Ratio (M‐H, Random, 95% CI) | 2.76 [0.62, 12.38] |
| 2.2 long term | 6 | 1123 | Risk Ratio (M‐H, Random, 95% CI) | 0.71 [0.49, 1.01] |
