Results of a Coordinated Specialty Care Program for Early Psychosis and Predictors of Outcomes

Multielement early psychosis programs, known as coordinated specialty care (CSC), have rapidly expanded in the United States (1–3). This expansion built upon positive findings from the Recovery After an Initial Schizophrenia Episode (RAISE) project, funded by the National Institute of Mental Health, and was enabled by community mental health block grant federal funding of $25 million starting in 2014 (4–7). Block grant funding was increased in 2016 to $50 milllion annually (1).

A number of well-controlled international studies found that multielement early psychosis programs led to enhanced treatment engagement (8,9), symptom (8–10) and functional improvement (8,11,12), medication adherence (12), reduced substance use (9), better quality of life (8,12), increased treatment satisfaction (9), and fewer hospitalizations (8,12). The first U.S. controlled study of a multielement early psychosis program found that the program was associated with decreased hospitalization and increased vocational engagement (13). The RAISE Early Treatment Program (ETP) study found that CSC was associated with increased treatment engagement, improved quality of life, and greater symptom reduction compared with usual care (5). A recent review of multicomponent interventions for early psychosis found that these interventions had significant advantages over usual care, including reduced treatment disengagement, symptoms, and hospitalization; improved functioning, treatment satisfaction, and quality of life; greater involvement in school and work; and reduced family burden (14).

Despite the growing evidence base for CSC, little is known about the outcomes of CSC in real-world settings where treatment is not provided as part of a research protocol. New York State (NYS) was an early adopter of CSC and has rapidly expanded its use across the state, providing an important opportunity to understand how research findings translate in real-world, community settings (2). This study prospectively evaluated patient outcomes and their predictors over time at OnTrackNY, the CSC program operated by New York State.

Methods

Participants and Study Design

The total sample included individuals ages 16 to 30 who had experienced nonaffective psychosis for less than two years, were enrolled at one of 10 OnTrackNY sites, and had completed at least one three-month follow-up assessment. OnTrackNY clinicians submit client-level data to the NYS Office of Mental Health (OMH) for quality improvement (QI) and fidelity monitoring purposes; data are submitted at enrollment, at the end of every quarter, and at discharge. Clinicians complete standardized admission, follow-up, and discharge forms quarterly on the basis of chart review and reports from participants and their families. Data collected at admission refer to the 90 days prior to the assessment. Teams at each OnTrackNY site submit program-level data quarterly. For research purposes, all protected health information was removed from the data set. The NYS Psychiatric Institute (NYSPI) Institutional Review Board approved the study procedures.

Intervention

OnTrackNY provides CSC, including evidence-based psychosocial interventions and medication, by a multidisciplinary team. OnTrackNY sites are located in licensed outpatient clinics at community agencies, state-operated facilities, and community and academic hospitals in urban and suburban areas throughout the state. At the time of this study, teams were staffed by 3.5 full-time-equivalent employees, including a team leader, one or more licensed primary clinicians, an outreach and recruitment coordinator, a supported education and employment specialist, a prescriber, and a nurse. Clinicians receive initial training and ongoing technical assistance (TA) from the Center for Practice Innovations, a training and TA center at NYSPI funded by the NYS OMH. The program model specifies that individuals are expected to receive services for an average of two years, although variation in length of treatment is expected. The OnTrackNY model is described more fully elsewhere (2), and intervention manuals are available at www.ontrackny.org/resources.

Measures

Domains assessed included demographic characteristics, diagnosis, substance use, antipsychotic use, and suicidal and violent ideation or behavior. Demographic characteristics included race-ethnicity, age, gender, highest level of education completed at time of admission, and health insurance status. Licensed team clinicians conducted a nonstandardized interview at intake. Because the goal of the interview was to establish a diagnosis of primary, nonaffective psychosis to determine eligibility, clinicians received training in differential diagnosis. Report of substance use included any use of alcohol, marijuana, or other drugs (excluding tobacco) in the 90 days prior to admission. Antipsychotic adherence was operationalized as yes (client was prescribed an antipsychotic in the past 90 days and the clinician reported adherence), no (the client was prescribed an antipsychotic in the past 90 days and the clinician reported nonadherence), or no medications prescribed (antipsychotics were not prescribed in the past 90 days). Adherence was defined as taking at least 80% of the prescribed medication. Clients were not prescribed medication if they chose not to take medication following a shared decision-making process. Suicidal and violent ideation or behavior included any report of suicidal ideation or attempts or of violent or aggressive ideation or behavior in the 90 days prior to admission. Time to intervention was defined as the time from onset of psychosis to enrollment in OnTrackNY.

Primary outcomes included the Global Assessment of Functioning (GAF) (15) symptom, occupational and social functioning scales, as adapted by the U.S. Department of Veterans Affairs Mental Illness Research, Education and Clinical Center (MIRECC) for early psychosis (3); education and employment status; and psychiatric hospitalization rate. Scores on this version of the GAF range from 0 to 100, with scores below 40 considered to be evidence of impairment and scores of 70 and above considered to be in the normal range. Primary clinicians were trained to perform the GAF through didactic instruction and case ratings until reliability was achieved (results within ±5 points of gold standard rating). Participation in school included enrollment in an education program, including high school, vocational training, college, or graduate study, either at admission or follow-up and on a full- or part-time basis. Participation in work included any paid employment, including competitive or noncompetitive work, self-employment, or internship at admission or follow-up. Competitive work was defined as work that pays at least minimum wage, involves a paycheck from the employer, and is not reserved for those with behavioral health problems. Hospitalizations included any psychiatric hospitalizations, excluding substance use rehabilitation or detoxification admissions, in the 90 days prior to admission and in the 90 days prior to each follow-up.

Procedures

Enrollment occurred between October 2013 and August 2016. Approximately 40% of referrals were from inpatient units, 20% were from outpatient mental health providers, and 20% were self-referrals or referrals from family. Other referrals were from schools, community organizations, and emergency departments. The data reported in this study were from admission and months 3, 6, 9, and 12 following enrollment, although the length of the follow-up periods varied by participant, depending on the date of enrollment.

Data Analysis

Differences in rate of discharge or dropout by baseline demographic and clinical characteristics were assessed by using t tests for continuous measures and chi-square tests for categorical measures. To assess outcome trends over time, longitudinal models were fit to each of the five outcomes separately by using generalized estimating equations with an autoregressive covariance structure to account for within-subject correlations over time. Linear models were fit for the three continuous and normally distributed GAF score outcomes, with model estimates interpreted as differences. Logistic models were fit for the binary education or employment status and psychiatric hospitalization outcomes, with model estimates interpreted as log odds ratios. Each model contained a main effect of time (three, six, nine, and 12 months) and assessed baseline site variability by adjusting for each respective outcome measure at admission and for site as a fixed effect. These longitudinal models do not require complete data and provide unbiased results under the assumption that the unobserved values are missing at random (16). A Kaplan-Meier estimator was used to estimate the probability that a client would have a psychiatric rehospitalization or have engaged in any education or employment by 12 months, with three, six, nine, or 12 months as the discrete time to event and censoring at last observed follow-up with no event.

To test how each outcome changed over time, prespecified contrasts were tested from the longitudinal model for the mean change in sequential follow-up visits. These contrasts included mean change from baseline to month 3, from months 3 to 6, from months 6 to 9, and from months 9 to 12. To test which predictors were associated with change in each outcome, we ran longitudinal models that included demographic and clinical characteristics separately as well as models that included both types of characteristic concurrently.

All analyses were run by using SAS, version 9.4, and all statistical tests were two-sided with a p value of 5%. Because analysis of dropout led to no meaningful differences between those who dropped out and those who remained engaged, unobserved values due to dropout were assumed to be missing at random and were not expected to lead to biased results.

Results

Demographic and clinical characteristics of the sample (N=325) are described in Table 1. For 54% of participants, time to intervention was less than six months. There were no differences in baseline demographic and clinical characteristics between those who dropped out before 12 months and those who remained enrolled, except in baseline medication adherence (p=.049) and GAF symptom score (p=.044). Those who were discharged before 12 months were less likely to be taking prescribed medication and were more likely to have higher (better) GAF symptom scores at baseline compared with those who were still enrolled at 12 months.

A total of 133 (41%) participants had complete data for up to 12 months. Sixteen percent were discharged or dropped out before 12 months, and 43% were censored before 12 months (because of staggered enrollment) (Table 2). Program evaluation data were also reported for QI purposes just after data were deidentified in August 2016. These data indicated that 334 individuals were enrolled during the study period and that 99% (N=329) were retained at three months. The slight lag between the time of deidentification and the QI report explains why there were 325 individuals in this study and 329 in the QI report.

Figure 1 demonstrates the model-estimated mean scores for the GAF over time and the model-estimated prevalence of engagement in education or employment and psychiatric hospitalization over time. The mean GAF score at baseline across the 10 sites ranged from 35.2 to 65.5 for social functioning, 25.5 to 54.0 for occupational functioning, and 25.4 to 43.1 for symptoms. Whereas more than 70% of participants were hospitalized in the three months prior to enrollment, the hospitalization rate following enrollment decreased to an estimated 10% per quarter. The estimated risk of rehospitalization for a psychiatric condition within one year of enrollment was 34% (95% confidence interval [CI]=28%–41%).

Approximately 40% of participants were engaged in school or work at baseline, which increased to an estimated 80% by six months. The estimated probability of being employed or in school at some time during the year after admission was 88% (CI=83%–92%). At baseline, 10% of clients were employed only, 29% were in school only, and 5% were both employed and in school. Of the 50 clients who were employed, 44 (88%) had a competitive job. Of the 260 clients with data at six-month follow-up, 77 (30%) were employed only, 75 (29%) were in school only, and 47 (18%) were both employed and in school. Of the 124 clients who were employed, 81 (65%) had a competitive job.

Table 3 reports the estimated change in outcomes between each time point. The greatest improvement in GAF scores and education or employment status occurred between baseline and three months, with continued improvement to month 6. Statistically significant improvements in social functioning scores were seen at each additional time point, whereas symptom scores continued to improve through month 9. Statistically significant reductions in hospitalization were seen from baseline to month 3 and from months 3 to 6, and hospitalization rates remained low thereafter. [A table showing the percentage of clients with GAF scores >40 and >70 is available in the online supplement.]

Table 4 shows the estimated effects of predictors on longitudinal outcomes. Site was significantly associated with all GAF outcomes (social functioning, χ²=27.30, df=9, p=.001; occupational functioning, χ²=26.59, df=9, p=.002; and symptoms, χ²=19.06, df=9, p=.025) but not with education or employment or hospitalization. However, only 4% of the variability in baseline social functioning scores was due to site, compared with 7% of the variability in occupational scores and 5% for symptom scores (data not shown). Baseline scores in each domain were highly associated with subsequent scores in that domain, and baseline participation in education or employment was highly associated with subsequent education or employment.

Compared with women, men had significantly lower (worse) occupational scores and lower odds of being in school or employed. Compared with non-Hispanic whites, Asians and Hispanics had significantly lower occupational scores and lower odds of education or employment. Compared with non-Hispanic whites, blacks had lower odds of education or employment. Compared with individuals who had not yet completed high school, individuals with a high school degree or GED had lower educational or employment status at follow-up. Having some college (versus less than a high school degree), shorter time from onset of psychosis to program enrollment, and medication not prescribed (versus nonadherence) predicted higher (better) symptom scores. A lower (worse) baseline symptom score was associated with higher odds of having a psychiatric hospitalization. We repeated the analysis while controlling for parents’ highest education, with the only difference being that age became significantly associated with the occupational score. Older clients had significantly lower (worse) occupational scores.

To determine whether some predictors that were not significantly associated with outcomes would become significant if the analyses did not control for the other predictors, we repeated the analyses, this time controlling for baseline demographic characteristics and baseline outcomes but not mutually controlling for other predictors. We found no differences in significance between the approaches, except that compared with medication nonadherence, not being prescribed an antipsychotic was significantly associated with lower odds of a psychiatric hospitalization.

Discussion

This is the first large-scale report of outcomes from a CSC program in the United States that is not part of a research study. The data demonstrate improved symptoms and occupational and social functioning, increased education and employment status, and decreased hospitalization rate. The greatest gains were seen in the first three months following enrollment, with continued improvement over time.

Participation in school or work was comparable to or exceeded rates seen in other CSC studies (5,13). OnTrackNY programs have a full-time supported employment and education specialist, which has been shown to improve vocational outcomes (17). The increased participation in school and work among OnTrackNY participants may also be due to the fact that the duration of untreated psychosis (DUP) among participants in this study was relatively short, compared with studies that did not limit DUP. Eligibility for OnTrackNY services is limited to individuals with DUP of less than two years; DUP in this study was an average of 225 days (median=160 days, or 5.3 months). Shorter DUP has been associated with better outcomes overall (14,18) and improved school or work outcomes (19–21). The shorter time to intervention may be due in part to the emphasis on outreach; each OnTrackNY has a part-time outreach and recruitment coordinator responsible for outreach (equal to .5 full-time-equivalent positions, approximately). DUP was longer in the RAISE-ETP study, which limited enrollment based on prior treatment but not duration of psychosis; the median DUP in the RAISE-ETP study was 74 weeks (5).

Rates of enrollment in the Supplemental Security Income (SSI) or the Social Security Disability Insurance (SSDI) programs among participants in OnTrackNY were somewhat lower than those observed in the RAISE-ETP study (22,23). In that study, the mean±SE percentage of participants enrolled in SSI/SSDI at baseline was 14%±2.9% for those enrolled in community care (CC) and 6%±1.5% for those enrolled in Navigate, the CSC model tested in the study. Rates rose to 36%±4.2% and 34%±3.2% at one year and to 48%±5.8% and 55%±4.2% at two years for CC and Navigate, respectively. In OnTrackNY, only 3% of individuals received SSI/SSDI at baseline. Participation was estimated to be 11% (CI=9%–15%) at one year and 18% (CI=14%–24%) at two years (14–21,24–26). Notably, some participants in the RAISE-ETP study may have needed to apply for SSI/SSDI in order to obtain Medicaid and receive treatment services, which was likely a factor in the SSI/SSDI participation rates in the RAISE-ETP study. It is possible that OnTrackNY participants were less reliant on applying for SSI/SSDI as a strategy to obtain health care coverage because OnTrackNY services are provided regardless of insurance status. We did not assess whether disability at baseline was a predictor of participation in work or school because baseline rates of disability were relatively low. Also, given the relatively high rates of employment and school participation, we anticipated that receipt of SSI/SSDI would result from poor work and school functioning rather than the other way around.

Other studies of early psychosis have also found that baseline levels of education and employment predicted school and work outcomes (19,20,24,25). We were not able to measure baseline cognitive and premorbid functioning, which have been found to predict vocational outcomes in first-episode psychosis (FEP) (25–27).

It is noteworthy that social functioning improved over every time point and that the percentage of clients with scores in the normal range was higher for social functioning than for the other GAF subscales at every time point. Other studies have also found that baseline social functioning predicted follow-up social functioning (28). Similar improvements in social functioning have been seen in other FEP studies (29).

Compared with medication nonadherence, no medication prescription was associated with higher (better) symptom scores. That may be because those with fewer symptoms were less likely to agree to antipsychotic medication.

The rate of hospitalization, 34% in one year, was similar to the rates in other studies of CSC. In the RAISE-ETP study, 34% of participants in CSC and 37% of participants in community care were hospitalized over a two-year period. In the Specialized Treatment in Early Psychosis study, 24% of participants in CSC and 44% in usual treatment (5,13) were hospitalized. Reductions in hospitalization are to be expected given the high rates of recent hospitalization at enrollment. Other studies have identified prior hospitalization, older age, lower GAF scores, and substance use as predictors of hospitalization (30–32).

Of note, substance use was not a significant predictor of any of the outcomes studied. This finding differs from some of the literature. Lower use or no use of substances has been identified as a predictor of recovery, and substance use has been identified as a negative predictor of education and employment (25,33,34). Substance use may have been underreported, given that we did not use a standardized assessment or collect self-report or collateral data.

The study was limited by the constraints of data collection in a context not associated with research. First, without a control condition, we cannot say that OnTrackNY caused the observed outcomes. All ratings were performed by clinical staff and, therefore, were subject to clinician bias. Diagnosis, substance use, and medication adherence were not determined by using a structured interview or tool. We did not use self-report or collateral measures and did not examine which components of the model were utilized. Although teams were closely monitored for adherence to the clinical model, we did not utilize a formal fidelity scale, although one is in development. OnTrackNY has the benefit of strong state support, including funding for a statewide TA center.

Conclusions

This study demonstrated that providing CSC in community settings and not as part of a research protocol was associated with significant improvements in education and employment and a decrease in the hospitalization rate among individuals with early psychosis. Several demographic variables and baseline education predicted education and employment outcomes. CSC teams should make particular effort to support the work and school goals of individuals who may be more likely to struggle in achieving engagement in work and school, including male participants and participants from racial and ethnic minority groups.