Inpatient Psychiatric Care Experience and Its Relationship to Posthospitalization Treatment Participation

After being discharged from inpatient care, psychiatric patients experience elevated symptoms, functional deficits, and increased chance of suicide (1,2). Mental health care participation during this time promotes adaptive functioning and reduces the chance of readmission (3,4). In recognition of the importance of ongoing care after inpatient treatment, the National Council for Quality Assurance has chosen posthospitalization follow-up as a quality measure by which to evaluate health plans (5), and the Veterans Health Administration (VHA) requires follow-up within seven days of discharge (6).

Unfortunately, a variety of barriers can interfere with postdischarge treatment participation. External barriers, such as distance to treatment, transportation concerns, and financial limitations, can hinder treatment participation (7–9). Internal barriers, such as stigma, negative expectations of treatment, and distrust of health care providers, may similarly limit engagement (10,11). In part because of the impact of such internal factors, there has been an increased interest in evaluating the treatment experiences of patients receiving mental health care (12).

Within the VHA, patient treatment experience has been measured by the Survey of Healthcare Experiences of Patients (SHEP), which includes a version specifically designed to assess recent inpatient care (I-SHEP). The I-SHEP is the only measure of veterans’ inpatient treatment experience that is widely distributed within the U.S. Department of Veterans Affairs (VA) treatment system and the only departmentwide source of information on veterans’ satisfaction with inpatient psychiatric treatment.

Veteran responses to SHEP items have been used to evaluate a variety of treatment-shaping factors, such as alcohol use and health functioning among older patients (13); gender differences in care satisfaction (14); and racial differences in the receipt and effectiveness of treatment for pain (15). Although information from the SHEP has proven to be useful for understanding these areas, studies using SHEP data have had little consistency in approach or definition of variables, and use of SHEP data has not been validated among patients with a recent psychiatric hospitalization.

For these reasons, this study adopted a factor-analysis approach to clarify underlying I-SHEP factors, create clearer definitions of these factors, and examine the utility of I-SHEP factors in evaluating treatment satisfaction among patients with a recent psychiatric hospitalization. The goal of this study was to determine subdomains of the inpatient psychiatric care experience that are measured by the I-SHEP. We then evaluated whether the resulting factors, representing different aspects of the inpatient care experience, were associated with postinpatient treatment participation. Such information has the potential to improve care by allowing clinicians to promote aspects of inpatient care that support postinpatient treatment participation.

Methods

Study population and data sources

Patients included in the study were the 34,237 veterans randomly selected to receive the annual I-SHEP in fiscal year (FY) 2009. To be eligible for the survey, patients must have been discharged from inpatient psychiatric care within the VHA medical system to outpatient care during FY 2009, with patients discharged from inpatient psychiatric care to other high-intensity settings, such as nursing homes or medical inpatient care, eliminated from the study sample. A total of 7,408 (21.6%) veterans completed and returned the FY 2009 I-SHEP (responders), and 26,829 (78.4%) did not (nonresponders). An additional comparison group was created from all VHA patients who received inpatient psychiatric care during FY 2009 (N=65,775) to evaluate the level of representativeness of I-SHEP responders of the overall VHA inpatient psychiatric population.

Additional information was collected from the National Patient Care Database (NPCD), a large administrative data set containing patient demographic, diagnostic, and treatment information. This study was conducted in accordance with institutional review board approval from the VA Ann Arbor Health System.

I-SHEP

The I-SHEP is mailed to randomly selected VHA patients following their discharge from inpatient care to assess various care aspects for the purposes of continuous quality assurance. Detailed information related to SHEP survey methods has been presented previously (14). Briefly, the I-SHEP is designed as a stratified random sample without replacement and is based on monthly samples from VHA national computerized data on discharges and visits. A random sample is created from VHA patients discharged alive from any VHA hospital inpatient care to the community, and the I-SHEP is mailed to sample patients within a month of their discharge from inpatient care. I-SHEP contains 53 items and assesses interactions with treatment staff, perceptions of treatment environment, overall impressions of care, and postinpatient treatment planning. Response options to I-SHEP items include a variety of scales, for example, yes or no, “always” to “never,” and “very satisfied” to “very dissatisfied.”

Measures

Adequacy of postinpatient care was assessed on the basis of whether patients attended an outpatient visit with a mental health provider within seven days or within 30 days (two separate measures) of their discharge from inpatient care. These measures have been included in the National Committee for Quality Assurance’s Healthcare Effectiveness Data and Information Set, which tracks performance on health-related quality measures among private, Medicare, and Medicaid health plans within the United States (5). Information related to postinpatient treatment participation was gathered from the NPCD.

Information related to patient gender, race, ethnicity, age, and diagnosis was collected from the NPCD. Additional NPCD information was gathered related to the length of inpatient stay, distance between hospital and patient residence, and attendance at a mental health visit in the 90 days prior to hospitalization. To account for the potential effects of serious medical comorbidity, Charlson Comorbidity Index (CCI) scores (16) were calculated for each patient. Cut-off scores for the variables were chosen on the basis of previous research that indicated that mental health treatment outcomes among specific clinical groups within a population of veterans could be distinguished by certain characteristics, such as distance to treatment center, age (17), and length of stay (18).

Analyses

The content structure of the I-SHEP was evaluated through the use of exploratory factor-analysis methods, with analyses focusing on identifying the underlying conceptual factors within patients’ I-SHEP responses. The factor analysis included 37 I-SHEP items, having eliminated items related to demographic characteristics and experiences associated with an inpatient medical or surgical stay rather than a psychiatric stay, such as “How often did you get help in getting to the bathroom as soon as you wanted?” We included an item regarding whether the patient had a specific complaint but excluded the follow-up items, for example, “If you had a complaint, how easy was it for you to find someone to hear your complaint?” Because of a lack of normally distributed I-SHEP items, a principal-factors extraction approach was utilized for the factor analysis. Varimax rotation was applied to the resulting factor structure.

I-SHEP factor scales were created to provide scores representative of each patient’s level of satisfaction with different I-SHEP factor areas. To accomplish this, I-SHEP items were converted to a standard 4-point value scale, and a composite score was calculated for each I-SHEP factor by summing individual item scores within each factor. Regression models were then created to identify associations between I-SHEP factor scores and patient variables. To investigate potential relationships between I-SHEP factors and outcome measures, two logistic regression models were created, with attendance at seven-day and 30-day mental health appointments as the outcome measures and I-SHEP factors as the predictors. Patient demographic, illness, and treatment variables were included in these models to control for the effects of these variables. All regression analyses included a weighing variable to correct for the effects of facility and patient factors that contribute to I-SHEP nonresponse to ensure that results were representative of all patients receiving inpatient psychiatric care within the VHA and not only of those returning the I-SHEP.

Results

Characteristics of I-SHEP responders (N=7,408), nonresponders (N=26,829), and all VA patients who received inpatient psychiatric treatment (N=65,775) in FY 2009 are shown in Table 1 for comparison. Chi square analyses were used to evaluate potential between-group differences between I-SHEP responders and nonresponders. Nonresponders were more likely than responders (absolute difference ≥5%) to be African American, to be under age 45, and to have a comorbid substance use disorder. They were less likely than responders to be age 45–65, to have been given a diagnosis of major depressive disorder or posttraumatic stress disorder (PTSD), or to have a postdischarge follow-up visit within 30 days.

Additionally, the group of all VA patients who received psychiatric inpatient care was less likely than I-SHEP responders and nonresponders to attend seven-day and 30-day follow-up appointments. The group of all VA patients also had a larger percentage of patients with inpatient stays of greater than 14 days, suggesting an increased need for intensive care that may have had an impact on postdischarge treatment participation. Only I-SHEP responders (N=7,408) were included in subsequent analyses.

Factor analysis

A three-factor model best fit the data, according to several indicators. Eigenvalues dropped sharply between factors 3 and 4 (factor 1, 13.83; factor 2, 1.85; factor 3, .99; and factor 4, .58), and a skree plot demonstrated a clear change in slope between factors 3 and 4. Finally, a three-factor solution allowed for a relatively clean factor model, with only three of 37 items loading on more than one factor at .40 or higher. These indicators represented adequate information to inform the selection of number of factors (19). This factor solution accounted for 16.7% (r²=.167) of the variance in patient responses.

Utilizing the suggested factor loading cutoff of .40 (19), we excluded five items because of inadequate loading. In the case of items with multiple factor loadings greater than .40, we assigned items to the factor with the higher loading—in each case, this represented a difference of >.10. The final three-factor solution is presented in Table 2. Each factor was assigned a name on the basis of common themes within the factor, resulting in the “respect and caring by nurses–overall hospital impression” factor (14 items, score range 14–56), the “involvement and information about care decisions” factor (13 items, score range 13–52) and the “respect and caring by doctors” factor (five items, score range 5–20).

The respect and caring by nurses–overall hospital impression factor, known as the “hospital-nurses” factor, contained items assessing satisfaction with the patient-nurse interaction and the overall hospital experience. The involvement and information about care decisions factor, known as the “care information” factor, consisted of items assessing the extent to which providers shared information related to mental health treatment decisions and symptom management. The respect and caring by doctors factor, known as the “doctors” factor, measured satisfaction with patient-doctor interactions. These factors had good internal reliability, with Cronbach’s alphas of .93 (hospital-nurses), .90 (care information), and .94 (doctors).

Patients endorsed moderate to high levels of satisfaction with their inpatient psychiatric care. The overall mean scores for each overall factor measure were in the upper third of the range of potential scores (46.83±8.66, hospital-nurses; 41.82±10.99, care information; 16.97±3.78, doctors).

Factor evaluation

Relationships between the three I-SHEP factors and patient characteristics are presented in Table 3. A more positive view of nurses and the overall hospital experience was associated with male gender (β=.97), Hispanic ethnicity (β=2.03), age 45 and older (45–65 versus <45, β=1.32; >65 versus <45, β=1.84), and the absence of PTSD (β=–.88) or a personality disorder (β=–1.28). Increased satisfaction with care information was associated with African-American race (β=.63), the absence of a personality disorder (β=–1.62), and an inpatient hospitalization of seven or more days (seven to 14 days versus less than seven days, β=.61; >14 days versus less than seven days, β=1.41). Increased satisfaction with doctors was associated with Hispanic ethnicity (β=1.11), older age (45–65 versus <45 years, β=.46; >65 versus <45 years, β=.66), the absence of a substance use disorder (β=–.32), the absence of PTSD (β=–.22), and the absence of a personality disorder (β=–.82).

Posthospitalization mental health treatment participation

Correlates of receipt of posthospitalization follow-up care, including the three I-SHEP factors, are presented in Table 4. Patients were more likely to attend a follow-up appointment within seven days if they had increased satisfaction with care information (one standard deviation [SD], 10.99 points on a 40-point scale, odds ratio [OR]=1.14), had a diagnosis of major depressive disorder (OR=1.48) or bipolar disorder (OR=1.30), or had attended an outpatient mental health visit in the 90 days prior to hospitalization (OR=2.07). Decreased likelihood of follow-up psychiatric treatment attendance was associated with male gender (OR=.80), African-American race (OR=.80), age older than 65 versus <45 (OR=.72), and increased medical comorbidity (CCI score 1 versus 0, OR=.81; CCI score ≥2 versus 0, OR=.78).

Increased follow-up within 30 days was associated with increased satisfaction with care information (one SD, 10.99 points on a 40-point scale, OR=1.17); diagnosis of major depressive disorder (OR=2.07), PTSD (OR=1.30), or bipolar disorder (OR=1.64); attendance at a prehospitalization mental health visit (OR=2.36); and hospitalization lasting seven to 14 days versus fewer than seven days (OR=1.28). Decreased follow-up within 30 days was associated with African-American race (OR=.66), older age (45–65 versus <45, OR=.74; >65 versus <45, OR=.49), diagnosis of a substance use disorder (OR=.66), moderate levels of medical comorbidity (CCI score 1 versus 0, OR=.83), and increased distance from hospital (>60 miles versus <30 miles, OR=.35).

Discussion

Similar to respondents to other surveys of health care experiences, respondents endorsed moderate to high levels of satisfaction with their inpatient psychiatric care (20). The I-SHEP factor analysis suggested that the survey measures three aspects of patients’ satisfaction with inpatient psychiatric treatment: hospital-nurses, care information, and doctors. The factor model accounted for a moderate amount of the variance in treatment satisfaction, and several relationships between I-SHEP factors and other variables emerged.

Factor evaluation

I-SHEP factors were related to several patient demographic and treatment variables. Consistent with previous research, patients with personality disorders, women, younger patients, and patients with shorter lengths of stay were less satisfied with their inpatient care (14,21,22). The consistency of these relationships between this study and previous work provides some support for the external validity of I-SHEP factors. Previous work found no relationship between African-American race, Hispanic ethnicity, and care satisfaction (23,24). However, perhaps because of the novel I-SHEP factor structure, our results showed that Hispanics were more satisfied with hospital-nurses and doctors and that African Americans were more satisfied with care information. Other novel findings were that patients with substance use disorders were less satisfied with doctors and that patients with PTSD were less satisfied with doctors and hospital-nurses. Whether such dissatisfaction was due to limited PTSD-specific treatments on general acute inpatient psychiatry units, an interaction between PTSD and the environment of care on inpatient units, or other possibilities warrants further study (25).

The I-SHEP factor structure shares similarities with previous evaluations of inpatient satisfaction. Previous studies have also found differential relationships between satisfaction and staff-patient relationships and information related to care, suggesting that these factors represent two distinct content areas (26,27). Although a positive correlation between satisfaction with nursing staff and overall mental health treatment experience has been found previously (22,28), we found that the two loaded onto the same factor.

Relationships between factors and postinpatient treatment

The group of all VA patients who received inpatient psychiatric treatment was less likely than I-SHEP responders or nonresponders to attend seven-day and 30-day follow-up appointments. This may be because the receipt of the I-SHEP served as a prompt to engage in aftercare or because the overall group contained patients who were transferred from inpatient care to another intensive-care setting, which would have interfered with participation in aftercare.

Several patient diagnostic, demographic, and treatment variables were associated with engagement in postinpatient care among I-SHEP responders. Increased attendance at appointments within both seven and 30 days after discharge was associated with Caucasian race, attendance at a prehospitalization mental health appointment in the previous 90 days, younger age, diagnosis of major depressive disorder or bipolar disorder, and less medical comorbidity.

Similar trends between older age and less treatment participation have been found in studies of outpatient care of depression and bipolar care (29–31). These articles suggested that older adults are less likely than younger adults to initiate mental health treatment (29,31), although those who initiate are more treatment adherent, particularly to medications, than younger patients (29,30). Similarly, attendance at prehospitalization mental health appointments has been linked to participation in posthospitalization care (32) and reduced medical comorbidity (29), and studies have found reduced participation in mental health care among African-American veterans (33). Increased postinpatient follow-up by veterans with depression or bipolar disorder represented a novel finding and may be due to better functioning among these patients than among veterans with PTSD or schizophrenia. Increased functioning may allow them to overcome barriers to care participation that more disabled psychiatric patients frequently encounter, such as incomplete remission of symptoms (34) or competing life obligations (35).

Care information was the only I-SHEP factor associated with postinpatient care participation. Many of this factor’s items involve the process of engaging patients in designing and participating in their mental health care. Such information sharing and patient-provider collaboration are central aspects of effective recovery-oriented care (36,37) and have been linked to increased care satisfaction (38) and treatment engagement in outpatient psychiatric settings (37). Our findings suggest that these principles are similarly important during the transition from inpatient to outpatient care.

A lack of relationship between the other I-SHEP factors and postinpatient follow-up may be due to several reasons. The two factors not associated with subsequent care involved a greater degree of interpersonal interaction. Because patients may be unlikely to follow up with the same providers they worked with during their inpatient stay, there may be minimal outpatient carryover from inpatient working relationships. The care information factor may represent content that could directly impact patient mental health treatment participation by reducing barriers to care, such as transportation, finances, and interpersonal support, whereas the other two factors may offer less assistance in addressing these barriers to care. Finally, strength of habit may play a central role in shaping mental health care engagement within a large, integrated health care system such as the VHA, where patients can be continuously engaged in care for years. If that is the case, patients with well-established expectations and engagement habits will be much less influenced by short-term relationships with treatment staff than by involvement with and clarity of posthospitalization treatment plans.

There were limitations to the approach adopted within this study. Restricted representation of some groups, such as women, may limit generalizability to non-VHA populations. The low survey response rate and differences between I-SHEP responders and nonresponders limit the generalizability of our findings to all VHA patients who were hospitalized for a psychiatric condition. The survey response patterns indicate that the findings may overrepresent patients with major depression and PTSD compared with those with substance use disorders and personality disorders.

Because responders were more likely than nonresponders to have a 30-day follow-up appointment, survey responders may have had overall greater satisfaction with their care than nonresponders. I-SHEP responses represent patients’ subjective report of their treatment experience, for example, degree of information provided at discharge, and not an objectively observed measure of these treatment aspects. I-SHEP factors were not compared with other similar measures in order to evaluate their concurrent validity. The gap between discharge from inpatient care and completion of the I-SHEP may have resulted in a recall bias regarding responders’ views of their inpatient care. We also cannot exclude the possibility that an unmeasured confounding factor, such as self-efficacy, underlay both patients’ perceptions of care provided and the likelihood of patients’ follow-up. The influence of external pressures to engage in postinpatient care, such as court-mandated treatment, on patient treatment participation was not independently assessed.

The limited association between I-SHEP factors and postdischarge mental health care participation may be due in part to the resources available for supporting postdischarge engagement. At sites where fewer resources are devoted to supporting postdischarge psychiatric treatment participation, such as telephone reminders of appointments and emergency transportation, the association between the I-SHEP factor areas and outpatient engagement is likely stronger, given that participation in postdischarge care is more reliant upon patient treatment motivation and investment. Given that the percentage of patients who attended postinpatient follow-up appointments was higher among VHA patients who completed the I-SHEP than among patients outside the VHA system (33) (70% and 30% at seven-day and 89% and 49% at 30-day follow-ups, respectively), other health systems may find patients’ experiences with their inpatient treatment play a stronger role in determining who attends follow-up appointments.

Conclusions

This study utilized factor-analysis methodology to identify factors related to inpatient psychiatric treatment satisfaction of patients as well as the relationships between these factors and postdischarge mental health care participation. Few previous studies have evaluated patients’ satisfaction with their inpatient psychiatric care and its relationship with later mental health care engagement, and no previous study has sought to determine the underlying factors within this relationship.

We found patients’ overall health care experience and interactions with nurses and doctors were not associated with participating in posthospitalization follow-up care. However, there was an association between increased patient involvement in care planning and subsequent follow-up. Although causality cannot be determined from this study, these findings suggest that providing more information and opportunities for involvement in care decisions during an inpatient stay may reduce the risk of nonadherence to subsequent outpatient appointments. These findings should be considered when designing interventions to improve transitions from inpatient to outpatient mental health settings.