Using a Computer Database to Monitor Compliance With Pharmacotherapeutic Guidelines for Schizophrenia

Performance assessment has become increasingly important because of efforts to control health service utilization and reduce costs while preserving the quality of care. Evidence-based clinical guidelines offer a formal means of assessing the quality of care provided by individual clinicians and health care organizations (1). Many guidelines have recently appeared in the mental health field, including the treatment recommendations of the Schizophrenia Patient Outcomes Research Team (PORT) (2). Health organizations are increasingly using administrative data to monitor clinical programs, measure outcomes, and assess providers' performance (3,4,5,6).

A recent study based on interviews and chart reviews for 719 patients demonstrated the use of intensive data-collecting methods to assess adherence to guidelines in the treatment of schizophrenia (7). Similar methods were used to evaluate the quality of care provided to 224 patients with schizophrenia (8). Unfortunately, these data-gathering methods were resource intensive, which potentially limited their relevance with large samples. To our knowledge, no studies describing the use of computerized patient data to monitor conformance with published mental health guidelines have been reported. The ability to flexibly examine such data to assess patterns of care efficiently is potentially of great value to program managers and health policy makers.

We present an approach to monitoring adherence to pharmacotherapeutic guidelines in the management of schizophrenia that was developed as part of an ongoing effort to monitor quality of care in Veterans Affairs mental health programs (5,6). The study had three primary objectives. First, we translated PORT guidelines into measures derived from a clinical patient database to assess guideline conformance. Second, we used the guidelines and additional measures to assess the care of 353 patients with a primary diagnosis of schizophrenia discharged from a large academically affiliated medical center over a four-year period. Finally, we identified individual patient characteristics significantly associated with guideline adherence. Thus the study was both a methodological and a substantive contribution. This report illustrates the challenge of such an effort and presents preliminary data on guideline conformance at one medical center.

Methods

Data collection

The Veterans Affairs medical center where the study was conducted is a large academically affiliated tertiary medical center located in a midsized Northeastern city. The center's clinical patient database contains data on outpatient prescriptions, such as information on prescription dosages; inpatient admission data, such as primary ICD-9 discharge diagnoses; and outpatient appointment data, such as the dates of visits. Data from March 31, 1994, to March 10, 1998, were extracted into a database program (Microsoft Access 97) for analysis.

Patient sample

We identified 484 patients with at least one hospitalization for a primary diagnosis of schizophrenia (ICD-9 codes 295.xx) during the four years. We then excluded 80 patients who were subjects in investigational protocols during the study period. Finally, we identified patients with two or more outpatient psychiatric visits to the medical center within a six-month period after discharge, yielding our final cohort of 353 patients. We used these criteria to select patients who most likely received their long-term outpatient psychiatric care at the VA medical center.

Data analysis

We reviewed the PORT treatment recommendations and developed protocols for querying the database to measure conformance with guidelines where possible. Of the PORT's 18 pharmacotherapeutic guidelines, three could be meaningfully addressed with the available computerized patient data. In addition, we analyzed other parameters felt to be relevant to evaluating pharmacotherapy for schizophrenia.

Measuring guideline conformance

As our indicator of an acute symptom episode, we identified a patient's most recent admission for schizophrenia that was followed by two or more outpatient mental health visits in the following six months. We then extracted data on all antipsychotic prescriptions issued in the period from five days before hospital discharge to 42 days after discharge. Next, we determined the most recent prescription during that interval and calculated the daily chlorpromazine-equivalent dosage, where possible. If a patient received more than one antipsychotic on a single day during the period, the chlorpromazine-equivalent dosages for all medications received that day were summed. Chlorpromazine-equivalent dosages were available for clozapine and risperidone (2) but not for olanzapine. Thus we could not perform this analysis for olanzapine.

These data were used to assess conformance to PORT recommendation 1, on first-line therapy, and recommendation 2, on dosage:

• Recommendation 1: Antipsychotic medications other than clozapine should be used as the first-line treatment to reduce psychotic symptoms for persons experiencing an acute symptom episode of schizophrenia.

• Recommendation 2: The dosage of antipsychotic medication for an acute symptom episode should be in the range of 300 to 1,000 mg of chlorpromazine equivalents per day for a minimum of six weeks. Reasons for dosages outside this range should be justified. The minimum effective dose should be used.

We used longitudinal prescription data to evaluate compliance with PORT recommendation 13, on clozapine trials:

• Recommendation 13 (abbreviated): A trial of clozapine should be offered to patients with schizophrenia or schizoaffective disorder whose positive symptoms do not robustly respond to adequate trials of two different classes of antipsychotic medications.

In this case, we identified patients who had received conventional therapy before being started on an atypical antipsychotic (clozapine, olanzapine, or risperidone) at any time during the four-year study period. We then calculated the daily chlorpromazine-equivalent dosage for patients' most recent conventional antipsychotic regimen before their first atypical antipsychotic prescription.

We determined the percentage of patients currently on depot antipsychotic medications and the percentage of patients who had at some point during the study period received depot therapy.

We also examined prescribing patterns for benzodiazepines for patients with a dual diagnosis of drug or alcohol abuse and identified patients with at least one benzodiazepine prescription.

Patient characteristics and guideline adherence

We used the chi square and Fisher's exact tests (SPSS 8.0) to examine the relationship between adherence to the chlorpromazine-equivalent-dosage guideline (recommendation 2) and sex, race, and type of antipsychotic regimen (conventional versus atypical). We computed Spearman rank correlation coefficients to determine the relationships between adherence to the dosage guideline and patients' age, distance of residence from the medical center, and time in the study. This approach translated to a significance level of .02 for each of the three associations for the purpose of maintaining an overall study significance of .05 after Bonferroni correction for multiple comparisons.

Results

Demographic characteristics

The group consisted of 338 men (96 percent) and 15 women. Ninety-one (26 percent) were African Americans, 245 (69 percent) were Caucasians, and 17 (5 percent) were Hispanics. The mean±SD age of our cohort was 48±12 years, with a range of 23 to 81 years. Forty-one of the 353 patients (12 percent) were 65 years of age or older.

Conformance and other measures

PORT recommendation 1 on first-line therapy. Prescriptions for antipsychotic medications (conventional agents, clozapine, risperidone, olanzapine, or depot therapy) were issued for 272 of the 353 patients (77 percent) in the five days before discharge or the 42 days after discharge. Of the 272 patients, only 15 (6 percent) were placed on an antipsychotic regimen that included clozapine.

PORT recommendation 2 on dosage. Among the 185 patients receiving only conventional antipsychotics, 77 (42 percent) were prescribed a chlorpromazine-equivalent dosage below the recommended range of 300 to 1,000 mg of chlorpromazine equivalents, and nine (5 percent) were prescribed a dosage above the recommended range. However, of the 53 patients on clozapine or risperidone, 46 (87 percent) were given dosages within the recommended range.

Overall, of the 238 patients either on conventional antipsychotics or on clozapine or risperidone, 83 (35 percent) were prescribed dosages below the accepted range, 145 (61 percent) had dosages within the range, and ten (4 percent) had dosages above it.

PORT recommendation 13 on clozapine trials. Of the 272 patients who received antipsychotic prescriptions, 15 (6 percent) were prescribed clozapine, 39 (14 percent) risperidone, and 19 (7 percent) olanzapine. (One patient was prescribed both risperidone and clozapine.) We examined dosages of the most recent regimens of conventional antipsychotics before atypical therapy was initiated. For 29 of 72 patients (40 percent), the dosages of conventional medications were below the recommended range. Similar analyses for clozapine revealed that six of 20 patients (30 percent) were receiving dosages of conventional medications greater than or equal to 1,000 mg of chlorpromazine equivalents, whereas only two of 20 patients (10 percent) were receiving chlorpromazine-equivalent dosages below the recommended range.

Depot therapy. Of the 272 patients who received an antipsychotic, the most recent antipsychotic regimen for 16 patients (6 percent) included either haloperidol decanoate or fluphenazine decanoate. (One patient received olanzapine in addition to depot therapy.) Overall, 31 of the 272 patients (11 percent) received a depot prescription during the study period.

Benzodiazepines. Of the 353 patients with schizophrenia in the study, 206 (58 percent) had a secondary diagnosis of drug or alcohol abuse. Of these patients, 76 (22 percent) had alcohol abuse alone, 34 of whom (45 percent) received a prescription for a benzodiazepine. Thirty-three (9 percent) had drug abuse alone, and 17 of them (52 percent) received a benzodiazepine. Ninety-seven patients (27 percent) had diagnoses of both drug and alcohol abuse, and 47 of them (48 percent) received a benzodiazepine.

Of the 147 patients with no history of drug or alcohol abuse, 79 (54 percent) were prescribed a benzodiazepine. Overall, 50 percent (177 of 353 patients) received at least one benzodiazepine prescription over the four-year study period.

Patients' characteristics and guideline conformance

Adherence to the dosage guideline was associated with the type of antipsychotic regimen prescribed (χ²= 19.165, df=1, two-tailed p<.001). Adherence to the dosage guideline was independent of race and sex. We observed a negative correlation between adherence to the dosage guideline and age (Spearman r=−.221, p=.001). Spearman correlations with distance of residence from the medical center and years in the study were not significant.

Discussion

In this investigation, we hoped to determine whether measures derived from a computerized clinical patient database could be used to help assess conformance to evidence-based guidelines. We were successful in evaluating guideline adherence, in addition to other measures, using simple, straightforward database queries. Therefore, generating these measures required minimal resources, in contrast to more costly manual chart reviews.

However, using this approach, we were able to address guideline conformance for only three of the 18 PORT recommendations on pharmacotherapy, and measurements of adherence to even these recommendations were imprecise. The limitations of computer-based studies and the ambiguities inherent in analyses of administrative data were readily apparent, for example, when measuring adherence to PORT recommendation 2 on dosage. We found that a significant number of patients treated for schizophrenia received less than the recommended chlorpromazine-equivalent dosage. Because our database does not contain fields describing the reasons for dosages outside this range, we could not differentiate between instances of improper dosing and cases in which dosages outside the range were medically appropriate.

We feel that these computer-based analyses are best used to efficiently identify potentially significant problem areas. Chart reviews should then be used to determine the full significance of the findings.

With regard to PORT recommendation 1, on first-line therapy, our analysis showed that only 15 of the 272 patients on antipsychotic pharmacotherapy (6 percent) received regimens that contained clozapine. This finding suggests that clozapine is not widely viewed as a first-line agent for treatment of schizophrenia in this medical center, which is in concordance with the guideline. In addition, we found that only 272 of 353 patients (77 percent) had filled a prescription for any antipsychotic medication within five days before and 42 days after the patient's most recent discharge for schizophrenia. This finding suggests that some of the remaining 81 patients might not have received any antipsychotic therapy. On the other hand, they may have filled an antipsychotic prescription outside the health system studied, received outpatient care from multiple centers, not filled their prescriptions, or refused pharmacotherapy.

To investigate these possibilities, we examined a sample of ten discharge summaries and found that five patients received their outpatient medications from other treatment facilities, three received instructions for having a depot medication administered as an outpatient but did not fill the prescription in our health care system, one left the hospital against medical advice without an outpatient antipsychotic prescription, and one refused antipsychotic therapy. In summary, five of the ten patients likely did not receive any antipsychotic medication, suggesting an area for quality improvement.

We found only moderate compliance with PORT recommendation 2 on dosage. The negative correlation observed between adherence to this recommendation and age suggests that some of the deviation from this recommendation may be due partly to providers' concerns about increased side effects of antipsychotic medications among elderly patients. Patients on atypical antipsychotics were less likely than those on conventional therapy to receive a chlorpromazine-equivalent dosage below the recommended range. Patients who receive atypical therapy may be more likely to have disabling disease and to require additional monitoring and drug dosing. In addition, newer antipsychotics are less likely to generate dose-limiting side effects, perhaps allowing providers to more easily prescribe within the recommended range.

Using longitudinal prescription data to study conformance to PORT recommendation 13, on clozapine trials, we found that of 20 patients started on clozapine therapy, 18 (90 percent) received dosages greater than or equal to the minimum recommended chlorpromazine-equivalent dosage in their most recent regimen of conventional antipsychotic medication. In fact, before clozapine therapy was initiated, six of the 20 patients (30 percent) were prescribed dosages of conventional antipsychotics that were greater than or equal to the maximal chlorpromazine-equivalent dosage. This finding suggests that patients were receiving adequate dosages of antipsychotics before the change to clozapine.

In contrast, we found that a substantial number of patients started on atypical antipsychotic therapy were not receiving maximal chlorpromazine-equivalent dosages in their previous regimen of conventional antipsychotic medication. However, these lower dosages do not necessarily imply substandard patient care. Instead, the data might reflect patients' intolerance of the side effects of conventional agents (extrapyramidal syndrome), which may have necessitated the change to an atypical agent.

This study has a number of limitations besides those outlined above. First, we were limited to analyzing data contained in the clinical database, prohibiting us from measuring adherence to many other PORT guidelines, such as psychological treatments. Second, our study population was predominantly male, and all patients had recent hospitalizations, limiting the generalizability of our results. Finally, as discussed by the schizophrenia PORT researchers, demonstrating conformance to guidelines does not necessarily ensure that better outcomes and quality of care have been achieved. Additional research is needed in this area.

This study documented the feasibility of using computerized pharmacy data to identify specific patterns of schizophrenia management and measure conformance to a subset of the PORT treatment recommendations. We also identified important limitations of these methods and the need for complementary data-gathering strategies. However, we hope that these cost-effective approaches will prove successful in identifying potentially problematic areas for quality improvement.

Acknowledgments

This work was supported by grants T15-LM-07056 and G08-LM-05583 from the National Library of Medicine of the National Institutes of Health and by the office of the chief information officer of VA New England. The authors thank Matthew Chinman, Ph.D., for assistance with the statistical analyses.

Dr. Chen, Dr. Nadkarni, and Dr. Miller are affiliated with the Center for Medical Informatics at the Yale University School of Medicine in New Haven, Connecticut. Mr. Levin is with Evergreen Design in Guilford, Connecticut. Dr. Erdos and Dr. Rosenheck are with the department of psychiatry at Yale University School of Medicine. Dr. Rosenheck is also with the Veterans Affairs Connecticut-Massachusetts Mental Illness Research, Education, and Clinical Center in West Haven. Send correspondence to Dr. Rosenheck at the Northeast Program Evaluation Center, 950 Campbell Avenue (116A), West Haven, Connecticut 06516 (e-mail, [email protected]).