Development and Delivery of a Quality Improvement Program to Reduce Antipsychotic Polytherapy

BACKGROUND
Although antipsychotic polytherapy is considered appropriate in limited circumstances (e.g., during a brief "cross-titration" period when switching medications), its increasing prevalence indicates use beyond this limited scope. Despite absence of support in the medical literature and higher costs, antipsychotic polytherapy is common in the treatment of schizophrenia and related disorders. The highest utilization of antipsychotic polytherapy occurs on psychiatric inpatient units, and in 2008, the Joint Commission released the first set of 7 hospital-based inpatient psychiatric services (HBIPS) core measures, 2 of which assess antipsychotic polytherapy at time of discharge.


OBJECTIVE
To describe the effect on antipsychotic polytherapy at time of discharge of a 2-part quality improvement program composed of educational seminars and prescriber-specific feedback provided to 11 psychiatrists in 4 acute inpatient psychiatric units in 2 hospitals.


METHODS
In a regional academic health care system, we determined the prevalence of antipsychotic monotherapy and polytherapy at time of discharge for all patients discharged on standing antipsychotic medications during 3 periods: (a) a 3-month baseline period (August 2006 through October 2006); (b) in July 2007, after delivery of 4 educational luncheon seminars to 11 psychiatrists from November 2006 through June 2007; and (c) in June 2008, following the provision of monthly prescriber-specific audit feedback from August 2007 through June 2008. To prepare nurses for the change and address possible safety concerns, an educational module was delivered to the psychiatric nursing staff at "best practice" day lectures held in the first quarter of 2007. General themes in the educational presentations included literature-based reviews of (a) safety and efficacy of antipsychotic polytherapy, (b) medical risks of antipsychotic medications, (c) specific versus nonspecific effects of these medications, and (d) effectiveness of first- versus second-generation antipsychotic medications. The prescriber-specific audit feedback was provided in paper form and masked the identity of the other prescribers. The chief of service reviewed audit feedback individually with each psychiatrist on a quarterly basis. The primary outcome measure was the percentage of patients prescribed 2 or more antipsychotics at discharge. A secondary outcome measure was the percentage of patients prescribed 3 or more antipsychotics at discharge. Differences in the primary outcome measure, comparing (a) July 2007 with the baseline period and (b) June 2008 with July 2007, were analyzed using Fisher's Exact tests. The Cochran-Armitage test for trend was used to assess the relationship between the primary outcome measure and the extent of the intervention, measured as the 3 time periods. For the secondary outcome measure, the Goodman-Kruskal gamma test for ordered categorical data was calculated to examine the association between the the proportion of patients receiving 1, 2, or 3 or more antipsychotics at discharge and the 3 time periods.


RESULTS
The percentage of patients prescribed 2 or more antipsychotics at discharge declined from 33.9% at baseline (132 of 389 patients), to 21.8% after delivery of the educational modules (44 of 202 patients, P = 0.002), and to 12.2% after audit feedback (18 of 147 patients, P = 0.023; Cochran-Armitage test for trend P < 0.001). When antipsychotic use was classified as 1, 2, or 3 or more antipsychotic medications, more extensive intervention was associated with decreased combination use (Goodman- Kruskal gamma = 0.39, P < 0.001). In the baseline period, 5.9% of patients were prescribed 3 or more antipsychotics at discharge. Following completion of the educational and audit components, respectively, the proportion of patients prescribed 3 or more antipsychotics declined to 2.5% and then to 0.0%.


CONCLUSION
Educational modules presented to psychiatrists and nurses in group settings were associated with a decrease in the rate of prescribing 2 or more antipsychotics at discharge from acute psychiatric inpatient units. Addition of monthly audit feedback provided to psychiatrists was associated with further decreases.

OBJECTIVE: To describe the effect on antipsychotic polytherapy at time of discharge of a 2-part quality improvement program composed of educational seminars and prescriber-specific feedback provided to 11 psychiatrists in 4 acute inpatient psychiatric units in 2 hospitals.
METHODS: In a regional academic health care system, we determined the prevalence of antipsychotic monotherapy and polytherapy at time of discharge for all patients discharged on standing antipsychotic medications during 3 periods: To prepare nurses for the change and address possible safety concerns, an educational module was delivered to the psychiatric nursing staff at "best practice" day lectures held in the first quarter of 2007. General themes in the educational presentations included literature-based reviews of (a) safety and efficacy of antipsychotic polytherapy, (b) medical risks of antipsychotic medications, (c) specific versus nonspecific effects of these medications, and (d) effectiveness of first-versus second-generation antipsychotic medications. The prescriber-specific audit feedback was provided in paper form and masked the identity of the other prescribers. The chief of service reviewed audit feedback individually with each psychiatrist on a quarterly basis. The primary outcome measure was the percentage of patients prescribed 2 or more antipsychotics at discharge. A secondary outcome measure was the percentage of patients prescribed 3 or more antipsychotics at discharge. Differences in the primary outcome measure, comparing (a) July 2007 with the baseline period and (b) June 2008 with July 2007, were analyzed using Fisher's Exact tests. The Cochran-Armitage test for trend was used to assess the relationship between the primary outcome measure and the extent of the intervention, measured as the 3 time periods. For the secondary outcome measure, the Goodman-Kruskal gamma test for ordered categorical data was calculated to examine the association between the the proportion of patients receiving 1, 2, or 3 or more antipsychotics at discharge and the 3 time periods.
RESULTS: The percentage of patients prescribed 2 or more antipsychotics at discharge declined from 33.9% at baseline (132 of 389 patients), to 21.8% after delivery of the educational modules (44 of 202 patients, P = 0.002), and to 12.2% after audit feedback (18 of 147 patients, P = 0.023; Cochran-Armitage test for trend P < 0.001). When antipsychotic use was classified as 1, 2, or 3 or more antipsychotic medications, more extensive intervention was associated with decreased combination use (Goodman-Kruskal gamma = 0.39, P < 0.001). In the baseline period, 5.9% of patients were prescribed 3 or more antipsychotics at discharge. Following completion of the educational and audit components, respectively, the proportion of patients prescribed 3 or more antipsychotics declined to 2.5% and then to 0.0%.
prescribed in up to 40% of psychiatric inpatients. [19][20][21] However, no antipsychotic is approved for adjunctive use with other antipsychotic agents, and such practice is discouraged in treatment guidelines. [22][23][24][25] Not all antipsychotic polytherapy is unjustified. For example, treatment guidelines suggest that cross-titration of antipsychotics is preferable to abrupt discontinuation of a medication when switching from one drug to another. [23][24][25] In addition, treatment guidelines suggest antipsychotic polytherapy for patients who are treatment resistant and have failed or are ineligible for a clozapine trial. [23][24][25] The guidelines caution that such use is not evidence-based, should be monitored closely, and discontinued if ineffective. [23][24][25] One of the most commonly cited justifications for antipsychotic polytherapy is treatment resistance. 26,27 However, data suggest that antipsychotic polytherapy is not being reserved for treatment-resistant populations. Lee and Walker (2008) reported that in patients continuously enrolled in the California Medicaid program for at least 3 months, 3.7% of new users of second-generation antipsychotics received antipsychotic polytherapy as their initial antipsychotic treatment. 28 Schumacher et al. (2003) found that 79 of 85 patients (93%) on antipsychotic polytherapy did not have an adequate trial of monotherapy (4-9 weeks of optimal antipsychotic monotherapy dose). 27 Even if we presumed that all patients on antipsychotic polytherapy were truly treatment resistant, Schumacher et al. found that 97.6% of patients did not receive a trial of clozapine, the only antipsychotic proven effective for treatment-resistant schizophrenia.
High prevalence and lack of evidence about safety and efficacy have produced concerns about antipsychotic polytherapy. 29,30 Heeding these concerns, the Joint Commission in October 2008 released the first set of hospital-based inpatient psychiatric services (HBIPS) core measures, including 2 measures of antipsychotic polytherapy at discharge ( Figure  1). 30 Patients discharged without antipsychotic medications represent a different population than those discharged with antipsychotic medications and are not included in the antipsychotic polytherapy core measure.
In anticipation of the release of the Joint Commission's antipsychotic polytherapy core measure, the study authors designed a quality improvement project to address antipsychotic polytherapy. In alignment with the Joint Commission's core measure, the study's primary outcome was use of antipsychotic polytherapy at discharge. Since the project began during development of the HBIPS core measure, initial data collection was based on early drafts of the measure, which did not include a measure of appropriate justification of antipsychotic polytherapy. Therefore, data on justification of antipsychotic polytherapy were not collected during the earlier portion of our intervention. Once the core measure criteria expanded to include appropriate justification of antipsychotic polytherapy, we began to collect these data.
A ntipsychotic monotherapy with first-and secondgeneration antipsychotics has demonstrated efficacy in the treatment of schizophrenia and other psychotic disorders. 1,2 However, 20%-35% of patients fail to respond to or have incomplete response to antipsychotic monotherapy. 1,2 For such treatment-resistant patients, clozapine is the only treatment proven effective. 1-3 However, clozapine has some particularly dangerous side effects and requires regular monitoring of white blood cell count, limiting its use. As a consequence of this problem, clinicians and patients struggle to find an effective medication regimen. One particularly prevalent medication regimen is concomitant prescription of 2 or more standing antipsychotics (antipsychotic polytherapy).
Antipsychotic polytherapy has not been consistently proven effective or safe for patients with no or partial response to antipsychotic monotherapy. [4][5][6] In the majority of randomized controlled efficacy trials, antipsychotic polytherapy did not significantly differ from monotherapy on primary outcome measures. [7][8][9][10][11][12] Of the randomized controlled efficacy trials that found differences between groups on primary outcome measures, 2 studies reported antipsychotic polytherapy superior, 11,12 and 1 reported antipsychotic monotherapy more effective. 7 Effects on secondary outcome measures were mixed, but benefits gained with polytherapy were often, although not always, small. Results of analyses of side effects with polytherapy are mixed. 7,9,[11][12][13][14][15] Adherence is decreased with antipsychotic polytherapy, 16 and the additional medication costs associated with antipsychotic polytherapy do not appear to be offset by cost savings in other areas of treatment. 17 Despite these mixed data, the use of antipsychotic polytherapy is common. [18][19][20] Within the California Medicaid population in 2004, an estimated 13.7% of outpatients with schizophrenia who were treated with antipsychotics received multiple second-generation antipsychotics. 18 Rates are even higher during psychiatric inpatient stays, with antipsychotic polytherapy • This quality improvement program can serve as a potential model for other inpatient facilities trying to meet the Joint Commission's antipsychotic polytherapy HBIPS core measure. • At baseline, 5.9% of patients discharged from an acute inpatient psychiatric stay on antipsychotic medication received discharge prescriptions for 3 or more antipsychotics. After the provision of education to psychiatrists and nurses, that rate declined to 2.5%. The rate further declined to 0.0% following the delivery of feedback from 11 monthly audits to psychiatrists. • Of 389 patients discharged on antipsychotics at baseline, 132 (33.9%) were prescribed 2 or more antipsychotics. After completion of the educational and audit feedback components of the program, 12.2% (18 of 147 patients) were prescribed 2 or more antipsychotics at discharge (P < 0.001).

What this study adds
of antipsychotic medications prior to initiation of the quality improvement program.

Educational Component
Reports of educational efforts to improve physician adherence with evidence-based medicine within and outside of psychiatry have been mixed. [31][32][33][34] However, an educational program was considered by the study authors to be crucial to this quality improvement program for several reasons, including the need to (a) open a dialogue regarding goals and concerns with the quality improvement program, (b) address the evidence base supporting the quality improvement program, (c) obtain clinician "buy-in" for the program, and (d) allow nurses and psychiatrists access to the same information regarding the purpose and methods of the quality improvement program. A pharmacist who is board certified in psychiatric pharmacy practice worked as part of the interdisciplinary team to develop and deliver the education. Educational modules were developed using a disciplinespecific approach. Specific learning objectives are presented in Table 1. The primary educational goal was to ensure realistic expectations of what an antipsychotic could do and in what time frame. The secondary educational goal was to initiate discussion of the risks of antipsychotic medications, especially with antipsychotic polytherapy. Discussions centered on risks and benefits of antipsychotic monotherapy and polytherapy, relative risks and benefits of first-and second-generation ■■ Methods All data were gathered as part of a quality improvement project. As such, our local institutional review board deemed publication of research using de-indentified patient and psychiatrist data exempt from institutional review board approval. To ensure confidentiality during the project, all data analyses were conducted and reported for internal use with de-identified patient data.

Setting
At the time of the quality improvement project, Cambridge Health Alliance (CHA), a regional academic health care system, consisted of 3 inpatient hospitals, more than 20 outpatient clinics, a Medicaid health maintenance organization (HMO), and the Cambridge Public Health Department. As a safety net hospital system, CHA serves a racially and ethnically diverse population. Approximately 50% of CHA patients are uninsured or enrolled in Medicaid or Commonwealth Care, the state health insurance program.
During implementation of the quality improvement program, CHA had 4 acute inpatient adult psychiatric units housed within 2 hospitals (2 units at each hospital). Each adult inpatient psychiatric unit cares for approximately 40 patients daily. One hospital follows an academic model, with residents, students, and interns involved in patient care. The other hospital follows a community model without trainee involvement. There were no prescribing guidelines for the use Development and Delivery of a Quality Improvement Program to Reduce Antipsychotic Polytherapy  antipsychotics, and use of antipsychotics for acute agitation versus primary psychosis. All 11 inpatient psychiatrists met as a group 4 times during the 9-month intervention period. The initial meetings were in November and December 2006. The remaining 2 seminars were conducted quarterly during the first and second quarters of 2007. The meeting site was varied in order to decrease travel burden on any one group of psychiatrists. Core information was presented during lunch by the psychiatric pharmacist, chief of adult psychiatry, and local psychiatry directors from each hospital. An interactive seminar format was used to present information and allow psychiatrists to provide input and express their concerns about antipsychotic polytherapy in the clinical setting. To engage psychiatrists in discussion, the seminars included discussions of recent patients who were particularly complex and challenging.
Nursing staff were included for 2 reasons: (a) to prepare nursing staff for possible changes in prescribing practices, and (b) psychiatrists' prescribing can be influenced by nursing input, specifically fears about violence on the unit. Given the goal of preparing nursing staff for possible prescribing changes, similar educational content was provided to nurses and psychiatrists. Education for nursing staff was conducted during 2007 Q1 through a series of lectures delivered by the psychiatric pharmacist during mandatory nursing "best practice" days. The format was a 45-minute lecture with a 15-minute question and answer period. A single lecture was delivered 7 times to allow maximal nursing participation. Supplemental written materials were provided. The 10% of psychiatric nurses unable to attend the lecture due to vacation or sick time received a printed copy of the lecture and contact information for the psychiatric pharmacist.

Audit Feedback
Because of the extensive literature on audit feedback as a method of quality improvement, 31 the educational component was combined with prescribing feedback to improve the likelihood of success. Psychiatrists were provided with monthly "dashboard" (visual presentation of prescribing data) reports of their antipsychotic prescribing from August 2007 to June 2008 ( Figure 2). They were also provided with de-identified peer data for comparison. Data presented in the monthly dashboard included the number of patients treated, number of patients treated with antipsychotics, number of patients treated with monotherapy or polytherapy, patient diagnoses, and specific antipsychotic combinations used. Psychiatrists met individually with the chief of service quarterly to review their antipsychotic prescribing dashboard, discuss concerns, and offer suggestions for improvements to the antipsychotic polytherapy project.

Outcome Measures
In alignment with the Joint Commission HBIPS-5 core measure, the primary outcome measure was the number of patients discharged on 2 or more standing antipsychotic medications divided by the number of patients discharged on at least 1 antipsychotic. 30 Given the potential for increased risks and paucity of data on use of 3 or more antipsychotics, we believed this practice could never be justifiable. Therefore, the secondary outcome measure was prescription of 3 or more standing antipsychotics at discharge. Information on patients discharged on no antipsychotic was not reviewed because such patients represent a different population with disorders not responsive to antipsychotic treatment.

Sample and Data Collection Process
All patients on regularly scheduled antipsychotic medication for any indication at time of discharge from the 4 acute adult inpatient units were included. Baseline data were collected prior to any discussion of antipsychotic polytherapy usage (August through October 2006). Data following the educational component and the monthly audit component were collected in July 2007 and June 2008, respectively. Appropriate justification data were collected from November 2007 until the conclusion of the program in June 2008. Collection of these data coincided with inclusion of appropriate justification of antipsychotic polytherapy criteria into drafts of the HBIPS core measure.
Data were collected from the Meditech Health Care Information System, the electronic database used in CHA's hospital system. Data collected included number of patients treated, standing antipsychotic medications at the time of discharge, and diagnoses. All cases of antipsychotic polytherapy were confirmed by a chart review conducted by the chief of adult psychiatry or chief of psychiatric quality improvement. Justification of antipsychotic polytherapy was confirmed through chart review by the chief of adult psychiatry.

Data Analysis
A comparison of the use of antipsychotic polytherapy in the 2 hospital settings prior to education was performed using a Pearson's chi-square test of independence. As there were no significant differences between usage in these settings, data Development and Delivery of a Quality Improvement Program to Reduce Antipsychotic Polytherapy  2). Patient case mix and average length of stay did not differ between baseline and active intervention time periods (data not shown).
The frequency of antipsychotic polytherapy decreased significantly after the educational and audit feedback components of the intervention (Pearson chi-square = 28.81, df = 2, P < 0.001; Figure 3). In July 2007, after the educational modules had been delivered, of 202 patients prescribed antipsychotics at discharge, 44 (21.8%) were prescribed 2 or more antipsychotics. This rate represented an absolute reduction of 12.1% compared with baseline (Fisher's Exact test, P = 0.002). Following the audit intervention, there was a further reduction to an antipsychotic polytherapy rate of 12.2% (18 of 147 patients), representing an absolute reduction of 9.6% (Fisher's Exact test, P = 0.023). The Cochran-Armitage test for trend showed a significant (Pearson chi-square = 28.69, df = 1, P < 0.001) linear decrease in the proportion of patients who were treated with 2 or more antipsychotic drugs over the 3 time periods (i.e., using the progressive education to audit strategy).
In July 2007, 5 patients (2.5%) had 3 or more antipsychotics prescribed at discharge; that number declined to 0 in June 2008. Combinations involving clozapine accounted for less than 1% of antipsychotic polytherapy at all time points. When antipsychotic usage was classified as the frequency of patients receiving 1, 2, or 3 or more antipsychotic medications, the incremental intervention (i.e., 3 time periods) was associated with decreased combination use (Goodman-Kruskal gamma = 0.39, P < 0.001).
Justification of antipsychotic polytherapy data were collected from November 2007 until conclusion of the quality improvement project. Rates of appropriate justification ranged from 21% to 100%.

■■ Discussion
Our findings of decreased antipsychotic polytherapy during a quality improvement program are consistent with those of other reports in the literature. Similar to our report, successful interventions typically include multifaceted approaches. [32][33][34] The present study is the first, to our knowledge, to find an association between the combination of discipline-specific group education for nursing and psychiatry staff with audit feedback and reduced rates of antipsychotic polytherapy.
Other reports address interventions to decrease antipsychotic polytherapy with different quality improvement programs. Mason et al. (1978) first reported more than 30 years ago on attempts to reduce antipsychotic polytherapy in state psychiatric hospitals. 35 The authors reported that peer review, small group educational sessions, audit feedback, and written educational materials on basic principles of antipsychotic prescribing led to a rate of antipsychotic polytherapy of 152 of 802 (19.0%) at follow-up compared with 357 of 1,190 (30.0%) at baseline.
were combined across settings. Fisher's Exact tests were performed on the primary outcome measure, receipt of 2 or more antipsychotics at discharge. These tests compared (a) July 2007 (the month following completion of the education) with August through October 2006 (the baseline period) and (b) June 2008 (the month following the completion of the monthly providerspecific feedback) with July 2007. A Pearson chi-square test of independence and a Cochran-Armitage test assessed the significance and trend of the relationship between the intervention and antipsychotic polytherapy, using a 2 X 3 table of the antipsychotic polytherapy measure (i.e., 1 vs. 2 or more antipsychotics at discharge over the 3 time periods-baseline, then education alone, then education plus monthly feedback). The strength of the association between the prescription of 1, 2, or 3 antipsychotics and increasing intervention (i.e., the 3 time periods) was assessed using a Goodman-Kruskal gamma test. Because no cells had less than 5 expected cases, continuity correction was not required for the Pearson chi-square tests. The a priori P value for statistical significance was 0.05. All statistical analyses were performed with Systat 12.0 (Systat Software Inc., Chicago, IL).

■■ Results
There were 389 patients prescribed at least 1 antipsychotic at discharge during the baseline period. Bipolar disorder was the most common diagnosis (37.0%) followed by schizophrenia/ schizoaffective disorder/schizophreniform disorder (32.9%), major depression (28.0%), other mood disorder with psychosis (2.1%), and other (3.1%). Patients could be diagnosed with more than 1 primary psychiatric disorder; hence, the total of reported diagnoses exceeds 100%. During the baseline period, 109 (28.0%) of patients were treated with 2 antipsychotics, and 23 (5.9%) were treated with 3 antipsychotics at discharge, for an overall antipsychotic polytherapy rate of 33.9% (Table   Development and Delivery of a Quality Improvement Program to Reduce Antipsychotic Polytherapy the modules during normal working shifts. This feature may have improved the palatability of educational programs to psychiatric physicians and nursing staff. This difference may partially account for the present study's finding of an absolute 22% decrease in rates of antipsychotic polytherapy. The present study's authors noted a decrease in the most egregious form of antipsychotic polytherapy (3 or more antipsychotics) after the introduction of educational programming only. Therefore, group education may be useful in decreasing the most egregious form of antipsychotic polytherapy. Study results, which represent academic and community hospitals, demonstrate that focused educational group sessions have the potential for broad application.

Limitations
First, the health care system in which the present study was conducted is a public health safety net hospital system. Additionally, 33% of sample patients were diagnosed with schizophrenia, schizoaffective disorder, or schizophreniform disorder, and 55% were diagnosed with bipolar disorder or major depression. As such, our population may not necessarily reflect other patient populations, including those in other facilities or with a different diagnostic profile. Second, this study reflects only a limited number of staff psychiatrists (n = 11). It is possible that other psychiatrists may be more resistant to prescribing change. However, the study investigators noted improvements in antipsychotic prescribing patterns in all 11 psychiatrists.
In another study, Chong et al. (2006) developed and implemented an evidence-based treatment algorithm for patients accepted into an early psychosis intervention program. 36 Introduction of the algorithm was associated with decreased antipsychotic polytherapy. However, the algorithm was for "first break" psychosis and may not be applicable to patients who have been ill for many years.
In the previous research most similar to the present study, Thompson et al. (2008) reported a decrease in antipsychotic polytherapy in a randomized controlled trial involving 19 psychiatric units that included a 3-part intervention of academic detailing, chart-based reminders, and an educational workbook documenting alternatives to polytherapy, including cognitive behavioral techniques. 32 Cognitive challenges to polypharmacy prescribing were identified from the rationales used by clinicians to justify polypharmacy, and alternatives to polypharmacy were reviewed in workbooks provided to nurses and physicians. Initial rates of polytherapy were 71 of 204 (34.8%) and 130 of 270 (48.1%) patients on the control and intervention units, respectively. Upon completion of the interventions, polytherapy was prescribed in 92 of 220 (41.8%) and 104 of 260 (40.0%) patients on the control and intervention units, respectively. However, given the modest gains and labor-intensive efforts, the authors recommended caution when interpreting their results. In contrast, the process described in the present study was less labor intensive. We did not rely on staff commitment of personal time for the educational program, providing Development and Delivery of a Quality Improvement Program to Reduce Antipsychotic Polytherapy

DISCLOSURES
There was no external funding for this quality improvement project or this research. Gören has served on advisory boards and speakers bureaus for Astra Zeneca and Eli Lilly and Company and has provided expert testimony for Morrison Mahoney, LLP, on the topics of drug pharmacokinetics, drug monitoring, and drug toxicity. Dufresne has served on an advisory board and speakers board for Eli Lilly and Company and Janssen Pharmaceutica. Mills has served on speakers boards for Janssen Pharmaceutica. The other authors reported no financial or other conflicts of interest related to the subject of this manuscript. Beck, Gören, Mills, and Shtasel were responsible for concept and design. Shtasel had primary responsibility for data collection with the assistance of Gören. Dufresne, Gören, and Shtasel had primary responsibility for interpreting the data. Gören had primary responsibility for writing and revising the manuscript; Dufresne assisted with the revisions.
Third, all units involved in the study were within 1 health care system. This factor may limit the generalizability of our findings. However, the study did demonstrate that group educational modules can be conducted despite practice sites being in more than 1 city or town. Although all educational meetings in the present study were conducted in person, use of teleconferencing could decrease the time commitment of psychiatry staff or allow inclusion of remote sites.
Fourth, because the psychiatrists who were the subject of the intervention were aware of the objective of the quality improvement initiative, they may have changed their prescribing behavior as a result of being observed and not as a direct result of either the educational or audit feedback interventions. However, the Hawthorne effect has been viewed as an acceptable and useful aspect of quality improvement initiatives. 37 Fifth, we did not measure whether patients in any time period also had observations in other periods. To the extent that this problem occurred, it would have violated the assumptions of the statistical tests used, which assume independent samples. However, persistence of educational effects into the audit feedback period for both new and repeat cases was an intended outcome of the program.

■■ Conclusion
Standardized group educational presentations were associated with decreases in concomitant use of 2 or more antipsychotics prescribed at discharge from an acute psychiatric inpatient stay. The addition of audit feedback was associated with an additional decrease in the concomitant use of 2 or more antipsychotics.