Global index of safety (GIS): a new instrument to assess drug safety.

Abstract

There is an asymmetry between the extraordinary development of measures and tools aimed at studying the beneficial effects of the drugs and the more limited methods to assess their safety profile. The goal of our study was to develop a global measuring tool to assess drugs' safety. We conducted a survey of Spanish psychiatrists in mental health centers and outpatient treatment units to assess the severity scores that they would assign to a list of the most common adverse events (AEs) that usually occur with antipsychotic treatment. The severity scores were then applied to the list of AEs that really occurred along a naturalistic pharmacoepidemiological study on the use of different antipsychotics in the treatment of schizophrenia. The Global Index of Safety (GIS) of the experimental group treated with olanzapine (OLZ) was compared with the GIS of the control group and with the GIS of specific antipsychotics for which the number of treated patients was greater than 100. A total of 194 psychiatrists rated the severity of each AE on a scale of 1 (insignificant) to 5 (extremely severe). The individual severity was applied to the 2949 schizophrenic patients included in a pharmacoepidemiological study. A GIS was calculated for every group of patients receiving the same treatments. The GIS of the control group was higher (4.3) than that calculated from the experimental group (2.5) (P < 0.001). The GIS of the risperidone (3.6) and haloperidol (6.0) subgroups were higher than that calculated from the OLZ group (P < 0.001). The development of a GIS may facilitate the comparison of the safety of several drugs and may constitute a very valuable aid for those involved in selecting drugs.

Full text

Page 1

Journal of Clinical Epidemiology 54 (2001) 1120–1125
0895-4356/01/$ – see front matter © 2001 Elsevier Science Inc. All rights reserved.
PII: S0895-4356(01)00384-5
Global index of safety (GIS): A new instrument to assess drug safety
José A. Sacristán*, Juan C. Gómez
Clinical Research Department, Lilly S.A., Madrid, Spain
b
Catalan Institute of Public Health, Barcelona, Spain
c
Center for Health Economics, University of York, York, UK
Received 16 October 2000; received in revised form 30 April 2001; accepted 2 May 2001
a,a
, Xavier Badía
b
, Paul Kind
c
a
Abstract
There is an asymmetry between the extraordinary development of measures and tools aimed at studying the beneficial effects of the
drugs and the more limited methods to assess their safety profile. The goal of our study was to develop a global measuring tool to assess
drugs’ safety. We conducted a survey of Spanish psychiatrists in mental health centers and outpatient treatment units to assess the sever-
ity scores that they would assign to a list of the most common adverse events (AEs) that usually occur with antipsychotic treatment. The
severity scores were then applied to the list of AEs that really occurred along a naturalistic pharmacoepidemiological study on the use of
different antipsychotics in the treatment of schizophrenia. The Global Index of Safety (GIS) of the experimental group treated with olan-
zapine (OLZ) was compared with the GIS of the control group and with the GIS of specific antipsychotics for which the number of
treated patients was greater than 100. A total of 194 psychiatrists rated the severity of each AE on a scale of 1 (insignificant) to 5 (ex-
tremely severe). The individual severity was applied to the 2949 schizophrenic patients included in a pharmacoepidemiological study. A
GIS was calculated for every group of patients receiving the same treatments. The GIS of the control group was higher (4.3) than that cal-
culated from the experimental group (2.5) (P
?
0.001). The GIS of the risperidone (3.6) and haloperidol (6.0) subgroups were higher than
that calculated from the OLZ group (P
?
0.001). The development of a GIS may facilitate the comparison of the safety of several drugs
and may constitute a very valuable aid for those involved in selecting drugs.© 2001 Elsevier Science Inc. All rights reserved.
Keywords:
Safety; Adverse events; Antipsychotics; Schizophrenia; Olanzapine; Risperidone; Haloperidol
1. Introduction
The drug selection process is based on the comparison of
the risk–benefit profile of the different options available.
Initially, efficacy and risk effects are assessed using ran-
domized clinical trials (RCTs) in which the new drugs are
compared with placebo or standard treatments. The limited
number of patients that can be included in RCTs, their ex-
perimental nature, and the relatively short follow-up period,
make it desirable to complete the information obtained in
RCTs with post-marketing data when drugs are used in ac-
tual practice.
Generally, the methods of studying adverse events are
less extensive than the methods used to assess the beneficial
effects of a drug. Although there are frequent controversies
on the pros and cons of using surrogate endpoints, or on the
appropriate temporal horizon to measure beneficial effects,
efficacy assessment presents two important advantages with
respect to safety evaluation: the high number and the so-
phistication of the measuring instruments, and the possibil-
ity of directly comparing identical outcomes (for example,
reduction of arterial blood pressure or decrease in mortality).
Spontaneous reporting systems remain the most common
way to detect rare adverse events (AEs) that are temporally
associated with drug use. However, these systems are not a
valid option when decisionmakers are interested in detect-
ing the true incidence of anticipated and common harmful
effects. To obtain the latter information, appropriate ana-
lytic studies such as case-control studies, cohort studies, or
postmarketing clinical trials need to be conducted [1]. But a
common problem of all analytic methods is that results are
often limited to a long list of the frequencies of occurrence
of all the AEs reported during the study. This line-item pro-
cess may assist in achieving an approximate understanding
of the safety profile of the drugs studied and may sometimes
be sufficient for deciding on the option with the most favor-
able profile. This is true in cases in which a particular drug
is associated with an AE that is much more serious than
those of other drugs, or if the frequency of one or more AEs
is clearly higher in one of the options. However, other times
the choice is not that obvious and the comparison of the
* Corresponding author. Clinical Research Department, Lilly S.A.,
Avda. de la Industria 30, 28108 Alcobendas, Madrid, Spain. Fax: 34-91-
6635231.
E-mail address:
Sacristan_jose@lilly.com (J.A. Sacristán)

Page 2

J.A. Sacristán et al. / Journal of Clinical Epidemiology 54 (2001) 1120–1125
1121
item-by-item AE does not allow for any conclusions to be
drawn.
The development of a global safety measuring tool,
which would represent a single weighted score for an over-
all AE profile, may facilitate the safety comparison of sev-
eral drugs and constitute a valuable aid for those involved in
selecting drugs. The objective of this study was to develop a
GIS in parallel to a recent naturalistic prospective pharma-
coepidemiological study with antipsychotics.
2. Materials and methods
The GIS was developed in the following steps: (1) Selec-
tion of AE; (2) Drafting a questionnaire and physician rat-
ing of the severity of each AE; (3) Application of AE sever-
ity scores to the pharmacoepidemiological study (EFESO)
and calculation of the GIS for each treatment group; and (4)
Comparison of the GIS of the different groups.
2.1. Selection of the adverse events
A list with the most common and severe AE that may oc-
cur in patients treated with antipsychotics was constructed.
AE selection took place by reviewing the AE reported in the
recent literature [2–5]. The initial list was completed by
searching the Eli Lilly and Company safety database for the
most common AE (frequency
comparable drugs, including those that occurred after com-
mercialization. Other known AEs of neuroleptics were also
included in our criteria. The AEs with similar terminology
were grouped according to the COSTART terminology [6].
?
1%) reported for OLZ and
2.2. Drafting of the questionnaire and obtaining the severity
scores for each AE
During the initial meetings for the start-up of a pharma-
coepidemiological study with antipsychotics, the purpose of
developing a new safety evaluation tool and its subsequent
application to the safety results of the study was explained
to the participating psychiatrists. In the GIS questionnaire
instructions, the psychiatrists were asked to assign, accord-
ing to their experience, scores to each of the AEs on the list,
rating the severity that each of the AEs would have in
schizophrenic patients treated with antipsychotics. The
scores ranged from 1–5 on the following assessment scale:
(1) Insignificant, (2) Mild, (3) Moderate, (4) Severe, and (5)
Extremely severe. This system for scoring the physician-
perceived severity of the AE has been reported previously
[7]. All questionnaires were administered by either JAS or
JCG and all participants received the same instructions. The
questionnaire included instructions for its completion.
Given that a particular AE could have different degrees of
severity in different patients, it was suggested that the score
should reflect a “standard” severity—in other words, the se-
verity they had most often observed in the schizophrenic pa-
tients they had treated.
All psychiatrists participating at the start-up meetings for
the study completed the questionnaire. Using the scores as-
signed, the severity of each AE was calculated (mean, stan-
dard deviation [SD], median, and 95% confidence intervals
[95% CI] for each AE). On closure of the database and be-
fore any statistical analysis, the list was completed by in-
cluding AEs detected during the pharmacoepidemiological
study that were not on the original list. For these AEs, se-
verity scores were assigned using the value of the AE most
similar to itself by means of a consensus meeting of the 5
investigators managing this component of the study.
2.3. Application of the AE severity scores to the EFESO and
calculation of the GIS for each treatment group
EFESO was an observational phase-IV prospective phar-
macoepidemiological study with an open comparison in
parallel groups to assess the safety of OLZ in comparison
with other antipsychotic drugs in the treatment of outpa-
tients with schizophrenia. Data were collected by a total of
293 psychiatrists, mostly in mental health centers or outpa-
tient treatment units. The main results of the study have re-
cently been published elsewhere [8]. To assess the main
goal, all AEs spontaneously reported by patients or identi-
fied in the course of the doctor–patient interview were re-
corded. An adverse event was defined as any undesirable
experience occurring after administration of the product. An
adverse event did not necessarily have a causal relationship
with the treatment. Extrapyramidal symptoms (EPS) were
collected by means of a short questionnaire, based on the
extrapyramidal symptoms section of the UKU scale: dysto-
nia, rigidity, hypokinesia, tremor, dyskinesia, and akathisia
[9]. All AEs were coded using the COSTART glossary.
Applying the AE previously obtained severity scores to
the EFESO AE results, the GIS was calculated for each anti-
psychotic. The first step was to calculate the safety index for
each patient by summing the severity scores for all of the re-
spective AEs in each patient. For example, the safety index
for a drug in a patient without any AEs would be 0, whereas
it would be 7.5 in a patient with 2 AEs with severity ratings
of 3.5 and 4.
The second step was to calculate the GIS for each drug;
this was determined by averaging the indexes calculated in
all patients in the respective drug treatment groups.
2.4. Comparison of the GIS of the different groups
The GIS of the experimental group treated with olanzap-
ine (OLZ) was compared with that of the control group (all
other antipsychotics). Moreover, the GIS for OLZ was com-
pared with that of specific antipsychotics for which the
number of treated patients was greater than 100. Two GIS
values were determined. The first included all patients of
the study, regardless of AE experience, and the second cal-
culation only included data from patients who experienced
at least 1 AE.
2.5. Statistical analysis
The EFESO study was designed to confirm the differ-
ences in the incidence of extrapyramidal symptoms between

Page 3

1122
J.A. Sacristán et al. / Journal of Clinical Epidemiology 54 (2001) 1120–1125
the OLZ group and the control group, with a 90% power
and a two-tailed
?
risk of 0.05, to which end a sample size
of approximately 1000 patients was estimated for each
group. In addition, the OLZ group contained twice the num-
ber of patients as the control group to detect, with an 80%
power and an
?
risk of .05, any adverse effects occurring
with a frequency of 1:1000 in the OLZ group. In short, to
meet our objectives the study was designed to gather infor-
mation on approximately 2000 patients treated with OLZ
and 1000 patients treated with other antipsychotic drugs [8].
All patients who received OLZ were included in the OLZ
group, and patients who did not receive OLZ were included
in the control group. Nevertheless, to test for differences be-
tween different drugs in the control group, it was decided a
priori to conduct a secondary analysis comparing the inci-
dence of AE in the OLZ group versus treatment-specific
subgroups with
n

?
100.
The data were keyed into two databases simultaneously
by different individuals and later compared to eliminate er-
rors. The system used for the verification, validation, and
analysis of the data was SAS
All patients with available information were included in the
analysis. The incidence of each AE in each group was cal-
culated for the number of patients presenting the event at
any time during the study for the total number of patients in
the group.
The incidence of AE was described by means of fre-
quency and percentage and compared with the
Fisher’s Exact test. The GISs were described using means,
medians, SDs, and ranges, and compared using single factor
analysis of variance (ANOVA). A two-tailed significance
level of 0.05 was applied for all tests.
®
version 6.12 for Windows.
?
2
test or
3. Results
Due to space restrictions, the list with the average sever-
ity assigned for each AE by the participating psychiatrists is
not included in this article (interested readers may request it
from the corresponding author). The highest perceived se-
verity scores corresponded to malignant neuroleptic syn-
drome (mean
?
SD, 4.81
?
0.76), agranulocytosis (4.64
0.88), and suicide attempt (4.45
sponded to an increase in coughing (1.93
(1.91
?
0.74), and thirst (1.88
In the EFESO study, 2128 (72,2%) patients were treated
with the experimental drug (OLZ) and 821 (28,8%) patients
were in the control group, treated as follows: risperidone
(RIS) 417 (14.1%), haloperidol (HAL) 112 (3.8%), sertin-
dole 84 (2.8%) zuclopenthixol 74 (2.5%), fluphenazine 33
(1.1%), trifluoperazine 31 (1.1%), and others 70 (2.3%).
Despite the observational nature of the study there were no
statistically significant differences between the two groups
in age, gender, time from onset, schizophrenia subtype, or
baseline severity [8]. The GIS of the OLZ group was com-
pared with the GIS of the control group. Additionally, the
GIS of the OLZ group was compared with the GIS of pa-
?
?
0.88). The lowest corre-
?
0.69), chills
0.71).
?
tients treated with RIS and HAL (antipsychotics for which
the number of treated patients was greater than 100).
A higher percentage of the patients in the control group
(64%) presented at least one adverse event at some time in
the study, a statistically significant difference (P
when compared with the OLZ group (48%). This percent-
age was also lower (P
?
0.001) in the OLZ group compared
with the HAL (80%) and RIS (57%) subgroups. In the same
way, a significantly lower (P
tients developed EPS in the OLZ group (37%) than in the
RIS (50%) and HAL (76%) subgroups. These results are
consistent with the profile shown in the registration clinical
trials [2–5] and are included in the product’s package insert
and in the only randomized, double-blind clinical trial that
has compared OLZ and RIS [10].
Table 1 compares the GIS for OLZ with the antipsy-
chotic control group as well as with RIS and HAL specifi-
cally, when all patients were included in the analysis. The
GIS calculated from the control group (4.3) was 72% higher
(worse score) than that calculated from the OLZ group (2.5)
(P
?
0.001). The GIS of the RIS (3.6) and HAL (6.0) sub-
groups were 44% and 140% higher than that calculated
from the OLZ group (P
?
0.001). Table 2 shows the results
of this same comparison when only including those patients
with at least one AE. In this case, the GIS of OLZ was also
significantly lower than that of the antipsychotics used in
the control group or with HAL and RIS specifically. In pa-
?
0.001)
?
0.001) percentage of pa-
Table 1
Global Index of Safety (GIS) of olanzapine (OLZ) vs. control group, and
OLZ vs. risperidone (RIS) and haloperidol (HAL). All-patients analysis
OLZ
N

?
a

1,896
Control
N

?
b

748
OLZ
N

?
c
1,896
RIS
N
?
383
HAL
N

?
104
Mean
Median
SD
Range
2.5
0
3.5
0–24.7
4.3
3
4.6
0–21.3
2.5
0
3.5
0–24.7
3.6
2.7
4.4
0–19.4
6.0
5.8
4.9
0–21.3
a
p
b
This group includes all patients in the control, including the RIS and
HAL subgroups.
c
p
?
0.001 versus RIS, and HAL (ANOVA).
?
0.001 versus control (ANOVA).
Table 2
Global Index of Safety (GIS) of olanzapine (OLZ) vs. control group, and
OLZ vs. risperidone (RIS) and haloperidol (HAL). Patients with at least
one adverse event
OLZ
N

?
a

903
Control
N
?
b

476
OLZ
N
?
c
903
RIS
N
?
218
HAL
N
?
83
Mean
Median
SD
Range
5.2
3.2
3.4
2–24.7
6.7
5.8
4.0
2–21.3
5.2
3.2
3.4
2–24.7
6.3
5.8
4.1
2–19.4
7.6
6.2
4.3
2.4–21.3
a
p
b
This group includes all patients in the control group, including the RIS
and HAL subgroups.
c
p
?
0.001 versus RIS, and HAL (ANOVA).
?
0.001 versus Control (ANOVA).

Page 4

J.A. Sacristán et al. / Journal of Clinical Epidemiology 54 (2001) 1120–1125
1123
tients who experienced at least one AE, HAL (7.6) and RIS
(6.3) had a higher a GIS than patients treated with OLZ (5.2).
Table 3 lists the AE reported in more than 1% of the
OLZ group or in the control group, with an indication of
events in which the incidence was statistically significant
between the groups. The control group was also associated
with a greater incidence of sexual disorders in males,
whereas OLZ was more frequently associated with somno-
lence and weight gain. There were no significant differences
in the incidence of anticholinergic symptoms (dry mouth,
constipation, diplopia, urinary retention, difficulties in con-
centration, and confusion). There were no cases of agranu-
locytosis in the study. A higher percentage of patients in the
control group (57,9%) received some kind of concomitant
treatment, in comparison with patients in the OLZ group
(36.3%; P
?
0.001). Specifically, a lower percentage of the
patients in the OLZ group received anticholinergic medica-
tion compared with the control group or the HAL and RIS
subgroups (at 6 months, OLZ 10.2% versus control group
26.8%, P
?
0.001; versus RIS 19.9%, P
sus HAL 44%, P
?
0.001).
Table 4 shows the AEs, classified and coded according
the COSTART dictionary, that were reported with a statisti-
cally significant difference frequency in the OLZ group
compared with the RIS and HAL subgroups.
?
0.001; and ver-
4. Discussion
The calculation of a GIS provides different information
than the conventional method of reporting the safety profile
of drugs in clinical trials and epidemiological studies. It al-
lows obtaining information on the relative severity assigned
by the researchers to different AEs. For example, in this
study, the researchers considered that weight gain and som-
nolence have a lower severity (2.6 and 2.42, respectively)
than the EPS most often reported with the other groups:
dystonia (3.21), hypokinesia (3.01), akathisia (3.05), tremor
Table 3
Incidence of adverse events reported with a frequency higher than 1%
OLZ
N
?
2,128
Control group
N
?
821
N
%
N
%
Akathisia
Dyskinesia
Dystonia
Extrapyramidal
syndrome
Hypertonia
Hypokinesia
Somnolence
Tremor
Weight gain
59
31
24
2.8
1.5
1.1
78
12
24
9.5
1.5
2.9
?
2

?
60.539,
p

?
0.001
?
2

?
11.927,
p

?
0.001
6 0.3
3.4
4.9
4.5
6.6
6.9
91.1
11.8
12.8
2.18
14.7
1.7
Fisher, 7.762,
?

?
76.665,
?

?
56.179,
?

?
9.577,
?

?
48.889,
?

?
30.690,
p
p
p

p
p



?
?
?
?

?

?
0.009
0.001
0.001
0.002
0.001
0.001
7397
2
104
96
140
146
105
17
121
14
2
2
p
2
2
Adverse events have been classified and coded according to the CO-
START dictionary
Table 4
Incidence of those adverse events that were reported with a significantly different frequency in the olanzapine group compared with the risperidone and
haloperidol subgroups, classified and coded according to the COSTART dictionary
Event
Olanzapine
N

?
2,128
n
(%)
Risperidone
N

?
417
n
(%)
Haloperidol
N

?
112
n
(%)
OLZ versus RIS
?
/
p
?2 ? 20.200
p ? 0.001
2
OLZ versus HAL
?
/
p
?2 ? 63.761
p ? 0.001
?2 ? 5.147
p ? 0.048
?2 ? 25.905
p ? 0.001
?2 ? 123.521
p ? 0.001
?2 ? 90.720
p ? 0.001
?2 ? 10.168
p ? 0.033
2
Akathisia59 (2.8)30 (7.2)19 (17)
Dystonia24 (1.1)9 (2.2)4 (3.6)
Extrapyramidal syndrome6 (0.3)3 (0.7)4 (3.6)
Hypertonia 73 (3.4)35 (8.4)29 (25.9)
?2 ? 21.134
p ? 0 .001
?2 ? 9.412
p ? 0.002
Hypokinesia104 (4.9)36 (8.6) 30 (26.8)
Hypotension4 (0.2)1 (0.2)2 (1.8)
Somnolence96 (4.5)7 (1.7)5 (4.5)
?2 ? 7.204
p ? 0.007
?2 ? 11.275
p ? 0.001
?2 ? 14.982
p ? 0.001
Tremor140 (6.6) 47 (11.3)29 (25.9)
?2 ? 56.900
p ? 0.001
?2 ? 6.180
p ? 0.013
Weight gain 146 (6.9)8 (1.9) 1 (0.9)
Males
Abnormal Ejaculation
N ? 1,349
0
N ? 274
2 (0.7)
N ? 68
0
?2 ? 9.859
p ? 0.028
?2 ? 9.859
p ? 0.028
Impotence 02 (0.73) 0
Females
Amenorrhea
N ? 769
5 (0.7)
N ? 142
5 (3.5)
N ? 44
0
?2 ? 9.100
p ? 0.011

Page 5

1124
J.A. Sacristán et al. / Journal of Clinical Epidemiology 54 (2001) 1120–1125
(2.74), or hypertonia (3.16). Also, the use of a global index
allows a safety comparison to be made between the differ-
ent groups, using the usual statistical techniques (Table 1
and Table 2). For example, it would be advisable in system-
atic application to pharmacological groups such as NSAIDs
in which the safety profile usually determines the optimal
option. Finally, the possible use of a GIS to compare drugs
from different therapeutic groups could prove to be attrac-
tive for decisionmakers.
A classical analysis of the safety profile is in many cases
sufficient to determine which of the options presents the
most favorable profile. In these cases, the use of a GIS
should be considered a complementary tool that may serve
to increase available information with the advantages noted
earlier. For instance, the comparisons made in Table 3 and
Table 4 may suggest that, as a whole, OLZ presents a more
favorable profile than the other treatment groups: statisti-
cally significant differences were detected in the incidence
of EPS in favor of OLZ compared with the other treatment
groups; sexual disorders in males were more frequent in the
control group, whereas OLZ was more frequently associ-
ated with somnolence and weight gain. However, the rela-
tive importance of each AE is not always clear; in the ab-
sence of a global safety measure, the selection of the safest
option may be a controversial decision. Obviously, the use
of a GIS derived from scores previously assigned by physi-
cians with experience in the treatment of a specific disease
may be a valuable tool to help decisionmakers select the
most favorable options.
In EFESO, a large proportion of patients from both the
OLZ and control groups received concomitant treatment
with antipsychotics, benzodiazepines, anticholinergics, or
other drugs. Therefore, it is difficult to attribute the safety
results unequivocally to OLZ or to other specific drugs. AEs
were assigned to groups reflecting the initially prescribed
drug in the study. This type of analysis may be appropriate
to reflect routine clinical practice where antipsychotics are
frequently used in combination.
The high degree of variability observed in the GIS values
(the SD is greater than the mean) could lead to a recommen-
dation to use the median values. In the case of the EFESO
study, the differences in safety between the different drugs
would be even more marked: the median of OLZ would be
0 (equivalent to an absence of AE), 3 (equivalent to a mod-
erate AE) for control antipsychotics, and 2.7 and 5.8 for RIS
and HAL, respectively.
Despite the advantages mentioned above, the proposed
method presents several limitations. First, the scores as-
signed a priori by a group of researchers (in this case psy-
chiatrists who work in the outpatient setting), may not re-
flect scores that would be assigned by other researchers. For
example, psychiatrists working in a hospital setting may as-
sign different severity scores, based on the different condi-
tions of the patients they usually treat. In such cases it
would be useful for each collective with different character-
istics to establish its own evaluation tool. In this example, it
is likely that hospitalized patients suffer a greater degree of
disease severity and may therefore require higher antipsy-
chotic doses.
Another drawback is that the same AE may have differ-
ent severity. For example, the EFESO researchers gave
vomiting a severity score of 2.65, but in a specific patient
the true severity of the vomiting may vary between a broad
range of mild to very severe, and these differences could be
related to the drugs administered. In this study these possi-
ble differences were not taken into consideration and each
AE was given a single severity score. It would be advisable
to perfect the method to adjust for the weight of each AE ac-
cording to its true severity.
Furthermore, the interpretation of the GIS is not exempt
from difficulties. The GIS of a drug may be high because
mild AEs are very common (for example, the high scores
shown by some patients in this study may be explained by
the simultaneous occurrence of several mild to moderate
AEs), or because severe AEs appear less often. Although
the clinical significance of both situations would be differ-
ent, the GIS provides an intuitive idea about the differences
in safety between different drugs, keeping in mind the fre-
quency and the severity at the same time. A potential way to
develop this assessment tool would be to allow researchers
to weight the scores on the scale according to their severity
(for example, a very severe AE could weigh 3 times more
than a severe one, and 5 times more than a moderate one).
Finally, this work presents the inherent limitations of ob-
servational studies. Although the OLZ and control groups
were not significantly different at baseline in the variables
analyzed [8], treatments were selected by the investigators
according to purely clinical criteria, so there is a clear po-
tential of bias in treatment assignment that could influence
the differences in the incidence of AEs between the groups.
There may also have been differences in how patients were
assessed or treated throughout the study, given the un-
blinded nature of the study. It is necessary to take into ac-
count that EFESO was only the vehicle to develop the GIS
methodology and that the instrument may be perfectly ap-
plied to other kind of designs such as randomized clinical
trials, database analyses, etc.
Several methods for drug decisionmaking, such as the
SOJA [11] or the Analytic Hierarchy Process [12] have
been developed in the recent years. However, the GIS is the
first methodological proposal to assess the safety profile of
drugs by using scores previously assigned by clinicians. Al-
though the validation of the instrument needs further inves-
tigation, in this study GIS results have been consistent with
the conclusions that could be drawn from the conventional
analysis of AE: GIS scores were highest in the control
group that had a higher incidence of severe neuroleptic ad-
verse events. It would be interesting to apply this instrument
to comparisons between groups that do not show such obvi-
ous differences as our analysis.
It would be desirable to compare the scores assigned by
clinicians with those assigned by patients to the same AE, or

Page 6

J.A. Sacristán et al. / Journal of Clinical Epidemiology 54 (2001) 1120–1125
1125
with the scores assigned by clinicians working in different
settings. To that end, our group conducted an interview to
obtain the AE scores assigned by researcher participants in
a pharmacoepidemiological study of schizophrenic hospi-
talized patients. The analysis of the ranking of the scores as-
signed to the same AE by the two groups of psychiatrists
(working in the ambulatory and hospital setting) may be a
first step in validation of the instrument. We plan to conduct
a test-retest of the instrument with a small group of clini-
cians to assess its reliability.
The use of GISs such as the one proposed in this study
may serve to achieve a better understanding of the benefi-
cial drug profile and be a great help in making decisions re-
garding their selection. The GIS is a new instrument that
must be perfected and should be considered as complemen-
tary tool to the current methods for evaluating drug safety.
Further development and validation will be useful in evalu-
ating the advantages and problems of the GIS.
Acknowledgments
We thank the psychiatrists that participated in the inter-
view, and Raul Vara and Fernando Gonzalo of the Biomet-
rics Department, Phoenix International Spain. This study
was supported by Lilly and was conducted under the R&D
Spanish Plan for the Pharmaceutical Industry (Farma II)
from the Spanish Ministries of Health and Industry.
References
[1] Brewer T, Colditz GA. Postmarketing surveillance and adverse drug
reactions. Current perspectives and future needs. JAMA 1999;281:
824–9.
[2] Beasley CM, Sanger TM, Satterlee W, Tollefson GD, Tran PV,
Hamilton SL. Olanzapine versus placebo results of a double-blind,
fixed-dose olanzapine trial. Psychopharmacol 1996;124:159–67.
[3] Beasley CM, Tollefson G, Tran P, Satterle W, Sanger T, Hamilton S.
Olanzapine versus placebo and haloperidol: Acute phase results of
the North American double-blind olanzapine trial. Neuropsychophar-
macol 1996;14:111–23.
[4] Beasley CM, Hamilton SH, Crawford AM, Dellva MA, Tollefson
GD, Tran PV, Blin O, Bevzen JN. Olanzapine versus haloperidol:
acute phase results of the international double-blind olanzapine trial.
Eur Neuropsychopharmacol 1997;7:125–37.
[5] Tollefson GD, Beasley CM Jr, Tran PV, Street JS, Tamura RN, Krue-
ger JA, Graffeo KA, Thieme ME. Olanzapine versus haloperidol in
the treatment of schizophrenia, schizoaffective and schizophreniform
disorders: results of an international collaborative trial. Am J Psychi-
atry 1997;154:157–65.
[6] COSTART. Coding symbol and thesaurus for adverse event termi-
nology. Rockville, MD: US Department of Health and Human Ser-
vices, 1990.
[7] Kind P, Dolan P, Martínez C. Adjusting outcomes for quality: scaling the
severity of adverse drug reactions. 13th Annual Meeting, International
Society Technology Assessment Health Care; Barcelona; May 1997.
[8] Gómez JC, Sacristán JA, Hernandez J, Breier A, Ruiz P, Antón C,
Fontova E. The safety of olanzapine compared to other antipsychotic
drugs: results of an observational prospective study in patients with
schizophrenia (EFESO study). J Clin Psychiatry 2000;61:335–43.
[9] Lingjaerde O, Ahlors UG, Bech P, Dencker SJ, Elgen K. The UKU
side effect rating scale. a new comprehensive rating scale for psycho-
tropic drugs and a cross-sectional study of side effects in neuroleptic-
treated patients. Acta Psychiatr Scand Suppl 1987;334:1–100.
[10] Tran PV, Hamilton SH, Kuntz AJ, Potvin JH, Andersen SW, Beasley
CM, Tollefson GD. Double-blind comparison of olanzapine versus
risperidone in the treatment of schizophrenia and other psychotic dis-
orders. J Clin Psychopharmacol 1997;17:407–18.
[11] Janknegt R, Steenhoek A. The system of objective judgement analy-
sis (SOJA). A tool in rationing drug selection for formulary inclusion.
Drugs 1997;53:550–62.
[12] Dolan JG. Medical decision making using the analytic hierarchy pro-
cess: choice of initial antimicrobial therapy for acute pyelonephritis.
Med Decis Making 1989;9:51–6.