pression),1monitoring disease symptoms,2monitoring be-
haviour (e.g., substance abuse),3conducting behavioural
counselling,4assessing adherence to medication regimens5
and increasing attendance at appointments. In a recent sys-
tematic review, our group comprehensively described the
populations, interventions and outcomes of clinical trials in-
volving interactive voice response systems.6Most of these
trials examined low-risk interventions and low-risk patient
populations. None of the studies examined an intervention
in which patients received instructions about their medicat-
ion therapy without human intervention.
Previous authors have reported a combined thromboembolic
and hemorrhagic event rate of 15% per year and a fatal hemor-
rhagic event rate of 1% per year among patients taking oral an-
ticoagulants.7–9Clinical event rates are typically minimal when
patients’ international normalized ratio are within the therapeu-
tic range,10but more than half of adverse events related to oral
anticoagulation are directly attributable to international normal-
ized ratio values outside the therapeutic range.11
More than half of the adverse drug events that affect am-
bulatory patients occur because of inadequate monitoring,12,13
making this aspect of care an attractive target for safety strate-
gies. Several interventions, including the use of automated
telephone messages14and academic detailing,15appear to im-
prove physician and patient compliance with laboratory mon-
itoring. Strategies to enhance monitoring could be improved
by targeting patients who take high-risk medications, such as
Many anticoagulation clinics already use computerized de-
cision support systems, because these systems are effective in
improving anticoagulation control.17–20However, even with the
help of a computerized decision support system, monitoring
patients is labour-intensive, because of the staff time required
to call patients to communicate medication instructions. Addi-
tional staff time is needed to follow up with patients who for-
get to go for scheduled blood tests. Therefore, we developed
an information technology-based solution to facilitate the
ealth care institutions are increasingly using inter-
active voice response systems. Applications of this
technology include screening for disease (e.g., de-
Natalie Oake MSc, Carl van Walraven MD MSc, Marc A. Rodger MD MSc, Alan J. Forster MD MSc
See related commentary by Gurwitz, page 909
Effect of an interactive voice response system on oral
From the Clinical Epidemiology Program (Oake, van Walraven, Rodger,
Forster), Ottawa Health Research Institute, The Ottawa Hospital; the
Department of Medicine (van Walraven, Rodger, Forster), University of
Ottawa, Ottawa, Ont.; and the Institute for Clinical Evaluative Sciences (van
Walraven), Toronto, Ont.
Background: Monitoring oral anticoagulants is logistically
challenging for both patients and medical staff. We evalu-
ated the effect of adding an interactive voice response sys-
tem to computerized decision support for oral anticoagu-
Methods: We developed an interactive voice response sys-
tem to communicate to patients the results of internation-
al normalized ratio testing and their dosage schedules for
anticoagulation therapy. The system also reminded pa-
tients of upcoming and missed appointments for blood
tests. We recruited patients whose anticoagulation control
was stable after at least 3 months of warfarin therapy. We
prospectively examined clinical data and outcomes for
these patients for an intervention period of at least
3 months. We also collected retrospective data for each
patient for the 3 months before study enrolment.
Results: We recruited 226 patients between Nov. 23, 2006,
and Aug. 1, 2007. The mean duration of the intervention
period (prospective data collection) was 4.2 months. Anti-
coagulation control was similar for the periods during and
preceding the intervention (mean time within the thera-
peutic range 80.3%, 95% confidence interval [CI] 77.5%
to 83.1% v. 79.9%, 95% CI 77.3% to 82.6%). The interac-
tive voice response system delivered 1211 (77.8%) of 1557
scheduled dosage messages, with no further input re-
quired from clinic staff. The most common reason for clinic
staff having to deliver the remaining messages (account-
ing for 143 [9.2%] of all messages) was an international
normalized ratio that was excessively high or low, (i.e., 0.5
or more outside the therapeutic range). When given the
option, 76.6% of patients (164/214) chose to continue with
the interactive voice response system for management of
their anticoagulation after the study was completed. The
system reduced staff workload for monitoring anticoagu-
lation therapy by 48 min/wk, a 33% reduction from the
baseline of 2.4 hours.
Interpretation: Interactive voice response systems have a po-
tential role in improving the monitoring of patients taking
oral anticoagulants. Further work is required to determine
the generalizability and cost-effectiveness of these results.
Une version française de ce résumé est disponible à l’adresse
CMAJ• APRIL 28, 2009 • 180(9)
© 2009 Canadian Medical Association or its licensors
monitoring of patients receiving oral anticoagulants. We were
uncertain whether our system could relay complex instructions
to patients in a manner that would be acceptable to the pa-
tients. In addition, the potential risk associated with using the
system was high, since patients would be receiving dosage in-
structions without human intervention for a medication with a
narrow therapeutic window. Therefore, we used a “proof-of-
concept” approach to evaluate the feasibility of adding an in-
teractive voice response system to a computerized decision
support system under optimal conditions.
Study design and setting
We conducted an observational study using a 1-group,
pretest–posttest design.21Data collection was retrospective for
the pre-intervention period and prospective for the interven-
tion period. The pre-intervention period (retrospective) was
the 3-month period before a patient’s enrolment in the study.
The intervention period (prospective) was the 3-month period
after study enrolment. We extended the prospective observa-
tion period for any patient who chose to continue with the in-
tervention beyond 3 months.
We conducted this study within the Oral Anticoagulation
Management Service of The Ottawa Hospital Thrombosis
Program in Ottawa, Canada. At the time of the study, this
service was monitoring about 1200 patients from eastern On-
tario who were receiving oral anticoagulation therapy. Pro-
gram staff used a computerized decision support system
(DawnAC, 4S Information Systems Ltd., Milnthorpe, Eng-
land) to assist in monitoring patients.
The study was approved by the Ottawa Hospital Research
Patients were potentially eligible for the study if they had
completed 3 months of oral anticoagulation therapy with war-
farin and their anticoagulation control was stable. We defined
“stable control” as 2 consecutive international normalized ratio
values within the therapeutic range during the month before
recruitment. We used stability of control as an eligibility cri-
terion because we wanted to evaluate the effectiveness of the
interactive voice response system under optimal conditions.
We did not exclude patients who had experienced previous
hemorrhagic or thromboembolic events. We did exclude pa-
tients whose anticoagulation control was unstable and those
who did not speak English, were receiving an oral anticoagu-
lant other than warfarin, had hearing problems, received calls
at a telephone number with an extension, self-managed their
warfarin dosage or had plans to stop being monitored by the
clinic. We recruited consecutive patients from the clinic who
met the inclusion criteria during the recruitment period.
We linked an interactive voice response system (Call-
AssureCDM, Vocantas Inc., Ottawa, Canada) to a computer-
ized decision support system and a telephone network to fa-
cilitate monitoring of oral anticoagulation therapy. Interactive
voice response systems allow a computer database to com-
municate with people via the telephone,22for example, by au-
tomatically calling to deliver and retrieve information. In our
case, health care professionals used the computerized deci-
sion support system to determine the new oral anticoagulant
dosage and timing of the next international normalized ratio
test. The interactive voice response system then communi-
cated this information to patients automatically.
The interactive voice response system communicated with
patients using 3 types of messages: dosage, reminder and
missed. The “dosage” message reported the patient’s latest in-
ternational normalized ratio, the weekly dosage schedule and
the date of the next appointment for testing of international
normalized ratio. This message also asked the patient if he or
she wanted to speak with someone from the clinic and if he or
she had started any new medications. If the patient responded
“yes” to either question, the interactive voice response system
notified a health care professional via email to follow-up with
the patient. The “reminder” message notified patients of up-
coming appointments for international normalized ratio test-
ing. We programmed the interactive voice response system to
deliver this message 2 days before the patient’s appointment.
The “missed” message notified patients who had missed an
appointment for testing of international normalized ratio and
asked them to go for testing the next day.
The interactive voice response system documented the de-
tails (e.g., date and time) of all calls made in a report that
could be accessed via a web-based interface. A health care
professional reviewed this report daily and contacted any pa-
tients for whom delivery of a dosage message had been un-
successful. We did not attempt to contact patients for whom
delivery of a reminder or missed message was unsuccessful.
Protocol for individual patients
We recruited eligible patients by telephone using a standard-
The intervention period for a particular patient started with
his or her first international normalized ratio test after provision
of consent. We followed each patient for a minimum of 3
months, during which time the patient had regular appoint-
ments for testing of international normalized ratio (Appendix 1,
available at www.cmaj.ca/cgi/content/full/180/9/927/DC2).
The laboratories forwarded international normalized ratio
results to the clinic, as usual. The pharmacist manually entered
each test result into the patient’s profile in the computerized
decision support system. The support system then recom-
mended a new oral dosage of anticoagulant and the date of the
next international normalized ratio test. The pharmacist re-
viewed and approved or changed these recommendations. The
interactive voice response system then called the patient to de-
liver a dosage message. If the system reached the patient, a
caregiver or an answering machine, the message was deliv-
ered. Otherwise, the system disconnected and attempted to
contact the patient later. We programmed the interactive voice
response system to make up to 3 attempts to contact each pa-
tient. At the end of the study, we contacted patients and used a
semistructured interview to elicit feedback about the interac-
tive voice response system. We gave patients the option of
CMAJ• APRIL 28, 2009 • 180(9)
continuing with the interactive voice response system or re-
turning to the clinic’s standard monitoring system. We ex-
tended the study observation period for those patients who
chose to continue with the interactive voice response system.
The primary outcome of the study was anticoagulation con-
trol.23We collected patients’ international normalized ratio
data from the clinic’s computerized decision support system
for both the pre-intervention and intervention periods.
We also evaluated the interactive voice response system us-
ing a health technology assessment framework that incorporated
outcome, process and structure indicators.24 Within this frame-
work, we measured 2 outcome indicators: the rate of hemor-
rhagic and thromboembolic events during the 2 study periods,
and patients’ satisfaction with the interactive voice response sys-
tem, defined as the proportion of eligible patients who continued
with the interactive voice response system after the study.
We used the clinic’s computerized decision support sys-
tem to identify hemorrhagic and thromboembolic events that
occurred during the pre-intervention and intervention periods.
We used semistructured interviews to assess patient satisfac-
tion. Also within the health technology assessment frame-
work, we measured 2 process indicators: the utility of the in-
teractive voice response system, defined as the proportion of
scheduled dosage messages that were successfully delivered
by the interactive voice response system and that did not re-
quire further input from clinic staff, and the change in work-
load of the clinic staff. We calculated the utility of the interac-
tive voice response system using data stored in the
Web-accessible report generated by the system. For the sec-
ond process indicator, we manually recorded and compared
the time required to monitor the interactive voice response
system and the time required to communicate with patients
using the standard method. To generate the structure indica-
tor, we described the setting and resources required to imple-
ment the interactive voice response system.
Data analyses and sample size calculation
We used 3 steps to determine the extent of anticoagulation
control. First, we used linear interpolation23to calculate inter-
national normalized ratio values for the days between actual
measurements. Second, for each patient, we calculated the
proportion of days, for his or her total observation period, on
which the international normalized ratio was within the thera-
peutic range. Third, we calculated the overall mean (i.e., the
mean of individual patient proportions) and 95% confidence
intervals (CIs). We applied this 3-step process to data for both
the pre-intervention and intervention periods. We then per-
formed a noninferiority test25,26to compare anticoagulation
control for the 2 study periods. We selected a margin of non-
inferiority of 5%, expressed as an absolute difference, based
on the smallest minimal important difference25reported by
published randomized controlled trials18,27–30in which antico-
agulation control was the primary outcome. We also con-
ducted subgroup analyses to investigate if anticoagulation
control during the intervention period differed according to
patients’ indication for oral anticoagulant use, sex, age, dur-
ation of oral anticoagulant use and satisfaction with the inter-
active voice response system.
We included 2 regression models in our analyses. We used
logistic regression31to investigate if patient factors were asso-
ciated with whether a patient continued with the interactive
voice response system. In addition, we used Poisson regres-
sion32to investigate whether the usefulness of the interactive
voice response system was related to age.
To estimate the time required to monitor patients using the
interactive voice response system, we measured the time re-
quired to perform each monitoring task over a 1-week period.
CMAJ• APRIL 28, 2009 • 180(9)
n = 355
Excluded n = 41
• Had discontinued warfarin n = 24
• Resided outside Ontario n = 8
• Had hearing problems n = 3
• Had unstable anticoagulation control n = 2
• Received instructions at a telephone number
with an extension n = 2
• Did not speak English n = 1
• Started self-management n = 1
Declined to participate n = 88
• Satisfied with existing system n = 36
• Preferred not to participate in research
n = 15
• Other or no reason given n = 37
n = 226
Dropped out of the study n = 21
• Instructions were confusing n = 13
• Instructions were too fast n = 6
• Missed personal contact n = 2
Excluded during the study n = 12
• Discontinued warfarin n = 8
• Stopped being monitored by the clinic n = 4
Chose not to continue with IVRS after the
study n = 29
Continued with IVRS
after the study
Lost to follow-up
n = 0
Included in analysis
n = 226
Figure 1: Flow diagram of patient recruitment, follow-up and
analysis. Note: IVRS = interactive voice response system.
The monitoring tasks included identifying patients whose
dosage messages were unsuccessful, contacting and receiving
calls from patients, and relaying information about patients to
the pharmacist. To estimate the time it would take staff to de-
liver the scheduled dosage messages, we determined the aver-
age time required to deliver 1 message, as follows. For 4 differ-
ent periods in 1 week, the clerk at the clinic recorded the exact
time required to successfully deliver information to 20 patients.
Because we had a single study group, we calculated the
sample size using the desired final CI, rather than a predeter-
mined effect size.33We selected a desired 95% CI around the
intervention estimate of anticoagulation control of 8%. We
also used data from a population-based study34to select an ex-
pected standard deviation estimate of anticoagulation control
in a population (30.1%). We determined that 226 patients
would be required for our study.
Patient characteristics and follow-up
We recruited patients between Nov. 23, 2006, and Aug. 1,
2007. We approached a total of 355 patients (Figure 1). We
excluded 41 patients because they did not meet the eligibility
criteria at the start of the study. Of the 314 eligible patients,
88 patients declined to participate, 15 (17%) of these because
they preferred not to participate in research. Therefore, we en-
rolled a total of 226 eligible patients in the study during the 9-
month recruitment period. About half of the study participants
were female (Table 1), and the median age was 58 (interquar-
tile range 48–68, range 21–88) years. By far, the most com-
mon indication for warfarin was venous thromboembolism
(179 patients [79.2%]). In total, 181 patients (80.0%) had
been taking warfarin for longer than 1 year.
The overall intervention period began on May 7, 2007, and
ended on Dec. 14, 2007. We followed patients prospectively
for a total of 942.2 months (78.5 years), with a mean follow-
up period of 4.2 months (standard deviation 1.8 months,
range 1 day to 7.2 months) (Table 1). We collected 3 months’
worth of retrospective data for each patient, for a total of 56.5
years. A total of 193 patients (85.4%) completed the 3-month
intervention period (Figure 1). Twenty-one patients (9.3%)
withdrew from the study because they found the automated
instructions confusing or too fast (n = 19) or they missed the
personal contact with clinic staff (n = 2). We excluded an ad-
ditional 12 patients (5.3%) because they discontinued war-
farin (n = 8) or stopped being monitored by the clinic (n = 4).
The primary outcome indicator was anticoagulation control,
expressed as proportion of time within the therapeutic range.
Anticoagulation control during the intervention period was
similar to that during the pre-intervention period. With the in-
teractive voice response system, international normalized ra-
tio values were within the therapeutic range a mean of 80.3%
of the time (95% CI 77.5% to 83.1%). In the pre-intervention
period, values were within the therapeutic range a mean of
79.9% of the time (95% CI 77.3% to 82.6%). The mean dif-
ference in anticoagulation control between the 2 periods was
0.36% (95% CI –2.95% to 3.67%). This difference was non-
inferior because the 95% CI of the mean difference included
zero and excluded the 5% margin of noninferiority.
According to our subgroup analyses, anticoagulation con-
trol during the intervention period did not differ significantly
by indication for oral anticoagulant use, sex, age, duration of
oral anticoagulant use or patients’ satisfaction with the inter-
active voice response system.
No hemorrhagic or thromboembolic events occurred dur-
ing the study period.
Most of the patients were satisfied with the interactive
voice response system. A total of 164 patients continued with
the interactive voice response system after the study (Table 2),
representing 76.6% of the 214 patients who were eligible to do
so. The most common reason for continuing with the interac-
tive voice response system, cited by 86 patients, was its clear
and timely delivery of information. Twenty-nine patients
(13.6%) did not continue with the system after the study. The
most common reasons for not continuing were missing the
personal contact with staff (n = 12) and finding the automated
instructions confusing or too fast (n = 12). Age was signifi-
cantly associated with the decision to continue using the sys-
tem. The likelihood of continuing use decreased with greater
age (odds ratio 0.96, 95% CI 0.93 to 0.99).
The interactive voice response system was useful for
communicating information to patients, as indicated by the
outcomes for dosage messages (Table 3). During the inter-
CMAJ• APRIL 28, 2009 • 180(9)
Table 1: Characteristics of the study population
No. (%) of patients*
n = 226
Age, yr, median (IQR)
Indication for use of oral anticoagulant
Mechanical heart valve
Target international normalized ratio
Duration of oral anticoagulant use, yr
Prospective follow-up, mo, mean (SD)
Total observation time, mo
Note: IQR = interquartile range, SD = standard deviation.
*Unless indicated otherwise.
vention period, we needed to deliver a total of 1557 dosage
messages. The interactive voice response system successfully
delivered 1211 (77.8%) of these without further input from
clinic staff. The remaining 346 messages (22.2%) required in-
put from clinic staff, most often because the patient’s interna-
tional normalized ratio was 0.5 or more outside the therapeu-
tic range (n = 143). The pharmacist contacted these patients to
identify possible explanations for the out-of-range values.
Overall, the system was unable to deliver 155 (10.0%) of the
dosage messages (Table 3). Our Poisson regression model re-
vealed a trend toward decreasing effectiveness of the interac-
tive voice response system with increasing age of the patients.
Clinic staff spent a total of 1.6 hours per week monitoring the
interactive voice response system. We found that it took 2 min-
utes and 50 seconds to deliver a single message in the absence of
the interactive voice response system and estimated that it would
have taken staff about 2.4 hours per week to deliver the sched-
uled messages. Therefore, we estimated that the system reduced
overall staff workload by about 48 minutes per week (33%).
We implemented the interactive voice response system in an
anticoagulation clinic. Physicians, nurses and pharmacists
work in the clinic and had experience using the computerized
decision support system before our study began.
The bulk of the work after implementation involved moni-
toring the system’s functioning. This required clerical training
and basic computing skills. We periodically required techni-
cal support from the manufacturer to trouble-shoot problems.
However, none of the problems necessitated turning off the
interactive voice response system.
In this proof-of-concept study, we evaluated the addition of
an interactive voice response system to a computerized deci-
sion support system to facilitate the management of oral anti-
coagulation therapy in patients whose anticoagulation control
was already stable. Anticoagulation control was similar with
and without the interactive voice response system. Most of
the patients were satisfied with the system. The system was
effective in communicating complex information, as indi-
cated by the high rate of successful delivery of messages
(77.8%) without input from staff. Importantly, the interactive
voice response system reduced the workload of clinic staff by
33%. However, these results may have limited generalizabil-
ity to unselected patients.
Anticoagulation control in our patient population was ex-
ceptionally good at baseline. In a systematic review35of stud-
ies evaluating anticoagulation control, we found that the aver-
age time within the therapeutic range was 56.7% in
community settings and 65.6% in anticoagulation clinics. In
contrast, the patients in this study spent 79.9% of the time
within the therapeutic range before the intervention, which
made it very unlikely that the intervention would result in a
significant improvement. Nonetheless, it was reassuring that
anticoagulation control did not decrease during the interven-
tion. In addition, the interactive voice response system re-
quired less work from staff members.
The absence of any hemorrhagic or thromboembolic
events in our cohort might be considered notable. However,
we feel that this result was unsurprising, for 3 reasons. First,
we followed a relatively small number of patients for a short
period of time (a total of 135.0 patient-years of observation).
Given the overall risk of adverse events of 15% per year, it is
possible that very few, if any, events would have occurred in
CMAJ• APRIL 28, 2009 • 180(9)
Table 2: Patients’ satisfaction with the interactive voice
Decision regarding continuation and reason
No. (%) of
n = 214*
Completed 3 mo follow-up
Continued with the system after the study
Received clear, timely information
Believed the system was easier for clinic staff
Received clear information and had the
option of speaking with clinic staff
Appreciated receiving reminder message
Did not continue with the system
after the study
Missed personal contact
Instructions were confusing
Instructions were too fast
System had problems recognizing the
patient’s answering machine and left
*Except where indicated otherwise. The denominator of 214 represents the
number of patients who were eligible to continue with the interactive voice
†Percentage based on a denominator of 226 patients.
Table 3: Overall utility of the interactive voice response
No. (%) of scheduled
n = 1557
Message delivered, with no additional
input required from clinic staff
Additional input required from clinic
Patient’s international normalized
ratio excessively low or high†
Message not successfully delivered
Patient asked to be contacted by
someone from the clinic
System recorded successful delivery of
the message, but patient called clinic
because he or she did not receive the
*Based on dosage messages with attempted delivery between May 7, 2007,
and Dec. 14, 2007.
†If the result was excessively high or low, a clinic staff member called the
patient to identify factors that might be causing the nontherapeutic
international normalized ratio.
our cohort simply by chance. Second, patients were within
the therapeutic range close to 80% of the time during the
study. Because the risk of adverse events is minimized when
the international normalized ratio is within therapeutic range,
the expected risk of events was low. Third, the patients in our
cohort were younger and healthier than patients evaluated in
previous studies. As age and comorbidity are strong predic-
tors of the risk of bleeding, our study might have been biased
toward a low rate of adverse events.
A previous evaluation of an interactive voice response sys-
tem used in managing oral anticoagulation therapy36was lim-
ited because it did not measure anticoagulation control or
structural indicators. Also, the system in the earlier study was
not integrated with a computerized decision support system
and was therefore more cumbersome to use. The interactive
voice response system in our study was easy to use and could
be implemented by large health management organizations.
This decision support tool would be even more efficient if the
computerized decision support system were integrated with a
laboratory information system, so as to make manual data
Strengths and limitations
Our study had important strengths. First, we had perfect fol-
low-up of our study population. All patients completed the
post-study interview. In addition, once the study was com-
plete, most chose to continue using the interactive voice re-
sponse system rather than reverting to the standard monitor-
ing system. Furthermore, the follow-up period for each
patient was adequate to accurately assess anticoagulation con-
trol. Although the total observation time for the intervention
period was 22 years longer than that for the pre-intervention
period, we believe that the 2 periods were sufficiently long to
justify our comparisons. Second, we selected our study popu-
lation using clinically relevant, transparent eligibility criteria.
These criteria yielded a clear inception cohort. In addition, a
high proportion of the patients whom we approached agreed
to participate, which indicates that the study population was
highly representative of the clinic’s population. Third, our
range of relevant, objective indicators provided a comprehen-
sive assessment of the interactive voice response system.
Our study also had limitations. First, the study population
consisted of a highly selected group of users of oral anticoagu-
lants. Most of the patients monitored by the anticoagulation
clinic had stable anticoagulation control and were therefore not
representative of most community-based patients.35Second, we
were unable to determine the true utility of the interactive voice
response system because our study lacked a concurrent control.
The results of this pretest–posttest study suggested that care
with the interactive voice response system may be noninferior.
However, the limited statistical power of the study and the lack
of a randomized design prevented us from conclusively
demonstrating noninferiority. A randomized controlled trial
would be required to obtain more robust evidence.
We have demonstrated that interactive voice response sys-
tems have a potential role in improving the monitoring of pa-
tients who are taking oral anticoagulants. Future randomized
studies will be required to conclusively demonstrate the effec-
tiveness of this technology. Such studies should also be based
on a more representative sample of users of oral anticoagu-
lants. Given our experiences, we also recommend that this
type of decision support tool be considered for management
of other high-risk medication therapies for which laboratory
monitoring is required.
1.Kim H, Bracha Y, Tipnis A. Automated depression screening in disadvantaged
pregnant women in an urban obstetric clinic. Arch Womens Ment Health
2.Shaw WS, Verma SK. Data equivalency of an interactive voice response system
for home assessment of back pain and function. Pain Res Manag 2007;12:23-30.
3. Gruenewald PJ, Searles J, Helzer J, et al. Exploring drinking dynamics using inter-
active voice response technology. J Stud Alcohol 2005;66:571-6.
4. Friedman RH. Automated telephone conversations to assess health behavior and
deliver behavioral interventions. J Med Syst 1998;22:95-102.
5. Stuart GW, Laraia MT, Ornstein SM, et al. An interactive voice response system to
enhance antidepressant medication compliance. Top Health Inf Manage 2003;24:
6.Oake N, Jennings A, van Walraven C, et al. The use and effectiveness of interac-
tive voice response systems for improving the delivery of ambulatory care: a sys-
tematic review and meta-analysis. Am J Manag Care. In press.
7.Ansell J, Jacobson A, Levy J, et al. Guidelines for implementation of patient self-
testing and patient self-management of oral anticoagulation. Int J Cardiol
8.Gitter MJ, Jaeger TM, Petterson TM, et al. Bleeding and thromboembolism during
anticoagulant therapy: a population-based study in Rochester, Minnesota. Mayo
Clin Proc 1995;70:725-33.
9.Steffensen FH, Kristensen K, Ejlersen E, et al. Major haemorrhagic complications
during oral anticoagulant therapy in a Danish population-based cohort. J Intern
10.Oake N, Jennings A, Forster AJ, et al. Anticoagulation intensity and outcomes
among patients prescribed oral anticoagulant therapy: a systematic review and
meta-analysis. CMAJ 2008;179:235-44.
11.Oake N, Fergusson DA, Forster AJ, et al. Frequency of adverse events in patients
with poor anticoagulation: a meta-analysis. CMAJ 2007;176:1589-94.
12.Gurwitz JH, Field TS, Harrold LR, et al. Incidence and preventability of adverse drug
events among older persons in the ambulatory setting. JAMA2003;289:1107-16.
13.Gurwitz JH, Field TS, Judge J, et al. The incidence of adverse drug events in two
large academic long-term care facilities. Am J Med 2005;118:251-8.
14.Mundt JC. Interactive voice response systems in clinical research and treatment.
Psychiatr Serv 1997;48:611-2.
15.Lafata JE, Gunter MJ, Hsu J, et al. Academic detailing to improve laboratory test-
ing among outpatient medication users. Med Care 2007;45:966-72.
CMAJ• APRIL 28, 2009 • 180(9)
This article has been peer reviewed.
Competing interests: There are no conflicts of interest between any of the
authors and the company that manufactured the interactive voice response
system, Vocantas Inc. The data were collected and analyzed independent of
Vocantas Inc. Natalie Oake had full access to all of the study data and takes
responsibility for the integrity of the data and the accuracy of the analyses.
Marc Rodger has received honoraria for speaking engagements and con-
sulting from makers of oral anticoagulants, including Bayer, Pfizer and
Boehringer Ingelheim. These funds were placed in the research trust funds of
the Ottawa Health Research Institute.
None declared for Natalie Oake, Carl van Walraven and Alan Forster.
Contributors: All of the authors were involved in implementing the interac-
tive voice response system at the study clinic. Natalie Oake recruited the
study participants and collected the data. All of the authors were involved in
the analyses and interpretation of the data, the writing and revision of the
manuscript and gave approval of the final version for publication.
Acknowledgements: We thank the staff of the Oral Anticoagulation Man-
agement Service, The Ottawa Hospital Thrombosis Program, especially
Shemina Kherani, Lesley Yeung and Geoff Lewis.
Funding: Marc Rodger is the recipient of a Career Investigator Award from
the Heart and Stroke Foundation of Canada. Alan Forster holds a Career Scien-
tist Award from the Ontario Ministry of Health and Long-Term Care and an
Early Research Award from the Ontario Ministry of Research and Innovation.
16.Forster AJ, Murff HJ, Peterson JF, et al. Adverse drug events occurring following
hospital discharge. J Gen Intern Med 2005;20:317-23.
Fitzmaurice DA, Hobbs FD, Murray ET, et al. Evaluation of computerized deci-
sion support for oral anticoagulation management based in primary care. Br J Gen
Fitzmaurice DA, Hobbs FD, Murray ET, et al. Oral anticoagulation management
in primary care with the use of computerized decision support and near-patient
testing: a randomized, controlled trial. Arch Intern Med 2000;160:2343-8.
Poller L, Shiach CR, MacCallum PK, et al. Multicentre randomised study of com-
puterised anticoagulant dosage. European Concerted Action on Anticoagulation.
Manotti C, Moia M, Palareti G, et al. Effect of computer-aided management on the
quality of treatment in anticoagulated patients: a prospective, randomized, multi-
center trial of APROAT (Automated PRogram for Oral Anticoagulant Treatment).
Harris AD, McGregor JC, Perencevich EN, et al. The use and interpretation of quasi-
experimental studies in medical informatics. J Am Med Inform Assoc 2006;13:16-23.
Lee H, Friedman ME, Cukor P, et al. Interactive voice response system (IVRS) in
health care services. Nurs Outlook 2003;51:277-83.
Rosendaal FR, Cannegieter SC, van der Meer FJ, et al. A method to determine the
optimal intensity of oral anticoagulant therapy. Thromb Haemost 1993;69:236-9.
Steuten L, Vrijhoef B, Severens H, et al. Are we measuring what matters in health
technology assessment of disease management? Systematic literature review. Int J
Technol Assess Health Care 2006;22:47-57.
Jones B, Jarvis P, Lewis JA, et al. Trials to assess equivalence: the importance of
rigorous methods. BMJ 1996;313:36-9.
Piaggio G, Elbourne DR, Altman DG, et al. Reporting of noninferiority and equiv-
alence randomized trials: an extension of the CONSORT statement. JAMA
Fitzmaurice DA, Murray ET, McCahon D, et al. Self management of oral antico-
agulation: randomised trial. BMJ 2005;331:1057.
Chan FWH, Wong RSM, Lau WH, et al. Management of Chinese patients on war-
farin therapy in two models of anticoagulation service — a prospective random-
ized trial. Br J Clin Pharmacol 2006;62:601-9.
Menendez-Jandula B, Souto JC. Comparing self-management of oral anticoagulant
therapy with clinic management: a randomized trial. Ann Intern Med 2005;142:1-10.
Wilson SJ, Wells PS, Kovacs MJ, et al. Comparing the quality of oral anticoagu-
lant management by anticoagulation clinics and by family physicians: a random-
ized controlled trial [published erratum appears in CMAJ 2004;170:451]. CMAJ
Kleinbaum DG, Gail M, editors. Modeling strategy guidelines: logistic regression
— a self learning text. New York (NY): Springer-Verlag; 1994.
Stokes ME, Davis CS, Koch GG. Categorical data analysis using the SAS system.
2nd ed. Cary (NC): SAS Institute Inc.; 2000.
Lewis RJ. Power analysis and sample size determination: concepts and software
tools. Proceedings of the 2000 Annual Meeting of the Society for Academic Emer-
gency Medicine; 2000 May 22–25; San Francisco (CA). Available: www.saem.org
/download/lewis4.pdf (accessed 2009 Mar. 30).
34.van Walraven C, Austin PC, Oake N, et al. The effect of hospitalization on oral an-
ticoagulation control: a population-based study. Thromb Res 2007;119:705-14.
van Walraven C, Jennings A, Oake N, et al. Effect of study setting on anticoagula-
tion control: a systematic review and metaregression. Chest 2006;129:1155-66.
Cervi PL, Everitt AS. Automatic voice mail for delivering computer-generated an-
ticoagulant dose advice to patients. J Telemed Telecare 2002;8:259-63.
CMAJ • APRIL 28, 2009 • 180(9)
Correspondence to: Dr. Alan Forster, Clinical Epidemiology
Program, Ottawa Health Research Institute, 1053 Carling Ave.,
Administrative Services Building 1-008, Ottawa ON K1Y 4E9;
Pfizer Canada Inc., Champix,
We require 6 to 8 weeks’ notice to ensure uninterrupted
service. Please send your current mailing label, new
address and the effective date of change to:
CMA Member Service Centre
1870 Alta Vista Dr.
Ottawa ON K1G 6R7
tel 888 855-2555 or
613 731-8610 x2307
fax 613 236-8864
Change of address