Use of a Handheld Computer Application for Voluntary
Medication Event Reporting by Inpatient Nurses and Physicians
Adrian W. Dollarhide, MD1,2, Thomas Rutledge, PhD1,2, Matthew B. Weinger, MD3,4,
and Timothy R. Dresselhaus, MD, MPH1,2
1VA San Diego Healthcare System, San Diego, CA, USA;2University of California, San Diego, CA, USA;3Vanderbilt University and VA Tennessee
Valley Healthcare System, Nashville, TN, USA;4VA Tennessee Valley Healthcare System, Nashville, TN, USA.
OBJECTIVE: To determine the feasibility of capturing
self-reported medication events using a handheld com-
puter-based Medication Event Reporting Tool (MERT).
DESIGN AND PARTICIPANTS: Handheld computers
operating the MERT software application were deployed
among volunteer physician (n=185) and nurse (n=119)
participants on the medical wards of four university-
to complete confidential reports on the handheld compu-
ters for medication events observed during the study
MEASUREMENTS AND MAIN RESULTS: Demographic
variables including age, gender, education level, and
clinical experience were recorded for all participants. Each
MERT report included details on the provider, location,
timing and type of medication event recorded. Over the
course of 2,311 days of clinician participation, 76 events
were reported; the median time for report completion was
231 seconds. The average event reporting rate for all
participants was 0.033 reports per clinician shift. Nurses
had a significantly higher reporting rate compared to
physicians (0.045 vs 0.026 reports/shift, p=.02). Sub-
events more frequently than resident physicians (0.042 vs
0.021 reports/shift, p=.03), and at a rate similar to that of
nurses (p=.80). Only 5% of MERT medication events were
reported to require increased monitoring or treatment.
a feasible method to record medication events in inpatient
hospital care units. Handheld reporting tools may hold
promise to augment existing hospital reporting systems.
KEY WORDS: incident reporting; medication errors; computers;
J Gen Intern Med 23(4):418–22
© Society of General Internal Medicine 2007
Since the publication of the Institute of Medicine’s (IOM) To Err
Is Human, national attention has been focused on medical
errors and especially hospital safety.1However, progress
toward developing safer health care systems has been slow,
impeded in large part by the difficulty in detecting hazards
using conventional monitoring or reporting mechanisms.2In
its review of the scope and impact of medication error in
hospitals, the IOM estimates that 400,000 preventable drug-
related injuries still occur each year, the majority of which are
undetected.3New methods of identifying safety hazards are
needed to generate the data to inform system redesign.4
Methods currently available for monitoring patient safety in
hospitalized patients include clinician event reporting, direct
observation, patient surveys, and extraction of practice data
from medical records or computerized databases. Direct
observation has been shown to have the highest rate of event
detection, but may be prohibitively expensive for routine
monitoring.2,5Patient surveys can identify certain types of
events such as adverse drug reactions, but are insensitive to
potential events or medication events that are unrecognizable
to patients.6Chart review relies on the retrospective analysis of
medical record documents, which may limit the rate of event
detection because of deficiencies in documentation.7In addi-
tion, although automated database trigger tools may increase
the yield of event detection, they are often limited to lab-related
measures and depend on computerized medical record data-
bases not yet available in many clinical settings.8–16
Medication event reporting remains a central element of
many hospital safety monitoring systems. Event reports
(sometimes called “incident reports”) offer providers at the
sharp end of patient care a means to describe and document
safety events that result from system failures.17,18Clinician-
reported events are rarely false-positive, the influence of recall
bias is limited,2and they offer the greatest amount of clinically
relevant data gathered at the time of the actual event. However,
consistent underreporting by practicing clinicians limits the
effectiveness of this component of safety monitoring.19–21
Analysis of currently available self-report methods demon-
strates that such reports may capture as little as 0.04% of
recognizable hospital events.2,22
Emerging handheld technologies could serve to augment
clinician self-reporting of medication events in the hospital
setting. Handheld computers offer a number of potential
advantages over stationary computers including convenience,
portability, confidentiality, and point-of-care reporting. Yet,
This project was supported by grant number 1 UC1 HS014283-01
from the Agency for Healthcare Research and Quality (AHRQ).
despite an increasing number of clinical applications,23hand-
held technology has been used on a very limited basis to
facilitate medical event reporting.24,25To determine the feasi-
bility of a handheld computer-assisted reporting system, we
report on the development and deployment of a Medication
Event Reporting Tool (MERT) among physicians and nurses on
inpatient medical units.
A total of 119 nurses and 185 physicians were recruited from
four university-affiliated hospitals. Physician participants were
enrolled from both general medicine and pediatric service
wards and included attending and resident physicians. Physi-
cians were recruited in rotating units of ward teams that were
typically comprised of attendings, residents, and interns. Al-
though teams were recruited to participate as a unit, individual
clinician participation was strictly voluntary. Nurse participants
were enrolled from medicine, surgery, and intensive care units
from three of these hospitals. All participant reports were
voluntary and confidential, and did not replace the standard
event reporting systems currently in place at participating
hospital sites. All participants provided written informed con-
sent, and Institutional Review Board (IRB) approval was
obtained for data collection at each hospital.
The Medication Event Reporting Tool (MERT), a customized
software application, was developed to allow self-reporting of
medication events on a Palm Zire™ 21 handheld computer
(Palm, Inc., Sunnyvale, CA, USA). The software application was
developed on an AppForg platform, with code written in Visual
Basic. The MERT interface was designed to be consistent with
the standard reporting systems used by participating institu-
tions to facilitate interpretation and classification. Data col-
lected in the report included information about the provider
involved, event date and time, location, type, severity, stage of
the medication process, and contributing factors (Table 1). The
application used a forced-choice, menu-driven response for-
mat that directed respondents to provide detailed descriptions
of a medication event on the handheld computer. A software
algorithm timed and recorded the number of seconds required
to complete each report. Individualized medication formularies
were constructed on the reporting tool specific to each
participating hospital. Collected data were transferred each
week via a custom hotsync conduit to a centralized server
database from the participating hospitals during the study.
Nurses and physicians were recruited to carry a handheld
computer for 1-week work intervals. Participants were eligible
to participate for more than 1 study interval based on hospital
staffing assignments. Participants received a nominal financial
incentive of 4 dollars per day for their participation in the
study, but there was no direct incentive tied to the completion
of MERT reports. Baseline demographic and professional
experience information including descriptive variables of par-
ticipants’ sex, age, professional experience, and educational
level were recorded for all participants by the handheld
application through an initial sign-on process. Participants
were prompted through an audible alarm feature of the
software program, which then subsequently elicited responses
from participants and collected survey data. Daily workload
information was captured at the beginning of each shift.
Random repeating daily surveys collected dynamic character-
istics such as work activity, stress, and perceived work
demands. These ecologic momentary assessments were eli-
cited from study participants randomly over 90-minute inter-
vals throughout their work shift (data reported separately). At
the end of each computer alarm and survey, an opportunity to
launch the event reporting application was provided. Partici-
pants were also given instructions on how to initiate the MERT
application at any time during the study period they wished to
report immediately an observed medication event. Event
reporting rates were determined as the number of reports
generated per shift of clinician participation.
For continuous variables, Student’s t tests were used to
determine statistically significant differences between partici-
Table 1. Survey Structure for MERT Tool
1. Who was involved in this event? (provider type)
2. In regard to this event, I would rate my involvement as (responsibility
a. Completely responsible
b. Mostly responsible
c. Somewhat responsible
d. Minimally responsible
e. Not at all responsible
3. What was the result of this event?
a. Error occurred; med not given to patient
b. Error occurred; med given to patient
i. No harm to patient
ii. No harm but increased monitoring
iii. Temporary harm not requiring treatment
iv. Temporary harm requiring treatment
v. Temporary harm prolonging hospital stay
vi. Permanent harm
vii. Near-death event
vii. Patient death
4. The medication involved in this event was (medication list)
5. When did this event occur? (date/time record)
6. What route was used? (route list)
7. Which steps in the medication process were involved?
a. Prescribing (steps list)
b. Transcribing/documenting (steps list)
c. Dispensing (steps list)
d. Administration (steps list)
e. Delays (steps list)
f. Administration-pump specific
8. What causes or factors contributed to this event?
a. Communication (causes list)
b. Information systems (causes list)
c. Equipment or devices (causes list)
d. Patient or clinical context (causes list)
e.Clinician (causes list)
f. Staffing and workload (causes list)
g. Organizational (causes list)
Dollarhide et al.: Reporting by Nurses and Physicians Using Handheld Computer
pant groups. A chi-square test of proportions was performed to
examine the association between education level and MERT
use by nurse participants, and also to compare the response
rate differences between participant groups. All statistical
calculations were performed using SPSS version 12.0. Statis-
tical comparisons were completed as 2-tailed tests, using an
alpha level of 0.05 for determining significance.
Physician participants were evenly distributed between male
(46%) and female (54%), while nurse participants were pre-
dominantly female (89%). Fifty participants (28 physicians and
22 nurses; 16% of all participants) completed at least 1
electronic MERT report over the 10-month study period
(Table 2). Over 2,311 days of clinician participation, reporters
generated 76 complete event reports (Table 3). The median
time required to complete an event report was 231 seconds
(interquartile range of 179.3–281.8). The overall MERT report-
ing rate for the study period was 0.033 reports/shift.
Physician study participants were overall younger and less
experienced on their units compared to nurse participants.
Physician reporters had significantly more days of study
participation than physicians who did not complete an event
report (12.9 vs 7.3 days; p<.001). There was no difference in
age or unit experience for physician reporters versus non-
reporters. Nurse reporters compared to nonreporters were
similar in age, experience, and days of participation. There
was a trend toward nurse reporters having a higher educa-
tional level compared with nurse nonreporters (p=.06). Physi-
cian reporters overall had more average days of participation
with the handheld tool compared to nurse reporters (12.9 vs
7.5 days; p=.016).
Nurses had significantly higher reporting rates compared to
all physicians (0.045 vs 0.026 reports/shift, p=.02). This
difference was primarily due to the relatively lower reporting
rate for resident physicians. Both attending physicians and
nurses reported events significantly more often than resident
physicians (0.042 vs 0.021 reports/shift, p=.03 and .045 vs
0.021 reports/shift, p=.003, respectively). In contrast, report-
ing rates for nurses and attending physicians were similar
(0.045 vs 0.042 reports/shift, p=.80). MERT reports were
generated in an even distribution across the days of participa-
tion. An assessment of the distribution of MERT events sug-
gested a nonsignificant trend toward increased reporting rates
during later days of participation (p=.07).
Event reports were fairly evenly distributed between those
observed as involving others (53%) and those involving self
(47%) (Table 4). House staff physicians reported themselves
personally responsible in a majority of medication events
(71%), whereas the level of personal responsibility recorded
by nurses (44%) and physician (19%) participants was lower.
Reported events were predominantly potential adverse events,
with medication either not administered to the patient (69%) or
administered but resulting in no discernible patient harm
Table 2. MERT “Reporter” Versus “Non-reporter” Provider
Sex (% male)
Average Age (years)*
Average Experience (mo)†
Average Study Days‡
*p=.001: all MD versus Nurse; p=.002 Resident MD versus Nurse
†p<.001: Nurse versus All MD, Resident MD, and Attending MD
‡p<.001: Reporter versus Nonreporter for All MD, Resident MD, and
Table 3. MERT Event Reports by All Providers
All MDRes MD Att MDNurse
Number of reports
Total days of participation
MERT reports per shift
All MD = all physicians; Res MD = resident physicians; Att MD =
*p=.003: Resident versus Nurse
†p=.03: Resident versus Attending MD
‡p=.02: All MD versus Nurse
Table 4. Events Reported by Physician and Nurse Subjects (N=76)
Res MDAtt MD Nurse
Yes (any degree)
Error made, med not given
Error made, med given
No harm, monitoring
Temporary harm with treatment
Process Step Involved
Causes or Factors
Equipment or devices
Patient or clinical context
Staffing and workload
Dollarhide et al.: Reporting by Nurses and Physicians Using Handheld Computer
(27%). An event resulting in an increased need for monitoring,
temporary patient harm, or the need for treatment, was noted
in only 5% of the MERT reports.
The MERT software application was developed to enable
convenient, real-time reporting of medication events on a
handheld computer in the hospital setting. Study participants
using the handheld tool required less than 4 minutes on
average to complete an event report, indicating a response
burden, which compares favorably to that described for other
computerized event reporting systems.24Age, clinical experi-
ence, and education level of participants did not discriminate
reporters versus nonreporters, suggesting acceptance of this
tool across a range of hospital clinicians.
Nurses used the handheld application to report medication
events at a greater rate than the physicians in this study. This
was not a surprising finding as hospital event reporting has
historically suffered from significant participation bias, with
nursing staff submitting up to 89% of all reports received in
some settings,26and typically very few reports gathered from
senior physicians. Subgroup analysis of our data, however,
demonstrated that attending physicians and nurses actually
displayed very similar rates of event reporting. The lower rate
of resident reporting largely accounted for the observed
reporting rate difference between nurse and physician groups.
These data would suggest that handheld systems may repre-
sent an opportunity to mitigate the impact of participation bias
and gather event reports more broadly across provider dis-
ciplines, particularly from attending physicians.
Resident physicians filed event reports at a rate significantly
lower than both attending physicians and nurses in our study.
Extended-duration work shifts have put resident physicians at
particular risk for medical error leading to adverse patient
events.27Although some studies demonstrate that house staff
are willing to report events when appropriately queried,28,29
actively engaging resident physicians in event reporting has
been a substantial challenge. Reporting systems are some-
times difficult for house staff to navigate, and lack of clinical
experience may interfere with successful reporting.30In addi-
tion, resident physicians experience substantially higher per-
ceived work stress levels in the hospital environment,31,32and
fear of professional or legal reprisal may limit willingness to file
event reports.33Yet, resident physicians are perhaps more
likely to utilize handheld computer technology,23and adapting
reporting tools to their needs could enhance their reporting. In
addition, house staff were the group most likely to report
events for which they were personally responsible (71%). These
data underscore the importance of developing tools to facilitate
event reporting from resident physicians, and reducing the
barriers inherent to conventional reporting systems.
Event reports collected by our handheld computer applica-
tion reflected a broad spectrum of both actual adverse drug
events (ADEs) and near misses (i.e., potential ADEs). Sixty nine
percent of events captured by MERT were recorded before
medication was administered, while 27% of events resulted in
no discernible patient injury after the administration of
medication. This observed reporting of near-miss medication
events with MERT was greater than that reported by other
monitoring systems, which have captured near misses in
approximately 35% of events.34Near misses are an important
component of successful reporting systems, as they typically
occur at a greater frequency than ADEs and allow for
quantitative analysis of system failures that ultimately lead to
adverse outcomes.18Emphasizing the reporting of near-miss
events tends to reduce the culture of blame when harmful
events do occur, and promotes the study of recovery strategies
to build more resilient safety systems.35,36
Finally, the results of this study suggest that this novel
handheld tool may hold promise to improve critical event
reporting in hospitals as part of an integrated health informa-
tion technology. There is substantial evidence that inpatient
medication events are grossly underreported,2,19,33and tradi-
tional reporting methods are often not well integrated into
hospital data systems. Handheld computers offer the potential
to bundle clinical care applications on a single portable device,
and to prompt users for critical event data reporting. As
hospitals increasingly integrate electronic health information
technology, bundling an event reporting mechanism into
clinical applications offers a unique opportunity to simplify
the process and potentially increase the yield of event reports
gathered in hospital settings. Future study should critically
evaluate the quantitative impact of a handheld event reporting
tool in comparison to traditional reporting systems, as hospi-
tals increasingly adopt novel health information technology.
There are limitations to interpreting the results and gener-
alizability of this study, as this was an unblinded evaluation of
a newly developed software application. The study participants
were physicians and nurses on inpatient units at academic
centers, and use in community hospitals will need to be
evaluated. A handheld computer-based system would likely
only be feasible at hospitals with IT resources to support data
download, software modification and maintenance. Consider-
able attention would need to be given to encrypting and
maintaining sensitive clinical information while more broadly
integrating this novel technology, and the associated security
and information technology (IT) costs of a larger application of
MERT were not specifically evaluated in this study. The format
of event reports captured on handheld computers also remains
limited in its ability to collect narrative data, although
advances in scripting and voice recognition software hold
promise to allow expanded narrative event reporting on
handheld devices in the near future. In addition, the overall
percentage of events captured by this handheld event report-
ing application likely remains quite small, still representing
only the “tip of the iceberg” of hospital events. An ideal event
reporting system would also include incentives for voluntary
reporting, place emphasis on a systems perspective for event
analysis, and promote culture change to value event reporting
as a nonpunitive means to provide critical feedback and
improve patient safety.32,34
A statistical limitation of this study is that our sample size
made it impractical to adjust for increases in family-wise error
rates introduced by completing multiple tests. The failure to
correct statistically for multiple comparisons makes it possible
that some of our statistically significant findings occurred by
chance. However, the importance of this limitation is mitigated
by the consistent pattern of results we observed across
multiple groups and the congruence of our results with that
found in the literature. Observed reporting rates could have
been also artificially inflated by the novelty of the handheld
tool, by a volunteer bias introduced by the recruitment of
Dollarhide et al.: Reporting by Nurses and Physicians Using Handheld Computer
clinician volunteers for the study, and by the nature of the
study itself. Data was not collected for clinicians choosing not
to participate in this study. Comparison of MERT-generated
reports with events identified by concurrent direct observation
would help to establish the effectiveness of this event reporting
modality in detecting medication events in the hospital setting.
Future studies may determine whether event reporting rates
would be sustained in routine practice with the use of the
Notwithstanding these limitations, several features of the
handheld MERT including ready accessibility and ease of use
suggest that this approach may prove an effective complement
to existing hospital event reporting systems. In our study, both
physician and nursing participants were able to effectively
utilize the handheld tool to generate medication event reports
on inpatient hospital units. Event reports were collected with a
reasonable response burden, and reflected a spectrum of both
potential and actual medication events. Future work will need
to examine if deploying MERT-enabled handheld devices could
translate into a sustainable increase in overall medication
event detection in hospital settings, and provide substantive
opportunities to improve patient safety.
Acknowledgments: The authors wish to thank Tod Kuykendall
and Jason Slagle for their invaluable technical assistance in the
development of the handheld computer application. We also express
our gratitude to project site leaders Erin Stucky, MD, Greg Maynard,
MD, and Martha Shively, RN. This project was supported by grant
number 1 UC1 HS014283-01 from the Agency for Healthcare
Research and Quality (AHRQ).
Conflict of Interest: None disclosed.
Corresponding Author: Timothy R. Dresselhaus, MD, MPH; VA San
Diego Healthcare System, 3350 La Jolla Village Drive (111), San
Diego, CA 92161, USA (e-mail: email@example.com).
1. Kohn LT, Corrigan JM, Donaldson MS, eds. To Err is Human: Building
a Safer Health System. Washington, DC: National Academy Press; 1999.
2. Flynn EA, Barker KN, Pepper GA, Bates DW, Mikeal RL. Comparison
of methods for detecting medication errors in 36 hospitals and skilled-
nursing facilities. Am J Health-Syst Pharm. 2002;59:436–46.
3. Aspden P, Wolcott J, Bootman JL, Cronenwett LR, eds. Preventing
Medication Errors. Committee on Identifying and Preventing Medication
Errors. Washington, DC: The National Academies Press; 2006.
4. Leape LL, Berwick DM. Five years after To Err Is Human: what have we
learned? JAMA. 2005;293:2384–90.
5. Kopp BJ, Erstad BL, Allen ME, Theodorou AA, Priestley G. Medication
errors and adverse drug events in an intensive care unit: direct
observation approach for detection. Crit Care Med. 2006;34:415–25.
6. Weingart SN, Toth M, Eneman J, et al. Lessons from a patient
partnership intervention to prevent adverse drug events. Int J Qual
Health Care. 2004;16:499–507.
7. Bates DW, Cullen DJ, Laird N, et al. Incidence of adverse drug events
and potential adverse drug events. JAMA. 1995;274:29–34.
8. Field TS, Gurwitz JH, Harrold LR, et al. Strategies for detecting
adverse drug events among older persons in the ambulatory setting.
J Am Med Inform Assoc. 2004;11:492–8.
9. Resar RK, Rozich JD, Classen D. Methodology and rationale for the
measurement of harm with trigger tools. Quality and Safety in Health
10. Classen DC, Metzger J. Improving medication safety: the measurement
conundrum and where to start. Int J Qual Health Care. 2003;15:i41–7.
11. Hardmeier B, Braunschweig S, Cavallaro M, et al. Adverse drug events
caused by medication errors in medical inpatients. Swiss Med Wkly.
12. Payne TH, Savarino J, Marshall R, Hoey CT. Use of a clinical event
monitor to prevent and detect medication errors. Proc AMIA Symp.
13. Murff HJ, Patel VL, Hripcsak G, Bates DW. Detecting adverse events for
patient safety research: a review of current methodologies. J Biomed
14. Murff HJ, Forster AJ, Peterson JF, Fiskio JM, Heiman HL, Bates DW.
Electronically screening discharge summaries for adverse medical
events. J Am Med Inform Assoc. 2003;10:339–50.
15. Centers for Disease Control. Assessing the national electronic injury
six sites, United States, January 1–June 15, 2004. MMWR Morb Mort
Wkly Rep. 2005;54:380–3.
16. Bates DW, Evans RS, Murff H, Stetson PD, Pizziferri L, Hripcsak G.
Detecting adverse events using information technology. J Am Med Inform
17. Nebeker JR, Barach P, Samore MH. Clarifying adverse drug events: a
clinician’s guide to terminology, documentation, and reporting. Ann
Intern Med. 2004;140:795–801.
18. Morimoto T, Gandhi TK, Seger AC, Hsieh TC, Bates DW. Adverse drug
events and medication errors: detection and classification methods.
Quality and Safety in Health Care. 2004;13:306–14.
19. Taylor JA, Brownstein D, Christakis DA, et al. Use of incident reports
by physicians and nurses to document medical errors in pediatric
patients. Pediatrics. 2004;114:729–35.
20. Lazarou J, Pomeranz BH, Corey PN. Incidence of adverse drug
reactions in hospitalized patients: a meta-analysis of prospective
studies. JAMA. 1998;279:1200–5.
21. Capuzzo M, Nawfal I, Campi M, Valpondi V, Verri M, Alvisi R.
Reporting of unintended events in an intensive care unit: comparison
between staff and observer. BMC Emerg Med. 2005;5:1471–8.
22. Tamuz M, Thomas EJ, Franchois KE. Defining and classifying medical
error: lessons for patient safety reporting systems. Quality and Safety in
Health Care. 2004;13:13–20.
23. McLeod TG, Ebbert JO, Lymp JF. Survey assessment of personal digital
assistant use among trainees and attending physicians. J Am Med
Inform Assoc. 2003;10:605–7.
24. Kobus DA, Amundson D, Moses JD, Rascona D, Gubler KD. A
computerized medical incident reporting system for errors in the
intensive care unit: initial evaluation of interrater agreement. Mil Med.
25. Bent PD, Bolsin SN, Creati BJ, Patrick AJ, Colson ME. Professional
monitoring and critical incident reporting using personal digital assis-
tants. Med J Aust. 2002;177:496–9.
26. Busse DK, Wright DJ. Classification and analysis of incidents in
complex medical environments. Top Health Inf Manag. 2000;20:1–11.
27. Barger LK, Ayas NT, Cade BE, et al. Impact of extended-duration shifts
on medical errors, adverse events, and attentional failures. PLoS Med.
28. Weingart SN, Ship AN, Aronson MD. Confidential clinician-reported
surveillance of adverse events among medical inpatients. J Gen Intern
29. Weinger MB, Slagle J, Jain S, Ordonez N. Retrospective data collection
and analytical techniques for patient safety studies. J Biomed Inform.
30. Weingart SN, Callanan LD, Ship AN, Aronson MD. A physician-based
voluntary reporting system for adverse events and medical errors. J Gen
Intern Med. 2001;16:809–14.
31. Thomas NK. Resident burnout. JAMA. 2004;292:2880–9.
32. Weinger MB, Reddy SB, Slagle JM. Multiple measures of anesthesia
workload during teaching and nonteaching cases. Anesth Analg.
33. Lawton R, Parker D. Barriers to incident reporting in a healthcare
system. Quality and Safety in Health Care. 2002;11:15–8.
34. Hicks RW, Cousins DD, Williams RL. Selected medication-error data
from USP's MEDMARX program for 2002. Am J Health-syst Pharm.
35. Firth-Cozens J. Barriers to incident reporting. Quality and Safety in
Health Care. 2002;11;7–8.
36. Barach P, Small SD. Reporting and preventing medical mishaps: lessons
from non-medical near miss reporting systems. BMJ. 2000;320:759–63.
Dollarhide et al.: Reporting by Nurses and Physicians Using Handheld Computer